Disney Comes Up With Automatic Re-dubbing Tech That Can Actually “Put Words Into Your Mouth”
TheTechPortal.in reports that Disney continues to churn out fascinating tech. The special research division under Disney has now come up with a new technology in the field of re-dubbing, extensively used in the field of television and movies.
The latest technology named as “Visually Consistent Acoustic Redubbing” in the paper written by Sarah Taylor and Iain Matthews of Disney labs, Pittsburgh, can detect facial expressions and lip movements in a video, i.e the visual elements of speech (visemes) and map them to different phoneme sequences synchronizing with the video.
When composed with a pronunciation dictionary and language model, this produces a vast number of word sequences that are in sync with the original video, literally putting plausible words into the mouth of the speaker.
In simple words, the technology can come up with a list of many things which you could be saying by detecting your facial features and lip movements while you are saying something else. So you may originally be speaking the dialogue by Batman but it can come up as a funny dialogue from a Bhojpuri movie perfectly synced to your lip movements..!!