Post-Doctoral Position: Synthesis Of Videos Driven By Text And Audio
Description of the research project
In recent years, voice interaction with computers has made significant progress. Virtual agents offer a user-friendly human-machine interface while reducing maintenance costs. Speechbased interaction is already effective, as proved by Siri, Google Assistant or Alexa virtual agents, however, their visual counterpart is still far behind. The level of user engagement for audiovisual interactions is much higher than for purely audio interactions. Therefore, it is desirable to be able to associate face visual animations with the generated audio.
A notable advancement in video generation was made by a team at Stanford University in 2019 in partnership with Adobe 1. Their work is aimed at enabling a video editing technology of a person's face-to-face scene to revise its speech script and adapt the rendering automatically based simply on this revised text.
The latest advances in the field of audio-driven face video synthesis were presented in 2. The proposed approach generalizes across different people, to synthesize videos of a target actor with the voice of any actor from an unknown source or even synthetic voices that can be generated using standard text-to-speech approaches.
During this post-doctoral contract, we want to work on the development of a prototype of a Text to Speech to Video technology with a sufficient level of accuracy.
Duration, place to work and supervisors
The funding covers 18 months of post-doc, the desired start is April-May 2022. The postdoctoral fellow will be attached to the LIRIS (Laboratory of Computer Science in Image and Information Systems) on the campus of the University Lyon 2 in Bron. Some stages of work can be conducted in the office of Mon Petit Placement in Lyon.
The post-doc will be supervised by IuliiaTkachenko and Serge Miguet (LIRIS). The project manager on the side of Mon Petit Placement is the technical director of the startup Thibault Jaillon.
References
1 O. Fried, A. Tewari, M. Zollhöfer, A. Finkelstein, E. Shechtman, D. Goldman, K. Genova, Z. Jin, C. Theobalt, M. Agrawala, “Text-based editing of talking-head video”, ACM Transactions on Graphics (TOG), Vol.38, 2019.
2 J. Thies, M. Elgharib, A. Tewari, C. Theobalt, M. Niessner, “Neural Voice Puppetry: Audio-driven Facial Reenactment”, ECCV 2020 (https: // justusthies.github.io/posts/neural-voice-puppetry/ )
Offer Requirements Specific Requirements
The candidate must have a PhD in computer science, specializing in image and video processing
Programming languages: Python/C++
Neural network libraries: PyTorch/Keras/Tensorflow
Programming tools for image analysis: OpenCV
Scientific knowledge: machine learning and deep learning, video analysis and processing
Good writing skills and proficiency in written and spoken English (French is not a requirement)
Contact Information
Organisation/Company: CNRS LIRIS
Organisation Type: Small Medium Enterprise, Start-up
Website: https: // perso.liris.cnrs.fr/itkachenko/
Country: France
為防止簡(jiǎn)歷投遞丟失請(qǐng)抄送一份至：boshijob@126.com（郵件標(biāo)題格式：應(yīng)聘職位名稱(chēng)+姓名+學(xué)歷+專(zhuān)業(yè)+中國(guó)博士人才網(wǎng)）
中國(guó)-博士人才網(wǎng)發(fā)布
聲明提示：凡本網(wǎng)注明“來(lái)源：XXX”的文/圖等稿件，本網(wǎng)轉(zhuǎn)載出于傳遞更多信息及方便產(chǎn)業(yè)探討之目的，并不意味著本站贊同其觀點(diǎn)或證實(shí)其內(nèi)容的真實(shí)性，文章內(nèi)容僅供參考。
