+
+
+
+
+ 
[](https://colab.research.google.com/github/Winfredy/SadTalker/blob/main/quick_demo.ipynb) [](https://huggingface.co/spaces/vinthony/SadTalker)
+
+
+
+ Wenxuan Zhang *,1,2
+ Xiaodong Cun *,2
+ Xuan Wang 3
+ Yong Zhang 2
+ Xi Shen 2
+ Yu Guo1
+ Ying Shan 2
+ Fei Wang 1
+
++
+ 1 Xi'an Jiaotong University 2 Tencent AI Lab 3 Ant Group
+
++CVPR 2023 +
+
+ + + +TL;DR: single portrait image 🙎♂️ + audio 🎤 = talking head video 🎞. + +
+ +
+
+- 🔥 Several new mode, eg, `still mode`, `reference mode`, `resize mode` are online for better and custom applications.
+
+- 🔥 Happy to see our method is used in various talking or singing avatar, checkout these wonderful demos at [bilibili](https://search.bilibili.com/all?keyword=sadtalker&from_source=webtop_search&spm_id_from=333.1007&search_source=3
+) and [twitter #sadtalker](https://twitter.com/search?q=%23sadtalker&src=typed_query).
+
+## 📋 Changelog (Previous changelog can be founded [here](docs/changlelog.md))
+
+- __[2023.04.08]__: In v0.0.2, we add a logo watermark to the generated video to prevent abusing since it is very realistic.
+
+- __[2023.04.08]__: v0.0.2, full image animation, adding baidu driver for download checkpoints. Optimizing the logic about enhancer.
+
+- __[2023.04.06]__: stable-diffiusion webui extension is release.
+
+- __[2023.04.03]__: Enable TTS in huggingface and gradio local demo.
+
+- __[2023.03.30]__: Launch beta version of the full body mode.
+
+- __[2023.03.30]__: Launch new feature: through using reference videos, our algorithm can generate videos with more natural eye blinking and some eyebrow movement.
+
+- __[2023.03.29]__: `resize mode` is online by `python infererence.py --preprocess resize`! Where we can produce a larger crop of the image as discussed in https://github.com/Winfredy/SadTalker/issues/35.
+
+- __[2023.03.29]__: local gradio demo is online! `python app.py` to start the demo. New `requirments.txt` is used to avoid the bugs in `librosa`.
+
+- __[2023.03.28]__: Online demo is launched in [](https://huggingface.co/spaces/vinthony/SadTalker), thanks AK!
+
+
+## 🎼 Pipeline
+
+> Our method uses the coefficients of 3DMM as intermediate motion representation. To this end, we first generate
+realistic 3D motion coefficients (facial expression β, head pose ρ)
+from audio, then these coefficients are used to implicitly modulate
+the 3D-aware face render for final video generation.
+
+
+## 🚧 TODO
+
+
+
+Then, retarting the stable-diffusion-webui, set some commandline args. The models will be downloaded automatically in the right place. Alternatively, you can add the path of pre-downloaded sadtalker checkpoints to `SADTALKTER_CHECKPOINTS` in `webui_user.sh`(linux) or `webui_user.bat`(windows) by:
+
+```bash
+# windows (webui_user.bat)
+set COMMANDLINE_ARGS=--no-gradio-queue --disable-safe-unpickle
+set SADTALKER_CHECKPOINTS=D:\SadTalker\checkpoints
+
+# linux (webui_user.sh)
+export COMMANDLINE_ARGS=--no-gradio-queue --disable-safe-unpickle
+export SADTALKER_CHECKPOINTS=/path/to/SadTalker/checkpoints
+```
+
+After installation, the SadTalker can be used in stable-diffusion-webui directly.
+
+
+
+
| |
+
+> Kindly ensure to activate the audio as the default audio playing is incompatible with GitHub.
+
+
+#### Generating 4D free-view talking examples from audio and a single image
+
+We use `input_yaw`, `input_pitch`, `input_roll` to control head pose. For example, `--input_yaw -20 30 10` means the input head yaw degree changes from -20 to 30 and then changes from 30 to 10.
+```bash
+python inference.py --driven_audio