primal-sage commited on
Commit
2dab9eb
·
verified ·
1 Parent(s): 172d845

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +49 -0
README.md ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - lip-sync
4
+ - emotion
5
+ - talking-face
6
+ - facial-animation
7
+ license: mit
8
+ ---
9
+
10
+ # EMOLIPS: Emotion-Driven Facial Animation for Realistic Lip-Sync Synthesis
11
+
12
+ ## Overview
13
+ Emotion-controllable lip-sync framework using FiLM-based modulation of 3DMM expression coefficients.
14
+
15
+ ## Architecture
16
+ - **Backbone:** SadTalker (3DMM-based talking face)
17
+ - **Novel Module:** Emotion-Conditioned Fusion Module (ECFM)
18
+ - **Conditioning:** FiLM layers + Lip Consistency Gate
19
+ - **Emotions:** Neutral, Happy, Sad, Angry, Fear, Surprise, Disgust
20
+
21
+ ## Results
22
+ See `EMOLIPS_Submission.ipynb` for full methodology, architecture diagrams, outputs, and metrics.
23
+
24
+ ### Output Samples
25
+ | Emotion | Video |
26
+ |---------|-------|
27
+ | Neutral | [emolips_neutral.mp4](outputs/emolips_neutral.mp4) |
28
+ | Happy | [emolips_happy.mp4](outputs/emolips_happy.mp4) |
29
+ | Sad | [emolips_sad.mp4](outputs/emolips_sad.mp4) |
30
+ | Angry | [emolips_angry.mp4](outputs/emolips_angry.mp4) |
31
+ | Fear | [emolips_fear.mp4](outputs/emolips_fear.mp4) |
32
+ | Surprise | [emolips_surprise.mp4](outputs/emolips_surprise.mp4) |
33
+ | Disgust | [emolips_disgust.mp4](outputs/emolips_disgust.mp4) |
34
+
35
+ ## Quick Start
36
+ ```bash
37
+ git clone https://huggingface.co/primal-sage/emolips
38
+ cd emolips/code
39
+ bash setup.sh
40
+ python inference.py --audio ../samples/input_audio.wav --image ../samples/input_face.jpg --all-emotions
41
+ ```
42
+
43
+ ## Citation
44
+ ```bibtex
45
+ @article{emolips2026,
46
+ title={EMOLIPS: Emotion-Driven Facial Animation for Realistic Lip-Sync Synthesis},
47
+ year={2026}
48
+ }
49
+ ```