Apple ML Research
· Cloud & Big Tech
Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction
This study focuses on Text-to-Sounding-Video (T2SV) generation, which aims to generate a video with synchronized audio from text, with both modalities aligned to the text conditions. Despite progress in joint audio-video training, two critical challenges remain: (1) text conditioning is a bottleneck—shared captions (TV