Meta AI Translation Tool Automatically Dubs and Lip-Syncs Your Reels

Don't miss out on our latest stories. Add PCMag as a preferred source on Google.

Meta has rolled out a new AI translation tool that will help creators dub and lip-sync their reels in a different language. The feature "helps you reach and better connect with audiences all over the world so you can grow a global following," Meta says .

Meta AI Translations was first previewed at the Meta Connect conference last year and is now rolling out to all regions where Meta AI is available. To use the feature, though, you'll need either 1,000 Facebook followers or a public Instagram account. 

At launch, the feature will only support English-to-Spanish and Spanish-to-English translations. Meta says support for more languages will be added in the future. 

To get started, tap the "Translate your voice with Meta AI" option before publishing a reel. Here, you'll see three separate options to translate your voice, add lip sync, and see a preview. If you select the last option, you'll receive a notification when the reel is ready, and then you can accept or reject the AI-generated dubbing. 

Meta says the tool will mimic your sound and tone to make the dubbed audio feel authentic. If you choose to lip-sync, the tool will also match your mouth's movements to the translated audio. 

Once the translated reel is published, it will be marked as Meta AI-generated. Viewers will see it in their preferred language and have the option to remove the translation in the three-dot menu. 

Creators, on the other hand, will also see a change in metrics. The views will be dissected based on language "to help you better understand how your translations are performing."

For now, the tool works best for face-to-face camera videos, Meta notes. It supports dubbing of up to two people in a video, but for accurate outputs, the two speakers shouldn't overlap with each other or have excessive noise in the background. 

Apart from automated audio tracks, Meta now also allows creators to upload 20 of their own dubbed audio tracks to a reel. To upload them, head to the Meta Business Suite and follow the instructions mentioned in a separate support page . However, there's a minor caveat here. If you have already dubbed your reel using Meta AI, you won't be allowed to add your own dubbed audio tracks manually.