Meta AI can clone your voice in another language: Here's how it works

Meta has launched an innovative AI-driven voice translation feature aimed at enhancing the reach of content for creators on Instagram and Facebook. This groundbreaking tool is designed to automatically dub videos in various languages, ensuring that the original tone and delivery of the creator are preserved, while also synchronizing lip movements for a more authentic viewing experience. Initially, the feature supports translations between English and Spanish, with plans to expand to additional languages in the future. However, it is important to note that the rollout is limited to public Instagram accounts and Facebook profiles with over 1,000 followers, and it is not available in several regions, including the European Union, the UK, South Korea, Brazil, Australia, Nigeria, Turkey, South Africa, and certain U.S. states like Texas and Illinois. Creators can easily enable this feature when posting a Reel by selecting the option "Translate your voice with Meta AI." They have the choice to activate lip syncing, so the dubbed audio aligns with their mouth movements. Once the dubbing is complete, creators receive a notification to review and approve the translation or to disable features like lip syncing before making the video public. A subtle indicator will be displayed on the final video, informing viewers that the translation was performed by Meta AI. According to Meta, this system aims to maintain the authenticity of dubbed content by mimicking the creator's voice instead of using a generic voiceover. Additionally, viewers will have the option to adjust their preferences regarding dubbed videos in specific languages. To assist creators in maximizing their content's performance across different languages, Meta has introduced new analytics tools that provide insights into viewership breakdowns by translation. Best practices have also been shared, advising creators to face the camera, articulate clearly, and minimize background noise to enhance translation accuracy. The current setup allows for up to two speakers per video, provided they don’t speak over each other. Furthermore, Facebook creators can upload up to 20 of their own dubbed audio tracks for a single Reel using the Meta Business Suite platform, offering a flexible way to add translations even post-publication. This announcement follows Meta's initial preview of the AI voice translation tool at last year's Connect developer conference and subsequent pilot tests on Reels. Adam Mosseri, head of Instagram, emphasized that this feature is intended to help creators expand their audiences by bridging language gaps. As Meta continues to refine its AI division with a focus on research and product development, the immediate benefit for creators is clear: they can now engage with a broader global audience through their Reels.