HeyGen Video Translation and Lip Sync: How It Works

HeyGen (heygen.com) can take a video of a person speaking and produce versions in other languages in which the speaker appears to say the translated lines: the voice sounds like them and their mouth moves with the new words. For anyone delivering content to several markets it replaces reshoots and subtitles with something that feels native. This guide covers how the process works, how to prepare and check a translation, and the rights questions you need to settle with clients and speakers before anything ships.

How HeyGen video translation works

The pipeline has four stages, all handled inside HeyGen:

1. Transcription. The original speech is converted to text with timings.
2. Translation. The transcript is translated into each target language you selected. In the current interface you can usually review and edit this text before the voice is generated; names of the option may differ.
3. Voice generation. A voice in the target language is produced that preserves the character of the original speaker’s voice, so the French version still sounds like the same person.
4. Lip sync. The speaker’s mouth region is regenerated to match the new audio, so the lips form the translated words rather than fighting them.

The output is a new video file per language with the original visuals, the translated voice and matched lip movement. Translation into many languages is supported; the exact list is on the platform.

Preparing a video that translates well

The quality of the translation depends heavily on the source. Before uploading:

• One speaker, facing camera. Lip sync is most reliable on a clearly visible face. Group shots, profiles and fast cuts between speakers reduce quality.
• Clean audio. Speech should be clear, with little music or background noise under it. If you can, export a version with music removed or lowered; add music back in your editor after translation.
• Measured pace. Translated sentences are often longer than the originals. A speaker who already talks fast leaves no room, and the translated voice may rush.
• No burned-in captions. Captions in the source language will remain on screen under the new audio. Use a clean version and add target-language captions afterwards.
• Short segments for tests. Translate a one-minute excerpt into one language first. Check voice, pace and lip sync before committing a long video to five languages.

Reviewing before you deliver

Automatic translation is good and still not final. Build a review step into every job:

• Edit the transcript first. Fix mis-heard product names and numbers in the source transcript before translation, otherwise the error travels into every language.
• Review the translated text. Have a native speaker read it for tone and terminology. Formal versus informal address, brand terms that should stay untranslated and local units are the usual issues. Edit in HeyGen before generating the voice where the interface allows it.
• Watch the output with a native speaker. Check pronunciation of names, pacing, and whether the lip sync holds through the whole clip, especially on close-ups.

Keep a short review log per language: who checked it, what changed, what was approved.

Rights, consent and client agreements

Translation with voice matching and lip sync creates new footage of a real person saying things they never said in that language. Treat that seriously.

• Speaker consent. Get written permission from the person on screen that covers translation into the agreed languages, voice replication and altered mouth movement, plus where and for how long the videos will be used. Store it with the project. HeyGen’s own terms also require you to have the rights to the content you upload; read the current terms before a client job.
• Client agreement. State in your quote which languages, how many review rounds, who provides the native-speaker check, and that automated translation needs human review for legal, medical or financial content.
• Disclosure. Some markets expect AI-altered speech to be labelled; agree with the client where a notice appears.
• Usage rights. What you are allowed to do with HeyGen output, including commercial use, depends on your plan. Check the current terms and keep a copy of the version you relied on with the project file.

For a broader record-keeping method for client work, see keep a record of AI work for clients.

Settings at a glance

Translation uses credits or minutes by plan and by number of languages; check the platform’s current pricing page.

Organising multi-language delivery

A translation job multiplies files quickly. One source video in six languages with two review rounds is dozens of assets. Keep them straight with a naming scheme (Title-LANG-vN, for example Launch-DE-v2), one folder per language holding the video, the approved transcript and the review log, and a single master sheet listing status per language: translated, reviewed, approved, delivered. If you also make avatar videos from scripts, the same scheme works there; see the HeyGen avatar video guide.

Try it

Source: a 60-second clip, one presenter facing camera, clean voice track, no captions, no music.
Target language: one (for example Spanish).
Steps: upload; open the transcript and correct any product names; translate; have a native speaker read the Spanish text and adjust formal/informal address; generate with the matched voice and lip sync on; review the output with sound; note changes in a review log.

Once the single-language test passes, add the remaining languages in one job, and change nothing about the source; if a language still sounds rushed, shorten the translated sentences rather than speeding up the voice.