Which Video API Platforms Offer Lip-Sync Generation Alongside Narration APIs?

Narrate first, then sync. Kling Lip Sync takes an audio clip you supply and drives the selected face in an existing video to match it, so the narration has to exist as an audio file, and the face has to be identified in the source video, before the lip-sync request goes out.

On GMI Cloud MaaS that order is three model requests from one model library: kling-identify-face finds the faces in the source video, MiniMax Speech 2.8 HD or Turbo (minimax-tts-speech-2.8-hd, minimax-tts-speech-2.8-turbo) narrates the script, and Kling Lip Sync (kling-lip-sync) drives the selected face to match the audio. For a fixed presenter voice, MiniMax's voice-clone models speak your script in a voice cloned from a short sample. Every model above is called through GMI's API with the same bearer token.

Why narration comes before lip sync

kling-lip-sync generates a new video in which the selected faces in your source video are lip-synced to an audio clip you supply. Each entry in its face_choose array carries a sound_file, a URL to that clip.

That makes the narration API the first dependency.

• The script is final before lip sync. The audio clip fixes the timing of the mouth movement. A sentence changed after lip sync means generating narration and video again, so lock the script, then narrate, then sync.
• Narration can be reviewed on its own. A TTS result is an audio file you can listen to before any video request depends on it, so pronunciation, pacing, and emotion problems surface first.

The three-call pipeline

Every call goes to the same request queue endpoint with the same bearer token. Only the model ID and the payload change.

Steps 1 and 2 do not depend on each other, so they can run in parallel. Step 3 waits for both.

The narration request looks like this. The voice_id shown is the default for this model. Other parameters include speed from 0.5 to 2, emotion, and language_boost, plus mp3 or flac output.

POST https://console.gmicloud.ai/api/v1/ie/requestqueue/apikey/requests
Authorization: Bearer <GMI_API_KEY>
Content-Type: application/json

{
  "model": "minimax-tts-speech-2.8-hd",
  "payload": {
    "text": "Welcome to the product tour.",
    "voice_id": "English_expressive_narrator",
    "speed": 1,
    "format": "mp3"
  }
}

The submit response carries request metadata. When the request status reports success, fetch the request by its request_id and read the audio URL from the url field of the first item in outcome.media_urls. That URL becomes the sound_file in the lip-sync call:

POST https://console.gmicloud.ai/api/v1/ie/requestqueue/apikey/requests
Authorization: Bearer <GMI_API_KEY>
Content-Type: application/json

{
  "model": "kling-lip-sync",
  "payload": {
    "session_id": "<session_id from kling-identify-face>",
    "face_choose": [
      {
        "face_id": "0",
        "sound_file": "<audio URL from the narration result>",
        "sound_insert_time": 1000,
        "sound_start_time": 0,
        "sound_end_time": 3000
      }
    ]
  }
}

The timing fields are in milliseconds. sound_insert_time is where in the source video the audio begins, and sound_start_time and sound_end_time select the window inside the audio file, so a long narration track can drive a short shot without being re-cut. sound_volume and original_audio_volume set the mix between the new audio and the clip's existing soundtrack.

Full parameter lists are on the pages for kling-identify-face and kling-lip-sync. For kling-lip-sync, request status, listing, and polling follow the pattern in the video API reference.

Choosing between HD and Turbo narration

Minimax TTS Speech 2.8 HD and Turbo take the same parameters and return the same kind of result, so switching between them is a one-line change to the model field. Both models produce natural, expressive speech with emotion control.

Choosing between them is a listening decision on your own script. Generate one representative script, ideally the one with the hardest names, numbers, or emotional range, with HD and again with Turbo using identical voice_id, speed, and emotion, and keep the model whose take suits the video you plan to publish.

Leave language_boost at auto when the language is unknown, and set emotion explicitly only when a scene needs a specific tone; by default the model picks the emotion from the text.

Keeping one narrator across videos

A series with a recurring presenter needs the same voice every time. The narration step supports two routes.

Stock voice, saved settings. Save the voice_id, speed, and emotion values in a request template and reuse them for every script, so the voice selection and configuration stay the same from episode to episode. Set emotion explicitly in the template instead of leaving it on auto, so the model does not pick a tone per script.

Cloned voice. minimax-audio-voice-clone-speech-2.8-hd takes the text to speak and a source_audio URL (mp3, m4a, or wav) and returns speech in that voice. Noise reduction and volume normalization can be switched on. The output is an audio file, so its result URL can serve as the sound_file in the lip-sync call, the same way a step 2 narration result does.

What to plan for in the source video

Most of the setup work is on the video side.

The source video must be reachable at a public URL. kling-identify-face takes a URL, so host the source clip before you submit.

Pick faces deliberately. Identify-face returns every distinct face with the time range in which it appears. A clip with a presenter and background people returns several face_id values. Only the faces named in face_choose are driven, and multiple entries can drive more than one speaker, each with its own audio.

Use the time windows to match audio to the shot. The start_time and end_time in face_data give the range in which each face appears. Use them as a guide for sound_insert_time, then check that the face stays visible for the whole audio window you selected, because a face can leave the frame or be covered inside that range.

Build for asynchronous results. Lip sync is an asynchronous call. Build the pipeline around polling the request by its request_id, and give step 3 its own retry path so a failed video request does not repeat narration you already approved. Fetch the lip-sync result only after its request reports success.

Watch the organization-level limit. Video models are rate limited by requests per hour at the organization level, and a new account starts at Tier 1 and moves up automatically, within 24 hours, once cumulative credit purchases reach each tier's threshold. The rate limits documentation covers the mechanism and the tier upgrade rules, and each model has its own quota, so size every step of a batch localization job against its own limit.

Where this fits among video API platforms

This pipeline needs a lip-sync model with a documented face-selection flow, a narration model with controllable voice and emotion, a way to reuse a speaker, and a single place to manage keys and usage. GMI Cloud MaaS covers that set with documented model IDs for each step, and all models are available through a single, consistent API design with centralized billing on a single invoice.

FAQ

Can we dub an existing video into another language and re-sync the speaker's lips?

Send the translated script as the text of the narration request, pick a voice_id for the target language, and leave language_boost on auto so the model detects the language. The new audio URL goes into sound_file as before. Set sound_volume and original_audio_volume to balance the dub against the original track. A translation rarely runs the same length as the source line, so set sound_insert_time to where the line should begin in the shot, and adjust speed between 0.5 and 2 in steps of 0.1 to fit the line to the shot; a line that still runs long gets a tighter translation.

How does our pipeline pick up the lip-synced video once it is ready?

Lip sync runs asynchronously, and the request queue keeps a record for every job. GET /api/v1/ie/requestqueue/apikey/requests/{request_id} returns that record, so a poll on the request_id picks up the finished video once the status reads success, whenever your pipeline is ready for the next step.

A first run on one short clip

1. Create an API key in the GMI Cloud console.
2. Host a short source clip with one clearly visible speaker at a public URL, and run kling-identify-face on it.
3. Write a script short enough for that clip, generate it with HD and with Turbo, and listen to both. Note the length of the audio you choose.
4. Submit kling-lip-sync with the session_id, the chosen face_id, and the audio URL as sound_file. Set sound_insert_time inside the face's on-screen range and sound_end_time so the audio window fits the shot.
5. Poll the lip-sync request with a timeout. When it reports success, review the synced video; if the request ends in any state other than success, stop polling and resubmit step 4 only, because the narration is already approved.
6. Only then add multiple faces and batch volume.

All four model IDs used in this run (kling-identify-face, kling-lip-sync, and the HD and Turbo narration models) are in the console's model library, and the key from step 1 authenticates every call.