OmniHuman 2.0 Prompt Guide & In-Depth Control

Core Principles for Writing Prompts

Writing prompts for OmniHuman 2.0 is different from standard image generation models, as this model focuses on cinematic storytelling and fluid acting:

  • Storytelling Approach: Write in natural, flowing sentences. Avoid stacking isolated keywords.
  • Dynamic Events Only: Only describe moving or changing elements. Do not describe static features already present in the reference image (e.g., clothing, facial features, accessories, or static backgrounds).
  • Clarity & Minimal Negation: Be specific about what you want to happen. Avoid using negative prompts or telling the model what not to do.
  • Trigger Keywords: Always include keywords like “talking”, “speaking”, “crying”, or “singing”. This helps the model process realistic lip-syncing and speech-related body language.

Recommended Prompt Template

To ensure the model fully understands the direction, sequence, and context, it is recommended to structure your prompt as follows:

[Camera Movement] + [Emotion of Speaking Character] + [Speaking State] + [Specific Actions] + [(Optional) Background Events]

Complete Prompt Example:

“The camera slowly moves from the side to a medium-front shot. A young woman is sitting by the window, calm, gently picking up the coffee cup and smiling as she talks to the camera. A boy beside her quietly watches her and occasionally turns to the camera and smiles. The lights in the background are gently flickering.”

In-Depth Model Control

Action Sequence Control

  • Use chronological words like “first” and “then” to dictate complex sequences of actions.
  • Example: “The man walks forward first, then stops and puts his hands on his hips while speaking. Then, he turns around to look at the explosion behind him…”

Choreographing Gestures Relative to Speech

  • You can instruct the character to perform an action before they start speaking, or after they finish speaking.
  • Action before speech: “The man quickly took out the cigarette from his mouth and then started speaking to the camera.”
  • Action after speech: “The woman spoke to the camera, and after finishing, she quickly turned around and ran backward.”

Camera Movement & Rhythm

  • Clearly define the direction, speed, and final focus of the camera (e.g., “The camera moves forward and focuses on [character]”).
  • Add the word “rapidly” to increase camera speed and responsiveness.
  • Caution: Avoid panning the camera so far that the character completely leaves the frame for a long time, as this may cause consistency issues when they return to the frame.

Camera Focus Shift Control

  • Control the camera’s focus between foreground characters and background events to match the narrative.
  • Example: “The reporter is reporting to the camera. Meanwhile, the king behind suddenly collapses. The camera focuses on the collapsed king behind the reporter. The reporter turns around incredibly quickly and speaks while looking behind.”

Emotion & Micro-expression Control

  • Command the character to change facial expressions and emotions dynamically to match the audio context.
  • Example: “First, the man looked calm and spoke to the camera. Then, he suddenly became nervous and panicked, frowning tightly.”

Environmental Interaction & Background Events

  • Control how characters interact with their background or trigger environmental events (e.g., wind).
  • Example: “Holding the camera, the woman looks into the distance. In the distance behind her, fireworks are going off, and the wind blows her hair and clothes.”

Special Effects & Visual Elements

  • Add objects or effects that are not present in the original reference image.
  • Example: “The chick put on sunglasses and held two guns in hand and spoke evilly.”

Best Practices & Limitations

  1. Image Clarity: The final video quality relies heavily on the sharpness and resolution of your input image.
  2. Face Proportion: Avoid extreme long-range shots. If the face is too small in the frame, the lip-sync accuracy may drop.
  3. Randomness: Generation involves randomness. If a scene is complex, try generating it multiple times to get the best result.
  4. Audio Comprehension Test (No Prompt): Because OmniHuman 2.0 has built-in audio comprehension, if your text prompt isn’t working, try generating the video without a text prompt. Let the model automatically infer emotions and gestures directly from the audio file.