If you’ve ever wanted to create AI-powered movies on your own, the Act-One tool from Runway makes it possible. Act-One allows you to take a simple video recording and transform it into an AI-driven video, complete with lip sync, facial expressions, and even head movements—just like an AI actor. Here’s a comprehensive guide on how to use the Act-One feature in Runway.
What is Runway Act-One?
Runway Act-One is an AI video generation tool that transforms simple video recordings into animated videos with realistic lip sync, facial expressions, and head movements. It uses a reference image to map these expressions onto different characters, enabling users to create AI-driven actors from their video clips.
Getting Started with Runway Act-One
The Act-One tool allows you to upload a video clip of yourself acting, select a reference image to use, and animate it with facial expressions and head movements.
Here’s a breakdown of the steps to help you get started.
Step 1: Record a Video Clip
- Create a Short Clip – Record a video clip of yourself acting out a scene. Make sure the clip is 30 seconds or shorter to ensure smooth processing.
- Focus on Face and Upper Body – Ensure your video captures only your head and upper chest area, with no hand movements. Good lighting and video quality are essential for accurate animation.
- Prepare Reference Images – Select or create a reference image of the character you want to animate. It’s best if the face in the image is facing forward with clear visibility of all facial features.
Step 2: Upload Your Video Clip
Access Act-One Feature – Open Runway, navigate to the Generate Videos tab, and select the Act-One feature.
Drag and Drop Video – Upload your video clip by dragging it onto the upload area. Double-check the video quality to ensure it’s ready for animation.
Step 3: Choose or Upload a Reference Image
Runway Gen 3 offers a selection of pre-made images, including photorealistic images, 3D pixel art, 2D cartoons, and illustrations.
- Using Pre-made Images – If you’re in a hurry, you can select from Runway’s library of images. These are optimized for easy animation.
- Uploading Your Own Image – If you prefer a custom look, upload a reference image of your choice. Ensure the face is centered and visible to help the AI detect facial features accurately.
Step 4: Generate the Animation
Once your video and reference image are uploaded:
- Crop the Image (if Needed) – Sometimes, you may need to crop the image slightly to center the face.
- Click Generate – Hit the Generate button to begin the animation process. Runway will then animate your video clip based on the selected reference image. This process may take a few minutes.
Exploring Animation Styles in Act-One
Runway’s Act-One feature supports different animation styles, so you have the flexibility to match the aesthetic you’re aiming for. For my project, I experimented with a James Bond-style conversation, animating both a male and female character.
Variety of Styles: From realistic photo-based animations to 3D, pixel art, 2D cartoons, and even painting-style characters, there’s plenty to choose from.
Best Results: Through my experiments, I found that photo-realistic images tend to work best, capturing subtle facial expressions and head movements more effectively than other styles.
Tip: While head movements and lip-sync look great, keep in mind that Act-One currently only supports facial and head animations. Full-body animations are not yet available.
Pika vs Runway: Text-to-Video and Image to Video
Adding AI Voice to Match the Character
After creating the animation, you may find that your voice doesn’t quite match your character. Here’s how to change your voice to fit the animated character.
Step 1: Use AI Voice-Changing Software
- Select a Voice-Changing Tool – For this guide, we’ll use Eleven Labs for voice conversion.
- Upload Your Original Audio – Record and upload the audio clip of yourself speaking.
- Choose a New Voice – Browse through the available voices and select one that matches your animated character.
- Generate New Audio – Once selected, generate the new voice to align with the character. This ensures the audio and visuals are in sync, creating a more immersive experience.
Step 2: Sync the Voice with the Animation
- Apply the New Voice – Replace the original audio with the AI-generated voice.
- Create Multiple Characters – To add a dialogue, repeat this process with different voices for each character. This enables you to create conversations between animated characters, enhancing storytelling.
Creating Non-Human Animations
If you want to push the boundaries and experiment with non-human characters, like animals, the Act-One tool has some flexibility here as well.
Step 1: Select or Upload Animal Photos
Try Multiple Images – Non-human images, such as animals, might not always work perfectly. Try different images, as some might work better than others.
Humanoid Features Work Best – Images of animals with slightly human-like features, such as forward-facing faces, are more likely to be accepted by the AI.
Step 2: Generate the Animal Animation
Runway can sometimes animate animal characters with basic lip-sync and head movements, though the expressions may be limited.
Run the Generation Process – Upload your video and reference image (in this case, an animal photo) and click Generate.
Review Animation Quality – Animal animations might not capture all expressions as effectively as human animations, but the lip-sync and basic head movements should still look good.
Tips for the Best Results
Here are some recommendations to get the best results with Runway’s Act-One tool:
- Opt for Photorealistic Images – Photorealistic images tend to produce the most accurate facial expressions and movements.
- Experiment with Different Styles – If photorealism isn’t your goal, try 3D, 2D, or illustration-based images. Each has its own unique animation style.
- Focus on Head Movements – Currently, the tool is limited to animating head and facial expressions only. Full-body animations are not yet supported.
Frequently Asked Questions
Below are some commonly asked questions and tips to further optimize your experience with Runway’s Act-One.
1. How Long Does the Generation Process Take?
The animation generation typically takes a few minutes, depending on the video and image complexity.
2. Can I Animate Multiple Characters at Once?
Yes! By uploading separate video clips and using different reference images, you can animate multiple characters and combine them for a dynamic scene.
3. What File Types Are Supported?
Act-One generally supports common video formats, but for the best results, use high-quality MP4 files.
4. Is There a Limit to Video Length?
Yes, Act-One is optimized for clips up to 30 seconds. Longer videos might not process correctly, so it’s best to keep each clip under this limit.
Summary
Runway’s Act-One feature provides a straightforward way to animate video clips with AI, from realistic facial movements to dynamic voice-syncing options. If you’re animating a fictional character or trying out a non-human avatar, this guide has provided the steps you need to start creating your own AI-driven videos.