Many years ago, I remember being amazed by the visual effects in movies and wishing to see such effects and stunts in animations and video games. And then, with the advent of green screen technology, it became possible to add CGI creatures and effects into films!
Soon enough, video games and animations started to resemble movies, with incredibly realistic characters, animals, backgrounds, and atmospheric effects. In fact, some movies were so realistic that people couldn’t tell if they were real or not.
What used to be transforming the physical world into a virtual one in post-production has now become transforming virtual worlds into our reality.
In this article, we’ll explore virtual production, its pros and cons, and the different stages involved. Since virtual production is no longer exclusive to big studios, I’ll guide you through the steps to build a successful virtual production setup.
Let’s dive in!
What is Virtual Production?
Virtual production is a modern video production technique that combines traditional filmmaking techniques with digital animation, VFX, computer-generated imagery (CGI), and game engine methodology.
Think of it as a mix of the traditional production process with real actors, real scenarios, world locations, and sets, and the VFX methodology to create digital worlds using a game engine, lighting effects, mocap, motion tracking, and LED walls.
Unlike traditional production processes, VFX artists create 3D assets and build digital backgrounds in the early stages of the virtual production pipeline to use during the shooting.
VFX artists have also been more actively involved in production since the beginning, creating the concepts and ideas and the first rough VFX to present during the pitchvis and previs stages.
The use of LED screens in virtual production replaces the traditional green screen methods. LED walls are prioritized because the virtual worlds can be ready and screened when shooting the scene instead of being left to the end. In this way, actors can interact with the digital world and 3D assets to better understand how the effects will look on screen, and the real-life cameras “copy” the virtual camera movement to film with the same perspective and depth as the virtual world.
Contrary to what many believe, virtual production is a technology introduced previously. It’s more like a development over the years of many techniques used in the past to achieve special effects and other methods from the animation and gaming industry.
A few movies filmed using virtual production methods are Oblivion (2013), Mandalorian (2019), Percy Jackson and the Olympians (2023), which used LED screens and LED volumes. In cases like The Lion King (2019), the movie was entirely filmed using virtual production. Everything in that movie was created using game engines, modeling, VFX, and CGI.
Virtual production is not limited to films and television shows. Video games, animated movies, content creators, and other industries are also starting to adopt virtual production technology.
It’s also not limited to big companies and productions. Many of the techniques can be adopted by many small studios and VFX teams. The game engine, crucial in virtual production, is available to anyone with a computer that can run it.
Pros of Virtual Production
Virtual production has many benefits. Here’s how it can make your production easier.
- Cost Effective
Many studios use virtual production because it requires fewer expenses than traditional filmmaking, such as traveling to locations, building sets, and acquiring or crafting props. Even if scouting a location is needed, only one crew member could use camera tracking technology instead of deploying the whole team. - Reduce Post-Production Times
VFX artists and designers work closely with the director and producers to make important decisions regarding the design of 3D assets and digital worlds, which are typically conceptualized before shooting begins. This early collaboration allows for changes to be made at the start of the process or in real-time during production, giving the team more flexibility to create professional visual effects. It also allows them to seamlessly integrate the CGI elements and the real-life footage in post-production, reducing the work and costs in the final stage. - No More Geographic Restrictions
Virtual production allows you to film anywhere you can imagine, even imaginary worlds. The limitations that used to come with traditional production are gone: if you want to shoot in space, underwater, inside a volcano crater, or in an old civilization city, you can do it! You can also use LED screens to render the virtual world on set in real-time and shoot all those scenes in less time. Finally, the VFX artists can work remotely from anywhere in the world, allowing you to save money and time to invest in other aspects of production. - Less Reshooting, More Continuity
Filming with real-time game engines allows the director to ensure the framing is correct and that you’re using the right camera angles, and it helps actors monitor their performances with VFX. Continuity is a common problem when shooting on location because things can change between shots, especially when shooting a scene in the same area that takes several days. With virtual production, you can pick up a scene where you left it under the same conditions, lighting, and weather, which brings us to the next benefit. - Controllable Weather and Lighting Conditions
In a virtual set, you can create and control the weather you need and avoid the natural conditions of our world. You have more control over the lighting, too, and the LED wall can generate realistic light effects that will reflect and blend more naturally with the talent. This also means no more green and blue spills and less editing! - Better Actor’s Performance
With real-time rendering, actors can see how the final scene will look and react naturally to what they see on the LED wall. For example, if you’re filming next to a volcano erupting, the actors can see the intensity and consistency of the lava and react more coordinated to it with multiple actors.
Cons of Virtual Production
Though minimal compared to its benefits, you should know about the (few) drawbacks of virtual production.
- Technical Knowledge Preparation Virtual production requires you and your team to be proficient in various fields, such as animation, 3D animation, design, game engines, motion tracking, mocap, and more. You must also know how to use digital software such as Maya, Unreal Engine, Unity, Houdini, After Effects, and many other compositions and VFX tools. The learning curve for beginners will be steep, but you should know that this is knowledge that will open many doors in the entertainment industry for you.
- Virtual Production Equipment You can do virtual production on a budget. However, you still need at least a camera, virtual reality equipment, LED walls or green screen materials, a computer, software, various subscriptions, and other virtual production resources and tools that can be expensive. Smaller studios and independent filmmakers might need more equipment to begin with.
- Dealing with Isolation Filming the whole project on an LED wall might be an experience that, over time, can feel isolating for some actors and crew, as opposed to visiting a real location and being in the open.
Virtual Production Stages
In a traditional film production, VFX is added in the post-production stage. Once the shooting has finished and actors have performed with green screens and props to imagine the final scene.
The traditional pipeline could be summarized in pre-production, production, and post-production stages, where VFX artists’ main work is in the final post-production stage. The virtual production process is less linear than the traditional process, with the visual effects team working from the start.
Here’s how the virtual production pipeline goes in general:
- Pitchvis Pitchvis is the first step, where the VFX studio pitches to the production team with a previsualization of the project. They preview part of the script, storyboard, or concept to get the project greenlit.
- Previs Once the project is greenlit, previsualization takes place. At this stage, the VFX artists generate the rough CGI and virtual assets that will be in the virtual environment to make creative decisions. In these preliminary versions of the visual effects, filmmakers and the VFX team prepare everything for the virtual scene, including camera movement and which assets will be on the virtual set and which ones will be on the real set. Previs can also take place in camera VFX (on-set previs). The VFX team is at the shooting location, rendering digital assets in the virtual environment in real-time. Here, the crew can make changes as they shoot, and actors can see both screens, one with the real-life camera footage and the other with the content from the virtual camera.
- Techvis Techvis is everything related to the technical intricacies of virtual production. The production team analyzes and determines how the sequence will be filmed, the virtual camera positions, lighting, what digital assets will be included in which scenes, and what will be used digitally versus on the physical set. Virtual scooting can take place during techvis, where you can visit the environments virtually and learn the limitations and dimensions of the digital world.
- Stuntvis Stuntvis, also called action design, uses motion capture technology to simulate the live-action stunts of the actors. It helps the stunt crew previsualize the stunts that will be choreographed. When motion capture is used for a CGI character or creature, stuntvis helps adapt the actions to the CGI creature.
- Postvis Once all the shots are done, postvis is done to visualize the work done so far. Here, VFX artists add the unfinished or missing VFX to the real-life footage. It provides a better look at the final sequence than the previous stages but offers the option to make last-minute changes. It takes place in the traditional post-production stage where other processes such as color grading, editing rendering VFX, replacing or reutilizing virtual assets from previs, and blending the 3D modeled creatures to the motion captured actor.
- Delivery In the final stage, once everything has been edited, rendered, and finalized, the VFX team delivers the product to be approved and distributed.
What Do You Need for a Successful Virtual Production Setup?
The equipment you need for a successful virtual production includes the following:
- Computer. A computer with enough GPU and CPU power can handle 3D modeling, animation software, and game engines. The computer is the core of your virtual production setup.
- Game engines. The game engine is where you create the digital world and composite your virtual sequences. The most used game engines in the market are Unreal Engine 5 and Unity.
- Virtual reality gear. For virtual scouting and previs, you’ll need a virtual reality headset, such as the HTC Vide or Oculus Quest, or an AR device.
- Cameras. You still need a regular camera for live-action footage.
- LED wall. Depending on the size and budget of your production, you will need an LED screen, an LED wall, or an LED volume.
- Software. To create the assets, you need 3D animation software, such as Maya, Blender, and Houdini. You also need VFX software to render the visual effects and motion graphics and add the final touches with color grading. Some examples of VFX software are After Effects and Boris FX plug-ins.
Final Words
One main difference and benefit between traditional film and virtual production is that virtual production is more flexible. As a filmmaker, you’re involved in most stages of the production, allowing you to enhance your creativity from start to finish and deliver the best possible work.
Many VFX artists with basic equipment can start with virtual production. You can enter the magical world of virtual production by learning how to use the game engines and applying your knowledge of VFX and animation.
FAQ
What is an example of virtual production?
Virtual production has become increasingly popular in the film industry, and by 2024, you can expect to see many more examples of this technique in action. Some notable examples of movies that used virtual production include The Mandalorian, Star Trek Discovery, Obi-Wan Kenobi, Thor: Love and Thunder, and Ant-Man and the Wasp: Quantumania.
Is virtual production VFX?
Virtual production encompasses VFX as well as other filming techniques and game engines.
