Transport Your Show Anywhere in the World
May 08, 2015 by Chuck Baker
**A Talk with Rex Olson about TriCaster LiveSet™ Virtual Set Technology **
NewTek’s Vice President of Media Production Rex Olson oversees the development and production of NewTek’s video content for all media. Prior to joining NewTek full-time, he operated his own production companies where he produced television programs, commercials and hundreds of videos and DVD titles.
In this interview, we ask Rex about the advantages producers can gain with TriCaster™ LiveSet™—our own virtual set technology—in multi-camera production, and the new possibilities they can consider with the added power of Holographic LiveSets.
Hey Rex! Tell us what NewTek’s LiveSet technology is all about. LiveSet is what makes it possible for producers to create multi-layered virtual sets within TriCaster. It integrates all the keying and matte-generating technology right in the system, along with live source inputs, realistic lighting and reflections, and multiple angles to switch between. By including all that, it means you don’t need any external equipment in order to create your own, custom sets, and instead of constructing a physical studio, props and furniture, you use digital files, imagery and video sources. It’s hard to believe, but all you really need to appear in an entire professional studio is just a few square feet of green screen.
What makes it different from typical virtual sets and chroma keyers? Because of the variety of angles, sources and layers you can composite in a single set, LiveSet gives producers many more creative options than typical virtual sets and keyers, which appear flat and unrealistic by comparison. A lot of other systems give you only one channel, or one angle, of a virtual set—you’d need to purchase more channels if you want a second or third angle. By contrast, every TriCaster model builds in multiple angles, so already you’ve doubled or tripled your possibilities without additional cost. And each angle you set up includes the capability to layer in several video sources. You can use live camera inputs covering all your on-screen talent, or you can use other video sources to feed on-screen digital signage or virtual monitors. These elements can be sized and positioned so you can create a photorealistic composition that places the video source anywhere on the set, layered in front of or behind other elements with surfaces that can actually reflect the sources and all their real-time movements, for a highly realistic, multi-dimensional feel.
What’s the advantage of using LiveSet virtual sets? It all comes down to improving your look, enhancing your brand, and how polished you want your show to appear. You can start with a small, basic production and make it look more professional, which helps maintain credibility with viewers and sponsors. Or if you have a professional production already, you can create a different design, brand or identity depending upon each program’s needs. You can have a spacious professional set for your show, or a series of dedicated sets for a whole lineup of shows for that matter, all in just a small studio space with a small green screen behind your onscreen talent.
You can also take your talent out of that studio to anywhere in the world you want to go without anyone moving an inch, with a Holographic LiveSet. This capability offers producers some pretty flexible and affordable ways to add instant production value to their shows.
Holographic LiveSet? What’s that? A Holographic LiveSet takes our virtual sets to a whole new level of immersion. The basic idea is that, rather than compose a virtual set with a couple of fixed camera angles to emulate a physical studio, this is a new type of virtual set developed by NewTek that lets you create a full 360 degree virtual environment from your own panoramic imagery. Since it’s a panoramic environment, the camera behaves as if it is mounted on a tripod right in the center of a 360-degree set, with the ability to pan and tilt in any direction throughout the set.
The beauty of this is that since you’re using your own imagery, you can put your talent anywhere in the world. You want your show host to be in Paris in front of the Eiffel Tower? You can do that with a Holographic LiveSet.
How are Holographic LiveSets different from other Virtual Sets on TriCaster? The easiest way to describe it is that—because the camera’s POV (point of view) is in the center of an environment and the talent is placed in that scene—the audience feels more immersed in a location…
With standard LiveSets, having one virtual camera angle per live camera shot allows you to zoom and position within that shot. With a Holographic LiveSet, the virtual camera can pan and tilt 360 degrees in any direction and completely at will. That’s what really adds to the “you are there” realism for your viewers. For the close-ups, you can use the standard virtual set with that scene in the background to provide the cutaway angles for each subject in the shot, for additional shot variation.
So a LiveSet simulates a studio set. Does a Holographic LiveSet simulate an on-location site? You can create any environment you want in either case, but if the advantage of our standard LiveSets is that they eliminate the space, resource, and financial requirements usually needed for a highly professional studio environment, think of a Holographic LiveSet as simulating an on-location environment: it minimizes travel, cost and logistics – while letting you produce virtually from any site in the world, including made-up locations.
Even if producers accept the costs and logistics of remote productions, there’s a bigger challenge—and that’s all the unexpected “x-factors” that can take place in a real environment. Is it an outdoor shot? You need to be prepared for the weather. Is it in a crowd, and will they get in the way of the cameras? Is there lots of noise? Traffic rumbling by? Reliable power sources? All these things that require so much tight coordination totally evaporate when you can capture that environment and produce the segment in a Holographic LiveSet instead. You control everything, right down to the weather.
Is this an extra cost add-on for TriCaster? Holographic LiveSet capability is a standard feature on the professional TriCaster models, with several sets included at no extra cost. That means you can immediately use those sets, or you can work with a broadcast designer who creates custom TriCaster sets, import those and use them. To create your own LiveSets, either standard or Holographic, you would purchase our optional VSE (Virtual Set Editor™) v2.5 software and install it on your TriCaster system or on a separate workstation.
How do I make a Holographic Virtual Set and get it into TriCaster? A lot of DSLR cameras, as well as mobile phones and tablets, already have the ability to take a panoramic picture. You just turn on the panorama function and take the shot by systematically aiming the camera all around and shooting until you have captured the complete image of the location in all directions from your central position.
If you have a camera without the built-in panoramic capability, making a panorama for a Holographic LiveSet is still pretty easy. Just pick a location, stand in the center, and take a series of pictures all around you. Make sure there is some overlap between each picture and the ones adjacent to it on either side. Now all you need to do is to take those images and combine them all into one big image containing the entire environment you just photographed. Any photo stitcher application, like Microsoft ICE, can do this automatically.
Once you have the panoramic image, take that file from your image editor program, load it into VSE, save it as a LiveSet to TriCaster, and it is ready to use.
Super Tip: Here is the technique I use: Capture one set of frames from ground to sky (bottom to top), stay in that spot and turn clockwise just one increment; now capture each frame top to bottom, turn clockwise another increment; now go bottom to top, then turn another increment, top to bottom, and so on. Keep doing this until you have covered the entire area, or as much of it as you want to use. Again, all pictures must overlap slightly, and in my experience it takes about 50 shots to get full coverage.
Is the only way to get a panoramic image to take the photo myself? Holographic LiveSets give you all kinds of options for creating your environment. There are stock photo libraries online where you can purchase panoramas. You can also create a model in LightWave 3D if you want to design an environment or an entire world yourself. You would simply need to render it as a 360-degree image.
Can I create different preset camera angles for Holographic Live Sets? Yes, this works the same way. Open the LiveSet Preset Editor, select the preset that you want to use, move the frame for the shot wherever you want it, and it is ready to go. You can have up to 8 presets for each LiveSet.
What are the lighting considerations if I’m putting studio talent into a Holographic LiveSet? In practice, soft lighting will work for most situations and is easy to key. With our keying algorithm, the blend will look good, regardless of the lighting of the panoramic. If you are intentionally striving for detailed subtleties of effect, as with any set design you should match the exposure and style of lighting of the Holographic LiveSet to get the live material to coordinate more seamlessly with the environment. If the set is a sunny scene, then you would consider that type of lighting for a very detailed match, bright with hard shadows. If the scene was of a shaded area or indoors, then consider a softer type of lighting.
Super Tip: The panorama file will be huge, so you’ll need to scale it down; 7000 pixels by 3500 pixels works well.
In any case, it’s always important to use a backlight for a LiveSet to get good separation within the environment. The real power in TriCaster virtual sets is the ability to look professional, realistic and highly custom to your own brand, without expensive add-ons. We include so many capabilities in the system, so you don’t have to spend additional money on production equipment—but good lighting is a small investment you should never be without.
Learn more about Live Production and Streaming
Determine Which TriCaster is Right for You