{"id":61141,"date":"2023-12-12T15:01:02","date_gmt":"2023-12-12T15:01:02","guid":{"rendered":"https:\/\/entertainment.runfyers.com\/index.php\/2023\/12\/12\/fuzzy-doors-viewscreen-on-set-ar-puts-cg-characters-and-locations-in-the-viewfinder-techcrunch\/"},"modified":"2023-12-12T15:01:02","modified_gmt":"2023-12-12T15:01:02","slug":"fuzzy-doors-viewscreen-on-set-ar-puts-cg-characters-and-locations-in-the-viewfinder-techcrunch","status":"publish","type":"post","link":"https:\/\/entertainment.runfyers.com\/index.php\/2023\/12\/12\/fuzzy-doors-viewscreen-on-set-ar-puts-cg-characters-and-locations-in-the-viewfinder-techcrunch\/","title":{"rendered":"Fuzzy Door’s Viewscreen on-set AR puts CG characters and locations in the viewfinder | TechCrunch"},"content":{"rendered":"


\n<\/p>\n

\n

Practically every TV and<\/span> film production uses CG these days, but a show with a fully digital character takes it to another level. Seth MacFarlane\u2019s \u201cTed\u201d is one of those, and his production company Fuzzy Door has built a suite of on-set augmented reality tools called Viewscreen<\/a> turning this potentially awkward process into an opportunity for collaboration and improvisation.<\/p>\n

Working with a CG character or environment is tough for both actors and crew. Imagine talking to an empty place marker while someone does dialogue off-camera, or pretending a tennis ball on a stick is a shuttlecraft coming into the landing bay. Until the whole production takes place in a holodeck, these CG assets will remain invisible, but Viewscreen at least lets everyone work with them in-camera, in real time.<\/p>\n

\u201cIt dramatically improves the creative process and we\u2019re able to get the shots we need much more quickly,\u201d MacFarlane told TechCrunch.<\/p>\n

The usual process for shooting with CG assets takes place almost entirely after the cameras are off. You shoot the scene with a stand-in for the character, be it a tennis ball or a mocap-rigged performer, and give actors and camera operators marks and framing for how you expect it to play out. Then you send your footage to the VFX people, who return a rough cut, which then must be adjusted to taste or redone. It\u2019s an iterative, traditionally executed process that leaves little room for spontaneity and often makes these shoots tedious and complex.<\/p>\n

\u201cBasically, this came from my need as the VFX supervisor to show the invisible thing that everybody\u2019s supposed to be interacting with,\u201d said Brandon Fayette, co-founder of Fuzzy Door. \u201cIt\u2019s darn hard to film things that have digital elements, because they\u2019re not there. It\u2019s hard for the director, the camera operator has trouble framing, the gaffers, the lighting people can\u2019t get the lighting to work properly on the digital element. Imagine if you could actually see the imaginary things on the set, on the day.\u201d<\/p>\n

<\/p>\n

Image Credits:<\/strong> Fuzzy Door<\/p>\n<\/div>\n

You might say, \u201cI can do that with my iPhone right now. Ever hear of ARKit?\u201d But although the technology involved is similar \u2014 and in fact Viewscreen does use iPhones \u2014 the difference is that one is a toy, the other a tool. Sure, you can drop a virtual character onto a set in your AR app. But the real cameras don\u2019t see it; the on-set monitors don\u2019t show it; the voice actor doesn\u2019t sync with it; the VFX crew can\u2019t base final shots on it \u2014 and so on. It\u2019s not a question of putting a digital character in a scene, but doing so while integrating with modern production standards.<\/p>\n

Viewscreen Studio wirelessly syncs between multiple cameras (real ones, like Sony\u2019s Venice line) and can integrate multiple streams of data simultaneously via a central 3D compositing and positioning box. They call it ProVis, or production visualization, a middle ground between pre and post.<\/p>\n

For a shot in Ted, for instance, two cameras might have the wide and close shots of the bear, which is being controlled by someone on set with a gamepad or iPhone. His voice and gestures are being done by MacFarlane live, while a behavioral AI keeps the character\u2019s positions and gaze on target. Fayette demonstrated it for me live on a small scale, positioning and animated a version of Ted next to himself that included live face capture and free movement.<\/p>\n

\"\"<\/p>\n

An example of Viewscreen Studio in action, with a live on-set image at bottom and the final shot up top. Image Credits:<\/strong> Fuzzy Door<\/p>\n<\/div>\n

Meanwhile, the cameras and computer are laying down clean footage, clean VFX, and a live composite both in-viewfinder and on monitors everyone can see, all timecoded and ready for the rest of the production process.<\/p>\n

Assets can be given new instructions or attributes live, like waypoints or lighting. A virtual camera can be walked through the screen, letting alternative shots and scenarios occur naturally. A path can be shown only in the viewfinder of a camera with movement so the operator can plan their shot.<\/p>\n

\"\"<\/p>\n

Example assets in a shot with a virtual character \u2013 the model of the girl will walk between the waypoints, which correspond to real space.<\/p>\n<\/div>\n

What if the director decides the titular stuffed bear Ted should hop off the couch and walk around? Or what if they want to try a more dynamic camera movement to highlight an alien landscape in The Orville? That\u2019s just not something that you could do in the pre-baked process normally used for this stuff.<\/p>\n

Of course virtual productions on LED enclosures address some of these issues, but you run into the same things. You get creative freedom with dynamics backgrounds and lighting, but much of a scene actually has to be locked in tightly due to the constraints of how these giant sets work.<\/p>\n

\u201cJust to do one setup for The Orville of a shuttle landing would be about seven takes and would take 15-20 minutes. Now we\u2019re getting them in two takes, and it\u2019s three minutes,\u201d said Fayette. \u201cWe found ourselves not only having shorter days, but trying new things \u2014 we can play a little. It helps take the technical out of it and allows the creative to take over\u2026 the technical will always be there, but when you let the creatives create, the quality of the shots gets so much more advanced and fun. And it makes people feel more like the characters are real \u2014 we\u2019re not staring at a vacuum.\u201d<\/p>\n

It\u2019s not just theoretical, either \u2014 he said they shot Ted this way, \u201cthe entire production, for about 3,000 shots.\u201d Traditional VFX artists take over eventually for final-quality effects, but they\u2019re not being tapped every few hours to render some new variant that might go straight in the trash.<\/p>\n

If you\u2019re in the business, you might want to know about the four specific modules of the Studio product, straight from Fuzzy Door:<\/p>\n