My SPW NewsPage View

Digital colorist Dave Franks enjoys helping to deliver bleeding-edge projects, like director Tony Kaye’s VR experience: “Pure McCartney VR”— a five-part series of 3D virtual reality (VR) experiences, produced by Jaunt Studios (www.jauntvr.com).

Innovation in the delivery of musical content has been a passion for Paul McCartney, so the VR space was a perfect medium for him to collaborate with director Tony Kaye (http://www.imdb.com/name/nm0443411/) and Jaunt to release these episodes. Each one of them is a unique vignette of McCartney discussing the origins of some of his most iconic music and other anecdotes within environmental montages of his artwork, photographs, and related imagery.

All live-action VR footage was captured at McCartney’s rural barn recording studio, and is intermixed with performance footage and music videos from the past four decades, along with CG and other imagery that help bring a particular experience or anecdote to life.

Three of the “Pure McCartney VR” sequences have been released and two others will be released later this month. Get in the McCartney groove at www.jauntvr.com

We recently sat down with Dave Franks to discuss the post-production process for this “Pure McCartney VR” Project. Learn more about Dave Franks at http://www.imdb.com/name/nm1389400/?ref_=fn_al_nm_10.

What cameras were used for the shoot?

Director, Tony Kaye, used a combination of the new Jaunt ONE camera system (with its 24 multi-direction stereoscopic cameras) as the main camera, along with Red, and other still cameras to capture ancillary footage and create stop-motion animation of artwork and other imagery relevant to each sequence. Jaunt’s team used Jaunt Cloud Services, their cloud base auto-stitching tools, to assemble 24 HD (1920 x 1200) feeds and deliver stitched footage through to editorial in 3D at 4k.

Each of the five VR sequences has its own look and is a complex assembly of the recently captured Jaunt VR 3D footage with decades of old 2D performance footage, composited with stop-motion 2D and CG animation as needed for the storytelling.

How did you manage the color grading and finishing?

I used Assimilate’s new SCRATCH VR Suite of tools. Having graded dozens of 2D and 3D features in SCRATCH for over the past decade, I found the VR Suite to simply be an extension of that well refined workflow –with all the needed VR effects and controls streamlined into one intuitive tool set. The grading, compositing, and other finishing tools are also ideal for working with this type of VR imagery, to craft and hone the look of each sequence in client supervised sessions.

For each “Pure McCartney VR” episode, this meant working with Tony to create the right look. For weeks following the UK shoot, our talented editor, Duncan Shepherd, painstakingly designed the VR environments with Tony at Jaunt Studios in Santa Monica.

The finishing work began after ingesting the footage and multiple matte layers, following the sequences turnovers from Duncan. The first step for me was to get a sense of each sequence on my own while grading unsupervised.

I wanted to play a little, before working with Tony, to see how the VR experience would respond to various looks. Tony and I would then kick-off each sequence together, setting looks after already having had some familiarity with the work – the scope of the experience, its story, environments, and theme. Following those sessions, I would then work on my own to refine the look we developed together for each vignette, along with doing technical image clean-up work to make the sequences feel more organic. Then, finally, I’d review again with Tony for final buy-off.

A critical aspect of finishing these types of bleeding-edge projects is to be able to precisely discern between conscious creative choices and unintended technical problems that may arise in post-production. Being able to know the difference by being buttoned-up technically, from a calibration, set-up, or imaging standpoint, allows me to work well with a client to make confident color choices together, because I know where the boundaries are.

The client isn’t often technical or doesn’t concern themselves with the underlying technical details – nor should they. But it’s critical on a show for me to know that the technical details are sorted as much as possible to then be able to fly creatively with a Director or DP developing a look together. In this way, we can confidently focus on the work to create compelling imagery that supports their vision for the project.

What challenges did you face for the post production of “Pure McCartney VR”?

With VR and 360-video post production, the challenge is often how to navigate the finishing stages and ensure the color remains consistent throughout the entire process – given that we truly can’t control the final output devices that are used to display the content. This often creates collaborative finishing challenges, especially if your ultimate client is viewing the work remotely, for example, in the UK or on location (on tour), as was the case with this project.

Reviewing the content on the various viewing platforms, such as with an Oculus Rift and Samsung GearVR  headsets, or in 360 on smart phones or a desktop machine, the color space and calibration can be quite different, creating a technical problem because each viewer isn’t having the same experience or seeing things the same way.  At Jaunt, we’re working toward minimizing these issues by refining existing standards, but right now, it’s a bit of a VR wild-west for color and other imaging processes that have long been standardized in other media spaces.

How do you control the color calibration issues to get the best result for viewers?

After calibrating the entire system using a probe or other colorimeter, I double check the environment (or multiple environments in the case of VR,) by using a set of confidence images I’ve carried with me for over a decade. This confidence imagery lets me know “where I am” using images I’ve seen literally hundreds of times, in dozens of varied viewing environments. Once I’ve a sense how these images look in a particular environment, I know by eye where the boundaries are, and I’m then able to set aside all of the underlying technical worries and focus on the creative color and finishing choices with a client.

Which headset did you use?

When color grading or doing any of the finishing tasks, I work directly with a calibrated viewing monitor, and navigate in 360 space using SCRATCH’s 360 tool set. In client-review sessions, and to verify the accuracy of my work throughout the post process, I’ll QC the work using an Oculus Rift or GearVR headset.  Using these head-mounted-displays, the client and I are fully immersed within the environment and can easily identify color, 3D or other VFX related tweaks that need to be made.

What aspect of the project did you enjoy the most?

This may sound cliché, but I actually enjoy the whole process: from working alone and initially banging on the ungraded work delivered from editorial, to helping to design stereoscopically compelling shots, to setting looks, to performing imaging fixes in the background that help to focus the eye and keep the imagery looking organic.  It’s personally very rewarding to do this type of work, at a high-level, on significant projects, using a combination of my creative experience and technical ability to help a client better tell his/her story using 3D visual effects, color and imaging.

I learn a lot from every project – from the collaboration with the creative team, to the content itself, and just by experimenting in a quiet moment on my own with what works and doesn’t work.

What applications do you foresee for the future in VR?

I’m excited by VR and its future possibilities – from entertainment and gaming, to medical exploration and simulations. From the entertainment standpoint, where I’ve been focused for many years, I think the tools (filmmaking language) for how to use VR in storytelling are still being developed. Prior to working at Jaunt, I had seen many experiences that seemed very rudimentray. And sometimes it was as if the storytellers were trying to fit square-pegs into round-holes – using VR where it might not quite work. But I praise these attempts to get in and work with the VR medium to find new ways of telling compelling stories that engage a viewer in this fully encompassing environment – quite a leap from traditional (proscenium framed) 2D or 3D viewing environments.

Learn more about the SCRATCH VR Suite and access tutorials at www.assimilateinc.com.

Pure McCartney: Dance Tonight: Paul takes us through the charming creation story of this song, and its music video. A VR reimagining, complete with a surprise ending performance. Directed by Tony Kaye. Soundscape by Geoff Emerick.

Pure McCartney: Coming Up  Be present with Paul McCartney as he takes us through the creation of the original "Coming Up" music video, complete with archive stills, and a continual pulse of color and image. Directed by Tony Kaye, Soundscape by Geoff Emerick.

Pure McCartney: My Valentine  A Valentine's journey... the writing of a love song, and friends coming together for the crafting of an artistic and poetic music video. Join Paul for an intimate view into his creative world.

Pure McCartney: Mull of Kintyre  Join Paul on a journey through memories of the Scottish countryside, the writing of one of his most enduring hits, and the making of a seminal music video. Directed by: Tony Kaye, Sounscape by: Geoff Emerick

Pure McCartney: Early Days  An emotional journey with Paul through Early Days and the formation of The Beatles. A visual and aural landscape of light, sound, and a few kids from Liverpool. Directed by Tony Kaye. Soundscape by Geoff Emerick.