“Avatar,” the highest-grossing film of all time, has taken viewers to a new world, Pandora, while advancing the field of virtual production. By leveraging performance capture, LED virtual environments, and advanced 3D imaging technologies, virtual production is transforming modern cinema. While millions have seen “Avatar,” only a fraction understand the magic behind the scenes. Daniel Pillis SM ’24, an alumnus of the MIT Media Lab, aims to expose filmmaking students to this magic. “Motion capture, like that in 'Avatar,' bridges real human movement with digital technology,” Pillis states. “In this digital age, as artificial intelligence becomes more involved in film studios, technology that enables the authenticity of human expression and performance is increasingly important.” As an assistant professor at Emerson College, Pillis teaches these concepts in his filmmaking courses. Each semester, the class travels to MIT, where Emerson undergraduate and graduate students utilize the capabilities of the MIT.nano Immersion Lab to create their own virtual productions. Donning full-body motion-capture suits that connect to the 28-camera OptiTrack system in the Immersion Lab, students become their own avatars—generating virtual characters that dance, fight, or play the guitar like The Beatles. They see their animation data immediately on a computer screen and can change or add to their character’s movements in real time. Later, they take their data back to Emerson to incorporate it into short films for their final projects. “It has been truly gratifying to support this course and to see the curiosity and ingenuity students have brought to the stage,” says Talis Reks, who manages the MIT.nano Immersion Lab. Pillis first learned about the MIT.nano Immersion Lab during his time as a graduate student in Professor Hiroshi Ishii’s Tangible Media group at the MIT Media Lab. Working with colleague Georine Pierre SM ’24, they collaborated on a Haitian folklore dance project, creating a motion capture-driven simulation of Haitian folkloric dance traditions, specifically the sacred Yanvalou dance. When he became faculty at Emerson, Pillis recognized that the Immersion Lab was a perfect fit for enhancing his students’ experiences. “The level of high-end film production that the Immersion Lab supports is out of reach for so many students who would benefit from this technology in their practice,” explains Pillis. The facility is unique, well-equipped, and accessible even to those outside of MIT—there is truly nothing else like it in the Boston area. With the mechanical character animation the Immersion Lab technology allows, the final projects end up light-years beyond what these students thought they could achieve. “They really get into it,” says Reks. “These students are not necessarily trained as actors, but the moment they see themselves as virtual characters, the realistic, granular movement enabled by motion capture fully engages them.” Over the past two years, more than 60 Emerson College students have utilized the Immersion Lab for Pillis’ class. Emerson undergraduate student Nick Forsch received an EVVY Award nomination for his project. “Being able to use the MIT.nano Immersion Lab really elevated my project,” says Forsch, who created “Enter,” a short film about a human transported into a digital world to meet an artificial intelligence. Another undergraduate student, Evan Costa, recently created a virtual recreation of The Beatles on “The Ed Sullivan Show,” capturing a version of each musician’s performance and reconstructing a simulation of 1950s television. Pillis and Reks plan to leverage advanced Immersion Lab technologies in the coming academic year to teach facial animation, hand and finger tracking, multi-modal data capture, and further advances in interactive generative motion capture for their next productions.
Blogger's Review: This project not only showcases cutting-edge virtual production technology but also emphasizes the close integration between education and the creative industry. By allowing students to engage with high-end technology, it fosters their creativity and practical skills. This interdisciplinary collaboration opens new possibilities for the future of filmmaking.