Not long ago, at the music awards ceremony held at the US Billboard, Madonna and Latino singer Maluma performed the song "Medellín". During the performance, Madonna danced a dance with her several AR clones on the same stage. It is reported that this special effect is achieved through volumetric photography and shooting 3D videos based on the Unreal engine.
According to Qingting.com, there are four images of Madonna in the performance: Chacha dance teacher, agent, bride and musician. The AR performance was produced by Sequin, and the volumetric photography part was done in a London studio, while the work of rendering digital content and environments in the Unreal engine was handed over to a Canadian company.
In the process of making AR live performances, Sequin needs to consider the issue of live camera tracking. They used a tool called Brainstorm to input live videos, character generation and real-time data into the Unrela engine to ensure the synchronization of real and digital performances.
In live performances, Sequin will also adjust the AR performance based on the live light, shadow, reflection, actual and position overlap.
Live audiences can watch this AR performance through a large screen, and the capture of AR content on the performance site is achieved through three AR-enabled cameras, which are located in the rocker arm, front camera and wireless camera stabilization device. It is understood that the wireless photography stabilization device is unique, allowing photographers to move back and forth on the scene to capture dynamic and natural AR images.
It is reported that Madonna AR performance is Sequin's first work, but the company's co-founders Lawrence Jones and Robert DeFranco also developed AR weather forecasts, League of Legends girls' group K/DA live AR performances, and this year's Super Bowl live AR effects. Their works were nominated for the Emmy Awards.
Similar cases include: the performances performed by Xue Zhiqian and AR version of Luo Tianyi at the 2019 Jiangsu TV New Year's Eve Gala, as well as the Teresa Teng's concert created by Prism Optical Entertainment and Digital Kingdom using holographic technology and CGI. And in 2014, the US Billboard Awards ceremony also revived the performance of "Michael Jackson" through Pepper's fantasy technology, which relies on the illusion generated by semi-inverted and semi-transparent glass to achieve. This article is translated by Qingting.com from: Engadget
