This July was full of excitement for video production. Firstly, in response to the many creative ideas, realized using Apple’s newly announced ARKit, Sky made a quick Pacific Rim video using HoloLens Actiongram. The warm reception of this video on Twitter was a pleasant surprise for all of us.
— Sky Z (@skyzdev) July 5, 2017
Later in July, we released 2 videos made with HoloLens, as entries towards the Dota 2 Short Film Contest of The International 2017. Although we don’t have high hopes for winning, we believe these 2 videos showcase the possibility of creating an entertaining film using mixed reality actions, and they were plenty of fun to make!
In filming these short clips, the director could see the 3D animations and live actions while filming, and immediately make adjustments as needed. The actors were also able to view the 3D animations on set and understand the director’s vision in advance of filming, and thus were better prepared for their roles and actions. The camera man was able to move through the scenes in many angles, and thus provide different perspectives. Moreover, there is no extensive requirement for post-processing because there is no need for inserting 3D animations and merging video streams with the live action footages.
The game characters in the videos were controlled via gaze, gesture, and voice commands. Additionally, the main player could be controlled via a wireless Xbox controller for further flexibility and responsiveness. Even though the character actions required some coding, the code base can be exported and reused for other projects. Some advanced HoloLens features were utilized to add another level of realism, such as spatial sound, spatial occlusion, and physical collision with real world objects.