Cinematography on the Fly
May 22, 2017 | MITEstimated reading time: 3 minutes
In recent years, a host of Hollywood blockbusters — including “The Fast and the Furious 7,” “Jurassic World,” and “The Wolf of Wall Street” — have included aerial tracking shots provided by drone helicopters outfitted with cameras.
Those shots required separate operators for the drones and the cameras, and careful planning to avoid collisions. But a team of researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and ETH Zurich hope to make drone cinematography more accessible, simple, and reliable.
At the International Conference on Robotics and Automation later this month, the researchers will present a system that allows a director to specify a shot’s framing — which figures or faces appear where, at what distance. Then, on the fly, it generates control signals for a camera-equipped autonomous drone, which preserve that framing as the actors move.
As long as the drone’s information about its environment is accurate, the system also guarantees that it won’t collide with either stationary or moving obstacles.
“There are other efforts to do autonomous filming with one drone,” says Daniela Rus, an Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT and a senior author on the new paper. "They can follow someone, but if the subject turns, say 180 degrees, the drone will end up showing the back of the subject. With our solution, if the subject turns 180 degrees, our drones are able to circle around and keep focus on the face. We are able to specify richer higher-level constraints for the drones. The drones then map the high-level specifications into control and we end up with greater levels of interaction between the drones and the subjects."
Joining Rus on the paper are Javier Alonso-Mora, who was a postdoc in her group when the work was done and is now an assistant professor of robotics at the Delft University of Technology; Tobias Nägeli, a graduate student at ETH Zurich and his advisor Otmar Hilliges, an assistant professor of computer science; and Alexander Domahidi, CTO of Embotech, an autonomous-systems company that spun out of ETH.
With the new system, the user can specify how much of the screen a face or figure should occupy, what part of the screen it should occupy, and what the subject’s orientation toward the camera should be — straight on, profile, three-quarter view from either side, or over the shoulder. Those parameters can be set separately for any number of subjects; in tests at MIT, the researchers used compositions involving up to three subjects.
Usually, the maintenance of the framing will be approximate. Unless the actors are extremely well-choreographed, the distances between them, the orientations of their bodies, and their distance from obstacles will vary, making it impossible to meet all constraints simultaneously. But the user can specify how the different factors should be weighed against each other. Preserving the actors’ relative locations onscreen, for instance, might be more important than maintaining a precise distance, or vice versa. The user can also assign a weight to minimize occlusion, ensuring that one actor doesn’t end up blocking another from the camera.
The key to the system, Alonso-Mora explains, is that it continuously estimates the velocities of all of the moving objects in the drone’s environment and projects their locations a second or two into the future. This buys it a little time to compute optimal flight trajectories and also ensures that it can get recover smoothly if the drone needs to take evasive action to avoid collision.
The system updates its position projections about 50 times a second. Usually, the updates will have little effect on the drone’s trajectory, but the frequent updates ensure that the system can handle sudden changes of velocity.
The researchers tested the system at CSAIL’s motion-capture studio, using a quadrotor (four-propeller) drone. The motion-capture system provided highly accurate position data about the subjects, the studio walls, and the drone itself.
In one set of experiments, the subjects actively tried to collide with the drone, marching briskly toward it as it attempted to keep them framed within the shot. In all such cases, it avoided collision and immediately tried to resume the prescribed framing.
Suggested Items
Siemens’ Breakthrough Veloce CS Transforms Emulation and Prototyping with Three Novel Products
04/24/2024 | Siemens Digital Industries SoftwareSiemens Digital Industries Software launched the Veloce™ CS hardware-assisted verification and validation system. In a first for the EDA (Electronic Design Automation) industry, Veloce CS incorporates hardware emulation, enterprise prototyping and software prototyping and is built on two highly advanced integrated circuits (ICs) – Siemens’ new, purpose-built Crystal accelerator chip for emulation and the AMD Versal™ Premium VP1902 FPGA adaptive SoC (System-on-a-chip) for enterprise and software prototyping.
Taiyo Circuit Automation Installs New DP3500 into Fuba Printed Circuits, Tunisia
04/25/2024 | Taiyo Circuit AutomationTaiyo Circuit Automation are proud to be partnered with Fuba Printed Circuits, Tunisia part of the OneTech Group of companies, a leading printed circuit board manufacturer based out of Bizerte, Tunisia. on their first installation of Taiyo Circuit Automation DP3500 coater.
Vicor Power Orders Hentec Industries/RPS Automation Pulsar Solderability Testing System
04/24/2024 | Hentec Industries/RPS AutomationHentec Industries/RPS Automation, a leading manufacturer of selective soldering, lead tinning and solderability test equipment, is pleased to announce that Vicor Power has finalized the purchase of a Pulsar solderability testing system.
Lockheed Martin Successfully Transitions Long Range Discrimination Radar To The Missile Defense Agency
04/23/2024 | Lockheed MartinThe Long Range Discrimination Radar (LRDR) at Clear Space Force Station in Clear, Alaska, completed DD250 final acceptance and was officially handed over to the Missile Defense Agency in preparation for an Operational Capability Baseline (OCB) decision and final transition to the Warfighter. In addition, prior to this transition, the system has started Space Domain Awareness data collects for the United States Space Force.
US Department of Defense Selects Intel Foundry for Phase Three of RAMP-C
04/23/2024 | IntelThe U.S. Department of Defense (DoD) has awarded Intel Foundry Phase Three of its Rapid Assured Microelectronics Prototypes - Commercial (RAMP-C) program.