Using a Camera to Spot and Track Drones
August 17, 2017 | EPFLEstimated reading time: 3 minutes
EPFL researchers have shown that a simple camera can detect and track flying drones. Plus, the lightweight, energy-efficient and inexpensive technology could be installed directly on the drones themselves and enhance safety in the skies.
The rising number of drones in air space poses numerous challenges. Topping that list is our ability to simply detect these small unmanned aerial vehicles. Periodic near-misses between drones and large airplanes raise the specter of disaster, and the drones themselves often lack the necessary technology to locate other moving objects. To address these issues, EPFL researchers have developed algorithms capable of detecting and tracking small flying objects using a simple camera. The proof of concept was conducted as part of a PhD dissertation, and a real-time detection and collision avoidance system is now being developed in a project funded by the Commission for Technology and Innovation (CTI).
Today’s collision avoidance systems operate actively: an airplane in flight calculates its position, altitude and course, and communicates this information to other aircraft using the same technology. Those aircraft can then evaluate the risk of a collision based on their own positioning data and, if necessary, alert the pilot. But this system is only effective as long as all aircraft are equipped with the same technology. In reality, drones often lack such systems, which are costly and heavy and consume more power.
Artificial intelligence and Deep learning
A camera can thus be an effective, non-cooperative (i.e., not every aircraft must be equipped with it) addition to that system, provided the camera can successfully detect a flying drone. Therein lay the obstacle that researchers at EPFL's Computer Vision Laboratory (CVLAB) sought to overcome. The biggest challenge for a moving camera is to spot another moving object. This is much more difficult on a drone than it is on a car, which only moves in two dimensions. Drones move in three dimensions, and the camera is called on to detect objects against the sky or the ground, depending on the angle of sight. Plus, drones need to locate objects as quickly as possible, such as when they are still fuzzy black dots against a dark forest. And the fact that no two drones look alike anymore – new models are constantly being developed – meant that the researchers had to find a way to teach the camera to recognize all sorts of drones.
In his thesis, Artem Rozantsev showed that these challenges can be overcome. The first step consisted in using artificial intelligence and deep learning to teach the camera to recognize drones. His method combined information on both appearance (types of drones, position, etc.) and motion (movement in the camera's field of view), as neither alone was capable of achieving sufficiently reliable detection. He therefore proposed a machine-learning technique that operates on spatio-temporal cubes of image intensities where individual patches are aligned using a regression-based motion stabilization algorithm.
Real-time performance and accuracy
But the recognition algorithm on its own was not enough. To train a detector to recognize all types of drones in all kinds of positions, it has to have "seen" as many as possible. The existing database of images, however, is limited. So Rozantsev filled in the gaps by generating realistic synthetic images. The generated images, which are based only on a small set of real examples and a coarse 3D model of the object, are used together with the real examples to train the detector. A key ingredient to his method, the generated images are as close as possible to the real ones – not in terms of image quality, but according to the features used by the machine-learning algorithm.
The researchers managed to develop a reliable algorithm capable of detecting a drone using a lightweight camera similar to those found in smartphones. The aim of the project, now financed by the CTI, is to train a detector using an even larger data set to improve its real-time performance and accuracy. EPFL's CVLAB researchers are working on this in collaboration with FLARM Technology AG, a leading supplier of affordable collision avoidance technology for civil aviation. The first commercial models are expected to be released next year.
Suggested Items
Koh Young Showcases Award-winning Inspection Solutions at SMTconnect with SmartRep in Hall 4A.225
04/25/2024 | Koh Young TechnologyKoh Young Technology, the industry leader in True 3D measurement-based inspection solutions, will showcase an array of award-winning inspection and measurement solutions at SMTconnect alongside its sales partner, SmartRep, in booth 4A.225 at NürnbergMesse from June 11-13, 2023. The following offers a glimpse into what Koh Young will present at the tradeshow:
Real Time with… IPC APEX EXPO 2024: Plasmatreat: Innovative Surface Preparation Solutions
04/25/2024 | Real Time with...IPC APEX EXPOIn this interview, Editor Nolan Johnson speaks with Hardev Grewal, CEO and president of Plasmatreat, a developer of atmospheric plasma solutions. Plasmatreat uses clean compressed air and electricity to create plasma, offering environmentally friendly methods for surface preparation. Their technology measures plasma density for process optimization and can remove organic micro-contamination. Nolan and Hardev also discuss REDOX-Tool, a new technology for removing metal oxides.
Nanotechnology Market to Surpass $53.51 Billion by 2031
04/25/2024 | PRNewswireSkyQuest projects that the nanotechnology market will attain a value of USD 53.51 billion by 2031, with a CAGR of 36.4% over the forecast period (2024-2031).
Cadence, TSMC Collaborate on Wide-Ranging Innovations to Transform System and Semiconductor Design
04/25/2024 | Cadence Design SystemsCadence Design Systems, Inc. and TSMC have extended their longstanding collaboration by announcing a broad range of innovative technology advancements to accelerate design, including developments ranging from 3D-IC and advanced process nodes to design IP and photonics.
SMC Korea 2024 to Highlight Semiconductor Materials Trends and Innovations on Industry’s Path to $1 Trillion
04/24/2024 | SEMIWith Korea a major consumer of semiconductor materials and advanced materials a key driver of innovation on the industry’s path to $1 trillion, industry leaders and experts will gather at SMC (Strategic Materials Conference) Korea 2024 on May 29 at the Suwon Convention Center in Gyeonggi-do, South Korea to provide insights into the latest materials developments and trends. Registration is open.