New Tech Enhances Depth-Sensing Camera Capabilities
August 11, 2015 | Carnegie Mellon UniversityEstimated reading time: 4 minutes
Depth-sensing cameras, such as Microsoft’s Kinect controller for video games, have become widely used 3-D sensors. Now, a new imaging technology invented by Carnegie Mellon University and the University of Toronto addresses a major shortcoming of these cameras: the inability to work in bright light, especially sunlight.
The key is to gather only the bits of light the camera actually needs. The researchers created a mathematical model to help program these devices so that the camera and its light source work together efficiently, eliminating extraneous light, or “noise,” that would otherwise wash out the signals needed to detect a scene’s contours.
Lamp Our Camera
A new depth-sensing camera technology developed by CMU and the University of Toronto can capture 3-D information in even brightly lit scenes; a prototype is able to sense the shape of a lit CFL bulb (above) that would create blinding glare for a conventional camera (below).
“We have a way of choosing the light rays we want to capture and only those rays,” said Srinivasa Narasimhan, CMU associate professor of robotics. “We don’t need new image-processing algorithms and we don’t need extra processing to eliminate the noise, because we don’t collect the noise. This is all done by the sensor.”
One prototype based on this model synchronizes a laser projector with a common rolling-shutter camera — the type of camera used in most smartphones — so that the camera detects light only from points being illuminated by the laser as it scans across the scene.
Lamp Regular Camera
This not only makes it possible for the camera to work under extremely bright light or amidst highly reflected or diffused light — it can capture the shape of a lightbulb that has been turned on, for instance, and see through smoke — but also makes it extremely energy efficient. This combination of features could make this imaging technology suitable for many applications, including medical imaging, inspection of shiny parts and sensing for robots used to explore the moon and planets. It also could be readily incorporated into most smartphones.
The researchers will present their findings today at SIGGRAPH 2015, the International Conference on Computer Graphics and Interactive Techniques, in Los Angeles.
Depth cameras work by projecting a pattern of dots or lines over a scene. Depending on how these patterns are deformed or how much time it takes light to reflect back to the camera, it is possible to calculate the 3-D contours of the scene.
The problem is that these devices use compact projectors that operate at low power, so their faint patterns are washed out and undetectable when the camera captures ambient light from a scene. But as a projector scans a laser across the scene, the spots illuminated by the laser beam are brighter, if only briefly, noted Kyros Kutulakos, U of T professor of computer science.
“Even though we’re not sending a huge amount of photons, at short time scales, we’re sending a lot more energy to that spot than the energy sent by the sun,” he said. The trick is to be able to record only the light from that spot as it is illuminated, rather than try to pick out the spot from the entire bright scene.
In the prototype using a rolling-shutter camera, this is accomplished by synchronizing the projector so that as the laser scans a particular plane, the camera accepts light only from that plane. Alternatively, if other camera hardware is used, the mathematical framework developed by the team can compute energy-efficient codes that optimize the amount of energy that reaches the camera.
“We have a way of choosing the light rays we want to capture and only those rays.” — Srinivasa Narasimhan
In addition to enabling the use of Kinect-like devices to play videogames outdoors, the new approach also could be used for medical imaging, such as skin structures that otherwise would be obscured when light diffuses as it enters the skin. Likewise, the system can see through smoke despite the light scattering that usually makes it impenetrable to cameras. Manufacturers also could use the system to look for anomalies in shiny or mirrored components.
William “Red” Whittaker, University Professor of Robotics at CMU, said the system offers a number of advantages for extraterrestrial robots. Because depth cameras actively illuminate scenes, they are suitable for use in darkness, such as inside craters, he noted. In polar regions of the moon, where the sun is always at a low angle, a vision system that is able to eliminate the glare is essential.
“Low-power sensing is very important,” Whittaker said, noting that a robot’s sensors expend a relatively large amount of energy because they are always on. “Every watt matters in a space mission.”
Narasimhan said depth cameras that can operate outdoors could be useful in automotive applications, such as in maintaining spacing between self-driving cars that are “platooned” — following each other at close intervals.
In addition to Narasimhan and Kutulakos, the research team included Supreeth Achar, a CMU Ph.D. student in robotics, and Matthew O’Toole, a U of T Ph.D. computer science student. The research was supported by the National Science Foundation, the U.S. Army Research Laboratory, and the Natural Sciences and Engineering Research Council of Canada.
Suggested Items
Inkjet Solder Mask ‘Has Arrived’
04/10/2024 | Pete Starkey, I-Connect007I was delighted to be invited to attend an interactive webinar entitled “Solder Mask Coating Made Easy with Additive Manufacturing,” hosted by SUSS MicroTec Netherlands in Eindhoven. The webinar was introduced and moderated by André Bodegom, managing director at Adeon Technologies, and the speakers were Mariana Van Dam, senior product manager PCB imaging solutions at AGFA in Belgium; Ashley Steers, sales manager at Electra Polymers in the UK; and Dr. Luca Gautero, product manager at SUSS MicroTec Netherlands.
NetVia Group Acquires Direct Imaging from Mivatek
04/09/2024 | Miva TechnologiesMiva Technologies is pleased to announce NetVia Group, Irving, TX has acquired a new Miva 2400NG Dual Tray Direct Imaging System with 30-micron capabilities for inner, outer and soldermask imaging.
Teledyne to Acquire Adimec
02/13/2024 | TeledyneTeledyne Technologies Incorporated announced that it has entered into an agreement to acquire Adimec Holding B.V. and its subsidiaries.
Real Time with... productronica 2023: MivaTek Global Advances Technology With High-res Imaging System
12/08/2023 | Real Time with...productronicaMivaTek's Brendan Hogan talks about how the company employs Digitally Adaptive Rasterization Technology (DART) in their high-res imaging equipment. He also shares how the blurred line between semiconductors and microelectronics is driving broader application of the imaging process.
Keysight Enables Validation of Arbe 4D Imaging Radar Chipset
11/30/2023 | Keysight Technologies, Inc.Keysight Technologies, Inc. announces that Arbe has selected the E8719A Radar Target Solution (RTS) to test the Arbe 4D imaging radar chipset for automotive applications.