MIT researchers have developed a new chip that can help tiny, low-power UAVs navigate industrial HVAC systems while avoiding obstacles to check for gas leaks. This chip enables small autonomous robots and other battery-limited devices to construct detailed 3D maps of their environments in real-time, consuming only about as much power as a single LED. Such maps allow robots to plan collision-free paths to their goals. Typically, generating such detailed maps requires power-hungry systems and extensive memory to build and store 3D representations of obstacles in the environment.
The MIT team took a different approach by combining an extremely efficient mapping algorithm with specialized hardware designed to accelerate its workload, minimizing both memory and power consumption. This system-on-a-chip consumes only about 6 milliwatts of power, a fraction of that required by other systems. This low-power operation also makes the chip well-suited for lightweight augmented reality headsets that can be worn for extended periods, useful for applications like educational medical simulation or detailed repair and assembly work.
"This paper showcases a key example of how you can leverage co-design of the algorithm and hardware to really push energy efficiency. While there has been much work into compact 3D maps, what stands out about this work is that it also ensures that the process to generate those maps is as efficient as possible. Our chip allows you to store very large maps in a very small space, and do it in a very energy-efficient manner," says Vivienne Sze, a professor in the Department of Electrical Engineering and Computer Science (EECS).
To generate a 3D map that includes the obstacles in its environment, a robot typically demands a lot of power as it must store images captured by its camera and process all 3D pixels in each image multiple times. Instead of representing the environment using 3D pixels (voxels), the MIT researchers utilized a technique that maps obstacles in space using ellipsoidal blobs called Gaussians. The size, shape, and thickness of these ellipsoids can be smoothly adapted to match the shapes of curved objects more efficiently than rigid, cube-shaped voxels. Importantly, the map captures both obstacles and free space around the robot, allowing it to plan a safe, collision-free path.
For their new system-on-a-chip, named Gleanmer, the researchers employed an algorithm called GMMap that efficiently generates a 3D map of the robot’s environment using Gaussians to represent obstacles. Traditional approaches require the robot to load and process each depth image multiple times to adjust the size and shape of the ellipsoids. However, the memory and power needed for this remain too high for many edge devices. To solve this, the researchers invented a technique that can generate highly accurate Gaussians from depth images with only one pass, after which they can discard the images, so the chip never has to store an entire image at once.
As the robot moves through space, it typically sees the same object from different viewpoints, causing some Gaussians to overlap. This can make the 3D map too large to store on an edge device. Fusing overlapping Gaussians makes the map more compact, but this process usually requires the algorithm to process numerous raw pixels stored in memory. The researchers developed a novel technique to perform this fusion process directly on overlapping Gaussians, without needing to revisit the original pixels.
The Gleanmer chip can reconstruct a range of diverse, pre-existing 3D environments and can also reconstruct obstacles and free space directly from live data streamed from an iPhone camera. It generates detailed 3D maps in real-time while consuming about 6 milliwatts of power, requiring only about 2.5% of the power that the best existing chip for map construction would need. By reusing compact Gaussians along the path as it plans, the chip allows a robot to chart a safe trajectory using only about 20% of the energy it would otherwise need.
"We reduce memory consumption by making sure the algorithm is efficient. Then we accelerate the workload performed by that efficient algorithm, so in the end, our chip is as efficient as possible," Li says. The researchers plan to further improve energy efficiency by moving the processing units on the chip closer to the sensors that gather environmental data and explore additional applications, such as using Gaussians to represent schematics, helping AI systems reason about complex blueprints more efficiently.
"Real-time 3D mapping has been the missing piece for small autonomous systems. A drone inspecting a pipeline or a pair of AR glasses navigating a room both need to understand the space around them — instantly, continuously, and at almost no power cost. Gleanmer makes that possible for the first time in a chip you can hold between your fingers," says Karaman. This work is supported, in part, by the MIT-MathWorks Fellowship, Amazon, the U.S. National Science Foundation, and Intel.
Blogger's Review: The development of this new chip marks a significant breakthrough in the autonomous navigation capabilities of tiny robots in complex environments. By utilizing efficient Gaussian mapping technology, the researchers have not only enhanced energy efficiency but also laid the groundwork for future applications in augmented reality and autonomous systems. Exciting to see this technology's potential across various fields!