praying mantis vision tech innovations in robotics

Researchers Develop a biomimetic vision system inspired by the ocular structure of praying mantises

Autonomous vehicles sometimes experience collisions due to limitations in their visual systems' ability to accurately perceive static or slow-moving objects within three-dimensional space. This challenge mirrors the monocular vision of many insects, whose compound eyes excel in motion detection and offer a broad field of view but lack depth perception.

Apart from the praying mantis.

The field of view of a praying mantis overlap between its left and right eyes, resulting in binocular vision that enables depth perception in three-dimensional space.

Leveraging this insight alongside advanced optoelectrical engineering and cutting-edge 'edge' computing--where data processing occurs within or near the sensors that capture it---researchers at the University of Virginia School of Engineering and Applied Science have created artificial compound eyes that address significant limitations in the current methods of machine visual data collection and processing. These limitations include issues with accuracy, data processing delays, and the requirement for substantial computational resources.

"Our study of praying mantis vision led us to understand that replicating their biological capabilities in a biomimetic system required the development of new technologies," said Byungjoon Bae, a doctoral candidate in the Charles L. Brown Department of Electrical and Computer Engineering.

Regarding The Biomimetic Visual System

The team's intricately designed 'eyes' replicate natural systems by incorporating microleneses and multiple photodiodes, which generate an electrical current upon exposure to light. Flexible semiconductor materials were employed to emulate the convex shapes and faceted configurations of mantis eyes.

"Creating a sensor with hemispherical geometry without compromising its performance marks a state-of-the-art breakthrough, enabling a broad field of view and superior depth perception," remarked Bae. "This system ensures precise real-time spatial awareness, vital for applications interacting with ever-changing surroundings."

These applications include energy-efficient vehicles and drones, self-driving cars, robotic manufacturing, security surveillance systems, and intelligent home devices.

The recent paper in Science Robotics lists Bae as the first author, who is supervised by Kyusang Lee, an associate professor with an additional role in materials science and engineering.

A key discovery by the team with the lab's prototype system was the potential to reduce power consumption by over 400 times compared to conventional visual systems.

Advantages of Edge Computing

By processing visual information locally in real time, Lee's system avoids the need for cloud computing, thus cutting down on the time and resource costs of data transfer and external computation, while also minimizing energy usage.

According to Bae, the key technological advancement in this research is the incorporation of flexible semiconductor materials, conformal devices that retain exact angles, an in-sensor memory element, and specialized post-processing algorithms.

The essential aspects is that the sensor array persistently monitors scene alterations, detects modified pixels, and encodes this data into smaller sets for subsequent processing.

This method emulates the way insects interpret their surroundings through visual signals, distinguishing pixels between scenes to comprehend motion and spatial information. For instance, similar to other insects--and humans--the praying mantis rapidly processes visual data using motion parallax, where nearer objects seem to move faster than distant ones. Although one eye can achieve this effect, motion parallax alone does not provide precise depth perception.

Praying mantis eyes are noteworthy for their use of stereopsis, similar to human vision, which enables depth perception. This capability is complemented by their hemispherical compound eyes and motion parallax, aiding in environmental comprehension.

According to Lee, an emerging leader in the field of thin-film semiconductors and intelligent sensors, 'The amalgamation of these cutting-edge materials and algorithms facilitates real-time, efficient, and precise 3D spatiotemporal perception.'

'Our research embodies a significant scientific revelation that could serve as an inspiration to engineers and scientists, presenting a sophisticated, biomimetic approach to solving complex visual processing issues,' he noted.

Source