Binocular 3D vision or “stereopsis” is a remarkable ability. Long thought to be limited to vertebrates, it is now known to have evolved independently in an insect: the praying mantis. Yet we currently know almost nothing about the capabilities of mantis 3D vision or how 3D can be implemented in an insect’s simple nervous system. My Leverhulme Research Leadership Award aims to uncover how mantis 3D vision works. This will suggest ways of engineering robot 3D vision with ultra-low size, weight, and power consumption. It will shed new light on why and how 3D vision evolves. And it will help us understand our own, vastly more complex, 3D perception.
To achieve this aim, we need a lot of people with a range of different skills. Dr Vivek Nityananda is a behavioural ecologist who designs and runs experiments to find out what mantis 3D vision can see. He’s been helped by Dr Diana Umeton and many students working in the lab (so far 11 undergraduates, 8 Masters and 2 PhD students have worked on the project). Dr Ronny Rosner is a neurophysiologist and neuroanatomist who finds out how and where in the mantis brain their 3D vision is achieved. Dr Ghaith Tarawneh transforms this knowledge into computer simulations of mantis vision. Adam Simmons looks after the mantids for us – no easy task as we can have several dozen at a time, and they all have to be housed in separate cages or they might eat each other! We work with many other collaborators, both here at Newcastle and further afield. For example, we’re developing links with robotics groups to see if we can create robots with insect 3D vision.
3D glasses… for insects?
Our first challenge was to figure out a way of presenting 3D images to an insect. After a lot of false starts, we eventually found a solution – blue and green coloured filters which we cut into the shape of spectacle lenses and glue onto the insect’s face with a dab of molten beeswax. This doesn’t seem to bother the insects and they wear their “3D glasses” and hunt quite happily back in their home cages. But it means that we can present different images in the blue and green channels of a computer monitor, and so manipulate what the insects see in 3D. This enables us to ask questions about how their 3D vision works – changing different aspects of the images and seeing what disrupts insect 3D vision, and what doesn’t affect it. This enables us to develop computational models of what the insect brain must be doing.
So what have we learnt so far?
Our major finding so far is that mantis 3D vision works very differently from human 3D, and indeed from almost all machine forms of 3D vision. Human and most machine 3D works by comparing the small differences between the detailed patterns of light and dark in the two eyes’ images. So, if we disrupt these patterns, humans can no longer detect the 3D depth. But we found that praying mantises only look for locations in the image where things are changing. They don’t care about the detailed pattern of light and dark, or whether it matches up between the two eyes. This means that the insects actually perform better than humans when we mess up the pattern. It was amusing to see undergraduates being outperformed by an insect, especially since scientists used to think that 3D vision was too complicated for insects to do at all. We created a video to try and explain this better.
In terms of the underlying brain mechanisms, Dr Ronny Rosner, working with collaborators in Germany, has discovered a new region within the area of the mantis brain known to be specialised for vision. This region, named the “stalk lobe” seems to be peculiar to mantids – it’s not found in their close relatives, the cockroach and locust, which we think don’t have 3D vision. Ronny’s also found neurons in the mantis brain which link the stalk lobes on either side of the brain. These neurons seem a good candidate to be involved in binocular and 3D vision. Ronny is recording electrical activity in these neurons while the mantis views 3D images. We hope this work will reveal more about how the tiny mantis brain achieves such good 3D vision.
To find out more
Check out our website: https://www.jennyreadresearch.com/research/m3/
Our scientific publications can be downloaded from: https://www.jennyreadresearch.com/research/m3/m3-publications/