Camera captures the world as animals see it, with up to 99% accuracy

It’s easy to forget that most animals don’t see the world the way humans do. In fact, with infrared and ultraviolet sight, many animals experience a world that is completely invisible to us.

Now, however, scientists have developed hardware and software that allows you to capture footage as if it was filmed through the eyes of animals such as honeybees and birds.

It’s an intriguing, revealing look at nature and animal behavior, and one the researchers from the University of Sussex and the Hanley Color Lab at George Mason University believe will have a wide range of uses. As such, they’ve made the software open-source, encouraging everyone from nature documentary makers and ecologists to outdoors enthusiasts and bird-watchers to take a peek inside these animals’ very different visual realities.

“We’ve long been fascinated by how animals see the world,” said senior author Daniel Hanley. “Modern techniques in sensory ecology allow us to infer how static scenes might appear to an animal; however, animals often make crucial decisions on moving targets (detecting food items, evaluating a potential mate’s display, etc.). Here, we introduce hardware and software tools for ecologists and filmmakers that can capture and display animal-perceived colors in motion.”

Color, depth and other vision capabilities are determined by our eyes’ photoreceptor makeup, as well as other biological hardware such as the cones and rods assembled. Animals like vampire bats and mosquitoes can perceive infrared (IR) light, while butterflies and some birds can see ultraviolet (UV) light. Both are beyond the spectrum of color that humans can see.

Naturally, this makes it difficult for humans to fully understand animal behavior and how we may inadvertently be impacting their ability to communicate and find food, shelter or a mate. So far, our ability to capture their vision through methods such as spectrophotometry have been time-consuming, relied on specific light conditions and have not been able to record moving images.

And this is where the researchers’ novel development differs. The researchers have painstakingly engineered a tool using multispectral photography, which captures light in diverse wavelengths, including in the IR and UV ranges. The camera records video in four color channels – blue, green, red and UV – which are then processed to deliver footage as if it’s through the eyes of a particular animal, based on what we know of their eye receptors.

The team created a portable 3D-printed device that contains a beam splitter, separating UV from visible light, with each captured by a dedicated camera. On its own, the UV-sensitive camera doesn’t record perceivable data, but when paired with the other, they together record high-quality video. Algorithms align the footage and present the visuals in the perspective of different animals’ sight. It had a mean accuracy of 92%, but some tests returned 99% positive results.

The hardware is designed to suit commercially available cameras, and the researchers have made the software open-source, in the hope that others may want adapt it for their own specific wildlife-filming needs.

While it has limitations – it can’t capture polarized light and has a limited frame rate, so would struggle to capture speedy creatures – it offers unique insights to further our understanding of animal behavior and help us moderate our impact on the natural world.

As for the footage?

The team filmed a museum specimen of a Phoebis philea butterfly in avian receptor noise-limited (RNL) false colors. The researchers noted: "Another potentially useful application of the system is the fast digitization of museum specimens. This butterfly possesses both pigmentary and structural UV coloration. Bright magenta colors highlight the predominantly UV-reflective areas, while the areas appearing purple reflect similar amounts of UV and long wavelength light. The specimen is mounted on a stand and slowly rotated, showcasing how the iridescent colors change depending on viewing angle.

A caterpillar’s anti-predator display in Apis (bee) vision.

"Conceal and reveal displays can pose a problem for spectroscopy and standard multispectral photography," the researchers said. "Here, we show a video of a black swallowtail Papilio polyxenes caterpillar displaying its osmeteria. We illustrate this video in honeybee false colors such that UV, blue, and green quantum catches are shown as blue, green, and red, respectively. The (human) yellow osmeteria as well as the yellow spots along the caterpillar’s back both reflect strongly in the UV and appear magenta when the colors are shifted into honeybee false colors (as the strong responses on the honeybee’s UV-sensitive and green-sensitive photoreceptors are depicted as blue and red, respectively). Many predators of caterpillars perceive UV, and accordingly, this coloration might be an effective aposematic signal."

Bees foraging and interacting on flowers in Apis vision. The team noted: "The camera system is capable to capture naturally occurring behaviors in their original context. This is illustrated with three short clips that depict bees foraging (first and second clips) and fighting (third clip) in their natural environment. The videos are shown in honeybee false colors (displaying the honeybee’s UV, blue, and green photoreceptor responses as blue, green, and red, respectively).

Finally, iridescent peacock feather through the eyes of four different animals. In this case, its own species, the peafowl, plus humans, honeybees and dogs.

"The camera system can measure angle-dependent structural colors such as iridescence," the team explained. "This is illustrated here through a video of a highly iridescent peacock (Pavo cristatus) feather. The colors in this video represent (A) peafowl Pavo cristatus false color, where blue, green, and red quantum catches are depicted as blue, green, and red, respectively, and the UV is overlaid as magenta. Although broadly similar to a standard color video, the UV-iridescence (annotated in the video at approximately five seconds) can be seen on the blue-green barbs of the ocellus (“eyespot”). Further UV iridescence can be seen along the perimeter of the ocellus (between the outer two green stripes). Interestingly, the iridescence is more notable to the peafowl than to (B) humans (standard colors), (C) honeybees, or (D) dogs.

The study was published in the journal PLOS Biology.

Source: University of Sussex via Scimex