How AI could help us talk to animals

Vox

7/31/2024

Transcript

00:00Way back in the 80s I noticed that sometimes when an elephant called a member of her family

00:08one individual would answer and everybody else ignored the calling animal and then she would

00:15call again and a different elephant would sort of lift her head up and you know rumble very loudly.

00:24This is Joyce Poole. She's been studying African elephants and their communication for 50 years.

00:30Then I started to think well okay so maybe they have a way of directing a call to a specific

00:36individual but we had no way of detecting that. Decades later she partnered up with Mickey Pardo

00:41who designed a study around her observations. I went out to the field I recorded calls with

00:46careful behavioral observations so we knew who made each call we knew who the call was addressed

00:51to we knew the context of the call. They encoded the acoustic information from the recordings into

00:56a long string of numbers along with the data Mickey collected about the calls. They fed nearly

01:02500 different calls like this into a statistical model and when given the acoustic structure of a

01:08new call the model could predict who the receiver of the call was much better than chance. In other

01:14words evidence suggesting African savannah elephants give each other names. When we posted

01:21it on Facebook somebody wrote back said that the earth just shifted a little bit

01:27and I think that's true. This is just one example of how machine learning is decoding complexities

01:34in animal communication that humans can't detect and now some AI researchers want to take the next

01:39step. Large language models like the ones that power chatbots but built for interspecies

01:45communication. Can we talk a little bit about love? There is still much to be learned about whales.

02:05When researchers study animal communication they usually employ a few methods recording

02:09their vocalizations observing and documenting the behavior and context around those sounds

02:14and sometimes doing a playback to measure the animal's response.

02:18All of these areas are already being impacted by AI. Recordings from the field don't usually sound

02:24like this. They often sound like this. Multiple animals vocalizing on top of one another in a

02:35noisy environment. This is known as the cocktail party problem and it's a common issue in the field

02:40of animal research. But machine learning solved a similar problem in human speech recognition.

02:46AI researchers trained a model called deep karaoke on lots of music tracks where instruments and

02:51vocals were recorded separately. Then also on the fully mixed tracks until it was able to do the

02:57task of separating out instruments and vocals in new music clips. Recently AI researchers have had

03:04some success applying similar algorithms to animal sound recordings which means you can take a clip

03:10of a group of macaque monkeys and single out one discernible call. Researchers could also start

03:19using AI in how they use playbacks in the field. You may have seen AI models that can be trained

03:25on lots of examples of a sound recording and then generate another unique version of it.

03:35Now the last movie woke up. AI researchers are starting to develop similar models for animal recordings.

03:45These are all types of supervised learning. That means that the model gets trained on lots of

03:50examples labeled by humans and in the elephant name study researchers were able to feed a model

03:55their observations which along with the sound data helped them detect something an elephant

04:00calls they couldn't through observation alone. You need to annotate a lot of data. Yossi Yovel

04:05trained a statistical model on 15,000 Egyptian fruit bat vocalizations which was able to identify

04:11the emitter of the call, the context of the call, its behavioral response and who the call was

04:16addressed to. And we annotated them manually or you know I'm already saying this is a restriction

04:21of the study because maybe we're missing something we're humans we're not bats. And that's the problem

04:26with supervised learning models. They are limited by what we humans already know about animal

04:31communication in order to label the training data and we don't know a lot. That's why some AI

04:38researchers say self-supervised models hold the most potential for decoding animal communication.

04:43This is how natural language processing models like CHAT-GBT are trained. Instead of human labeled

04:48examples they are trained on a large amount of unlabeled data and they can sort it according

04:54to patterns and categories it detects all on its own. In the example of CHAT-GBT it learned from

04:59all the books, websites, social media feeds and anything else it could scrape from the internet

05:04and came to its own conclusions about how language works. Every language has a shape that AI discovers.

05:12This is Eze Raskin. He co-founded the Earth Species Project, one of a few organizations that

05:17want to build models like this for animal communication. What he means by language

05:21having a shape is that language models are built out of relationships among words. Words that mean

05:27similar things are placed near each other, words that share a relationship, share a distance and

05:32direction. So man is to king as woman is to queen. So this is the shape of all those relationships

05:38among the English language's 10,000 most common words visualized here by the Earth Species Project.

05:44Flattened out it looks something like this. Something really miraculous happened in 2017

05:49and that was researchers discovered that you could take the shape of any one language

05:57and match it to the shape of any other language and the point which is dog ends up in the same

06:03spot. This idea that similar words can be located in other languages in roughly the same place

06:09is what gives the Earth Species Project hope we could do a version of this for animal communication.

06:14To do a translation without needing any examples, without needing a Rosetta Stone.

06:19This is complicated though because we know that animals don't just communicate with sound

06:23but with other senses too. But Eiza points out that we can learn from the fact that image

06:28generation models like DALI and MIDJOURNEY are built on the same large language model structure

06:32used for text. It turns out behind the scenes it's again these kinds of shapes. There's the

06:39shape that represents sound. The shapes that represents images. Those two shapes get aligned

06:44and now you can translate between images and text. Their expectation is that where non-human

06:52animals communication would line up with ours will tell us even more about what we have in common.

06:58Dolphins look in mirrors and recognize themselves. Elephants too. That's a kind of self-awareness.

07:04One concern with this plan is related to a step in self-supervised learning called

07:08validation. Meaning humans still need to refine these models by grading them on their answers.

07:14How would we do that in a communication so foreign from our own? We also might have too high of

07:19expectations of this overlap or the capacity for having a conversation with a non-human animal in

07:25a shared language and about shared experiences. Hey Kurt, how are you doing dude? So I'm about to

07:30translate that into a meow. We said hi, hi, hi, hi, you know. Next time you want to say how are you?

07:40I do not think that humans should be considered more important than other species but that doesn't

07:45mean that there's no usefulness in distinguishing between language which is this very specific

07:51behavior that at least based on what we currently know seems to be unique to humans and other forms

07:58of communication. In order to build these models the first step is collecting a lot more data on

08:03animal sounds than exists right now. And so I'm actually at the moment building up a database with

08:10all the individual calls so close to 10,000 records in that which is very small actually.

08:18Around the world animal researchers are in the midst of a massive data collection effort

08:22tagging and recording animals with video and sound and spatial data to feed these

08:27data thirsty models. Time will tell whether true interspecies communication will be facilitated by

08:32AI but researchers hope that discoveries along the way will continue to have an impact on our

08:37appreciation and protection of the species we share the planet with. We're not the only ones

08:44on the planet who can communicate, who care about one another, who have thoughts about the past and

08:51about the future. They also have a right to be here and a reason for being here.

How AI could help us talk to animals

Category

Transcript

Recommended