On the screen, a guy appears in front of a whiteboard chalked full of mathematical calculations. What is it? "No idea, those belong to my colleagues," laughs Leonard Niekerken. It's still early for him, the reporter had momentarily forgotten. In fact, Niekerken is calling from Princeton, where he did his internship and now has a job as a research assistant. "During my master's I read a lot of articles from this group. When I had to choose a topic for my thesis, I decided to send them an e-mail."
Black box
Never before have daily conversations been so closely scrutinized. Yes, speech had been studied before, but in the lab. Niekerken emphasizes that this differs substantially from his research: "I want to look at behavior 'in the wild'. In previous studies, people had to read out fixed sentences, whereas we recorded completely spontaneous conversations, meaning there was no script.” The conversations came from patients with epilepsy - the disease, incidentally, was not relevant to the study - who spoke to doctors, family and friends during their hospitalization. A microphone in the room recorded everything. Electrodes implanted in the patients’ brains allowed to measure brain activity. And the German compared this with a speech AI model using the same underlying technology as ChatGPT. "You have to see it this way: both the brain and the speech AI model receive sound waves as input and transform it to words we can make sense of. But what happens inside?" The familiar concept of the ‘black box’ described by philosopher Bruno Latour, right? According to Niekerken, that's not entirely true. "We can indeed take a look at this, it’s rather a 'transparent box,' in other words."
In the wild
And what is there to see? In the AI model all kinds of complicated mathematical functions. In the brain, electrical activity in different areas showing "what happens where. So where the phonetic part is, for example." Can you put the two models side by side or is that comparing apples and oranges? Certain processes appear to use both. "But a big difference is that ChatGPT has no intentions, a human does. We are much more than a model.”
Boost
When Niekerken began his internship, he was able to start processing the data right away. It was intense, but for the ambitious German, it felt like an opportunity to be part of a greater project: a publication. "It really was a team effort," he said. And working with that team boosted his confidence: "I felt really valued. When I came up with ideas, they didn't just listen, they let me actually try it out. If I could give one tip to others: 'Don't be shy, dare to take initiative.'"