Masaki Takeuchi is using A.I. to give a voice to the voiceless

Masaki Takeuchi, 24, and Jaesol Ahn, 23, are unapologetically driven — obsessed, one might even say.

Takeuchi likes to grind in video games like Super Smash Brothers and Pokémon, a process that involves spending hours on repetitive tasks for an eventual payoff. Ahn commits a self-own by calling himself an otaku, which is a term in Japanese for a young person with all-consuming interests in electronics and anime, often used in a derogatory sense. Ahn says he chose to study at the University of Tokyo in part for its proximity to Akihabara, a district in central Tokyo famous for its electronics shops.

These obsessive tendencies paid off when the pair developed the concept for Syrinx (pronounced sigh-rinks) with a few other classmates.

Syrinx is a hands-free electrolarynx that enables a person who has lost their voice to communicate with others. An electrolarynx works by producing vibrations that a person’s tongue and lips can then shape into speech, but traditionally, these devices had to be held in one hand and produced a monotonic, robotic-sounding voice.

Here's how it works: When a person realizes they are losing their voice, they record themselves speaking and upload it — before they lose it completely — into a Microsoft Azure machine learning network, which will then be used to generate more lifelike voice sounds from the device later. Takeuchi noted in a pitch for the idea that he thought of people who had lost their voice due to cancer as his inspiration.

Using machine learning, the Syrinx team produced a prototype that more closely mimics natural human vocal patterns and can be tailored to a single user. (The team was recognized this month for their innovation as runners-up in Microsoft’s 2020 Imagine Cup, an annual contest that attracts thousands of submissions.)

This week, Inverse spoke to Takeuchi and Ahn about their innovation. (This interview has been edited for length and clarity, and Takeuchi’s responses were interpreted from Japanese.)

Inverse: WHAT WAS YOUR FIRST EXPERIENCE WITH SCIENCE?

Masaka Takeuchi: The thing that really left an impression on me early in my childhood was going to science museums, particularly how the Earth was created and developed to what it is now. They had these globes with information semi-projected on with the history of the Earth from as early as such things like asteroids hitting the dinosaurs.

WHAT WAS YOUR FIRST BREAKTHROUGH IN SCIENCE?

Jaesol Ahn: The breakthrough with science for me is when I thought that I could make a living off science and become a scientist or engineer. I'm actually from the Republic of South Korea, and in South Korea, there's a special kind of high school where all the students study only math and science.

Inside the science high school, we did a lot of experiments using equipment for normal university students, and the most memorable was an experiment we did with a really high-powered laser. We were actually performing a physics test to calculate the exact value of g — the gravitational acceleration constant — and we got 9.8, and I think about three digits down the decimal. That was kind of a breakthrough.

WHAT WAS YOUR FIRST FAILURE IN SCIENCE?

MT: In an elementary science class, I was doing the classic magnifying glass and refracting point and focal point experiment, and was burning paper as an experiment lightly. But I did it a little too much, and almost caused a fire. The teacher was able to stomp it out.

JA: When I was in high school, in physics class we had to give a presentation to our other classmates on physics, and my part was on potential energy. I did a horrible presentation on that, and all the students and the teachers were just baffled by my explanation of potential energy. I actually failed that class.

WHAT’S THE ORIGIN STORY OF YOUR INNOVATION?

MT: We saw videos on YouTube of people losing their voice because of cancer and talking through machines that made them sound not at all human. Upon further investigation, and going to these communities of people who couldn't talk, one of their wishes was to speak like people. The mission was taking a device that exists already, the electrolarynx, and improving it and making it better, making it hands-free, to produce human voices using the mouth.

HOW IS SYRINX DIFFERENT FROM A TRADITIONAL ELECTROLARYNX?

MT: The big difference is the sound, and the sound quality. A [traditional electrolarynx] sounds like a robot and the new device sounds like a human voice. The way we measured that was through the sound waves, comparing sound waves of humans versus what the electrolarynx produced. We also designed our device to be hands-free.

WHAT’S A PROBLEM YOU’D LIKE TO SOLVE IN YOUR CAREER?

MT: One of the things that I noticed from looking at engineers and people with needs is that they often don't communicate together. And so the people with needs can't get what they need across to engineers. One of my goals is to look at what people's needs are and as an engineer develop a product to solve those needs.

JA: Through doing Syrinx and through the competitions such as the Imagine Cup, I think I got to know a lot of social problems that haven't been solved around the world. We were only looking at voice loss due to cancer, but there were a lot of other social problems with other teams like Parkinson's disease or depression

WHO WAS A TEACHER WHO INSPIRED YOU?

JA: The most inspiring teacher for me is Ifukube Tohru sensei [Tohru Ifukube, a professor emeritus at the University of Tokyo]. When I first met him at his laboratory in the University of Tokyo, he gave me his book on his life’s work solving social problems through engineering. He said, “This is a treasure box. Dig in, because there are so many problems that haven't been tackled by engineers in the past.” That really inspired me and Takeuchi to do this project in the first place.

WHAT’S THE MOST USEFUL PIECE OF ADVICE YOU’VE EVER RECEIVED, AND WHO GAVE IT TO YOU?

MT: I think I learned this from a book — if you want to do something and are passionate and can do it, then make sure you prioritize that. And just because someone above you hierarchically has different priorities, doesn't mean that you should bend.

"You should look at the real-life problems of real people first."

JA: There’s a maker community at the University of Tokyo, and there was one person who said that when doing a project, you should start from the problem, not from the solution. That was kind of a mind-blowing experience for me, because I was always looking for exciting, new-looking technologies such as artificial intelligence or machine learning, but what he said is that you should look at the real-life problems of real people first.

WHAT’S ONE THING MOST PEOPLE DON’T KNOW ABOUT YOU?

JA: When I was in high school, I tried to hide the fact that I'm an anime otaku. I watch Japanese animation in my free time and read a lot of manga stories. Actually, that’s a major reason why I’m studying abroad in Japan right now.

MT: Yeah, he often goes to Akihabara.

JA: One of the reasons I enrolled at the University of Tokyo is because it’s the closest national university to the Akihabara.

MT: One thing people don’t know about me is that when I do game, I go hardcore, especially in Nintendo games such as Super Smash Brothers and Pokémon. As another hobby, I love watching sumo wrestling.

Who’s your favorite character in Smash Bros?

MT: Mr. Game & Watch, he’s a very niche character but he’s been part of the game series for a long time.

Of the three classic starter Pokémon (Bulbasaur, Squirtle, Charmander), which is your favorite?

MT: Charmander, definitely. He’s the coolest.

WHAT’S A PREDICTION YOU HAVE FOR 2030?

MT: This is a little bleak, but Japan might have a natural disaster like an earthquake, or a tsunami or something like that. So I think I would be thinking about how to best prepare but also recover using technology.

JA: I thought of this before the coronavirus pandemic, but I thought that everyone in the world will wear a mask. Not like a face mask, but the full-blown helmet mask and hazmat suit in 2030, maybe not year-round but for flu season.

WHO WOULD YOU WANT TO PLAY YOU IN THE MOVIE OF YOUR INNOVATION?

MT: Ken Watanabe or Ezra Miller — a little mysterious and a cool guy.

WHO’S A SCIENTIST YOU WANT TO SHOUT OUT?

MT: Bill Gates and Steve Jobs. They are the people I shout out because they're visionaries of American technology. Japanese engineers and scientists tend to only see the now, whereas visionaries in a lot of other foreign countries like America and China think about what society will be like in the future.

JA: Richard Feynman and Nikola Tesla.

ANYTHING ELSE?

MT: It's really easy to explore science through books and the internet, but through Syrinx, I really felt that working with my physical hands and tinkering and touching was important.

Young Innovators is an occasional series from Inverse.