How AI Audio Conversation Tools Can Help With Learning Languages

So you can gain confidence, as well as listening and speaking skills right away

This has been a long time coming.

As a paid ChatGPT subscriber, I’ve been using ChatGPT’s Voice system to have conversations for quite some time. I started about a year ago with the old voice system we couldn’t interrupt, which still had a rather robotic voice.

It wasn’t the best but it was still quite useful as listening practice for my more advanced languages: Japanese and Korean. Especially during walks.

I asked it to create fake news articles in the target language and proceed to ask me questions about their content. Sometimes, I’d give it a specific topic, such as environment or politics, to learn new vocabulary. Others, I’d ask it to create a much longer news article. Yet others, I’d request it to create a conversation between two people on the topic of that fake news piece.

It was useful but limitations were clearly there:

• I couldn’t interrupt it to ask for the translation of a specific word

• It never felt like a real conversation

• Lower-level languages were out of the question because it wasn’t capable of slowing down its talking speed, something ChatGPT now can do.

Also, it regularly struggled to understand words and responded way off base. I get I was speaking with an accent but this has barely happened once or twice since the new Advanced Voice for ChatGPT.¹

Things have evolved quite a bit since then.

Some caveats

Only two AI tools offer this type of live conversation at the moment: Gemini (from phone only) and ChatGPT.

Gemini is entirely free but misunderstands or needs refreshes often. ChatGPT allows for about 15 minutes or so daily for free users and one hour for paid users.²

Both tools write down the conversation in the background, which is convenient to refer to it afterward for review. But it often miswrites or writes “Transcript Unavailable” for the speaker’s text, even if it did reply on point at the time.

For Japanese, it often writes the wrong kanji despite the context being clear. For example, in a conversation about blind people and braille, it wrote moujin as 毛人 (non-Yamato people in northern Japan) instead of 盲人³, and tenji as 展示 (exhibition, display) instead of 点字 (literally “dot character”).

Alright, let’s dive in.

Vocabulary-learning heaven

No language partner is an expert on every topic you can imagine.

If you want to develop your vocabulary about traditional Korean architecture, you might find one person who can do that but would they also be able to discuss the Japanese braille structure or advances in space as well? Likely not. Even if they could, they’d need preparation to look up a few words/expressions.

With AI voice tools, you can decide in the middle of the conversation that you’re done with one topic and switch to a new one.

You can also ask it to explain new terms in the target language, your native language, or any other language you might know. I often ask the Japanese equivalent for Korean words I don’t know because they’re often similar which allows me to strengthen my Japanese at the same time.

You can also finish the conversation and ask it to create a table filled with the vocabulary you asked about. Or you can ask it to figure out the important expressions on its own, as I did below.

Unfortunately, this one was not so useful as I already understood all of them, but it was good as a reminder

You can then ask it to create Anki (or other SRS) cards to import with a specific format you have in mind. That saves time creating them—although creating them yourself also has its own benefits. Combine this with a tool like ElevenLabs to get an audio version and you’re golden.

One rule I still struggle to get ChatGPT to consistently follow is the request to correct me as we go. Many of my sentences will have mistakes. Some bigger than others, especially in some languages like Spanish or Korean. Sometimes it does it but it’s inconsistent so I need to try a few more things, notably defining more precisely what should trigger the correction.

One roundabout I found is to ask it every few exchanges to fix my mistakes. I wish it were a bit more strict but well. If you have ideas to make it work, let me know!

Instant grammar tutor

Sometimes ChatGPT replies to my audio messages with a grammar pattern I recognize but am not sure of. In these situations, I switch to asking for more example sentences using it or directly explaining it in English if I’m really lost.

Once I’ve got it down, I can just ask it to pick up where we left off. Even if I forgot where we were at, the AI does remember.

Pronunciation practice support

This isn’t quite there yet for all I wish I could do, but I have no doubt 2025 will be the year we’ll start getting this so I’m adding it.

One of the main reasons people—and by that I mean “I”—fear speaking a language they’re learning with native speakers is their pronunciation. They’re afraid people will make fun of them so they—again, I—miss the opportunity.

Well, AI won’t make fun of you. It might misunderstand and frustrate you but at least you’ll have all the time you want to make it repeat or rephrase so it understands.

Unfortunately, it’s also too kind.

I’ve asked it dozens of times to correct my pronunciation, rhythm, and so on in Korean but all I got was some sort of “it’s really good!,” even if I asked it to be strict. I even stumbled three times on a specific word on purpose only to hear it say another one could have been said more clearly.

In the near future, I’m sure we’ll be able to do this but that’s not here yet.

In the meantime, I sometimes ask it to create a conversation or monologue in Korean so I can repeat it multiple times and try to have a similar rhythm, take a breath at the right times, or at least not stumble in the middle of long sentences.

Extend knowledge in a new language

AI Advanced Voice tools like Gemini and ChatGPT allow us to dive into new topics we hadn’t even thought of.

I sometimes start the conversation by asking ChatGPT to provide three or four rather specialized topics, and then I choose one or ask for more. All too often, we stick to topics we know. This allows me to learn new things. Even if it may make mistakes, it doesn’t matter because it usually gets the basics right, and if I truly get interested, I can look it up afterward.

Combining language learning with curiosity is a sure-fire way to not get bored!

Anyway, that’s it for today. Time to go have a quick chat in Korean about tatamis around the world.

Cheers for reading!
Mathias

1. For Japanese and Korean. It happens a lot with Mandarin but I get that 😅

2. Every 24 hours so if you use it one evening, you can use it the next morning. Not a huge problem but worth knowing.

3. I know the currently used term for blind is 視覚障害者 (visually impaired person) but it’s also longer and I wanted to insist on the ones who don’t see anything at all.