site stats

ChatGPT can now see, hear, and speak

ChatGPT can now see, hear, and speak

We are beginning to roll out new voice and image capabilities in ChatGPT. They offer a new, more intuitive type of interface by allowing you to have a voice conversation or show ChatGPT what you’re talking about.

ChatGPT Can Now See Hear And Speak
September 25, 2023

Authors

We are beginning to roll out new voice and image capabilities in ChatGPT. They offer a new, more intuitive type of interface by allowing you to have a voice conversation or show ChatGPT what you’re talking about.

Voice and image give you more ways to use ChatGPT in your life. Snap a picture of a landmark while traveling and have a live conversation about what’s interesting about it. When you’re home, snap pictures of your fridge and pantry to figure out what’s for dinner (and ask follow up questions for a step by step recipe). After dinner, help your child with a math problem by taking a photo, circling the problem set, and having it share hints with both of you.

We’re rolling out voice and images in ChatGPT to Plus and Enterprise users over the next two weeks. Voice is coming on iOS and Android (opt-in in your settings) and images will be available on all platforms.

Speak with ChatGPT and have it talk back

You can now use voice to engage in a back-and-forth conversation with your assistant. Speak with it on the go, request a bedtime story for your family, or settle a dinner table debate.

Rectangle Speak with ChatGPT and have it talk back

Use voice to engage in a back-and-forth conversation with your assistant.

To get started with voice, head to Settings → New Features on the mobile app and opt into voice conversations. Then, tap the headphone button located in the top-right corner of the home screen and choose your preferred voice out of five different voices.

The new voice capability is powered by a new text-to-speech model, capable of generating human-like audio from just text and a few seconds of sample speech. We collaborated with professional voice actors to create each of the voices. We also use Whisper, our open-source speech recognition system, to transcribe your spoken words into text.

Listen to voice samples

Once in a tranquil woodland, there was a fluffy mama cat named Lila. One sunny day, she cuddled with her playful kitten, Milo, under the shade of an old oak tree.

“Milo,” Lila began, her voice soft and gentle, “you’re going to have a new playmate soon.”

Milo’s ears perked up, curious. “A new playmate?”

Lila purred, “Yes, a baby sister.”

Milo’s eyes widened with excitement. “A sister? Will she chase tails like I do?”

Lila chuckled. “Oh, she’ll have her own quirks. You’ll teach her, won’t you?”

Milo nodded eagerly, already dreaming of the adventures they’d share.

Chat about images

You can now show ChatGPT one or more images. Troubleshoot why your grill won’t start, explore the contents of your fridge to plan a meal, or analyze a complex graph for work-related data. To focus on a specific part of the image, you can use the drawing tool in our mobile app.

Comments are closed.