Thinking with Sound
Instead of treating a recording as a fixed input to describe, let the model actively work with the signal while it reasons, cleaning up noise, separating overlapping sources, and aligning events, so it can hear better before it answers.
*These are learning tools, not prompts. They teach you to write your own, think of them as training wheels that guide you while you learn, then fall away once you can ride on your own.
Reasoning That Works the Signal
Most audio reasoning treats the recording as a static input. The model listens once, forms an impression, and reasons about that impression in words. When the audio is clean, this works. When it is noisy, crowded with overlapping voices, or poorly aligned, the model is stuck reasoning over a signal it could not fully make out in the first place.
Thinking with Sound treats the audio as something to actively work with, not just listen to. During reasoning, the model can perform numerical analysis and digital manipulation on the signal itself, applying acoustic operations to hear more clearly before drawing a conclusion. Rather than accepting a muddy recording as-is, it can clean it up mid-thought and then reconsider.
The operations it reaches for are the familiar tools of audio engineering: noise suppression to reduce background interference, source separation to pull apart overlapping sounds or voices, and temporal alignment to line up events in time. Crucially, this is training-free. It equips an existing audio-language model with acoustic tools it can invoke while reasoning, which is what makes it especially robust when the audio has been degraded or perturbed.
You cannot reliably reason about a voice you cannot hear over the noise. By letting the model denoise, separate, or realign the audio in the middle of its reasoning, Thinking with Sound removes the obstacle before it becomes a wrong answer. This is why it holds up under real-world acoustic perturbations that cause static listening to fail: the model does not just cope with a bad signal, it improves the signal and then listens again.
How Thinking with Sound Works
Four stages that fold signal manipulation into reasoning
Listen and Spot the Obstacle
The model takes in the audio and the question, and begins reasoning. Early on, it identifies what is standing in the way of a confident answer: background noise, overlapping sources, or events that are hard to place in time.
“I am asked what the announcement says, but there is heavy crowd noise over the speech, I cannot make out the words yet.”
Apply an Acoustic Tool
Mid-reasoning, the model manipulates the signal with the operation that fits the obstacle: suppress the noise, separate the sources, or align events in time. This is the step that sets the method apart, it changes the audio it is working from rather than only describing it.
“Applying noise suppression to reduce the crowd noise, then isolating the announcer’s voice with source separation.”
Re-Listen to the Cleaner Signal
With the processed audio in hand, the model listens again to the improved signal and continues its reasoning. Details that were buried, words, timing, a faint source, may now be audible enough to reason about reliably.
“On the cleaned, separated track, the announcement is now intelligible: it names a gate change.”
Answer With Restored Confidence
The model answers from what it could hear after improving the signal. Because it addressed the acoustic obstacle directly, the conclusion holds up even on degraded or perturbed audio that would defeat a model reasoning over the raw recording alone.
“The announcement states that the departure gate has changed, which was only recoverable after suppressing the crowd noise.”
See the Difference
Reasoning over raw audio versus improving the signal first
Static Audio Reasoning
Over this noisy recording of two people, what is the second speaker’s main concern?
It is hard to tell over the noise; the second speaker seems to be talking about the weather.
Thinking with Sound
“The voices overlap and there is background noise. I will separate the two speakers and suppress the noise, then focus on the second speaker’s isolated track. On that cleaner track, the second speaker repeatedly mentions a missed payment and a due date.”
The second speaker’s main concern is a missed payment and its upcoming due date, clear only after separating the voices and reducing the noise.
Thinking with Sound in Action
One acoustic tool per obstacle, applied mid-reasoning
“A voice memo recorded near a busy road. What instruction does the speaker give?”
Obstacle: steady traffic noise masks parts of the speech.
Tool applied: noise suppression to attenuate the broadband road noise while preserving the voice band.
Re-listen and answer: On the cleaned audio, the instruction is now clear, the speaker says to leave the package with the neighbor in apartment 4B.
“Two people talk at once in this clip. What does the quieter speaker ask for?”
Obstacle: a louder voice dominates and masks the quieter one.
Tool applied: source separation to split the mixture into two isolated voice tracks, then focus on the quieter track.
Re-listen and answer: On the separated quiet track, that speaker asks for a glass of water, which was inaudible in the original mixture.
“In this recording, did the alarm sound before or after the door closed?”
Obstacle: the two events overlap and their order is ambiguous by ear.
Tool applied: temporal alignment to place each event precisely on a shared timeline, separating their onsets.
Re-listen and answer: The aligned timeline shows the alarm onset preceding the door-close by a clear margin, so the alarm sounded first.
When to Use Thinking with Sound
Best when the signal itself is the obstacle
Perfect For
Real-world recordings with background interference, where suppressing the noise before reasoning recovers content that static listening misses.
Clips with multiple simultaneous voices or sounds, where separating the mixture lets the model reason about one source at a time.
Questions that hinge on when things happen, where aligning events on a shared timeline resolves ambiguity that is hard to judge by ear.
Settings where audio may be perturbed or corrupted and you need reasoning that improves the signal rather than failing on it.
Skip It When
When the recording is clear and single-source, there is no signal obstacle to remove, and plain audio reasoning answers just as well with less work.
The benefit depends on the model being able to invoke operations like separation and denoising. Without them, this reduces to ordinary listening.
For high-level questions that do not depend on hard-to-hear detail, manipulating the audio adds steps without changing the answer.
Use Cases
Where Thinking with Sound delivers the most value
Speech in Noisy Environments
Understand voice memos, field recordings, and calls captured in traffic, wind, or crowds by suppressing noise before reasoning about the words.
Multi-Speaker Meetings
Follow individual contributions in overlapping conversation by separating speakers, so each person’s point can be reasoned about on its own track.
Event Timeline Reconstruction
Determine the order and timing of sounds in a recording by aligning events, useful in incident review and forensic audio analysis.
Robust Audio Assistants
Build voice features that hold up in the messy acoustics of real life, where the ability to improve the signal is the difference between working and failing.
Audio Forensics and Restoration
Recover intelligible content from degraded archival or evidentiary recordings by cleaning and separating the signal as part of the analysis.
Accessible Listening Support
Help people parse difficult audio by isolating and clarifying the part that matters, then explaining what was heard on the improved signal.
Where Thinking with Sound Fits
From reasoning about audio to reasoning that reshapes it
Audio Chain-of-Thought let models reason about what they hear, but it still accepted the recording as given. Thinking with Sound takes the next step: it lets the model act on the signal, denoising, separating, and aligning it in the middle of reasoning, so a poor recording becomes a workable one before any conclusion is drawn. Because it adds acoustic tools to an existing model without retraining, it is especially valuable in the noisy, overlapping, imperfect audio of the real world, where the ability to improve the signal is exactly what static listening lacks.
Related Techniques
Explore techniques connected to Thinking with Sound
Improve the Signal, Then Answer
Apply Thinking with Sound to your own noisy or crowded audio tasks, or build structured audio prompts with our interactive tools.