Tips and tricks for reliable voice AI agents

Last episode we talked about the challenges teams face building production voice AI systems. This time, Aidan and Jack share the practical tips, tricks, and hacks they've learned from dozens of teams building voice agents in the real world.

From audio isolation tools to parallel transcription, from thinking noises to turn-taking tuning—these are the techniques that actually work.

---

What We Cover

Audio Quality & Transcription

• Audio isolation tools like Krisp and AI Acoustics that "solve challenges overnight"
• Using LLMs to filter out background noise from transcriptions
• The simple fix: prompting agents to ask users to repeat themselves
• Parallel transcription with fast/slow models running simultaneously
• Post-call processing with higher-quality models for summaries and verification

Speed vs. Accuracy Trade-offs

• Deliberate pauses to buy time for slower, more accurate transcription
• Language-specific model performance (Mistral for German, etc.)
• Why teams keep evaluating new models—and why it's so time-consuming
• WhatsApp voice memos as a transcription quality hack

Conversation Flow & UX

• Thinking noises: why silence breaks conversations
• The elevator mirror principle—distraction beats optimization
• Proactive agents that ask "can you hear me?" during silence
• Turn-taking: why being patient beats being eager
• Using IPA (International Phonetic Alphabet) for brand name pronunciation

Hybrid Approaches

• Text as a fallback when voice isn't working
• Boardy.ai as an example of mixing voice and text effectively

---

Timestamps

• 0:00 — Intro: tips & tricks from production teams
• 1:02 — Audio isolation tools (Krisp, AI Acoustics)
• 2:06 — LLM filtering for background noise
• 3:11 — Prompting agents to ask users to repeat themselves
• 4:43 — Parallel transcription with fast/slow models
• 7:30 — Post-call processing for better transcripts
• 10:06 — Speed vs accuracy trade-offs
• 11:51 — Language-specific model performance
• 15:31 — WhatsApp voice memo hack
• 17:17 — Text as a fallback, Boardy.ai example
• 18:55 — Thinking noises and conversation flow
• 21:07 — The elevator mirror story
• 22:46 — Proactive silence handling
• 23:57 — Turn-taking: patience over eagerness
• 25:40 — IPA for pronunciation + wrap-up

---

Resources

• Layercode
• Layercode Docs
• Follow Aidan on X
• Follow Jack on X

Mentioned in This Episode

• Krisp — AI-powered noise cancellation
• AI Acoustics — Audio isolation (powers Sennheiser/Bose)
• Boardy.ai — Voice + text hybrid onboarding example
• Deepgram Flux — Speech-to-text with embedded turn-taking

---

Have a topic you'd like us to cover? Reach out on X @uselayercode or email us at podcast@layercode.com