Connect AI Agents to Phone Calls with Sinch Voice API v2

Connecting an AI agent to a live phone call means bridging the phone network and the agent. That can involve SIP integration or a raw media path over a WebSocket, plus speech-to-text, text-to-speech, and managing conversational turns between the caller and the agent. That is a lot of telephony and media infrastructure to build and operate around an AI agent.

Sinch Voice API v2 takes that work off you. Its Voice Relay destination connects a live call to your WebSocket endpoint, Sinch runs speech-to-text and text-to-speech, and your application exchanges plain text. Voice API v2 also adds Voice Streams, a raw bidirectional audio path for the cases where you do want the media yourself.

It’s in public preview, so features, limits, documentation, and behavior can still change and no SLA applies, but it’s open for testing, evaluation, and early production traffic. I ran the whole path end to end against the tutorial example: a LangChain agent answering a real phone call, interruptions included.

An AI agent usually starts as a text application. It accepts a message, calls a model, and returns a text response. Voice Relay moves the audio boundary into the Sinch platform, so you keep the agent, its tools, prompts, and state, and you work with text rather than raw audio. Interruptions come with it, so callers can barge in while a response is playing.

That makes Voice Relay a practical entry point for an AI receptionist, product demo bot, and internal helpdesk.

Voice API v2 organizes an interaction around three resources:

• A session groups related calls and connections and remains active until its associated calls end.
• A call represents one participant’s connection, such as a phone, SIP, or streaming leg.
• A bridge connects calls within a session when multiple participants need to communicate.

SVAML, the Sinch Voice API Markup Language, describes what happens during the interaction. A dial command can create a call leg. Nested events can define what happens when the call is answered, is busy, times out, or fails. A webhook can take over when the application needs to make a dynamic decision.

This model matters for AI agents because the conversation and the call are related, but they aren’t the same thing. The AI agent manages the conversation. SVAML and the Voice API manage the call flow. That separation lets you add a human transfer, another call leg, or a different media path without putting all of that logic inside the model prompt.

• A Sinch account with Voice API v2 access, at the Sinch Build Dashboard
• A Sinch project ID, access key ID, and access key secret
• An activated Sinch virtual number
• A Voice API v2 service ID, from Voice > Programmable Voice > Services in the dashboard
• Python 3.10 or newer
• An API key for OpenAI, Anthropic Claude, or Google Gemini
• ngrok with a registered authtoken, or another way to expose a local server over wss://

The walkthrough uses the sinch-voice-tutorials repository. Its 4.1-voice-relay tutorial is a Python WebSocket server backed by LangChain, which reads its configuration from environment variables and supports OpenAI, Anthropic Claude, and Google Gemini.

Clone the tutorials repository and move into the Voice Relay example:

Create an environment and install the example’s dependencies:

The commands here are for macOS and Linux. On Windows, use python in place of python3 and activate with .venv\Scripts\activate.

Copy the example configuration:

Set a provider and its API key in .env:

The repository’s .env.example uses openai as its default provider. The example also supports claude for Anthropic Claude and gemini for Google Gemini, and requirements.txt installs the packages for all three, so switching provider means changing PROVIDER and API_KEY. You can also set MODEL, TEMPERATURE, MAX_TOKENS, and PORT, whose defaults show up in the startup log below, plus GREETING to change the Hello! the agent opens with.

Start the local server:

The server reports the system prompt it loaded, then its provider and model settings, then the local address it is listening on:

Check that output before you make a call. A missing provider key or a broken dependency shows up here rather than halfway through a conversation.

The agent’s behavior comes from system_prompt.md. Edit that file to change the persona, domain, or instructions, then restart server.py because the prompt is loaded at startup.

Sinch connects inward to your relay server, so it needs a public address. Leave server.py running in the first terminal and open a second one. If you don’t already have ngrok, install it and register an authtoken from the Your Authtoken page in the ngrok dashboard. The free tier covers this walkthrough, and ngrok refuses to start a tunnel until a token is configured:

ngrok takes over the terminal and prints a status table. The line that matters is the forwarding address:

Take that address, change only the scheme to wss://, and use it as the Voice Relay endpoint:

That wss:// value goes in the Sinch service configuration, not in the relay server, which stays on local port 8765. Leave both server.py and ngrok running from here on.

The minimal Voice Relay destination looks like this:

                                

                                    {
  "type": "VOICE_RELAY",
  "voiceRelay": {
    "endpoint": "wss://YOUR_NGROK_ID.ngrok-free.dev",
    "ttsVoice": "Tiffany",
    "sttLanguage": "en-US"
  }
}
                                
                            

Your service needs the number before any of this matters. In the dashboard, open the service, go to Voice channels, and choose Configure on the Phone row, then Add numbers and pick your virtual number. A number belongs to one service at a time, so if it already sits with another service the dashboard asks you to confirm the reassignment, and incoming calls to that number follow the new service from then on.

With the number in place, route the call by giving the service a static call behavior or a webhook that returns SVAML. The static configuration does five things:

1. Answer the incoming call.
2. Add the inbound leg to main-bridge.
3. Create a call leg with a VOICE_RELAY destination.
4. Add the relay leg to the same main-bridge when it answers.
5. Hang up the relay leg when the inbound leg ends.

server.py and ngrok are both holding their terminals now, so open a third one for this step. Fill in the five values at the top of the request and run it. If you’d rather use the dashboard, paste the inner SVAML body from static.txt into its predefined call behavior editor instead.

                                

                                    export PROJECT_ID=YOUR_PROJECT_ID
export KEY_ID=YOUR_ACCESS_KEY_ID
export SERVICE_ID=YOUR_SERVICE_ID
export VOICE_RELAY_ENDPOINT=wss://YOUR_NGROK_ID.ngrok-free.dev
 
# prompts without echoing, so the secret stays out of shell history
printf 'Access key secret: '
read -rs KEY_SECRET
echo
export KEY_SECRET
 
curl -X PATCH \
  -u "$KEY_ID:$KEY_SECRET" \
  "https://voice.api.sinch.com/v2/projects/$PROJECT_ID/services/$SERVICE_ID" \
  -H "Content-Type: application/json" \
  -d @- <<JSON
{
  "callBehavior": {
    "type": "STATIC",
    "static": {
      "callName": "caller",
      "commands": [
        { "command": "answer" },
        { "command": "bridgeCall", "bridgeName": "main-bridge" },
        {
          "command": "dial",
          "callName": "voice_relay_call",
          "to": {
            "type": "VOICE_RELAY",
            "voiceRelay": {
              "endpoint": "$VOICE_RELAY_ENDPOINT",
              "ttsVoice": "Tiffany",
              "sttLanguage": "en-US"
            }
          },
          "events": {
            "onAnswer": [
              { "command": "bridgeCall", "bridgeName": "main-bridge" }
            ]
          }
        }
      ],
      "events": {
        "onHangup": [
          { "command": "hangup", "callName": "voice_relay_call" }
        ]
      }
    }
  }
}
JSON

                                
                            

A successful PATCH returns the updated service, so you can read your endpoint back out of the response and confirm it took:

                                

                                    {
  "serviceId": "YOUR_SERVICE_ID",
  "projectId": "YOUR_PROJECT_ID",
  "name": "voice-relay-test",
  "isDefault": true,
  "callBehavior": {
    "type": "STATIC",
    "static": {
      "callName": "caller",
      "commands": [
        { "command": "answer" },
        { "command": "bridgeCall", "bridgeName": "main-bridge" },
        {
          "command": "dial",
          "to": {
            "type": "VOICE_RELAY",
            "voiceRelay": {
              "endpoint": "wss://YOUR_NGROK_ID.ngrok-free.dev",
              "ttsVoice": "Tiffany",
              "sttLanguage": "en-US"
            }
          },
          "events": {
            "onAnswer": [
              { "command": "bridgeCall", "bridgeName": "main-bridge" }
            ]
          },
          "callName": "voice_relay_call"
        }
      ],
      "events": {
        "onHangup": [
          { "command": "hangup", "callName": "voice_relay_call" }
        ]
      }
    }
  }
}
                                
                            

The inner SVAML body is also available in static.txt.

Call the Sinch number associated with the service. You should hear Hello!, or the greeting you set with GREETING. Then speak a question and listen for the agent’s response.

The server terminal logs both directions of the WebSocket, and that log is the most useful thing on screen during a first call. Mine looked like this, trimmed to one exchange:

Two things in that log are worth knowing before your first call. The connect message carries interruptionsEnabled: true, which is where you can confirm the interruption default rather than taking it from the configuration reference. And the connection closes with code 1006 when the caller hangs up, which reads as an abnormal closure but is what a normal hangup looks like here.

An interrupt with reason=speech-detected appears on every turn, because Sinch sends one whenever it hears the caller. Its position is what tells you something: after textPlaybackStop it just marks the start of the caller’s turn, while between textPlaybackStart and textPlaybackStop it is a barge-in. When I talked over a response, playback stopped and the next transcript was what I had said over the top:

That’s a voice interface in front of a text-based AI agent, without handling a single audio frame yourself.

Sinch and the WebSocket server exchange a small JSON protocol, one command per message, and server.py handles it for you. Transcript corrections are the part worth understanding before you build on the example.

Sinch sends an early transcript, then may send a corrected one for the same utterance with isCorrection: true, and the corrected text includes the earlier text rather than replacing only the changed part. It happened once in three calls: Thank you. was followed by Thank you.\nOh, that's nice., and both went to the model, so the agent answered the same utterance twice. Decide whether you wait for a correction, cancel the in-flight request, or ignore corrections entirely.

The application boundary looks like this:

Voice Streams suits a custom speech pipeline, a specialized audio processor, or a provider whose protocol requires direct audio access. It also means you own more of the latency and failure behavior.

Voice interactions make latency visible. A slow model response creates silence for the caller, so measure the time from the incoming speech event to the first response and to the completed response. Use a model and prompt that fit a phone conversation rather than optimizing only for maximum answer quality.

The repository example waits for the full model response before sending anything back, so the caller hears nothing until the model has finished generating. That is the first place to look if the pauses feel long.

The example also keeps conversation history in memory for each WebSocket connection. That’s fine for a first test, but it isn’t a durable conversation store. Decide what state belongs to a call, what belongs to a customer, and what has to survive a reconnect.

Interruptions need explicit handling. If the caller speaks while a response is playing, you can receive an interruption event while an LLM request is still running. Cancel the request where you can, or tag each response with a turn ID and discard stale output before sending it back to Sinch. The same turn ID handles corrected transcripts, which arrive as a second text message for an utterance you have already sent to the model.

The local example also needs stronger failure handling before production use:

• Send a short fallback response when the model provider fails.
• Persist conversation state when the call needs to survive a process restart.
• Keep the WebSocket endpoint available for the full call lifetime.
• Log call, session, connection, and model timing identifiers without logging sensitive conversation content unnecessarily.
• Use OAuth 2.0 client credentials for production API access. Basic authentication is useful for initial testing.

Check the provider configuration before you make a call. A bad or missing LLM key shows up as a silent call rather than an error, because the model isn’t called until the first turn, so confirm that PROVIDER and API_KEY match each other in .env.

The free ngrok URL changes when the tunnel restarts. Update the endpoint in the Voice API v2 service configuration whenever that happens, or Sinch can’t reach the current server. server.py doesn’t need restarting for this.

The preview terms allow testing, evaluation, early commercial use, and live traffic, and state that the platform is designed to support production-level volumes. The service is provided as is and as available, usage limits may apply, and no SLA applies. Sinch may add functionality, and breaking changes cannot be ruled out before general availability.

Voice Relay is one path into Voice API v2. The same platform model also supports:

• Outbound voice alerts with text-to-speech
• Answering machine detection
• Batch calling with call pacing
• Call recording and transcription
• Number masking
• SIP connections
• Raw audio streaming
• Live call control through SVAML and webhooks

The Sinch Voice API v2 tutorials repository has working examples for these paths. Start with the 4.1-voice-relay tutorial for a text-based agent, then move to 4.2-stream-audio when you need direct access to the audio stream.

Voice API v2 gives an existing text agent a route into a live voice interaction without turning that agent into an audio-processing system. Clone 4.1-voice-relay, point system_prompt.md at your own agent’s prompt, and call the number.

• Sinch Voice API v2 overview
• Voice API v2 getting started
• Voice API v2 API reference
• Text-to-Speech voices reference
• Voice API v2 tutorials repository

Author: Gunnar Grosch Gunnar Grosch is a Principal Developer Advocate at Sinch and an engineer with over 25 years of experience building software, scaling developer communities, and making complex technology accessible. He shares code, insights, and lessons through conference talks, blog posts, and open source.