Premium

Voice AI Support Backend

Twilio WebSockets + OpenAI Realtime API integration. Native VAD & interruption.

Architecture Overview

Transform your company's phone support. This server acts as an ultra-fast bridge between PSTN calls (via Twilio) and OpenAI's new Realtime Voice API. It supports ultra-low latency, Voice Activity Detection (VAD) to stop the robot from speaking if the human interrupts, and continuous audio streaming using Fastify WebSockets.

Folder Structure

├── src/
│   ├── server.ts
│   ├── websockets/
│   │   ├── twilioHandler.ts
│   │   └── openaiBridge.ts
│   └── utils/
│       └── audioCodec.ts
├── .env.example
├── package.json
└── tsconfig.json

Code Preview

typescript

import Fastify from "fastify";
import fastifyWebsocket from "@fastify/websocket";

const app = Fastify();
app.register(fastifyWebsocket);

app.get('/media-stream', { websocket: true }, (connection, req) => {
  const twilioWs = connection.socket;
  const openAiWs = new WebSocket('wss://api.openai.com/v1/realtime', {
    headers: { Authorization: `Bearer ${process.env.OPENAI_KEY}` }
  });

  twilioWs.on('message', (msg) => {
    const data = JSON.parse(msg.toString());
    if (data.event === 'media') {
      // Stream G711 audio directly to OpenAI
      openAiWs.send(JSON.stringify({
        type: 'input_audio_buffer.append',
        audio: data.media.payload
      }));
    }
  });
});

Technical FAQ

Yes, the OpenAI Realtime API handles multiple languages automatically.

Yes, this architecture is specifically designed to handle Twilio Media Streams.

Lifetime Access

$149

Secured by Stripe

256-BIT ENCRYPTION SOC 2 COMPLIANT 99.9% UPTIME

Included in the box

Bidirectional G.711 Audio Streaming
Human Interruption Detection
Latency < 300ms
Integrated Twilio TwiML Webhook
Scalable Architecture

Tech Stack

FastifyWebSocketsTwilio Media StreamsOpenAI Realtime APINode.js