Real-Time Speech-to-Text and Diarization with WhisperLiveKit: A Dev's Guide


Real-Time Speech-to-Text and Diarization with WhisperLiveKit: A Dev's Guide

QuentinFuxa/WhisperLiveKit

2025-08-27

From a software engineering perspective, WhisperLiveKit is incredibly useful because it handles the complex, resource-intensive parts of real-time audio processing.

Real-time Transcription
It lets you add live transcription features to your applications without needing to build a complex system from scratch. This is great for applications like live captioning for virtual meetings, real-time customer support chat, or accessibility tools.

Speaker Diarization
The ability to tell who is speaking is a huge plus. Imagine building an application that needs to create meeting transcripts with speaker labels (e.g., "John
...", "Jane
..."). WhisperLiveKit makes this a breeze. It can differentiate between multiple speakers in a single audio stream, which is a difficult problem to solve manually.

Local Processing
Because it runs locally, you don't have to worry about sending sensitive audio data to a third-party API. This is critical for applications in healthcare, finance, or any field with strict data privacy regulations. It also eliminates network latency, resulting in faster and more reliable performance.

FastAPI Integration
The package comes with a built-in FastAPI server and a web interface. This means you can get a fully functional transcription server up and running with minimal effort. It's an excellent way to expose a real-time speech processing service as a web API for other applications to consume.

Getting WhisperLiveKit up and running is pretty straightforward. You'll need Python and pip installed.

First, you need to install the package using pip. It's a good practice to use a virtual environment to avoid conflicts with other Python projects.

# Create and activate a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install the package
pip install git+https://github.com/QuentinFuxa/WhisperLiveKit.git

Once installed, you can start the FastAPI server with a single command. The server will download the necessary models (the Whisper model for STT and pyannote for diarization) the first time you run it.

python -m whisper_live_kit

By default, the server runs on http://127.0.0.1:9000. You can open this URL in your browser to access the web interface.

Now, let's look at how you might use this server from another Python application. The package provides a simple client class to handle the communication. Here’s a quick example of a client that connects to the server and listens for real-time transcription.

import asyncio
from whisper_live_kit.client import WhisperLiveKitClient

async def main():
    # Instantiate the client. Replace with your server's IP if it's not local.
    client = WhisperLiveKitClient("ws://127.0.0.1:9000/ws")

    # Define a callback function to handle incoming transcriptions
    async def on_transcription(data):
        print(f"Transcription received: {data}")
        # The data dictionary will contain keys like 'text', 'start', 'end', and 'speaker'
        # Example output: {'text': 'Hello world.', 'start': 1.2, 'end': 2.5, 'speaker': 'SPEAKER_00'}
        # You can process this data as needed for your application

    # Set the callback function
    client.set_transcription_callback(on_transcription)

    # Start the client. This will run in the background.
    await client.start()

    print("Client started. Now waiting for transcription data...")

    try:
        # Keep the event loop running to listen for data
        while True:
            await asyncio.sleep(1)
    except KeyboardInterrupt:
        print("Stopping client...")
        await client.stop()

if __name__ == "__main__":
    asyncio.run(main())

QuentinFuxa/WhisperLiveKit