wss://api.pascalai.co/v1/speech-to-text/realtime

Speech to text realtime

Stream live audio and receive the transcript on the same WebSocket.

wss://api.pascalai.co/v1/speech-to-text/realtime?model=sarvam-saaras-live

Send an API key as Authorization: Bearer. A browser sends the key as api_key on the query string.

Field	Type	Description
`model`Required	string	Catalogue id of the live model. nemotron-3.5-asr-streaming-0.6b or pascal-transcribe-live.
`api_key`	string	API key, sent on the query string.

Audio

Send binary frames of little-endian PCM16, mono. A file upload carries its rate in the header, and that job resamples it. A live frame is raw PCM and has no header. Set session.sample_rate to the rate of those bytes before the first audio frame. Omit it and the socket treats the audio as 16 kHz. The bytes have to already be that rate. If they are a different rate, the socket still accepts them and the transcript can come out degraded. A rate that model does not list is rejected.

Nemotron accepts 8 kHz and 16 kHz. The other live models accept the rates on their model page. The studio microphone always sends 16 kHz. An API caller converts first. A microphone that captures at 48 kHz has to be turned into a rate that model lists before it is sent. Send 8 kHz only when the audio really is 8 kHz.

Client messages

Wait for session.created. Send one session.update, then audio. session.update is accepted only before the first audio frame.

Field	Type	Description
`session.update`	object	JSON text frame. session holds the fields in the Session section. Send it once, before audio.
`binary frame`	bytes	Little-endian PCM16, mono, at session.sample_rate. Omit the rate and it is 16 kHz. The bytes have to already be that rate. A mismatch is still accepted and the transcript can come out degraded.
`session.commit`	object	JSON text frame. Commits the current turn when endpointing is manual.
`session.clear`	object	JSON text frame. Drops audio buffered since the last commit.
`session.cancel`	object	JSON text frame. Cancels the current turn.
`session.close`	object	JSON text frame. Ends the session and closes the socket.

Server events

Field	Type	Description
`session.created`	object	The socket is open. session.id, session.model, session.sample_rate, session.prepaid_seconds, and session.request_fee_credits. session.sample_rate is 16000 until session.update sets 8000 or 16000. Nemotron accepts both and also sends session.chunk_ms_applied.
`transcript.delta`	object	A partial transcript while the turn is still open. delta is the new text.
`transcript.completed`	object	The transcript for one turn. transcript is the text. A model may also send start, end, and speaker.
`error`	object	error.type, error.code, error.message, and error.param. The socket then closes.

Billing

A session pays the request fee once, then connected time at that model's rate, for up to 5 hours. The rate is on the model.

Session

These fields are the session object inside session.update. Each model accepts its own set. A field the model does not list is rejected. The tables below are that model’s fields. speaker_diarization is on Pascal Transcribe Live and Deepgram Nova-3 Streaming. Sarvam Saaras Live does not diarize.

`sarvam-saaras-live`

Field	Type	Description
`session.language` Default unknown	string	Same codes as the file model: en, hi, bn, gu, kn, ml, mr, pa, ta, te, or the -IN form. Odia is od or or. Both are the same. unknown detects. language_code is the same field.
`session.mode` Default transcribe	string	Applied to the final transcript. Partials stay a straight transcription.
`session.sample_rate` Default 16000	integer	8000 or 16000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.encoding` Default linear16	string	linear16, mulaw, or alaw. Omit it for linear16.

`nemotron-3.5-asr-streaming-0.6b`

Field	Type	Description
`session.language` Default auto	string	auto detects the language. An ISO 639-1 or BCP-47 code selects it, such as en or en-US.
`session.sample_rate` Default 16000	integer	8000 or 16000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.chunk_ms` Default 160	integer	The session runs at 160 ms. The first caption arrives after about 1.2 seconds of speech.
`session.endpointing` Default silence	string	silence or manual. silence commits after stop_history_eou_ms of quiet audio. manual commits the turn when the client sends session.commit.
`session.stop_history_eou_ms` Default 3200	integer	200 to 10000. Silence, in milliseconds, that ends a turn when endpointing is silence.
`session.speech_context`	string	Up to 500 characters. A phrase to boost, such as a name or a product term.
`session.speech_context_boost` Default 10	number	0 to 20. When speech_context is set and this field is omitted, the boost is 10.
`session.profanity_filter` Default false	boolean	Mask profanity in the transcript.

`pascal-transcribe-live`

Field	Type	Description
`session.sample_rate` Default 16000	integer	8000, 11025, 16000, 22050, 32000, 44100, or 48000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.language` Default auto	string	auto detects the language. An ISO 639-1 code selects it. ['af', 'am', 'ar', 'as', 'az', 'ba', 'be', 'bg', 'bn', 'bo', 'br', 'bs', 'ca', 'cs', 'cy', 'da', 'de', 'el', 'en', 'es', 'et', 'eu', 'fa', 'fi', 'fo', 'fr', 'gl', 'gu', 'ha', 'haw', 'he', 'hi', 'hr', 'ht', 'hu', 'hy', 'id', 'is', 'it', 'ja', 'jw', 'ka', 'kk', 'km', 'kn', 'ko', 'la', 'lb', 'ln', 'lo', 'lt', 'lv', 'mg', 'mi', 'mk', 'ml', 'mn', 'mr', 'ms', 'mt', 'my', 'ne', 'nl', 'nn', 'no', 'oc', 'pa', 'pl', 'ps', 'pt', 'ro', 'ru', 'sa', 'sd', 'si', 'sk', 'sl', 'sn', 'so', 'sq', 'sr', 'su', 'sv', 'sw', 'ta', 'te', 'tg', 'th', 'tk', 'tl', 'tr', 'tt', 'uk', 'ur', 'uz', 'vi', 'yi', 'yo', 'yue', 'zh']
`session.speaker_diarization` Default false	boolean	Label each turn with a speaker number. When true, connected time is billed at $0.25 per hour.
`session.custom_terms`	string or array	Names and jargon to bias the transcript. A comma-separated string or a list of strings. At most 1000 terms and 8000 characters.
`session.timestamp_granularity` Default none	string	none returns transcript only. segment also sends start and end, in seconds, on each transcript.completed.

`deepgram-nova-3-streaming`

Field	Type	Description
`session.language`	string	BCP-47 tag, or multi.
`session.diarize`	boolean	Speaker labels. speaker_diarization is the same switch. Adds $0.0020 per minute.
`session.interim_results`	boolean	Partial results. Deepgram’s Results events are forwarded.
`session.endpointing`	integer or false	Milliseconds of silence, or false.
`session.keyterm`	string or array	Keyterm prompting. custom_terms is the same list.
`session.sample_rate` Default 16000	integer	8000, 16000, 24000, 44100, or 48000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.encoding` Default linear16	string	linear16, mulaw, or alaw. Omit it for linear16.

`deepgram-flux-en`

Field	Type	Description
`session.eot_threshold` Default 0.7	number	End-of-turn confidence from 0.5 to 1.
`session.eager_eot_threshold`	number	Eager end-of-turn confidence from 0.3 to 0.9.
`session.eot_timeout_ms` Default 5000	integer	Force the end of a turn after this many milliseconds.
`session.keyterm`	string or array	Keyterm prompting. custom_terms is the same list.
`session.sample_rate` Default 16000	integer	8000, 16000, 24000, 44100, or 48000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.encoding` Default linear16	string	linear16, mulaw, or alaw. Omit it for linear16.

`deepgram-flux-multi`

Field	Type	Description
`session.language_hint`	string or array	en, es, fr, de, hi, ru, pt, ja, it, nl. language is the same field.
`session.eot_threshold`	number	End-of-turn confidence from 0.5 to 1.
`session.sample_rate` Default 16000	integer	8000, 16000, 24000, 44100, or 48000. Omit it and the rate is 16000 Hz. The bytes have to already be that rate. A live frame has no header, so a mismatch is still accepted and the transcript can come out degraded.
`session.encoding` Default linear16	string	linear16, mulaw, or alaw. Omit it for linear16.

Example

const socket = new WebSocket(
  'wss://api.pascalai.co/v1/speech-to-text/realtime?model=sarvam-saaras-live&api_key=' + key
);
socket.binaryType = 'arraybuffer';
socket.onopen = () => {
  socket.send(JSON.stringify({
    type: 'session.update',
    session: {
      language: 'auto',
      endpointing: 'manual',
    },
  }));
};
// binary Int16 PCM16 mono frames at session.sample_rate (default 16000)
// when endpointing is manual:
socket.send(JSON.stringify({ type: 'session.commit' }));
socket.send(JSON.stringify({ type: 'session.close' }));

Models

sarvam-saaras-liveLive Saaras. $0.005 / min. PCM16, μ-law, or A-law at 8000 or 16000 Hz.
nemotron-3.5-asr-streaming-0.6bLive speech-to-text. WebSocket only. $0.00333 / min plus the $0.05 request fee.
pascal-transcribe-liveLive speech-to-text. WebSocket only. $0.22 per audio hour, or $0.25 per audio hour with speaker diarization, plus the $0.05 request fee.
deepgram-nova-3-streamingLive Nova-3. $0.0048 per minute for one language, $0.0058 per minute when language is multi. Diarization adds $0.0020 per minute.
deepgram-flux-enLive English Flux. $0.0065 per minute. Turn detection is part of the model.
deepgram-flux-multiLive multilingual Flux. $0.0078 per minute.