MOSS-Transcribe-Diarize

1. Project Overview

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model. The core goal is Speaker-Attributed, Time-Stamped Transcription, or SATS for short, that is, "transcription with speaker attribution and timestamp". It receives raw audio or video files and directly generates compact, structured transcribe text:

[start_time][Sxx]transcribed speech[end_time]

For example:

[0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready for evaluation[13.81]

This means that it is not the traditional "ASR model first to text, then a diarization model, and finally by post-processing alignment" splicing scheme, but through a multi-modal model joint to complete transwriting, speaker separation and timestamp generation.

2. What does it mostly do?

2.1 long audio transcription

It can convert long audio into text for meetings, lectures, podcasts, interviews, videos, calls, etc. Both the official paper abstract and the README emphasize that it can process up to about 90 minutes of input and use 128K context to carry long audio encoded sequences.

Pre-sales value: customers do not need to cut long meetings into many small segments and then splice them manually, which is suitable for directly entering the process of "retrievable, summarizable, quality inspection and discipline generation" from "recording files.

2.2 speaker separation

The model directly outputs anonymous speaker labels such as '[S01]' and '[S02]. It is not to identify the real name, but to try to keep the same speaker's label consistent within the same audio.

Pre-sales value: meeting minutes, interview arrangement, customer service quality inspection all need to know "who is talking", otherwise only get a whole paragraph of text, follow-up summary and responsibility tracking will become weak.

2.3 Timestamp Callout

Each segment has a start and end timestamp, which makes it easy to jump back to the original audio/video and to generate subtitles.

Pre-sales value: customers can directly locate the original audio and video evidence from summary, to-do and quality inspection issues, which is suitable for meeting resumption, training review, legal leaving marks and customer service sampling.

2.4 Acoustic Event Perception

The official introduction mentions that acoustic event annotations, I .e. acoustic event annotation, can be selectively output. This is more suitable as a "potential enhancement capability". In actual PoC, customer audio needs to be used to verify whether the desired event label can be stably output.

Suitable for exploration:

-Meeting/class events such as applause, laughter, silence, noise, etc.

-Abnormal pauses or ambient noise in the customer service scene.

-Non-speech event assisted annotation in video content.

2.5 hot word tips

It supports custom prompt and hotwords. The official suggestion is to add "hot word prompt: hot word 1, hot word 2, hot word 3" after the default transcription prompt ".

This is critical for industry customers, as the most common ASR mistakes are:

-Name of person

-Company Name

-Product name

-Proprietary Terms

-Place names

-Abbreviations

-Business system name

2.6 Subtitle Web App and Batch Processing

The GitHub package provides a local subtitle workflow:

-Upload audio/video.

-Review the parsed subtitle fragment.

-Export JSON/SRT/ASS.

-If FFmpeg/ffprobe is installed, MP4 generation can be suppressed.

-Web UI supports Simplified Chinese and English.

This makes it not just a model, but also a tool interface that can be demonstrated to the business side.

3. Core Competence List

4. Architecture and technical roadmap

The official model architecture is as follows:

Main components:

The inference pipeline given by the SGLang Omni cookbook is as follows:

It can be understood as three stages:

1. Encoder: Audio waveform to log-mel spectrogram, and then through Whisper encoder to obtain continuous audio features.
2. LLM Prefill: Audio features are projected into the LLM embedding space, replacing the audio placeholders in the prompt to build a KV cache.
3. AR Decode: Qwen3 style decoder self-regression generates transcribed text with speaker and timestamp.

SGLang Omni optimization focuses on CUDA Graph, chunked prefill, continuous batching, KV cache management, async decode, encoder cache, etc. For long audio, autoregressive decoding is the main time consuming point.

5. Applicable Scenario

5.1 Intelligent Meeting Minutes

This is the most direct scene. The client uploads the meeting recording, the model outputs a verbatim draft with a timestamp and speaker, and then the LLM does the summary, issues, to-do, risk, and decision items.

Value points:

-From "Recordings not retrievable" to "Structured Text Assets".

-Each sentence can be traced back to the original audio time point.

-Speaker tags make the summary closer to the real meeting process.

-Can be integrated with enterprise knowledge base, OA, flying book/enterprise micro/DingTalk conference system.

5.2 Call Center Quality Inspection

Customer service recording is generally a conversation between two or more people, not only to identify the content, but also to distinguish between seats and customers. MOSS-Transcribe-Diarize native output speaker-aware transcript, suitable for entering quality inspection rules, sentiment analysis, sensitive word detection, sales speech analysis.

Value points:

-Reduce the cost of manual sampling.

-Automatically locate offending speech and critical fragments.

-Linkage with subsequent quality inspection models, summary models, and CRM work orders.

5.3 Interviews and Survey Arrangement

User interviews, expert interviews, and market research recordings are usually very long, and many people speak alternately. Traditional ASR only gives words, and post-finishing is still very tired. Verbatim drafts with a speaker and time stamp are more suitable for researchers to do coding, excerpting and attribution.

5.4 Classroom/Lecture/Training Content Precipitation

Course recording, live playback and offline training videos can be converted into subtitles and text to regenerate course abstracts, knowledge points, question and answer questions and review materials.

Value points:

-Improve the reuse rate of training content.

-Support video subtitle generation.

-Easy retrieval of learning materials.

5.5 podcast and video caption production

The project comes with subtitle Web App and batch command, supports export SRT/ASS/JSON, and can use FFmpeg to suppress video. It's friendly to the media operations, the course team, the short video team.

5.6 compliance recording and evidence tracing

In judicial, financial, insurance, government hotline and other scenarios, time stamp and original audio positioning is very important. The model output is not final evidence, but can be used as an entry point for retrieval and manual review.

6. Not quite the scene

7. Deployment Resource Estimates

Officials did not directly write "minimum memory", but disclosed the model size, weight size and single H100 benchmark. The following is a pre-sales estimate based on official information and engineering experience. Formal projects need to be tested with real customer audio.

The official SGLang Omni benchmark is given on the single H100 80GB:

RTF less than 1 means faster than real time. For example, RTF 0.1237, theoretically about 1 hour of audio can be processed in 7-8 minutes, but this is under the condition of H100 80GB, and the official benchmark shall prevail.

Pre-sales advice:

-Demo only: Single 24GB GPU verified first.

-Customer meeting minutes PoC:24GB starting, 48GB preferred.

-Call center batch processing: 48GB start, estimated by concurrent and daily recording hours.

-90 minutes long sessions with multiple concurrency: Evaluate directly by 80GB card.

-When deploying 24GB/48GB cards, reduce the max-running-requests from 16 in the official example to 1-4.

How to use #8.

8.1 Environment Preparation

The official GitHub README uses Python 3.12 and Transformers 5.x:

git clone https://github.com/OpenMOSS/MOSS-Transcribe-Diarize.git
cd MOSS-Transcribe-Diarize
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e ".[torch-runtime]" --torch-backend=auto

The main Python package requirement comes from 'pyproject.toml ':

8.2 Python calls

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

from moss_transcribe_diarize import parse_transcript
from moss_transcribe_diarize.inference_utils import (
    build_transcription_messages,
    generate_transcription,
    resolve_device,
)

model_id = "OpenMOSS-Team/MOSS-Transcribe-Diarize"
audio_path = "audio.wav"

device = resolve_device("auto")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
).to(dtype=dtype).to(device).eval()

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
)

messages = build_transcription_messages(audio_path)
result = generate_transcription(
    model,
    processor,
    messages,
    max_new_tokens=2048,
    do_sample=False,
    device=device,
    dtype=dtype,
)

print(result["text"])

for segment in parse_transcript(result["text"]):
    print(segment.start, segment.end, segment.speaker, segment.text)

Note: Hugging Face loading needs 'trust_remote_code = True', and the enterprise must do code audit before production.

8.3 SGLang Omni Servitization

Officially recommended SGLang Omni:

hf download OpenMOSS-Team/MOSS-Transcribe-Diarize

sgl-omni serve \
  --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --port 8000 \
  --max-running-requests 16 \
  --cuda-graph-max-bs 16 \
  --mem-fraction-static 0.80

Call:

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F file=@audio.wav \
  -F response_format=verbose_json

Long Audio Adjustable Large:

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F file=@audio.wav \
  -F response_format=verbose_json \
  -F max_new_tokens=65536

8.4 vLLM Servitization

The CUDA 12 environment may consider vLLM. The official tip requires a pinned vLLM nightly that contains the model registration.

vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code

Call:

curl http://localhost:8000/v1/audio/transcriptions \
  -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
  -F file=@"audio.wav" \
  -F response_format="json" \
  -F temperature="0"

8.5 Subtitles Web App

mtd-subtitle-web \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --host 127.0.0.1 \
  --port 7860

After opening 'http:// 127.0.0.1:7860 ', you can upload audio/video, review segments, and export JSON/SRT/ASS. MP4 can also be suppressed if FFmpeg/ffprobe is installed.

Batch:

mtd-subtitle /path/to/input.mp4 \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --out-dir runs/example \
  --render

9. What can I say before sales

9.1 for Meeting Minutes Customers

"It's not simply turning the audio into a whole paragraph of text, but directly outputting who said what at what time. Following the big model summary, you can generate meeting minutes, to-do items, controversial points and key decisions, and you can jump back to the original recording."

9.2 for Call Center Customers

"Call center quality inspection is most afraid of only the full text, no role. MOSS-Transcribe-Diarize can output speaker tags and time stamps, the follow-up can distinguish between agents and customers, do speech compliance, complaint identification, sales opportunity analysis."

9.3 for audio and video content customers

"It provides a local captioning workflow, supports exporting SRT/ASS/JSON, and can also work with FFmpeg to suppress video. For courses, podcasts, interviews, and live playback, it can convert content assets into retrievable, distributable, and reusable text and captions."

9.4 for information security/government and enterprise customers

"The model is Apache-2.0 open source, which can be deployed privately, and audio can not go out of the intranet. For meeting recordings, hotline recordings, law enforcement records, and training videos with privacy and compliance requirements, this is a route that is easier to pass security reviews than pure SaaS transcriptions."

10. Frequently Asked Customer Questions

11. PoC Recommendations

11.1 PoC Data Preparation

Customers are advised to provide 3 types of real data:

11.2 acceptance index

11.3 Demo script

1. Upload a 10-20 minute multi-person meeting recording.
2. Output verbatim with '[S01]', '[S02]', timestamp.
3. Presentation of 'verbose_json segment' results.
4. Export SRT subtitles.
5. Use LLM to generate meeting minutes, to-do, risk points for the transfer results.
6. Click the timestamp to listen back to the original audio clip.
7. Add hot words in the customer industry and compare the term recognition effect.

Comparison of 11.4 and Competitions/Alternatives

12. Risks and Considerations

12.1 New Project Maturity

Project 2026-07 is open source and is growing fast, but production stability still requires customer scenario stress testing. Don't just look at the official benchmark and promise direct production.

12.2 'trust_remote_code = True'

Hugging Face loading requires remote custom code execution. The enterprise environment must:

-Fixed model version and commit sha.

-Auditing documents such.

-Do mirror curing.

-Prohibit the production environment from dynamically pulling unaudited code.

12.3 long audio memory and delay

Long audio results in a long encoded sequence and output. The actual consumption is affected by the audio length, the number of people speaking, the output text length, the number of max_new_tokens, and the number of concurrency. Pre-sales quotes are estimated by "audio hours per day" and "need to return in real time.

12.4 speaker label is not identity authentication

'[S01]' is just an anonymous label generated by the model. It does not prove true identity, nor does it replace voiceprint recognition or real-name participation information.

12.5 transcription results still need manual review

In high-risk scenarios such as finance, justice, medical care, and government affairs, automatic transcription can only be used as an aid. The manual review process shall be retained for official texts, evidentiary materials and compliance reports.

12.6 Pro Version Differences

README mentioned that MOSS-Transcribe-Diarize Pro is more powerful and can be used through online Playground. Before the sale to distinguish:

-Open source 0.9B: suitable for privatization, self-deployment, PoC.

-Pro: may be more effective, but involves online services, commercial terms, data flow and fees.

13. My Pre-Sales Judgment

MOSS-Transcribe-Diarize is a project that is closer to the business floor than the early MOSS dialogue model. It solves a very clear high-frequency problem: large amounts of audio/video content cannot be structured, cannot be retrieved, and cannot be quickly reproduced.

I think it is best suited to cut into three categories of scenarios:

1. Meeting Minutes/Training Content Precipitation : Audio to verbatim draft, followed by large model summary and knowledge base.
2. Call Center Quality Inspection: Speaker-differentiated timestamp business rules/LLM quality inspection.
3. Video subtitles and content production: Automatically generate subtitles, manuscripts, abstracts, and slice materials.

If customers are concerned about data security and want to deploy it privately, its Apache-2.0 license and open source weight are obvious advantages. If customers are looking for extreme real-time, extremely high concurrency, or legal-level accuracy, they need to be evaluated carefully and suggest adding manual review, proper noun hot words, voiceprint/identity mapping, and post-processing rules.

My suggestion: PoC can be made before sales with the combination of "30 minutes of meeting recording, 100 minutes of customer service recording, 5 course videos. As long as the transcription, speaker, timestamp and subtitle export can meet the customer's expectations, this project can easily be linked to several mature business topics such as meeting minutes, quality inspection, knowledge base and capitalization of audio and video content.

14. REFERENCE

-Hugging Face model card:OpenMOSS-Team/MOSS-Transcribe-Diarize

-GitHub repository:OpenMOSS/MOSS-Transcribe-Diarize

-Chinese README:README_zh.md

-English README:README.md

-SGLang Omni Cookbook:moss_transcribe_diarize.md

-Technical Report:MOSS Transcribe Diarize Technical Report

-Online Playground:MOSS Transcribe Diarize Pro

-OpenMOSS official website:open-moss.com

-MOSI.AI:mosi.cn