Introducing Realtime-Venus

AI that listens
while it speaks.

Realtime-Venus keeps listening while it responds, connects sound with visual context, and handles delegated tasks in the background.

Explore the demos Read the technical report

Venus Team, Ant GroupTsinghua University

GitHub Hugging Face ModelScope

Demos

See Realtime-Venus in action.

Three everyday moments: a follow-up question, a microwave beep, and a flight search.

Realtime-Venus-AudioRecorded audio demo

A new question. A new direction.

About this scene

Full-duplex conversation

Hear a road-trip conversation shift from planning the route to arranging daily breaks and meals.

What to notice

Around 15 s, you ask a follow-up while Realtime-Venus is still speaking. Its next response addresses breaks and meals.

How it works

Two loops.
One conversation.

One loop handles live perception and speech. The other runs delegated tasks and prepares replies, returning results to the same conversation.

Dual-loop architecture: the user exchanges audio, video, text and speech with the Realtime-Venus frontend. Delegated tasks pass through capture, dispatch and return in the asynchronous Harness capability loop. — Figure 3 The two loops share one conversation: delegated results return while live interaction continues.

9B · Audio + video

Realtime-Venus-Omni

Understand visual events, respond proactively, and recall relevant moments from long videos.

9B · Audio

Realtime-Venus-Audio

Understand sound and speech, respond by voice, and distinguish interruptions from brief acknowledgments.

Background execution

Realtime-Venus-Harness

Preserve each request’s context, run delegated tasks, and prepare replies for the conversation.

Inside the systemDelegation, training, and long-video memory

Capture the right context

Save the context available when a request arrives and link it to the conversation it came from.

Keep interaction running

Route each task to a multimodal model, a general model, or a registered skill while perception and conversation continue.

Return results to the conversation

The harness prepares a reply, and the frontend decides when to speak. Playback tracking confirms that the reply was delivered.

Realtime-Venus-Harness captures a frozen request snapshot, routes work to multimodal, general or skill capabilities, prepares a spoken reply, and returns it to the same session with playback tracking. — Figure 7 From context capture to task execution and reply delivery. The harness prepares the reply; the frontend controls when it is spoken.

Long-video memory

Keep useful context within reach

For longer audio–visual sessions, Realtime-Venus-Omni stores selected video frames and retrieves them based on relevance and novelty. This recalled context is combined with recent observations.

The memory module requires no additional training and operates separately from the fine-tuned dialogue policy.

Realtime-Venus-Omni architecture and audio-visual memory: construct a long-term archive, retrieve frames by relevance and novelty, and assemble retrieved evidence with recent context. — Figure 5 Model architecture and audio–visual memory.

Results

Understanding meets
interaction

Evaluations cover video and audio understanding, full-duplex conversation, and delegation. Scores shown here come from the September 9, 2026 manuscript; each benchmark uses its own evaluation protocol.

6 of 8

Video benchmarks where Realtime-Venus-Omni leads the online models evaluated in the report.

70.2%

Understand the scene

StreamingBench · Omni

78.0%

Understand sound and speech

MMAU · Audio

97%

Continue through backchannels

Full-Duplex-Bench v1.5 · Audio
C_RESUME continuation rate

Omni and Audio refer to Realtime-Venus-Omni and Realtime-Venus-Audio.

Benchmark results and evaluation details

Video and audio understanding

Selected benchmark scores from the technical report
Benchmark	Model	Score
StreamingBench	Omni	70.2%
OVO-Bench	Omni	64.7%
Daily-Omni	Omni	81.3%
MMAU	Audio	78.0%
MMAU-Pro	Audio	63.2%
Llama Questions	Audio	83.8%
Speech CMMLU	Audio	67.8%

See the technical report for datasets, prompts, and the full model comparisons.

Knowing when to keep speaking

Full-Duplex-Bench v1.5 measures whether Realtime-Venus-Audio continues speaking when a cue does not call for interruption (C_RESUME).

User backchannels97%

Speech directed at others88%

Background speech86%

75%

Response rate to user interruptions
C_RESPOND · measured separately

Original benchmark radar charts comparing Realtime-Venus-Omni on eight video benchmarks and Realtime-Venus-Audio on eight audio benchmarks against the models evaluated in the paper. — Figure 1 Video and audio benchmark comparisons. RT-Venus is short for Realtime-Venus; the video chart separates online and offline models.

Technical report

Explore the research

Realtime-Venus: A full-duplex interaction system with asynchronous delegation

arXiv:2609.13814 · September 12, 2026

Read the technical report

BibTeX citation

BibTeX

@misc{zhao2026realtimevenus,
  title         = {{Realtime-Venus}: A full-duplex interaction
                   system with asynchronous delegation},
  author        = {Ruixiang Zhao and Hualei Wang and
                  Renhe Sun and Enzhi Zhou and
                  Jincenzi Wu and Xujie Song and
                  Kexin Shi and Zihang Liu and
                  Pengcheng Zhu and Jiayi Zhou and
                  Baoyue Zhang and Changhao Zhang and
                  Zitong Wang and Jinhong Wang and
                  Tong Niu and Jingjing Liu and
                  Junan Lin and Haolin He and
                  Hengshuo Chu and Yuhui Chen and
                  Jian Liu and Yuge Huang and
                  Junliang Xing and Yuntao Wang and
                  Weiqiang Wang and Chun Yu and
                  Yuanchun Shi},
  year          = {2026},
  eprint        = {2609.13814},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2609.13814}
}

Other work

AI that listens
while it speaks.