Abstract
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation $\Phi(\tau)$, which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability.
Brownian Bridge and Bridge Forcing
LeapTalk reformulates talking-head generation as reference-anchored data-to-data transport instead of repeatedly reconstructing each autoregressive chunk from Gaussian noise. In Bridge Forcing, the Brownian bridge starts from the persistent reference image $\mathcal{I}$ and moves toward the target frame as $X_t=(1-t)\mathcal{I}+tX_1+\sqrt{t(1-t)}\,\epsilon$, so every streaming chunk remains tied to the same identity while the prefix from the previous chunk preserves motion continuity. This design avoids the error accumulation and identity drift caused by conventional noise-to-data flow matching, enabling stable long-form generation.
Heterogeneous Distillation Framework
To achieve real-time one-step synthesis, LeapTalk distills a pre-trained flow-matching teacher into a Brownian-bridge student through a heterogeneous DMD framework. Because the teacher and student follow different generative trajectories, the same timestep corresponds to different noise levels; LeapTalk resolves this mismatch with an SNR-aligned time transformation $t=\Phi(\tau)$, allowing both score functions to be evaluated consistently. The framework further introduces audio-driven classifier-free guidance during distillation, preserving fine-grained lip motion and visual fidelity under extreme step reduction.
BibTeX
@misc{zhang2026leaptalkbreakinglatencyqualitytradeoff,
title={LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation},
author={Rongxiang Zhang and Songhua Liu},
year={2026},
eprint={2608.00079},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2608.00079},
}