In this project, in collaboration with NVIDIA<\/strong> and Pika<\/strong>, our core motivation is to build a practical distillation pipeline that Pika can deploy to cut inference costs for their video generation services, while maintaining the visual quality their users expect.<\/p>\n\n\n\n Naive attempts to simply compress the teacher\u2019s denoising trajectory into fewer steps tend to produce over-smoothed, low-contrast results and still rely on costly teacher generation. This motivates our use of Distribution Matching Distillation<\/strong> (DMD), which matches distributions rather than trajectories<\/strong> to train a 4-step<\/strong> student generator.<\/p>\n\n\n\n To accelerate Wan while keeping its visual quality, we build on Distribution Matching Distillation (DMD)<\/strong>, introduced by Yin et al. in \u201cOne-step Diffusion with Distribution Matching Distillation\u201d (CVPR 2024)<\/em> and extended in \u201cImproved Distribution Matching Distillation for Fast Image Synthesis\u201d (NeurIPS 2024).<\/em><\/p>\n\n\n\n At a high level, DMD provides a way to distill a slow, multi-step diffusion model into a fast few-step generator by matching distributions<\/strong> rather than individual sampling trajectories.<\/p>\n\n\n\n Traditional distillation for diffusion often tries to make the student mimic the teacher\u2019s denoising trajectory<\/strong> step by step. This tends to:<\/p>\n\n\n\n DMD takes a different approach:<\/p>\n\n\n\n In our implementation, we also avoid precomputing a large teacher dataset of trajectories<\/strong> and the heavy regression losses that come with it, following ideas from improved DMD variants. This makes training more storage-friendly and scalable<\/strong>, which is crucial in the video setting.<\/p>\n\n\n\n We explore two distillation pipelines for fast video generation.<\/p>\n\n\n\n First, we directly distill the base Wan 2.1 model with Distribution Matching Distillation, turning a slow 50-step teacher into a 4-step student that keeps most of the visual quality while cutting inference time dramatically.<\/p>\n\n\n\n Second, in collaboration with NVIDIA and Pika, we finetune Wan on a custom \u201cSquish\u201d video effect and then distill this effect-specific model into a few-step generator\u2014showing that the same DMD pipeline can power both general-purpose and commercial, effect-specialized video generation.<\/p>\n\n\n\nMethod<\/h1>\n\n\n\n
Distribution Matching Distillation<\/a><\/h3>\n\n\n\n
![\"\"](\"https:\/\/mscvprojects.ri.cmu.edu\/2025team2-2\/wp-content\/uploads\/sites\/136\/2025\/12\/image-1-1024x319.png\")
Setup<\/h4>\n\n\n\n
\n
A pre-trained, many-step diffusion model (Wan 2.1 in our case) that we treat as the target distribution<\/strong>.<\/li>\n\n\n\n
A generator that produces samples in 4 denoising steps<\/strong>. This is the model we ultimately want to deploy.<\/li>\n\n\n\n
A diffusion-style network that estimates the score of the student\u2019s distribution<\/strong> at different noise levels.<\/li>\n<\/ul>\n\n\n\nKey idea: match distributions, not trajectories<\/h4>\n\n\n\n
\n
\n
Our Pipelines<\/h3>\n\n\n\n
Results<\/h1>\n\n\n\n
General-purpose few-step generator samples<\/h3>\n\n\n\n