How AI Generates Clothing — The Diffusion Pipeline Explained
Every modern try-on system uses a diffusion model to generate the final clothed image. Here is a plain-English explanation of how AI generates a photorealistic image of you wearing a new garment.
How do diffusion models work in AI virtual try-on?
A diffusion model in virtual try-on starts with pure random noise the same shape as the output image and iteratively removes that noise over 20–50 steps, conditioned on three signals: the segmented garment image, the human-parsed person map, and the pose skeleton. At each step the model predicts how much noise to remove based on its training on millions of (person + garment + result) triplets. The final step produces a clean photorealistic image of the person wearing the new garment. The conditioning signals constrain the result so identity (face, skin, accessories) is preserved while the existing garment is overwritten.
- Diffusion = iterative denoising from random noise to clean image.
- Conditioned on garment + person-parse + pose-skeleton.
- Typically 20–50 denoising steps; more steps = higher quality, slower.
- Trained on millions of paired (person, garment, result) examples.
- Modern try-on models are diffusion + ControlNet-style conditioning.
The denoising idea in plain English
A diffusion model is trained on a simple game: take a clean image, gradually add noise until it is pure static, then learn to reverse the process. After enough examples, the model can start from pure noise and produce a coherent image — because it has learned the structure of the data distribution.
At generation time, the process runs in reverse. Start with random noise the same shape as your desired output. Pass it through the model along with conditioning signals (the garment, the parsed person, the pose). The model predicts a slightly less noisy version. Repeat 20–50 times. The final iteration is a clean image.
Why conditioning matters
A vanilla diffusion model would generate any plausible image. Try-on requires a very specific image: this person, this garment, this pose. Conditioning is how we constrain the output.
ControlNet-style architectures inject the conditioning signals (garment + parse + pose) into the diffusion U-Net at multiple resolution scales. The denoising network learns to respect those signals at every step. The result is constrained generation: the model can only produce images consistent with the given garment on the given person in the given pose.
FASHN v1.6, the model behind Drape Try-On, layers additional conditioning channels for identity preservation — explicitly copying face and accessory pixels through from the input.
Why diffusion replaced GANs
Before 2022, virtual try-on used Generative Adversarial Networks (GANs). GANs were faster but struggled with identity preservation and fine-detail garment patterns. Diffusion models are slower (20–50 forward passes vs 1 for a GAN) but produce dramatically higher quality, especially on edge cases like patterned fabric and small accessories.
The quality gap was decisive enough that the entire try-on field shifted to diffusion by 2024. Today every major try-on model — FASHN, IDM-VTON, OOTDiffusion, Kling video try-on — is diffusion-based.
Definitions
- U-Net
- A symmetric encoder-decoder architecture with skip connections, used as the backbone of most diffusion models.
- ControlNet
- A conditioning architecture that injects auxiliary signals into a pretrained diffusion model at multiple resolution scales.
- Denoising step
- One forward pass of the diffusion model that predicts how much noise to remove at the current step.
Frequently asked questions
Are diffusion try-on results deterministic?+
Why does generation take 12 seconds?+
Can diffusion try-on hallucinate features?+
Drape Editorial is the in-house research team behind Drape Try-On. We test virtual try-on models against real garment photography weekly and publish what we learn.