What Is Pose Estimation?
Pose estimation detects 18–25 anatomical keypoints on the wearer — shoulders, elbows, wrists, hips, knees — producing a skeleton the try-on model warps the garment along.
What is pose estimation in AI virtual try-on?
Pose estimation is a computer-vision task that detects the location of human body joints in a photo. For AI virtual try-on, a pose model (typically DWPose or OpenPose-class) finds 18–25 keypoints — head, shoulders, elbows, wrists, hips, knees, ankles. These keypoints form a skeleton that tells the synthesis stage how the wearer’s body is oriented. The garment is then warped along this skeleton so sleeves bend with elbows, the hem hangs at the hip line, and the collar curves with the neck. Without pose estimation, the diffusion model would render the garment in a default frontal pose regardless of how the wearer is actually standing.
- Pose estimation = detecting joint keypoints on the wearer.
- Typical keypoint count: 18 (OpenPose) or 25 (DWPose).
- Output is a skeleton overlay used to warp the garment.
- Essential for non-frontal poses; frontal poses tolerate weaker pose models.
- Errors here manifest as garments that do not follow the body.
How keypoint detection works
Modern pose models are convolutional or transformer networks trained on millions of pose-annotated photos (the COCO Keypoints dataset is the standard). The network takes the person photo as input and outputs a per-keypoint heatmap — a 2D probability distribution for where each joint lives. The argmax of each heatmap gives the predicted joint location.
DWPose, the dominant pose model in 2026 try-on pipelines, predicts 25 keypoints including extra refinements on hand and foot positions. Hand keypoints matter because rings and watches sit on hands and must be preserved through the synthesis.
How pose drives the garment warp
Once the skeleton is known, the garment is warped to fit. For tops, the shoulder keypoints set sleeve attachment points, elbow keypoints set sleeve bends, and wrist keypoints set cuff endpoints. The warp is a thin-plate spline (TPS) transform parameterised by the keypoints — a smooth deformation that maps the flat garment onto the curved body geometry implied by the skeleton.
The warped garment is then handed to the diffusion stage as a conditioning input alongside the human-parsing map. The diffusion model synthesises the final pixels using both signals.
Pose failure modes
Three classes of pose failure degrade try-on. First, missing keypoints — occluded joints (hands behind back, legs cut off the frame) produce missing skeleton segments, and the warp guesses badly. Second, swapped keypoints — left and right limbs mislabelled, producing inside-out garment renders. Third, low-confidence keypoints — partial-body or low-resolution photos give the model weak signal, and the warp drifts.
Front-facing full-body photos at 1024+ pixel resolution avoid all three failures. This is why Drape’s onboarding strongly recommends them.
Definitions
- Keypoint
- A single labelled anatomical landmark (e.g., "left shoulder") with an (x, y) coordinate in the image.
- COCO
- Common Objects in Context dataset — the standard benchmark for pose estimation; defines the canonical 17-keypoint set.
- TPS
- Thin-Plate Spline — a smooth interpolation method used to warp an image based on a set of control point correspondences.
Frequently asked questions
Why does my try-on fail when I am sitting?+
Does Drape need a full-body photo?+
Can I see the detected pose?+
Drape Editorial is the in-house research team behind Drape Try-On. We test virtual try-on models against real garment photography weekly and publish what we learn.