How MediaPipe sees a hand — and exactly where it fails
Twenty-one points, thirty frames a second, on a CPU, from a flat RGB image with no depth information. This is how that is possible — and the four failure modes you will meet the first time you rely on it. Every vision-driven robotics project has a moment where the camera stops being a camera and starts being a sensor. For this one, that moment is MediaPipe Hands: a webcam frame goes in, and 21 numbered points in space come out.
It is easy to treat that as a black box. It is also a mistake, because the box has a specific shape, and its failure modes follow directly from how it was built.
It is two networks, not one # The single most useful thing to know about MediaPipe Hands is that it is a cascade: a detector and a regressor, with completely different jobs.
flowchart TB A["📷 Full frame
e.g. 640 × 480"] --> B["Stage 1 — BlazePalm
SSD detector"] B --> C["Oriented palm crop
256 × 256"] C --> D["Stage 2 — Landmark regressor
MobileNetV2-style encoder"] D --> E["63 floats
21 landmarks × (x, y, z)"] D --> F["Presence score"] D --> G["Handedness
left / right"] F -->|"confidence ≥ 0.5"| C F -->|"confidence < 0.5"| B The regressor’s output, drawn back onto the frame: 21 points and the connections between them. Stage 1 — BlazePalm detects palms, never fingers # This is the design decision the whole system rests on.