Projects on Mulham Fetna

A webcam, some vector geometry, and a hand that moves

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Everything in this series in one read: how a $20 webcam ends up driving a 15-DOF CAD model in real time, why every step is deliberately explicit rather than learned, and what broke along the way.

You hold your hand up to a laptop camera. On the other half of the screen, a robotic hand — designed in CAD, never manufactured — closes its fingers at the same moment yours do.

There is no glove, no marker, no depth sensor. Just an RGB webcam, two small neural networks, about forty lines of vector geometry, and a middleware stack that thinks it is talking to a real robot.

All of it is open source under AGPL-3.0 and archived with a DOI: 10.5281/zenodo.22658556.

flowchart LR
    A["📷 Webcam
/dev/video0"] --> B["BlazePalm
palm detector"]
    B --> C["Landmark regressor
21 × (x, y, z)"]
    C --> D["Dot-product geometry
15 interior angles"]
    D --> E["Normalize → flexion
0.0 straight · 1.0 curled"]
    E --> F["Lerp onto the URDF's
mechanical limits"]
    F --> G["/joint_states"]
    G --> H["robot_state_publisher
→ /tf"]
    H --> I["🖥️ RViz digital twin"]

The rule that shaped the build
#

There is an easier version of this project. Collect a few thousand frames of a hand next to the corresponding CAD poses, train a network to map one to the other, and let gradient descent work out the relationship.

Containerizing ROS 2 without losing the hardware

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Web developers containerize to isolate processes. Roboticists containerize and then spend the rest of the day punching holes back through the isolation — for the camera, the GPU, the display server and the network.

ROS 2 Jazzy is hard-locked to Ubuntu 24.04. That single fact is reason enough to containerize: without it, adopting a ROS distribution means adopting an operating system version, on every machine that will ever run the code.

But a ROS container is a strange artifact. A web service container wants isolation — that is the product. A robotics container needs a webcam, a GPU, a display server and multicast networking, all of which live outside it. You end up building a box and then carefully cutting four holes in it.

Here is every hole in this project’s compose file and what it is for.

Cold start to live tracking. Note hand_msgs being compiled separately inside two different containers — that duplication is deliberate and load-bearing.

flowchart TB
    subgraph HOST["🖥️ Host — Ubuntu / Kubuntu"]
        X11["X11 socket
/tmp/.X11-unix"]
        CAM["/dev/video0"]
        GPU["/dev/dri"]
        NET["host network
UDP multicast"]
    end
    subgraph C["Active containers · ROS_DOMAIN_ID=42"]
        HT["hand_tracker"]
        RV["ros_rviz"]
        TS["topic_sniffer"]
    end
    GZ["gazebo_sim
(commented out)"]:::parked
    classDef parked stroke-dasharray: 5 5,opacity:0.55
    CAM --> HT
    GPU --> HT
    GPU --> RV
    X11 --> HT
    X11 --> RV
    NET <--> HT
    NET <--> RV
    NET <--> TS

Hole 1 — the network wall has to come down
#

network_mode: "host"
ipc: host
pid: host

This is the one that breaks stacks silently, so it is worth understanding rather than copying.

From an Onshape assembly to a robot ROS 2 can reason about

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Bridging a modern parametric CAD platform and a fifteen-year-old XML standard is where most roboticists lose days. The exporter translates exactly what it sees — so every shortcut taken in CAD becomes a bug in ROS.

→ Open the assembly on Onshape — it is public, so everything below is checkable against the source.

The assembly: four fingers on blue linkages, the thumb on an orange one, all mounted to a single palm block.

onshape-to-robot is a compiler. Assembly in, robot description out. It reads mate names, mate limits and material densities directly from the CAD document and writes them into URDF as joint names, joint limits and inertia tensors.

That is a genuinely good deal — done properly, the physical properties of your robot are generated rather than hand-typed, and they stay correct when the mechanism changes. Done improperly, you spend your evenings editing XML by hand and discovering that your ring finger has no knuckle.

flowchart LR
    A["Onshape assembly
mates · limits · materials"] --> B["onshape-to-robot
API pull"]
    B --> C["robot.urdf
links · joints · inertials"]
    B --> D["assets/*.stl
visual + collision meshes"]
    C --> E["robot_state_publisher"]
    C --> F["JOINT_MAPPING
limits transcribed by hand"]
    D --> G["RViz / Gazebo"]

Five rules that move work back into CAD
#

Each of these exists because its absence cost real time on this project.

From camera coordinates to mechanical radians

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Twenty-one points in a camera’s coordinate system on one side. A CAD assembly with hard mechanical stops on the other. This is the arithmetic that makes them agree — and the one row where it currently does not.

The vision layer gives you 21 points floating in a normalized coordinate space that has no physical units and a faked depth axis. The mechanical layer gives you fifteen revolute joints, each with a lower and upper bound in radians that came out of a CAD mate.

Nothing connects them. Building that connection is the actual work of this project, and it happens in about forty lines of Python.

flowchart LR
    A["21 landmarks
(x, y, z) normalized"] --> B["Triplet selection
15 × (p₁, p₂, p₃)"]
    B --> C["Two vectors per joint
v₁ = p₁ − p₂ · v₂ = p₃ − p₂"]
    C --> D["Dot product → arccos
θ in radians"]
    D --> E["Normalize
flexion ∈ [0, 1]"]
    E --> F["Lerp onto URDF limits
θ_urdf"]
    F --> G["JointState
names + positions"]

Step 1 — pick three points
#

To measure a joint you need the joint itself and the two bones meeting at it. In landmark terms: the vertex, plus its two neighbours.

How MediaPipe sees a hand — and exactly where it fails

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Twenty-one points, thirty frames a second, on a CPU, from a flat RGB image with no depth information. This is how that is possible — and the four failure modes you will meet the first time you rely on it.

Every vision-driven robotics project has a moment where the camera stops being a camera and starts being a sensor. For this one, that moment is MediaPipe Hands: a webcam frame goes in, and 21 numbered points in space come out.

It is easy to treat that as a black box. It is also a mistake, because the box has a specific shape, and its failure modes follow directly from how it was built.

It is two networks, not one
#

The single most useful thing to know about MediaPipe Hands is that it is a cascade: a detector and a regressor, with completely different jobs.

flowchart TB
    A["📷 Full frame
e.g. 640 × 480"] --> B["Stage 1 — BlazePalm
SSD detector"]
    B --> C["Oriented palm crop
256 × 256"]
    C --> D["Stage 2 — Landmark regressor
MobileNetV2-style encoder"]
    D --> E["63 floats
21 landmarks × (x, y, z)"]
    D --> F["Presence score"]
    D --> G["Handedness
left / right"]
    F -->|"confidence ≥ 0.5"| C
    F -->|"confidence < 0.5"| B

The regressor’s output, drawn back onto the frame: 21 points and the connections between them.

Stage 1 — BlazePalm detects palms, never fingers
#

This is the design decision the whole system rests on.

Why ROS 2 earns its complexity — and how this graph is wired

contact@mulhamfetna.com (Mulham Fetna) — Tue, 08 Sep 2026 00:00:00 +0000

Every mechatronics engineer hits the wall where ROS 2 feels like an enormous tax just to move a few servos. Here is when that instinct is right, when it stops being right, and what this project’s graph actually looks like.

Writing raw sockets on an ESP32 is cleaner on day one. It is genuinely simpler, genuinely faster to get moving, and for a single microcontroller driving a handful of servos it is often the correct engineering decision.

ROS 2 is not a plug-and-play convenience layer. It is distributed middleware built to solve problems you do not have yet — time-synchronizing asynchronous nodes, standardizing message types across C++ and Python, managing coordinate transform trees that nest six levels deep. Adopting it before you have those problems is pure overhead.

The question is when you cross over. For this project, the crossing point was concrete: the moment a second consumer needed the same hand data. One script drawing on a frame is trivial. One script producing angles, another rendering a kinematic tree, a third logging telemetry, all needing the same data at the same instant without knowing about each other — that is when the framework starts paying rent.

Projects on Mulham Fetna

A webcam, some vector geometry, and a hand that moves

The rule that shaped the build #

Containerizing ROS 2 without losing the hardware

Hole 1 — the network wall has to come down #

From an Onshape assembly to a robot ROS 2 can reason about

Five rules that move work back into CAD #

From camera coordinates to mechanical radians

Step 1 — pick three points #

How MediaPipe sees a hand — and exactly where it fails

It is two networks, not one #

Stage 1 — BlazePalm detects palms, never fingers #

Why ROS 2 earns its complexity — and how this graph is wired

The rule that shaped the build
#

Hole 1 — the network wall has to come down
#

Five rules that move work back into CAD
#

Step 1 — pick three points
#

It is two networks, not one
#

Stage 1 — BlazePalm detects palms, never fingers
#