Skip to main content
  1. Projects/
  2. Tendon-Driven Robotic Hand — A Vision-Teleoperated MuJoCo Digital Twin/

From finger flexion to tendon force: linear interpolation onto a MuJoCo motor

Mulham Fetna
Author
Mulham Fetna
Renaissance Engineer
Table of Contents
ROS 2 Tendon-Driven Hand MuJoCo Twin - This article is part of a series.
Part 10: This Article
On the far side of the ROS 2 topic, five flexions arrive and five tendon motors wait. One line of linear interpolation connects them — plus a name lookup that can fail silently, and a string that is allowed to push.

The formula
#

$$F(t) = F_\text{open} + t\,(F_\text{closed} - F_\text{open}) = 50 + t\,(-50 - 50) = 50 - 100\,t$$
FORCE_OPEN = 50.0
FORCE_CLOSED = -50.0

def _lerp(self, start_val, end_val, t):
    return start_val + t * (end_val - start_val)

def apply_flexions(self, flexions):
    for finger, flexion_amount in flexions.items():
        target_force = self._lerp(FORCE_OPEN, FORCE_CLOSED, flexion_amount)
        self.data.ctrl[self.motors[finger]] = target_force

Three parts of one line
#

With \(t = 0.75\), a finger 75% closed:

  1. Range: \(F_\text{closed} - F_\text{open} = -100\) N — the whole span of the scale.
  2. Distance along it: \(0.75 \times -100 = -75\) N.
  3. Anchor: the scale starts at +50, not 0, so \(50 + (-75) = -25\) N.
Flexion \(t\) Force Meaning
0.00 +50 N open — the motor pushes the tendon
0.50 0 N no actuation
1.00 −50 N fist — maximum pull

Seen live
#

OK sign, live: pull_thumb −32.9 N and pull_index −9.67 N; the other fingers near +46 N
MuJoCo’s Control panel during an OK sign: thumb flexion 0.83 → −32.9 N, index 0.60 → −9.7 N, open fingers near +46 N.

Looking actuators up by name
#

self.motors = {
    "thumb":  mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_ACTUATOR, "pull_thumb"),
    "index":  mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_ACTUATOR, "pull_index"),
    ...
}

data.ctrl is a flat array ordered by the XML — here pinky, ring, middle, index, thumb (IDs 0–4). Resolving names once at startup means reordering or adding actuators never breaks the Python.

The silent failure. Rename an actuator and mj_name2id returns -1 — no exception. Then data.ctrl[-1] writes to the last actuator: one finger’s command drives another finger. Assert that every ID is >= 0 at startup.

On the ROS side, incoming names are filtered against the same dictionary, so an unknown finger name in a message is ignored instead of crashing the callback:

flexions = {name: pos for name, pos in zip(msg.name, msg.position) if name in self.sim.motors}

A string that pushes
#

A MuJoCo <motor> on a tendon applies force along the tendon’s length (gear = 1):

  • negative → shortens the tendon → pulls, exactly what a servo winding a string does;
  • positive → lengthens it → pushes.

Real strings can’t push. MuJoCo lets a tendon motor apply compression anyway, and this model uses that to drive fingers open at +50 N — which presses them about 3° past their open stop into the soft joint limit. On hardware the equivalent is a second, antagonistic tendon or a return spring. The model has extensor tendons, but they’re passive — and that has consequences (Part 11).

ctrlrange must agree with Python
#

<motor name="pull_index" tendon="tendon_flex_index" ctrlrange="-50 50" .../>

The limits live in two files. MuJoCo clamps ctrl to ctrlrange (the model uses autolimits="true"), so raising FORCE_CLOSED to −80 without editing the XML silently saturates at −50.

Mulham Fetna
Author
Mulham Fetna
Renaissance Engineer
ROS 2 Tendon-Driven Hand MuJoCo Twin - This article is part of a series.
Part 10: This Article

Related

Normalizing finger curl: from radians to a 0–1 flexion

Radians belong to the camera. Newtons belong to the robot. The number that crosses between them is a plain percentage — and producing it takes one inverted formula and one clip. The formula # RAW_STRAIGHT_ANGLE = 3.10 # ~177.6° — open finger RAW_CURLED_ANGLE = 1.60 # ~91.7° — fully curled finger flexion = (straight_limit - avg_angle) / (straight_limit - curled_limit) flexions[finger_name] = float(np.clip(flexion, 0.0, 1.0)) $$\text{flexion} = \operatorname{clip}\!\left(\frac{\theta_\text{straight} - \bar\theta}{\theta_\text{straight} - \theta_\text{curled}},\; 0,\; 1\right)$$ The three pieces # Denominator — the range. \(3.10 - 1.60 = 1.50\) rad of travel. It sets the scale.

Finger joint angles from three hand landmarks: the dot-product geometry

Every knuckle angle in this project comes from three landmarks and one dot product. No learning, no lookup table — just the definition of the angle between two vectors, plus two guards that keep a single glitchy frame from sending NaN into a motor command. Three points make an angle # An angle needs a vertex and two rays. A knuckle is the vertex; the two bones meeting there are the rays: a base point, where the previous bone starts, the vertex — the knuckle being measured, an end point, where the next bone ends. flowchart LR P1(("p1 base")) -- "v1 = p1 − p2" --- P2(("p2 vertex knuckle")) P2 -- "v2 = p3 − p2" --- P3(("p3 end")) The triplet table # self.finger_triplets = { "thumb": [(0, 1, 2), (1, 2, 3), (2, 3, 4)], "index": [(0, 5, 6), (5, 6, 7), (6, 7, 8)], "middle": [(0, 9, 10), (9, 10, 11), (10, 11, 12)], "ring": [(0, 13, 14), (13, 14, 15), (14, 15, 16)], "pinky": [(0, 17, 18), (17, 18, 19), (18, 19, 20)] } Landmark 0 — the wrist — starts every finger’s first triplet. Finger Triplet Vertex Joint measured Index (0, 5, 6) 5 MCP — joins finger to palm Index (5, 6, 7) 6 PIP — middle knuckle Index (6, 7, 8) 7 DIP — fingertip knuckle Thumb (0, 1, 2) 1 CMC — saddle joint at the wrist Thumb (1, 2, 3) 2 MCP Thumb (2, 3, 4) 3 IP Why every first triplet starts at 0. The palm has no landmark of its own, so the wrist → knuckle line stands in for the metacarpal bone. It isn’t exactly collinear with a straight finger — ring and pinky metacarpals fan outward — so a relaxed straight finger rarely measures a full \(\pi\). That’s part of why the calibrated “straight” threshold is 3.10 rad rather than 3.14.

The ROS 2 topic contract: one JointState topic, two nodes, and rclpy inside a render loop

The whole ROS 2 layer is one topic carrying five numbers. The interesting decisions are what those numbers mean, which message type carries them, and how to run a ROS subscriber when a 3D viewer owns your main thread. Why ROS 2 between vision and physics at all? # The single-process script works well. Splitting it across ROS 2 buys: Benefit Concretely Process isolation a MediaPipe crash doesn’t kill the simulator, and vice versa Independent environments vision and physics get their own container, dependencies and restarts Swappable endpoints replace the twin with a servo driver, or the tracker with a data glove Free observability ros2 topic echo, hz, bag record on the live stream Network transparency any machine on the LAN can subscribe The costs — a bigger stack, DDS configuration, one more hop — are small next to 19–85 ms of inference (Part 19). The node graph # flowchart LR subgraph C1["🐳 vision_tracker"] V["/vision_tracker_node timer · 30 Hz"] end subgraph C2["🐳 mujoco_twin"] T["/mujoco_twin_node spin_once in viewer loop"] end V -- "/hand/target_flexions sensor_msgs/JointState · depth 10" --> T V -. "any LAN subscriber ROS_DOMAIN_ID=42" .-> X["ros2 topic echo · rosbag future servo driver"] Every frame of the twin is driven by one JointState message like the one below. The contract # Field Value Topic /hand/target_flexions Type sensor_msgs/msg/JointState Publisher vision_tracker_node, timer at 30 Hz — effective rate bounded by inference Subscriber mujoco_twin_node QoS default reliable, keep-last 10 header.stamp publisher clock at publish time name ["thumb", "index", "middle", "ring", "pinky"] position flexion per finger, 0.0 open … 1.0 closed, same order as name velocity, effort empty A real message, captured with ros2 topic echo during testing (published by hand with ros2 topic pub, hence the zero stamp):

From an Onshape assembly to a MuJoCo model with onshape-to-robot

The simulated hand was never modelled by hand. It is an Onshape assembly — five SG90 servos, fifteen knuckle mates, a palm full of tendon channels — pulled through the Onshape API and written out as MuJoCo XML. Here is the design, and every setting that steers the export. Open the Onshape assembly The design # Your browser cannot play this video. Download video. Palm and fingers: four three-phalanx fingers and a three-segment thumb, every knuckle a revolute mate with limits. The RGB triads in the views are mate connectors. Tendon channels: one per finger, running down the palm into the base. Servo block: five SG90-class servos, staggered so each horn sits under a tendon exit. The design has a history # Start 2026-09-02 The first version in the history. v1.0.0 — MediaPipe 2026-09-06 The joint-angle-driven hand behind the RViz predecessor project. v1.0.1 → Main 2026-09-08 Point release and the main line the later work branches from. V3 → Mujoco branch 2026-09-16 The current design used by this twin — the version with the servo base block shown above. Onshape version history Mate features 43 part instances, 112 mate features: the 15 dof_* knuckle mates, the servo mates, and many Fastened mates.

How a webcam moves a simulated tendon-driven hand

One vision container turns webcam frames into five numbers. One simulation container turns those numbers into tendon forces. Everything else in this series is detail inside one of those two boxes — or the pipe between them. Left to right, three layers in one frame: vision (landmarks), simulation (the twin), actuation (live motor forces from MuJoCo’s Control panel). End to end # flowchart LR subgraph VISION["🐳 vision_tracker container"] direction TB A["Webcam frame 640×480 BGR"] --> B["MediaPipe Hands 21 landmarks"] B --> C["3 knuckle angles / finger dot product"] C --> D["mean → 1 curl angle (underactuation)"] D --> E["normalize + clip flexion 0..1"] end subgraph TWIN["🐳 mujoco_twin container"] direction TB F["lerp +50 N … −50 N"] --> G["data.ctrl on pull_{finger} motor"] G --> H["spatial tendon through 6 sites"] H --> I["3 passive hinge joints curl"] I --> J["MuJoCo viewer"] end E -- "ROS 2 · /hand/target_flexions sensor_msgs/JointState" --> F Stage What comes out Deep dive MediaPipe Hands 21 (x, y, z) landmarks per frame Part 4 Triplet angles 3 interior angles per finger, in radians Part 5 Averaging 1 curl angle per finger Part 6 Normalization flexion 0..1 (the thumb has its own window) Parts 7–8 ROS 2 topic JointState: names are fingers, positions are flexions Part 15 Lerp force in newtons per tendon Part 10 Tendon physics joint angles Part 11 Why the pipe carries flexions, not angles or forces # The contract between the containers is five unitless numbers: 0.0 is an open finger, 1.0 is a closed one. That choice is the architecture.