Skip to main content
  1. Projects/
  2. Tendon-Driven Robotic Hand — A Vision-Teleoperated MuJoCo Digital Twin/

Why the simulated tendon finger is a switch — and three fixes tested in MuJoCo

Mulham Fetna
Author
Mulham Fetna
Renaissance Engineer
Table of Contents
ROS 2 Tendon-Driven Hand MuJoCo Twin - This article is part of a series.
Part 11: This Article
The vision layer sends a carefully normalized, smooth 0-to-1 signal. The simulated finger turns it into two states: open and closed. This is the most important thing to understand about the twin — and every number below was measured on the model in the repository.

The measurement
#

The same flexion was applied to all five motors, the model simulated for 3 s from rest, and the steady-state joint angles recorded — 41 flexion values from 0 to 1.

Flexion Force Index MCP / PIP / DIP Thumb CMC / MP / IP
0.000 +50.0 N −3.3° / −3.2° / −3.3° −2.9° / −2.7° / −2.7°
0.250 +25.0 N −1.7° / −1.6° / −1.7° −1.5° / −1.3° / −1.4°
0.500 0.0 N 0.0° / 0.0° / 0.0° 0.0° / 0.0° / 0.0°
0.625 −12.5 N 91.1° / 91.0° / 91.0° 90.6° / 53.9° / 45.6°
1.000 −50.0 N 94.0° / 93.8° / 93.9° 92.3° / 54.2° / 46.0°

Joint limits are 90°; values just above it are MuJoCo’s soft limit being pressed.

Zooming into the wall (index MCP, steady state from rest):

Tendon force 0 N −0.1 N −0.5 N −1.0 N −2.5 N
Index MCP bend 0.0° 1.0° 62.1° 90.1° 90.2°

The finger’s entire proportional range fits between 0 and −1 N — 1% of the ±50 N command range, a flexion band of 0.50 to 0.51.

Filmstrip at flexion 0.0, 0.5, 0.505, 0.51, 0.52 and 1.0 — only the 0.505 frame is partially curled
Flexion 0.0 · 0.5 · 0.505 · 0.51 · 0.52 · 1.0. Only 0.505 — a −0.5 N pull — shows a partial curl.
Open (+50 N) Closed (−50 N)
Side view of the twin pushed open
Side view of the twin in a fist

Timing and hysteresis
#

Test Result
Close from open at −50 N (MCP reaches 85°) 48 ms
Re-open from closed at +50 N (MCP below 10°) 44 ms
From closed, command 0 N for 2 s still 69.6° — barely drifts
From closed, command +1 N for 2 s fully open (−0.1°)

A finger does not return to open at flexion 0.5; it opens only once the command crosses to the push side. Around 0.5 the finger simply stays where it last was.

Why: nothing pushes back
#

A proportional position needs a restoring force that grows with displacement — a spring. At equilibrium, per joint:

$$\underbrace{r\,F_\text{tendon}}_{\text{tendon torque}} \;=\; \underbrace{k_\text{joint}\,q}_{\text{joint spring}} \;+\; \underbrace{\tau_\text{springs}}_{\text{elastic tendons}} \;+\; \text{friction} + \text{gravity}$$
Restoring element In robot.xml Contribution
Joint stiffness \(k_\text{joint}\) 0 — not set none
Joint damping 0.01 slows motion; no equilibrium force
Joint friction loss 0.001 a tiny dead-band
Tendon stiffness (flexor + extensor) 15 N/m a full curl stretches the extensor ~12 mm → ≈ 0.2 N
Gravity on phalanges a few grams each tiny, orientation-dependent

Nothing grows fast enough to balance even 1 N of pull, so any net pull accelerates the finger into its joint limit, and the limit becomes the equilibrium. Force control against a near-zero spring is a switch.

The thumb’s MP and IP joints stopping near 54° and 46° even at −50 N hasn’t been isolated; the likely explanation is the thumb tendon’s via-point geometry once the CMC hits its stop.

Is the project wrong, then?
#

No — it’s a binary grasp twin, which is exactly how the live demo looks: a crisp open ↔ fist mirror. The fingers travel in ~45 ms, faster than one vision frame, so it looks instantaneous. The smooth flexion signal isn’t wasted either; it’s on ROS 2 for any consumer. But partial poses can’t be mirrored, and calibration mostly moves when the switch flips.

Three fixes, tested
#

Every option below stayed numerically stable over a 3 s settle. Numbers are index MCP / PIP / DIP bend.

Keep force control; give each finger joint a return spring (jnt_stiffness on the 15 finger joints):

Stiffness (N·m/rad) −5 N −12.5 N −25 N −50 N
0 — current 90 / 90 / 90 91 / 91 / 91 92 / 92 / 92 94 / 94 / 94
0.03 86 / 75 / 75 91 / 90 / 90 92 / 91 / 92 94 / 93 / 93
0.1 24 / 21 / 20 72 / 62 / 62 91 / 90 / 90 92 / 92 / 92

At 0.1 the finger is proportional over the first ~25 N of pull (flexion 0.5 → 0.75); go stiffer (start near 0.2) to spread the curl across the full half.

<!-- default class of robot.xml — finger joints, not the servo_* hinges -->
<joint frictionloss="0.001" armature="0.0001" damping="0.01" stiffness="0.1"/>

Force control still only curls on the pull half. To map all of 0 → 1 onto curl, set FORCE_OPEN = 0 and let the springs open the hand.

Raise stiffness on the passive tendon_ext_* tendons (currently 15 N/m) so the extensor becomes the return spring — the rubber-band approach many 3D-printed hands use. Same principle as option A, applied along the tendon path rather than per joint.

Untested here — sweep it the same way before relying on it.

Real SG90 servos are position devices: a horn angle winds the string to a length. Model that directly — a position actuator on each flexor tendon — and interpolate flexion onto length. From the model, the index flexor is 0.1294 m open and 0.0870 m closed.

Gain t = 0.25 t = 0.5 t = 0.75 t = 1.0
kp = 200 33 / 22 / 18 61 / 43 / 36 85 / 61 / 55 90 / 85 / 82
kp = 1000 30 / 24 / 22 58 / 46 / 42 81 / 65 / 63 90 / 90 / 90

kv = 2·√(kp·0.001): ≈ 0.9 for kp 200, 2 for kp 1000.

<position name="pull_index" tendon="tendon_flex_index" kp="1000" kv="2" ctrlrange="0.0870 0.1294"/>
target_length = L_open[finger] + flexion * (L_closed[finger] - L_open[finger])
self.data.ctrl[self.motors[finger]] = target_length

A smooth, monotonic curl over the whole range — and the same command a hardware servo driver needs.

Option C is tracked for implementation in issue #1.

Reproduce it
#

env -u PYTHONPATH MUJOCO_GL=glfw venv/bin/python docs/tools/make_figures.py all

It rewrites docs/data/flexion_sweep.json and regenerates every chart and render from the model and the constants in standalone/main.py.

Mulham Fetna
Author
Mulham Fetna
Renaissance Engineer
ROS 2 Tendon-Driven Hand MuJoCo Twin - This article is part of a series.
Part 11: This Article

Related

How a webcam moves a simulated tendon-driven hand

One vision container turns webcam frames into five numbers. One simulation container turns those numbers into tendon forces. Everything else in this series is detail inside one of those two boxes — or the pipe between them. Left to right, three layers in one frame: vision (landmarks), simulation (the twin), actuation (live motor forces from MuJoCo’s Control panel). End to end # flowchart LR subgraph VISION["🐳 vision_tracker container"] direction TB A["Webcam frame 640×480 BGR"] --> B["MediaPipe Hands 21 landmarks"] B --> C["3 knuckle angles / finger dot product"] C --> D["mean → 1 curl angle (underactuation)"] D --> E["normalize + clip flexion 0..1"] end subgraph TWIN["🐳 mujoco_twin container"] direction TB F["lerp +50 N … −50 N"] --> G["data.ctrl on pull_{finger} motor"] G --> H["spatial tendon through 6 sites"] H --> I["3 passive hinge joints curl"] I --> J["MuJoCo viewer"] end E -- "ROS 2 · /hand/target_flexions sensor_msgs/JointState" --> F Stage What comes out Deep dive MediaPipe Hands 21 (x, y, z) landmarks per frame Part 4 Triplet angles 3 interior angles per finger, in radians Part 5 Averaging 1 curl angle per finger Part 6 Normalization flexion 0..1 (the thumb has its own window) Parts 7–8 ROS 2 topic JointState: names are fingers, positions are flexions Part 15 Lerp force in newtons per tendon Part 10 Tendon physics joint angles Part 11 Why the pipe carries flexions, not angles or forces # The contract between the containers is five unitless numbers: 0.0 is an open finger, 1.0 is a closed one. That choice is the architecture.

Inside the MuJoCo tendon hand model: bodies, joints, sites and sign conventions

A fixed palm, five three-segment digits, five free-spinning servo horns, ten strings through seventy points, five motors. Every number here was read from the compiled model with MuJoCo 3.13.0. Open (+50 N) Neutral (0 N) Fist (−50 N) Index tendon only Middle tendon only Ring tendon only Pinky tendon only Thumb tendon only Previous Next At a glance # Quantity Value Notes Bodies 22 world, palm, 15 phalanges, 5 servo horns Joints / DoF 20 / 20 all hinges; no free joint — the palm is welded to the world Knuckle joints 15 passive, 90° limits Servo horn joints 5 unlimited and unactuated Geoms 87 43 visual + 43 collision meshes + floor Meshes 14 STL Sites 70 tendon via-points and anchors Tendons 10 5 flexor (actuated) + 5 extensor (passive) Actuators 5 <motor> on flexors, ±50 N Timestep / integrator 0.002 s / Euler 500 steps per simulated second Total mass 0.398 kg palm block 341.8 g The kinematic tree # flowchart TB W["world"] --> P["part_1 · palm + servo block 341.8 g · 20 sites"] P --> I1["part_2_4 · index proximal 4.6 g · index_mcp"] --> I2["part_3_4 1.8 g · index_pip"] --> I3["part_4_4 4.6 g · index_dip"] P --> M1["part_2_3 · middle middle_mcp"] --> M2["part_3_3 middle_pip"] --> M3["part_4_3 middle_dip"] P --> R1["part_2_2 · ring ring_mcp"] --> R2["part_3_2 ring_pip"] --> R3["part_4_2 ring_dip"] P --> K1["part_2 · pinky pinky_mcp"] --> K2["part_3 pinky_pip"] --> K3["part_4 pinky_dip"] P --> T1["part_5 · thumb 5.4 g · thumb_cmc"] --> T2["part_6 2.4 g · thumb_mp"] --> T3["part_7 2.5 g · thumb_ip"] P --> H["servo_horn … servo_horn_5 0.3 g each"] Body names are CAD part names, not finger names. Repeated instances get _2, _3, … appended, so the pinky chain is part_2 → part_3 → part_4 and the index chain is part_2_4 → part_3_4 → part_4_4. Address joints, sites, tendons and actuators in code — never bodies. Joints and their signs # Joint Range (rad) Bend direction index_mcp, index_pip, index_dip [−1.571, 0] negative middle_mcp [−1.571, 0] negative middle_pip, middle_dip [0, +1.571] positive ring_mcp, ring_dip [−1.571, 0] negative ring_pip [0, +1.571] positive pinky_mcp, pinky_dip [−1.571, 0] negative pinky_pip [0, +1.571] positive thumb_cmc, thumb_mp, thumb_ip [−1.571, 0] negative servo_* × 5 unlimited — Bend direction is inconsistent, because each Onshape mate’s axis was exported as-is. Anything that reads joint angles must normalize per joint:

The manual MJCF edits that turn a CAD export into a tendon-driven hand

A raw CAD export gives you bodies, joints, meshes and sites — and a hand that does nothing. Four edits and one extra file turn it into a tendon-driven twin. One of those edits, it turns out, does nothing at all — which is worth understanding too. Which edits survive a re-export # flowchart TB subgraph generated["robot.xml — regenerated by onshape-to-robot"] D["① joint defaults manual — re-apply"] T["② tendons + ③ contacts auto-injected from tendons.xml"] B["bodies · joints · sites · meshes generated"] A["④ actuators manual — re-apply"] end S["⑤ scene.xml floor · lights · skybox never regenerated"] -->|"include robot.xml"| generated # Edit Lives in Survives re-export? ① Joint friction / armature / damping defaults robot.xml <default> ✘ re-apply ② Flexor + extensor spatial tendons tendons.xml → injected ✔ ③ Contact exclusions tendons.xml → injected ✔ ④ Five tendon motors robot.xml <actuator> ✘ re-apply ⑤ Environment scene.xml ✔ separate file tendons.xml is kept byte-identical to the <tendon> and <contact> region of robot.xml. Tune a tendon in robot.xml without copying it back and the next export silently reverts it. ① Joint defaults — stability # <default class="ros2-tendon-driven-hand-gazebo-digital-twin"> <joint frictionloss="0.001" armature="0.0001" damping="0.01"/> Attribute Value Role damping 0.01 N·m·s/rad stops a 2 g phalanx reaching absurd speed when 50 N yanks it armature 0.0001 kg·m² rotor-like inertia on each joint’s diagonal — conditions the solver for very light bodies frictionloss 0.001 N·m a small dry-friction dead-band so joints settle instead of creeping Phalanges weigh 1.8–5.4 g. Without these, tiny inertias under large tendon forces blow up the integrator — an earlier, larger revision of the model logged Nan, Inf or huge value in QACC at DOF 128. The simulation is unstable.

From an Onshape assembly to a MuJoCo model with onshape-to-robot

The simulated hand was never modelled by hand. It is an Onshape assembly — five SG90 servos, fifteen knuckle mates, a palm full of tendon channels — pulled through the Onshape API and written out as MuJoCo XML. Here is the design, and every setting that steers the export. Open the Onshape assembly The design # Your browser cannot play this video. Download video. Palm and fingers: four three-phalanx fingers and a three-segment thumb, every knuckle a revolute mate with limits. The RGB triads in the views are mate connectors. Tendon channels: one per finger, running down the palm into the base. Servo block: five SG90-class servos, staggered so each horn sits under a tendon exit. The design has a history # Start 2026-09-02 The first version in the history. v1.0.0 — MediaPipe 2026-09-06 The joint-angle-driven hand behind the RViz predecessor project. v1.0.1 → Main 2026-09-08 Point release and the main line the later work branches from. V3 → Mujoco branch 2026-09-16 The current design used by this twin — the version with the servo base block shown above. Onshape version history Mate features 43 part instances, 112 mate features: the 15 dof_* knuckle mates, the servo mates, and many Fastened mates.

From finger flexion to tendon force: linear interpolation onto a MuJoCo motor

On the far side of the ROS 2 topic, five flexions arrive and five tendon motors wait. One line of linear interpolation connects them — plus a name lookup that can fail silently, and a string that is allowed to push. The formula # $$F(t) = F_\text{open} + t\,(F_\text{closed} - F_\text{open}) = 50 + t\,(-50 - 50) = 50 - 100\,t$$FORCE_OPEN = 50.0 FORCE_CLOSED = -50.0 def _lerp(self, start_val, end_val, t): return start_val + t * (end_val - start_val) def apply_flexions(self, flexions): for finger, flexion_amount in flexions.items(): target_force = self._lerp(FORCE_OPEN, FORCE_CLOSED, flexion_amount) self.data.ctrl[self.motors[finger]] = target_force Three parts of one line # With \(t = 0.75\), a finger 75% closed: