Gesture Controlled Robotic Hand

Mechanical Design

System Architecture and Human Input

Human hand motion serves as the reference input for the system. 

A wearable glove is instrumented with magnetic encoders mounted at the joints of the fingers, allowing direct measurement of joint rotation. 

This joint-level sensing approach enables the robotic hand to mirror natural finger motion continuously, rather than executing discrete gestures or predefined poses.

This architecture places the human operator directly in the control loop, creating an intuitive and responsive mapping between human motion and robotic actuation.

Sensing Architecture and Encoder Selection

Each sensed joint uses an AS5600 magnetic rotary encoder to measure angular position directly at the joint.

•  Magnetic encoders were selected due to their non-contact operation, resistance to mechanical wear, and ability to provide position feedback even when joints are manually back-driven or experience compliance.

• Direct joint sensing avoids reliance on servo-reported position, which can be inaccurate in multi-joint systems due to backlash, compliance, and load variation. This decision was made for maintaining consistent motion mapping across a high number of joints.

The system uses ten magnetic encoders in total. Because the encoders share identical I2C addresses, an I2C multiplexer is used to allow independent access to each sensor while maintaining a clean and scalable communication architecture.

Electronics and Hardware

This board required careful consideration of connector placement, bus routing, grounding strategy, and power delivery.

As system complexity increased, it became clear that PCB layout and power architecture had a direct impact on sensor stability, communication reliability, and overall system behaviour.

Control System and Firmware

I developed embedded firmware on the Teensy 4.1 to manage sensor acquisition, data processing, and servo actuation across all joints. The firmware is structured to handle a large number of sensors in a deterministic and testable manner.

During each update cycle, the firmware:

1. Selects each encoder through the I2C multiplexer

2. Reads raw angular position data from the magnetic encoders

3. Converts raw values into normalized joint angles

4. Maps joint angles to corresponding servo commands

5. Outputs PWM signals to actuate each joint
   
   

This creates a direct, low latency mapping between human finger motion and robotic hand motion.

To improve stability and reduce jitter, I implemented multiple layers of signal conditioning and motion limiting:

• Median sampling (multiple reads per update) to reject random noise spikes

• Low-pass filtering to smooth high frequency sensor fluctuations

• Deadband to ignore tiny angle changes that would otherwise cause servo jittering near a steady position

• Slew rate limiting to cap how fast commanded angles can change, preventing sudden jumps, overshoot, and chatter

• Fixed update rate tuned to servo behaviour (instead of over-updating), improving consistency and reducing twitching
  
  
  

The firmware is modular, allowing individual joints or subsystems to be isolated during debugging, and enabling future expansion, such as automated calibration routines and improved filtering.

Testing and Iteration

Testing this system required extensive debugging across mostly electrical and firmware domains. With many sensors and actuators, similar symptoms could arise from different failure modes, including sensor misalignment, magnet selection, electrical noise, bus timing, or power instability.

Magnetic encoder performance was particularly sensitive to magnet strength, spacing, and alignment. Selecting appropriate magnets and mounting them consistently required repeated experimentation. Small deviations often resulted in unstable or non-linear readings, which had to be isolated through structured testing.

As the number of sensors increased, signal integrity became a major challenge. Noise on shared signal lines and I2C bus interactions occasionally caused readings from one sensor to influence or interfere with others, especially when multiple devices were accessed in rapid succession through the multiplexer. Addressing this required careful firmware sequencing, filtering, and attention to grounding and routing practices.

Power distribution also emerged as a critical limitation as the system scaled. Driving multiple servos from a shared supply introduced voltage drops, which in turn affected both sensor reliability and actuator behaviour. In some cases, servos would behave erratically, spin continuously, or fail to respond altogether, symptoms that were ultimately traced back to power sag, shared return paths, and insufficient isolation between logic and actuation loads. These issues highlighted the importance of robust power architecture and proper decoupling in mixed-signal mechatronic systems.

Testing this system required extensive debugging across primarily the electrical and firmware domains. With many sensors and actuators operating simultaneously, similar symptoms often originated from different failure modes, including sensor misalignment, magnet selection, electrical noise, I²C bus behavior, timing issues, or power instability.

Conclusion and Next Steps

This project represents a fully integrated robotic system, combining wearable sensing, dense electronics, embedded firmware, and multi-joint actuation. It strengthened my ability to design, debug, and iterate complex hardware systems where failures are rarely isolated to a single domain.

Future work will focus on improving the power architecture, adding calibration and fault detection routines, reducing latency, and increasing robustness under continuous operation. The current system architecture was designed to support these improvements without fundamental redesign.