Pantomime – Gestures to actions for disability aid and more

Pantomime is a configurable gesture controller unit that can be used for reliable human-machine interaction. For the demo purpose, we have shown a use case of the Pantomime in a chassis of 4WD, which here can be easily altered with a smart wheelchair. The Pantomime responds to the hand gestures to control the 4wd rover. The motions we were able to show using the prototype are forward, backward, left, right.

Video

Inspiration

Being in the field of electronic engineering, we wanted to explore and experiment with the different possibilities of working with several sensors and components. The idea of controlling mobile devices using simple hand gestures is very fascinating and useful. We also wanted to make a product that solves several problems, makes tasks easier and all-in-all be a useful tool for people.

What it does

Pantomime is a configurable gesture controller unit that can be used for reliable human-machine interaction. For the demo purpose, we have shown a use case of the Pantomime in a chassis of 4WD, which here can be easily altered with a smart wheelchair. The Pantomime responds to the hand gestures to control the 4wd rover. The motions we were able to show using the prototype are forward, backward, left, right.

How we built it

The prototype of the project was built using 4 300 RPM BO motors, 4 wheels, an L293d motor driver, a 433mHz RF (transmitter and receiver) modules to transmit and receive the gesture commands, and two Arduino Uno (one for the receiver i.e. on the rover, and the other for the transmitter circuitry) for communication and controlling purposes. Along with that, to power the rover, we used 2 3.7v Li-On batteries. The chassis was built using an acrylic board. Some basic Arduino coding was done to communicate between the transmitter and receiver with checksums. For the gesture identifying wearable, we interfaced the transmitter with a three-axis accelerometer to calculate the angle of inclination in each direction. And lastly to show the utility of the receiver we incorporated it in a 4WD rover.

For the software part, we used EasyEDA to design the PCBs and used Arduino IDE to develop the firmware.

Challenges we ran into

Setting up the transmitter and receiver modules to send and receive the gesture commands was a bit tricky, there is a lag between the gesture made and the rover to perform the respective action, but later by modifying the code with some filters and adjusting the hardware circuitry as well, we got comparatively better results. We were also having some troubles with the 5v input port of the Arduino connected to the receiver that is on the rover, but we tackled that by moving the connections to the 5v port of the ICSP headers.

Accomplishments that we’re proud of

Completing the project on time including the PCB design, getting the prototype to work, and overcoming the obstacles faced while building it were some of our greatest achievements.

What we learned

We learned about RF communication and interfacing the modules with an accelerometer and a 4WD rover. We also learned about checksums, the eradication of errors, and haptic feedback.

What’s next for Pantomime

Next, we would definitely try to make the interface much smoother and efficient. We would also like to make Pantomime market-ready and user-configurable. We are planning to test the same gesture controller on different mobile devices like drones in the future.

Source: Pantomime – Gestures to actions for disability aid and more