Summary of Conversational Gloves using arduino
This project records friends telling stories while wearing sensor gloves; Grasshopper (with Firefly) captures Arduino Mega and Kinect inputs to turn finger bends and hand rotation into pentagon-based 3D forms, which are then 3D printed. Accelerometers and flex sensors map finger motion and rotation to vertex positions; Kinect tracks hand movement through space. The resulting geometries are abstracted representations of gestures rather than precise spatial reproductions.
Parts used in the Conversational Gloves:
- Arduino Mega (two units)
- Flex sensors for fingers
- Accelerometers mounted on the back of the hand
- Kinect sensor for hand tracking
- Glove hardware to mount sensors
- Firefly add-on for Grasshopper
- Grasshopper (Rhino plug-in)
- 3D printer and filament for printing forms
This is what came out of my attempt at combining some of my favorite tools and themes…Grasshopper, 3D printing, kinetic motion, and linguistics. I put these gloves on my friends, asked them to tell me a story, and used Grasshopper to record the data from their hand gestures. While these gloves are far from precise instruments, they still produced some interesting forms which I later 3d printed.
Step 1: Grasshopper Script
I used the Firefly add-on in order to receive input from Arduino and Kinect in Grasshopper. Two Arduino Megas track finger positions and hand rotation while a Kinect tracks each hand’s movement through space.
Step 2: Specifics
Three dimensional shapes were made in Grasshopper by extruding a pentagon through space. Each vertex on the pentagon corresponds with a finger. The more a finger is bent, the further the vertex moves inward. The accelerometer on the back of the hand controls the overall rotation of the polygon. Of course, the accelerometers are not meant to know their spatial position so the resulting geometry is very much an abstraction of the gesture and not a completely accurate representation.
Step 3:
In the above picture, I bend my pointer finger. As a result, the Y value from the accelerometer decreases, and so does the radius value of the corresponding vertex, causing it to move inward. A similar effect happens with the other fingers (as shown below).
- How are finger positions tracked in the project?
Finger positions are tracked using flex sensors connected to two Arduino Mega boards and mapped to pentagon vertices in Grasshopper. - Can hand rotation be recorded by the gloves?
Yes, accelerometers on the back of the hand control the overall rotation of the polygon in the Grasshopper model. - Does the Kinect play a role in the system?
Yes, the Kinect tracks each hand's movement through space and provides additional input to Grasshopper via Firefly. - What software is used to process sensor data?
Grasshopper with the Firefly add-on is used to receive and process input from Arduino and Kinect. - How are the 3D shapes generated?
Three-dimensional shapes are made by extruding a pentagon through space, with each vertex linked to a finger's bend amount. - Are the glove measurements precise spatial representations?
No, the accelerometers are not meant to know spatial position, so the resulting geometry is an abstraction of the gesture, not an exact spatial record. - What happens when a finger bends?
When a finger bends, the corresponding pentagon vertex moves inward because the accelerometer Y value decreases and reduces the radius of that vertex. - What is the final output of the captured gestures?
The captured gesture data is used to produce interesting 3D forms that are later 3D printed.
