Gesture Controlled Robotic Arm Using Kinect & Arduino

This is a Simulink Model to Control a Robotic Arm By Gestures that are captured using Kinect. A Robotic Arm is developed using Servo Motors.

Step 1: Things Used in This Project

Hardware components:

1. Arduino Mega 2560
2. Microsoft Kinect for Xbox 360
3. Microsoft Xbox Kinect Adapter
4. Adafruit Servo Motor
5. SparkFun USB to Serial Breakout – FT232RL
6. Gripper
7. Solderless Breadboard Full Size
8. Jumper wires (generic)

Software apps and online services:

1. Arduino IDE
2. Microsoft Kinect for Windows SDK v1.6
3. Microsoft Windows SDK 7.1
4. MATLAB 2013a

Step 2: Story

This is a Simulink Model to Control a Robotic Arm By Gestures that are captured using Kinect Sensor. A Robot Arm is developed using Servo Motors that replicates the Right Arm Shoulder, Elbow and Hand Movements.

My File at Mathworks File Exchange : https://in.mathworks.com/matlabcentral/fileexchan…

A Project By

B.Avinash & J.Karthikeyan

The Zip file contains the Matlab Simulink Support for Kinect and the Simulink Model named “GestureControlledRoboticArm” that WE HAVE CREATED and also the Arduino Code/Wiring Diagram. The Simulink Support for Kinect was downloaded from Mathworks File Exchange and the full credits go to “Takashi Chikamasa” for providing it. We do not own the copyrights or have developed the Simulink Support For Kinect. Simulink

Support for Kinect consists of IMAQ, Image, IR, Depth, Motion and Skeleton blocks to achieve real-time image data acquisition from Kinect devices such as Microsoft Kinect or ASUS Xtion PRO under Simulink simulation environment. It works with 32bit/64bit version of MATLAB R2010b or later in Windows7 32bit/64bit and it requires the following 3rd party software:

[C MEX Compiler] – Microsoft Visual Studio 2010 Express Edition (VC++) – Microsoft Windows SDK 7.1 [To use OpenNI http://openni.ru/ ] – OpenNI – PrimeSense NITE – PrimeSensor Module [To use Microsoft Kinect SDK http://openni.ru/ ] – Microsoft Kinect for Windows SDK v1.6 For more detailed information, please refer to document in the zip file and help of the blocks. [Optional MATLAB products] – Computer Vision System Toolbox (highly recommended) – DSP System Toolbox (to acquire Kinect Microphone Array data) – Simulink Coder (to generate *.exe file)

Acknowledgements : Kinect Matlab, Simulink Support For Kinect, and Kinect Microsoft Sdk inspired this Project.

Step 3: Custom Parts and Enclosures

Simulink Model

Open the GestureControlledRoboticArm.mdl using Matlab after installing the Simulink Support for Kinect.

Download : GestureControlledRoboticArm.mdl

Matlab Simulink Code & Library Files (Instructions included)

In the .ZIP, you can find the Simulink for Kinect Support developed by Takashi Chikamasa, Mathworks. Inside the “slkinect” folder, you can find the instruction on how to install the library file in Matlab.

Download : Gesture Controlled Robotic Arm Using Matlab Simulink Support For Kinect.zip

Step 4: Schematics – ​Arduino – Servo Connection

Arduino – Servo Connection

The data output (angle) from the matlab software is sent to the arduino via TTL device. The connection from the TTL – Arduino & Arduino – Servo motors are as shown in the diagram.

Step 5: Schematics – ​Simulink Model (Gesture Controlled Robotic Arm)

Simulink Model (Gesture Controlled Robotic Arm)

This is the simulink model we have developed to aquire the data from kinect, convert xyz co-ordinates to angle, decrypt the data and send it to Arduino via Serial Communication.

Step 6: Code – Arduino Program to Receive Angle Data From Matlab

Arduino Program to Receive Angle Data from Matlab – Arduino

The angle output from the Matlab Software is encypted. In the Arduino IDE, we have written a rogram to receive the angle data, decrypt it and send it to the servo motors.

Download : Arduino Code

Step 7: Awards and Recognitions

“Gesture Controlled Robotic Arm Using Kinect (Simulink Support For Kinect) and Arduino” has been chosen in the week’s ( 29/10/2016) Pick of the Week! – Matlab Simulink Hardware Challenge 2016

Won the 4th place in “MATLAB International Simulink Hardware Challenge 2016″

http://blogs.mathworks.com/pick/2016/10/28/2016-ma…

https://in.mathworks.com/matlabcentral/fileexchang…

Read more: Gesture Controlled Robotic Arm Using Kinect & Arduino