Real Time Face Detection and Tracking Robot using Arduino

Contents hide

1 Materials Required:

2 3D Printing the Required Parts (Optional):

3 Schematic and Hardware:

4 Setting up your Android Application:

5 Programming your Arduino:

6 Working:

Ever wanted to build a Face Tracking Robotic Arm or Robot by simply using Arduino and not any other programming like OpenCV, visual basics C# etc? Then read along, in this project we are going to implement face detection by blending in the power of Arduino and Android. In this project, the mobile camera will move along with your face with the help of servos. The advantage of using the Android Mobile Phone here is that you do not need to invest on a camera module and the whole image detection work can be done in the phone itself, you do not need your Arduino connected to your computer for this to work. Here we have used Bluetooth Module with Arduino to communicate with Mobile wirelessly.

The Android application used in this project was created using Processing Android, you can either directly install the application by downloading the APK file (read further for link) or put on your programming cap and make your own more appealing Android Application using the Processing Code given further in the Tutorial. Learn more about Processing by checking our previous Processing Projects.

By the end of this tutorial you will have a Mini Tilt and Span Robotic Arm that could track your face and move along with it. You can use this (with further advancement) to record your vlog videos or even take a selfie with the rear camera of your mobile phone since it positions your face exactly at the centre of your mobile screen. So!! Sounds interesting? Check the Demo Video at the end this tutorial to see it working. Let’s see how we can build one…

I have tried my best to make this project to work as simple as possible, anyone with minimum knowledge on hardware or coding can use this guidelines to make this project work in no time. However once you make it I suggest you to get behind the codes so that you can really know what makes this thing work and how.

Materials Required:

Arduino Nano
Servo motor SG90 – 2Nos
Android Phone with decent camera
HC-05/HC-06 Bluetooth Module
Computer for programming
3D printer (optional)
9V Battery

3D Printing the Required Parts (Optional):

In order to pan and tilt our mobile phone we need some mechanical structures like a mobile holder and a few servo brackets. You can use a cardboard to make one, since I have a 3D printer I decided to 3D print these parts.

3D printing is an amazing tool that can contribute a lot when building prototype projects or experimenting with new mechanical designs. If you have not yet discovered the benefits of a 3D printer or how it works you can read The beginners Guide to 3D printing.

If you own or have access to a 3D printer then you can use the STL files which can be downloaded from here to directly print and assemble them. However few parts like the mobile phone holder might need some modifications based on the dimensions of your phone. I have designed it for my MOTO G mobile phone. I have used a very basic printer of mine to print all the parts. The printer is FABX v1 from 3ding which comes at an affordable price with a print volume of 10 cubic cm. The cheap price comes with a trade off with low print resolution and no SD card or print resuming function. I am using software called Cura to print the STL files. The settings that I used to print the materials are given below you can use the same or change them based on your printer.

Once you print all the required materials you can secure them in position by using screws and some hot glue. After you assembly is complete it should look something like this below.

Schematic and Hardware:

The Circuit for this Face Tracking on Smart Phone project is shown in the image below:

The Circuit Consists of two Servo motors, out of which one is used to move the mobile phone left/right and the other is used to tilt the mobile phone up/down. The direction in which the servo has to move will be instructed by the Arduino Nano which itself gets information from the Bluetooth (HC-05) module. The whole circuit is powered by a 9V battery.

This circuit can be connected easily on your breadboard or you can also solder these on a small Perf board like I have done here.

Setting up your Android Application:

As, I said earlier the main brain working behind this project is this Android application. This android application was developed using Processing Android. You can directly install this application on your mobile phone and launch that by following the steps below.

Download the APK file from here.
Power on the circuit shown above.
In your phone settings search for Bluetooth module named “HC-05”
If you have named it something else other than “HC-05” change it back to HC-05 since only then the application will work.
Pair with your Bluetooth module with the password “1234” or “0000”.
Now, launch the Application in portrait mode. You should see your camera screen and also “Connected to: HC-05” on the top of your screen.
Try moving your camera over a face and a green box should appear on top of it and its position will also be displayed on the top left corner of your screen as shown below.

You can take this Arduino Face Tracking Project to next level by bringing in lot of advancements for which you won’t need to code your own Android application. Creating an Android application might sound difficult but trust me with the help of Processing you can learn it in no time. The complete processing code that is used to build this application can be downloaded here. You are free to make any advancement with your own creativity. Check below projects to learn more about Processing:

Programming your Arduino:

The Android application will detect the face and its position on screen; it will then decide which direction it should move based on the position of the face so that the face gets to the centre of the screen. This direction is then sent to the Arduino via Bluetooth Module.

The Arduino program for this project is fairly simple, we just have to control the direction of the two servo motors based on the values received from the Bluetooth Module. The complete code can be found at the end of this tutorial, I have also explained few important lines below.

Below line of code establishes a serial connection with pins D12 as RX and D11 as TX. Hence the pin D12 must be connected to the TX of the BT module and the pin D11 to the RX of the BT module.

SoftwareSerial cam_BT(12, 11);  // RX, TX

Then we have initialised the Bluetooth module at a baud rate of 9600. Make sure you module also works on the same baud rate. Else change it accordingly.

cam_BT.begin(9600); //start the Bluetooth communication at 9600 baudrate
cam_BT.println("Ready to take commands");

Below line reads what is coming in through the Bluetooth module. Also the data is saved in the variable “BluetoothData”.

if (cam_BT.available()) //Read whats coming in through Bluetooth
   {
BluetoothData=cam_BT.read();
Serial.print("Incoming from BT:");
Serial.println(BluetoothData);
}

Based on the data received from the Bluetooth the motors direction is controlled. To turn a motor left the motor is decrement by a value of 2 from its previous position. You can increase this value 2 to 4 or 6 if you need the arm to move faster. But, it might create some jerks making the camera unstable.

if (BluetoothData==49) //Turn Left
{pos1+=2; servo1.write(pos1);}

if (BluetoothData==50) //Turn Right
{pos1-=2; servo1.write(pos1);}

if (BluetoothData==51) //Turn Up
{pos2-=2; servo2.write(pos2);}

if (BluetoothData==52) //Turn Down
{pos2+=2; servo2.write(pos2);}

Working:

Once we are ready with our hardware, code and Android Application its time for some action. Simply power your Arduino and open the android application. The Application will automatically connect to the HC-05 (must be named HC-05) Bluetooth module and will wait for a face to be detected. Simply place the phone in our mobile holder and sit in front of it. You should notice your servo motors moving your phone so that your face will be placed at the centre of the screen. Now move around within the range of the camera and your mobile phone will follow your movements. You can also try it by placing and moving any picture.