Gesture Controlled PPT Changer

What is the Project About

This is a device capable of recording and recognizing sophisticated 3d gestures and carrying out a preset action if the gesture matches, maybe changing the slide of a presentation, increasing volume or anything else that can be done using a keyboard and mouse.

For example: rotate the hand counter-clockwise to go to the previous slide & twist the hand to go to the next slide.

Some of its key features

  • Can identify any sophisticated 3-Dimensional gestures.
  • Recognise gestures using Dynamic Time Warping (DTW) algorithm.
  • It communicates wirelessly to PC via Bluetooth, no need to connect any additional device to PC for communication.
  • Has an OLED display and menu styled user interface for easy navigation.
  • It is a standalone device, entire processing is done internally in Arduino Mega.
  • It can remember gestures even after power is lost and retrieve after it is repowered again.

How it all started

It all started as fantasy if I could change the slides of the presentation with hand gestures instead of pressing those boring old buttons of a slide changer or a laptop.

Before actually building the device, I had some requirements for it

Requirements

  • Should be as cheap as possible.
  • Need to be small, portable and battery powered.
  • All processing should be done internally, it should be a standalone device.
  • Should communicate wirelessly, there should not be any additional receivers.
  • Should be able to record custom gesture

Step 1: Parts Required

  1. Arduino Mega – link
  2. 0.96-inch I2C OLED display – link
  3. MPU-6050 accelerometer gyroscope combo – link
  4. HC-05 Bluetooth module – link
  5. FTDI programmer – link
  6. Perfboard – link
  7. Push button * 2 – link
  8. Battery – link

And some other common electronics components and tools such as soldering iron, USB cables etc…

Please use the above affiliate links to buy any of the above components, it will help in the future existence of the project.

Step 2: ​Configure the HC-05 Bluetooth Module As HID (Human Interface Device) Device

In this project, we need to send commands to the PC to change the slides, as we don’t want any receivers to insert in the USB port of PC, we can proceed further with Bluetooth.

By default, the firmware which is present in the most common Bluetooth module, HC-05 can act as a slave only.

We need an HID (Human Interface Device) compatible Bluetooth module configured as a wireless Bluetooth keyboard to send commands to the PC to change slides.

We have two options for an HID device, one is to directly buy a HID compatible Bluetooth module RN-42 like this which costs 10x of ordinary Bluetooth module or to flash the firmware of RN-42 to HC-05, both of the modules are based on similar hardware.

I have learnt to change the firmware of HC-05 from Brian Lough and Evan Kale, they have done a wonderful job in explaining how to change the firmware, it makes no sense in reproducing the same here again, so I would like to leave links to their videos and give you the firmware files I have.

Find Brian Lough’s video here

Find Evan Kale’s video here

All the files required for firmware change can be downloaded from the google drive link here

Step 3: Building the Hardware

Now that you have an HID compatible Bluetooth module, you can start building the hardware.

Download the Fritzing circuit schematic file from here.

I recommend to build all the hardware on a piece of perf board and keep it as a shield for Arduino mega.

Step 4: Building the Software

This is the most important part of the project, the code.

Before actually opening my code, I would like to recommend you go to preferences and enable “Code Folding” if you are using Arduino IDE, now you can fold the parts of the code to get a clear idea of the code.

You can find the most updated code from my Github page.

There are many parts in the code, let me explain each part individually.

If you have any doubts, comment it down, I will be happy to help.

Step 5: Recording Gestures

The first step in the process of recognising gestures is to record them, and to record gestures, we are using accelerometer readings from MPU-6050.

The function that I have created for recording gestures is take_reading(), refer it in the code.

For gesture to be clear the sample size needs to be large and for faster processing, the gesture size needs to be small, I have found 50 to be a good fit for both the conditions. Now if we want to record longer gestures, we can average out every 2 or 3 elements to get gesture to 50 elements.

#define DOF 3	//3-degrees of freedom acc_x, acc-y, acc_z
#define avg_lenght 2 	//average out every 2 elements of reading
#define sample_size 50	

int reading[DOF][avg_lenght*sample_size];	//creating a 2-D array to store readings

//taking readings
for(i=0; i<avg-lenght*sample_size; i++)
{
  MPU6050.update();

  reading[0][i] = mpu6050.getAccX();	//taking readings of acceleration in g's, 1g, 1.2g
  reading[0][i] = reading[0][i]*50 + 50;   //ofsetting the value to 50
  if(reading[0][i]<0)	//limiting its value from 0 to 100 
    reading[0][i]=0;
  else if(reading[0][i]>100)
    reading[0][i]=100;

  //repeating the same for remaining 2 degrees of freedom, acc_y, acc_z
}

We have taken the readings, but these are not of sample_size elements, these are of sample_size*avg_lenght elements, we need sample_size elements, so we need to average it out.

if(avg_lenght>1)	//if we need to average
{
  for(i=0; i<DOF; i++)		//for each DOF
  {
    for(j=0; j<sample_size; j++)
    {
      for(k=0; k<avg_lenght; k++)
      {
        sum=sum+reading[i][avg_lenght*j+k];	//add every avg_lenght elemens
      }
      temp_values[i][j]=sum/avg_lenhgt;		//save avg of avg_lenght elements here
      sum=0;
    }
  }
}


else if(avg-lenght==1)	//no need to average
{
  for(i=0; i<DOF; i++)
  {
    for(j=0; j<sample_size; j++)
      temp_values[i][j]=reading[i][j];	//simply copying values to temp_values
  }
}<br>

Now readings for all DOF’s are taken and saved to temp_values array.

Source: Gesture Controlled PPT Changer


About The Author

Muhammad Bilal

I am a highly skilled and motivated individual with a Master's degree in Computer Science. I have extensive experience in technical writing and a deep understanding of SEO practices.

Scroll to Top