Blind Maze Navigation using 2-DOF Haptic Joystick

The domain of haptics has immense potential as a technology to assist visually-impaired individuals with their navigation of the physical world. In this project, we sought to develop a 2-DOF haptic joystick to provide force-feedback to a user as she attempts to navigate a maze – with no visual feedback whatsoever. The proof of concept we show here takes a given maze, and renders it as a 2D navigable virtual environment.

Introduction

We see many applications in the field of haptics to improving the quality of life for individuals with disabilities. In particular, the reliance of haptics of the sense of touch lends itself to aiding the visually-impaired. Consider the case of a wheelchair-bound individual with a visual impairment. Even with modern wheelchair technology, the burden of safe and effective navigation through the environment is entirely on the operator of the wheelchair. If the user is unable to see obstacles and visually sense a clear path, the assistive technology that the wheelchair provides is effectively useless. The 2-DOF haptic joystick is an attempt at remedying that problem. By coupling traditional joystick navigation with force-feedback, haptic joystick technology could allow visually-impaired, wheelchair-bound users to safely navigate their environments.

Our project is half of the solution: not having the ability to generate real-time map information, our device is reliant on pre-determined “mazes” to simulate the notion of a navigable space. The next step would be using real-time environmental mapping (such as with the Kinect’s depth sensor) to generate “mazes” on the fly, allowing the virtual environment to truly mimic the user’s surroundings and provide useful haptic feedback.

Background

There exist many other haptic devices that allow for 2-DOF movement. For example, Jared and Sam created a planar 2-DOF device to emulate gravitational forces. All-purpose haptic feedback joysticks are also common, which often provide vibrational feedback during video games. Other special-purpose joysticks have also been created, for MRI applications, wheelchair navigation, and motor driving. A number of different implementation schemes are employed, both for sensors and actuators. Popular alternatives to our optical encoder sensing set-up include potentiometers and magneto-resistive (MR) sensors, as in Prof. Okamura’s 1D Hapkit design. On the actuation front, gears and chain-drives are common alternatives to the rubber friction drive model we show here.

Design

Hardware

For our device that would allow for both 2-DOF movement and force feedback, we designed a two-axis gimbal. We reused the transmission from the Hapkit (paddle and friction drive) to couple each axis with a motor. Motor positioning data was obtained with optical encoders in place of the Hapkit’s MR sensors. The optical encoders are Maxon HEDS-5540 encoders, coupled with a 10-watt Maxon RE 25 motor (datasheet).

Besides four 1/4-20′ fasteners, the smaller parts (fasteners, bearings, shaft collars, etc) used were taken from the Hapkit parts list or found in the CHARM lab. All other parts besides the paddle were modeled in Solidworks, then laser cut from either 1/8” acrylic or Duron in the Product Realization Lab. Acrylic was used for the gimbal and duron for the joystick’s container. The frame was initially assembled using epoxy, but following an unfortunate incident where gravity got the better of the structure, hot glue usurped the favored spot.

The joystick itself is a repurposed wooden chopstick (no longer sanitary) mounted to the top of the device with rubber bands and duct tape. The duct tape is flexible enough to allow the chopstick to move and rotate as a 2D pivot, while the rubber bands provide enough friction for the joystick to remain vertically anchored and consistently coupled with the gimbal.
!!!# Electronics

The joystick was powered by an Arduino Uno and the Arduino Motor Shield. The Uno reads and processes the motor encoder values, while the shield’s built-in motor controller allows it to drive the two motors via PWM.

In order to read both encoder values, each encoder had to communicate with the Arduino via a pin capable of performing software interrupts. The Arduino Uno only has two interrupt pins (2 and 3). Unfortunately, the Arduino Motor Shield is hard-wired to use pin 3 as Motor A’s PWM input. This caused us a sufficient amount of trouble – we explored a number of options to resolve the issue, finally deciding on externally wiring the shield to the the Arduino instead of piggybacking the motor shield onto the board. Note that all the pins do not need to be connected from the shield to the Uno – only the pins specified below are necessary for function of both encoders and motors. A circuit diagram of the wiring we used is shown below:

With this configuration, the Uno writes to pins 5 and 6 as the two motor’s PWM pins instead of the default 3 and 10, reserving pins 2 and 3 as interrupt pins for the encoders.

Software

A map can be read from a text file, with an arbitrary number of rows and columns. Each character represents a 40px by 40px tile. ‘X’ denotes a solid wall in that tile, and any other character denotes navigable space. A sample map:

The user’s x, y position is initially set to (20, 20), in the middle of the first tile. The functionality we desired was for the physical position of the joystick to translate directly into the user’s velocity. Since both axes of the joystick were coupled with the two motor shafts, reading the encoder values from both motors would allow us to determine the 2-axis displacement of the joystick.

To interface with the encoders, we use 3 C header files used in the ME327 Haptics class (TimerOne.h, PinChangeInt,h, and Encoder.h). Using these external libraries allows us to abstract away the process of parsing data from the digital encoders, giving us the angular displacement of both motor shafts in degrees. These raw encoder values easily translated into the joystick’s physical displacement, which is then converted into x and y velocities. If no collisions occur, the virtual (x, y) position of the user change accordingly.

 

For more detail: Blind Maze Navigation using 2-DOF Haptic Joystick