While the field of robotics is continuously expanding at a remarkable rate and better performing robots are created every year, robotics still remains out of reach for many students and researchers. The main reasons for this difficulty are the high complexity of the hardware and software of robots, and their typically high cost. We believe that the computing power, sensing capabilities and intuitive programming interfaces of modern smartphones afford an inexpensive yet highly capable robotic platform. Smartphone based robots are becoming increasingly popular, with many exciting applications emerging in both academia and industry. As a case in point, we provide a detailed description of a simple robotic platform based on this approach.
I. INTRODUCTION
A. Existing Robotic Platforms for Education and Research
A large number of robotic platforms are available for research and education (see Table 1 for a non-exhaustive list). Robotic platforms such as the Lego Mindstorms EV3 , Mobsya Thymio II , IRobot Create , VEX, TETRIX, Surveyor (SRV-1) and Bioloid (see rows 1-6 of Table 1) are simple and inexpensive enough to be used in education and robotic research. These platforms come in kits or preassembled and can be used by educators, and students to program behaviors. However, these platforms have difficulties working outdoors on uneven terrain, and usually do not have powerful onboard computers and a large suite of sensors. Robots such as the Khepera, Koala and Pioneer (see rows 7-10 of Table 1) are very popular in the research community. While they are more capable than the above platforms, the base models are also more expensive and additional equipment, such as onboard computers, cameras and sensors, drastically increase the total cost. Humanoid robots such as the NAO and the DARwIn-OP are also becoming more affordable and they are now used for research and education. Kits provided by competitions, such as the FIRST Robotics Competition and other robotic platforms provide enough modularity and flexibility to be used for education and research although they do not use onboard computers, sensors and cameras of the same caliber as recent smartphones, although more recent robots like the Lego Mindstorms EV3 and the Thymio II allow a certain degree of interaction with smartphones.
| Robotic platform | Price |
|---|---|
| IRobot Create | $130 |
| Thymio II | $190 |
| VEX Robotics (VEX IQ; VEX) | $250; $400 |
| Lego Mindstorms EV3 | $350 |
| Robotics (Bioloid; DARwIn-OP) | $350; $12,000 |
| TETRIX | $380 |
| Surveyor (SRV-1) | $495 |
| K-Team Corporation (K-Junior; Kilobot; Khepera; Koala) | $800; $1,200 (for 10); $3,200; $8,400 |
| Adept MobileRobots (AmigoBot; Pioneer DX; Pioneer AT) | $1,695; $4,000; $6,495 |
| Scout (Dr Robot) | $8,750 |
| Aldebaran Robotics (NAO) | $15,600 |
Table 1. Popular robotic platforms used for education and research. Prices are shown for the base models/kits.
A new and exciting alternative to these platforms is to build a robotic platform with a smartphone acting as an onboard computer, as well as a sensing and interacting device. The computational power of handheld devices, such as mobile phones and tablets, increases every year at a remarkable rate. Even though smartphones have compact form factors, they are currently equipped with powerful quad-core processors and graphical processing units, video cameras, location providers (GPS, Wi-Fi, Cell-ID), long lasting batteries, and a multitude of sensors such as acceleration and orientation sensors. They also have an impressive suite of communication options (Bluetooth, Wi-Fi, Wi-Fi Direct, 3G, 4G), are powered by small long-lasting batteries, run modern operating systems (OS), and are reasonably priced. For software development, smartphone OS’s provide a Software Development Kit (SDK) that enables programmers to readily create applications. Programmers often share their applications with the community enabling rapid prototyping and development. For these reasons, we believe that smarthphones are promising candidates for onboard computing and sensing in autonomous robots.
B. Smartphone Based Robots and Vehicles
A growing interest in having smartphones interacting with peripheral devices such as motors, servos and sensors led to the recent creation of electronic interface boards that can be purchased online or built at a small cost. These boards serve as communication bridges between Android™ smartphones and external devices. The two main boards available to the public are the IOIO ($30-$39.95) and the Arduino ADK Rev3 (€49.00 ~$65.96), although other boards exist (e.g. Amarino, Microbridge, PropBridge).
An increasing number of projects realized by hobbyists, students or teachers, which utilize these electronic boards, are available online (see Table 2). A significant number of these projects involve remote-controlled (R/C) cars, or other four wheeled based robots, controlled by Android phones via IOIO or Arduino boards. Most of them involve remote controlled functionalities, occasionally with video, sensory and location feedback to another phone or a computer.
| Robot/Project Name | Cost |
|---|---|
| Cellbots Robots Everywhere Collection of IOIO based projects IOIO based sailing boat IOIO based search and rescue robot Android based Soccer Robot Collection of Arduino based projects The Android Car ( see below) |
$30 to $300 (estimations) unknown $100 to $400 (estimations) $1200 (phone included) unknown unknown $200 to $700 (estimations) $200 |
| NASA – MIT SPHERES NASA PhoneSats MIT DragonBot GEORGIA TECH project Brain-controlled robot Visual obstacle avoidance Android Based Robotic Platform (see below) |
unknown $3500-$7000 (phone included) $1000 (phone included) unknown unknown unknown $350 |
| Romo Botiful Wheelphone Shimi Double Albert iRiver Kibot |
$149 S299 S215 (+45$ for docking station) S200 S2499 unknown $40 (+ $30/month 2 years KT) |
Table 2. Smartphone based robotic projects: hobbyists/students and/or open source (top), research (middle), commercial (bottom). Both The Android Car and the Android Based Robotic Platform were completed at the Cognitive Anteater Robotics Laboratory, University of California Irvine.
For example, a group of hobbyists (Cellbots team) developed open source platforms for Android phones that can be used to control different robotic platforms such as the IRobot Create, Lego Mindstorms, VEX Pro, and Arduino based Truckbot or Tankbot. A group of high school students also built sailboats controlled by Android phones via IOIO boards. These examples show that existing robots can be used as bases, and Android phones as onboard computers. A company called Robots Everywhere develops Open Source control software for Android based robots. A team participated to the RoboCup Search and Rescue Junior competition with an impressive little robot built using an Android phone and a IOIO (rescue robot). Interestingly, some of these projects emerge from developing countries, where the use of Smartphone based robots is attractive due to their low cost and high computational power. For example, a group in Thailand created Android based robots using IOIOs that are fast and can play soccer with ping-pong balls (Android based Soccer Robot).
Our group has built a remote controlled vehicle, named The Android Car, using a R/C car, an Android phone, a phone holder and IOIO. The vehicle can be controlled over Wi-Fi and stream video and sensory information back to a computer (see Table 2 and video below). As will be described later, we are using an upgraded version of this platform for teaching and for research in computational neuroscience. For source code, click here.
Smartphone robots and applications are becoming increasingly prevalent in research projects (see Table 2). Scientists at NASA and MIT built free-flying satellites called SPHERES that are self-contained with power, propulsion, computing and navigation equipment. These devices were tested in the International Space Station and were equipped with Android phones. More recently, NASA launched three nanosatellites in orbit around Earth called NASA PhoneSats. These satellites used smartphones as control systems and a UHF radio beacon to transmit data and images to the ground. The smartphones monitored the cameras, accelerometers, magnetometers, and gyroscopes, which were onboard the satellites. In the field of human-robot interaction, researchers at MIT built a robot companion equipped with a smartphone named DragonBot. This cloud-connected robot utilizes the smartphone for motor control, 3D animation, image streaming, data capture, and is used to study human/robot interaction and potentially help kids learn. It has five physical degrees of freedom and an animated face that can display a wide range of emotional expressions. Other smartphone based robots have been developed to help remote users to communicate with each other through the robotic interface, which utilizes facial expressions and body gestures [1]. Researchers at Georgia Tech are working on a musical robotic swarm composed of cell-phone based robots that can communicate with humans and with each other and coordinate their movement in order to explore real time algorithmic musical composition and performance [2] (musical swarm). Android phones have also been used successfully with the LEGO Mindstorms NXT for robotics/software engineering classes [3]. An Android based robot was also used for brain-computer interface (BCI) research where the brain activity of a subject controlled a robot over the internet. The robot was composed of a Lego Mindstorms NXT 2.0 and an Android phone which sent video feedback to the subjects [4]. Finally, Thomas used an HTC Android phone mounted on a iRobot Create to compute optic flow with OpenCV. The robot was able to perform visual obstacle avoidance for safe navigation.
Smartphone based robots are finding their way into commercial applications (see Table 2). For example, Romo is a small robot using an iPhone as an onboard computer. It can be trained to perform face tracking, controlled using another iOS device over Wi-Fi, or over Internet for telepresence. A simple SDK is also provided to users in order to create their own apps. Similarly, Botiful is a small telepresence robot built for Android phones. Wheelphone is also a small robot that can be used for telepresence. It can also avoid obstacles and cliffs, and find a docking station on its own to recharge its battery. The Wheel phone is compatible with both Android phones and iPhones, and the software is open source and uses ROS and OpenCV libraries. Shimi is a robotic musical companion that reacts to songs played by a smartphone when connected to it. Kibot and Albert are robotic companions that can play with children and help them learn. Although not based on an Android phone, Double is a tall telepresence robot using an iPad as a computing and interacting device. These different companion or telepresence robots are an interesting market. However, they are not modular and not really suitable for many education or research purposes.
II. ANDROID BASED ROBOTIC PLATFORM
Similar to the examples discussed above and in Table 2, our group has developed a smartphone robot platform for hobbyists, students and researchers. We believe our platform provides flexibility over other available options making it attractive to a wide range of enthusiasts. In this section and the next, we describe the components of the platform and instructions on how to construct a smartphone robot.
The robot is constructed from an Android phone, IOIO board, which is connected via Bluetooth or USB, a R/C vehicle and additional sensors and actuators. The robot takes advantage of the sensors on the phone (e.g., camera, accelerometers, GPS), as well as additional sensors external to the phone (e.g. IR sensors, Hall Effect Sensors) via the IOIO. The Android phone interacts with actuators, such as speed controllers or pan/tilt units, via the IOIO board. Our Android Robotic Platform is an inexpensive do-it-yourself (DIY) smartphone based robotic platform using off-the-shelf components and open source software libraries that could easily be built by students, hobbyists or researchers, but still perform complex computations and tasks.
Our goal was to minimize both expenses and time spent on building robots, allowing users to focus on more fundamental research and robotic problems. The platform also had to be modular and flexible enough to support different sensors and actuators that could be incorporated and relocated very easily. Furthermore, the platform had to be able to traverse a wide range of indoor and outdoor terrains.
We believe that three main off-the-shelf components can be used in order to fulfill these requirements:
1) A smartphone running the Android operating system used as onboard computers and sensing devices;
2) an electronic board (e.g. IOIO, Arduino ADK) used to interact with peripheral devices such as servos, motors and sensors not included in the phone;
3) a R/C vehicle, or inexpensive robotic base.
Due to the variability in complexity of these components, the total cost to build such a robotic platform can change, especially depending on the phone and vehicle used.
Figure 6. Diagram showing the main components of the Android based robotic platform and their interactions. The robotic platform gets sensory input from the phone’s internal sensors, as well as external sensors via the IOIO board. The Android phone sends commands to the robot’s actuators via the IOIO board. The Android phone interacts with the IOIO board through a USB cable or a Bluetooth connection.
In the following sections, we will describe a robotic platform that can be built for approximately $350 (excluding the phone). The main difference, compared with other smartphone based robots, is that our platform is more modular, and can be used outdoors on uneven terrain.
A. COMPONENTS
1) Android phone
Hardware
An important advantage of using a smartphone for an onboard computer is that the size of a robot can be kept relatively small, yet still have great features. Its cost can also be minimal since the phone itself can handle computation, sensing and battery power. Many different phones are now available on the market. Before purchasing an Android phone to be used as an onboard computer for a robot, one has to consider the uses and needs of that particular robot. A hobbyist or student may consider using an older less expensive phone. For example, the HTC Google Nexus One can be found unlocked for less than $200, and is a suitable onboard computer. This phone has a 1 GHz Qualcomm Scorpion CPU, 512MB of RAM memory, a microSD card reader (supports up to 32 GB), and a 1400 mAh Li-ion battery. It can provide a number of sensory inputs such as a capacitive touch screen, a 3-axis accelerometer, a digital compass, a satellite navigation system (aGPS), a proximity sensor, an ambient light sensor, push buttons, a trackball and a 5.0 megapixel rear camera with a LED flash. For connectivity, it includes a 3.5mm TRRS audio connector, and hardware supporting Bluetooth 2.1, micro USB 2.0, Wi-Fi IEEE 802.11b/g/n, 2G/3G networks.
A researcher may desire more features and computational power. In this case, a recent phone such as the Samsung Galaxy S3 might be considered. This phone can be found unlocked for around $400, has a 1.4 GHz quad-core Cortex-A9 CPU, 1-2GB of RAM, a microSD card reader (supports up to 64 GB), and a 2,100 mAh Li-ion battery. For sensing, it has a multi-touch capacitive touchscreen, 3 push buttons, satellite navigation systems (aGPS, GLONASS), a barometer, a gyroscope, an accelerometer, a digital compass, an 8.0 megapixel rear camera with a LED flash, and a 1.9 megapixel front camera. For connectivity, it includes a 3.5mm TRRS audio connector, and hardware supporting Bluetooth 4.0, Wi-Fi (802.11 a/b/g/n), Wi-Fi Direct, 2G/3G networks, Micro-USB, NFC, and DLNA.
Software
The Android operating system is open source and Linux-based. Programmers can develop software for Android in Java using the SDK or in native language (C/C++) using the native development kit (NDK) . It is also possible for developers to modify the Linux kernel if needed. Implementation of an Android application can be achieved using the Eclipse IDE with the Android Development Tools (ADT) plug-in. Using this SDK, the developer has easy access to different functionalities of an Android phone such as graphical interfaces, multi-threading, networking, data storage, multimedia, sensors, location provider, speech-to-text, text-to-speech, and more. Since Android phones can connect to the Internet, cloud based applications can also be used when high performance computing is needed. In the field of robotics, this feature can allow the development of cloud based robotics applications. When developing an application that is CPU-intensive but doesn’t allocate much memory, an alternative programming option is to use the Android NDK. With the NDK, a programmer can create an Android Java application that interacts with native code (C/C++) using the Java Native Interface (JNI). Programming in C/C++ on an Android platform can result in an increase of performance, but also increases complexity. The NDK also enables usage of existing C/C++ libraries.
For more detail: ANDROID BASED ROBOTICS
