Summary of CMUcam3: Working Module But Not Working CMUcam3-Arduino System
### Summary The article details the development of a CMUcam3-Arduino system featuring facial-recognition-driven motors. It operates in two modes: "interaction mode," where the camera tracks humans to control DC motors and LED matrices, and "solar tracking mode," where LDR sensors adjust the surface toward sunlight. The author implemented Viola-Jones face detection to calculate coordinates for maintaining focus on targets, overcoming initial communication hurdles between modules to achieve a functional smart surface.
Parts used in the CMUcam3-Arduino System:
- CMUcam3 camera
- Arduino Mega
- Pololu motor controller
- DC motors
- LDR light sensors
- LED matrix
- Peggy2 board (mentioned as previously fixed)
After over spending nearly 20 hours extra on fixing peggy2 board I mentioned in the previous post, there is only a few hours left for me to work on CMUcam3-Arduino system and its facial-recognition driven motor system.
The basic programming architecture is shown below.
When in “interaction mode”, CMUcam3 camera dictates the only input information of the surface when there are people around. Based on different information provided by the camera, programming stored in Arduino Mega decides and sends corresponding commands to motor controllers that I mentioned in the post about Pololu motor controller. Each motor controller that controls different parts of the surface such as arm, shoulder, head, ect. then executes the commands they received.
What we get in the end is the resulting mechanical motions carried out by DC motors. Besides, CMUcam3 camera can also provide related visual feed that controls color mixing of the LED matrix through Arduino Mega.
While there is no people around, the surface is in “solar tracking mode”, which means LDR light sensors dictates the command over entire system. LDR senses change in sunlight intensity. Similar to CMUcam3, it provides information to Arduino Mega, which controls the motion of the surface. That’s how the surface can be “heliotropic”. Based on the architecture shown below, it seems as long as each module is working and be able to establish proper communications with other modules, it will be a piece of cake to have a “function” and “smart” surface. While, things always work as it should in theories. However, that’s not the case in reality.
Facial detection is implemented by a algorithm based on on the well known paper “Robust Real-Time Face Detection” by P. Viola and M. Jones from 2004. However, facial detection is not same as facial tracking. The first thing that the surface must accomplish under “interaction mode” is being able to “recognize and focus” on a person. If it keeps staring at a red ball on the ground, there is no way that the human-machine interaction can begin. After several hours of trying to understand lines and lines of codes written, I managed to gain access to the coordinate of the sub-frame of the face detected by the camera relative to the whole frame of the picture, as shown below. This breakthrough means I can now set a threshold around the center of the frame. If the coordinate of the sub-frame that contains the face of a person is at outside of that threshold, CMUcam3 can send command to Arduino Mega, which would tell DC motors to move the body so it can always “facing” the target person in front of it.
For more detail: CMUcam3: Working Module But Not Working CMUcam3-Arduino System
- How does the system behave when no people are present?
The surface enters solar tracking mode where LDR light sensors dictate commands based on sunlight intensity. - What algorithm is used for facial detection?
The system uses an algorithm based on the 2004 paper Robust Real-Time Face Detection by P. Viola and M. Jones. - Can the system distinguish between a person and a red ball?
No, without proper tracking logic, it might stare at a red ball instead of recognizing a human for interaction. - What information does the Arduino Mega receive from the CMUcam3?
The camera provides coordinates of the detected face sub-frame relative to the whole picture frame. - How does the system maintain focus on a target person?
It sets a threshold around the center; if the face coordinate is outside this, motors move the body to face the target. - What controls the color mixing of the LED matrix?
The CMUcam3 provides visual feed data that the Arduino Mega uses to control the LED matrix colors. - Which component executes the commands sent by the Arduino Mega?
Motor controllers execute commands received from the Arduino to drive different parts like the arm, shoulder, and head. - What happens if each module fails to communicate properly?
The system will not function correctly despite theoretical designs suggesting it should work easily.
