Use Arduino for Projects > Projects > Sound – Audio Projects > NES Controller Piano / Ultrasonic Theremin Mash-Up

NES Controller Piano / Ultrasonic Theremin Mash-Up

Friday December 11, 2020 / Muhammad Bilal
Categories: Sound – Audio Projects
Tags: breadboard, gamepad, sensor
Contents hide
1 Step 1: Parts and Equipment *
2 Step 2: NES Controller
3 Step 3: Ultrasonic Sensor
4 Step 4: Piezo Buzzer Module
5 Step 5: NES Controller Code
6 Step 6: Ultrasonic Sensor Code
7 Step 7: “If” and “&&” Arguments
8 Step 8: Play!

I recently built a simple 8 key piano using an original NES controller and Arduino UNO board.
Unlike conventional Arduino piano projects which require one digital pin for each key / button the NES controller piano used only three digital pins (D5 for nesClock, D6 for nesLatch, D7 for nesDataIn) leaving 5 extra digital pins available for experimentation.

The NES Controller sent pulses of 8-bit information to the Arduino UNO and the pulses were interpreted by the Arduino (“bit-banging “) to make sounds via the buzzer module – Build the NES Controller piano and download the sketch then open the serial monitor to watch this process in action.

The NES controller piano only played eight different notes (C4 to C5) and my kids tired of playing it pretty quickly – I decided to increase the piano’s octave range (C1 to C8) by adding an ultrasonic sensor and create a NES Controller piano / “Ultrasonic theremin” mash-up that can play 50 notes using only 5 digital pins (+1 for piezo buzzer).

I’m still a learner when it comes to coding and welcome any feedback regarding my sketch.

Step 1: Parts and Equipment *

*I had already built my NES Controller Piano and just needed to add a breadboard, HC-SR04 Ultrasonic Sensor, and a few more male-male jumper wires.

Step 2: NES Controller

The NES Controller has 8 buttons, uses a 4021 8-Bit Shift Register IC, and only requires connection to 3 digital pins on the Arduino (as well as connections to +5V and GND)

The plug has 7 pins (5 of which are used in this instructable) – In the second picture click on the pins in the plug to find out each function.

Male jumper wires are a perfect fit for the pins on the NES Controller plug.

  1. Connect red wire to positive rail on breadboard and to 5V pin (Arduino)
  2. Connect another red wire from positive rail to +5V (pin 7) on NES Controller
  3. Connect black wire to GND on NES Controller and GND on Arduino
  4. Connect orange wire to Clock (NES pin 2) and D5
  5. Connect yellow wire to Latch (NES pin 3) and D6
  6. Connect green wire to Data (NES pin 4) to D7

Step 3: Ultrasonic Sensor

  1. Insert ultrasonic sensor into breadboard
  2. Connect a black wire from Arduino GND to ground rail
  3. Connect another black wire from ground rail to sensor Gnd
  4. Connect purple wire from D9 to Echo
  5. Connect grey wire from D10 to Trig
  6. Connect red wire from positive rail to Vcc
  7. Ultrasonic sensor code is covered in Step 6

Step 4: Piezo Buzzer Module

The connections for the piezo buzzer module is a little different in this project compared to the NES Controller Piano

  1. Connect a black wire from ground rail to – on buzzer module
  2. Connect a blue wire from D8 to S

Step 5: NES Controller Code

I based the code for my NES Controller Piano / Ultrasonic Theremin Mash-Up on my NES Controller Piano code (which itself was based on the “Teachbot gamepad controller sketch” by Gordon McComb in Arduino Robot Bonanza published by McGraw-Hill in 2013).

The NES controller sends short pulses of 8-bit information when a button is pressed* or not pressed** which the Arduino catches and then translates into tones to be played by the buzzer module (for example if bit 0 (01111111) is captured the buzzer will play the tone F4 – see bitRead commands in the code below).

The code for the NES Controller Piano is in full in the box below:

#include "pitches.h"   // include pitches library
 #define nesClock 5     // label pinout
 #define nesLatch 6
 #define nesDataIn 7
 #define buzzer 9
 byte nes = 0;         // global variable

void setup() {
  // put your setup code here, to run once:
Serial.begin (9600); // serial terminal speed 
pinMode (nesDataIn, INPUT); // set up pins
pinMode (nesClock, OUTPUT);
pinMode (nesLatch, OUTPUT);
pinMode (buzzer, OUTPUT);
}
void loop() {
  // put your main code here, to run repeatedly:
nes = nesRead ();
if (bitRead (nes, 0) == 0){ // If right button NES Controller pressed
  tone (buzzer, NOTE_F4, 250); // play tone NOTE_F4
  Serial.println ("NOTE_F4"); // NOTE_F4 displayed on Serial monitor
}
if (bitRead (nes, 1) == 0){
  tone (buzzer, NOTE_E4, 250);
  Serial.println ("NOTE_E4");
}
if (bitRead (nes, 2) == 0){
  tone (buzzer, NOTE_D4, 250);
  Serial.println ("NOTE_D4");
}
if (bitRead (nes, 3) == 0){
  tone (buzzer, NOTE_C4, 250);
  Serial.println ("NOTE_C4");
}
if (bitRead (nes, 4) == 0){
  tone (buzzer, NOTE_G4, 250);
  Serial.println ("NOTE_G4");
}
if (bitRead (nes, 5) == 0){
  tone (buzzer, NOTE_A4, 250);
  Serial.println ("NOTE_A4");
}
if (bitRead (nes, 6) == 0){
  tone (buzzer, NOTE_B4, 250);
  Serial.println ("NOTE_B4");
}
if (bitRead (nes, 7) == 0){
  tone (buzzer, NOTE_C5, 250);
  Serial.println ("NOTE_C5");
}
if (nes == 255) //Checks to see if no button pressed (255 = 11111111)
{
  noTone; //if no button pressed no tone will be played
  Serial.println ("NO_TONE"); 
}
delay (180);
}
byte nesRead (){ // "bit banging" 
  byte value = 0;
  digitalWrite (nesLatch, HIGH);
  delayMicroseconds(5);
  digitalWrite (nesLatch, LOW);
  for (int i=0; i<8; i++) {
    digitalWrite (nesClock, LOW);
    value |= digitalRead (nesDataIn) << (7 - i);
    digitalWrite (nesClock, HIGH);
  }
  return (value);
}

*Information sent when a button is pressed:

  • Right = 01111111bit 0
  • Left = 10111111 bit 1
  • Down = 11011111 bit 2
  • Up = 11101111 bit 3
  • Start = 11110111 bit 4
  • Select = 11111011 bit 5
  • B = 11111101 bit 6
  • A = 11111110 bit 7

**Information sent when no button is pressed:

  • 11111111

Step 6: Ultrasonic Sensor Code

  • I based my code for the ultrasonic sensor on Ammar Atef Ali’s excellent instructable.
  • In theexample code below if my right hand is 0-4cm from the sensor C1 is played, if my hand is 4-8cm from the sensor C2 is played and so on and so on…
#define buzzer 8
#define echo 9
#define trig 10


void setup() {

pinMode (buzzer, OUTPUT);
pinMode (echo, INPUT);
pinMode (trig, OUTPUT);
}
void loop () {int duration, distance;
digitalWrite (trig, HIGH);
delay (1);
digitalWrite (trig, LOW);
duration = pulseIn (echo, HIGH);
distance = (duration/2)/29.1;
if (distance <4 && distance >=0){ 

  tone (buzzer, NOTE_C1, 250); 
 }
if (distance <8 && distance >=4){ 
  tone (buzzer, NOTE_C2, 250); 
 }
if (distance <12 && distance >=8){ 
  tone (buzzer, NOTE_C3, 250); 
  }
if (distance <16 && distance >=12){ 
  tone (buzzer, NOTE_C4, 250); 
}
if (distance <20 && distance >=16){ 
  tone (buzzer, NOTE_C5, 250); 
}
if (distance <24 && distance >=20){ 
  tone (buzzer, NOTE_C6, 250); 
 }
if (distance >=24){ 
  tone (buzzer, NOTE_C7, 250);
}
}

Step 7: “If” and “&&” Arguments

I mashed the NES Piano Controller Sketch and my ultrasonic sensor sketch together using lots of if‘s, &&‘s, (‘s and )‘s to be able to get 50 different notes (C1-C8) out of 5 digital pins on the Arduino Uno – If anyone has any tips on how to neaten the code I would appreciate it as I am still bit of a novice. Full code is posted below: 

/* NES Controller Piano Ultrasonic Theremin Mash-Up
 * 
 *
 * 
 * ----------------------------------------------------------------
 * Requires:
 * ----------------------------------------------------------------
 *  -   1 x Arduino UNO
 *  -   1 x Active Piezo Transducer Module
 *  -   1 x HC-SR04 Ultrasonic Sensor Module
 *  -   Male-Male Jumper Wires
 *  -   Male-Female Jumper Wires
 *  -   1 x NES controller
 *  -   1 x Breadboard
 * -----------------------------------------------------------------
 * NES Controller Plug ("End On" View)
 * -----------------------------------------------------------------
 *                _________
 *               /      01 |GND
 *       +5V     |  70  02 |Clock
 *       ?       |  60  03 |Latch
 *       ?       |  50  04 |DataIn
 *               __________
 * -----------------------------------------------------------------
 * Connections:
 * -----------------------------------------------------------------
 * Arduino Uno Pinout     NES Controller 
 *                        (Use Male-Male Jumper Wires)
 * -----------------------------------------------------------------
 * +5V      ------------> Pin 7
 * D7       ------------> Pin 4
 * D5       ------------> Pin 2
 * D6       ------------> Pin 3
 * GND      ------------> Pin 1
 *------------------------------------------------------------------                        
 * Arduino UNO Pinout     Active Piezo Transducer Module 
 *                        (Use Male-Female Jumper Wires)
 * -----------------------------------------------------------------
 * D8       ------------> S
 * GND      ------------> GND
 * -----------------------------------------------------------------
 * Arduino UNO Pinout     HC-SR04
 * -----------------------------------------------------------------
 * +5V      ------------> Vcc
 * D10      ------------> Trig
 * D9       ------------> Echo
 * GND      ------------> GND
 * -----------------------------------------------------------------
 * NES Controller Button Values:
 * -----------------------------------------------------------------
 * Right  = 11111110  bit 0
 * Left   = 11111101  bit 1
 * Down   = 11111011  bit 2
 * Up     = 11110111  bit 3
 * Start  = 11101111  bit 4
 * Select = 11011111  bit 5
 * B      = 10111111  bit 6
 * A      = 01111111  bit 7
 * -----------------------------------------------------------------
 * Libraries Required: 
 * -----------------------------------------------------------------
 * "pitches.h" library (https://www.arduino.cc/en/Tutorial/toneMelody)
 * copy and paste to "New Tab" 
 * -----------------------------------------------------------------
 * Reference List:
 * -----------------------------------------------------------------
 * "Teachbot gamepad controller sketch" by Gordon McComb Arduino Robot Bonanza McGraw-Hill 2013
 * and
 * "https://www.instructables.com/id/Detecting-Obstacles-and-Warning-Arduino-Ultrasonic/" */
 #include "pitches.h"   // include pitches library
 #define nesClock 5     // label pinout
 #define nesLatch 6
 #define nesDataIn 7
 #define buzzer 8
 #define echo 9
 #define trig 10
 byte nes = 0;         // global variable
 
void setup() {
  // put your setup code here, to run once:
Serial.begin (9600); // serial terminal speed 
pinMode (nesDataIn, INPUT); // set up pins
pinMode (nesClock, OUTPUT);
pinMode (nesLatch, OUTPUT);
pinMode (buzzer, OUTPUT);
pinMode (echo, INPUT);
pinMode (trig, OUTPUT);
}</p><p>void loop (){ // put your main code here, to run repeatedly:
int duration, distance;
digitalWrite (trig, HIGH);
delay (1);
digitalWrite (trig, LOW);
duration = pulseIn (echo, HIGH);
distance = (duration/2)/29.1;
nes = nesRead ();
if ((bitRead (nes, 0) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_F1, 250); 
  Serial.println ("NOTE_F1");
}
if ((bitRead (nes, 0) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_F2, 250); 
  Serial.println ("NOTE_F2"); 
}
if ((bitRead (nes, 0) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_F3, 250); 
  Serial.println ("NOTE_F3"); 
}
if ((bitRead (nes, 0) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_F4, 250); 
  Serial.println ("NOTE_F4"); 
}
if ((bitRead (nes, 0) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_F5, 250); 
  Serial.println ("NOTE_F5"); 
}
if ((bitRead (nes, 0) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_F6, 250); 
  Serial.println ("NOTE_F6"); 
}
if ((bitRead (nes, 0) == 0) && distance >=24){ 
  tone (buzzer, NOTE_F7, 250); 
  Serial.println ("NOTE_F7"); 
}
if ((bitRead (nes, 1) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_E1, 250); 
  Serial.println ("NOTE_E1");
}
if ((bitRead (nes, 1) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_E2, 250); 
  Serial.println ("NOTE_E2"); 
}
if ((bitRead (nes, 1) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_E3, 250); 
  Serial.println ("NOTE_E3"); 
}
if ((bitRead (nes, 1) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_E4, 250); 
  Serial.println ("NOTE_E4"); 
}
if ((bitRead (nes, 1) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_E5, 250); 
  Serial.println ("NOTE_E5"); 
}
if ((bitRead (nes, 1) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_E6, 250); 
  Serial.println ("NOTE_E6"); 
}
if ((bitRead (nes, 1) == 0) && distance >=24){ 
  tone (buzzer, NOTE_E7, 250); 
  Serial.println ("NOTE_E7"); 
}
if ((bitRead (nes, 2) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_D1, 250); 
  Serial.println ("NOTE_D1");
}
if ((bitRead (nes, 2) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_D2, 250); 
  Serial.println ("NOTE_D2"); 
}
if ((bitRead (nes, 2) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_D3, 250); 
  Serial.println ("NOTE_D3"); 
}
if ((bitRead (nes, 2) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_D4, 250); 
  Serial.println ("NOTE_D4"); 
}
if ((bitRead (nes, 2) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_D5, 250); 
  Serial.println ("NOTE_D5"); 
}
if ((bitRead (nes, 2) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_D6, 250); 
  Serial.println ("NOTE_D6"); 
}
if ((bitRead (nes, 2) == 0) && distance >=24){ 
  tone (buzzer, NOTE_D7, 250);
  Serial.println ("NOTE_D7");
}
if ((bitRead (nes, 3) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_C1, 250); 
  Serial.println ("NOTE_C1");
}
if ((bitRead (nes, 3) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_C2, 250); 
  Serial.println ("NOTE_C2"); 
}
if ((bitRead (nes, 3) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_C3, 250); 
  Serial.println ("NOTE_C3"); 
}
if ((bitRead (nes, 3) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_C4, 250); 
  Serial.println ("NOTE_C4"); 
}
if ((bitRead (nes, 3) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_C5, 250); 
  Serial.println ("NOTE_C5"); 
}
if ((bitRead (nes, 3) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_C6, 250); 
  Serial.println ("NOTE_C6"); 
}
if ((bitRead (nes, 3) == 0) && distance >=24){ 
  tone (buzzer, NOTE_C7, 250);
  Serial.println ("NOTE_C7");
} 
if ((bitRead (nes, 5) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_G1, 250); 
  Serial.println ("NOTE_G1");
}
if ((bitRead (nes, 5) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_G2, 250); 
  Serial.println ("NOTE_G2"); 
}
if ((bitRead (nes, 5) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_G3, 250); 
  Serial.println ("NOTE_G3"); 
}
if ((bitRead (nes, 5) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_G4, 250); 
  Serial.println ("NOTE_G4"); 
}
if ((bitRead (nes, 5) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_G5, 250); 
  Serial.println ("NOTE_G5"); 
}
if ((bitRead (nes, 5) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_G6, 250); 
  Serial.println ("NOTE_G6"); 
}
if ((bitRead (nes, 5) == 0) && distance >=24){ 
  tone (buzzer, NOTE_G7, 250);
  Serial.println ("NOTE_G7");
}
if ((bitRead (nes, 4) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_A1, 250); 
  Serial.println ("NOTE_A1");
}
if ((bitRead (nes, 4) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_A2, 250); 
  Serial.println ("NOTE_A2"); 
}
if ((bitRead (nes, 4) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_A3, 250); 
  Serial.println ("NOTE_A3"); 
}
if ((bitRead (nes, 4) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_A4, 250); 
  Serial.println ("NOTE_A4"); 
}
if ((bitRead (nes, 4) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_A5, 250); 
  Serial.println ("NOTE_A5"); 
}
if ((bitRead (nes, 4) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_A6, 250); 
  Serial.println ("NOTE_A6"); 
}
if ((bitRead (nes, 4) == 0) && distance >=24){ 
  tone (buzzer, NOTE_A7, 250);
  Serial.println("NOTE_A7");
}
if ((bitRead (nes, 6) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_B1, 250); 
  Serial.println ("NOTE_B1");
}
if ((bitRead (nes, 6) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_B2, 250); 
  Serial.println ("NOTE_B2"); 
}
if ((bitRead (nes, 6) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_B3, 250); 
  Serial.println ("NOTE_B3"); 
}
if ((bitRead (nes, 6) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_B4, 250); 
  Serial.println ("NOTE_B4"); 
}
if ((bitRead (nes, 6) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_B5, 250); 
  Serial.println ("NOTE_B5"); 
}
if ((bitRead (nes, 6) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_B6, 250); 
  Serial.println ("NOTE_B6"); 
}
if ((bitRead (nes, 6) == 0) && distance >=24){ 
  tone (buzzer, NOTE_B7, 250);
    Serial.println ("NOTE_B7"); 
}
if ((bitRead (nes, 7) == 0) && (distance <4 && distance >=0)){ 
  tone (buzzer, NOTE_C2, 250); 
  Serial.println ("NOTE_C2");
}
if ((bitRead (nes, 7) == 0) && (distance <8 && distance >=4)){ 
  tone (buzzer, NOTE_C3, 250); 
  Serial.println ("NOTE_C3"); 
}
if ((bitRead (nes, 7) == 0) && (distance <12 && distance >=8)){ 
  tone (buzzer, NOTE_C4, 250); 
  Serial.println ("NOTE_C4"); 
}
if ((bitRead (nes, 7) == 0) && (distance <16 && distance >=12)){ 
  tone (buzzer, NOTE_C5, 250); 
  Serial.println ("NOTE_C5"); 
}
if ((bitRead (nes, 7) == 0) && (distance <20 && distance >=16)){ 
  tone (buzzer, NOTE_C6, 250); 
  Serial.println ("NOTE_C6"); 
}
if ((bitRead (nes, 7) == 0) && (distance <24 && distance >=20)){ 
  tone (buzzer, NOTE_C7, 250); 
  Serial.println ("NOTE_C7"); 
}
if ((bitRead (nes, 7) == 0) && distance >=24){ 
  tone (buzzer, NOTE_C8, 250);
    Serial.println ("NOTE_C8"); 
}
if (nes == 255) //Checks to see if no button pressed (255 = 11111111)
{
  noTone; //if no button pressed no tone will be played
  Serial.println ("NO_TONE"); 
}
delay (100);
}
byte nesRead (){ // "bit banging" 
  byte value = 0;
  digitalWrite (nesLatch, HIGH);
  delayMicroseconds(5);
  digitalWrite (nesLatch, LOW);
  for (int i=0; i<8; i++) {
    digitalWrite (nesClock, LOW);
    value |= digitalRead (nesDataIn) << (7 - i);
    digitalWrite (nesClock, HIGH);
  }
  return (value);
}

For example if (if command) I press the UP button (bitRead (nes, 3)) with my left hand and (&& command) my right hand is <4cm from the ultrasonic sensor note C1 will play:

if ((bitRead (nes, 3) == 0) && (distance <4 && distance >=0)){ 

 tone (buzzer, NOTE_C1, 250);}

However if I press the UP button and my right hand is 12-16cm away from the ultrasonic sensor note C4 will play:

if ((bitRead (nes, 3) == 0) && (distance <16 && distance >=12)){ 

 tone (buzzer, NOTE_C4, 250);

}

Step 8: Play!

Have fun!

There are still plenty of pins available for more experimentation too…

Source: NES Controller Piano / Ultrasonic Theremin Mash-Up


About The Author

Muhammad Bilal

I am a highly skilled and motivated individual with a Master's degree in Computer Science. I have extensive experience in technical writing and a deep understanding of SEO practices.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top