Simon Sings November 2016 - December 2016
Interaction Design


As the final project for DSGN 395: Designing Product Interactions, students ideated and created a bilateral, multi-sensory interaction to demonstrate understanding of both intuitive and innovative product interactions.


My group began brainstorming by thinking of routines that could be implemented in a different way such as managing personal finances, navigating to an unfamiliar destination, and initiating conversations when networking. We also reflected on a class discussion on how as a glass of water is filled up, the pitch increases, which is an audible cue to turn off the faucet. We decided to turn this feedback into control so that the power of a user’s voice could fill a glass of water.

High level concept of Simon Sings


Our first prototype was a play on the game Simon Says. “Simon” would “teach” the user how to fill their own cup with water by playing a pitch and then filling a glass. The user would then be prompted to repeat after Simon until the glass was filled. We created a low fidelity foamcore mockup and presented our idea to the class to get feedback.

Foamcore prototype of Simon Sings

User Research

In response to the feedback we received, we decided to conduct further user research to understand the most intuitive way to control the filling of the glass. We wanted to understand if it was more intuitive to: sing a continuously ascending pitch, sing three predetermined ascending notes, or sing any three ascending notes. From our research, we found that singing a single ascending pitch was not intuitive, as users didn’t understand the mapping of their pitch to water level. Matching three tones was also difficult for some users. We finally decided to design our interaction so that any three ascending notes would fill the glass in three discrete amounts.

Final Design

Simon Sings, turning feedback into control

Simon Sings consists of two parts: Learn and Do. To begin the interaction, a user presses the big button in the center of the board. The Learn side then lights up. The speakers play a tone of a person whistling and 1/3 of the LEDs behind the bottle lights up.

LEDs lighting up to symbolize the bottle filling up

This is repeated two more times until all of the LEDs are lit up, symbolizing that the bottle has been filled. A triumphant “ta-da” sound is played to signal the end the Learn side of the interaction and to convey a sense of accomplishment. The light on the Learn side then turns off and the light on the Do side turns on so that the user knows that it is their turn.

The user presses the now lit green button on the Do side, which evokes the feeling of using an intercom system, and then whistles into the microphone. The green button was also designed to be a safety feature so that if the microphone picked up external noises, water would not unexpectedly flow. A solenoid valve then opens for a set amount of time and fills 1/3 of the bottle with water. The user repeats this process two more times, whistling a pitch that was higher than the last, to fill the bottle.

Water filling up the bottle

Again, a triumphant “ta-da” sound is played, prompting the user that they have successfully filled the bottle. The light on the Do side turns off, signaling that the entire interaction has been completed.

My Role

My primary roles were to design/implement the voice controlled part of Simon Sings and ensure that all of the electrical components were integrated correctly. To analyze the pitch the user sang, I used a Fast Fourier Transform (FFT) library on a Teensy LC microcontroller. Due to the library’s large processor and memory footprint, we chose to break of some of the interaction to a separate Arduino Uno. We also decided to center our interaction around whistling because whistling has a more harmonically pure spectrum than singing, which greatly simplified our signal processing algorithm without deviating from people’s mental models of the game Simon Says.

Troubleshooting the electronics

What Else?

We presented Simon Sings at the Fall Design Expo 2016 hosted by the Segal Design Institute at Northwestern University and had a great time explaining and demonstrating our interaction.

Presenting Simon Sings at the Fall Design Expo 2016