MOAI

MOAI is an instrument designed for Study 2, which investigated the impact of gesture size on audience perception of DMI performance, and queries whether the DMI’s physicality can influence the gestures used to play it.

Gesture is a primary area of interest in DMI research, and the community has produced a huge amount of literature on the topic. In HCI gesture is also a major area of inquiry, and a range of models for understanding how audiences perceive gesture exist. But, in these two disciplines there is no work that creates a comparative case between two physical repertoires. This work was motivated by the desire to understand this more precisely, by comparing audience perception of the same DMI at two contrasting physical sizes.

To provide a comparative case, this interface was produced in two sizes: One small, and the other 3.5 times larger. The internal hardware, the physical proportions, and the functionality were identical.

Technical details

MOAI is an acronym for Multiply Oscillating Actuated Interface. MOAI consists of three percussion elements that each sit on a strip of steel that is clamped to a table, so they bounce in response to being hit. All 3 elements are connected to a central interface that contains the computer (which continually processes the three streams of sensor data, one from each element). The control interface has a few basic system controls (mute, shutdown, volume, and potentiometer for controlling the overall sensitivity of the instrument). The percussive elements are identical in materiality, proportion, and internal hardware, and differ only in terms of scale — one version is large, with each element measuring approximately 30cm x 40cm x 20cm, and one is 3.5 times smaller, with each element measuring 12cm x 15cm x 5cm (see image below).

The percussive elements each sit on a steel slat that is clamped to a table. These steel slats are flexible, and cause the elements to bounce up and down as they are struck. This further reinforces visibility, as the elements oscillate in response to being played.

The hardware inside each MOAI element is identical, and consists of a piezo network to sense the velocity of strikes, and an accelerometer to sense the movement of the box. Inside the control interface is a Bela board that processes all six streams of analogue data, and produces the sound output in response.

The musicians: Ex Easter Island Head

This instrument was designed with input from the participating musician, Benjamin Duvall of Liverpool-based percussion ensemble Ex Easter Island Head. The instrument’s name, MOAI, is an acronym standing for Multiply Oscillating, Articulated Interface. This also references the moai, which are the giant head sculptures on Easter Island:

The band has gained notoriety in the UK music scene for their percussion performances, in which they use  open tuned electric guitars as percussion surfaces, which they strike with a variety of mallets and sticks. The tonal qualities of the ringing guitar strings give their performances a droning and meditative nature.

Design process

MOAI was inspired by B.U.R.T., an instrument I had built in my first year in collaboration with Robin Rimbaud aka Scanner. It consisted of wooden boxes that hung on springs. This collaboration was the subject of the Advanced Placement Project that took place during the first year of my PhD study, and was the first time I was able to explore how my existing interaction design skills could apply to DMI design.

I had spent a considerable amount of time on the construction and proportions of the boxes, and wondered if there was a way I could use this shape again. I attached one to a piece of steel I found in the workshop I was working in, tested it, and discovered the bounce in response to hit was a very interesting dynamic element. This general shape and behaviour was the point of departure for MOAI.

As I played it I noticed that bouncing the elements was interesting, but so was the ability to stop the bouncing. For this reason I adjusted the box design to have a rounded front in order to lend itself to grabbing with the hand, and incorporated this bouncing/grabbing into the musical interaction.

I specifically designed each MOAI element so it had only one playable surface, and that it oscillated in only one direction and at one frequency (determined by where the steel strip was clamped to the table and the length of the overhang over the table’s edge). In this way, all plurality of interaction was focused into how the player started and stopped the oscillation, and the manner in which they made their strikes.

Physical form factor and materials

Each version of MOAI consists of three identical percussion elements, and the contrast in size between the two versions was of primary importance as this would provide the point of comparison in the study. How large was ‘large’ and how small was ‘small’ was dependent on the hardware and fabrication process. For the small version, I made it as small as I could while still containing the requisite hardware. For the large version, I was constrained by the size of the laser cutter bed to cut the largest parts of the housing.

The material choices for MOAI were made by considering their function. They had to be sturdy, so I used plywood for the internal support. The grabbing surface was made by cutting \corrections{a} living hinge into the plywood, but was fragile. To support it, I created internal rounded supports out of cardboard.

Two functional aspects were mass, and that the boxes had to be able to be opened to be attached to the screws that held them to steel slats with a nut and bolt. I had to consider weight at every turn to achieve a bounce on the steel slats. Because of this, and to make sides that could be taken off and re-attached, I made the sides out of  cardboard that had been covered in lightweight black fabric.

I chose a black vinyl to cover the elements. Materials have a language of their own, and I wanted MOAI to communicate primarily through form and interaction instead of looking like a plywood `maker’ project (thereby not communicating as a musical instrument). I also wanted to avoid any communication of wood to avoid a connotation of acoustic instruments.

Sound

The core sound of the MOAI system was created from the NASA recording of the moon landing, where Neil Armstrong climbs off the space shuttle and says `That’s one small step for man, one giant leap for mankind.’ The first few seconds of this recording have a mechanical hum in the background. I was interested in isolating this hum as it was rich and brought to mind the drone qualities of Ex-Easter Island Head’s use of tuned guitars.

For this sound synthesis I used SPEAR (Sinusoidal Partial Editing Analysis and Resynthesis). SPEAR is free software that resynthesizes audio input by representing that input through many individual sinusoidal tracks, or partials. Each partial corresponds to a single sinusoidal wave with time varying frequency and amplitude. I isolated the 60Hz hum and the complementary harmonics I was interested in and synthesised a sample, then gave this sample a percussive quality with the addition of an envelope.

Each MOAI box produces three sounds:

– The base tone, described above
– The base tone with upper partials
– A sound with low partials

The high partials added to each element’s base tone were created using a recording of percussion on the upper strings of a grand piano. These partials were then added to each box’s base tone (though the partials themselves were not transposed). For the low partials, I was inspired by the richness of the bass tones of The Sinking Belle, a collaboaration by Sunn o))), Boris, and Jesse Sykes and the Sweet Hereafter. Using SPEAR I analysed this track, and I found a combination of sinusoidal partials that I liked. I then synthesised a sample by combining sinusoidal partials at the relevant frequencies (directly synthesizing from the track wasn’t possible as the partials didn’t have the amplitude, clean tone or duration I was interested in). This low tone was applied to Box 2, then transposed for boxes 1 and 3 as before.

Sensor design and programming

Inside MOAI’s control interface is a Bela unit that continually processes the analogue data from the three elements.
The hardware for the large and small versions were identical, except that a network of multiple small piezo discs were used for the small version.

MOAI plays one of three samples — base tone, tone with high partials, or tone with low partials — in response to strikes. MOAI begins playing a sample when it detects a strike on one of the elements, but it determines which sample to play based on the movement state of that box, determined through the accelerometer data.

To detect piezo strikes, the control interface constantly samples the signal generated by each element’s embedded piezo network. The piezo input is used to detect strike events, and assign a velocity to the strike. The piezo input undergoes considerable conditioning each time it is sampled (Bela’s analogue sample rate is 22.05kHz). First, it is full wave rectified and a DC blocking filter applied.

When any piezo velocity is sensed above a threshold (this threshold is applied to reject noise), Bela looks both forward and backward by 5ms of input to find the highest signal peak. This is because I found that there was no guarantee that the strike that is the first detected would be the highest, so this process was implemented to make sure the highest velocity is applied. This did method did not add any noticeable latency.

The sound that is played in response to a strike is determined by accelerometer data. The accelerometer data from each element is also processed continually, the movement state determined by this data indicates which sample will be played. The velocity input detected by the piezo network determines the velocity assigned to that sample.

The sound-producing states determined by box movement are are moving, still, or stopped suddenly. Moving is by far the most common state as the boxes bounce when hit, and continue to bounce after. An element that is moving plays the element’s base tone in response to a strike. Starting from a still position, as determined by accelerometer data, produces the element’s base tone with added low partials. Striking the box with sufficient velocity produces the element’s base tone with added high partials. The threshold of velocity needed for this tone is adjustable using a potentiometer on the control box, to allow for variations in strike velocity between performers.