Help us crowd-source our drums!

My students have been working hard this summer on a project I’ve talked about before. Here’s the gist:

• Normal drums aren’t melodic. They have resonant frequencies but they aren’t in a pattern that we think sounds good. That’s why they’re used for percussion.
• If you explore different shapes you can move the resonant frequencies quite a bit. We’re interested in finding cool shapes that might sound melodic.
• If we can find a shape using simulation, we can then print it using a 3D printer

That sounds cool and all, but the details are proving to be tough. I’d like to brag a little about what we’ve been up to in this post (mostly so there’s a good record of it somewhere), but if you’re wondering about the title of this post, just go here where there’s a little explanation for what we need for you. Read on for more details.

Calculating resonant frequencies

This is actually the easy part. If you know the shape of the drum head you’re interested in (and can describe it mathematically — see below for that hassle) you just need a single command in Mathematica:

{frequencies, functions}=NDEigensystem[{-Laplacian[f[x, y], {x,y}],
    DirichletCondition[f[x,y], True]}, {f}, {x,y} \Element region, {10}]

where “region” is your mathematical description of the shape of the drum head. This command uses a Finite Element approach and returns the 10 lowest eigen frequencies. Note that you have to take the square root of the frequencies you get from this command to get the audio frequencies.

Here’s a sample of listening to various frequencies on a slowly changing shape:

Describing shapes

Simple shapes are easy: a circle? Disk[], a rectangle? ImplicitRegion[-1<=x<=1 && -2<=y<=2, {x,y}]. But what about crazy shapes? And what about shapes that Mathematica can programmatically shift around while it hunts for cool shapes that produce cool spectra?

What we’ve decided to do is to use control points around the edge that Mathematica can make slight adjustments to. When it does, it redraws a smooth, closed curve that includes all the points and it then uses a cool command that turns that border into a region:

region = BoundaryMeshRegion[controlpoints, Line[{1,2,3,4,6,1}]]

The problem is that you have to make sure that the control points are in the right order around the border (say, clockwise, for example). Luckily it turns out that the traveling salesperson problem comes to the rescue here. If you want to find the shortest path visiting all the points in a plane (and returning to the first one), that path will not cross itself and hence will be a proper region border. So:

fst = FindShortestTour[points];

comes to the rescue. So Mathematica does this:

1. takes some random points in the  plane
2. Find the shortest path around them
3. Use BoundaryMeshRegion to make that border a region
4. calculate the frequencies for that shape
5. decide what small adjustments to make (see below).
6. Make those adjustments to the locations of the control points (first described in (1) above)
7. repeat 2 – 7 until a cool shape is found

Decide on adjustments

Ok, so let’s say you have six control points. Each one is an x and y value so you have a 12-dimensional optimization problem. What could we use? We’ve decided to use Mathematica’s implementation of an evolutionary algorithm (or genetic algorithm). Really it’s the same thing I was using when trying to see if Mathematica could learn to race around corners. Evolutionary approaches work well where there’s a humongous parameter space and you don’t really know any other way to explore it other than brute force.

The big problem (yes, I’m getting to the title of this post, hold your horses) is that a set of frequencies from a drum head (the result of step 4 above) needs to be converted to a single number that can be used to rank various drum heads in the evolutionary algorithm.

Single number

Ok, so we realized that we needed to be able to look at a spectrum from a drum head and rate it on the scale of “is it melodic?” We thought of some interesting approaches. Mostly they centered around measuring how close the frequency spectrum is to an evenly spaced one (which is what a stringed instrument gives you). We ran into lots of potential problems, though, not least was that orchestra chimes have a “missing fundamental” and still sound good.

We also realized that maybe we could handle mostly evenly spaced frequencies if we could determine where to thump the drum head to kill the offending non-evenly-spaced ones.

Thump predictions

Ok, so now we had to go back to Mathematica to determine where on a particular drum head you could thump it to control the relative amplitudes of the various frequencies (think about how a stringed instrument sounds very different depending on where you hit it.

Here’s an example of how the frequencies from the shape of Minnesota change their relative amplitudes if you thump in the center of every county in Minnesota (note that the find shortest tour command was used to do that):

Luckily the NDEigensystem gives us the resonant shapes for every resonant frequency so finding the relative amplitude for a given thump location (and shape) really just amounts to doing this integral:

where is just the ith resonant shape and thump(x,y) is the function that describes the thump shape (and location).

It’s taken us a while to find a good way to do this integral fast, but we’re getting there (right now we’re at one second per frequency per shape).

So now we can look for a good candidate of frequencies and then hope there’s a thump location that’ll shut off the bad ones (fingers crossed!).

Back to single number (Neural networks and you!)

So then we hit on the way we could pull all of this together (we hope). We’ve decided to let the crowd (you!) help us rate a collection of frequencies and relative amplitudes on a scale of 0 – 5 where 0 is like white noise and 5 is a pure tone. We figured that since we’re making drums for people we ought to let people determine the single number that our evolutionary algorithm needs.

One of the researchers in the math department this summer is working on an artificial neural network to recognize handwriting and my students realized that approach could work here. All we need is to train the network on what are good, bad, and medium sounding collections of frequencies and relative amplitudes.

Luckily Mathematica has recently built in some really powerful functions that implement the major algorithms in neural network theory. The one we’re planning on using is “Predict” which just needs a whole bunch of these:

{{216, 456, 786, 890, 1012}, {0.5, 0.3, 0.6, 0.7, 1}}->2

where the first list of numbers is the random frequencies and the second is the relative amplitudes. It then trains on whatever you give it and then it can be used on future untrained ones.

So, we need your help! Please go to our new site and score a few random sounds on our 5 point scale (decimals are welcome). It just takes 1 second per sound and we’d love to just get a ton to train the neural network. Then our workflow will look like this:

1. set a generation of random control points
2. find the region for all of them (using FindShortestPath)
3. find the frequencies for all of them
4. Check them against the neural network to determine goodness
5. Make babies with better ones (evolution)
6. monkey with the thumping (yeah, I know, this part isn’t as clear).

HTML 5 sounds

We started developing the training set using Mathematica to generate sounds. This is pretty easy (just use the Play command) but it was tedius and we weren’t generating enough. This notion of crowdsourcing came from my wonderful students so I decided to give it a try over this holiday weekend.

I knew making a database-driven website wouldn’t be a problem (I rail against Blackboard so much because I finally just wrote my own LMS). But I didn’t know how to generate the sounds. So, I decided to dig into the HTML5 audio standards. It turns out that just a few lines of javascript code will generate a sound with a controllable frequency and amplitude:

oscillator$key = context.createOscillator();
 gainNode$key = context.createGain();
 oscillator$key.frequency.value = $value;
 currentTime = context.currentTime;
 oscillator$key.connect(gainNode$key); // Connect sound source 2 to gain node 2
 gainNode$key.connect(context.destination); // Connect gain node 2 to output
 gainNode$key.gain.value = $amps[$key];
 oscillator$key.start(currentTime);
 oscillator$key.stop(currentTime + 1);

where $key is set up as the loop variable (goes from 1 to 5). Feel free to take a look at the html source of our page to see how it all goes together.

So thanks for any help you can give. We really hope we get enough data so that the training is robust.

Thoughts? Here are some starters for you:

• I love this! Here’s something you could do in addition . . .
• This is dumb. All of this has been done before and you’re just repeating what someone else did (insert reference here).
• Why do you do all of this in Mathematica?
• Are you going to sell these drums?
• How are you mounting them to keep the tension constant in the crazy shapes?
• Why don’t you just build tympani?
• What are you talking about with the “missing fundamental” for orchestra chimes?
• The web page doesn’t work for me, what’s the deal?
• Can I have access to the training set?
• Wait, you wrote your own LMS? That sounds cool, tell us about it!
• I’m still confused how you’re going to do the thumping in the evolutionary program.