How does audiosurf work

This maybe a step in the right direction, but it is something that does not make any sense to me.. This tries to calculate the tempo of the song and where the beats in a measure are and hence place the obstacles the appropriate distance apart to coincide with each beat. The way that the games know when to "kick in" etc can range from being very simple and measuring the amplitude volume of the waveform or something more complex like isolating the volume of certain frequencies and measuring their volume.

If you're interested, look into Digital Signal Processing to see how you can analyse waveforms, which is essentially what these games are doing in their loading phase. EDIT: I just saw your edit regarding Fourier transforms and thought I'll add some insight into it, although I'm by no means an expert on it!

FFT is a way of calculating the actual Fourier transform of a waveform. Basically, if you load up an audio file into Audacity , you'll see the wave form with the timeline along the top, this is known as the time domain. The FFT will convert a signal from the time domain into the frequency domain basically all the frequencies that occur within the audio.

This conversion is useful for spectral analysis. In a game example, if you were to do a Fourier transform, you could easily calculate the amount of high frequency occurrences in the audio, and from that you could add twinkly visual effects, stars, or something associated with typically high frequency sounds.

For the low frequencies you could have big, gluttonous monsters moving in time to the bass sounds, etc. Here is a great seven part tutorial series on this topic by Badlogic Games. They cover everything from the basics to implementation. The data that come from the analysis oft the spectral energy variation are enough to generate this kind of maps.

Here the problem may be if there are too much data to process: not what kind of data are used, but how. The software generates data by using the spectral energy changes and try to recognise known features, then it use the information about the features to setup the map.

Recognition can be done by clusterization, maximum likelihood, neural networks, Genetic algorithm and so on. After completing the recognition, you have infomations like: where the feature if found in time and frequency , what type of featuature is found, the velocity the feature vector is moving and so on; you can use these data to feed a map generation algorimth, leaving room for improvements like making better recognition algorithms, recognize more family of features, extract more data, find new ways to "render" these data and so on.

Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Thanks for the quick answer! On question 2 I meant when I choose some song from my hard drive, will the rythm of the game synch automatically with the song beats?

I'm not really sure how to explain this lol, hope you kinda understand what I mean. Luzzifus View Profile View Posts. Just watch some gameplay videos in the community hub or on youtube. Last edited by Luzzifus ; 27 Jun, pm. Originally posted by skzk :. Edit: AAC works as well for me but I don't know where it's getting that codec from.

Last edited by skzk ; 27 Jun, pm. Per page: 15 30 I mean, Steam's the only place to buy right now, but that's not the entire story. There's no contractual requirement or anything. Your track analysis for building a game level from the music is pretty cool. Can you talk about how it grabs that information and turns it into a level? It's based on frequency analysis. The basic gist is that when the music is at its most intense, that's when you're on a really steep downward slope, like you're flying down a rollercoaster in a tunnel.

When the music is calmer, that's when you're chugging your way up the hill, watching that peak in the distance you're going to reach. And music is not all about just going uphill and downhill; lots of music has speed bumps and waves that you ride, so that's all pulled out of the song. And I guess the other important element is the traffic pattern to match with the music. The way it works is that there's a block in the highway. And whenever there's a spike in the music, the intensity of that spike determines the block's color.

So the most distinct spikes, like a snare drum, that would tend to be a red block, a really hot block. If something is a little more subtle, like a quiet high hat, that would be a purple block, which is worth less points.

Uh, yeah, that's the gist of it. Although to tell you the truth, I don't know entirely how to speak about it with other people in the field, because it's all just kind of trial and error on my part.

I don't come from a DSP background, really. You mentioned the hills on the tracks. How do you figure intensity to get those? It's primarily volume, really, the wave amplitude. But it is skewed based on the frequency analysis, depending on the song. It tries to normalize it, so that whatever song you choose, you get an interesting ride.

There's a trend in pop music to compress the volume, to make everything louder. Did that cause problems with your track analysis? I can't remember entirely. It doesn't have too much to do with the transients when it comes to the shape of the track, that's more of a smooth waveform. It's interesting you pointed out that pop music is compressed, like a lot of producers will put… they want it to be loud, and it loses so much dynamic range.

Yeah, I wish they wouldn't do that. Did you ever look at something that would be more procedural, like generating music based on the player's actions? Yes, that was actually the first thing I tried, but I didn't come up with anything that was worth pursuing further. The Chemical Brothers' Star Guitar video. I didn't see that until later in the project, but that was inspiring.

Various music visualizers, which I enjoy staring at. There was one someone released from Wild Tangent that was this spaceship flying over a terrain, and the terrain would morph and bump with the music in realtime. It kind of gave you a sense of motion with the music.

There was a game called Barcode Battler , which I've never actually played but I heard of it. The idea was that you scanned in a barcode, and every barcode was a monster. It was big in Japan, apparently, and there was one brand of suit that was sold out everywhere, because the barcode made a huge dragon. I really liked that idea, of exploring something outside the game, bringing it in, and seeing what it does. Is there a certain kind of track that you used while you were tuning the code?

I guess trying to tune it for a large variety of music really came later. At first, I was only interested in making my favorite songs into really cool tracks, and it stayed that way for quite a while. I guess what the songs I was really drawn to had really quiet parts and really loud parts, so that was the kind of stuff I was doing when I was first building it.