Home       About Me       Contact       Downloads       Part 48    Part 50   

Part 49: Sound

February 16, 2012

I started the week working on character animation for a small model I had put together. I decided I didn't know enough to do that right, and ended up buying a book: Game Engine Architecture by Jason Gregory. It tries to cover the entire range of game engine issues, and on many topics, it's just a summary with (badly formatted) links. It does have a pretty extensive chapter on character animation. But the more I read, the more I realized it was a huge project.

I looked around for something else to do for this part, and settled on Sound. The final result of this is just some new code for the project, but to have some fun with it, I also made a short movie. Skip to that below if you get bored with the technical stuff.

Ogg and Vorbis

During my previous attempt at this project in 2005, I had worked through the spec for Windows DirectSound and written a test program. Back then, I wasn't worried about distributing a demo, so I had just used uncompressed Wav files. This time, I wanted longer pieces of music and so I needed a codec. I originally thought I'd use MP3, but some discussion on the net convinced me to use Ogg Vorbis instead. There's a reference implementation and supposedly no license issues.

Ogg turns out to be the container format, which can handle a variety of streaming content. Vorbis is the audio codec.

With the Vorbis library, there's a sample decoder that reads the Ogg file, decompresses the audio with Vorbis and writes it to a file. There's also "VorbisFile" layer that reads an Ogg file without exposing any of the underlying API. Both of these produce output buffers of arbitrary length, determined by the size of the compressed packets.

This was a problem for my DirectSound test program -- it wants me to fill a fixed size buffer. Yes, I could have coded around that, but the guts of the Vorbis file layer are really complicated, since it supports seeking to the middle of the file and re-syncing the audio. Even the simpler decoder example was a problem -- I need to understand it well enough to unwind the loop, and read more audio data when DirectSound needed it, not just run through the entire file and convert it in one pass.

I spent a couple of days reading the Ogg and Vorbis documentation and puzzling over the example code before I got my version to work. Ogg supports "pages" which contain "packets", although packets can apparently span pages as well. I also found the names of routines a bit confusing. Would you call ogg_sync_pageout, followed by ogg_stream_pagein to start a new page?

Once you work through the docs, it's really simple (simpler than their example, actually.) For reference, you structure your code like this:

  • Read and process the Vorbis header packets from the file.
  • Start a new page.
Write a "readPacket" routine:
  • Try to read a packet. If this succeeds, you are done.
  • If it fails with "need more data", read a page.
  • If this fails with "need more data", read more of the source file and push it to Ogg.
  • Go back to the top and read a packet.

Once you have this routine, decoding the file is simple:

  • Read a packet.
  • Decode it with Vorbis.
  • Copy wave data to your audio buffer.
  • Repeat until you have filled the buffer.
If Vorbis returns more data than you want, you can leave some of it in the codec. This makes it easy for me to just fill the fixed-size audio buffer I use with DirectSound. After about 20 attempts, music played from my own demo. Whee!

OpenAL

DirectSound is Windows-only, and I wanted to support other platforms. OpenAL was recommended by a reader back early in this blog, and seemed to do everything I needed, so I went with that.

OpenAL builds with CMake, and everything I've done so far under Windows uses Visual C++ project files. I really didn't want anyone grabbing the code to have to mix different methods of building the libraries, so I rebuilt the OpenAL library with Visual C++.

That involved some messing with configuration files, and one mystery problem with initialization. I cannot figure out how the original code initializes itself when used as a static library, not a DLL. I had to fix that, by bashing it with a rock.

Unlike DirectSound, which uses Windows Events, the OpenAL API requires you to poll the various sounds to see if they need more data. When I saw that, I imagined some spin loop running in its own thread, burning CPU, but of course that's not needed here. For each sound, the example sets up three buffers -- one playing, one ready to play, and a third being loaded. By making these hold 1 second of audio each, I have plenty to time to load new data even if the polling loop only checks every half second.

Once that worked, I wrote a cover for the whole mess -- Ogg, Vorbis and OpenAL -- to just support attaching Ogg files to objects and moving them around in three dimensions. I packaged it all up as a single audio library, to go with the Jpeg lib and my multiplatform "mgFramework" library.

A Short Rant

I know this is all open source and we should be grateful for what we get, even if the documentation is a bit terse and the code a bit dense. But these guys aren't working on a hobby project. This is intended to be a reference implementation of a standard. And when that's your goal, make the damn code compile without errors!

Both Vorbis and OpenAL spit out type conversion warning messages -- "conversion from 'double' to 'float', possible loss of data" etc. The Vorbis code may have been written assuming that library functions like "cos" and "floor" return float instead of double. When I replaced them with "cosf" and "floorf", a lot of errors were fixed.

That can't be the only explanation though. There are plenty of explicit uses of the "double" type, and the code mixes float and double constants (0.5f, and 0.5) freely. It just seems a bit confused over which floating point type it's using. I assume that's due to multiple authors, but it's also trivial to fix. It's not like this is the first release!

The code also consistently rounds numbers by adding 0.5 and truncating to integer, without an "(int)" cast, and it assigns pointers without regard to type (void* to char*). That doesn't bother the compiler, although the IDE underlines them all as errors. And I suppose the code could be really, really old (before C++?), but using "new" and "this" as variable names really throws me when reading the code.

There's even less excuse for this in OpenAL. It actually defines its own type names for everything and has conversion routines. Why would you write something like this and not explicitly cast the result to shut up the compiler?

ALfloat Conv_ALfloat_ALint (ALint val) 
{
  return val * (1.0/2147483647.0); 
}

I went through the Vorbis code and fixed all the warnings, and some of these variable names. I didn't fix all the pointer assignments because they don't produce error messages (and they are everywhere.) I haven't tackled the OpenAL code yet. This is really not the sort of thing I should spend time on...

A Movie

To have some fun with all this code, I decided to make a very short movie. A search for music found the piece I was looking for in a Wikipedia topic. As with all the other background music I've used, YouTube flags it but doesn't ban the video.

Next, I needed some sound effects. I found a cheesy laser weapon sound (free!) at AudioMicro, but was stumped for the flying saucer. Google just pulled up a lot of "spaceship whoosh" sounds, not the classic 1950s saucer warble. AudioMicro had an effect that sounded OK, although I needed to edit it. Unfortunately, it wasn't free. After several failed searches for an alternative, I just paid the $9 they wanted for it.

To edit my sounds, I downloaded the Audacity open source sound editor. This has done everything I needed, although that hasn't been much so far. The resulting movie is here. Trivial as this movie is, it takes a bit of implementation. Watch it first, then read the list of steps in the construction.

The movie program is implemented as a state machine, with each state transitioning to the next once time or other conditions are satisfied.

  • Look at the stars for a few seconds while the music plays.

  • Create some saucers and let them pass by for a few seconds.

  • Create the "broken" saucer offscreen and let it drift to a stop in view.

  • Create the first "killer" saucer.

  • Wait a bit, then create the second killer saucer.

  • Let the first killer come to a stop, then start his attack.

  • Let the second killer stop, then start his attack.

  • Attacks continue until broken ship is red hot.

  • End the beams and wait while the broken ship fades.

  • Killers start moving again.

  • Pan towards Earth.

  • Play background music to end.

None of this is complicated, but the timing does take some fussing to get right, as do the camera angles, grouping of ships, etc. I found that after working on the video a bit, I lost the ability to tell if I was presenting things too quickly. I had to come back to it after a day and look at it again.

The Linux Port

I had used the OpenAL audio package because it ran on all three platforms, so I expected the port to Linux to be no problem. And the library did compile and link without any issues. The SaucerMovie program ran, and I saw all my animation. But no sound!

Looking at the diagnostics, OpenAL was reporting that there were no input or output devices. When CMake builds the makefiles, it includes the drivers it thinks are supported on the system. It turns out that it doesn't add drivers unless you have the development libraries for them installed (Naturally, since it can't compile the drivers without the header files.) Googling, it looked like I wanted "Pulse Audio" and perhaps "ASLA". I installed both of those, rebuilt OpenAL and tried again.

This time I got sound, and life was great -- for 20 seconds. Then the program hung and had to be killed. Sigh. The same web page that told me I needed PulseAudio also complained that it could be unstable with OpenAL and Ubuntu, and I thought "not this again!"

I noticed however that the sound was cutting out at the same point every time -- right after the saucers stop firing and the "laser" sound is deleted. So I thought perhaps OpenAL wasn't thread safe, or that I was messing up in my background thread where I keep feeding buffers of audio to the system. After hours of walking through that code, I found nothing.

Finally, I noticed something odd -- the dead saucer was disappearing. This is the step after the sounds end, indicating the demo was getting a bit farther than I had assumed. I put in some more diagnostics, and sure enough, it was hanging at the point where it creates the huge fleet of saucers just offscreen. I thought perhaps I was creating too many sounds and overloading something. That would be a nuisance.

More tracing revealed that the whole thing was a bug in my code. When I create the fleet of saucers, I just do it brute force. It has a loop where it creates a saucer, gives it a random position, then checks to make sure that position is not on top of another saucer. That loop was never exiting -- as if it could never find a good position for the new saucer! The positioning code looks like this:

saucer->m_origin.x = -150+300*rand()/(double) RAND_MAX;
I couldn't see anything wrong with this, and it works fine under Windows. It turns out that on Linux, RAND_MAX is the max integer value (2,147,483,647). On Windows, it's 32,767. When I wrote this code, I parenthesized it wrong. What I have there is equivalent to

saucer->m_origin.x = -150+ (300*rand()) / (double) RAND_MAX;
and when you multiply a max integer value (such as rand can return) by 300, you can get an overflow. That meant most of my saucers were on top of one another, and the loop couldn't find a good position for new ones. On Windows, this would never happen, because 300*32767 is still in range. Isn't debugging fun?

Changing this to

saucer->m_origin.x = -150+ 300*(rand() / (double) RAND_MAX);
fixed the whole problem, and my SaucerMovie demo worked fine under Linux. In fact, when I searched my code, this same rand expression was all over the place, so I replaced them all with a routine to return a random double.

Installation Issues

I was still perplexed about what to do with the OpenAL build though. I didn't really want to tell people to install two development libraries, rebuild OpenAL and relink the app to run it! I didn't see how I could distribute my own binary version of OpenAL though, since it needs to be customized to each system it runs on.

I finally realized that OpenAL was probably an installable package under Linux, and that I should just link to the dynamic library in my code, picking up whatever version is present on the system. And sure enough, if you install "libopenal1" under Ubuntu, that all works fine. It's kind of a nuisance that it isn't installed by default though. Does anyone know what game developers for Linux are supposed to use?

On the Mac, OpenAL is their default sound support, and everything worked right out of the box. I also did a system update and discovered I now have OpenGL 3.2 on my MacBook Pro! Yay! So you might have nicer, faster graphics on your system. The graphics processor in my MacBook Pro is the same as the one in new models of the Air and Mini. Hopefully, OpenGL 3.2 works across the line now.

So I now have a sound demo on all three platforms. Let me know if you have any problems. You can play with the options if you want to change the sounds. It should play any ogg files you have lying around.

For Windows, download SaucerMovie Part 49 - Windows.

For Linux, download SaucerMovie Part 49 - Linux.

For Mac, download SaucerMovie Part 49 - Mac.

Hit "M" to start the movie animation.

Home       About Me       Contact       Downloads       Part 48    Part 50   

blog comments powered by Disqus