I'm not a musician, but I've always loved tinkering with synthesizers, sequencers and drum machines. I worked at a music store in high school, so I got to play with all the new toys. Since then, I've bought myself a new synth-related toy every once in a while.

Music composition and tracker tools on the web have always been interesting to me. There have been a few different Silverlight drum machine/sequencer projects over the years, plus my own Silverlight Synthesizer project. This one, however, is a cut above the rest. It's far more than just a sequencer, it's a musician's collaboration tool, and it's written in a combination of Silverlight, MVC, jQuery and more.

Have you seen AudioOrchard?

AudioOrchard is awesome. However, rather than just wax poetically about it, I figured I'd talk to the author, Eric Herbrandson, about what it took to create. Here's the interview.

What is AudioOrchard, and what will it be?

There are over 62 million amateur musicians in the US, and every one will tell you it's more fun to make music with other musicians than by themselves. The problem is finding a time and place to get together. For many musicians it's difficult just to find other musicians who have similar tastes in music. AudioOrchard.com, a collaborative music recording community, is the solution these musicians need.

Here's how it works: Let's say Kermit the Frog has a great idea for a new song about rainbows. He goes to AudioOrchard.com and within seconds he's recording his song. He has nothing to download or install: he just gets started. He programs a simple drum beat and then he gets out his banjo and records the basic song.

Now he'd like to get Miss Piggy involved, but he doesn't want to invite her over because that never really goes well for him. Instead he sends an invitation to her Facebook account from within AudioOrchard. Miss Piggy checks out the song, likes what she hears and records a vocal part. Kermit and Miss Piggy are happy with their song but they'd like to add some percussion.

They both agree that Animal isn't the right style of musician for this song, so they search the AudioOrchard database for percussionists with similar musical tastes. They find a few prospects, listen to some of the songs they've recorded on the site, settle on someone they think is perfect, and send him an invitation to collaborate. When the song is finished, they use AudioOrchard to tell all their Facebook/Twitter/MySpace friends to check out their masterpiece.

AudioOrchard's goal is to be more than just another digital recording studio. AudioOrchard is a truly unique community for artists to make music together in a way that's never been possible before. Musicians can record part of a song, invite their friends to contribute to their work, and search for other musicians who may be able to contribute. They listen to music created by other artists, see how those musicians made their songs, and "remix" a song created by someone else to give it a new twist. When a song is complete, AudioOrchard is also the way for musicians to share their work with friends and music fans. AudioOrchard even provides the platform for musicians to sell their completed work.

AudioOrchard's goal is to be more than just another digital recording studio. AudioOrchard is a truly unique community for artists to make music together in a way that's never been possible before.

Tell me about yourself

I've been in software for almost 11 years. Started out doing MS Access, then vb6, to vb.net, to c#. However, for the last year or so my day job has had me doing mostly C/C++, Java (for Android), and now Objective-C (iPhone). Here's a quick little summary of some of the projects I've worked on recently…

• Lead Developer and Architect for PicksPal, an online sports picking site (i.e. gambling on sporting events with points instead of money). Also developed a successful March Madness application for PicksPal that saw over 3000 signups an hour in the days leading up to the tournament.
• Lead Architect/Developer for EA Games' "Dodgeball" Facebook application. Dodgeball was built as a marketing tool for EA's "Playground for WII" release.
• One of three Developers for Solid State Networks "Solid Publisher" product, a patching/downloader tool for video game companies. Solid Publisher is being used by companies such as Riot Games for their award winning "League of Legends" release.
• Developer for Shotzoom's "Golfshot GPS" platform. (Golfshot GPS was the 5th highest revenue-generating application in the iTunes store for 2009).

Did you have any experience with Silverlight prior to developing this?

I've been playing around with Silverlight since the 2.0 beta, however this is the first project I've really built with it. However, I did work on one other project that used WPF to build some Admin screens (for the patching/downloader project mentioned above).

About a year ago I started talking with a friend about how fun it would be to build some music software. We immediately came up with the idea of putting it online in order to add a "community" aspect to it (something no other recording tool is really doing today). I started looking at Flash and Silverlight. SL3 beta had literally just come out that week, so I was really excited to find that it had a low level audio API.

After a little comparison between Flash and Silverlight, I ended up going with Silverlight for a few reasons. One, I was a lot stronger with c# than ActionScript. Two, having the ability to spin up threads seemed pretty important for making this type of app work well (the fact that the MediaStreamSource calls back on a non-UI thread is critical). And third, the way that Flash implements mic access really doesn't support what AudioOrchard is trying to do (the raw audio bits can't be accessed on the client. They have to be sent to the server first and then back down to the client). Of course, I had no way to know for sure at the time that SL4 would handle Mic support in a better way. Luckily, that part has worked out really well for me.

When I started this project, SL3 was still in beta and there was no mic support, but I felt 90% sure that the next version of SL would. So, I had actually originally built an Active-X plug-in (coded in c++/ATL) as a stop gap measure until SL added native support. I actually had the code working and was just about to start on the oh-so-fun process of signing an Active-X control when Microsoft announced the beta of SL4 (with mic support). At that point I decided to ditch my Active-X control and wait for SL4. Thankfully, it only took a couple days to replace my Active-X implementation with working SL4 code!

Are you a musician? What is your background?

I'm primarily a drummer, but I also play guitar, bass, and some keyboards. I attended ASU as a jazz performance major and currently play regularly for everything from big bands to top 40 dance bands, to small jazz ensembles, to church music, to rock and country bands. I'm actually playing a Country Western gig this weekend. Not country, country western (think "All My Ex's Live in Texas").

Also, here's a link to a tune from a jazz/funk gig I played a few weeks ago (the video is boring, but the music's pretty good I think). http://www.youtube.com/watch?v=Rs-dBN6UZT4

What was the coolest application you've ever built?

This would currently be my hands down favorite. But it's also fun to walk into Target or Wal-Mart and see "League of Legends" on the shelf and know that I created some of those bits (albeit a very small number of them). That game uses Adobe Air for the game lobby, but the game itself is in c++.

Riot games had purchased a downloader/patcher product from the company I worked for at the time and asked if we could also built a communication layer so that their lobby and game could talk back and forth. I ended up building a little sockets based library to make that work (actually it was 2 libraries, one in c++ and one in ActionScript).

How was AudioOrchard built?

AudioOrchard is built in c# from top to bottom (except for the above mentioned Active-X control that has since been scrapped). On the back end it uses MVC, JQuery, LinqToSql, WCF, MS-Sql, and Lucene.net (used for indexing/searching the loops library). I used the ORM w/ Repositories pattern for data access. I built my own little library on top of LinqToSql that makes a few things easier and ensures that all business objects call a "validate" function before being persisted (in order to enforce business rules).

On the client/SL side, I'm consuming WCF services to communicate with the server. I also used the MVVM pattern, but ended up creating my own commanding library for it because I need some rather non-standard commands (like a sorta drag-n-drop-ish command).

All of the audio processing is done on the client side. Wave files are downloaded from the server, but then manipulated on the client. After a recording is made on the client, the wave file is uploaded back to the server. This was actually a little tricky because there is no way to stream an html post to the server in SL and the recorded wave files can get quite large. I had to build my own little utility to break the files into chunks, upload them one at a time, and reassemble them on the server (as well as do some validation on the server to make sure no one tried to use my server for porn file sharing).

I had to build my own little utility to break the [wav] files into chunks, upload them one at a time, and reassemble them on the server (as well as do some validation on the server to make sure no one tried to use my server for porn file sharing).

Oh, I also built my own little SL unit testing framework. I started out using the one from MS, but it just ran too slow (it was taking over a minute to run 100 tests). Now I can run my 300 odd unit tests in about 4 seconds.

I hired a designer (who's also a good friend) to paint all the pretty pictures for both the html side and the SL side. In both cases, he sent me raw Photoshop files and I had to turn them into Html/Xaml. On the Xaml side of that, I used the "import" functionality of Blend to import the psd layers directly into SL. I actually imported them into a separate project, did some massaging, and then copied them into my actual AudioOrchard project. I was actually pretty happy with how this part worked out.

I had been developing for a few months before I actually brought in the designer. I had used MVVM, but put all the ViewModels in a separate project. This allowed me to create a dummy UI project to build against while I was just getting started (which looked ugly as sin). Then when I got the actual UI design, I created a new SL project and used the same ViewModels that I had already created. This allowed me to actually write 90% of the UI code that I needed before I even know for sure what the UI would look like.

What was the hardest part about building AudioOrchard?

There were a few parts of this project that were pretty tricky. First, I had no previous experience with audio software. There was quite a bit of learning on the fly. I've got a good friend who does a lot of audio work (used to write effects for Fender and now does DSP stuff for General Dynamics), and he was able to point me in some good directions. But for the most part it was a lot of searching the web and good old fashioned trial and error. It's been a bit slow going at points, but very rewarding.

Second, unit testing was a real problem. As I mentioned before, the MS unit testing framework for SL is great, but just too slow. It's really built around unit testing UI elements, but for this project I really had a ton of non-UI logic that I wanted to test. I ended up creating my own testing framework that ran quite a bit faster, but that only solved part of the issue.

The second half of the problem was that there are no line numbers in the stack traces of SL exceptions. This meant that when a test failed, I would get some wonderfully helpful message like "expected 4.5 but was 3.2". In a test with multiple asserts, this was often not much help. Without a line number I would often have to set a break point and step through the code to see what exactly was failing (or add a detailed text explanation as the last parameter for each assert, but what self respecting coder wants to type? That's one step away from writing documentation!). So I ended up creating something that I'm really proud of. I created a framework that takes lambdas for the asserts (actually, they take expression trees). I then created a way to turn the expression tree back into a c# text representation of the original code if a test failed. So an assert might look something like this:

Assert.That(() => mixer.ReadNextSample() == 14);

Then, if the test fails, I parse the lambda back into a string so that in my test runner UI I have an message that says:

Test failed [mixer.ReadNextSample() != 14]
Evaluated to [142 != 14]

This gives me a ton more detail about what actually failed and means that most of the time I know exactly what's going wrong without having to set a break point or even open the failing test. I wrote a blog post about it here.

Third, trying to optimize the performance in Silverlight was pretty tricky. Now that SL assemblies can be loaded into non-SL projects, I was able to create a little console-based test harness that exercised most of the critical path code and run that through Red Gates ANTS profiler.

Now that SL assemblies can be loaded into non-SL projects, I was able to create a little console-based test harness that exercised most of the critical path code and run that through Red Gates ANTS profiler.

Fourth, trying to find enough time to build the dang thing using my nights and weekends was pretty tricky. It was almost a year ago that I started research on the feasibility of the project, and over nine months since I started actively working on it. Like most programmers, I've always had some side project that I'm working on, but this has been the largest scale project I've ever undertaken.

What was the most rewarding part of this project?

It's been really fun to have a project where I'm not beholden to any sort of timeline. It's been great to have the freedom to rework each piece until I'm really happy with it, instead of having to say "good enough" all the time in order to meet a deadline. I'm really happy with how the project has turned out so far. It's also been fun to work on a project I believe in. Having a project that combines my love of music with my love of code has been really exciting.

What's next for you?

This is really only the very beginning of AudioOrchard. I've been working on it so long, that I really wanted to release something as soon as SL4 was live. But there's still a lot to do. The immediate next step is to fix any bugs that come up as people start using it. After that, I really want to get the community side of things up to snuff. Coming very soon will be the ability to listen to mp3 versions of everyone's tunes without having to open them in the editor. Also, the ability to create a "remix" of any song on the site (i.e. create your own copy of the song and change it around).

Once that side of things is a little stronger, it's back to the editor itself to add a lot of missing functionality. Effects (reverb, delay, chorus, compression, etc), pitch and tempo shifting of loops, more control over where loops are placed on the time line, and some EQing are all on the project plan. I'd really like to make AudioOrchard into a sort of "notepad" for song creation. In other words, I'd love to see musicians put their basic song ideas on the site and then work with others to flesh them out into a completed piece of music.

I'd really like to make AudioOrchard into a sort of "notepad" for song creation. In other words, I'd love to see musicians put their basic song ideas on the site and then work with others to flesh them out into a completed piece of music.

After that, I'd really like to bring in some of the "google docs" ideas. I think it would be great to actually work together in real time with others. Not real time jamming (networks just aren't fast enough yet), but incorporation a chat board and actually seeing the volume of a track change in real time as someone else tweaks with it.

Finally, the ultimate goal is to actually allow musicians to sell their completed work directly from the site. I'd love to create a really raw indie music scene where small groups (think high school bands) would have a platform to create, market, and monetize their work. It's kinda a big dream, I know. But that's where I'd like to get it to.

In fact, I'm really hoping to raise some funding so I can turn this into a full time project. I think the potential is pretty big. So, if you know of anyone with about $200k to burn… um, I mean invest… let me know.

Thanks!

Thanks for the great interview, Eric. I wish you luck in taking this to the next level. If what we've seen so far is any indicator, you'll definitely get this one in the bag.

You can try out AudioOrchard here. Be sure to save anything cool!