HTML5 audio visualizations
For a while now I've been playing around with music visualization, you can see some of my previous endeavours here. The canvas element gives us everything we need to draw all sorts of cool stuff in the browser but the audio part still needs Flash to be any fun. While the new HTML5 audio element does allow for audio playback, it doesn't give us much in terms of audio data information - which is crucial if the visualizations are to react to the music. In another experiment of mine I tried to solve it by doing a pre-analysis of the MP3 file and saving the data as JavaScript. The result was decent, I think, but having to pre-bake the data is a pretty severe limitation.
However, a few people have been hacking away at Firefox, attempting to provide just that functionality for the audio element. Some documentation of the new API is available here and if you're interested, you can follow the development in this bugzilla thread.
Anyway, I've been wanting to mess around with this and I figured that retro-fitting my "Pocket Full of Canvas" experiment to use HTML5 audio would be a fairly easy task. You can check out the result here.
At the moment you'll need a patched version of Firefox to get access to these extras but if you're not comfortable with building your own version, there are builds available here. It will still run in other (canvas-enabled) browsers but it will fall back to using SoundManager2 for the audio. While Flash audio is not that exciting anymore, it lets it run in eg. Chromium and Opera - and man, that's something else. Even if Firefox/Minefield has the upper hand with the audio stuff, it is feeling reaaally sluggish compared to those two (and you should definitely be using a nightly Minefield build). Chromium has always been speedy and Opera has gotten really crazy fast as well, running the visualizations in full-screen is just great (click the viz area to toggle).
And if you feel like playing around with the demos yourself, just click the "Show code" button and start hacking. I made a quick write-up of the very simple API here. The emphasis has been on making it easy to get from zero to something and not so much on making a lot of fancy functionality, so it is not very advanced.
Try it out: http://www.nihilogic.dk/labs/pocket_full_of_html5/
More examples:
For a while now I've been playing around with music visualization, you can see some of my previous endeavours here. The canvas element gives us everything we need to draw all sorts of cool stuff in the browser but the audio part still needs Flash to be any fun. While the new HTML5 audio element does allow for audio playback, it doesn't give us much in terms of audio data information - which is crucial if the visualizations are to react to the music. In another experiment of mine I tried to solve it by doing a pre-analysis of the MP3 file and saving the data as JavaScript. The result was decent, I think, but having to pre-bake the data is a pretty severe limitation.
However, a few people have been hacking away at Firefox, attempting to provide just that functionality for the audio element. Some documentation of the new API is available here and if you're interested, you can follow the development in this bugzilla thread.
Anyway, I've been wanting to mess around with this and I figured that retro-fitting my "Pocket Full of Canvas" experiment to use HTML5 audio would be a fairly easy task. You can check out the result here.
At the moment you'll need a patched version of Firefox to get access to these extras but if you're not comfortable with building your own version, there are builds available here. It will still run in other (canvas-enabled) browsers but it will fall back to using SoundManager2 for the audio. While Flash audio is not that exciting anymore, it lets it run in eg. Chromium and Opera - and man, that's something else. Even if Firefox/Minefield has the upper hand with the audio stuff, it is feeling reaaally sluggish compared to those two (and you should definitely be using a nightly Minefield build). Chromium has always been speedy and Opera has gotten really crazy fast as well, running the visualizations in full-screen is just great (click the viz area to toggle).
And if you feel like playing around with the demos yourself, just click the "Show code" button and start hacking. I made a quick write-up of the very simple API here. The emphasis has been on making it easy to get from zero to something and not so much on making a lot of fancy functionality, so it is not very advanced.
Try it out: http://www.nihilogic.dk/labs/pocket_full_of_html5/
More examples:
Too bad there's no Linux x64 build available :(
April 15, 2010 1:57 PM Bebenhmmm...what is this, i am not understand, just looking :)
April 20, 2010 2:29 PM general fuzzI'm seriously impressed that you were able to build in HTML 5. Good work! I clearly need to investigate its potential . . . .
April 27, 2010 8:25 AM Jacob SeidelinGreat, glad you like it :-) And thanks for the tunes, they are most delicious.
April 27, 2010 12:00 PM ScottSomehow I missed seeing this earlier, great stuff; glad to see you're looking at the Audio data API patch for Firefox. I think it has a lot of potential, hopefully it can get in and adopted etc. It's a huge piece missing from the current Audio API spec (even if only at v1.0 itself), in my opinion.
April 27, 2010 3:01 PM Jacob SeidelinRe: SM2, you may be able to get higher framerates if you flip on useHighPerformance and useFastPolling = true. (It doesn't look like either are on right now.) The useHighPerformance affects flash movie on-screen positioning (fixed vs. absolute, long story short, fixed = on-screen = better performance) - useFastPolling means a smaller interval within Flash for increased rate of whileplaying() calls, thus faster UI redraw.
Also FWIW, comment #3 re: "Epic" data is pure spam; identical posts all over the web elsewhere. ;)
@Scott: Yes, I know about those settings but they (or at least one of them..) were giving me trouble with Firefox not correctly loading SM2 unless I turned them off (or turned Flash debugging on). Thanks though!
April 28, 2010 12:24 AM SchillHm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!
April 29, 2010 5:31 PM ScottJS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)
Hm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!
April 29, 2010 5:38 PM www.richonet.comJS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)
Thanks for your content
May 12, 2010 11:02 PM CenaThanks,
www.oracledba.in
www.richonet.com
Hi, I am giving this party, and I have got a beamer. I thought it would be nice to show some nice visualizations on the big screen.(And show track names.) Is there such a program?
June 9, 2010 1:58 PM KampeAnd no window media player, since I have had some bad experiences with in the past.
Thanks
Amazing work man, keep it up!
June 13, 2010 2:04 AM XeallexxHey, Can you make this unreliant on the song thats being played so you could play your own music to it? I'd truly enjoy that. Another thing i would like to ask is if you could convert it to an embed-able format to use anywhere.
June 17, 2010 5:22 PM HavuzOther than that...
Wow! I really enjoy all projects posted on this blog and this one just blows my mind, thanks for all your effort!
thanks it is for me ı take it.
July 7, 2010 4:57 AM borgThis is very interesting game...........I would like to visit once in a week....... I would like to play it again.......
July 7, 2010 7:05 AM Fotoğrafçıonline games
thanks you for this wonderfull game. ı am playing it twice a day.
July 8, 2010 2:25 AM Havuz Kimyasallarıthanks.
July 8, 2010 2:26 AM Perukthanks you. ı am triying to work with asp.net are they ant good think for me.
July 12, 2010 2:03 AM Social Security Number LookupI have tried this new HTML 5 and it is very powerful even the iphone 4 is using the same software. So keep exploring more on this software and let us all know about it.
July 28, 2010 10:24 PM Ted MeyerHi, I've been combing the web for about 6 hours now, looking for an HTML5 audio visualizer. this is the closest i got, but i cant use it on my own site. if theres a way to download, please let me know, if not, are there any others that are available for export to my website? thanks.
September 16, 2010 7:00 PM Fenerbahçe Beşiktaş Maçını İzlethanks very much Fenerbahçe Beşiktaş Maçını İzle
September 17, 2010 4:25 AM BrownThe canvas element gives they everything necessary to draw all sorts of cool stuff in the browser, but the audio continues to flash a little funny. While the new HTML5 audio element allows audio playback, It does not give they much in terms of audio data - crucial if the visualisations are reacting to the music.
November 9, 2010 4:22 AM GrayFree Poker Money
Woah, I'm surprised this article doesn't have more relevant comments. This is great :/.
January 2, 2011 4:50 AM knutI'm trying to do something similar in AS3, but ran into the issue of getting the framebuffer of a playing mp3 hosted on another domain.
I want to visualize bass strokes. I'm not even close to know how to extract the bass level from the frameBuffer property on the MozAudioAvailable event. Can you please point me in the right direction? What are the numbers in the frameBuffer array? Which of them has to do with bass?
January 5, 2011 1:22 PM Kosmösvery cool stuff - by the way, I am using the latest build of Firefox 3 (3.6.13) and the visualizations run without issue. It appears to me you're looping images at a variable framerate tied to events in the music, correct? From the size of the initial file transfer, it would appear this is the case?
January 27, 2011 8:05 PM Gaurav MWOW!!
March 4, 2011 3:57 AM mishaThanks for sharing an amazing stuff ..cool..
March 24, 2011 10:32 AM Gabriel Guerreirovinyl siding installation virginia
try:
December 24, 2011 7:24 AM(function() {
return function() {
quality(1);
decay(0.05);
zoom(.95)
move(0.025,0);
drawPath([[0,.2],[0.05,0.215],[0.05,0.785],[0,0.8]], false, "hsl(" + (((time * 10) % 360) >> 0) + ",100%,15%)");
for(var i = 0; i < 11; i++) drawRect(.005 * i, 0.5 - soundData.waveDataL[i * 12] * .5, .005, soundData.waveDataL[i * 12], "#FFF");
commit();
};
})();