Sonification: Now Hear This

Sonification is visualization's poor cousin, probably because vision is a more dominant sense than hearing.  As such I've not come across as many interesting sonification projects as visualization projects.  In my opinion most sonification projects border on the artistic rather than providing deep insight into the data they represent.

Listen up and decide for yourself.

Solarbeat
Solarbeat is a musical orrery.  It's based on the orbit's of the solar system's planets.  Each time a planet completes an orbit a chime sounds.  The inner planets chime rapidly compared with the longer orbits of the outer planets.  The pitch of each planet's chime is higher the longer its orbit is.

Baroque.me
Baroque.me by Alexander Chen explores the first Prelude from Bach's Cello Suites using string lengths to express the pitch of each note (in contrast to classical music notation).  I'm not certain whether this qualifies as sonification or visualization of audio.



Radioactive Orchestra
Radioactive Orchestra consists of 3175 radioactive isotopes.  You can listen to the radioactive decay signature of each.  The fun part comes when you mix (up to) five isotopes to form an audio loop.  You can tweak the volume, pitch, tempo and waveform of each isotope's signature to produce some radioactive beats. 

Mix your own or listen to the clip created by DJ Axel Boman (below).




1945-1998
Artist Isao Hashimoto created a video and audio clip to represent the 2053 nuclear weapons detonated during the period from 1945 to 1998.  Each month lasts one second.  For each nuclear explosion a sound is heard and a marker is displayed briefly on the map at the location of the event.  The colour of the marker and the pitch of the sound signify the country of origin of the nuclear weapon.  A running tally of the countries' detonations is displayed at the top of the clip.  There is also a subtle (at least to me) stereo effect; the left/right balance of the sound corresponds to the east/west location of the explosion.

Unfortunately, there are additional sounds played for each passing month (every second) and year.  These interfere with the main focus of the project.  "Sonic junk" perhaps?



20 Hz
20 Hz was produced using data collected by the CARISMA radio array as it observed the solar wind during a geo-magnetic storm in the Earth's upper atmosphere.  The data was collected at a frequency of 20Hz and interpreted as audio.



If you are aware of any other interesting sonification projects then please leave a comment below.

Infinite Jest Visualization

My last blog post reviewed several visualizations of narrative text.  In it I mentioned that I was working on a similar project.  Here is version 1.0 - a visualization of David Foster Wallace's amazing book Infinite Jest.



I created the visualization using D3.js.  It's essentially a scatter plot in which each scene is represented by a dot.  The scenes are arranged along the x-axis in the order they occur in the book, and are chronologically positioned along the y-axis.  You can use the interactive version if you have a "modern" browser, i.e. recent versions of Firefox, Chrome, Opera and Safari.  IE users will have to make do with the images in this post.

The main ideas I was interested in visualizing were what happened, when, and who was present.  The when question is particularly interesting due to Wallace's use of Subsidised Time.  It became clear as I was preparing the data that much of the book is set during the Year of the Depend Adult Undergarment, but I pressed on with the visualization anyway.

Mouse-over a scene's dot and a summary of the scene is displayed below the chart.  Names of the characters present in the scene are highlighted in the legend to the right of the chart.  You can also mouse-over a character's name in the legend, which will highlight those scenes in which the character makes an appearance.

Not all scenes can be pinned down to a particular date; for some only the year is known (in such cases the scene's dot is positioned at the start of the year).  Some scenes have no obvious date so they're located at the bottom of the chart; the unspecified year.

Highlighting particular characters you can sometimes pick out narrative threads, for example the conversation that takes place between Remy Marathe and Hugh/Helen Steeply on the outcropping overlooking Tucson, Arizona.

I'd also like to be able to visualize references to characters.  James ("The Mad/Sad Stork") Incandenza is referred to often but is present in only a few scenes.  So too the final scenes where Don Gately is in hospital refer to characters (and events) earlier in Don's life but those characters, e.g. Gene Fackelman and Whitey Sorkin, aren't present in the hospital scenes, and so don't appear in this visualization.  Something for version 2.0 perhaps.

I'm indebted to Dr. Keith O'Neill who kindly permitted me to use his invaluable Infinite Jest Scene-by-Scene Guide, which forms the basis of the data used in the visualization.  I also relied heavily on The Infinite Jest Wiki.

The visualization is shared using a Creative Commons license, and the source-code is available on GitHub.