Previous Work: StarSound

The previous research and work done at Cork Institute of Technology in 2012 already had data included the visualizations of live data. The data was the live feed of a radio telescope, providing a constant number (normally between about 1-10), representing the amount of hydrogen at the position it's pointed at. In the final piece, the amount hydrogen was represented by little hydrogen atoms projected on a wall, as well as a sonification of hydrogen:

There are many different ways to visualize radio telescope data. Most commonly used data sheets used by astronomers look fairly complicated and probably quite boring for the general public. Where optical telescopes can provide astonishing imagery that can simply be projected on a screen, data from the radio telescope has to be converted and shown in a way, that the user enjoys and in some way understands, what that data means. For the visualization and auralization of hydrogen, the Rydberg Formula and the corresponding Lyman Series where used. Inspiration for this came from science blog ‘New Scientist’. Visual sine waves where arranged according to the Lyman Series and are representing the frequency of hydrogen, which naturally has the frequency of 1.4 GHz. One sine wave in the series consists of hexagons being surrounded by a single dot, which represents the atomic model of hydrogen and its typical hexagonal structure. The amplitude represents the amount of hydrogen at the particular position. Additional visualizations for the projection are also based on the atomic model of hydrogen, which consists of one core and single electron flying around it. The more of those models are flying around, the more hydrogen exists at the particular position. The sound design for the incoming data is also based on the Lyman Series. The sound consists of in total ten sine waves that are arranged according to the Lyman Series. The  volume of the sine waves represents the amount of hydrogen. A dynamic band pass filter, which is added onto the main signal, also influences the sound according to the data. This series of sine waves could theoretically even be expanded, possibly to an infinite amount, just as the Lyman Series itself.

(From the Research Paper of the StarSound Project)

Also see: http://clemensanzmanncitbahons.blogspot.de/2012/03/animated-poster.html

In fact, the sonification of hydrogen played a major part in the final piece, providing an understanding of what the data represents, which is in that case, something you usually cannot see or hear. If and how sonification can play a supportive part in other data representation, will be further examined during the research phase.

Data & Sound

The use of sound to transport information from a computer or any other device to a user is called Auditory Display. The most famous and only widely used example for the sonification of data is the Geiger Counter invented in 1908. The international community for Auditory Display ICAD still researches how to perceptualize data in an acoustic way. Sound however, will probably always play a subordinate role when it comes to the presentation of data.

Since interesting large data sets are easily available lately, very interesting and artistic projects came up that used these data sets to create music or sound spheres to experience the data in different and new ways. 
Related projects are for instance projects on LiveScience, where data from blue-green algae is transposed into musical notes, or songs are created from data from the Hubble telescope or the Higgs Boson.

Sea Songs, by Biologist Peter Larsen

by composer, physicist and engineer Domenico Vicinanza

Though this type of project does not really create a new practical way of how to work with data, they sure provide interesting cause for thoughts of how to combine visuals and sound to create new audio visual interfaces and installations to work, experience and present massive data sets.

One project that already walks this path is the AlloSphere at the University of Colifornia, Santa Barbara. Standing on a bridge inside a massive cube, the users can interact and work with different sorts of data sets, being totally immersed into the data on the visual and the audio layer. 

This large-scale immersive laboratory has a scientific and an artistic component:

Scientifically, it is an instrument for gaining insight and developing bodily intuition about environments into which the body cannot venture: abstract, higher-dimensional information spaces, the worlds of the very small or very large, and the realms of the very fast or very slow, in fields ranging from nanotechnology to theoretical physics, from proteomics to cosmology, from neurophysiology to the spaces of consciousness, and from new materials to new media.

Artistically, the AlloSphere is an instrument for the creation and performance of avant-garde new works and the development of entirely new modes and genres of expression and forms of immersion-based entertainment, fusing future art, architecture, music, media, games, cinema, and more.

- AlloSphere Website

Through combination of art and sciences, new ways are discovered of how to present different data in the most practical but also the most comfortable way, to make it easy and fun to work with. Different approaches are made of how to include sounds, so that sound would not be there as some sort of additional 'ear-candy', but actually increase and improve the workflow with the data. Dr. Joann Kuchera-Morin, working on the AlloSphere explains the sound design the following:

I take visual data, drop it 27 octaves and do the proper transformations so I can hear it. Think of a mathematical equation that can be seen and heard. If you take any math data and sonify and visualize it, you’ll be able to see and hear patterns — you’ll see balance, continuity, contrast, surprise, all the things that catch people’s attention.

- Dr. Joann Kuchera Morin on TED Talk 2009 

Just as the AlloSphere already did in some ways, the research being conducted this term will search for new ways to use immersive sound to support the work with data in a not too artistic, but scientific way.

References

Allosphere.ucsb.edu (2001) The AlloSphere at the California NanoSystems Institute, UC Santa Barbara. [online] Available at: http://www.allosphere.ucsb.edu/index.php [Accessed: 20 Nov 2012].

Blog.ted.com (2009) 

TED Blog | Mapping terrain in space and time: Exclusive interview with JoAnn Kuchera-Morin of the AlloSphere. [online] Available at: http://blog.ted.com/2009/04/15/allosphere_interview/#more [Accessed: 20 Nov 2012].

Icad.org (2012) International Community for Auditory Display. [online] Available at: http://icad.org/ [Accessed: 20 Nov 2012].

Livescience.com (1997) Sea Songs – Microbe Data Sound Like Music | Video | LiveScience. [online] Available at: http://www.livescience.com/23622-sea-songs-microbe-data-sound-like-music-video.html [Accessed: 20 Nov 2012].

Livescience.com (1997) Here's What the Higgs Boson Sounds Like | 'God' Particle Music | LiveScience. [online] Available at: http://www.livescience.com/21521-higgs-boson-music.html [Accessed: 20 Nov 2012].

Sonification.de (2010) sonification.de. [online] Available at: http://sonification.de/ [Accessed: 20 Nov 2012].

Data Visuals and 3D

Most Data Visualizations and Representations are in two dimensional pictures or maps, such as graphs and data charts, being static or interactive. This type of visual data is what people are used to and usually prefer to work with. But still, interactive data representations are still largely evolving, especially in their design, structure and interactivity.

David McCandless, data journalist and information designer presented beautiful interactive 2D maps and infographs of data, showing how beautiful data representations can be a "relief" in all the information overload. His designs are as beautiful as informative, giving data meaning and putting it into interesting context.

David McCandless at TED Talk August 2010

Considering this knowledge, when you think of the next step of how to design new ways for presenting and interacting with big data, the third dimension and audio sonification might be the next evovlement for big data solutions.

Three dimensional ways of visualization scientific data is nothing completely new though, as are for instance the 3D visuals of particle accelerator results, just recently becoming more famous due to the discovery of the Higgis Bossom.  

Computer visualization of particle debris from the decay of the Higgs boson

In that case though, the visuals refer more to the actual movement of particles, than translating or interpreting numerical data in a more understanding and meaningful way. Still, it is a beautiful and stunning visualization style that could also be used for many different applications.

Some big data solution providers, such as TerraEchos, FMS and Mindtel are already using state of the art 3D technology for big data visulizations. Achieved with the augemnted reality tool zSpace, data becomes a immersive, interactive and physically touchable. "While 3D capabilities are still advancing — the potential to use these types of technologies as a way to discover valuable organizational trends is certainly within reach." - Dave Chavez, 11.09.12, WIRED Magazine 

ZSpace promotional Video

References

Sadowa.tumblr.com (2012) The Reviewer Library (A computer visualization depicts one array of...). [online] Available at: http://sadowa.tumblr.com/post/26485355751/a-computer-visualization-depicts-one-array-of [Accessed: 14 Nov 2012].

Ted.com (2010) David McCandless: The beauty of data visualization | Video on TED.com. [online] Available at: http://www.ted.com/talks/david_mccandless_the_beauty_of_data_visualization.html [Accessed: 29 Nov 2012].

Wired.com (2012) Is 3D Visualization the Next Step For Big Data? | Innovation Insights | Wired.com. [online] Available at: http://www.wired.com/insights/2012/11/3d-visualization-big-data/ [Accessed: 14 Nov 2012].

Zspace.com (2012) zSpace - revolutionary virtual-holographic computing. [online] Available at: http://zspace.com/ [Accessed: 14 Nov 2012].

First Ideas / Basic Concept

The basis of the planned research for this term are big data representations. With focusing on gained knowledge from previous works, such as virtual reality and sound or user experience in immersive museum installations, new ways for presenting and experiencing massive data bases will be created and tested. Research in the large field of big data will be conducted, prototypes created and at a later stage tested through user studies. A detailed Gantt chart will follow.
The type of data used for the application obviously has a vast impact on the actual look and outcome of the final piece. There are plenty of interesting databases and live data sources on the web, that shall be examined  and worked with throughout the research phase. Possible data bases / data streams are:

• Cosm (all kinds of mostly private data live streams, such as weather, etc.)
• World Bank (massive data base of finance, health and population data as excel files)
• Google Public Data (lots of different data bases)
• Gapminder.org Brilliant Project by Hans Rosling
• Stock Market (Stock Market live stream provided by Yahoo!)

Kick Off

This Blog will document the research being made for the Master thesis at Hochschule Darmstadt. The major topic of this thesis will be (until now):

New ways for interactive and immersive Big Data presentations through new media, VR technolgies and sonification.

This topic is generally based on previous works and research done at Hochschule Darmstadt and Cork Institute of Technology, dealing with immersive sound applications for games and live data representations for a museum installation. This research and the outcome can be viewed here:
The Interactive Soundscape
Starsound
(or on my Behance Profile)