Objectives
At the end of the course, students should be familiar with the following topics:

- Basics of analog and digital sound theory
- Operation of field recording equipment and microphones
- Use of multitrack sound software, including editing, mixing, use of effects
- Introductory sound programming, including sound synthesis and sample manipulation
- Sound production for relevant topics such as podcasts, film, games, artistic installations
- Develop and complete a substantial independent project

Students can access the (password-protected) course website at:
http://daveblack.org/HfK/WS2011/

The first was the Sparkasse Bremen Bank’s first prize for Nature and Technology, awarded for FoleySonic. The second was the Grundig Business System prize for “Psst! The Speaking Library”, a concept for a silent library system that makes use of personal handheld devices to deliver information to library visitors through audio without disturbing others.

At the award ceremony demonstrating FoleySonic:

Smart, young inventors (Pfiffige junge Erfinder) – Weser Kurier, October 27, 2010

CAMPUSideen – Bremer Magazine, December 2010

Download

In cooperation with the Syzygy Group.

Quiz question pops up

Countdown timer

Correct answer

Game music excerpts

Introduction

“Full” loop

“Light” loop

“Minimalist” loop

Game ends

Image courtesy Man-ucommons

Now available in the iTunes store!

Also check out the quote of the day (in Portuguese) at the app website.

Sounds include:

“Introduction Animation”

“Search”

“Random Quote”

“My Quotes”

“Favorites List”

“Positive Click”

“Negative Click”

“Home”

“Configuration”

“About”

Some screenshots:

Pedestrians pass by this installation and trigger sounds in real time. The movements of the visitors shape a familiar melody. An exploratory space is created, which visitors can use to try out different means of interaction with the system. Motion detection reacts to dancing, jumping, or standing still.

More information at Design In Situ. For sound samples, contact me.

Images copyright Design In Situ

Similar to a movie’s soundtrack, the music score of a computer game must provide an immersive and captivating experience to the user. Hence, the score needs to adapt and react to the gameplay in real time. Many software tools have been developed to support music composition for interactive computer applications such as games. The most advanced of these tools rely on graph-based representations of a composition using MIDI scores. The graph is formed from nodes, which represent MIDI files, and transitions between them, which are governed by conditions. This describes the music to be synthesized depending on the state of the game and the history of states. Some of these tools even come with an integrated API (Application Programming Interface) and a ready-to-use game sound engine to facilitate game soundtrack production. However, none of these tools focuses on the “long tail” market of low-end games for mobile devices or browser-based games, especially those written in Adobe Flash. Such a technically limited environment requires more specific tools, such as the one we demonstrate here.

Some of the compositions generated for this piece include:

Mellow

A Little Faster

Chill

Energy

The test application using a version of PONG with custom hockey puck controllers at the Audio Mostly 2009 Conference in Glasgow:

The exhibit from Fraunhofer MEVIS shows a physical 3D model of the brain that can be rotated. This 3D model is outfitted with blue-lit sensors that, when clicked, show on a screen the body parts that are connected to that particular nerve pathway.

The Fraunhofer MEVIS press release can be found in English and German.

Update: Video report of the exhibit from Radio Bremen

The exhibit can be seen at the Haus der Wissenschaft (House of Science) in Bremen. Check out the press release from Fraunhofer MEVIS.

Sound design for this exhibit included custom-synthesized tones for clicking each of the 10 sensors on and off, as well as rotating the device and changing the amount of rendered 3D volume of the brain shown on the screen.

Clicking sensors on the right brain side, three examples

Clicking sensors on the left brain side, three examples

Unclicking (turning off) sensors

Other interesting developed sounds include:

Rotating the base of the physical brain model (style 1)

Rotating the base of the physical brain model (style 2)

Changing the amount of rendered 3D volume of the on-screen brain model

The brain exhibit on the MS Wissenschaft on the Main river in Frankfurt:

Download

Download

Download

Automated performance at Hochschule für Künste’s Hochschultage 2011:

Download

Download

Starting from a range of requirements and use cases, this paper discusses existing techniques from such fields as music visualization and music notation. It presents a number of novel techniques, aimed at better fulfilling the needs of the human operator. To this end, the paper draws upon methods from signal processing as well as computer graphics.

Presented at the AES 128th Convention in London, England. See the AES E-Library for the publication.

However, the surgeon must frequently switch between viewing the patient and the computer screen and is often not aware of the presence of anatomical risk areas which may be located near the instrument. This thesis presents an auditory display that addresses these two limitations. The auditory display extends the image-guided surgery system by helping guide the surgeon to keep the instrument on the desired resection surface and notifying the surgeon when the instrument approaches nearby risk structures.

Results from evaluations of the auditory display with surgeons and surgical assistants confirm that the system is a beneficial addition to image-guided surgery that reduces the dependence of the surgeon on the visual display and increases awareness of risk areas.

Presented at the CARS 2010 conference in Geneva, Switzerland.

See proceedings.
Auditory Display for Liver Surgery – Thesis PDF

The game is based on the scenario of an open-air market, in which the player roams freely by actually walking around, guided by the auralization played back via headphones. We created a wireless stride- and direction-sensing device with attached manual input controls to simulate future mobile gaming devices that allow the user to explore an open space while playing a game.

Presented at the Audio Mostly 2008 conference in Piteå, Sweden.

Download the conference proceedings.

Demo at the Audio Mostly conference, 2008:

Getting the custom wireless contoller built – out of an old orange juice bottle, heavy duty telephone cord, and a huge battery/transmitter pack:

An audio-only track can be heard here:

Synopsis:
A family lived in a spacious, beautiful and generous house for generations. The generosity of the house became part of their culture that guests after others were always welcomed to enjoy a beautiful stay. Until one guest arrived with a different plan in mind.

Download Video: Closed Format: MP4

How could we grasp eternity? We are living in the past. We are making decision now. We are holding the future in hands. We are all meeting in Garden Eden.

Entering a dark cabin, the visitor puts on a pair of headphones and is transported in the dark to various atmospheres. The visitor opens six cans, each containing a specific scent. Opening the can plays back ambient mixes of atmospheric sounds and lets the visitor experience through sound and smell.

Here are some sound output examples generated in real-time by the installation:

Garden

L.A. Market

Parisian Café

Party

Sea

Church

Visitors during the SHAMBA Event explore the inside of the cabin:

Constructing the cabin inside the TAB Building at the University of Bremen:

A view of the can-controllers which contained the various scents and controlled the sound generation process:

Sample of sounds used in the production include:

Across the Seasons

Heart falling on the ground

Battleground

Sword falling on ground

Sword test

Sword being destroyed #1

Sword being destroyed #2

Sizzling sword after forging #2

We demonstrate a novel approach that harnesses the motion-sensing capabilities of readily available input devices, such as the Nintendo Wii Remote or modern smart phones, to provide intuitive and fluid arrangement of samples on a timeline. Users can watch a video while simultaneously adding sound effects, providing a near real-time workflow. The system leverages the user’s motor skills for enhanced expressiveness and provides a satisfying experience while accelerating the process.

Download Video: Closed Format: MP4

Visitors of Tangible and Embedded Interaction 2011 in Madeira, Portugal trying out the FoleySonic:

As well as at the AES Conference 2010 in London:

Previous studies show that interaction with scrollable timelines can be enhanced by an intelligent interpretation of the mouse movement. We apply similar techniques to virtual faders as used for audio control, leveraging such approaches as controllable zoom levels and pseudo-haptic interaction. Tests conducted on five such methods provide insight into how to decouple the fader from the mouse movement to improve accuracy without impairing the speed of the interaction.

Presented at the AES Convention 126 in Munich, Germany, and Haptic Audio and Interaction Design 2009 in Dresden, Germany.

See Link – AES E-Library for download.

In this paper, we demonstrate the results of user evaluations of interaction with screen elements using the surround-screen setup. Results of these tests have shown that applying surround-screen sound can enhance response times in a simple task, and that users can localize the approximate origin of a sound when played back with this technique.

Presented at the AES Convention 129 in San Francisco, CA.

See Link – AES E-Library for download.

A girl goes through a city to forget. She is throwing away her past. She got lost and she is not strong enough to live with these feelings, they are too painful for her. She goes through a city to bury her past, her memories.

Shown at X-Screen 2008 in Bremen.

Download Video: Closed Format: MP4

Presented at the AES 126th Convention in Munich, Germany, see the publication at the AES E-Library.