Music, visual narratives & creatures

Visual narratives are multisensory invitations.

I’m interested in a biological synaesthesia. How to create stories about insects, trees, mosses, mushrooms, that are not just a bunch of static images for eyes alone, but also blend with textures, movements, sounds — to convey something about their expressive and dynamic lives?

One way into this is through music. There is close affinity between sounds and visual narratives, eg. as discussed for comics here and here. Sequences of images can have a rhythmic quality, and music can be visualised in many ways. Can a sound be a comic? Maybe. Sounds can certainly tell a story. Looking at visual narratives broadly, nature documentaries often use sound to create interesting illusions. Sound effects are used as if they were the real sounds of insects munching (hugely exaggerated!), or to present a seamless harmony of a sneaky melody with a sneaky lion that slowly approaches their prey. It’s tempting to see nature that way. But it’s also important to remember that they are constructs.

I’ve started recording insects and spiders around the house, and made music for them. We might not understand each other very well but we do live together — to a degree, we might even process the same indoor sounds. These are synthesizer experiments; tiny stories through silence & sound, harmony & mismatch. Music and creatures were recorded indoors: both are part of the home environment.

Adventure of a jumping spider in the kitchen and on me.

It’s not so much about finding rhythms and sounds that are a perfect harmonious match with the images. It’s more about capturing something about their expressive mystery. A way to ‘give voice’ without speech and with plenty of imperfection. After all, what seems a neat harmonious match from a human perspective, how could I possibly know if it’s anything close to a harmony from a spider perspective? The point is not to claim to understand them; I can only hint at their experiences through my own expressions. In these micro compositions there is plenty of silence and mismatch.

a weak bumblebee consumes a drop of honey, but only barely.

Sound to give ‘voice’, video to give ‘face’. And they are rather cute, no?

These sound/image narratives only begin to approach an answer to nature sound illusions. How could we present an honesty about nature constructs, a self-awareness that takes responsibility for the fantasy*?

*Philosopher Timothy Morton discusses this issue in his book Ecology without Nature.

Note: I briefly touched upon biological synaesthesia through the paintings and notes of Hilma af Klint, whose observations of ‘moist moss’ and ‘cool trees’ led to colourful works of figurative shapes (plants, shells, flowers) in combination with abstract geometries. Music provides another angle for creating a sensory & visual language to express something about creatures and humans alike.