Music, visual narratives & creatures

Visual narratives are multisensory invitations.

I’m interested in a biological synaesthesia. How to create stories about insects, trees, mosses, mushrooms, that are not just a bunch of static images for eyes alone, but also blend with textures, movements, sounds — to convey something about their expressive and dynamic lives?

One way into this is through music. There is close affinity between sounds and visual narratives, eg. as discussed for comics here and here. Sequences of images can have a rhythmic quality, and music can be visualised in many ways. Can a sound be a comic? Maybe. Sounds can certainly tell a story. Looking at visual narratives broadly, nature documentaries often use sound to create interesting illusions. Sound effects are used as if they were the real sounds of insects munching (hugely exaggerated!), or to present a seamless harmony of a sneaky melody with a sneaky lion that slowly approaches their prey. It’s tempting to see nature that way. But it’s also important to remember that they are constructs.

I’ve started recording insects and spiders around the house, and made music for them. We might not understand each other very well but we do live together — to a degree, we might even process the same indoor sounds. These are synthesizer experiments; tiny stories through silence & sound, harmony & mismatch. Music and creatures were recorded indoors: both are part of the home environment.

Adventure of a jumping spider in the kitchen and on me.

It’s not so much about finding rhythms and sounds that are a perfect harmonious match with the images. It’s more about capturing something about their expressive mystery. A way to ‘give voice’ without speech and with plenty of imperfection. After all, what seems a neat harmonious match from a human perspective, how could I possibly know if it’s anything close to a harmony from a spider perspective? The point is not to claim to understand them; I can only hint at their experiences through my own expressions. In these micro compositions there is plenty of silence and mismatch.

a weak bumblebee consumes a drop of honey, but only barely.


Sound to give ‘voice’, video to give ‘face’. And they are rather cute, no?

These sound/image narratives only begin to approach an answer to nature sound illusions. How could we present an honesty about nature constructs, a self-awareness that takes responsibility for the fantasy*?

*Philosopher Timothy Morton discusses this issue in his book Ecology without Nature.

Note: I briefly touched upon biological synaesthesia through the paintings and notes of Hilma af Klint, whose observations of ‘moist moss’ and ‘cool trees’ led to colourful works of figurative shapes (plants, shells, flowers) in combination with abstract geometries. Music provides another angle for creating a sensory & visual language to express something about creatures and humans alike.

Comics, scientific illustration & sleeping insects

What do scientific articles have in common with comics?

Well, they can both use arrangements of images to convey ideas.

Abstract comics expert Andrei Molotiu inspired me to keep an eye out for their overlapping approaches. He gave compelling examples of how “natural, physical phenomena can be structured by human activity to seem to echo, or be echoed by, abstract comics. For example, visual records of various scientific processes are usually arranged for publication in sequential art form”.

This is not only true for abstract comics — abstract forms that morph, meet, grow outward or shrink over time; it also happens in more figurative depictions. Image arrangements can suggest the movement of time, or can represent a kind of mosaic that gives an overview of different situations.

In two lovely studies about sleeping behaviours of bees and ants I learned that not only do insects sleep, they may also dream.

Source  link

Source link

A sequence that shows how an active bee (A) becomes a sleeping bee (D): time moves on as you ‘read the panels’.

Source  link

Source link

A ‘mosaic’ that shows different states of ant activities: awake and asleep. A contrast-and-compare puzzle. Note the differences in antennae and mouth. A dozing fire ant queen sleeps with her mouth open and antennae slightly raised; a queen in deep sleep has her mouth closed and antennae folded inward. When her antennae start to quiver in deep sleep, it may be a sign that she’s dreaming — kind of like rapid eye movements in larger dreaming animals, such as humans.

Knowing this, insects become ever-more mysterious yet relatable at the same time. These types of studies blow my mind, and also provide good starting points for how to convey aspects about these little lives through visual narratives. Yep, there is a comic in the making here.