The Software That Listens to Trees (And Taught Me I Was Building It for the Wrong Reason)
MIDI For Trees started as a technical challenge. It turned into a tool for listening — and it doesn't care that trees aren't its only subject.
When I started building MIDI For Trees, I thought I was building software that gives trees a voice. That was the pitch I told myself, the narrative I was working from. It took finishing the project to realise I'd been wrong about what I was making.
How it works
The system uses computer vision to detect coloured regions in video footage — the specific greens of leaves in sunlight, the browns of dead leaves, the particular tones that distinguish tree elements from background. Once it's found them, it tracks their movement across multiple scales simultaneously.
That movement data feeds into two musical layers:
- An Overall Movement Vector that maps the tree's primary sway to notes — the big, slow gestures of a trunk bending in the wind
- Individual vectors for each detected colour region, creating layered compositions where different parts of the tree produce different musical lines
Expression through data
Beyond note generation, the software produces control data from motion patterns. Direction, density, energy distribution — all of it can drive musical parameters like volume, filter cutoff, and effects depth. The result is dynamic, responsive soundscapes that breathe with the movement they're watching.
This is where things got interesting. The expressiveness wasn't something I designed in. It emerged from the data. Trees move in ways that are rhythmically complex but never random — they respond to wind with a kind of structured unpredictability that turns out to be musically rich.
The unexpected discovery
Here's what I didn't expect: the software doesn't actually care about trees.
It tracks moving colour. That's what it does at a fundamental level. Point it at storm clouds, flags blowing in the wind, lava lamps, traffic — it doesn't matter. Anything that moves and has colour becomes a musical instrument.
I built what I thought was a tree-listening tool. What I actually built was a system for making music with anything that won't sit still.
What that means
MIDI For Trees isn't really about trees. It's about the principle that movement contains music — and that genuine collaboration emerges when neither participant maintains full control. The wind doesn't know it's composing. The software doesn't know it's listening. But the result is something neither could produce alone.
That's the version I should have been building all along. Not a tool that gives trees a voice, but a tool that lets us hear what was always there.