VisionMIDI Live
Any scene, turned into music
An AI-powered system that uses YOLOv11 object detection technology to convert any video feed into music in real time. Detected objects are dynamically mapped to notes on a configurable virtual piano overlay, with intelligent tracking that maintains consistent musical output as objects move through the scene.
VisionMIDI Live is not available to buy or download yet, and there is no demo recording of it running. What follows is a description of the system as built. Get in touch if you want to see it or know when it ships.
How it works
YOLOv11 object detection
Uses cutting-edge YOLOv11 technology for real-time object detection. Identifies and tracks multiple objects simultaneously across diverse scene types — urban, natural, or staged.
Dynamic musical translation
Detected objects are mapped to notes on a configurable virtual piano overlay. Position, size, and movement all influence the musical output, creating compositions that reflect the spatial dynamics of the scene.
Intelligent tracking
Objects are tracked persistently across frames, ensuring consistent musical output even as they move, overlap, or temporarily leave the frame. No jarring note drops or resets.
Universal scene compatibility
Works with any scene — traffic patterns, bird flight, wildlife movement, dance performances, pedestrian flow, or abstract visual compositions. If it moves, it makes music.
Technical specs
Use cases
- Urban sound installations that turn traffic and pedestrian movement into music
- Natural rhythm capture — bird flight paths, wildlife movement, water flow
- Live performance installations where dancers or performers generate the soundtrack
- Generative art and music projects using found footage or live streams
- Theatre and stage productions with dynamic, scene-responsive musical scoring
Interested in VisionMIDI Live?
We'd love to walk you through the system, answer questions, or discuss how it fits your workflow.