System Architecture

Ljómi uses advanced computer vision technology to detect and analyze aurora characteristics, translating these into musical elements through a sophisticated processing pipeline:

Video Processing

The system begins by capturing aurora footage through either a camera (for live performances) or pre-recorded video files. The Video Processor serves as the central orchestrator, efficiently managing frame acquisition and coordinating all analysis components.

Colour & Pattern Analysis

Ljómi identifies the three primary aurora colours - green, pink, and purple - analysing their:

• Intensity: The brightness and prominence of each colour

• Movement: Direction and speed of motion across the frame

• Wave patterns: Coherence, wavelength, and organisation of aurora movements

• Area coverage: Percentage of the frame covered by each colour

Musical Translation

These visual characteristics are transformed into musical elements through:

• Note generation: Based on brightness regions within each colour

• Chord complexity: Evolving from simple triads (low brightness) to rich ninth chords (high brightness)

• Control signals: Wave direction, speed, coherence, and other parameters mapped to MIDI CC values

• Musical scales: Extensive selection from traditional to world music scales

Key Configurable Parameters

Ljómi offers extensive customisation through its configuration system:

• Video Settings

  • Input source: Camera or video file

  • Processing resolution: From 240p (very fast) to HD (high detail)

  • Frame skip rate: Process every Nth frame for performance optimisation

  • Display options: Configurable visualisation overlays

• Color Detection

  • HSV ranges: Fine-tune detection for each aurora colour

  • Brightness thresholds: Minimum brightness for detection

  • Area requirements: Minimum size for meaningful detection

• Musical Configuration

  • Scale selection: From traditional (major/minor) to exotic (Hungarian minor, Japanese, Persian)

  • Root note and octave: Base musical foundation

  • Note generation mode: Single notes or chord progressions

  • MIDI channel assignment: Per-colour channel routing

• Wave Analysis

  • Sensitivity settings: Control response to subtle movements

  • Coherence thresholds: Detect level of wave organisation

  • Direction stability: Smooth transitions between movement patterns

Performance Optimization

Ljómi employs several strategies to ensure smooth operation:

• Intelligent Processing

  • Resolution scaling: Separate processing and display resolutions

  • Frame skipping: Process only key frames for natural musical transitions

  • Value deduplication: Prevent MIDI message flooding