Artists: Borut Savski, Francisco Tomsich
Title: Elegija - soundtracks for a silent movie
Dates: 10th to 21st July 2020 at Cirkulacija 2 Ljubljana Slovenia
Production: Cirkulacija 2 & Borut Savski
Web: https://anticlimacus.wordpress.com/2020/07/05/11616/

Generative algorithmic composition excerpts based on realtime video data for Francisco Tomsich "ELEGIJA | ELEGY" video installation from 10th to 21st July 2020 at Cirkulacija 2 Ljubljana Slovenia.

Borut Savski: About the generative sound approach in Francisco Tomsich - Elegia project

For the project (that is silent) - but could generate sound from the video dynamics - i used an approach that has so-far managed to generate likeable (-> I like...) music-like structures (and some years before that: life-like mechanical movements – in some of my moving structures / sculptures). This dynamics generator (called: \"analogic\"; not synonymous to \"analog\"!) is based on idea that two sorts of sensors (data inputs) are needed that register outside activities and produce two different flows of pulses. These pulses have time-related occurences and are counted from 1 to some predefined number – and then reset to beginning. So: a kind of cyclical behaviour. Only when ther two cycles coincide on the last pulse of the two cycles – a decision-making pulse is output that triggers behaviour logic.

In this way we get rid of simplistic reactive linear relationship (every pulseis sacred...) and introduce a varying timely actions.

Analogic generator is an idea from 2008 and is presented here:http://www.3via.org/records/index.php?opt=item&id=41

The next part is the preparation of behavioral part on the level of music-like logic. In 2019 I started some studies in artificial intelligence principles – especially the ones that are not reductionist – but instead multiplicative (they generate). Clearly, this is not the aim of today's AI. However there is an area of stochastic approaches – made especially for this purpose (in the "seventies" -> when chaos was not order).

Music is easy: it has timely intervals, tonal intervals, harmonic relations, repetitions.

 

If outside of tonal control – we are in atonal domain. We are totally happy with this! Some additional dynamics had to be done with timely and tonaly intervals (the sequences) and repetitions. This was done by limited randomization ("select random from 1 and 3") and adding this to tonal result. In this same way also the repetitive nature (!) of music was approached. The final result is never-repeating, relatively pleasing – undoubtfully musical / arpeggio-like structures (which are reset when new trigger comes). Most of this work was done in 2019.

There is a parallel line of my similar work with MIDI data, where there is much more control over the parameters. In both occasions there is never an overall (AI) evaluation of what is happening. We are it.

The sounds used in Elegia are two independent sine generators in Puredata – with deliberate harmonic (octave) transpositions to lower and higher ends of spectrum of both – with medium frequencies eleminated totally. This outputs an organ-like sound. The interactions in the sound field between both generators produce occasional very noticable low frequency demodulations – again a superb thing. The sound system was set up to produce the 4-channel sound – but with no crossfading between the channels yet.

 

The software for video analysis was OpenCV with some Python programming, for the sound Puredata was used. There was a quite narrow communication pipeline between the two (only one set of data per frame - or Puredata crashes). Raspberry PI 4 was much too slow for this job (the frame analysis was the bottleneck). Occasional video hickups didn\'t bother too much. Some additional visual effects glitches were added to further affect the data for the sound.