Hello,
I am trying to create a sound according to this.
It seemed simple to me, but I’m making a mistake somewhere.
For the individual lines of the image, I will assign frequencies (now for the test only by linear interpolation), for example for line 1 it is 4000 and for the last 16000.
Then I scan the image line by line where I add up the individual sinuses and save them in the sample.
The resulting sound is completely different from the original.
for i=0, length - 2 do -- length = duration * samplerate
local t = i / length -- my time
local x = floor(Lerp(0, bmp.width, t)) -- interpolation, bmp is image class
local sum = 0 -- sum for all sines in column
for y = 1, lines do -- sum image columns
local r, g, b, a = bmp:get_pixel(x, y - 1) -- get pixel color
local amplitude = ((r + g + b) / 3) / 255 -- average brightness / 255 for 0 to 1 values
sum = sum + Sine(t, amplitude, frequencyDistribution[y], 0) -- sum sines with distributed frequencies with 0 phase
end
sum = sum / lines -- average output value
new_buffer:set_sample_data(1, i + 1, sum)
end
function Sine(time, amplitude, frequency, phase)
return amplitude * sin(2 * pi * frequency * time + phase)
end
One thing wrong is the computation for “brightness”. See image - Formula to determine perceived brightness of RGB color - Stack Overflow for a pretty thorough discussion on computing various notions of brightness/luminance/value, and the underlying assumptions you may be making about the colorspace your data is in (linear vs gamma-corrected, etc.).
I am not sure if this helps, it used to be an active plugin for VCV 1.6 that processed images into cv.
The code is still available and you might find it useful.
That’s true,
but this is enough for the test.
First the problem and then everything around.
I can control the amplitude with just the blue component of the color, for example.
But good point
Thanks,
I knew I had seen that module somewhere before.
But that’s not exactly it.
if I had to push it to the extreme, then VirtualANS by Alexander Zolotov is a model, but that will never happen
I am currently developing a module that takes an image and creates control signals – gates triggers and v/oct – from one or more “read heads”. I’m not focusing on generating audio, at least not in the first release, so it’s a bit of a different beast than described here.
It won’t be available for while, but I’m making slow progress amidst a bunch of competing tasks, like a kitchen remodel, water leaks in the house and the ferry to our island is on passenger-only service. etc ;-).
When it’s in good enough shape I’ll post a new thread asking for beta testers.