I feel at this point that I should state categorically that you can’t give booze to a baby no matter what your tipsy protocol eventually says.
The old Irish grandmas used to dip the finger in whiskey and rub the baby’s gums with it to quieten them down if they got too screamy.
1 Like
I was thinking the exact same thing when I saw the post! Ha!
Thinking on this, we may want ‘tipsy flavors’ of ports so you can negotiate message types.
So like
struct TipsyParticipant_V1
{
virtual ~TipsyParticipant_V1() = default;
virtual bool isInputTipsy(int) const = 0;
virtual bool isOutputTipsy(int) const = 0;
virtual std::string getInputMessageFormatForPort(int) const = 0;
virtual std::string getOutputMessageFormatForPort(int) const = 0;
}
then I could implement a port where that returned, say, application/json;lilpond and when i attach to a tipsy port i could see if it stands a chance of understanding me. Maybe.
Or perhaps a simple “supports type” bool question
This is all just to say: an interface here requires more thought than a few forum posts before my coffee
The ‘tipsy’ port format negotiation would allow for any data format (e.g. application/binary), and not just text, which is good – the simple encodings we’ve seen for 8-bit bytes of text works for arbitrary bytes of data. Could be any number of tagged formats like TIFF, Video, WAV, OSC.
If you look at OSC, it includes timing information so that precise synchronization for music can occur.
The two ends don’t necessarily need to agree on the meaning of the text. In the prototype shared earlier there is a low-level protocol the sends text chunks with a checksum, and at a higher level, it sends json text with elements to identify a module (I think the intended destination or perhaps sender and intended receiver). I didn’t spend enough time with the code to be sure but I think it also sends a “type” marker with the data. This would also simplify how a module decides whether and how to interpret the data.
1 Like
yes i think that’s right. change std::string to a buffer type everywhere (and have a utility to think of it as utf8). make the format have some two arg form like mime-type, extension-info so the squinky clipboard could go down a pipe as application/json, vcv-sequencer-clipoard, that kinda thing.
and then have good binary start and end message delimiters which are short but noticeable
and then make sure your binary encoding sort of stays within +/- 10 when mis-represented as an IEEE float (because remember cables suppress NaN and INF so you don’t get all 32 bits)
and there you kinda go.
heck. if you can put 24 bits on it, you can do almost 1.5 fps of 320 x 120 raw encoded video at 48khz down a pipe line that, if you send scanlines as quickly as you can. Or if you did 6 bit grayscale instead of 24 bit color get almost 6 fps.
And that’s the sort of truly terrible idea which kinda makes me want to have this library exist.
LOL
2 Likes
@baconpaul Ah, great detail about Lilypond. Preparing for this to allow for mime-type and thus a generalized data transfer interface is clever.
There is a small concern I have. At this point, we are effectively overloading the “cable” to just point between two ports, but not otherwise following the behavior for cables. Am I correct in thinking that this model only works when there are no non-tipsy intermediate steps between producer and consumer of data? Is there a concern that this will violate user’s expectation of how signals propagate?
I’m now wondering if there should be a notably different port UI for ports that only accept or send text. This might help users understand that this port doesn’t transmit the same kind of information.
1 Like
Well it depends how you encode. I was thinking last night whether we need the dynamic interface s Al vs just a binary decoder with a header and mime type. The. Things like delays and routers would work just fine. But this thought is still ill formed.
I may just write some prototype code and see but perhaps the only thing we need is “binary blob plus mime type to float stream decode and encode” and then really just use cables. Basically a modem but which assumes lossless digital floats
Like this?
I don’t think a negotiation protocol is necessary. Restricting what you could connect such an output to would just get in the way of doing weird things if you want to try. If it doesn’t sound good using text as CV or audio, then it just won’t be used for that. I don’t think it’s something you need to protect users from unless it tends to break speakers or damage hearing, but that can be adjusted by the encoding – 32 bits of float have a lot of headroom for representing a byte, and it’s something users have to deal with when doing modular anyway.
I am coming to the same conclusion @pachde - the only thing we need is an encoding and decoding from binary data into floats; and a start and end message protocol with a type and message version and then a library to handle both sides.
An IEEE float has plenty of room to back in 24 bits. You can see here https://en.wikipedia.org/wiki/Single-precision_floating-point_format
so you get 23 bits if you just use the fraction part but since we want values between -10 and 10 (roughy) I would propose that you put, like, 22 bits into the fraction, then one bit into the sign bit, then one bit into making the exponent either 63 or 64. And then you get a signal which is in range mostly and avoid the nan and inf indiciators. Pretty easy to write that efficiently also.
Then there’s the handshake. Since you have left a large part of IEEE float outside the 24 bit valid you are communicating, just have some magic floats which you send as start of message, start of header, end of header, end of message. Then have a header with a type and stream version. And then we can all just make data on cables. And you can just route it with switches and mutes like you would anything else.
No need for a dynamic cast or other end of the module
basically just making a “sample rate * 24 bits per second” modem protocol assuming 100% float fidelity.
See what I mean? Or would it be clearer if I wrote a bit of code?
That’s a perfect description.
I like the idea of using the control characters to mark message blocks (reference: ASCII - Wikipedia):
NULL NULL SOH TYPE<mime type> RS SLUG<transmitting module slug> RS ID__ <transmitting module id> RS LEN_<text-length> STX <text> NULL ETX ETB
Header may contain metadata, separated into chucks by RS. Each record begins with a 4-byte record type id (similar to TIFF AIFF, etc). followed by data up to next RS or STX. Text begins after STX, null-terminated, followed by ETX. Header may be empty (NULL NULL SOH STX <text> ETX).
We have all the markers we need in the ASCII portion of UTF-8. Perhaps a float value unlikely to occur in audio/CV and not a valid message block value is a good way to fence the text messages. And that ETB seems not necessary.
I think packing no more than 2 bytes in a float is convenient. No need to spread bits around – just 16 bits in the fraction. Just 1 byte per float simplifies some things, but writing the code will tell if packing more creates issues. If 2 bytes, then messages will have to be padded if they’re an odd length (zero-padding makes sense to me).
yeah agree with all of that. the only reason i thought about 3 bits was (1) we had room and (2) it is 50% more bandwidth for when someone inevitably makes an insane video codec with this
To your header I would add some version information so like SOH VERSION # TYPE<mine> so we can migrate a protocol and replace text with binary but otherwise agree completely
despite my strong desire i’m not going to write an encoder/decoder this weekend but would happily review, use, comment, collaborate if you want to. Or if you aren’t going to I might do it later in the week.
what a nifty idea.
oh and I guess we should make sure the idea still works for @StochasticTelegraph
oh and if we do it the first module to write is a ‘text scope’ of course.
1 Like
As soon as there’s a module that sends text, I’ll add it to my Info module.
I just opened the issue to submit my plugin to the library.
It might be a variant of Info or an extender geared for dynamic text display.
I imagine a full height scrolling text billboard with options for however many parallel lines you want scrolling across. For the retro roadside light sign look, I know of some fonts to make that look right.
yeah just someone has to write an encoder and decoder library
I guess “first one of us to start typing something in say something on this thread so we don’t dup work”! And if none of us ever type anything then it won’t happen.
but i think it could be neat.
I’m a little busy with the last steps of publishing my first plugin, but as soon as that’s done (and I finish splitting the logs my wife asked for), I’ll set up a new repo and scaffold a couple of modules that we can use as a vehicle for working this out. I’ll follow up here when that’s done. probably 3-4 days.
2 Likes
@pachde My concern was less about protecting users and more about reducing their confusion. I guess we’ll see how that goes.
This is a far, far better result than I expected when I started this thread!
I can get started on writing a “console” module (current name, “TTY”) and on adding text creation to BASICally. That should be sufficient to test out the encoder/decoder once someone starts writing it, yes?
Yeah. Also its raining here. Maybe I’ll take a shot at a standalone encoder/decoder lib.
Actually made some pretty good progress today. Might have something to show tomorrow. If you are interested in reading half finished GitHub pushes, PM me.
Here’s what @pachde @StochasticTelegraph and I have been up to together last day or so
there’s the code for that video’s modules.
9 Likes