DanT's new adventures in TouchDesigner

[dan.tilley (DanT)]

I have seen some amazing TouchDesigner visualisations around recently, and I know others have posted about TD here, @Omri_Cohen even did a video on it some years back.

But I didn’t fully appreciate the power of TD and the world of possibilities it opens, until I actually tried using it.

To start with I just wanted to test out sending data between TD and VCV as simply as possible, so I opted to use midi. TD has midi built in, and of course so does VCV.

Others tend to recommend using OSC because it is higher resolution, but that doesn’t concern me at this early stage of learning.

I am on Windows11 so the details here are based on that platform, but for most things it should easily translate to other platforms.

Obviously you need VCV and TD installed.

You also need a midi device, I believe you can use a suitable audio interface, or plug in a usb device to get this. But I am using a virtual midi device, I decided on loopmidi because it can create multiple ports (so that you can have bi-directional messages without feedback).

Here I have setup two midi ports, one for messages sent out of VCV and one for messages sent out of TD

Within TD you have to setup the midi in and out, which is a menu found at Dialogs > MIDI Device Mapper

Create a new mapping and then set the in and out devices

I also use the fantastic showMidi to verify the midi connections are working as intended

In VCV I use the CV to MIDI CC module to send out messages, and note that the value cannot be negative for a CC

In TD to receive the messages you add a midiin CHOP, note that the value will be 0 to 127 so normally a math CHOP would be used to rescale the value to something more appropriate

Notice how the channels are named, this is important because you have to set the right name to send messages out of TD.

You can send messages out of TD using a midiout CHOP, depending on the network setup, normally you would use a rename CHOP to set the channel name that is input.

In this screenshot I am using channel 1 on both devices, but I think it might be a good idea to use a different channel just to be safe.

Back in VCV I use the MIDI CC to CV module to bring the messages in

And done… bi-directional midi between VCV and TD

TD ↔️ midi ↔️ VCV Rack

Very simple start, but I feel very creative now, got lots of ideas for visualisations for patches, and generating interesting modulation sources…

[selected (colour)]

Thanks Dan - this is a brilliant post - inspires me to get into this. Excellent!

[ale47p (Alessandro Paglia)]

I have a little bit of experience with TD combined with Rack (inside Reaper, live show, everything running together on a win laptop + an external gigantic screen)

its modular philosophy made me feel at home since day one.

I was litteraly surprised when I realized how light that software is. My teacher told me “they are just pixels”. but to me it was day/night difference from what I was experiecing with video up to that day. like the day you use Reaper for the first time. “wtf was I doing before?” same feeling. fast, snappy, reliable.

the most “strange” thing for me was its “timeline”, but there is just the need to understand it

I managed,via a script on a DAT module, to sync Reaper and TD timelines perfectly, with pauses, stops, resets and so on working flawless. if anybody wants, I can dig and find it out…

[aetrion-music]

Fantastic introduction / overview @dan.tilley. I’ve been going along exactly the same path though I am on a MacBook and have been super excited about the similarities and possibilities with the two applications. OSC works great too.

One thing I had to get my head around is some of the nomenclature (parameters, binding etc.) And the fact that usually the operators in TD are more bare bones and you’re meant to build out every functionality vs. the plethora of modules in VCV that can create complex signals in a tiny footprint.

[dan.tilley (DanT)]

Yes, exactly.

I wanted to just get a random value on a trigger and thought it made sense to have some kind of random op, but no, you have to use a noise and then a hold, and maybe a math too

which is totally fine, just have to get used to a different way of working

I suppose if I really wanted a random op then I could just make a network to do that and then hide it all inside a component… or something… havent got that far in my learning yet

[ale47p (Alessandro Paglia)]

@aetrion-music and @dan.tilley that’s true.

the key difference for us (Rack users) imho is to get used to formulas/expressions on parameters, because they stay there

[shachaf.polakow]

I also found osc and audio analysis to work well, you can route each channel from the mixer. TD is really great with almost unlimited options. If you didn’t tried media pipe, you should. It is an easy gesture recognition module made by Torin Blankensmith - https://www.youtube.com/@blankensmithing/featured

GitHub - torinmb/mediapipe-touchdesigner: GPU Accelerated MediaPipe Plugin for TouchDesigner

I am working on patch that use media pipe to send signal both to the visual in TD and out to VCV to control some of the sounds elements.

[ale47p (Alessandro Paglia)]

thank you, I’ll take a look!

[dan.tilley (DanT)]

Episode 2

VCV Audio reactive

I am following a YouTube playlist to learn the basic of TD

Video 14 on SOPs & MATs has a nice audio reactive thing that I am recreating, but I want the audio to be from VCV…

This is where we run into a few issues.

I believe that Windows11 supports loopback audio via WASAPI (ie, you can route the audio from one application into another application) and VCV can send audio to WASAPI, but TD doesn’t support WASAPI, only DirectSound and ASIO, so thats a no go.

On a Mac you could use the loopback app which is just fantastic, but the offerings on Windows are all a bit cryptic and fiddley.

I tried VB-Cable, HiFi-Cable & VoiceMeeter.

I tried VAC, I even tried ReaRoute with Reaper, but nothing was quite working how I wanted, clean and easy…

Eventually I just hard-wired my headphone jack to my line-in jack on my PC, although even this is not that simple as you must make sure the hardware is setup correctly

Once you have your audio loopback figured out (I could have just plugged in one of my audio interfaces and done this easily, but I am trying to stay fully software for now), it is then simple to get the audio into and out of TD with the Audio CHOPs

So the audio generated in VCV is sent to my headphones jack

Then down the physical cable to the line-in jack on my mobo

Then into TD, then out of TD to my Speakers…

---

So now we just build the rest of the TD patch as per the video, and make some tweaks of our own

This one is quite nice

VCV Rack ➡️ audio ➡️ TD

I am seriously considering buying a MacBook now

[pachde (Paul Dempsey)]

Great job working the problem.

FWIW some these Windows limitations in their audio stack are in the works. No idea when they’ll be available – I’m guessing it’s at least a year away. The new MIDI service is late but getting much closer to release.

Instead of a MacBook, you might consider a Mini – much less $$$.

[Jens.Peter.Nielsen]

I went through many programs that didn’t really work for me, before I found “VB Audio Matrix”

And they recently released “coconut” with even more I/O.

it can run in a free mode - or you can donate min $10 to get a license key.

[ale47p (Alessandro Paglia)]

and there is always the good old “Jack”…if you figure out how to make it work…it works!

[dan.tilley (DanT)]

Episode 3

Pixel format

Feedback is used a lot in audio visualisations, nice easy way to create a ghost trail of some element.

At first I found that I kept forgetting how to correctly setup the feedback TOP, if I get something slightly off, then it just doesn’t do what I expect.

So I watched a few videos to get more familiar with it.

In this (quite painful to watch) video I learned something very interesting, that applies to feedback, but also could be important to many other TOPs; Common > Pixel Format

The above screenshot is a simple feedback making a trail from the moving banana.

Notice how the trail never actually fully fades away, there is a lighter area from the feedback loop, compared to the next screenshot where the feedback does fully fade away

The reason for this is the pixel format of the feedback and its input.

The default pixel format is normally 8-bit fixed (RGBA) which is causing the fading numbers not to fully reach zero.

To correct for this I just change the pixel format to a float value (either 16-bit or 32-bit)

---

Sorry, no video this time, been playing with my webcam and making feedback loops.

I think I have reached the limit of midi too, going to have to upgrade to OSC I suppose.

I also tried the media pipe, very easy to install and use which was great, but I don’t think much of its object detection

[dan.tilley (DanT)]

Episode 4

OSC

OK, so I upgraded from MIDI to OSC, and yes it is much better and not at all difficult to get working.

I am using the trowaSoft cvOSCcv module to send OSC between VCV and TD

trowaSoft cvOSCcv

I didn’t change any of the settings, just plugged cables into the VAL ports, by default the module is not enabled, so I first configured TD before trying to enable it.

For the incoming OSC to TD you just have to set the Network Port and Local Address as per the settings on the module

I normally want to tweak the values (similar to using an attenuvertor in VCV) so I use a Select CHOP to single out one signal and then add a math or lag etc, it is helpful to add a Trail CHOP as well to actually see what is going on…

For OSC going out of TD, again just set the Network Port and Local Address as per the module and values should show up, though you will likely need to rename the channel and rescale the values

Back over in VCV, click the MASTER CONFIG button and then click the ENABLE button, I am using the Sapphire env to send the audio levels into TD

Sapphire Env

…and thats about all there is to it, the rest is just messing around in TD until I get something I like

---

TD ↔️ OSC ↔️ VCV Rack

[ale47p (Alessandro Paglia)]

yes, for Rack + TD “cvOSCcv” is the way to go

[cosmoproductions (Gabe)]

Great stuff! Thanks for this comprehensive documentation of your first steps and journey thus far. Looking forward to seeing more of what you do with it!

I’ve looked around on youtube and had a hard time finding videos with advanced examples of touchdesigner usage that are not tutorials. I found a few non-tutorial videos by searching for ‘touchdesigner demo reel’, but not much. Does anyone have recommendations for touchdesigner vids, done by people who are expert level with the software, which are not tutorials, but rather just pure inspiration for what’s possible with it? Videos like yours simply showing the thing in action, are a great idea, and seemingly rare.

[dan.tilley (DanT)]

There are occasionally some good posts on Reddit

[shachaf.polakow]

These are some that I often check their work Torin Blankensmith - https://www.youtube.com/@blankensmithing/shorts Pao Olea - https://www.youtube.com/@pao_olea/shorts Tekt, who created Raytk, has this list of showcases - https://www.youtube.com/watch?v=CK9cqJKI2ss&list=PL74x1dpaScWO1s1f-0BzO7ZX2NJ0cSfsr Samo is wild and great - https://www.youtube.com/@roguesamo/videos Alexxxxi - https://www.youtube.com/@Alexxi-generativeartist/shorts

In general, i found that IG or YouTube shorts have good examples of work. Also, i signed up for a Patreon account for free to see work samples.

At the end of the day, it depends on what you are looking to do with TD, very similar to VCV, the options are endless.

And for some self-promotion, both these pieces are VCV sound going live to TD

Red Giant November 2023

Metzolut - מצולות

Always happy to discuss more about TD

[cosmoproductions (Gabe)]

Nice. Yeah that’s a treasure trove of bite sized inspirations mixed with some very full meals to digest. In particular to this forum, searching that community for ‘VCV’ yields some interesting posts. Thanks

[dan.tilley (DanT)]

I am also following a TouchDesigner feed on BlueSky which has some good stuff

https://bsky.app/profile/rolico.bsky.social/feed/aaahikiqcs7ba