Module expanders tutorial?

Yes, our modular mixers are very similar to what you are asking for. And they are open source so if you want to make you own you may find some info there. That said it was very difficult for us to get this to work, especially the rather trivial details around getting solo to work across the modules. don’t know if it’s cause or effect, but we have to confess our source code for the expander part of it is not super clear.

2 Likes

Just to be 100% sure … there should be nothing preventing a module to have expanders connected both to its left side and to its right side, right?

Correct.

1 Like

we put out mixer ex panders on the left, but that’s just because “real” mixers tend to be organized that way.

1 Like

I’m revisiting this thread because my bi-directional expander code isn’t working. It’s easy to get confused with this stuff and I could use an example to follow. However, the link is broken to Andrew’s compact example. Does anyone know if it’s still around somewhere?

The github issues were removed in that repo, so unless someone has a copy, it might be best to look at some code of an open source module that uses expanders. I have a few (as do many others), and I’m definitely not claiming to have the most elegant code, but perhaps some of my simpler modules like Clocked or ChordKey could be worth a look. In those cases, the buffers are declared as float arrays, but it can in fact be a struct to make management easier. I’ve done the struct method in MixMaster/AuxSpander, but that’s way more code to sift through perhaps than what is needed.

Drat, I’m having trouble. Maybe I can talk it out here.

My main module (GrooveBox) expects the expansion module (GrooveBoxExpander) to be on the left-hand side.

I’m using two different structs for passing around the data: One for the expandergroovebox communication, and a different struct for the grooveboxexpander communications…

I’ve been successful in sending data from the expander module to the groovebox module.

Here’s the structure used for sending information from the expander to the groovebox:

#pragma once

struct GrooveBoxExpanderMessage
{
  bool message_received = true;
  bool mutes[8];
  bool solos[8];
  float track_volumes[8];

  GrooveBoxExpanderMessage()
  {
    for(unsigned int i=0; i<8; i++)
    {
      mutes[i] = false;
      solos[i] = false;
      track_volumes[i] = 1.0;
    }
  }
};

In the Groovebox module, I define the following class variables:

  GrooveBoxExpanderMessage *producer_message = new GrooveBoxExpanderMessage;
  GrooveBoxExpanderMessage *consumer_message = new GrooveBoxExpanderMessage;

I’ve created a function that sends data from the Groovebox module to the expander which is called every frame:

  void processExpander()
  {
    if (leftExpander.module && leftExpander.module->model == modelGrooveBoxExpander)
    {
      // Receive message from expander
      GrooveBoxExpanderMessage *expander_message = (GrooveBoxExpanderMessage *) leftExpander.producerMessage;

      // Retrieve the data from the expander
      if(expander_message && expander_message->message_received == false)
      {
        // Copy the information in the structure to local variables.
        // This code was left out because it doesn't really pertain to
        // my question, and it's working fine.
        //
        // But saying that, here's just a taste:
        for(unsigned int i=0; i < NUMBER_OF_TRACKS; i++)
        {
           bool expander_mute_value = expander_message->mutes[i];
           // (I then store expander_mute_value locally)
        }

        // Set the received flag so we don't process the message every single frame
        expander_message->message_received = true;
      }

      leftExpander.messageFlipRequested = true;
    }
  }

So far, so good. That seems to work perfectly. However, now I wish to do the opposite: Send information from the groovebox module to the expander. First step, I created a struct to hold the data:

#pragma once

struct GrooveBoxMessage
{
  bool track_triggers[8];
  bool message_received = true;
  
  GrooveBoxMessage()
  {
    for(unsigned int i=0; i<8; i++)
    {
      track_triggers[i] = false;
    }
  }
};

And here’s my code in the GrooveBox that’s not working, with a clue to why:

It probably has something to do with this line:

GrooveBoxMessage *groovebox_message = (GrooveBoxMessage *) leftExpander.module->rightExpander.producerMessage;

And because I’m a bit confused about what’s going on. Thanks for any help you can muster!

I think it would be easier to combine the structures. Rely on one of the modules, either the parent, or the expander; to own those structures and coordinate the swap.

To keep things as clear as they are now; enclose one of each of your structures into a new structure.

I cant see anything in your code to set message received to false. But that is what you are testing for.

Also i notice that you have dynamically allocated your message structures. Are you disposing of them correctly? (Not related to your current issue)

You can avoid the dynamic allocation but using a static array of two structures.

Perhaps I’m understanding the code wrong and my comment will be besides the mark, but I think for a given direction of communication you always have to declare an array of 2 elements of the buffer you wish to use. For example, if I want to share 6 floats in one direction, it has to be declared as a double buffer:

And when using a struct, it looks like this:

The whole mechanism has to be diligently followed, and for each direction the double buffer is really needed.

You are trying to do 2 way communication between a module and an expander; am I understanding that right?

Yes, correct!

That’s interesting. I think that I may be essentially doing the same thing when I assign the structures like:

// defined as class variables
GrainEngineExpanderMessage *producer_message = new GrainEngineExpanderMessage;
GrainEngineExpanderMessage *consumer_message = new GrainEngineExpanderMessage;
// Assigned in the constructor
leftExpander.producerMessage = producer_message;
leftExpander.consumerMessage = consumer_message;

The above :point_up_2: is code that is successfully running in my GrainEngineMK2 expander. Here’s a thread where the author takes the same approach:

This is all starting to come back to me, so I may be able to sort this out tonight, or at least bring some more specific questions.

Perhaps one question is – am I supposed to be reading from leftExpander.consumerMessage? This might be something I’ve been doing wrong. ooohhhhhh… :bulb: I think it’s starting to all make sense. I’ll work on this again tonight!

1 Like

I knocked up a successful 2-way expander last night . I’m just about to start work; I’ll post more info about how I did it tonight unless you solve it in the meantime.

@CountModula That’s super nice of you, but I just figured it out as well. I used two different message structures instead of arrays. :slight_smile:

Thanks for reminding me. I did not, and I’ll get that fixed. :slight_smile:

@CountModula Woops,

I take it back. I did not succeed. I kind-of just got lucky but was doing it the wrong way. I’d love to see your solution. :bowing_man:

This is an abstract of what worked for me and assumes the expander is to the right of the module.

Module code:

// this is the struct we'll use to pass info back and forth
struct myExpanderMessage {
	bool somethingToGetFromTheExpander;
	int someThingToSendToTheExpander;
};


struct MyModule : Module {

	enum ParamIds {
		NUM_PARAMS
	};
	enum InputIds {
		NUM_INPUTS
	};
	enum OutputIds {
		NUM_OUTPUTS
	};
	enum LightIds {
		NUM_LIGHTS
	};
	
	// assuming the expander is on the right
	 myExpanderMessage rightMessages[2][1]; // messages to right module (expander)
	
	MyModule() {
		config(NUM_PARAMS, NUM_INPUTS, NUM_OUTPUTS, NUM_LIGHTS);
		
		// parameter/port config goes here
		
		// set the right expander message instances
		rightExpander.producerMessage = rightMessages[0];
		rightExpander.consumerMessage = rightMessages[1];
	}

	void process(const ProcessArgs &args) override {

		// do your processing here
		
		// set up details for the expander
		if (rightExpander.module) {
			if (rightExpander.module->model == modelMyExpander) {
				
				myExpanderMessage *messageToExpander = (myExpanderMessage*)(rightExpander.module->leftExpander.producerMessage);
				myExpanderMessage *messageFromExpander = (myExpanderMessage*)(rightExpander.module->leftExpander.consumerMessage);

				messageToExpander->somethingToSendToTheExpander = someModuleValue;
				
				someExpanderValue = messageFromExpander->someThingToGetFromTheExpander;
				
				rightExpander.module->leftExpander.messageFlipRequested = true;
			}
		}

	}
};

Model *modelMyModule = createModel<MyModule, MyModuleWidget>("MyModule");

Expander code:


// this is the struct we'll use to pass info back and forth must be the same for module and expander
struct myExpanderMessage {
	bool somethingToGetFromTheExpander;
	int someThingToSendToTheExpander;
};


struct MyExpander : Module {

	enum ParamIds {
		NUM_PARAMS
	};
	
	enum InputIds {
		NUM_INPUTS
	};
	
	enum OutputIds {
		NUM_OUTPUTS
	};
	
	enum LightIds {
		NUM_LIGHTS
	};
	
	int processCount = 8;
	bool doReset = false;
	int count = 0;
	
	// assuming the controller is on the left
	myExpanderMessage leftMessages[2][1]; // messages from left module (controller module))
	
	MyExpander() {
		config(NUM_PARAMS, NUM_INPUTS, NUM_OUTPUTS, NUM_LIGHTS);
		
		// parameter/port config goes here

		// set the left expander message instances
		leftExpander.producerMessage = leftMessages[0];
		leftExpander.consumerMessage = leftMessages[1];	
	}

	void process(const ProcessArgs &args) override {

		// do your processing here
		
		if (leftExpander.module) {
			if (leftExpander.module->model == modelMyModule) {
					
				myExpanderMessage *messagesFromModule = (myExpanderMessage *)(leftExpander.consumerMessage);
				myExpanderMessage *messagesToModule = (myExpanderMessage *)(leftExpander.producerMessage);

				someValueFromTheModule = messagesFromModule->someThingToSendToTheExpander;
				
				messagesToModule->somethingToGetFromTheExpander = someValueForTheModule;
			}
		}

	}
};

Model *modelMyExpander = createModel<MyExpander, MyExpanderWidget>("MyExpander");

In the module, you set the data what you want to send to the expander in the producerMessage and you get the expander data you want from the consumerMessage then you flip it.

In the expander, you get the module data you want from the consumerMessage and you set the data you want to send to the module in the producerMessage and it appears that you don’t need to flip it although now I’ve said that I’m sure someone far cleverer than I am will correct me.

I tested this with a dummy expander on a sequencer module with the sequecner passing the current step to the expander for display and the expander passing a button value back to the sequencer to make it manually reset and it worked very nicely:

image

7 Likes

Cool! I got my own code working before I saw you posting. Here’s my version:

First, I have two different structures for the data being passed, depending if it’s being sent from the module to expander, or from the expander to the module.

#pragma once

struct ExpanderToGrooveboxMessage
{
  bool message_received = true;
  bool mutes[8];
  bool solos[8];
  float track_volumes[8];

  ExpanderToGrooveboxMessage()
  {
    for(unsigned int i=0; i<8; i++)
    {
      mutes[i] = false;
      solos[i] = false;
      track_volumes[i] = 1.0;
    }
  }
};

and…

#pragma once

struct GrooveboxToExpanderMessage
{
  bool track_triggers[8];
  bool message_received = true;

  GrooveboxToExpanderMessage()
  {
    for(unsigned int i=0; i<8; i++)
    {
      track_triggers[i] = false;
    }
  }
};

Next, I had to make myself a little diagram to keep my brain from exploding:

Let’s start with the message that goes from the

groove box expander => groovebox

Step #1:

In GrooveBox, I created two message containers:

struct GrooveBox : Module
{
  ExpanderToGrooveboxMessage expander_to_groovebox_message_a;
  ExpanderToGrooveboxMessage expander_to_groovebox_message_b;

Step #2: In the groovebox constructor, I set the following:

GrooveBox()
{
  this->leftExpander.producerMessage = &expander_to_groovebox_message_a;
  this->leftExpander.consumerMessage = &expander_to_groovebox_message_b;

Step #3: In GrooveBoxExpander.hpp (the expander’s main code), I added the following function which is called whenever it’s time to send information from the expander to the module. I check to see if anything has changed before calling this function to avoid unnecessary work:

  void writeToGroovebox()
  {
    if (rightExpander.module && rightExpander.module->model == modelGrooveBox)
    {
      // Prepare message for sending to Grain Engine MK2
      // When writing to the groovebox, we're using the __GrooveBox's__ producer and consumer pair
      // Always write to the producer and read from the consumer
      ExpanderToGrooveboxMessage *message_to_groove_box = (ExpanderToGrooveboxMessage *) rightExpander.module->leftExpander.producerMessage;

      // Wait until the groovebox received the last message
      if(message_to_groove_box && message_to_groove_box->message_received == true)
      {
        for(unsigned int i=0; i < NUMBER_OF_TRACKS; i++)
        {
          message_to_groove_box->mutes[i] = mutes[i];
          message_to_groove_box->solos[i] = solos[i];
          message_to_groove_box->track_volumes[i] = params[VOLUME_KNOBS + i].getValue();
        }

        // Tell GrooveBox that the message is ready for receiving
        message_to_groove_box->message_received = false;
      }
    }
  }

Notice that there’s no “flip” request. I have the receiving code ask for the flip.

Step #4:

In the GrooveBox module’s code, I wrote the following function to receive the messages from the expander. This function is called from the GrooveBox’s Process loop:

  void readFromExpander(float rack_sample_rate)
  {
    if (leftExpander.module && leftExpander.module->model == modelGrooveBoxExpander)
    {
      // Receive message from expander.  Always read from the consumer.
      // when reading from the expander, we're using the __GrooveBox's__ consumer and producer message pair
      ExpanderToGrooveboxMessage *consumer_message = (ExpanderToGrooveboxMessage *) leftExpander.consumerMessage;

      // Retrieve the data from the expander
      if(consumer_message && consumer_message->message_received == false)
      {
        this->any_track_soloed = false;

        for(unsigned int i=0; i < NUMBER_OF_TRACKS; i++)
        {
          bool expander_mute_value = consumer_message->mutes[i];
          bool expander_solo_value = consumer_message->solos[i];

          //  THIS IS ALL MY IMPLEMENTATION SPECIFIC
          // ======================================
          if((this->mutes[i] == false) && (expander_mute_value == true) && (this->solos[i] == false))
          {
            this->selected_memory_slot->tracks[i].fadeOut(rack_sample_rate);
          }

          this->mutes[i] = expander_mute_value;
          this->solos[i] = expander_solo_value;
          this->track_volumes[i] = consumer_message->track_volumes[i];
          if(this->solos[i]) this->any_track_soloed = true;
          // ======================================
        }

        // Set the received flag
        consumer_message->message_received = true;
      }

      leftExpander.messageFlipRequested = true;
    }
  }




groove => groovebox box expander

This is essentially the opposite!

Step #1:

In the GrooveBox Expander, I added the following variables:

struct GrooveBoxExpander : Module
{
  GrooveboxToExpanderMessage groovebox_to_expander_message_a;
  GrooveboxToExpanderMessage groovebox_to_expander_message_b;

Step #2: In the GrooveBox Expander constructor:

  GrooveBoxExpander()
  {
    rightExpander.producerMessage = &groovebox_to_expander_message_a;
    rightExpander.consumerMessage = &groovebox_to_expander_message_b;

Step #3: In the main Groovebox module, I added code for sending the data to the expander. This is called from the main module’s Process loop:

  void writeToExpander()
  {
    if (leftExpander.module && leftExpander.module->model == modelGrooveBoxExpander)
    {
      // Always write to the producerMessage
      GrooveboxToExpanderMessage *groovebox_to_expander_message = (GrooveboxToExpanderMessage *) leftExpander.module->rightExpander.producerMessage;

      if(groovebox_to_expander_message && groovebox_to_expander_message->message_received == true)
      {
        for(unsigned int i=0; i < NUMBER_OF_TRACKS; i++)
        {
          groovebox_to_expander_message->track_triggers[i] = this->track_triggers[i];
          if(this->track_triggers[i]) this->track_triggers[i] = false;
        }

        groovebox_to_expander_message->message_received = false;
      }
    }
  }

(Notice again that there’s no “flip” request in the sending code. I put that in the receiving code.)

Step #4 Here’s the code in the Expander that receives the information. This is called from the expander’s Process() loop:

  void readFromGroovebox()
  {
    if (rightExpander.module && rightExpander.module->model == modelGrooveBox)
    {
      // Receive message from expander
      GrooveboxToExpanderMessage *groovebox_message = (GrooveboxToExpanderMessage *) this->rightExpander.consumerMessage;

      // Retrieve the data from the expander
      if(groovebox_message && groovebox_message->message_received == false)
      {
        for(unsigned int i=0; i < NUMBER_OF_TRACKS; i++)
        {
          if(groovebox_message->track_triggers[i])
          {
            // Trigger output
            triggerOutputPulseGenerators[i].trigger(0.01f);
            triggerLightPulseGenerators[i].trigger(0.05f);
          }
        }

        groovebox_message->message_received = true;
      }
      rightExpander.messageFlipRequested = true;
    }
  }
6 Likes