Avoiding graphs from processing multiple times

As logged in #578 we have discovered that graphs with N sinks (e.g. multiple jcom.unpack≈ objects in Max) cause the entire graph to process N times.

As a first step to address this, we should develop a number of test cases.

@Nils: It will be really useful if you can provide a simple patch illustrating the problem. This can then be modified for the test cases below so that we have tests to check if possible solutions work for all of different scenarios we can think of.

Contents:

Possible use case scenarios

Case A

  • 1 graph
  • 1 sink
  • 1 sink environment (AudioGraph/PortAudio)

This is a fairly trivial scenario, with one sink only, e.g. jcom.dac≈

Case B

  • 1 graph
  • 1 sink
  • 1 sink environment (MSP)

Yet another trivial scenario, with one sink only: jcom.unpack≈

Case C

  • 1 graph
  • 2 sinks
  • 1 sink environment only (AudioGraph/PortAudio)

This patch use two jcom.dac≈ sink objects

Case D

  • 1 graph
  • 2 sinks, all sinks directed to MSP

This patch use two jcom.unpack≈ objects for passing audio to MSP

Case E

  • 1 graph
  • AG and MSP
  • 2 sink environments (AudioGraph/PortAudio + MSP)

This patch use jcom.unpack≈ as well as jcom.dac≈

Class F cases

This is a class of cases using 2 graphs set up as separate Max patches, with any combination of A to F. There is a total of {{latex($N!$)}} possoble combinations:

  • F-AA
  • F-AB
  • F-AC
  • F-AD
  • F-AE
  • F-BB
  • F-BC
  • F-BD
  • F-BE
  • F-CC
  • F-CD
  • F-CE
  • F-DD
  • F-DE
  • F-EE

Class G cases

This is a class of cases using 2 graphs coexisting next to each other within the same Max patch. There is a total of {{latex($N!$)}} possible combinations, labeled G-AA, G-AB, etc.

Class H cases

This is a class of cases using two graphs. One is hosted in a Max patch, and the other is embedded in a poly~ object within the same patch. There are {{latex($N^2$)}} possible combinations.

Class I cases

This is a class of cases using two graphs. One is hosted in a Max patch, and the other is embedded in a pfft~ object within the same patch. There are {{latex($N^2$)}} possible combinations.

Identification and separation of what problems we are facing

More than one sink

More than one graph

More than one sink environment

Up until now we have tried permitting an audio graph to be able to sink to more than one environment (e.g. MSP and PortAudio). It might be a good time for asking ourselves why, and what potential benefits and problems might be.

Pros

  • Any?

Cons

  • Two sinks might interact with different sound cards. This might cause the sinks to have:
    • different vector sizes and sampling rates
      • How (and where) do we deal with this in the graph?
      • Will the user be able to know and understand whats going on inside the graph in a transparent and predictable way?
      • Do this smell like "convention over configuration" or is it just overly awkward?
    • Clocking issues if they are not both synced to the same clock (e.g. both using their internal clock)
  • On the Mac platform Aggregate device or Jack can be used to dynamically route audio in the direction of multiple sund cards, so it is hard to see the additional benefits of support for several Audio Cards in Audio Graph

Peer or server approach

  • The problem is related to the sinks. So whatever solution we end up with, it should not need to interfere with the general peer-related model that we have applied upstream.

Possible solution:

We permit an audio graph to use one sink environment only. Sink objects might then subscribe to a metasink object. This object is the one driving all of the audio graph by notifying the sinks that it's time to calculate a new vector of audio. This notification can be tagged with a vector ID counter that is increasing by one for each new vector. Hence, when several sinks pulls audio from upstream objects, they will be able to recognize when they are being requested repeatedly within the same audio vector.

A practical way of doing this in MSP would be to make jcom.dac≈ interact with MSP dac~objects rather than PortAudio. The PortAudio solution could still be useful for other environments such as Ruby/Rails.

This could work for case A-D.

Case E would be ruled out.

Somehow we need to be able to have audio graphs understanding if they belong at the top level or if they rather belong to a separate thread, as is the case for poly~ and possibly pfft~. If so they should be constructed as independent graphs, but in Max still be driven by MSP.

Solution Proposed by Tim:

I propose that we don't actually deal with this as a systemic issue in the JAG framework. Instead, I propose that we deal with this as an issue of the host environment. Maybe this is wrong?

In Max/MSP it could work this way:

The unpack≈ object would contain a global (class) variable which is a pointer to a linked-list of all unpack≈ instances. Then the process method would check this list of instances to see if the preprocess message needs to be sent by other objects (or not). If it does need to be sent, then tell those objects to send it. Now all of the objects have received the 'preprocess' message, and processing can carry on with no trouble at all.

I suppose dac≈ would work the same way?

We should also think about this in other environments. In Ruby what happens? In Max we can avoid the issue altogether through the use of join≈ and split≈ instances, but that is noticeably clumsy. Is it also clumsy in Ruby? What would a Ruby graph with multiple sinks look like?

Another way to phrase this problem we are solving: graphs only know about themselves. They don't know about the presence of other graphs. But the host environment would know about the presence of 0 or more graphs, which is why it makes sense to deal with it there.

Responses?