Feature request: support for model compositionality

[trondarild]

For longer term consideration: It would be useful to be able to compose larger models from smaller models, as described in this paper:

https://msp.cis.strath.ac.uk/act2022/papers/ACT2022_paper_8631.pdf

Thanks for a great initiative!

[highperformancecoder]

I did take a look at that paper - thanks for the reference. It is kind of a highly mathematical representation.
However, I think Minsky already provides what you are looking for. Composition of submodels is handled by groups, which can be individually imported or exported, and interfaces defined by means of I/O variables.

I would love to work with people to iron out the rough edges of the implementation of groups.

[trondarild]

Thank you, I will make sure to look into how groups work!
As I understand it, the novelty of the paper referenced above was the ability to compose two models that use the same stock. In Minsky, one application would be composing models that both reference the same Godley table. Is this also supported with groups?

[highperformancecoder]

On Fri, Jul 29, 2022 at 02:21:28AM -0700, Trond Arild Tjostheim wrote: Thank you, I will make sure to look into how groups work! As I understand it, the novelty of the paper referenced above was the ability to compose two models that use the same stock. In Minsky, one application would be composing models that both reference the same Godley table. Is this also supported with groups?

You can certainly refer to the same stock variable within two groups. The stock variable's definition can be (eg) part of a Godley table that is in the outer scope containing the two groups. Also, you can define two different stock with the same name in two different contexts (eg Treasury reserves for USA and Australia), by defining the Godley tables in different groups. This allows replicating Godley tables for economic sectors. Flows can be either explicitly represented, via the use of I/O variables, which add ports to the groups, or implicit via the local/global scope mechanism I described above. I'm pretty sure Minsky supports the maximum flexibility of model composition, replication and so on, that is theoretically possible. Of course, this is "advanced" stuff, and nobody has really tried to do this in anger, so there may well be bugs/glitches in the implementation :).

…

--

---------------------------------------------------------------------------- Dr Russell Standish Phone 0425 253119 (mobile) Principal, High Performance Coders ***@***.*** http://www.hpcoders.com.au ----------------------------------------------------------------------------

[aleith]

I did take a look at that paper - thanks for the reference. It is kind of a highly mathematical representation. However, I think Minsky already provides what you are looking for. Composition of submodels is handled by groups, which can be individually imported or exported, and interfaces defined by means of I/O variables.

I would love to work with people to iron out the rough edges of the implementation of groups.

I'm interested in this. I've only used nodal/visual programming in video programming (effects/generation/vision mixing apps for sports stadiums to use to display all kinds of live data from an XML stats feed). It seems like some of this is already quite easy to do in Minsky but I'm a day one novice (in spite of me backing the original Kickstarter, I was a designer/hack-coder at the time but now I'm an MMT/Ecological Econ post-grad student!)

There are a few different uses for groups in nodal systems I've used (different nodal apps have different conventions around these too) and people often are talking about different purposes depending on the use situation they foresee. Even within the "compositional" FR theme paradigm, scope is a big issue to think through b/c lots of implications for the future of the software depending on choices made today if you don't want to break backwards compatibility.

Based on throught I've given to this in the past (see this FR thread for a different app entirely, Vuo.org), there's three main types of group that people want to execute and they are a bit mutually exclusive, and so to cover all of then actually require seperate implementations. This is a copy and paste from a FR I made on Vuo.org years ago (still waiting!) but I've changed the terms to be minsky centric.

Whereas I want three kinds of sub-compositions.

1. Global — effects all instances in any minsky file that opens if the "group" of functions is actively loaded in a Global Modules folder.

2. Local — affects only instances within the Open composition, always belongs to a single minsky file and if copied to a new comp then is a new thing and belongs to that other composition. any edits in the new minsky file don't edit the instance in the original file. But changes to any instance changes all of the instances within the single Minsky project. Edit one, edit all local instances.

3. Group Nodes | Macros — effects only a single instance and if that instance is copied to elsewhere in any comp it is completely independent of the parent sub-comp and any changes to it or the Group/Macro it was spawned from have no effect on the other. So this is just to reduce screen clutter and speed up canvas rendering when in "play" mode.

In addition are two other desirables: (maybe I need to make these as separate FR to make it transparent that they are part of the concept).

An example of group-type 1.
I could make a simple conversion node on the minsky canvas that takes an input value in ºC units and converts it to ºF units. If I'm using these two functions a lot on the canvas, and across multiple minsky projects, I want to just type "ºC" or "ºF" onto the canvas or in some kind of function browser (think MS Excel) and just get a small node that has one input and one output a label I've assigned eg "Convert 𝑇" or "𝑥 ºC → 𝒙 ºF", and have none of maths visible. or I might want to make it for an array of temperatures "𝔁𝒔 ºC → 𝔁𝒔 ºF".

An example of group-type 2.
might be I write some helper functions to help parse incoming data from various files. As I improve the parser function, I want it to be update for every place where I parse a CSV file, but say the function filters out all columns but one, the input parsing tokens (say, column name or "country") may change in each instance but the underlying code needs to stay consistent.

An example of group-type 3.
is where I have spaghetti and vegan balls in part of the canvas, and rather than drag to one side and run wires across that cannot be traced easily, I put them all inside a "group" or "folder" for neatness showing the published input and output port names only, both at the parent level where it's a single node in appearance, and at the child level where its all the nodes selected for grouping, with some labels for the input ports on the left and output ports on the right, so we can see what is coming in and what is going out. This is a unique case and I don't want it being adjusted at all if some other group that I copied from this one is edited, so local scoping but also local (mutable) code.

The easiest way for developers to build this is be that a second minsky window gets created with the group of nodes that have been collapsed into a group and exported to the node/function library but feeding and sending data from whichever instance that was double clicked to open it (but then as a separate threaded 'application' maybe that's actually pretty hard?)*. But which values to show? Those sent from the parent that is also running I guess because Minsky can only have one main file open (but this is almost like a second file open at the same time, needs careful thought)

A down/up level mode in the Editor is the obvious way to do it, like in grid view (no sub levels are ever visible) in Finder on Mac and the File Browser on Windows/Linux. Although would only really solve 3. Group Node | Macros not Global and Local sub comps (1 and 2 in list above). EDIT: As Jaymie noted, there is already a FR for that behaviour here: Open sub-composition in same window as composition

You can see an animation of opening and closing groups on the same canvas which I made over decade ago animating a sub-composition that explodes from a single "group" node and then the window in window collapses back into the same single node when closed. It's embedded at the bottom of this post (takes forever to load, be patient). A bit like a tree structure in a file browser, but in 2D graphics on a Vuo.org feature request. A lot of these aspects above are discussed with outers at length there too.

[highperformancecoder]

On Sun, Feb 12, 2023 at 02:55:59AM -0800, Alastair Leith wrote: I did take a look at that paper - thanks for the reference. It is kind of a highly mathematical representation. However, I think Minsky already provides what you are looking for. Composition of submodels is handled by groups, which can be individually imported or exported, and interfaces defined by means of I/O variables. I would love to work with people to iron out the rough edges of the implementation of groups. I'm interested in this. I've only used nodal/visual programming in video programming (effects/generation/vision mixing apps for sports stadiums to use to display all kinds of live data from an XML stats feed). There are a few different uses for groups in nodal systems I've used (different nodal apps have different conventions around these too) and people often are talking about different purposes depending on the use situation they foresee. Based on through I've given to this in the past, there's three main types of group that people want to execute and they are a bit mutually exclusive, and so to cover all of then actually require seperate implementations. This is a copy and paste from a FR I made on Vuo.org years ago (still waiting!) but I've changed the terms to be minsky centric. Whereas I want three kinds of sub-compositions. 1. Global — effects all instances in any minsky file that opens if the "group" of functions is actively loaded in a Global Modules folder. 2. Local — affects only instances within the Open composition, always belongs to a single minsky file and if copied to a new comp then is a new thing and belongs to that other composition. any edits in the new minsky file don't edit the instance in the original file. But changes to any instance changes all of the instances within the single Minsky project. Edit one, edit all local instances. 3. Group Nodes | Macros — effects only a single instance and if that instance is copied to elsewhere in any comp it is completely independent of the parent sub-comp and any changes to it or the Group/ Macro it was spawned from have no effect on the other. So this is just to reduce screen clutter and speed up canvas rendering when in "play" mode. In addition are two other desirables: (maybe I need to make these as separate FR to make it transparent that they are part of the concept).

So currently, number 3 is supported by Minky groups. Number 2 is an interesting use case, and could be implemented via naming, like with variables - groups with the same name are the same entity. An empty name would be an anonymous group, and be treated like case 3. I'm not sure we can realistically support use case 1, minsky model files are meant to be standalone. To support this feature, the group feature would need to be exported to another file, and a watcher thread placed on that file to reimport it when it changes. I remember Steve raised a feature request for this, but I can't find it.

So an example of 1. I could make a simple conversion node that takes a value in ºC and converts it to ºF. If I'm using a lot in multiple minsky projects, I want to just type "ºC" or "ºF" onto the canvas or in some kind of function browser (think MS Excel) and just get a small node that has one input and one output and no maths visible.

What we currently have, which is supported, is that you can save your group that does the temperature conversion, and then reuse it in later models by including it. Of course, any edits to that group will not affect the version on disk - you would need to open it separately to edit it, or save the edited group.

So an example of 2. might be I write some helper functions to help parse incoming data from various files. As I improve the parser function, I want it to be update for every place where I parse a CSV file, but say the function filters out all columns but one, the input parsing tokens (say, column name or "country") may change in each instance but the underlying code needs to stay consistent.

Yes - I think this is an interesting feature request.

An example of 3 is where I have spaghetti and vegan balls in part of the canvas, and rather than drag to one side and run wires across that cannot be traced easily, I put them all inside a "group" or "folder" for neatness showing the published input and output port names only, both at the parent level where it's a single node in appearance, and at the child level where its all the nodes selected for grouping, with some labels for the input ports on the left and output ports on the right, so we can see what is coming in and what is going out. This is a unique case and I don't want it being adjusted at all if some other group that I copied from this one is edited, so local scoping but also local (mutable) code. The easiest way for developers to build this is be that a second minsky window gets created with the group of nodes that have been collapsed into a group and exported to the node/function library but feeding and sending data from whichever instance that was double clicked to open it (but then as a separate threaded 'application' maybe that's actually pretty hard?)*. But which values to show? Those sent from the parent that is also running I guess because Minsky can only have one main file open (but this is almost like a second file open at the same time, needs careful thought)

I'm not too keen on doing the multi-process thing. It sounds like an unnecessary complication.

A down/up level mode in the Editor is the obvious way to do it, like in grid view (no sub levels are ever visible) in Finder on Mac and the File Browser on Windows/Linux. Although would only really solve 3. Group Node | Macros not Global and Local sub comps (1 and 2 in list above). EDIT: As Jaymie noted, there is already a FR for that behaviour here: Open sub-composition in same window as composition

Do you mean "Open group in canvas"?

You can see an animation of opening and closing groups on the same canvas which I made over decade ago animating a sub-composition that explodes from a single "group" node and then the window in window collapses back into the same single node when closed. It's embedded at the bottom of this post (takes forever to load, be patient). A bit like a tree structure in a file browser, but in 2D graphics on a Vuo.org feature request. A lot of these aspects above are discussed with outers at length there too.

The link didn't come through in the email :(. I'll see if I can track it down on Github.

…

--

---------------------------------------------------------------------------- Dr Russell Standish Phone 0425 253119 (mobile) Principal, High Performance Coders ***@***.*** http://www.hpcoders.com.au ----------------------------------------------------------------------------

[aleith]

Hi Russel Responses in-line below in boldface Thanks Alastair On Sat, 25 Feb 2023 at 11:58 am, Russell Standish ***@***.***> wrote:

On Sun, Feb 12, 2023 at 02:55:59AM -0800, Alastair Leith wrote: > I did take a look at that paper - thanks for the reference. It is kind of a > highly mathematical representation. However, I think Minsky already > provides what you are looking for. Composition of submodels is handled by > groups, which can be individually imported or exported, and interfaces > defined by means of I/O variables. > > I would love to work with people to iron out the rough edges of the > implementation of groups. > > I'm interested in this. I've only used nodal/visual programming in video > programming (effects/generation/vision mixing apps for sports stadiums to use > to display all kinds of live data from an XML stats feed). > > There are a few different uses for groups in nodal systems I've used (different > nodal apps have different conventions around these too) and people often are > talking about different purposes depending on the use situation they foresee. > > Based on through I've given to this in the past, there's three main types of > group that people want to execute and they are a bit mutually exclusive, and so > to cover all of then actually require seperate implementations. This is a copy > and paste from a FR I made on Vuo.org years ago (still waiting!) but I've > changed the terms to be minsky centric. > > Whereas I want three kinds of sub-compositions. > > 1. > Global — effects all instances in any minsky file that opens if the > "group" of functions is actively loaded in a Global Modules folder. > > 2. > Local — affects only instances within the Open composition, always > belongs to a single minsky file and if copied to a new comp then is a > new thing and belongs to that other composition. any edits in the new > minsky file don't edit the instance in the original file. But changes > to any instance changes all of the instances within the single Minsky > project. Edit one, edit all local instances. > > 3. > Group Nodes | Macros — effects only a single instance and if that > instance is copied to elsewhere in any comp it is completely > independent of the parent sub-comp and any changes to it or the Group/ > Macro it was spawned from have no effect on the other. So this is just > to reduce screen clutter and speed up canvas rendering when in "play" > mode. > > In addition are two other desirables: (maybe I need to make these as > separate FR to make it transparent that they are part of the concept). So currently, number 3 is supported by Minky groups. Number 2 is an interesting use case, and could be implemented via naming, like with variables - groups with the same name are the same entity. An empty name would be an anonymous group, and be treated like case 3.

*Ok when I can use the editor canvas more easily on Mac I’ll get into exploring this feature. Thanks. *

I'm not sure we can realistically support use case 1, minsky model files are meant to be standalone. To support this feature, the group feature would need to be exported to another file, and a watcher thread placed on that file to reimport it when it changes. I remember Steve raised a feature request for this, but I can't find it.

For Vuo this was an important way to speed the development of user compositions and also to improve performance because in Vuo to “run” compositions. In Vuo to “run” the graph, they compositions is quickly compiled and the binary code is executed on a seperate thread(s). Messages are sent between the the Vuo editor and the running composition using the ZeroMQ <https://zeromq.org> protocols which share the values of data coming in and out of ports. Changes in then composition graph can be passed as messages if simple (changes in constant values) or recompiled (changes in the functions). Having pre-compiled Groups of nodes speeds up compilation and the execution of the composition itself because less messaging and canvas redrawing with data flows. *Can’t speak for Minsky till i use it in danger!*

> > So an example of 1. I could make a simple conversion node that takes a value in > ºC and converts it to ºF. If I'm using a lot in multiple minsky projects, I > want to just type "ºC" or "ºF" onto the canvas or in some kind of function > browser (think MS Excel) and just get a small node that has one input and one > output and no maths visible. > What we currently have, which is supported, is that you can save your group that does the temperature conversion, and then reuse it in later models by including it. Of course, any edits to that group will not affect the version on disk - you would need to open it separately to edit it, or save the edited group. > So an example of 2. might be I write some helper functions to help parse > incoming data from various files. As I improve the parser function, I want it > to be update for every place where I parse a CSV file, but say the function > filters out all columns but one, the input parsing tokens (say, column name or > "country") may change in each instance but the underlying code needs to stay > consistent. > Yes - I think this is an interesting feature request. > An example of 3 is where I have spaghetti and vegan balls in part of the > canvas, and rather than drag to one side and run wires across that cannot be > traced easily, I put them all inside a "group" or "folder" for neatness showing > the published input and output port names only, both at the parent level where > it's a single node in appearance, and at the child level where its all the > nodes selected for grouping, with some labels for the input ports on the left > and output ports on the right, so we can see what is coming in and what is > going out. This is a unique case and I don't want it being adjusted at all if > some other group that I copied from this one is edited, so local scoping but > also local (mutable) code. > > The easiest way for developers to build this is be that a second minsky > window gets created with the group of nodes that have been collapsed into a > group and exported to the node/function library but feeding and sending > data from whichever instance that was double clicked to open it (but then > as a separate threaded 'application' maybe that's actually pretty hard?)*. > But which values to show? Those sent from the parent that is also running I > guess because Minsky can only have one main file open (but this is almost > like a second file open at the same time, needs careful thought) > I'm not too keen on doing the multi-process thing. It sounds like an unnecessary complication.

*All depends on whether the need justifies the means and end, I think. Dont’ know minsky at all well yet!*

> A down/up level mode in the Editor is the obvious way to do it, like in > grid view (no sub levels are ever visible) in Finder on Mac and the File > Browser on Windows/Linux. Although would only really solve 3. Group Node | > Macros not Global and Local sub comps (1 and 2 in list above). EDIT: As > Jaymie noted, there is already a FR for that behaviour here: Open > sub-composition in same window as composition > Do you mean "Open group in canvas"?

*Yes one way of doing it is that you can only ever see one level of the composition/graph. the easiest way to implement it is that if you are inside a group of nodes you cannot see the nodes of the levels above or below. You have to navigate up and down like a folder structure without any tree-type-visibility,always leaving one level to go to another level up or down. * Another way to do it is you open seperate levels in seperate windows. Another way is to have sub graphs visible on the canvas of the parent level. Window in a window type of thing. I explored that many years ago in this video: https://vuo.org/sites/default/files/feature/Sub-Graphs-ii.m4v Lots of discourse around this on these Vuo forum threads (may or may not be relevant to minsky of course): https://vuo.org/node/1723#comment-4221 https://vuo.org/node/1184

> You can see an animation of opening and closing groups on the same canvas which > I made over decade ago animating a sub-composition that explodes from a single > "group" node and then the window in window collapses back into the same single > node when closed. It's embedded at the bottom of this post (takes forever to > load, be patient). A bit like a tree structure in a file browser, but in 2D > graphics on a Vuo.org feature request. A lot of these aspects above are > discussed with outers at length there too. > The link didn't come through in the email :(. I'll see if I can track it down on Github.

*Links to video and threads in comment above this one* Best Alastair

-- ---------------------------------------------------------------------------- Dr Russell Standish Phone 0425 253119 (mobile) Principal, High Performance Coders ***@***.*** http://www.hpcoders.com.au ---------------------------------------------------------------------------- — Reply to this email directly, view it on GitHub <#316 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALVHWSSAZS6JV666EEK4T3WZFKMPANCNFSM54RNFCTA> . You are receiving this because you commented.Message ID: ***@***.***>

-- Best Alastair Alastair Leith +61 432 889 831 Outreach for Sustainable Energy Now Founding Co-Convenor of Renew WA

[highperformancecoder]

On Sun, Feb 26, 2023 at 10:59:54PM -0800, Alastair Leith wrote: For Vuo this was an important way to speed the development of user compositions and also to improve performance because in Vuo to “run” compositions. In Vuo to “run” the graph, they compositions is quickly compiled and the binary code is executed on a seperate thread(s). Messages are sent between the the Vuo editor and the running composition using the ZeroMQ <https://zeromq.org> protocols which share the values of data coming in and out of ports. Changes in then composition graph can be passed as messages if simple (changes in constant values) or recompiled (changes in the functions). Having pre-compiled Groups of nodes speeds up compilation and the execution of the composition itself because less messaging and canvas redrawing with data flows. *Can’t speak for Minsky till i use it in danger!*

Yes - it is architecturally quite different. In Minsky, the graph, or nodal diagram, is compiled into a list of equations, which is applied to 2 vectors of numbers, one representing the stocks, and the other the flows. It is mainly done this way in order to run a standard integrator - eg Runge-Kutta. To evaluate the model function means iterating through the equations, evaluating each one.

> > I'm not too keen on doing the multi-process thing. It sounds like an > unnecessary complication. *All depends on whether the need justifies the means and end, I think. Dont’ know minsky at all well yet!*

I think in Minsky's case, running each wire between operations (eg +, * etc) as a 0MQ pipe is a huge overhead, compare with the evaluation cost of the operation. You would need much larger components in order to justify the overhead of such a pipe. Incidently, if talking about interthread communication, 0MQ leaves a lot to be desired, but for interprocess, and/or intermachine communication it is rather a nice library.

> > > > > A down/up level mode in the Editor is the obvious way to do it, like in > > grid view (no sub levels are ever visible) in Finder on Mac and the File > > Browser on Windows/Linux. Although would only really solve 3. Group Node > | > > Macros not Global and Local sub comps (1 and 2 in list above). EDIT: As > > Jaymie noted, there is already a FR for that behaviour here: Open > > sub-composition in same window as composition > > > > Do you mean "Open group in canvas"? *Yes one way of doing it is that you can only ever see one level of the composition/graph. the easiest way to implement it is that if you are inside a group of nodes you cannot see the nodes of the levels above or below. You have to navigate up and down like a folder structure without any tree-type-visibility,always leaving one level to go to another level up or down. * Another way to do it is you open seperate levels in seperate windows. Another way is to have sub graphs visible on the canvas of the parent level. Window in a window type of thing. I explored that many years ago in this video: https://vuo.org/sites/default/files/feature/Sub-Graphs-ii.m4v Lots of discourse around this on these Vuo forum threads (may or may not be relevant to minsky of course): https://vuo.org/node/1723#comment-4221 https://vuo.org/node/1184

I did end up finding and seeing your video, which was elegant. In Minsky's case, we borrowed from the zooming user interface https://en.wikipedia.org/wiki/Zooming_user_interface, which I still prefer. When you zoom in, at a certain level the contents of groups become visible, and you can see the contents of the group in the context of the surrounding graph. However, due to inevitable difficulty in implementing this UI correctly, I ended up giving in to the pressure of having a "Open group in canvas" feature.

…

--

---------------------------------------------------------------------------- Dr Russell Standish Phone 0425 253119 (mobile) Principal, High Performance Coders ***@***.*** http://www.hpcoders.com.au ----------------------------------------------------------------------------