Auto delays, regions, scopes, routing mechanism - future of MeshCore

[stachuman]

I really start to doubt of the future of MeshCore solution. It is simply breaking apart and I don't see dedication of the core dev team to address it at the right level.
I mean - I see that 'dev team' sees that network simply breaks with amount of collisions and try to address it (default scope, regions) - but I feel - it is going the either wrong direction or it is not enough. All of that will need to be configured manually - so how can we assume - it will be done in an optimal way?

I've built a simulator to better see what is happening, I've spent significant amount of time over building test cases, simulations, together with @KPrivitt and few other guys - we tried to propose something. What I see? (maybe I'm wrong) - zero interest from the dev team.
Sure, it's their project, they can do whatever they want.

After couple of weeks going deep into the topic - honestly - I see only one way to make it solid. Network needs to implement auto-adjustment in a much larger scale. Not only including auto-delay mechanism, but what is more important, far better routing mechanism... simply, network needs to start operate as network - not as individual repeaters. Maybe even including internal protocol to let the nodes self-adjust, route packets, limit collisions...

I know, it's not easy and I'm aware - network in any case will have limited capacity we can't extend. But guys - a single flood message is making whole region red...

I hope, I'm wrong and dev team has ideas how to move on, simply I'm not aware.

[KPrivitt]

Copied from PR #2337 to this discussion thread #2375

stachuman,
Thank you for your tremendous effort. Your results validate my original supposition that as the density increases the need for additional backoff increases and density (which is an easy thing to measure using a neighbor count) is a valid metric to use to adjust and set the amount of backoff.

From a theoretical point of view this is easy to see: the probability of a collision drops off as the number of slots increases, however if the number of neighbors increases it counteracts that benefit. Essentially, if the number of neighbors doubles the number of slots needs to double.

This does lead to high delay values in a very dense mesh. But remember in my comments on "success" I did mention that as this increases it does introduce a delay and when that delay becomes humanly noticeable (40 seconds definitely exceeded that threshold) it would need to be capped. A "balanced" or throttled setting is needed.

One thing is clear, an automatic tuning does improve mesh performance, and the current defaults are bad settings, in that they provide insufficient backoff. The mesh works better when collisions are reduced and backoff reduces collisions.

The question and discussion at hand is what should the table entries be for each neighbor count?

The discussion also needs to be about what the "success criteria" should be: what metric should be used? Just one? A combination? How should balancing be done. This will likely be contentious and have many different and sometimes opposing views. But the discussion is GOOD, all views should be entertained. That usually will generate a better result. I hope the dev experts will participate.

So, given the data we have: Can we start with a conservative table to enable automatic tuning and finalize the table later? Lets take the low hanging fruit that is right in front of us.

One comment regarding some comments that this is a centralized approach. It is not, it is decentralized since each repeater can have its own setting. This can be disabled and any repeater owner can set the values they choose. It is not forcing any particular setting, it allows choice and optimization.

The immediate value here is getting the defaults changed to a better setting (eliminate the majority of repeaters being set at the insufficient default values, we need to be good neighbors to each other, just one repeater setting it to better values will help their neighbors, but if the neighbors continue to stomp on you... well, it's best if we are all good neighbors) Plus having the ability to adjust based on density, "the mesh" can adjust for the future.

Optimization of the optimizer can come later.

My opinion: For what it is worth..

Last item: do we know why rxdelay does not affect the simulation results? It should...

That can easily be added to the table as another column. But with what settings... Can it be added and left all 0's (the current default) until we get a future better table and understand what is going on and the "optimum" settings. rxdelay is a secondary benefit, but it is a tool in our chest, why not use it.