Batch API

[klamike]

@amontoison

Here is what I have so far for the batch API and some "dumb" implementations of it, ForEachBatchNLPModel and InplaceBatchNLPModel as discussed. I'm opening this mostly to start discussion about how best to do batches of models, not necessarily suggesting we merge it as is (after all, it is "dumb" 😉).

ForEachBatchNLPModel just loops over the children's functions, and InplaceBatchNLPModel makes users pass updates, a list of functions that update a base_model to a batch element. The idea is if you are just varying e.g. the lcon, you can do something like

base_model = ...
lcons = [[1.0, 2.0], [1.0, -2.0], ...]
batch_size = length(lcons)

bnlp = InplaceBatchNLPModel(base_model, [nlp -> copyto!(get_lcon(nlp), lcons[i]) for i = 1:batch_size])

[klamike]

This is not related, just hides the deprecation warning (#501)

[amontoison]

cc @frapac

[klamike]

Docs are broken due to #520

[tmigot]

Hi @klamike ! Can you expand a bit more on the context for this and potential use of it?

[klamike]

@tmigot The idea is to define an API to talk about batches of models in a standardized way across JSO. Then each solver (up for discussion -- maybe we should do that here instead?) can define its own batch model types, which would mark the relevant assumptions (e.g. same jac/hess structure, same nvar/ncon, etc.) to enable exploiting that shared structure (importantly, not just in the evaluation of the model, but also in the solvers themselves). For example, if we assume the same jac/hess structure, we can use CUDSS uniform batch API for solving KKT systems instead of looping over single solves. This should bring substantial speedup over something like madnlp.(nlps::Vector{<:AbstractNLPModel}).

To be honest, this kind of change is very big for me to think about all the consequences of different design decisions at once. So, we decided it might be best to just implement something dumb first to get some visibility into tradeoffs.

[klamike]

Note the counters are not properly wired up yet, both here and in foreach

[klamike]

Had to exclude the inplace version here since the closures in the operators reference the model state, which we are mutating without their knowledge

[klamike]

It should be doable to put the update function inside the operator closure..

[klamike]

I am hoping that making Julia specialize on the N in Vararg{Any,N} will make this compile down to something reasonable