Repository navigation
Handle process externally in ModelicaSystem.simulate and ModelicaSystem.linearized #230
Description
Activity
@kiroorooki The suggestion looks fine, can you update your OMPython to the latest master and make a PR with your changes
I guess we could also add some facility to OMPython similar to what we have for Library Testing in parallel. Basically we first build the model, then run it in parallel with the Python Parallel jobs feature.
The only problem with the suggestion might be conflicting generated files and result file names. If those are different for each parallel simulate command then it should work fine.
ModelicaSystem() creates a separate working directory for each session, so there should not be a problem in conflicting files with same names. We can use python's
Multiprocessinglibrary to achieve parallel simulationThis would be very interesting, e.g., to run massively parallel black-box optimization on the cloud.
Hi everyone! :) This is what we tried to achieve with this change:
We first needed to build multiple versions of a model (tweaked) in different directories (In order to have multiple instances of a model using different parameter values).
This is why we requested the feature to build in a specific folder, as we re-use this path later to simulate a specific instance of the model. (#204)Later we needed to simulate each model instance multiple times with different parameter values (using mod.setParameters()).
However, this step needed to be parallelized. This is why we made the current request.We are now simulating each model instance (sequentially) multiple times with different parameter values (parallelized).
As the next step, it would be great if we could also parallelize at the instance level to speed up the simulation even more.--
This is our current approach (with the current feature request), hopefully it brings something to the table.
Feel free to ask more questions if anything is unclear.mod.simulate() returns a process instead of waiting and terminating the process.
p = mod.simulate(resultfile = RESULT_FILEPATH + resultFileName + ".mat", simflags = simflag)One issue we had was that a process could fail (for no clear reason), so we added this loop to retry up to 4 times:
while not os.path.exists(myFileName) and retryIndex < 4: poll = p.poll() if poll is None: #if process is still running if os.path.exists(myFileName): break else: if os.path.exists(myFileName): break p = mod.simulate(resultfile = RESULT_FILEPATH + resultFileName + ".mat", simflags = simflag) retryIndex += 1We then check that the processes are done before post-processing the results:
if omcProcess != None: poll = omcProcess.poll() if poll is None: omcProcess.wait() omcProcess.terminate() while(True): try: results = DyMat.DyMatFile(RESULT_FILEPATH + resultFileName + ".mat") except: continue break postProcess(results)Reacted by Glucas@FarzanehMousaviM , @kiroorooki could you please test if the changes in PR #312 help you? These should enable you to run a DoE based on OMPython.ModelicaSystemDoE ...
Hello!
I've noticed that we could run simulations in parallel if we handled the process outside the simulate() method.
It is useful to me because I need to run a log of simulations, and running them in parallel saves me a lot of time.
In order to keep the same functionality, I added a boolean handleProcessOutside, and if set to true, I don't wait() and terminate() the process, I simply return it, so the user can handle the process themselves. That way people can use the library the same way, or get extended functionality if needed with just a boolean.
In my case, I run all my simulations, and I make sure all of them are done later when I save my results (opening .mat file, converting things to csv...).
I just realized the library changed quite a bit, but I believe the same functionality can still be implemented in _run_cmd and propagated to the methods using it (simulate and linearized).
Thanks for your hard work, I hope my suggestion makes sense!
This is how I currently implemented it in my version for reference (quite outdated):
`
`