CLI netlab status allow --json for API handling

[sdargoeuves]

I have no idea if this is something we want to consider. After discovering the new HTTP server, I wanted to test and thought it was a shame to not a have readable output in the browser.

If we are happy to go down this rabbit hole, I don't mind checking what other commands we could json-ify, if not feel free to close this, and leave that for a distant future!

Overview

Added --json flag to netlab status for programmatic access, improving API usability and automation capabilities.

Changes

status.py

• Added --json argument to CLI parser
• Created data collection layer: build_*_payload() functions that construct structured data once
  • build_active_labs_payload() - all labs overview
  • build_lab_detail_payload() - single lab with nodes/tools
  • build_lab_log_payload() - lab event log
• Display functions now consume payloads instead of reimplementing logic
• Fixed duplicate snapshot file reads

api.py

• Using the --json when calling the netlab status command
• Added option to specify instance ID or ALL with /status/<id/all>

Example Usage

CLI

# Table (default)
netlab status

# JSON (new)
netlab status --json
netlab status --json --instance 1
netlab status --json --all

HTTP

[ipspace]

A few initial comments. I think it would make sense if you do another pass and simplify things, and then I'll look at them some more.

Most important: there should be a single data-gathering function, and the resulting data structure should be used directly for the JSON dump (potentially with an extra wrapper around a list), or to generate the text printout.

[ipspace]

This should not be necessary. Hopefully we can agree (between ourselves) what data format to use ;)

[ipspace]

I would love to keep this decision tree clean - one parameter, one action, and then handle the specifics in the action

[ipspace]

Every action should follow the same blueprint -- get data, create a data structure out of that, and print the data structure in table, YAML, or JSON format (see "show" commands).

Having a totally different approach to collect data for the JSON format makes little sense.

[ipspace]

So you're taking a data structure that we built somewhere and create a different data structure for the JSON printout? That makes no sense. We should get it right in the first place.

Also, please use Box, not dict. The code will become much easier to read (that's why I started using Box in the first place). There are two utility functions in netsim.data: get_box to convert a dict into a Box and get_empty_box that returns an empty Box. Do not create boxes by calling Box directly, the two utility functions set just the right combination of flags.

[sdargoeuves]

A few initial comments. I think it would make sense if you do another pass and simplify things, and then I'll look at them some more.

Most important: there should be a single data-gathering function, and the resulting data structure should be used directly for the JSON dump (potentially with an extra wrapper around a list), or to generate the text printout.

Thanks (and sorry) for taking the time to review this.
I got excited about the idea, and carried away trying this out with AI. While it worked, I can now see I’d taken on more than I could chew: I don’t understand the netlab status internals to do it cleanly, and the result is overcomplicated and too hacky.

I’ve got a few other things on the back burner right now, so I’d prefer to park/close this PR for the moment and come back in ~a month to do a clean refactor if you think the idea of adding a --json option is worth it.

Lesson learned: I’ll be more careful about raising this kind of AI experiments on something that seems cool, without correctly understanding the changes.

[ipspace]

Lesson learned: I’ll be more careful about raising this kind of AI experiments on something that seems cool, without correctly understanding the changes.

Been there, done that ;) No worries, I love the idea. I hope you don't mind if I just go ahead and implement it?

[ipspace]

One other thing (I thought I wrote that down, but evidently not) -- these changes would break the exiting API introduced in #2969. @captainpacket -- Are you OK with that, or should we return existing text in "status" field and a copy of the data in JSON-encoded format?

[sdargoeuves]

I love the idea. I hope you don't mind if I just go ahead and implement it?

I'm glad you like the idea as well, and yes, please feel free to implement it! I'll be looking at how you solve this, it should help me better understand that part of the codebase.

these changes would break the exiting API introduced in #2969. @captainpacket -- Are you OK with that, or should we return existing text in "status" field and a copy of the data in JSON-encoded format?

@captainpacket could you please share more information on how you are using this API?

Before seeing that PR, I was thinking it would be nice to have something that lets other team members view lab status and potentially take actions without needing CLI access. Our main lab is currently poorly managed by an unreadably long bash script that gets the job done but isn't ideal. This HTTP server could be a great foundation for building a more user-friendly wrapper around netlab.

That said, this could grow in scope pretty quickly depending on where we want to take it. It comes down to how much we want to solve this problem — is it worth diving deeper into, or are there bigger fish to fry?

[captainpacket]

@sdargoeuves I wrote a small FastAPI wrapper to solve the same problem you did - a lot of bash trickery. The PR I submitted is a simplified version of that; it implements some extra things like multilab, graphs, etc. I'm using a job execution engine (Semaphore UI) with a small client runner script to access the different functions, but it could easily have an actual frontend.

Feel free to use - https://github.com/forwardnetworks/netlab/blob/dev/netlab_api.py