Code generation for ThingClient subclasses

[rwb27]

Currently, ThingClient subclasses are generated on-the-fly from a Thing Description. This "gets the job done" and means it's always possible to have a client that's up to date with the server. However, it would be nice to support static analysis, e.g. type checking, autocompletion, etc. with importable client classes for specific instruments.

Functionality

This PR implements client code generation, specifically:

• A client class is created, subclassing ThingClient and named as per the title of the Thing Description.
• Properties are added using their names from the TD
  • Property types are converted to Python type hints (with very basic conversion)
  • Read-only properties are made read-only. Write-only properties still have a read method.
  • Implementations use get_property, set_property as provided by ThingClient
• Actions are added using names from the TD
  • The input schema is broken down into its properties (i.e. arguments), and each property's schema is converted to an argument with a type hint and default.
  • Required properties of the input schema become required arguments
  • Additional keyword arguments are always accepted and passed to the endpoint - TD DataSchema objects don't distinguish whether extra properties are allowed or not.
  • The output schema is converted to a return type hint
  • The function body calls invoke_action.
  • ... is used to distinguish between explicitly set None values, and unspecified values (i.e. not included in the JSON). This allows the server to fill in default values, which is safer than trying to generate default values in the Python code.
• ast is used to generate a syntax tree that's then turned into code. This has the significant advantage that:
  • the generated code should always be valid Python
  • it should be very robust against attack.

Schema conversion

Conversion from DataSchema to Python types is implemented partially. Types are converted recursively, with support for:

• integer -> int
• number -> float
• boolean -> bool
• string -> str
• anyOf -> Union
• null -> None
• array -> list (if an item type is specified, it's converted recursively) or tuple (if several item types are specified)
• object -> dict[str, Any] or a generated pydantic.BaseModel subclass (if properties are specified)
• Anything else -> Any

object schemas will be recursively converted to pydantic models. This can be turned off by disabling recursion, and more fine-grained control may be helpful.

Still to do

Before this can be merged, we need to add a few things:

• More testing, including all the data types
• Actual use of the created class. This is done, because ThingClient.from_url now uses this module.
• Better deduplication of models.
• Recognition of Blob in the output (rather than autogenerating a model for it).
• It would be nice to check that the methods/properties of the generated client class match the Thing Description of the thing we're connecting to. Extra affordances on the thing could be added dynamically, but it would be good to check the methods/properties of the client are all implemented on the server. This may be left for the future.

[VigneshVSV]

I remember us talking about this. Now that I am somewhat finished with my obligations, I will have deeper look at the latest changes in labthings.

FYI, I already released pydantic based property types from this code base as part of my repository.

[rwb27]

I'm going to bump this to 0.0.13 as it's not yet critical. I am keen to get in in though, as it will really improve the experience of using lab things from e.g. a python notebook.

[barecheck]

Barecheck - Code coverage report

Total: 96.39%

Your code coverage diff: -0.58% ▾

Uncovered files and lines

File	Lines
src/labthings_fastapi/base_descriptor.py	181, 195-196
src/labthings_fastapi/client/__init__.py	41, 77, 80-81, 237, 286-289, 403
src/labthings_fastapi/code_generation.py	125, 138, 169, 286, 290, 310-311, 318, 320, 355-356, 375-377, 450, 455-456, 471-472, 474, 508, 586, 590, 680, 683