dataverse-client

A simple client to interact with MS Power Platform Dataverse tables via REST API.

Installation

pip install git+https://github.com/AllenNeuralDynamics/dataverse-client.git

To install a specific version, append @{version} with a tag name. For example with v0.0.0:

pip install git+https://github.com/AllenNeuralDynamics/dataverse-client.git@v0.0.0

Configuration

Configuration is automatically loaded from C:\ProgramData\AllenInstitute\dataverse_client\config.yml. The configuration options and their defaults are listed in the DataverseConfig object in rest_client.py. The file should look like this:

tenant_id: "**********"
client_id: "**********"
org: "**********"
additional_scopes: ["offline_access"]
domain: "*********.org"
username: "**********"
request_timeout_s: 60

To set the password to be used alongside the username for authentication, set an environment variable:

• set DATAVERSE_password=<password>

Environment variables prefixed with DATAVERSE_ can also be used to set the other configuration options, e.g. DATAVERSE_tenant_id.

Usage

See the examples/example_dataverse_client.py for a working usage example.

Once setting up the config file and environment variable,

client = DataverseRestClient() # configuration is automatically loaded

entry = client.get_entry(table, entry_id)

The client provides the following methods:

add_entry(table: str, data: dict) -> dict ...

# id can either be guid or a length-1 dict with an alternate key
get_entry(table: str, id: str | dict) -> dict ...
update_entry(table: str, id: str | dict, update_data: dict,) -> dict: ...

query(
  table: str, 
  filter: Optional[str] = None,  # OData filter query, e.g. "column eq 'value'".
  order_by: Optional[str | list[str]] = None, # Column or list of columns to order by
  top: Optional[int] = None, # Return the top n results
  select: Optional[str | list[str]] = None # Columns to include in the response
) -> list[dict]: ...

REST API and Queries

The DataverseRestClient has some builtin url formatting (_construct_url and _format_queries) that implement odata-style urls.

URLs are formatted like https://<ORG_ID>.crm.dynamics.com/api/data/v9.2/<TABLE><QUERY>

• To fetch an entity by its primary key:
  • https://<ORG_ID>.crm.dynamics.com/api/data/v9.2/<TABLE>({entry_primary_id})
• To fetch an entity by an alternate key:
  • https://<ORG_ID>.crm.dynamics.com/api/data/v9.2/<TABLE>({alt_key_name}={entry_primary_id})
    • Note: string values must include single quotes: e.g. (mouse_id='123456')

To filter and query:

• odata query docs
• https://<ORG_ID>.crm.dynamics.com/api/data/v9.2/<TABLE>?$filter=contains(crb81_mouse_id, 614)
• https://<ORG_ID>.crm.dynamics.com/api/data/v9.2/<TABLE>?$filter=crb81_sex eq 0

Dataverse setup

Permissions

In order to configure table access, set up a security role at admin.powerplatform.microsoft.com - Environment - Settings -Security Roles. Assign the role to your service account

Important note on primary columns

Primary Column listed in the dataverse table is NOT the actual primary key - that's the autogenerated guid column <tablename>_baseid. In order to refer to an entry by a custom value, you must add an Alternate Key to the table (make.powerapps - table - Schema - Keys). This alternate key must be unique; adding a key will fail if duplicate values are found

For instance, the dim_mice table has a dim_mice_baseid column with uids, but we'd like to refer to mice by their 6-7 digit mouse_id. This table should be given an Alternate Key for the mouse_id column, so you can refer to a mouse with (mouse_id='614173')

Level of Support

Please indicate a level of support:

• Supported: We are releasing this code to the public as a tool we expect others to use. Issues are welcomed, and we expect to address them promptly; pull requests will be vetted by our staff before inclusion.
• Occasional updates: We are planning on occasional updating this tool with no fixed schedule. Community involvement is encouraged through both issues and pull requests.
• Unsupported: We are not currently supporting this code, but simply releasing it to the community AS IS but are not able to provide any guarantees of support. The community is welcome to submit issues, but you should not expect an active response.

Development instructions

• Clone the repo
• uv sync