Skip to content

%dds list and get, and valid folders in every folder error (ref #309773) - #104

Merged
oskaresparza merged 13 commits into
developmentfrom
dds_list_get
Oct 21, 2026
Merged

oskaresparza merged 13 commits into
developmentfrom
dds_list_get

Conversation

@oskaresparza

Copy link
Copy Markdown
Collaborator

Ref #309773.

What's in it

  • %dds list(folder, pattern=None, path=None). Shows the file names in a DDS document folder, sorted. path defaults to the %catalog use(...) context.
    • pattern filters the names, ignoring case. It can be a wildcard ("*.pdf", "report_202?.*"), an extension (".xlsx"), text in the name ("report"), or a list of these.
    • It uses DDS's listing route, GET /api/v1/files/{folder}/, and reads every page. The old guessed ?prefix= route got a 307 redirect.
  • %dds get(folder, filename, path=None, output_dir=None, overwrite=True). Downloads one file into the notebook's folder and returns the local file, which open, pandas and %dds push accept as is.
    • A missing file is a short error: no file 'x.pdf' in …/deliverables.
    • DdsController.get returns the file in memory instead.
  • Folder errors. In push, list, get and target, a wrong, quoted or missing folder now lists the valid DdsFolder values and suggests the closest one. This includes a misspelt bare name on a %dds line, which used to end in a NameError traceback.
  • exists() uses HEAD. push(..., overwrite=False) no longer downloads each file just to check whether it exists.
  • Updated %dds help, the README and the notebook example.

Notes

  • DDS's OpenAPI spec doesn't describe the listing's JSON shape. list reads the usual layouts, and if a response is none of them, the error names the keys it found.
  • This branch starts from main, so it also brings main's 0.1.34 and 0.1.35 release commits into development: version bumps and README install lines. It merges cleanly.

Testing

  • New tests for the listing route, pagination, subfolders and listing shapes; the pattern filter; get (download, output_dir, overwrite, missing file); and the folder errors in both the controller and %dds.
  • The full test suite passes locally.
  • In the notebook, against the real DDS, push and then get of an 860 KB GeoPackage worked.

oskaresparza and others added 13 commits October 19, 2026 14:54
staging → main: %dds push, ingest PY-1–PY-13 and the main ingest fix
%gpkg: stream Dremio tables and views to a GeoPackage, pushable with %dds (ref #309764)
Release 0.1.34: %gpkg GeoPackage export, pushable with %dds (ref #309764)
Update README main install example to v0.1.34
…or, ref #309773

- list(folder, pattern=None, path=None): the file names in a DDS folder,
  sorted; pattern filters by wildcard, extension or text, or a list of them.
  Uses DDS's real listing route, GET /api/v1/files/{folder}/, page by page.
- get(folder, filename, path=None, output_dir=None, overwrite=True): download
  one file into the notebook's folder and return the local file; a missing
  file is a short error.
- A wrong, quoted or missing folder names the valid DdsFolder values in push,
  list, get and target, including a misspelt bare name in %dds.
- exists() uses HEAD instead of downloading the file.

Redmine-Hook: v1
@oskaresparza
oskaresparza merged commit 54941b5 into development Oct 21, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant