Skip to content

Implement UUID proposer, fix compile_timestampltz_ddb and pyproject.toml - #169

Merged
yhong123 merged 4 commits into
snowflakefrom
yhong123/149-uuid_proposer
Nov 12, 2026
Merged

yhong123 merged 4 commits into
snowflakefrom
yhong123/149-uuid_proposer

Conversation

@yhong123

@yhong123 yhong123 commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a generator and proposer for UUID primary keys, so columns that hold UUIDs, including databases like Snowflake that have no native UUID column type and store them as plain strings, get sensible synthetic data instead of a generic string/password generator.

Fixes SAFEHR-data/datafaker#149

UUID proposer (datafaker/proposers/uuid.py, new)

  • UuidProposer generates fresh, random version-4 UUIDs (generic.cryptographic.uuid), treated as structurally fresh/unique for ranking purposes.
  • UuidProposerFactory offers it for native UUID types (SQLAlchemy Uuid, PostgreSQL UUID, MS-SQL UNIQUEIDENTIFIER), and for String primary keys without a native type (e.g. Snowflake) by sampling real values and checking the UUID pattern — avoiding false positives from a coincidentally UUID-length natural key, and catching unconstrained-length columns a length-only check would miss. Falls back to a length heuristic only when there’s no data to sample.
  • Wired into everything_factory.

Default generator fallback (datafaker/make.py)

  • _string_generator now also treats a 32/36-char String PK as a UUID when building the no-data config skeleton.

Snowflake TIMESTAMP_LTZ fix (datafaker/dialects.py)

  • Fixed infinite recursion: the snowflake-specific compiler was calling compiler.process() on its own element, re-entering itself. Now calls compiler.visit_TIMESTAMP_LTZ directly.

Dependency fix (pyproject.toml/poetry.lock)

  • Added snowflake-connector-python[secure-local-storage] (pulls in keyring) to silence the “keyring not installed” warning on every Snowflake connection. Purely additive lock change.

Tests

  • New tests/test_proposers_uuid.py, plus added coverage in tests/test_make.py.

@yhong123 yhong123 self-assigned this Nov 9, 2026

@tim-band tim-band left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good. Thanks!

Comment thread datafaker/proposers/uuid.py Outdated
``as_generic``) are recognised directly; a plain ``String`` *primary key*
column is checked against a sample of its real values instead, matching each
one against the canonical UUID pattern. Fixed column length alone is too
weak a signal: an unrelated natural key can coincidentally share a UUID's

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say these last two sentences are not the best thing for a docstring, but more a comment in the code. Doesn't really matter though.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have moved the last two sentences next to the code it explains: uuid.py:128-131

Comment thread datafaker/dialects.py
) -> str:
"""Render TIMESTAMP_LTZ as TIMESTAMP_LTZ."""
return compiler.process(element, **kw)
return compiler.visit_TIMESTAMP_LTZ(element, **kw)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, OK, this is nice.

Comment thread datafaker/make.py Outdated
# type and reflect a UUID primary key as a plain fixed-length string, so
# this is the only signal available (from column metadata alone, with no
# access to the real data) that such a column actually holds UUIDs.
_UUID_STRING_LENGTHS = frozenset({32, 36})

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love that this is repeated from uuid.py, but I see why you might not have wanted to add a dependency. Alternatively, just removing these changes would be OK. Not sure.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve removed the make.py change and the three related tests in test_make.py. I checked, and propose correctly recommends the UUID proposer for a Snowflake string UUID column without it, so the length-based default wasn’t needed. UUID detection now lives only in the proposer, which samples the real values instead of relying on length.

Comment thread tests/test_proposers_uuid.py Outdated
from tests.utils import DatafakerTestCase


class UuidProposerUnitTests(DatafakerTestCase):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these test don't really test anything. What would we do if one failed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve removed those tests because they only restated constants.

@nikk-nikaznan nikk-nikaznan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I managed to run with snowflakes, and it works

Comment thread datafaker/proposers/uuid.py Resolved
yhong123 and others added 2 commits November 11, 2026 17:18
Co-authored-by: nikk-nikaznan <48319650+nikk-nikaznan@users.noreply.github.com>
@yhong123

Copy link
Copy Markdown
Collaborator Author

I’ve also included a small fix in datafaker/evaluators/feature_extractors.py.

Problem: running propose on a column with a native UUID type (tested on DuckDB and PostgreSQL with id UUID PRIMARY KEY) crashed with:

Binder Error: No function matches the given name and argument types 'length(UUID)'

ColumnEvaluator sends a native UUID column down the string path, and it gets the SHORT_TEXT profile. That profile’s extractors build SQL with length(column) and substr(column, …), which DuckDB and PostgreSQL don’t accept for a UUID column. Snowflake UUID columns weren’t affected because they’re stored as VARCHAR.

Fix: LengthExtractor, FirstLetterExtractor and LastLetterExtractor now cast the column to String before applying length/substr. For real string columns the cast changes nothing, and it’s the same approach the word/sentence extractors already use for tsvector columns. After the fix, propose on a native UUID column ranks generic.cryptographic.uuid as expected.

@yhong123
yhong123 merged commit 38cd58f into snowflake Nov 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants