feat: add branch/tag metadata maps and tag timestamps by majin1102 · Pull Request #6364 · lance-format/lance

majin1102 · 2026-03-31T15:22:35Z

What Changed

Changed branch and tag metadata to structured key/value maps.
Added tag timestamp fields:
- createdAt / created_at
- updatedAt / updated_at
Exposed the new tag metadata shape consistently in Rust, Python, and Java.
Updated docs and tests to match the new ref metadata model.

Rust

TagContents now includes:
- metadata: HashMap<String, String>
- created_at: Option<DateTime<Utc>>
- updated_at: Option<DateTime<Utc>>
Tag creation initializes timestamps.
Tag updates preserve created_at and refresh updated_at.
Ref serialization tests now cover:
- map-based metadata
- tag timestamps
- backward compatibility for missing optional fields

Python

Python tag/branch metadata now uses dict-based metadata.
Tag listing APIs now return:
- branch
- version
- created_at
- updated_at
- manifest_size
- metadata
Updated the public Tag TypedDict.
Updated tests for tag metadata and timestamp behavior.
Fixed the Tags.list() docstring to describe the full returned metadata.

Java

Java Tag now includes:
- Map<String, String> metadata
- Optional<Instant> createdAt
- Optional<Instant> updatedAt
JNI tag listing now passes timestamps and metadata maps into Java.
Updated Java tests to validate timestamp and metadata behavior.

Docs

Updated the branch/tag format spec to document tag timestamps and map-based metadata.
Updated the tags and branches guide examples to show the current tag metadata shape.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca167a2b8f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

java/src/main/java/org/lance/Dataset.java

python/python/lance/lance/__init__.pyi

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb7b97bda1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

python/python/lance/dataset.py

codecov · 2026-03-31T16:22:52Z

Codecov Report

❌ Patch coverage is 57.40741% with 46 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance/src/dataset/refs.rs	57.40%	45 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

westonpace

The change seems clean and straightforward. What is your motivation for branch / tag metadata? Arguably this might be a spec change (requiring 3 votes) though it isn't actually modifying any .proto files.

westonpace · 2026-04-03T12:39:44Z

docs/src/format/table/branch_tag.md

+| `createdAt`     | string | Yes      | RFC 3339 timestamp for when the tag was first created.                  |
+| `updatedAt`     | string | Yes      | RFC 3339 timestamp for the latest tag reference update.                 |


Since we have manifest_size should this be created_at and updated_at for consistency?

Or actually, looking at the test, I wonder if this is supposed to be manifestSize below?

westonpace · 2026-04-03T12:53:43Z

python/src/dataset.rs

        for (tag_name, tag_content) in tags {
            let dict = PyDict::new(py);
            dict.set_item("version", tag_content.version)?;
+            dict.set_item("created_at", tag_content.created_at)?;


Do you also need to set branch?

westonpace · 2026-04-03T12:55:21Z

python/python/lance/dataset.py

+        """
+        Replace metadata for an existing tag.
+        """


Can you expand on this a little? Will this merge in the metadata or completely replace any existing metadata? Will this cause updated_at to change?

majin1102 · 2026-04-04T04:09:33Z

The change seems clean and straightforward. What is your motivation for branch / tag metadata? Arguably this might be a spec change (requiring 3 votes) though it isn't actually modifying any .proto files.

Thanks for driving this forward.

For reference, the motivation thread is here:
#6361

And the vote thread is here:
#6374
Would you mind voting there as well?

github-actions bot added enhancement New feature or request python java labels Mar 31, 2026

majin1102 marked this pull request as draft March 31, 2026 15:23

chatgpt-codex-connector bot reviewed Mar 31, 2026

View reviewed changes

java/src/main/java/org/lance/Dataset.java Outdated Show resolved Hide resolved

java/src/main/java/org/lance/Dataset.java Show resolved Hide resolved

python/python/lance/lance/__init__.pyi Outdated Show resolved Hide resolved

majin1102 force-pushed the add_metadata_field branch from ca167a2 to cb7b97b Compare March 31, 2026 15:30

majin1102 marked this pull request as ready for review March 31, 2026 15:34

chatgpt-codex-connector bot reviewed Mar 31, 2026

View reviewed changes

python/python/lance/dataset.py Outdated Show resolved Hide resolved

majin1102 changed the title ~~feat: add metadata field for branches and tags~~ feat: add metadata and timestamp fields for branches and tags Apr 1, 2026

majin1102 force-pushed the add_metadata_field branch from 4bec46e to bcbeed3 Compare April 1, 2026 15:26

majin1102 mentioned this pull request Apr 1, 2026

feat: add timestamps to tag metadata #6331

Closed

feat(refs): add branch/tag metadata maps and tag timestamps

a080281

majin1102 changed the title ~~feat: add metadata and timestamp fields for branches and tags~~ feat: add branch/tag metadata maps and tag timestamps Apr 2, 2026

majin1102 force-pushed the add_metadata_field branch from bcbeed3 to a080281 Compare April 2, 2026 07:39

westonpace approved these changes Apr 3, 2026

View reviewed changes

fix: address tag metadata review feedback

7ab286b

majin1102 force-pushed the add_metadata_field branch from 8999446 to 7ab286b Compare April 4, 2026 05:01

test: fix tag metadata expectation after update

08a5a3b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add branch/tag metadata maps and tag timestamps#6364

feat: add branch/tag metadata maps and tag timestamps#6364
majin1102 wants to merge 3 commits intolance-format:mainfrom
majin1102:add_metadata_field

majin1102 commented Mar 31, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

codecov bot commented Mar 31, 2026 •

edited

Loading

Uh oh!

westonpace left a comment

Uh oh!

westonpace Apr 3, 2026

Uh oh!

westonpace Apr 3, 2026

Uh oh!

westonpace Apr 3, 2026

Uh oh!

majin1102 commented Apr 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		\| `createdAt` \| string \| Yes \| RFC 3339 timestamp for when the tag was first created. \|
		\| `updatedAt` \| string \| Yes \| RFC 3339 timestamp for the latest tag reference update. \|

Conversation

majin1102 commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Changed

Rust

Python

Java

Docs

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

codecov bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

westonpace left a comment

Choose a reason for hiding this comment

Uh oh!

westonpace Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

westonpace Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

westonpace Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

majin1102 commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

majin1102 commented Mar 31, 2026 •

edited

Loading

codecov bot commented Mar 31, 2026 •

edited

Loading

majin1102 commented Apr 4, 2026 •

edited

Loading