Skip to content

docs: add SQL schema documentation - #8493

Merged
link2xt merged 2 commits into
mainfrom
link2xt/sql-schema-docs
Aug 13, 2026
Merged

docs: add SQL schema documentation#8493
link2xt merged 2 commits into
mainfrom
link2xt/sql-schema-docs

Conversation

@link2xt

@link2xt link2xt commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

There is currently no place to document up to date SQL schema. New database is initialized with src/sql/tables.sql and then updated by migrations,
So tracking down how some column is used requires grepping the code, looking for a migration adding it and corresponding commits.

Comment thread docs/schema.sql
Comment thread docs/schema.sql Outdated
Comment thread docs/schema.sql
Comment thread docs/schema.sql Outdated
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 8e4c7f3 to 4e7ee48 Compare July 31, 2026 23:05
@hpk42

hpk42 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Quite like it. I didn't review the comments except in passing (maybe @r10s is also a review authority to reckon with there :), but did a PR #8514 to make the schema become tested in CI.

I wonder if we could evolve to introduce an env DC_DBCREATE_MIGRATIONS=1 or so which would retain today's behaviour, but the default could become creating the DB from the actual documented schema, basically bumping the tables.sql squash point from 68 to current. Not in this PR of course.

@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch 2 times, most recently from 8bafac5 to 7ac86fc Compare August 3, 2026 14:28
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch 3 times, most recently from 5d396e1 to e89e8b2 Compare August 11, 2026 12:32
@link2xt
link2xt marked this pull request as ready for review August 11, 2026 12:32
@link2xt
link2xt marked this pull request as draft August 11, 2026 12:33
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from e89e8b2 to 2f032d3 Compare August 11, 2026 19:32
Comment thread docs/schema.sql
-- For incoming messages, SELF (1).
-- For outgoing messages, ID of the first recipient.
-- TODO: it is not useful in group chats, can it be deprecated?
to_id INTEGER DEFAULT 0,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • for groups to_id is 0 - so it seems not to be needed to really deliver messages.

  • to_id is used to check for "system messages" as to_id == ContactId::INFO - but at a first glance only in combination with from_id == ContactId::INFO - which also makes more sense.

apart from that, i do not know about where it is used in business logic - only that it is set and passed around like crazy :) but maybe i have overseen smth

This comment was marked as outdated.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turned out to be impossible to get rid of to_id for webxdc info messages, they have from_id of the sender and to_id of INFO. from_id is then used to collapse messages by the same author, and to_id is used to trigger displaying the message as an info message.

to_id is used to check for "system messages" as to_id == ContactId::INFO - but at a first glance only in combination with from_id == ContactId::INFO - which also makes more sense.

In all places i have seen the condition is to_id == ContactId::INFO || from_id == ContactId::Info. add_info_msg_with_cmd creates messages with info to_id, but optionally different from_id.

I documented the webxdc usecase instead of trying to get rid of to_id. It is probably possible to still cleanup other usage where to_id is set to the first recipient and document that only values 0 and 2 should be written and this is essentially an info marker. Otherwise all this to_id management makes it look like the first recipient is somehow important, but it is actually not used for anything.

@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 2f032d3 to 62e27c9 Compare August 11, 2026 20:23
@link2xt
link2xt changed the base branch from main to link2xt/mwsmuvztpxpl August 11, 2026 20:23
Comment thread docs/schema.sql
@link2xt

link2xt commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author

#8569 and #8566 deprecate msgs.to_id and msgs.msgrmsg

Comment thread docs/schema.sql
@link2xt
link2xt changed the base branch from link2xt/mwsmuvztpxpl to link2xt/yunrlkmkmwwu August 12, 2026 05:35
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch 2 times, most recently from 86c762e to d4beb82 Compare August 12, 2026 05:56
@link2xt
link2xt force-pushed the link2xt/yunrlkmkmwwu branch from 83e88fc to 8819296 Compare August 12, 2026 05:56
Base automatically changed from link2xt/yunrlkmkmwwu to main August 12, 2026 07:44
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch 2 times, most recently from 54ddfd4 to 5034f45 Compare August 12, 2026 11:20
@link2xt
link2xt marked this pull request as ready for review August 12, 2026 11:21

@hpk42 hpk42 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, just a few nits. definitely a nice improvement.

Comment thread docs/schema.sql Outdated
Comment thread docs/schema.sql Outdated

def normalize(stmt):
stmt = re.sub(r"\s+", " ", stmt).strip()
stmt = stmt.replace("CREATE TABLE IF NOT EXISTS ", "CREATE TABLE ")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this better should handle all "IF NOT EXISTS" forms.

Suggested change
stmt = stmt.replace("CREATE TABLE IF NOT EXISTS ", "CREATE TABLE ")
stmt = re.sub(r"^(CREATE (?:UNIQUE )?(?:TABLE|INDEX|VIEW|TRIGGER)) IF NOT EXISTS ", r"\1 ", stmt)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left it as is, we will notice when this breaks if some migration adds new syntax. It's unlikely we will add views or triggers. Unique index maybe, but then the script can be tweaked as needed.

link2xt and others added 2 commits August 13, 2026 11:51
There is currently no place to document up to date SQL schema.
New database is initialized with src/sql/tables.sql
and then updated by migrations,
So tracking down how some column is used requires grepping the code,
looking for a migration adding it and corresponding commits.

Co-Authored-By: bi酶rn <r10s@b44t.com>
finds things like "exists in the db, but is not documented" or "documented but not in the db".
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 5034f45 to ff750e2 Compare August 13, 2026 11:51
@link2xt
link2xt merged commit 56a99c2 into main Aug 13, 2026
30 checks passed
@link2xt
link2xt deleted the link2xt/sql-schema-docs branch August 13, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants