MDEV-34805 provide various information about vector indexes by shabbann · Pull Request #4955 · MariaDB/server

shabbann · 2026-04-18T00:31:41Z

Adds INFORMATION_SCHEMA.VECTOR_INDEXES table exposing statistics for MHNSW vector indexes via a MYSQL_INFORMATION_SCHEMA_PLUGIN registered alongside the existing mhnsw daemon plugin. columns covered: (TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, VECTOR_DIMENSIONS, SUBDIST_ENABLED, INDEX_SIZE, TOTAL_NODES, DELETED_ROWS, MEMORY_SIZE, CACHE_OVERFLOWS).

Copilot

Pull request overview

Adds an INFORMATION_SCHEMA.VECTOR_INDEXES table to expose runtime/disk statistics for MHNSW vector indexes, implemented as a MYSQL_INFORMATION_SCHEMA_PLUGIN registered alongside the existing mhnsw daemon plugin.

Changes:

Add INFORMATION_SCHEMA.VECTOR_INDEXES schema-table plugin and row population logic in sql/vector_mhnsw.cc.
Track MHNSW cache overflow events and expose cache memory/node statistics.
Add MTR coverage for the new I_S table and update expected outputs.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
sql/vector_mhnsw.cc	Adds I_S plugin/table definition + row filler; tracks cache overflows and exposes cache stats.
mysql-test/main/vector.test	Adds end-to-end test coverage for `INFORMATION_SCHEMA.VECTOR_INDEXES`.
mysql-test/main/vector.result	Captures expected output for the new test section.
mysql-test/main/information_schema.result	Updates global I_S table list expectations to include `VECTOR_INDEXES`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

vuvova · 2026-06-12T17:01:07Z

+  }
+  size_t cache_mem_size()
+  {
+    return root_size(&root);


but to avoid taking the lock twice, I'd introduce a method, like info() that takes a lock, reads all values you need, and releases the lock. once.

gkodinov

Thank you for your contribution! This is a preliminary review.

Please:

squash the commits into a single commit and have it have a commit message according to the MariaDB condig standards. There's 2 commits currently.
make sure the build-bot tests are passing (or at least, not failing because of your changes). Right now there's at least 2 test failures that are related.

shabbann · 2026-04-27T11:44:25Z

squash the commits into a single commit and have it have a commit message according to the MariaDb condig standards. There's 2 commits currently.

Hi George, Sergei actually asked me (in Zulip) to keep the commits separate to make reviewing easier. I will squash at the end

make sure the build-bot tests are passing (or at least, not failing because of your changes). Right now there's at least 2 test failures that are related.

I'm working on it

gkodinov

LGTM. Please keep working with Serg on the final review.

vuvova

See comments below. If you'd let me, I can apply these changes myself so that you could concentrate on your GSoC project. This way we'll have this ready in time for a 13.1 preview, there're only few days left.

vuvova · 2026-06-12T17:01:07Z

+  }
+  size_t cache_mem_size()
+  {
+    return root_size(&root);


but to avoid taking the lock twice, I'd introduce a method, like info() that takes a lock, reads all values you need, and releases the lock. once.

shabbann · 2026-06-12T20:09:59Z

@vuvova I tried to fix them quickly. Feel free to ask me for any edits, or you can do it yourself if it will be quicker.

gkodinov

Please consider rebasing your PR on top of the main branch and resolving the conflicts mentioned.

CLAassistant · 2026-06-17T10:57:34Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ shabbann
❌ vuvova
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

vuvova · 2026-06-17T10:59:08Z

Thanks! I've squashed your commits and added post-review commit on top with my (hopefully not controversial) fixes. I intentionally kept it separate to keep your changes clean.

Did not rebase on top of main though

vuvova

looks good to me, but needs to be rebased on top of main

gkodinov · 2026-06-18T07:35:23Z

I'll do the merge. hang on.

gkodinov

LGTM. now in testing.

shabbann · 2026-06-18T10:32:39Z

sorry I couldn't do the rebase yesterday because I was busy with my graduation project. I will check the build issues

shabbann · 2026-06-18T15:35:37Z

The test main.events_processlist (the one that failed in the buildbot) works fine on my device so I think the issue is from the buildbot. @gkodinov could you restart it please?

gkodinov · 2026-06-19T06:51:43Z

The test main.events_processlist (the one that failed in the buildbot) works fine on my device so I think the issue is from the buildbot. @gkodinov could you restart it please?

yes, this is a know unstable test. I've restarted it.

shabbann · 2026-07-01T17:17:07Z

@vuvova if you aren't going to do any more changes I could pull and '--record' the tests to fix the buildbot issues

gkodinov · 2026-08-11T12:11:11Z

FYI: According to our development cycle all testing of features must be completed before 30 Apr, 30 Jul, 30 Nov and 31 Jan (approximate release dates). You might get comments from the tester before that date or your feature will be approved as is and merged. But it'll all likely happen closer to the release date, not right now.

Adds INFORMATION_SCHEMA.VECTOR_INDEXES table exposing statistics for MHNSW vector indexes via a MYSQL_INFORMATION_SCHEMA_PLUGIN registered alongside the existing mhnsw daemon plugin. columns covered: (TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, VECTOR_DIMENSIONS, SUBDIST_ENABLED, INDEX_SIZE, TOTAL_NODES, DELETED_ROWS, MEMORY_SIZE, CACHE_OVERFLOWS).

* keep `vec_len >= subdist_part * 2` logic in one place only * keep "distance-greater-than" mode logic in one place only * simplify VECTOR_DIMENSIONS (no need to have a special ctx->vec_len path if the other one always works) * new plugin = maturity beta * remove redundant casts, etc * moved vector_indexes_fields_enum to the global scope to use it for setting schema->idx_field1/schema->idx_field2 * open the hlindex graph table, if needed, otherwise most values are unknown unless a user did vector search before * added TABLE_CATALOG column * remove CACHE_OVERFLOWS column, doesn't work as implemented, the fix is complex and isn't worth it in the test: * prefer query_vertical for readability * select all columns at least once * select INDEX_SIZE even if engine-dependent, use rdiff files * test how get_all_tables only open one specific table, and even only .frm file, if possible

shabbann force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch 2 times, most recently from 2591dd6 to da6f17d Compare April 18, 2026 01:22

vuvova requested review from Copilot and vuvova April 18, 2026 11:27

Copilot started reviewing on behalf of vuvova April 18, 2026 11:27 View session

Copilot AI reviewed Apr 18, 2026

View reviewed changes

gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Apr 20, 2026

gkodinov requested changes Apr 27, 2026

View reviewed changes

itzanway mentioned this pull request Apr 28, 2026

MDEV-39196: SELECT from information schema fails when FederatedX loses underlying table #4876

Draft

gkodinov approved these changes May 7, 2026

View reviewed changes

gkodinov assigned vuvova May 7, 2026

vuvova requested changes Jun 12, 2026

View reviewed changes

shabbann commented Jun 12, 2026

View reviewed changes

Comment thread sql/vector_mhnsw.cc Outdated

shabbann commented Jun 12, 2026

View reviewed changes

Comment thread sql/vector_mhnsw.cc Outdated

gkodinov requested changes Jun 15, 2026

View reviewed changes

vuvova force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch from 2f39416 to d0e8c40 Compare June 17, 2026 10:57

vuvova approved these changes Jun 17, 2026

View reviewed changes

gkodinov force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch from 7e6ed9b to 5dae70e Compare June 18, 2026 09:05

gkodinov approved these changes Jun 18, 2026

View reviewed changes

gkodinov assigned elenst and unassigned vuvova Jun 18, 2026

vuvova force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch from 0e0e88d to b9d7dd0 Compare July 1, 2026 16:32

vuvova force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch 3 times, most recently from 0a0c3e0 to 994c3bd Compare August 14, 2026 09:52

shabbann and others added 2 commits August 14, 2026 14:21

vuvova force-pushed the MDEV-34805-provide-various-information-about-vector-indexes branch from 994c3bd to 7a32192 Compare August 14, 2026 12:21

Uh oh!

Uh oh!

Conversation

shabbann commented Apr 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

vuvova Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

shabbann commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

vuvova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vuvova Jun 12, 2026

Choose a reason for hiding this comment

Uh oh!

shabbann commented Jun 12, 2026

Uh oh!

Uh oh!

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vuvova commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vuvova left a comment

Choose a reason for hiding this comment

Uh oh!

gkodinov commented Jun 18, 2026

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

shabbann commented Jun 18, 2026

Uh oh!

shabbann commented Jun 18, 2026

Uh oh!

gkodinov commented Jun 19, 2026

Uh oh!

shabbann commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gkodinov commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

6 participants

shabbann commented Apr 27, 2026 •

edited

Loading

CLAassistant commented Jun 17, 2026 •

edited

Loading

vuvova commented Jun 17, 2026 •

edited

Loading

shabbann commented Jul 1, 2026 •

edited

Loading