Skip to content

fix(server): keep the hbase meta table when clearing a graph - #3176

Open
bitflicker64 wants to merge 1 commit into
apache:masterfrom
bitflicker64:fix/hbase-truncate-keep-meta
Open

fix(server): keep the hbase meta table when clearing a graph#3176
bitflicker64 wants to merge 1 commit into
apache:masterfrom
bitflicker64:fix/hbase-truncate-keep-meta

Conversation

@bitflicker64

Copy link
Copy Markdown
Contributor

Purpose of the PR

closes #2209

Replaces #2911 by @LYD031106, whose head branch lives on a personal fork and has been inactive since January. The bug is still live on master.

DELETE /graphs/{name}/clear ends in HbaseStore.truncate() for each of the three stores. HbaseSystemStore.tableNames() appends the meta table, so the truncate also wiped the backend version that init() writes there. The next start of the server then failed BackendStoreInfo.checkVersion() with "The backend store version is inconsistent", and the known workarounds were to re-create the backend database and init the graph again, or to insert the version row by hand.

Main Changes

  • HbaseStore: new tableNamesToTruncate(), returning tableNames() by default. truncate() uses it, and the rollback path (enableTables) re-enables exactly the tables it disabled instead of iterating tableNames() again.
  • HbaseSystemStore: overrides tableNamesToTruncate() to return every table but the meta table. Only the meta table is kept. The schema store is untouched, so its counters table is still truncated and schema ids restart from the beginning after a clear, the same as RocksDB (clear(false) + init()). The version row is never deleted and no store init() runs after the truncate, so the version row is never rewritten and there is no window during the clear in which the version check could fail. truncateBackend() waits for the running tasks and takes the graph lock, the same as clearBackend() and initBackend(); a concurrency test for the clear is out of scope for this fix.
  • MultiGraphsTest.testTruncateBackendKeepsVersionAndResetsSchemaIds: opens a graph, writes schema and data, calls truncateBackend(), then checks that checkVersion() still passes, the schema and the data are gone, a vertex label created again gets the same id as before, and the graph is usable. It is part of CoreTestSuite, so it runs on memory, rocksdb and hbase in server-ci.yml. It is skipped on hstore (pd-store-ci.yml): there the schema lives in PD meta, which truncateBackend() does not clear (only clearBackend() does), and IdCounter hands out ids from a cached range, so the schema and id checks do not apply. The HBase change does not touch hstore.

Differences from #2911, on purpose:

  1. fix(server): ensure graph clear API does not clear meta table for hbase #2911 filtered this.tables on HugeType.META, but META is never a key of that map (the meta table is added in HbaseSystemStore.tableNames()), so the filter was a no-op. The fix worked only because the new method read this.tables directly and skipped the tableNames() overrides. That also skipped HbaseSchemaStore.tableNames(), so the counters table was no longer truncated and schema ids kept growing after a clear, which the Critical review on fix(server): ensure graph clear API does not clear meta table for hbase #2911 (only the meta table should be kept) had asked to avoid. Here the exclusion is explicit in the system store and the schema store is not touched.
  2. The HBase-only unit test of fix(server): ensure graph clear API does not clear meta table for hbase #2911 (BaseHbaseUnitTest / HbaseUnitTest) is not carried over. It talked to HBase through raw sessions, asserted only the version and three data rows, could not see the counters regression, swallowed setup failures with a warning, and on a setup failure ran teardown twice, with provider.truncate() (the method under test) outside any try/catch. The core test above covers the same behaviour through the public graph API on every backend, without HBase-specific plumbing, and it also covers the counters.
  3. RocksDB keeps its clear(false) + init() in truncate(): it does not have this bug, and unifying the backends on one approach, as raised on fix(server): ensure graph clear API does not clear meta table for hbase #2911, is a separate refactor. The MySQL backend that the sibling PR fix(server): prevent graph clear API from clearing meta table in MySQL backend storage #2888 targets was removed from master by chore: remove legacy backends #3116.
  4. On the question left open on fix(server): ensure graph clear API does not clear meta table for hbase #2911 on 2026-01-14 (does CI start the backend for backend-specific tests): it depends on the suite. run-unit-test.sh runs UnitTestSuite on the memory leg only, so a backend-specific test there never sees HBase, which is why fix(server): ensure graph clear API does not clear meta table for hbase #2911 needed the Assume. run-core-test.sh runs CoreTestSuite with -P core-test,$BACKEND once per backend of the build-server matrix of server-ci.yml, after install-backend.sh hbase has started HBase 2.6.5. That is why the new test lives in CoreTestSuite and needs no Assume.

Verifying these changes

  • Need tests and can be verified as follows:
    • MultiGraphsTest.testTruncateBackendKeepsVersionAndResetsSchemaIds (new)
    • HBase 2.6.5 standalone (the CI hbase-site.xml apart from the data directories), JDK 11, -P core-test,hbase: on master without the HbaseStore change the test fails right after the truncate, at checkVersion() (stored version null); with the change it passes. Full CoreTestSuite on hbase with the change: 805 run, 0 failures, 0 errors, 52 skipped, 795 s. The skips are the existing hstore-only gates and HBase feature gates (aggregate and olap properties, scan, ramtable, CONTAINS queries): the hbase job on master at 321ba4d reports 804 run and the same 52 skipped.
    • memory and rocksdb, JDK 11, MultiGraphsTest: 11 run, 0 failures, 0 errors, 2 skipped (the two existing hstore-gated tests) on both backends.
    • mvn checkstyle:check on hugegraph-hbase and hugegraph-test: 0 violations.
    • Not run: hstore, where the test skips itself (see above).

Does this PR potentially affect the following parts?

  • Nope

Documentation Status

  • Doc - No Need

Clearing a graph (DELETE /graphs/{name}/clear) calls truncate() on each
of the three stores. HbaseSystemStore.tableNames() appends the meta
table, so the truncate also wiped the backend version that init() had
written there, and the next start of the server failed the version
check with "The backend store version is inconsistent" (apache#2209).

Add HbaseStore.tableNamesToTruncate(), all the tables by default, and
override it in HbaseSystemStore to leave out the meta table only. The
schema store still truncates its counters table, so schema ids restart
from the beginning after a clear, as they do on RocksDB. The rollback
path re-enables the same tables it disabled.

Add a core test that clears a graph and checks that the version check
still passes, the schema and the data are gone, and the schema ids are
handed out again from the start. It runs on every backend of the CI
matrix.

Supersedes apache#2911 by LYD031106.

closes apache#2209
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working store Store module tests Add or improve test cases labels Aug 29, 2026
@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 62.50000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 37.74%. Comparing base (321ba4d) to head (40f7288).

Files with missing lines Patch % Lines
...ache/hugegraph/backend/store/hbase/HbaseStore.java 62.50% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3176      +/-   ##
============================================
+ Coverage     32.66%   37.74%   +5.08%     
- Complexity     5500     6535    +1035     
============================================
  Files           789      800      +11     
  Lines         67703    68886    +1183     
  Branches       8945     9147     +202     
============================================
+ Hits          22116    26004    +3888     
+ Misses        42983    39832    -3151     
- Partials       2604     3050     +446     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:M This PR changes 30-99 lines, ignoring generated files. store Store module tests Add or improve test cases

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

[Bug] The backend store version is inconsistent

1 participant