Skip to content

Constrain CSV formats supported by ENSRainbow to single-column #1648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
djstrong wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Constrain CSV formats supported by ENSRainbow to single-column #1648

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
765c6c8
chore: update .gitignore and modify CSV conversion tests to enforce s…
djstrong Feb 17, 2026
cabb653
add changeset
djstrong Feb 17, 2026
5004fd6
refactor(tests): remove redundant test for two-column CSV format
djstrong Feb 17, 2026
d197694
update documentation for CSV conversion to specify single-column format
djstrong Feb 17, 2026
c865e21
refactor(tests): remove test for invalid multi-column CSV and enhance…
djstrong Feb 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/single-column-csv.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"ensrainbow": major
---

Constrain CSV input to single-column format (label only). The two-column format (label + labelhash) is no longer supported. All labelhashes are now computed deterministically from labels, removing the risk of incorrect mappings from untrusted labelhash values.
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,6 @@ apps/ensrainbow/data

# fallback-ensapi dist
apps/fallback-ensapi/dist
apps/ensrainbow/data*
apps/ensrainbow/temp*
apps/ensrainbow/v2*
6 changes: 3 additions & 3 deletions apps/ensrainbow/src/cli.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ describe("CLI", () => {
// Successful convert with args (convert-sql always creates version 0)
// To test version 1, we need to use convert command with existing database
// But for this test, we'll create version 0 and then manually test the ingestion failure
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_2col.csv");
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_1col.csv");
const tempDbDirForV1 = join(tempDir, "temp-db-for-v1");
const version0FileForV1 = join(tempDir, "test_ens_names_0_for_v1.ensrainbow");

Expand Down Expand Up @@ -340,7 +340,7 @@ describe("CLI", () => {

// Create an ensrainbow file with label set version 2
// To create version 2, we need to create version 0, ingest it, create version 1, ingest it, then create version 2
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_2col.csv");
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_1col.csv");
const labelSetId = "test-ens-names";

// Create temporary directory for building up versions sequentially
Expand Down Expand Up @@ -453,7 +453,7 @@ describe("CLI", () => {
const thirdInputFile = join(tempDir, "different_label_set_id_1.ensrainbow");

// Create an ensrainbow file with different label set id
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_2col.csv");
const csvInputFile = join(TEST_FIXTURES_DIR, "test_labels_1col.csv");
const labelSetId = "different-label-set-id"; // Different from test-ens-names

// Create temporary directory for version 0 database
Expand Down
116 changes: 12 additions & 104 deletions apps/ensrainbow/src/commands/convert-csv-command.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,53 +63,6 @@ describe("convert-csv-command", () => {
await db.close();
});

it("should convert two column CSV with provided hashes and ingest successfully", async () => {
const inputFile = join(TEST_FIXTURES_DIR, "test_labels_2col.csv");
const outputFile = join(tempDir, "output_2col.ensrainbow");
const dataDir = join(tempDir, "db_2col");

// Convert CSV to ensrainbow format
await convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-csv-two-col" as LabelSetId,
silent: true,
});

// Verify the output file was created
const stats = await stat(outputFile);
expect(stats.isFile()).toBe(true);
expect(stats.size).toBeGreaterThan(0);

// Ingest the converted file into database
const cli = createCLI({ exitProcess: false });
await cli.parse(["ingest-ensrainbow", "--input-file", outputFile, "--data-dir", dataDir]);

const db = await ENSRainbowDB.open(dataDir);
expect(await db.validate()).toBe(true);
const recordsCount = await db.getPrecalculatedRainbowRecordCount();
expect(recordsCount).toBe(10);
expect(
(await db.getVersionedRainbowRecord(labelHashToBytes(labelhash("test123"))))?.label,
).toBe("test123");
expect(await db.getVersionedRainbowRecord(labelHashToBytes(labelhash("1234")))).toBe(null);
await db.close();
});

it("should fail when CSV has inconsistent column count", async () => {
const inputFile = join(TEST_FIXTURES_DIR, "test_labels_invalid_first.csv");
const outputFile = join(tempDir, "output_invalid.ensrainbow");

// Convert CSV to ensrainbow format (should fail on inconsistent columns)
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-csv-invalid" as LabelSetId,
}),
).rejects.toThrow(/Failed on line 1: Expected 1 or 2 col/);
});

it("should handle CSV with special characters, emojis, unicode, and quoted fields", async () => {
const inputFile = join(TEST_FIXTURES_DIR, "test_labels_special_chars.csv");
const outputFile = join(tempDir, "output_special.ensrainbow");
Expand Down Expand Up @@ -150,20 +103,6 @@ describe("convert-csv-command", () => {
expect(await db.getVersionedRainbowRecord(labelHashToBytes(labelhash("1234")))).toBe(null);
await db.close();
});

it("should fail when CSV contains invalid labelhash format", async () => {
const inputFile = join(TEST_FIXTURES_DIR, "test_labels_invalid_hash.csv");
const outputFile = join(tempDir, "output_invalid_hash.ensrainbow");

// Convert CSV to ensrainbow format (should fail on invalid hash format)
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-csv-invalid-hash" as LabelSetId,
}),
).rejects.toThrow(/Failed on line 2: Invalid labelHash/);
});
});

describe("Error handling", () => {
Expand Down Expand Up @@ -562,22 +501,8 @@ describe("convert-csv-command", () => {
it("should process all CSV rows including potential headers", async () => {
const inputFile = join(tempDir, "with_header.csv");
const outputFile = join(tempDir, "output_header.ensrainbow");
const csvContent =
"label,labelhash\nalice,0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501\nbob,0x38e47a7b719dce63662aeaf43440326f551b8a7ee198cee35cb5d517f2d296a2";
await writeFile(inputFile, csvContent);

// Should process the file (header will be treated as a regular row and fail validation)
// Actually, the header row will be processed and fail because "label" is not a valid hex hash
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-header" as LabelSetId,
silent: true,
}),
).rejects.toThrow(/Invalid labelHash/);

// For a proper test, let's create a CSV where the header is valid data
// Single-column CSV where the header is valid data
const csvContentValid = "label\nlabel1\nlabel2";
await writeFile(inputFile, csvContentValid);

Expand All @@ -603,40 +528,22 @@ describe("convert-csv-command", () => {
await db.close();
});

it("should handle CSV with malformed rows (extra columns)", async () => {
it("should reject CSV rows with extra columns", async () => {
const inputFile = join(tempDir, "malformed_extra_cols.csv");
const outputFile = join(tempDir, "output_malformed.ensrainbow");
const csvContent =
"alice\nbob,0x38e47a7b719dce63662aeaf43440326f551b8a7ee198cee35cb5d517f2d296a2,extra\ncharlie";
await writeFile(inputFile, csvContent);

// Should fail when column count is inconsistent
// Should fail because second row has more than 1 column
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-malformed" as LabelSetId,
silent: true,
}),
).rejects.toThrow(/Expected \d+ columns/);
});

it("should handle CSV with malformed rows (missing columns)", async () => {
const inputFile = join(tempDir, "malformed_missing_cols.csv");
const outputFile = join(tempDir, "output_malformed2.ensrainbow");
const csvContent =
"alice,0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501\nbob\ncharlie,0x87a213ce1ee769e28decedefb98f6fe48890a74ba84957ebf877fb591e37e0de";
await writeFile(inputFile, csvContent);

// Should fail when column count is inconsistent
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-malformed2" as LabelSetId,
silent: true,
}),
).rejects.toThrow(/Expected \d+ columns/);
).rejects.toThrow(/Expected 1 column \(label only\)/);
});

it("should handle CSV with quoted fields containing commas", async () => {
Expand Down Expand Up @@ -683,22 +590,23 @@ describe("convert-csv-command", () => {
await db.close();
});

it("should handle CSV with empty labelhash column (should fail validation)", async () => {
const inputFile = join(tempDir, "empty_hash.csv");
const outputFile = join(tempDir, "output_empty_hash.ensrainbow");
it("should reject CSV with two columns (label + labelhash not supported)", async () => {
const inputFile = join(tempDir, "two_columns.csv");
const outputFile = join(tempDir, "output_two_columns.ensrainbow");
const csvContent =
"alice,0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501\nbob,\ncharlie,0x87a213ce1ee769e28decedefb98f6fe48890a74ba84957ebf877fb591e37e0de";
"alice,0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501\nbob,0x38e47a7b719dce63662aeaf43440326f551b8a7ee198cee35cb5d517f2d296a2";
await writeFile(inputFile, csvContent);

// Should fail when labelhash is empty
await expect(
convertCsvCommand({
inputFile,
outputFile,
labelSetId: "test-empty-hash" as LabelSetId,
labelSetId: "test-two-columns" as LabelSetId,
silent: true,
}),
).rejects.toThrow(/LabelHash cannot be empty/);
).rejects.toThrow(
/Expected 1 column \(label only\).*Multi-column CSV formats are not supported/,
);
});
});
});
Loading
Loading