+ 
-
+ 
@@ -27,15 +27,15 @@
> [!NOTE]
-> HashPrep is now in its first stable release (v0.1.0). Core features are fully tested with CI.
+> Tablint 0.1.0b1 is the first public beta. `critical` means "do not train this table as-is."
## Overview
-**HashPrep** is a Python library for intelligent dataset profiling and debugging that acts as a comprehensive pre-training quality assurance tool for machine learning projects.
+**Tablint** is a Python library for intelligent dataset profiling and debugging that acts as a comprehensive pre-training quality assurance tool for machine learning projects.
Think of it as **"Pandas Profiling + PyLint for datasets"**, designed specifically for machine learning workflows.
It catches critical dataset issues before they derail your ML pipeline, explains the problems, and suggests context-aware fixes.
-If you want, HashPrep can even apply those fixes for you automatically.
+If you want, Tablint can even apply those fixes for you automatically.
---
@@ -48,7 +48,7 @@ Key features include:
- **ML-Specific Checks**: Identify data leakage, dataset drift, class imbalance, and high-cardinality features.
- **Automated Preparation**: Get suggestions for encoding, imputation, scaling, and transformations.
- **Rich Reporting**: Generate statistical summaries and exportable reports (HTML/PDF/Markdown/JSON) with embedded visualizations.
-- **Production-Ready Pipelines**: Output reproducible cleaning and preprocessing code (`fixes.py`) that integrates seamlessly with ML workflows.
+- **Reviewed preparation code**: Write a pandas cleaning script and a scikit-learn pipeline. Fit the pipeline on the training split. The pandas script computes statistics from the frame you pass in.
- **Modern Themes**: Choose between "Minimal" (professional) and "Neubrutalism" (bold) report styles.
---
@@ -57,7 +57,7 @@ Key features include:
### Using pip
```bash
-pip install hashprep
+pip install tablint
```
### Using uv (recommended)
@@ -65,29 +65,29 @@ pip install hashprep
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
-# Install hashprep
-uv pip install hashprep
+# Install tablint
+uv pip install tablint
# Or for development from source
-git clone https://github.com/cachevector/hashprep.git
-cd hashprep
+git clone https://github.com/cachevector/tablint.git
+cd tablint
uv sync
```
-After installation, the `hashprep` command will be available directly in your terminal.
+After installation, the `tablint` command will be available directly in your terminal.
---
## Usage
-HashPrep can be used both as a **command-line tool** and as a **Python library**.
+Tablint can be used both as a **command-line tool** and as a **Python library**.
### CLI Usage
#### 1. Quick Scan
Get a quick summary of critical issues in your terminal.
```bash
-hashprep scan dataset.csv
+tablint scan dataset.csv
```
**Options:**
@@ -97,36 +97,38 @@ hashprep scan dataset.csv
- `--target COLUMN`: Specify target column for ML-specific checks
- `--checks CHECKS`: Run specific checks (comma-separated)
- `--comparison FILE`: Compare with another dataset for drift detection
-- `--sample-size N`: Limit analysis to N rows
-- `--no-sample`: Disable automatic sampling
+- `--preset {gate,profile}`: `gate` is the default and is the training-blocker set. `profile` runs every check.
+- `--fail-on {critical,warning}`: Exit 2 when that severity is present. Exit 0 means the bar was not hit. Exit 1 is a usage or runtime error.
+- `--sample-size N`: Analyze at most N rows. Sampling stays off unless this is set.
+- `--no-sample`: Disable sampling even when `--sample-size` is set.
- `--config FILE`: Load thresholds from a YAML/TOML/JSON config file
**Example:**
```bash
-# Scan with target column and specific checks
-hashprep scan train.csv --target Survived --checks outliers,high_missing_values,class_imbalance
+# Block a training job when the table is not trainable
+tablint scan train.csv --target Survived --preset gate --fail-on critical
# Quick scan with JSON output
-hashprep scan dataset.csv --json --quiet
+tablint scan dataset.csv --json --quiet
```
#### 2. Detailed Analysis
Get comprehensive details about all detected issues.
```bash
-hashprep details dataset.csv
+tablint details dataset.csv
```
**Options:** Same as `scan` command (including `--config`)
**Example:**
```bash
-hashprep details train.csv --target Survived
+tablint details train.csv --target Survived
```
#### 3. Generate Reports
Generate comprehensive reports in multiple formats with visualizations.
```bash
-hashprep report dataset.csv --format html --theme minimal
+tablint report dataset.csv --format html --theme minimal
```
**Options:**
@@ -139,43 +141,44 @@ hashprep report dataset.csv --format html --theme minimal
- `--target COLUMN`: Specify target column
- `--checks CHECKS`: Run specific checks
- `--comparison FILE`: Compare with another dataset for drift detection
-- `--sample-size N`: Limit analysis to N rows
-- `--no-sample`: Disable automatic sampling
+- `--preset {gate,profile}`: `profile` is the default for reports. `gate` runs only training blockers.
+- `--sample-size N`: Analyze at most N rows. Sampling stays off unless this is set.
+- `--no-sample`: Disable sampling even when `--sample-size` is set.
- `--config FILE`: Load thresholds from a YAML/TOML/JSON config file
**Examples:**
```bash
# Generate HTML report with minimal theme
-hashprep report dataset.csv --format html --theme minimal --full
+tablint report dataset.csv --format html --theme minimal --full
# Generate PDF report without visualizations (faster)
-hashprep report dataset.csv --format pdf --no-visualizations
+tablint report dataset.csv --format pdf --no-visualizations
# Generate report with automatic fix scripts
-hashprep report dataset.csv --with-code
+tablint report dataset.csv --with-code
# Generate report with custom output path
-hashprep report dataset.csv --format html --output my_reports/analysis.html
+tablint report dataset.csv --format html --output my_reports/analysis.html
# This creates:
-# - dataset_hashprep_report.md (or .html/.pdf/.json)
-# - dataset_hashprep_report_fixes.py (pandas script)
-# - dataset_hashprep_report_pipeline.py (sklearn pipeline)
+# - dataset_tablint_report.md (or .html/.pdf/.json)
+# - dataset_tablint_report_fixes.py (pandas script)
+# - dataset_tablint_report_pipeline.py (sklearn pipeline)
# Compare two datasets for drift detection
-hashprep report train.csv --comparison test.csv --format html
+tablint report train.csv --comparison test.csv --format html
```
#### 4. List Available Checks
-Discover all data quality checks that HashPrep can perform.
+Discover all data quality checks that Tablint can perform.
```bash
-hashprep checks
+tablint checks
```
#### 5. Version
-Check HashPrep version.
+Check Tablint version.
```bash
-hashprep version
+tablint version
```
#### Available Checks
@@ -218,7 +221,7 @@ hashprep version
#### Basic Analysis
```python
import pandas as pd
-from hashprep import DatasetAnalyzer
+from tablint import DatasetAnalyzer
# Load your dataset
df = pd.read_csv("dataset.csv")
@@ -283,7 +286,7 @@ summary = analyzer.analyze()
#### Generate Reports Programmatically
```python
-from hashprep.reports import generate_report
+from tablint.reports import generate_report
# Analyze dataset
analyzer = DatasetAnalyzer(df, include_plots=True)
@@ -325,10 +328,10 @@ generate_report(
#### Generate Fix Scripts
```python
-from hashprep.checks.core import Issue
-from hashprep.preparers.codegen import CodeGenerator
-from hashprep.preparers.pipeline_builder import PipelineBuilder
-from hashprep.preparers.suggestions import SuggestionProvider
+from tablint.checks.core import Issue
+from tablint.preparers.codegen import CodeGenerator
+from tablint.preparers.pipeline_builder import PipelineBuilder
+from tablint.preparers.suggestions import SuggestionProvider
# After running analysis
analyzer = DatasetAnalyzer(df, target_col='target')
@@ -361,17 +364,17 @@ with open('pipeline.py', 'w') as f:
#### Load Config from File
```python
-from hashprep.utils.config_loader import load_config
-from hashprep import DatasetAnalyzer
+from tablint.utils.config_loader import load_config
+from tablint import DatasetAnalyzer
# Load thresholds from YAML, TOML, or JSON
-config = load_config("hashprep.yaml") # or .toml / .json
+config = load_config("tablint.yaml") # or .toml / .json
analyzer = DatasetAnalyzer(df, config=config)
summary = analyzer.analyze()
```
-Example `hashprep.yaml`:
+Example `tablint.yaml`:
```yaml
missing_values:
warning: 0.3
@@ -386,7 +389,7 @@ Only the keys you specify are overridden; all others fall back to defaults.
#### Custom Sampling
```python
-from hashprep.utils.sampling import SamplingConfig
+from tablint.utils.sampling import SamplingConfig
# Configure sampling for large datasets
sampling_config = SamplingConfig(max_rows=10000)
@@ -406,6 +409,35 @@ if 'sampling_info' in summary:
---
+## Production gate
+
+`DatasetAnalyzer` defaults to the `gate` preset. `tablint report` defaults to `profile`.
+
+`critical` is reserved for defects that should stop a training job: an empty table, an all-null column, missingness at or above the critical threshold, infinite values, duplicate rows above the critical ratio, a column identical or nearly identical to the target, and drift whose effect size is large. Skew, non-normality, ordinary correlation with the target, and zero-inflation of a binary target are warnings, and most of them run only in the `profile` preset.
+
+Stable keys in `analyze()` and in `scan --json` / `details --json`:
+
+- `schema_version` (`"0.2"`), `critical_count`, `warning_count`, `total_issues`
+- `issues[]`: `category`, `severity`, `column`, `description`, `impact_score`, `quick_fix`, `statistic`, `effect_size`
+- `sampling_info`
+
+`summaries` and plots are best-effort and can change without a major version.
+
+```bash
+tablint scan train.csv --target Survived --preset gate --fail-on critical --json
+```
+
+Exit 2 when `--fail-on` matches. A job that only checks the exit code should pass `--fail-on critical`.
+
+### Limitations
+
+- Input is CSV, or Parquet when `tablint[parquet]` is installed. HTML, PDF, and plots need `tablint[reports]`.
+- Leakage means near-deterministic association with the target, not a proof that a column was built from the label.
+- Sampling is off unless you pass `--sample-size` or `auto_sample=True`. Checks then describe the analyzed frame, which is the full table by default.
+- Generated pandas code learns fill values from the frame you pass. Pass the training split, or fit the generated sklearn pipeline on that split.
+
+---
+
## License
This project is licensed under the [**MIT License**](./LICENSE).
@@ -414,7 +446,7 @@ This project is licensed under the [**MIT License**](./LICENSE).
## Contributing
-We welcome contributions from the community to make HashPrep better!
+We welcome contributions from the community to make Tablint better!
Before you get started, please:
diff --git a/RELEASE_TODO.md b/RELEASE_TODO.md
index 5c3bb49..b0620ea 100644
--- a/RELEASE_TODO.md
+++ b/RELEASE_TODO.md
@@ -1,10 +1,10 @@
-# HashPrep Official Release TODO
+# Tablint Official Release TODO
## Release Scope
- [x] Decide the official version target, likely `0.1.0` unless the release should signal `1.0.0`.
- [x] Freeze public API expectations for `DatasetAnalyzer`.
-- [x] Freeze public API expectations for `HashPrepConfig`.
+- [x] Freeze public API expectations for `TablintConfig`.
- [x] Freeze public API expectations for `load_config`.
- [x] Freeze public API expectations for `generate_report`.
- [x] Freeze CLI behavior for `scan`, `details`, `report`, and `version`.
@@ -65,11 +65,11 @@
### Strongly Recommended
-- [x] Add an `--output` option to `hashprep report`.
-- [x] Allow `hashprep report data.csv --format html --output reports/data.html`.
-- [x] Add machine-readable JSON output for `hashprep details`.
-- [x] Add check discovery through a command such as `hashprep checks`.
-- [x] Alternatively add check discovery through an option such as `hashprep scan --list-checks`.
+- [x] Add an `--output` option to `tablint report`.
+- [x] Allow `tablint report data.csv --format html --output reports/data.html`.
+- [x] Add machine-readable JSON output for `tablint details`.
+- [x] Add check discovery through a command such as `tablint checks`.
+- [x] Alternatively add check discovery through an option such as `tablint scan --list-checks`.
- [x] Document why issues are classified as `critical` versus `warning`.
- [x] Review whether PDF/reporting dependencies should move to optional extras in a future release.
- [x] Document any dependency-extra plan if it is deferred.
@@ -84,7 +84,7 @@
## Packaging
-- [x] Update `hashprep/__init__.py` from `0.1.0b3` to the official release version.
+- [x] Update `tablint/__init__.py` from `0.1.0b3` to the official release version.
- [x] Update the beta note in `README.md`.
- [x] Update the beta support table in `SECURITY.md`.
- [x] Add or verify PyPI classifiers in `pyproject.toml`.
@@ -96,10 +96,10 @@
- [x] Build wheel distribution.
- [x] Inspect built package artifacts.
- [x] Install the built wheel in a clean environment.
-- [x] Smoke test `import hashprep` from the built wheel.
-- [x] Smoke test `hashprep version` from the built wheel.
-- [x] Smoke test `hashprep scan datasets/train.csv` from the built wheel.
-- [x] Smoke test `hashprep report datasets/train.csv --format html` from the built wheel.
+- [x] Smoke test `import tablint` from the built wheel.
+- [x] Smoke test `tablint version` from the built wheel.
+- [x] Smoke test `tablint scan datasets/train.csv` from the built wheel.
+- [x] Smoke test `tablint report datasets/train.csv --format html` from the built wheel.
- [x] Confirm `MANIFEST.in` excludes dev/demo files intentionally.
- [x] Confirm `MANIFEST.in` includes all runtime files needed by reports and templates.
@@ -165,15 +165,15 @@
- [x] Tag the release, for example `v0.1.0`.
- [x] Publish to PyPI.
- [x] Create GitHub release with release notes.
-- [x] Verify public install with `pip install hashprep`.
-- [x] Verify public CLI with `hashprep version`.
+- [x] Verify public install with `pip install tablint`.
+- [x] Verify public CLI with `tablint version`.
- [x] Announce the release as the first stable release after alpha and beta.
## Known Immediate Gaps
- [x] Version is still `0.1.0b3`.
- [x] `README.md` still says beta.
-- [x] Website hero still says beta and shows `hashprep-0.1.0b3`.
+- [x] Website hero still says beta and shows `tablint-0.1.0b3`.
- [x] `SECURITY.md` only describes beta support.
- [x] `CHANGELOG.md` is not present.
- [x] CI does not currently build or check publish artifacts.
diff --git a/SECURITY.md b/SECURITY.md
index 4255a0d..3c900f6 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -2,23 +2,21 @@
## Supported Versions
-hashprep has reached a stable `0.1.0` release. Only the latest minor release is supported for security updates.
+Only the latest minor release is supported for security updates.
| Version | Supported |
| ---------- | ------------------ |
-| `0.1.0` | :white_check_mark: |
-| `< 0.1.0` | :x: |
-
-Once hashprep reaches a stable `0.1.0` release, this table will be updated to reflect supported minor versions.
+| `0.1.0b1` | :white_check_mark: |
+| `< 0.1.0b1`| :x: |
## Reporting a Vulnerability
-If you believe you have found a security vulnerability in hashprep (the Python package, the CLI, or the documentation website under `web/`), **please do not open a public GitHub issue**.
+If you believe you have found a security vulnerability in tablint (the Python package, the CLI, or the documentation website under `web/`), **please do not open a public GitHub issue**.
Instead, report it privately through one of the following channels:
-- **GitHub Private Vulnerability Reporting** β preferred. Open a report at VERSION: 0.1.0a4 VERSION: {{ version }} VERSION: {{ version }}
-
- HashPrep // Report
+
+
+
+
Tablint // Report
@@ -2846,13 +2849,13 @@
@@ -116,7 +116,8 @@ def _minimal_template(self):
Tail (last 5 r
- HashPrep
+ {{ logo_svg }}
+ Tablint
v{{ version }}
@@ -281,7 +282,7 @@ def _minimal_template(self):
{% if not pdf_mode %}
@@ -678,7 +679,7 @@ def _minimal_template(self):
@@ -731,9 +732,12 @@ def _neubrutalism_template(self):
-
- HashPrep // Report
-
+ {{ logo_svg }}
+
+
Tablint // Report
+
@@ -1172,7 +1176,7 @@ def _neubrutalism_template(self):
{% endif %}
@@ -1184,7 +1188,7 @@ def _neubrutalism_template(self):
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
- a.download = 'hashprep_config.json';
+ a.download = 'tablint_config.json';
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
diff --git a/hashprep/reports/json.py b/tablint/reports/json.py
similarity index 98%
rename from hashprep/reports/json.py
rename to tablint/reports/json.py
index ec9b92a..79b6035 100644
--- a/hashprep/reports/json.py
+++ b/tablint/reports/json.py
@@ -3,7 +3,7 @@
import numpy as np
-import hashprep
+import tablint
def json_numpy_handler(obj):
@@ -24,7 +24,7 @@ def generate(self, summary, full=False, output_file=None):
report: dict = {
"metadata": {
"generated": datetime.now().isoformat(),
- "version": hashprep.__version__,
+ "version": tablint.__version__,
},
"dataset_overview": {
"statistics": {
diff --git a/tablint/reports/logo.py b/tablint/reports/logo.py
new file mode 100644
index 0000000..9517e23
--- /dev/null
+++ b/tablint/reports/logo.py
@@ -0,0 +1,22 @@
+"""The Tablint mark, inlined so reports render without fetching a file."""
+
+from markupsafe import Markup
+
+LOGO_SVG = Markup(
+ """"""
+)
diff --git a/hashprep/reports/markdown.py b/tablint/reports/markdown.py
similarity index 98%
rename from hashprep/reports/markdown.py
rename to tablint/reports/markdown.py
index 1af75c4..d09b028 100644
--- a/hashprep/reports/markdown.py
+++ b/tablint/reports/markdown.py
@@ -3,7 +3,7 @@
import pandas as pd
-import hashprep
+import tablint
from ..utils.logging import get_logger
from .base import BaseReport
@@ -60,7 +60,7 @@ def generate(self, summary, full=False, output_file=None):
content += f"| Analysis started | {analysis_started[:19] if analysis_started != 'N/A' else 'N/A'} |\n"
content += f"| Analysis finished | {analysis_finished[:19] if analysis_finished != 'N/A' else 'N/A'} |\n"
content += f"| Duration | {reproduction_info.get('duration_seconds', 0)} seconds |\n"
- content += f"| Software version | hashprep v{hashprep.__version__} |\n\n"
+ content += f"| Software version | tablint v{tablint.__version__} |\n\n"
if full:
img_dir = None
@@ -241,7 +241,7 @@ def generate(self, summary, full=False, output_file=None):
content += "### Tail (last 5 rows)\n\n"
content += pd.DataFrame(summary["summaries"]["tail"]).to_markdown(index=False) + "\n\n"
- content += "\n---\nGenerated by HashPrep\n"
+ content += "\n---\nGenerated by Tablint\n"
if output_file:
with open(output_file, "w") as f:
diff --git a/hashprep/reports/pdf.py b/tablint/reports/pdf.py
similarity index 97%
rename from hashprep/reports/pdf.py
rename to tablint/reports/pdf.py
index 609dfe6..2a9374d 100644
--- a/hashprep/reports/pdf.py
+++ b/tablint/reports/pdf.py
@@ -1,17 +1,17 @@
from datetime import datetime
import pandas as pd
-from jinja2 import Template
from weasyprint import HTML
-import hashprep
+import tablint
-from .base import BaseReport
+from .base import BaseReport, render_html
+from .logo import LOGO_SVG
class PdfReport(BaseReport):
def generate(self, summary, full=False, output_file=None, **kwargs):
- template = Template(self._get_template())
+ template_str = self._get_template()
head_df = pd.DataFrame(summary["summaries"]["head"])
tail_df = pd.DataFrame(summary["summaries"]["tail"])
@@ -41,7 +41,8 @@ def generate(self, summary, full=False, output_file=None, **kwargs):
"analysis_started": reproduction_info.get("analysis_started", ""),
"analysis_finished": reproduction_info.get("analysis_finished", ""),
"duration_seconds": reproduction_info.get("duration_seconds", 0),
- "version": hashprep.__version__,
+ "version": tablint.__version__,
+ "logo_svg": LOGO_SVG,
"full": full,
"head_table": head_df.to_html(index=False, border=0),
"tail_table": tail_df.to_html(index=False, border=0),
@@ -53,7 +54,7 @@ def generate(self, summary, full=False, output_file=None, **kwargs):
"missing_plots": summary["summaries"].get("plots", {}),
}
- html_content = template.render(context)
+ html_content = render_html(template_str, context)
pdf_content = HTML(string=html_content).write_pdf()
if output_file:
with open(output_file, "wb") as f:
@@ -82,6 +83,9 @@ def _get_template(self):
h4 { font-size: 9pt; margin: 10pt 0 4pt 0; color: #555; text-transform: uppercase; letter-spacing: 0.5pt; }
.header { margin-bottom: 16pt; }
+ .header .brand { display: flex; align-items: center; gap: 8pt; }
+ .header .brand h1 { margin: 0; }
+ .header .brand svg { width: 28pt; height: 28pt; }
.header .version { font-size: 9pt; color: #888; }
.header .badges { margin-top: 4pt; }
.badge {
@@ -269,7 +273,7 @@ def _get_template(self):
-
@@ -612,6 +616,6 @@ def _get_template(self):
HashPrep Report
+ {{ logo_svg }}
v{{ version }}
Tablint Report
{{ critical_count }} Critical
@@ -333,7 +337,7 @@ def _get_template(self):
Software version
- hashprep v{{ version }}
+ tablint v{{ version }}
{{ tail_table | safe }}
{% endif %}
-
+