Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
27 changes: 27 additions & 0 deletions .agents/skills/export-database-excel/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
name: export-database-excel
description: Exports the entire database (Manuscripts, Authors, Nisbas, Subjects) into a consolidated, compressed, multi-sheet Excel file.
---

# Export Database Excel Skill

Use this skill when you need to export the complete database to a compressed spreadsheet format suitable for Google Sheets import.

## When to use this skill
- The user requests an export of the entire database.
- Generating a full backup spreadsheet of the synced tables.

## Execution Steps for the AGENT
1. Run the database export script:
```bash
node bin/export-database.js
```

## Expected Results
- A dated, compressed Excel workbook generated at `data/spreadsheet_imports/waamdb-MM-DD-YYYY.xlsx` containing worksheets:
- `Manuscripts`
- `Authors`
- `Nisbas`
- `Subjects`
- All text fields are cleaned of edge spaces/diacritics/entities, and internal newlines are escaped.
- File size is compressed to ~40-50MB (below Google Sheets' 100MB conversion limit).
28 changes: 28 additions & 0 deletions .agents/skills/generate-missing-references-excel/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
name: generate-missing-references-excel
description: Takes the dated missing references JSON report and compiles it into a dated, multi-tab Excel spreadsheet separating missing subjects, authors, and groups.
---

# Generate Missing References Excel Skill

Use this skill when you need to compile a validation report of missing references (foreign keys referenced in manuscripts but not present in the database tables) into a readable Excel format for editors.

## When to use this skill
- Generating Excel spreadsheets for missing foreign key references after running a spreadsheet sync.
- A spreadsheet sync script has created a dated JSON report file in `data/spreadsheet_imports/`.

## Execution Steps for the AGENT
1. Ensure the dated JSON file exists (e.g. `data/spreadsheet_imports/missing_references_report-MM-DD-YYYY.json`).
2. Run the Excel compilation script:
```bash
node bin/create-excel-report.js
```
*Note: This script automatically reads the dated JSON report and generates the matching dated Excel report in `data/spreadsheet_imports/`.*

## Expected Results
- A dated Excel workbook generated in `data/spreadsheet_imports/missing_references_report-MM-DD-YYYY.xlsx` containing sheets:
- `manuscripts-subjects`
- `manuscripts-authors`
- `manuscripts-groups`
- `authors-nisbas`
- The file has custom headers (such as `Manuscript ID`, `Arabic Manuscript Title`, `English Manuscript Title`) with cleaned values.
39 changes: 39 additions & 0 deletions .agents/skills/sync-database/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
name: sync-database
description: Synchronises the local database with spreadsheet CSV files. Instructs on production backup, database seeding, CSV placement, running the sync script, and updating search indexes.
---

# Sync Database Skill

Use this skill when you need to synchronize the local database with the editor's spreadsheet data.

## When to use this skill
- The user requests to import, sync, or update the database with new CSV exports.
- A spreadsheet sync needs to be executed alongside search index regeneration.

## Prerequisites and Instructions to show the USER
Before executing the sync, explicitly ask/instruct the user to perform the following:
1. **Back up Production Database**: Always request the user to take a dump/backup of the production database first.
2. **Seed Local Database**: Instruct the user to seed the local database with the latest production backup so that the sync operates on fresh production-equivalent records.
3. **Download CSV Exports**: Prompt the user to export the editor's Google Sheets tabs as CSV files and save them to `data/spreadsheet_imports/` as:
- `manuscripts.csv`
- `authors.csv`
- `nisba.csv`
- `subjects.csv`

## Execution Steps for the AGENT
1. Run the synchronization script:
```bash
node bin/sync-spreadsheet.js
```
*Note: This will check for missing references and output a dated JSON report `data/spreadsheet_imports/missing_references_report-MM-DD-YYYY.json`.*

2. Rebuild search indexes:
```bash
node -r dotenv/config ./bin/generate-authors-search-indexes.js && node -r dotenv/config ./bin/generate-manuscripts-search-indexes.js
```

## Expected Results
- Updated database tables containing the new CSV data.
- A dated JSON report containing missing references generated in `data/spreadsheet_imports/missing_references_report-MM-DD-YYYY.json`.
- Refreshed search indexes for authors and manuscripts.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ npm-debug.log
!.yarn/releases
!.yarn/sdks
!.yarn/versions


cypress/videos
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
legacy-peer-deps=true
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v22.12.0
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/amp-npm-0.3.1-4dbe54e43f-d87c786ff0.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/atob-npm-2.1.2-bcb583261e-dfeeeb7009.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/co-npm-4.6.0-03f2d1feb6-5210d92230.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/cron-npm-1.8.3-b8d674f690-e0d8800573.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/depd-npm-2.0.0-b6c51a4b43-abbe19c768.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/etag-npm-1.8.1-54a3b989d9-571aeb3dbe.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/fsevents-patch-61ccaa93a2-8.zip
Binary file not shown.
Binary file removed .yarn/cache/ftp-npm-0.3.10-348fb9ac23-ddd313c1d4.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/glob-npm-7.2.3-2d866d17a5-29452e97b3.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/got-npm-6.7.1-f61570d59b-e816306dbd.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/has-npm-1.0.3-b7f00631c1-b9ad53d53b.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/hpkp-npm-2.0.0-6cacef2064-01f8df366e.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/ini-npm-1.3.8-fb5040b4c0-dfd98b0ca3.zip
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/ip-npm-1.1.5-af36318aa6-30133981f0.zip
Binary file not shown.
Binary file removed .yarn/cache/ip-npm-1.1.8-abea558b72-a2ade53eb3.zip
Binary file not shown.
Binary file removed .yarn/cache/ip-npm-2.0.0-204facb3cc-cfcfac6b87.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading