sync mentored trainees lists from CV to people.xlsx #10

Activity

1. jeremymanning commented on Dec 18, 2025

   

   jeremymanning

   on Dec 18, 2025

   MemberAuthor

   More actions

   there might also be people who appear on the website but NOT in the CV. in that case, those people should be added to the CV at the appropriate positions. (but when in conflict, e.g., with respect to start/end dates or status, prefer the CV info over the people.xlsx info).
2. jeremymanning commented on Dec 18, 2025

   

   jeremymanning

   on Dec 18, 2025

   MemberAuthor

   More actions

   note: we are working on this issue in parallel with issue #9, so we should stick with the same feature branch. we'll pull all the changes in together.
3. jeremymanning commented on Dec 18, 2025

   

   jeremymanning

   on Dec 18, 2025

   MemberAuthor

   More actions

   Implementation Plan for Issue #10: Sync Mentored Trainees from CV to people.xlsx

   Overview

   This issue requires bi-directional synchronization between the CV (documents/JRM_CV.tex) and the people spreadsheet (data/people.xlsx). The CV is the source of truth for trainee information.

   Key Requirements

   1. CV → people.xlsx: Add any trainees in CV but not in spreadsheet
   2. people.xlsx → CV: Add any trainees in spreadsheet but not in CV
   3. Active vs Alumni: Trainees with no end date (e.g., "2024 -- )") are active members; those with end dates are alumni
   4. Sorting:
      • Members: by category (postdoc > grad > lab manager > research scientist > undergrad), then reverse chronologically by start year
      • Alumni: as appropriate for each section

   Branch Strategy

   Per comment: Stay on branch issue-9-news-feed - will be merged together with Issue #9.

   ---

   Execution Plan

   Phase 1: Build CV Parser Script (Sequential)

   Agent 1: CV Parser Development

   Create scripts/parse_cv_trainees.py:

   • Parse LaTeX file to extract trainee information from mentorship sections
   • Handle all trainee types: Postdoctoral Advisees, Graduate Advisees, Undergraduate Advisees
   • Extract: name, role/type, start year, end year (if any), current position (if any)
   • Detect active vs alumni status based on date format
   • Return structured data suitable for comparison with people.xlsx

   Key parsing patterns:

   • \item Name (2024 -- ) → Active member
   • \item Name (2024 -- 2025; current position: Company) → Alumni
   • \item Name (Doctoral student; 2021 -- ) → Active grad student
   • \item Name* (2024 -- ) → Undergraduate with thesis (asterisk)

   Phase 2: Comparison & Sync Logic (Sequential - depends on Phase 1)

   Agent 2: Sync Script Development

   Create scripts/sync_cv_people.py:

   • Load CV trainees using parser from Phase 1
   • Load people.xlsx data
   • Compare and identify:
     • People in CV but not in spreadsheet (add to spreadsheet)
     • People in spreadsheet but not in CV (add to CV)
     • Discrepancies in dates/status (CV wins)
   • Generate sync report showing all changes needed

   Phase 3: Data Updates (Sequential - depends on Phase 2)

   Agent 3: Apply Updates

   • Update data/people.xlsx with missing trainees from CV
   • Generate patch/additions for CV if spreadsheet has unique entries
   • Ensure proper sorting in members sheet:
     • Category order: postdoc > grad student > lab manager > research scientist > undergrad
     • Within category: reverse chronological by start year
   • Run build_people.py to regenerate people.html

   Phase 4: Testing (Parallel after Phase 3)

   Agent 4a: Unit Tests
   Create tests/test_parse_cv_trainees.py:

   • Test LaTeX parsing for all trainee types
   • Test date extraction (active vs alumni detection)
   • Test name extraction with special characters
   • Test current position parsing
   • Test edge cases (missing data, malformed entries)

   Agent 4b: Integration Tests
   Create tests/test_sync_cv_people.py:

   • Test comparison logic
   • Test sync direction (CV → spreadsheet, spreadsheet → CV)
   • Test sorting logic
   • Test full sync workflow with real CV and spreadsheet files

   Phase 5: Validation & Cleanup (Sequential)

   Single Agent

   1. Run all tests locally: python -m pytest tests/ -v
   2. Run linters/validation: python scripts/validate_data.py
   3. Rebuild all pages: python scripts/build.py
   4. Manual verification:
      • Check people.html renders correctly
      • Verify all trainees from CV appear in correct sections
      • Confirm sorting is correct
   5. Clean up:
      • Remove any temporary files
      • Update notes in notes/ folder
   6. Commit with descriptive message

   ---

   Files to Create/Modify

   New Files

   • scripts/parse_cv_trainees.py - LaTeX CV parser for mentorship sections
   • scripts/sync_cv_people.py - Bi-directional sync script
   • tests/test_parse_cv_trainees.py - CV parser tests (NO MOCKS - uses real CV file)
   • tests/test_sync_cv_people.py - Sync logic tests (NO MOCKS - uses real files)

   Modified Files

   • data/people.xlsx - Updated with synced trainee data
   • documents/JRM_CV.tex - Updated with any trainees only in spreadsheet
   • people.html - Regenerated from updated spreadsheet

   ---

   Testing Requirements

   CRITICAL: NO MOCK TESTS OR SIMULATIONS

   All tests must use:

   • Real documents/JRM_CV.tex file for parsing tests
   • Real data/people.xlsx for spreadsheet tests
   • Real file I/O operations
   • Actual comparison against expected structured data

   Test categories:

   1. CV Parsing Tests - Parse actual CV, verify extracted trainee counts and data
   2. Sync Logic Tests - Use real files, verify correct additions/updates identified
   3. Integration Tests - Full workflow with real data files
   4. Validation Tests - Ensure generated HTML is valid

   ---

   Verification Checklist

   Before marking complete:

   • All tests pass locally: python -m pytest tests/ -v
   • Data validation passes: python scripts/validate_data.py
   • Build succeeds: python scripts/build.py
   • people.html renders correctly in browser
   • All CV trainees appear in correct spreadsheet sections
   • Members sorted correctly (category, then reverse chronological)
   • Alumni sorted correctly in respective sheets
   • No temporary files left behind
   • Notes updated in notes/ folder
   • GitHub Actions CI passes after push

   ---

   Notes on CV Structure

   From JRM_CV.tex (lines 653-859):

   Postdoctoral Advisees (lines 654-658):

   • Format: \item Name (YYYY -- YYYY; current position: Company)
   • 3 entries total

   Graduate Advisees (lines 661-676):

   • Format: \item Name (Type; YYYY -- YYYY; current position: ...)
   • Types: "Doctoral student", "Masters student, Program"
   • Active: no end year (e.g., "2021 -- )")
   • 10 entries total

   Undergraduate Advisees (lines 734-858):

   • Format: \item Name* (YYYY -- YYYY) (asterisk = thesis student)
   • Multi-column layout
   • ~125 entries total

   Lab Managers: Not explicitly listed in CV - need to cross-reference spreadsheet's alumni_managers sheet.

   ---

   Estimated Complexity

   • Phase 1: Medium - LaTeX parsing requires careful regex handling
   • Phase 2: Medium - Comparison logic with multiple edge cases
   • Phase 3: Low - Straightforward data updates
   • Phase 4: Medium - Comprehensive test coverage
   • Phase 5: Low - Standard validation workflow

   Total: ~3-4 hours of agent work with parallelization
4. jeremymanning commented on Dec 18, 2025

   

   jeremymanning

   on Dec 18, 2025

   MemberAuthor

   More actions

   Updated Plan

   Per feedback:

   • Additional pattern: \item Name (2023 -- 2024) → Alumni (date range without current position)
   • Skip tests - this is a one-time sync operation
   • Keep scripts - for systematic execution and potential future re-use
   • Future onboarding - will use separate process for adding people

   Simplified Execution

   Phase 1: Create scripts/parse_cv_trainees.py - CV parser
   Phase 2: Create scripts/sync_cv_people.py - comparison and sync logic
   Phase 3: Run sync, update files, validate output
   Phase 4: Cleanup and commit

   Proceeding with implementation now.
5. jeremymanning

   added a commit that references this issue on Dec 19, 2025

   Sync CV mentored trainees with people spreadsheet (Issue #10)

   ...

   0f77c97
6. jeremymanning commented on Dec 19, 2025

   

   jeremymanning

   on Dec 19, 2025

   MemberAuthor

   More actions

   Completed in PR #11

   CV Trainee Sync Scripts

   • Created scripts/sync_cv_people.py - main synchronization script that reads trainees from JRM_CV.tex and updates people.xlsx
   • Created scripts/parse_cv_trainees.py - parser for extracting trainee information from the CV LaTeX format

   Sorting Implementation

   Members sheet is now sorted by:

   1. Role priority: Postdocs → PhD students → Masters students → Lab managers → Undergrads
   2. Join date: Chronologically within each role category (earliest first)

   Alumni Handling

   • Trainees with end dates are placed in appropriate alumni sheets (alumni_postdocs, alumni_grads, alumni_managers, alumni_undergrads)
   • Current position information is parsed from CV and stored in spreadsheet

   Additional Improvements

   • build_people.py now uses placeholder.png for members without profile photos
   • Added hand-drawn borders to new profile photos using add_poster_borders.py
   • Updated validate_data.py to check for sync discrepancies

   How to Use

   Run python scripts/sync_cv_people.py to synchronize trainees from the CV to the spreadsheet. The script will report any additions, updates, or discrepancies found.
7. jeremymanning

   closed this as completedon Dec 19, 2025