Skip to content

Stable sort for school escorting#1085

Open
americalexander wants to merge 4 commits into
ActivitySim:mainfrom
RSGInc:stable_sort
Open

Stable sort for school escorting#1085
americalexander wants to merge 4 commits into
ActivitySim:mainfrom
RSGInc:stable_sort

Conversation

@americalexander

@americalexander americalexander commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Address multiple locations where the school escorting model does not sort in a stable manner. Also use 64-bit integers for tour and person IDs.

Changes

Stable Sorting

Added kind=""stable"" to sort_values calls throughout the school escorting model to ensure deterministic, reproducible results regardless of the initial row ordering of input data. Without stable sorting, ties in the sort key can be broken arbitrarily, leading to non-reproducible outputs across runs or platforms.

Affected locations:

  • determine_escorting_participants: stable sort of chaperones by chaperone_weight (descending) when assigning chaperone_num, and stable sort of escortees by age (ascending) when assigning escortee_num
  • school_escorting: stable sort of school escort tours by household_id and school_escort_direction before further processing
  • create_pure_school_escort_tours: stable sort of pure escort tours by household_id, person_id, and start time

64-bit Integer Types

Replaced .astype(int) with .astype(""int64"") for tour IDs, person IDs, and related integer columns. Using the platform-native int type can produce 32-bit integers on Windows, which risks integer overflow for large ID values and inconsistent behavior across operating systems.

Affected locations:

  • chauf_num and chauf_id in create_school_escorting_bundles_table
  • Column values in join_attributes
  • tour_id in create_chauf_escort_trips
  • num_escortees in process_tours_after_escorting_model

@americalexander americalexander marked this pull request as ready for review June 24, 2026 21:28
@jpn-- jpn-- requested a review from Copilot June 25, 2026 19:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves determinism and cross-platform consistency in the school escorting model by (1) enforcing stable sorting where ordering affects assignment/processing and (2) standardizing key ID/counter columns to 64-bit integers to avoid Windows 32-bit int pitfalls.

Changes:

  • Add kind="stable" to key sort_values calls to make tie-breaking deterministic.
  • Replace platform-native astype(int) casts with astype("int64") for tour/person-related integer columns.
  • Apply stable ordering before downstream computations (e.g., pure escort tour construction and escort bundle ordering).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
activitysim/abm/models/util/school_escort_tours_trips.py Uses int64 for ID-like columns and applies stable sorting when ordering impacts pure escort tour processing.
activitysim/abm/models/school_escorting.py Applies stable sorts in participant numbering and bundle ordering; switches chauffeur-related numeric fields to int64.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread activitysim/abm/models/school_escorting.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@jpn--

jpn-- commented Jun 25, 2026

Copy link
Copy Markdown
Member

@americalexander Is this PR motivated by an actual finding of non-reproducibility, or just having discovered the theoretical potential of such?

@janzill

janzill commented Jun 25, 2026

Copy link
Copy Markdown

@jpn-- this is an actual problem, we found this independently when looking into unexpected differences in EET scenarios, let's discuss this in today's engineering meeting. I think it would be good to have a test here that shows how this manifests and how the new code fixes it.

@americalexander

Copy link
Copy Markdown
Contributor Author

As Jan mentioned, this was something we discovered in practice. CCing @dhensle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants