Improve the speed of the database user retrieval for the student nav.

[drgrice1]

Now that the student nav is on every set and problem page there is a need to make the retrieval of all users assigned to the set faster. I noticed in a large class with more than 25,000 users this is quite slow now. It takes about 8 seconds for each page to load. The problem is the query is passing all of the user ids of those assigned to the set to the getUsersWhere method. That means that SQL::Abstract has to process all of those and compile a rather long sql statement. That is slow. So this breaks the list of user ids into chunks of 500. For the class with more than 25,000 users this drops the page load time to just under 2 seconds. Stil not great, but considerably better.

On my production placement exam server that has just over 32,000 users it currently takes more than 35 seconds for each page to load. With this pull request it drops the time to about 8 seconds. Again, not great but considerably better. Clearly the placement server is not as fast as my local computer also.

Note that for classes with less than 500 users this won't change anything.

Also note that this is only an issue for those that have the student nav shown.

[Alex-Jordan]

Code looks good and I hit no issues when using this. I also did not test this with a course that has thousands of students.

1. Did you give any thought to a course config option to disable the student nav? In a placement exam course where there might be many student users, it might be preferable to just disable it.

2. When navigating from one problem to the next (or from a set page to a problem) is there any way to pass the list of student names as a hidden input, so that it is only the first page load that takes 8 seconds, in your example? Of course, after navigating somewhere else in WeBWorK, it would do the database query again.

3. Why does the list of names get sorted for homework, but not for tests?

4. I noticed while testing this, that dropped students are included in the student nav. That's not new with this issue, but maybe you'd want to filter them out while you are making changes here.

[drgrice1]

1. Did you give any thought to a course config option to disable the student nav? In a placement exam course where there might be many student users, it might be preferable to just disable it.

I did/am considering it. I am just trying to figure out the best way that this should be done. One way to do it is to always do the initial query for the users assigned to the set, and then if there are too large of a number of users don't show the student nave. That initial query listUserSetsWhere call to obtian the users assigned to the set is fast and is not the slow down. It is the subsequent call to getUsersWhere that is passed the massive list of user_ids that is slow since SQL::Abstract has to parse that massive list.

2. When navigating from one problem to the next (or from a set page to a problem) is there any way to pass the list of student names as a hidden input, so that it is only the first page load that takes 8 seconds, in your example? Of course, after navigating somewhere else in WeBWorK, it would do the database query again.

I don't really think that there is a good way to do this. The thing is that the list can change from page to page. In fact for a course with a large number of users it will when changing users. The student nav only shows 100 before and 100 after the user currently being acted as (or just the first 200 if not acting). You wouldn't want to embed all of the users into the page as that would a cause client side slow down that would negate any speed gained from not querying server side.

3. Why does the list of names get sorted for homework, but not for tests?

The list does get sorted for tests. This happens on lines 1420 - 1425 of lib/WeBWorK/ContentGenerator/GatewayQuiz.pm.

4. I noticed while testing this, that dropped students are included in the student nav. That's not new with this issue, but maybe you'd want to filter them out while you are making changes here.

This is issue #2848. It seems that what should be done here is to add a filter to show/hide dropped students. There might be cases where one wants to act as a dropped student, so always removing them may be a mistake.

[Alex-Jordan]

Any chance it's faster to query for @allUserRecords as literally all user records, then let perl filter that using @users? That would avoid the very long query parameter.

On Mon, May 18, 2026, 4:43 AM Glenn Rice ***@***.***> wrote: *drgrice1* left a comment (openwebwork/webwork2#2975) <https://urldefense.com/v3/__https://github.com/openwebwork/webwork2/pull/2975*issuecomment-4477313197__;Iw!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-dKeXBx3I$> 1. Did you give any thought to a course config option to disable the student nav? In a placement exam course where there might be many student users, it might be preferable to just disable it. I did/am considering it. I am just trying to figure out the best way that this should be done. One way to do it is to always do the initial query for the users assigned to the set, and then if there are too large of a number of users don't show the student nave. That initial query listUserSetsWhere call to obtian the users assigned to the set is fast and is not the slow down. It is the subsequent call to getUsersWhere that is passed the massive list of user_ids that is slow since SQL::Abstract has to parse that massive list. 2. When navigating from one problem to the next (or from a set page to a problem) is there any way to pass the list of student names as a hidden input, so that it is only the first page load that takes 8 seconds, in your example? Of course, after navigating somewhere else in WeBWorK, it would do the database query again. I don't really think that there is a good way to do this. The thing is that the list can change from page to page. In fact for a course with a large number of users it will when changing users. The student nav only shows 100 before and 100 after the user currently being acted as (or just the first 200 if not acting). You wouldn't want to embed all of the users into the page as that would a cause client side slow down that would negate any speed gained from not querying server side. 3. Why does the list of names get sorted for homework, but not for tests? The list does get sorted for tests. This happens on lines 1420 - 1425 of lib/WeBWorK/ContentGenerator/GatewayQuiz.pm. 4. I noticed while testing this, that dropped students are included in the student nav. That's not new with this issue, but maybe you'd want to filter them out while you are making changes here. This is issue #2848 <https://urldefense.com/v3/__https://github.com/openwebwork/webwork2/issues/2848__;!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-dBG9sfm4$>. It seems that what should be done here is to add a filter to show/hide dropped students. There might be cases where one wants to act as a dropped student, so always removing them may be a mistake. — Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/openwebwork/webwork2/pull/2975*issuecomment-4477313197__;Iw!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-dKeXBx3I$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABEDOAGLLKGSMGD4X7OGSK343LZMTAVCNFSM6AAAAACY3RYLJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DINZXGMYTGMJZG4__;!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-dqUjRaqo$> . Triage notifications on the go with GitHub Mobile for iOS <https://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-deTTPwP8$> or Android <https://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!Ka_JY85zDv0FFw!lCLZTCZwpsgYCwaGgDWREb-bsokde-JS8PZK36hK8ypKISzNbB0M_eYDXDpgxppnt-0yKQ0WrLgJiVtyAl-d_Iy0M-s$>. You are receiving this because you commented.Message ID: ***@***.***>

Alex Jordan Mathematics Instructor Portland Community College

[Alex-Jordan]

I created a course with 32k students, each assigned to the Orientation set. So far on the WeBWorK-2.21 branch, it takes 34 seconds to load a page. I'll test the PR soon. But I am only seeing 200 names in select element for students to act as. Is it correct to only be seeing 200? If so, is the 500 number the right number to use, instead of 200?

[Alex-Jordan]

On your branch, it takes 4 seconds.

[drgrice1]

The initial load of users from the database gets all users. Then the list is narrowed to the 200 before and after the current user in subsequent processing. So the 500 is correct.

[Alex-Jordan]

I tried the approach where the filtering happens with perl, and 4 seconds reduces to 1.5 seconds. What I have starting at line 17 of lib/WeBWorK/HTML/StudentNav.pm is:

    # Find all users for the given set (except the current user) sorted by last_name, then first_name, then user_id.
    my %users = map { $_->[0] => 1} $c->db->listUserSetsWhere({ set_id => $setID, user_id => { '!=' => $userID } });
    my @allUserRecords = grep { defined($users{$_->{user_id}}) } $c->db->getUsersWhere({}, [qw/last_name first_name user_id/]);

[drgrice1]

I switched to using your suggestion @Alex-Jordan. Thanks, that does seem to be better.

[drgrice1]

This approach seems to be fast enough that I don't think an option to disable the student nav is needed anymore.