docs: add a Research section with participant instructions - #181
Conversation
ActivityWatch is used as a data-collection instrument in academic studies, but the participant-facing instructions have been circulated over email and go stale between releases. This adds a canonical, versioned page studies can link to. - research/index: why the tool suits research (local-first, auditable, category-only collection), the IJmker 2008 validity citation, and how to cite ActivityWatch via the Zenodo DOI - research/participant-instructions: install and permissions per platform, what is recorded, and the end-of-study export, plus a "Notes for researchers" section covering the mistakes that generate support mail (the per-row export menu instead of "Export all buckets as JSON", CSV vs JSON, two buckets being normal, and the UI auto-selecting a locale from the browser language) Studies keep their own branding, upload location and translations; only the mechanics live here.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| - **Two buckets is correct**, not a symptom of a double installation: one window watcher and | ||
| one AFK watcher. Four rows would mean the participant opened the standard dashboard on port | ||
| 5600 instead. |
There was a problem hiding this comment.
Port 5600 serves the standard installation’s separate bucket set; it does not add two rows to the Research Edition’s data. Optional watchers can also create four or more legitimate rows on either server. This fixed row-count test could therefore misidentify a valid Research Edition session or fail to recognize the standard dashboard. Diagnose the edition using the port and Research Edition badge instead.
| - **Two buckets is correct**, not a symptom of a double installation: one window watcher and | |
| one AFK watcher. Four rows would mean the participant opened the standard dashboard on port | |
| 5600 instead. | |
| - **Two buckets is correct** for the default Research Edition setup: one window watcher and | |
| one AFK watcher. Additional watchers may create more rows, so use the **Research Edition** | |
| badge and port 5667—not the row count—to distinguish it from the standard dashboard. |
Knowledge Base Used:
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Fixed in be65c9a — you, Codex and Bob all landed on this independently, and the suggestion was right.
Adopted with one addition: the row count is not just unreliable, it is not a discriminator at all. A standard install also shows exactly two buckets (window + AFK), so "two rows" is the expected default on either edition rather than a Research Edition signal. The page now says to diagnose by the Research Edition badge and port 5667, and notes that buckets are created per watcher and per host (linking the data model), so extra watchers or a hostname change can add rows on either server.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0254b612c9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - **Two buckets is correct**, not a symptom of a double installation: one window watcher and | ||
| one AFK watcher. Four rows would mean the participant opened the standard dashboard on port | ||
| 5600 instead. |
There was a problem hiding this comment.
Opening port 5600 queries only the standard server; it does not merge the buckets served on port 5667. Bucket count is also not fixed: the documented data model creates one bucket per watcher and host (src/buckets-and-events.rst:7-13), so a hostname change or an additional watcher can produce four rows, while a standard installation may show only the two core buckets. This guidance can therefore misdiagnose the correct Research dashboard; use the port or Research Edition badge as the discriminator instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in be65c9a. Your reading of the data model was the decisive part: src/buckets-and-events.rst documents one bucket per watcher and host, so a hostname change alone can produce four rows on a perfectly correct Research Edition install. That is now stated on the page, with a :doc: link to the data model.
The separate-servers point is also now explicit — opening 5600 queries the standard server and does not merge anything into the 5667 view — and the discriminator is the Research Edition badge plus the port.
|
Reviewed the Research section, with attention on "Notes for researchers" (that list came out of the Lund instruction reviews). The notes are the right support-mail generators: Export-all vs per-row, JSON vs CSV, locale auto-select, fixed filename, live Raw Data unsanitised. Put the unsanitised-live-Raw-Data warning first. Researchers on a support call will open the dashboard, see real titles, and think the ethics filter is broken. The rewrite to One correction, same line Greptile/Codex flagged:
That is wrong, and extra-watchers is the smaller reason. The standard dashboard on 5600 also has two buckets (window + AFK). Opening 5600 does not add two rows to the 5667 view — they are different servers. Row count does not distinguish editions at all. Diagnose by the Research Edition badge and port 5667. Two rows is the expected default for a participant; it is not a discriminator. Four rows is a different failure mode (hostname change, extra watcher, or they exported the wrong server and something else is also installed). Suggested replacement:
Otherwise this is the page studies should link to instead of emailing a copy. I would not block on Android: Windows unsigned / SmartScreen is documented. Keep it. That friction repeats on every study until the installer is signed. |
ActivityWatch appears in methods sections rather than titles or abstracts, so it is largely invisible to citation indexes - both Zenodo DOIs currently report zero tracked citations despite real academic use, including a 2,000-participant Oxford study and a multi-year ERC-funded deployment at Ghent. A public list serves two purposes: it lets researchers choosing an instrument see prior use, and it gives existing users a reason to tell us they exist.
|
The studies list is the right addition — methods-section uses are invisible to indexes, and this is how they become visible. One correction on The rest checks out: Ballou et al. Stage 1 RR is real; Parry & Klingelhoefer name AW as an exemplar desktop logger; Rivera/Booz/Hammerstein (CMU SEI, ECCWS 2025) actually instrument with AW. The two notes-for-researchers edits from the previous comment are still outstanding on Optional miss, not blocking: Timár (Masaryk, 2025) is a thesis that actually builds on AW. |
…s, and study classification - Notes for researchers now lead with where filtering actually happens: transform_window runs before client.heartbeat, so titles/URLs are filtered at capture, not export. The research-participant rewrite is the hostname, which is export-only. Adds the fail-closed export refusal (pre-existing database) as the failure mode that actually generates mid-study support mail. - Row count is not an edition discriminator: a standard install also shows two buckets, and the two editions are separate servers. Diagnose by the Research Edition badge and port 5667. Buckets are per watcher and per host. - Split the studies list by actual relationship. Activity Frames (Iyamu 2026) cites ActivityWatch once as related work and runs its own capture pipeline; it is not an instrument use. Traces as Data and the student theses likewise move out of the instrument list.
|
All addressed in be65c9a. Two of your three points landed; the sanitisation one is inverted, and it matters. Activity Frames — you were right, and I checked the source. Its only mention is in related work:
and it provisions its own capture via Rather than just drop it, I fixed the thing that let it in. The heading "Studies using ActivityWatch" was flattening three different relationships into one claim, so the list is now split: Used as a data-collection instrument (Oxford, On/Off, Sabermetrics), Discussed, compared or built upon (Traces as Data, Activity Frames, the student theses), and Unpublished or unlisted deployments. Traces as Data names AW as an exemplar and the theses build on it — neither is instrument use either, so the same correction applies to them. Worth noting my own source notes already drew this distinction correctly; it was lost in compressing them into the page, which is the failure mode to watch rather than the single entry. The sanitisation note is backwards, so I did not write it as suggested. From So filtering happens at capture. Live Raw Data already shows categories, not real titles — a researcher on a support call will not see raw titles on a clean Research Edition profile. Your conclusion still holds for a different reason, so it is now the first bullet: the export fails closed. If the Research Edition was installed over an existing ActivityWatch database, the old unfiltered events are still there and the export refuses rather than shipping them. That is the panicky end-of-study email, and it arrives when there is no time left to re-collect. The page now tells researchers to require a clean profile up front. Bucket count — fixed, and stronger than flagged: row count is not a weak discriminator, it is not one at all, since a standard install also shows two buckets. Badge and port 5667 instead, with the per-watcher-and-host data model linked. Timár — left out, deliberately. The Masaryk IS PDF sits behind a meta-refresh wall and the landing page returns nothing, so I could not confirm the ActivityWatch relationship or which of the three categories it belongs in. Adding it unverified is precisely what produced the Activity Frames entry. If you can reach the full text and classify it, I will add it. Android follow-up agreed — out of scope here, tracked separately for the VR pilot. Builds clean; the one warning is the pre-existing theme deprecation on master. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be65c9a61f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Do this once, at the end of the study period, and keep the Research Edition running while you | ||
| do it. It takes a couple of minutes. |
There was a problem hiding this comment.
For participants whose machine contains a prior ActivityWatch database, the later note explicitly says the Research Edition will refuse to export, yet this workflow instructs them to try exporting only once at the end of the study. That delays discovery until there is no time to recollect the experiment data—particularly problematic because lines 17–18 also describe installation alongside normal ActivityWatch as safe. Add an export preflight during setup or prevent enrollment until a clean profile has been confirmed.
Useful? React with 👍 / 👎.
2 participants
ActivityWatch is used as a data-collection instrument in academic studies. Until now the participant-facing instructions have been written per-study and circulated over email, which means they go stale between releases: the current round of review caught a study's instructions still describing pre-
b5autostart behaviour, pointing participants at the per-bucket export menu instead of "Export all buckets as JSON", and assuming an English-only UI.This adds a canonical, versioned page that studies can link to instead of maintaining their own copy of the mechanics.
What's here
research/index— why the tool fits research use (local-first, auditable, category-only collection), the canonical validity citation for why self-reported computer use is a poor measure (IJmker et al. 2008: self-report agreed with registration for only 18% of participants), and how to cite ActivityWatch via the Zenodo DOI.research/participant-instructions— install and permissions per platform, what is and isn't recorded, and the end-of-study export. Plus a "Notes for researchers" section covering the things that reliably generate support mail:Studies keep their own branding, upload location, ethics wording and translations. Only the mechanics live here, since those are what change between releases.
Notes
v0.14.0b5-researchbuild and the source at that tag.