feat(omnisearch): add Reddit as a 7th research source - #415
Open
sreeramakhil wants to merge 8 commits into
Open
Conversation
- Add webcmd npm versions <name> command that lists all published versions of a package newest-first, with publishedAt, isLatest flag, and a direct npmjs.com URL per version. Mirrors the pypi releases cmd. - Add optional equest parameter to pmFetch in utils.js so commands can inject a fake fetch function in tests without patching globals (matches the pattern used in the pypi plugin). - Add test/npm.test.js with 14 tests covering all four commands: package, versions, downloads, search. Includes happy paths, empty result / 404 handling, input validation, and a contract test asserting browser: false for every registered command. - Expand README.md with a full command table (including the new versions command), argument descriptions, and copy-paste examples for all four commands.
Slicing publishedAt to 10 chars before sorting caused versions published on the same calendar date to lose sub-day precision, producing non-deterministic newest-first ordering. Fix: sort on the raw full timestamp first, then format to date-only inside .map(). Add a regression test with two versions sharing the same date (08:00 and 14:00) to pin the correct ordering.
… entries A version key can exist in body.time without a matching entry in body.versions (e.g. yanked or unpublished releases). The previous code returned bogus rows for those keys with an invalid URL and misleading date. Fix: cross-filter timeMap entries against body.versions so only keys that exist in both are returned. Also guard that the timestamp is a string before sorting. Update the same-day regression fixture to include matching body.versions entries and add a time-only ghost key (0.0.1-ghost) to assert it is excluded from results.
Reddit is one of the most valuable developer discussion platforms
(r/programming, r/webdev, r/javascript, r/rust, r/python, etc.) but
was missing from the omnisearch aggregator entirely.
- Add redditSearch() to sources.js using the free public JSON search
API (reddit.com/search.json, no auth or key required). Returns
normalized rows: platform/title/author/score/commentCount/createdAt/
url/text — matching the shared schema used by all other sources.
Falls back to permalink when the url field is absent (self posts).
- Wire redditSearch into research.js: added to the fetchers map and
included in the default sources string so every agent using
webcmd omnisearch research gets Reddit results automatically.
- Wire redditSearch into verdict.js so community sentiment analysis
now includes Reddit engagement alongside HN, SO, GitHub, arXiv,
Dev.to, and Lobsters.
- Expand test/research.test.js from 1 test to 8 tests:
- redditSearch: normalized rows, permalink fallback, empty results,
correct endpoint URL
- research command: Reddit rows returned, default sources includes
reddit, Reddit failure isolated via Promise.allSettled
- Original HN limit regression test preserved
A non-numeric or out-of-range created_utc value would cause new Date(...).toISOString() to throw, rejecting the entire Reddit result set for that query. Fix: construct the Date object first, then check Number.isNaN on getTime() before calling toISOString() — returning empty string for malformed timestamps so one bad post never kills the whole fetch. Add regression test: a post with created_utc='not-a-number' must produce createdAt='' without throwing.
…tion
child?.data ?? {} silently converted null entries and entries missing
a data object into fake normalized rows, which downstream research
could count or render as real Reddit results.
Fix: filter children to only those where child.data is a non-null
object before slice/map. Add regression test: a mix of null, no-data,
and null-data children alongside one valid entry — only the valid
entry must appear in results.
typeof [] === 'object' is true in JavaScript, so {data:[]} passed the
previous filter and mapped to a fake row that could displace a valid
result when limit was applied after slicing.
Fix: add !Array.isArray(child.data) to the filter so only plain objects
are accepted as valid post data.
Add regression test: one array-data entry + one valid entry with limit=1
asserts the valid result is returned, not the array-shaped fake.
…imeout Without a timeout, a slow or stalled Reddit connection could keep the fetch pending indefinitely, blocking the entire research aggregation. Fix: pass AbortSignal.timeout(10_000) in the get() init options so the request is automatically aborted after 10 seconds. The existing headers and response handling are preserved.
Contributor
🟠 Maintainer review suggested — low confidenceThe automated review could not reach a fully supported conclusion. This review is advisory and does not block merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds Reddit as the 7th research source to
webcmd omnisearch researchand wires it into theverdictaggregator. Uses Reddit's public JSON API (reddit.com/search.json), requiring no auth or API keys. Includes robust validation to handle malformed posts, array-shaped payloads, or invalid timestamps safely.Related issue: None
Type of Change
Checklist
skill-src/, I ranmake buildand committedskills/Adapter Notes
CliErrorsubclasses instead of rawErrorScreenshots / Output
$ npx vitest run plugins/omnisearch/test/research.test.js RUN v4.1.4 C:/Users/akhil/OneDrive/Desktop/holidays/webcmd ✓ plugin plugins/omnisearch/test/research.test.js (11 tests) 30ms ✓ redditSearch (5) ✓ returns normalized rows from Reddit JSON API 29ms ✓ falls back to permalink when url field is absent 1ms ✓ returns empty array when Reddit returns no children 1ms ✓ returns empty string for createdAt when created_utc is invalid 0ms ✓ hits the correct Reddit search endpoint 1ms ✓ omnisearch research with reddit source (3) ✓ returns Reddit rows when sources=reddit 1ms ✓ includes reddit in default sources 0ms ✓ handles reddit failure gracefully when other sources succeed 1ms ✓ omnisearch research — limit enforcement (1) ✓ honors the total limit when research is narrowed to one source 1ms Test Files 1 passed (1) Tests 11 passed (11) Duration 379ms