Skip to content

Migrate network clients to new unified Scraper format (Fix #113)#142

Merged
samaloney merged 11 commits intosunpy:mainfrom
Amityush-lgtm:fix-scraper-issue-113
Feb 19, 2026
Merged

Migrate network clients to new unified Scraper format (Fix #113)#142
samaloney merged 11 commits intosunpy:mainfrom
Amityush-lgtm:fix-scraper-issue-113

Conversation

@Amityush-lgtm
Copy link
Contributor

@Amityush-lgtm Amityush-lgtm commented Feb 9, 2026

PR Description

Migrate network clients to unified Scraper format. Fixes: #113

Summary

This PR updates all radiospectra network clients (RSTN, eCallisto, I-LOFAR, and PSP) to use the modern, unified sunpy.net.scraper.Scraper interface introduced in SunPy 6.1 and enforced in 7.1.0.
It fixes CI failures caused by breaking changes in SunPy 7.1.0 and brings the networking code in line with current SunPy standards. This PR restores compatibility and modernizes the implementation to match current SunPy expectations.

What changed

  • Unified Scraper patterns: Replaced separate baseurl + pattern usage with a single modern pattern string using the {{ }} format required by Scraper.
  • Bug fixes across clients:
    • PSP: Fixed errors caused by regex-style braces conflicting with Scraper’s internal .format() handling.
    • RSTN: Fixed initialization errors and a KeyError by aligning pattern capture groups with metadata usage.
    • eCallisto & I-LOFAR: Migrated to the unified pattern format and removed redundant configuration.
  • Client logic updates: Adjusted search() and pre_search_hook() implementations to match the updated Scraper API.

Verification

  • Tested on Python 3.12 with SunPy 7.1.0
  • All networking-related tests now pass:
    • test_ecallisto_client
    • test_rstn_client
    • test_ilofar
    • test_psp_client
  • All pre-commit hooks (ruff, isort, codespell) pass.

Reviewer:

@samaloney, Please let me know if this change looks good to you or if you’d like any revisions.

@Amityush-lgtm
Copy link
Contributor Author

All the CI Checks are green now. The PR is ready for review now @samaloney.
Thanks!

@samaloney
Copy link
Member

This is not specific to this PR. I'm posting this to all open PRs. Please be aware of the update to our AI usage policy, specifically on the disclosure of and acceptable uses of AI.

Copy link
Member

@samaloney samaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR this looks pretty good, almost ready. This needs to come with a deprecation changelog and an update to project.toml as this code won't work with old version of sunpy

hooks:
- id: codespell
args: [ "--write-changes" ]
exclude: "^(check_scraper|reproduce_|test_issue_|prototype_slicing|test_prototype).*\\.py$"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these local files if so it would be better to remove them locally than change this config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, those were local files I accidentally committed. I’ll remove them from the PR and revert the .pre-commit-config.yaml.

@Amityush-lgtm
Copy link
Contributor Author

Amityush-lgtm commented Feb 18, 2026

@samaloney, I have made the suggested changes. All the CI checks are green now. Thanks for the response. Let me know if any more changes are required.

Copy link
Member

@samaloney samaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, LGTM thanks for your contribution!

@samaloney samaloney merged commit 722c476 into sunpy:main Feb 19, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update scraper based client for new format

2 participants

Comments