Skip to content

Reuse HTTPS connections, add new response fields - #1

Merged
jonathan-kosgei merged 4 commits into
claude/perl-ipdata-sdk-8p48pfrom
http-reuse-and-new-fields
Nov 3, 2026
Merged

jonathan-kosgei merged 4 commits into
claude/perl-ipdata-sdk-8p48pfrom
http-reuse-and-new-fields

Conversation

@jonathan-kosgei

@jonathan-kosgei jonathan-kosgei commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Four commits on top of claude/perl-ipdata-sdk-8p48p (the repository's default branch): the two logical changes, then two CI-driven follow-ups.

  1. 6243dbb Reuse the HTTPS connection across requests. Net::IPData already built one HTTP::Tiny per object, and HTTP::Tiny has kept one connection alive per object since 0.041, but nothing made that explicit, configurable or documented. new now passes keep_alive => $args{keep_alive} // 1 (and keep_alive_timeout, only when the caller supplied it) to HTTP::Tiny->new, accepts ua => $obj to inject any object with HTTP::Tiny-compatible get/post (returning {status, content, headers, success}) instead of building an HTTP::Tiny, stores all three values and adds the accessors keep_alive, keep_alive_timeout and ua (the previously unused Scalar::Util::blessed import now validates ua). bulk is a non-idempotent POST that HTTP::Tiny never retries, so the module retries it exactly once when the response is HTTP::Tiny's status 599 with Socket closed|Unexpected end of stream|Connection reset, i.e. the reused socket was dead before any response arrived; HTTP statuses are never retried. Keep-alive only: HTTP::Tiny speaks HTTP/1.1 and nothing advertises h2 via ALPN (SSL_options is untouched). Makefile.PL requires HTTP::Tiny 0.049 (fork- and thread-safe keep-alive). Docs: POD section "CONNECTION REUSE" after CONSTRUCTOR, README section "Connection reuse" (after the Usage section, before Constructor Options) plus the keep_alive, keep_alive_timeout and ua rows in the options table, Changes entry 1.1.0 2026-11-03, $VERSION 1.0.0 -> 1.1.0. CI: push.branches: ['*'] could never match the slash-containing default branch and pull_request.branches: [main] targeted a branch that does not exist, so the workflow had never run; both filters are removed and cpanm --installdeps . runs after the setup-perl step so every Perl in the matrix tests the HTTP::Tiny the prerequisite promises. release.yml is unchanged.
  2. a5fae02 Add is_satellite, is_anycast and blocklist date fields. is_satellite and is_anycast join %VALID_FIELDS (fields => filtering) and %PATH_FIELDS (lookup_field), with is_satellite($ip) and is_anycast($ip) convenience methods beside is_eu($ip). The decoded JSON passes through unchanged (JSON::PP tolerates unknown keys), so the new keys already reach callers; POD and README now document the five names and the "present only when true" behaviour. New t/05-response-fields.t with fixtures built from the three API sample responses (t/fixtures/), driven through the new ua => seam.
  3. 7e71d73 Make Makefile.PL work with the MakeMaker bundled in old Perls (CI follow-up). The first CI run failed on Perl 5.14: cpanm --installdeps . installed nothing because ExtUtils::MakeMaker 6.57 does not know TEST_REQUIRES and cannot express a version-2 META_MERGE (it writes a MYMETA.yml that claims metadata version 2 with the 1.4 layout, so CPAN::Meta drops the top-level requires), and the tests then ran against core HTTP::Tiny 0.012. Makefile.PL now builds its arguments in a hash and, on MakeMaker older than 6.64 folds TEST_REQUIRES into PREREQ_PM, older than 6.68 drops META_MERGE; modern MakeMaker (including the Perl 5.38 used by release.yml) gets exactly the same arguments as before.
  4. 223aad3 Make keep_alive_timeout effective on HTTP::Tiny 0.092 and later (follow-up). HTTP::Tiny 0.092 added keep_alive_timeout (an idle timeout: seconds a kept-alive connection may sit unused before the next request opens a fresh one) and documents it as a constructor attribute, but its constructor, checked up to 0.096/0.097, copies only @attributes, which does not include it, so the argument is silently ignored (its own t/170_keepalive.t misses this because the can_read mock alone decides the reuse outcome). The module forwards the option to the constructor and, on HTTP::Tiny 0.092 or later, also sets the attribute directly; older HTTP::Tiny has no such setting and keeps ignoring it. POD, README and Changes were corrected (they had described the option as inert with any stock HTTP::Tiny, true only up to 0.090), 60 s is recommended, and t/01 checks that the value reaches the HTTP::Tiny object where the attribute exists (runs for real on CI's Perl 5.14 and 5.20, skipped on older HTTP::Tiny).

What changes for users

  • Nothing to do for existing code: every constructor argument and method keeps working and the default behaviour is unchanged (HTTP::Tiny already kept the connection alive); it is now explicit, configurable and documented. Create one Net::IPData per process (or per API key or endpoint), keep it and reuse it for every lookup: each new object pays a fresh TLS handshake, and HTTP/2 is not available with HTTP::Tiny, so the saving is the handshake, not multiplexing.
  • New constructor options: keep_alive (default 1; 0 opens a connection per request), keep_alive_timeout (seconds a kept-alive connection may sit idle before the next request opens a fresh one; honoured by HTTP::Tiny 0.092 and later, ignored by older HTTP::Tiny, where no idle timeout is emulated; forwarded only when set, so HTTP::Tiny's own default of no idle limit applies otherwise; 60 suits api.ipdata.co, which keeps idle connections open for roughly two minutes), and ua => $obj (inject a pre-configured HTTP::Tiny, for example with verify_SSL/proxy settings, or a mock in tests). New read accessors keep_alive, keep_alive_timeout, ua.
  • bulk retries once when a kept-alive socket turns out to be dead before any response arrived (599 + Socket closed / Unexpected end of stream / Connection reset); never on an HTTP status. GET requests are retried by HTTP::Tiny itself, as before.
  • Lifecycle: nothing to close; the socket is released when the object is destroyed (HTTP::Tiny has no public way to drop its kept-alive socket, so no close method is added). A Net::IPData object is not shared between threads; HTTP::Tiny 0.049+ never reuses a socket inherited across fork or threads, so a forked child opens its own connection.
  • New prerequisite: HTTP::Tiny >= 0.049 (bundled with Perl 5.22 and later; on older Perls cpanm --installdeps . installs it, which the fixed Makefile.PL now makes work with their old MakeMaker too).
  • New response fields: is_satellite and is_anycast are accepted in fields => and lookup_field, and is_satellite($ip) / is_anycast($ip) exist. Both keys are present only when true, so $result->{is_satellite} is undef (false) when absent; test with a plain boolean, not exists. Each threat.blocklists entry may carry first_seen and last_seen (YYYY-MM-DD strings, UTC, not parsed into dates) and days_seen (integer); some feeds omit all three.

Release notes

Version in the repository: 1.1.0 ($Net::IPData::VERSION, Changes). Tag to push after merge: v1.1.0; publishing the GitHub release triggers release.yml, which uploads Net-IPData-1.1.0.tar.gz to CPAN.

Net::IPData 1.1.0

Connection reuse
- A Net::IPData object keeps one persistent HTTPS connection (HTTP/1.1 keep-alive)
  for as long as it lives; create one per process and reuse it for every lookup.
  keep_alive => 1 is now explicit and configurable. HTTP/2 is not available with
  HTTP::Tiny, so the saving is the TLS handshake, not multiplexing.
- New constructor options keep_alive (default 1), keep_alive_timeout (idle timeout
  in seconds, honoured by HTTP::Tiny 0.092 and later, forwarded only when set) and
  ua (inject any object with HTTP::Tiny-compatible get/post), with matching
  accessors.
- bulk() retries the POST once when a kept-alive connection turns out to be dead
  before any response arrived (HTTP::Tiny status 599 with Socket closed /
  Unexpected end of stream / Connection reset); HTTP statuses are never retried.
- Requires HTTP::Tiny 0.049 or later (fork- and thread-safe keep-alive; bundled
  with Perl 5.22+, older Perls install it from CPAN).

New response fields
- Top-level is_satellite and is_anycast (present only when true; a missing key
  reads as false), accepted in fields => and lookup_field, with is_satellite($ip)
  and is_anycast($ip) convenience methods.
- threat.blocklists entries may carry first_seen and last_seen (YYYY-MM-DD strings,
  UTC) and days_seen (integer); all three are optional and passed through unchanged.

CI now runs on every push and pull request and installs the declared prerequisites
before testing; Makefile.PL works with the MakeMaker bundled in Perl 5.14.

Verification

Local, perl 5.38.2 with core HTTP::Tiny 0.086, JSON::PP 4.16, Test::More 1.302194 and IO::Socket::SSL 2.085:

  • Baseline on the default branch: prove -l t -> Files=4, Tests=52, Result: PASS (t/04-live.t skips without IPDATA_API_KEY).
  • After commit 1: prove -l t -> Files=4, Tests=95, Result: PASS; perl Makefile.PL && make && make test -> 95 PASS.
  • After commit 2: prove -l t -> Files=5, Tests=155, Result: PASS (t/01 34, t/02 11, t/03 50, t/04 skipped, t/05 60); perl Makefile.PL && make && make test -> 155 PASS; make distcheck -> MANIFEST complete; perl -Ilib -c lib/Net/IPData.pm and podchecker lib/Net/IPData.pm -> OK.
  • After commit 3 (Makefile.PL): the fallback branch was exercised by temporarily raising the version thresholds so that it ran under MakeMaker 7.70: the resulting MYMETA lists HTTP::Tiny 0.049 (plus Test::More) under requires and make test passed 155; with the real thresholds MYMETA.json is unchanged (metadata version 2, repository and recommends intact); prove -l t 155 PASS.
  • After commit 4 (keep_alive_timeout), before the final push: prove -l t -> Files=5, Tests=156, Result: PASS (the new t/01 check is skipped on HTTP::Tiny 0.086, which predates the attribute); perl Makefile.PL && make && make test -> 156 PASS; make distcheck clean; podchecker OK.

Two GETs and one POST to api.ipdata.co through the module's own HTTP::Tiny were served over HTTP/1.1 with Connection: keep-alive (HTTP::Tiny does not negotiate HTTP/2).

CI (the workflow had never run on this repository before this change):

Noticed, not changed

  • verify_SSL is not set, so HTTP::Tiny below 0.083 does not verify server certificates (0.083 and later verify by default). Users who need a specific TLS setup on an older HTTP::Tiny can now pass a configured HTTP::Tiny through ua.
  • t/02-validation.t sends real requests to api.ipdata.co with a dummy key (its "valid fields pass validation" cases rely on a network or 401 error), so make test needs network access even though the README says no API key is needed.
  • Content-Type: application/json is sent as a default header on GET requests too.
  • The CI matrix stops at Perl 5.40 (5.42 exists) and keeps GitHub's default fail-fast, so one failing Perl cancels the jobs still running (seen in run 1); not modernised.
  • The documented threat shape (POD and README) still omits is_vpn and scores, which the API returns (both appear in its sample responses); only the five new names were added.
  • Single-field lookups of bare JSON booleans (is_eu, and now is_satellite / is_anycast) rely on decode_json accepting a scalar top-level value: JSON::PP before 4.0 (core before Perl 5.30) croaks, the module falls back to returning the raw text, and the string 'false' is true in Perl. Pre-existing for is_eu.
  • Upstream HTTP::Tiny 0.092-0.097: the documented keep_alive_timeout constructor argument is ignored by new, which copies only @attributes (allow_credentialed_redirects allow_downgrade cookie_jar default_headers http_proxy https_proxy keep_alive local_address max_redirect max_size proxy no_proxy SSL_options verify_SSL), while _open_handle reads $self->{keep_alive_timeout} and HTTP::Tiny::Handle::_has_keep_alive_expired applies it against last_used (idle semantics); worked around here (see commit 4), worth reporting to Perl-Toolchain-Gang/HTTP-Tiny. Nothing in 0.087-0.097 adds ALPN or HTTP/2.
  • $orig_get / $orig_post in t/03-mock-responses.t are saved but never used.
  • examples/basic.pl uses use lib '../lib', so it only runs from inside examples/.
  • release.yml builds and tests on Perl 5.38 only and does not run cpanm --installdeps .; its core HTTP::Tiny 0.086 satisfies the new prerequisite.

🤖 Generated with Claude Code

Net::IPData already built one HTTP::Tiny per object, and HTTP::Tiny has
kept one connection alive per object since 0.041, but nothing made that
explicit, configurable or documented. A first request to api.ipdata.co
costs roughly 660-980 ms (TCP plus TLS 1.3 handshake); a request on a
reused connection about 220-300 ms, so a client that is kept and reused
saves the handshake on every lookup after the first.

- Pass keep_alive (default 1) and, only when the caller supplied it,
  keep_alive_timeout to HTTP::Tiny->new; accept ua => $obj to inject any
  object with HTTP::Tiny-compatible get/post instead of building an
  HTTP::Tiny. Store all three and add accessors next to the existing
  ones. HTTP/1.1 keep-alive only: HTTP::Tiny cannot speak HTTP/2 and
  nothing advertises h2 via ALPN.
- bulk() is a non-idempotent POST that HTTP::Tiny never retries, so
  retry it exactly once when the response is HTTP::Tiny's status 599
  with a "Socket closed", "Unexpected end of stream" or "Connection
  reset" error, i.e. the reused socket was dead before any response
  arrived. HTTP statuses are never retried.
- Require HTTP::Tiny 0.049, the first release whose kept-alive socket is
  never reused across fork or threads.
- Tests: new options stored, ua injection used, same HTTP::Tiny object
  for every request, POST retried once (and only once) on the three
  stale-socket errors, not on other 599s or HTTP statuses, GET not
  retried by the module.
- Docs: CONNECTION REUSE POD section, README "Connection reuse" section
  and option rows, Changes entry for 1.1.0; $VERSION 1.0.0 -> 1.1.0.
- CI: push.branches ['*'] could never match the default branch (it
  contains a slash) and pull_request targeted a non-existent "main", so
  the workflow never ran; drop both filters and run cpanm --installdeps
  so every Perl in the matrix tests the HTTP::Tiny the prerequisite
  promises. Release workflow unchanged.

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The API now returns two more top-level booleans, is_satellite and
is_anycast, siblings of is_eu that are present only when true, and each
threat.blocklists entry may carry first_seen and last_seen (UTC calendar
dates as YYYY-MM-DD strings) and days_seen (integer); some feeds, such as
ipdata's own vpn feed, omit all three.

Net::IPData passes the decoded JSON through unchanged, so the new keys
already reach callers; this commit makes them first-class:

- is_satellite and is_anycast join %VALID_FIELDS (fields => filtering)
  and %PATH_FIELDS (lookup_field), with is_satellite($ip) and
  is_anycast($ip) convenience methods beside is_eu($ip).
- POD and README document the five names, the "present only when true"
  behaviour (a missing key reads as false; test with a plain boolean
  check, not exists), that the dates stay strings, and that the three
  blocklist fields are optional. The SYNOPSIS, README example and
  examples/basic.pl show the two flags.
- t/05-response-fields.t exercises fixtures built from the API samples
  (t/fixtures: satellite, anycast with dated blocklist entries, VPN exit
  with an undated entry) through the ua => injection seam: typed values,
  false-when-absent, optional blocklist fields, bulk items with the same
  shape, and field-name validation accepting both new names. JSON::PP
  tolerates unknown keys, which the fixtures' extra keys (scores, is_vpn)
  confirm.
- Changes entry for 1.1.0 mentions the new fields; MANIFEST lists the
  new test and fixtures.

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI on Perl 5.14 ran "cpanm --installdeps ." and installed nothing, then
failed the new keep-alive tests against the core HTTP::Tiny 0.012 (no
keep_alive accessor): ExtUtils::MakeMaker 6.57 does not know
TEST_REQUIRES and cannot express a meta-spec 2 META_MERGE. It writes a
MYMETA.yml that claims meta-spec 2 but keeps the 1.4 layout, so
CPAN::Meta treats the top-level "requires" as an unknown key and cpanm
sees no prerequisites at all.

Build the WriteMakefile arguments in a hash and, on MakeMaker older than
6.64, fold TEST_REQUIRES into PREREQ_PM; older than 6.68, drop
META_MERGE. The release tarball's META files are generated on Perl 5.38,
whose MakeMaker takes the same arguments as before, so nothing changes
for modern Perls; old ones now get a MYMETA that lists HTTP::Tiny 0.049
and cpanm installs it before the tests run.

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HTTP::Tiny 0.092 added keep_alive_timeout (seconds a kept-alive
connection may sit idle before the next request opens a fresh one) and
documents it as a constructor attribute, but its constructor, checked up
to 0.096, never copies the argument into the object, so forwarding the
option to HTTP::Tiny->new alone leaves it inert. Keep forwarding it, as
specified, and on HTTP::Tiny 0.092 or later also set the attribute
directly; older HTTP::Tiny has no such setting and keeps ignoring it.

Correct the POD, README and Changes, which described the option as
having no effect with any stock HTTP::Tiny (true only up to 0.090), and
recommend 60 seconds against api.ipdata.co, which keeps idle connections
for roughly two minutes. t/01 checks that the value reaches the
HTTP::Tiny object where the attribute exists (CI's Perl 5.14 and 5.20
install HTTP::Tiny 0.096; the check is skipped on older versions).

Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jonathan-kosgei
jonathan-kosgei merged commit 1e538d3 into claude/perl-ipdata-sdk-8p48p Nov 3, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants