From 6243dbb5596360565640fe6eddc61477e42cdc69 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 10:26:02 +0000 Subject: [PATCH 1/4] Reuse the HTTPS connection across requests Net::IPData already built one HTTP::Tiny per object, and HTTP::Tiny has kept one connection alive per object since 0.041, but nothing made that explicit, configurable or documented. A first request to api.ipdata.co costs roughly 660-980 ms (TCP plus TLS 1.3 handshake); a request on a reused connection about 220-300 ms, so a client that is kept and reused saves the handshake on every lookup after the first. - Pass keep_alive (default 1) and, only when the caller supplied it, keep_alive_timeout to HTTP::Tiny->new; accept ua => $obj to inject any object with HTTP::Tiny-compatible get/post instead of building an HTTP::Tiny. Store all three and add accessors next to the existing ones. HTTP/1.1 keep-alive only: HTTP::Tiny cannot speak HTTP/2 and nothing advertises h2 via ALPN. - bulk() is a non-idempotent POST that HTTP::Tiny never retries, so retry it exactly once when the response is HTTP::Tiny's status 599 with a "Socket closed", "Unexpected end of stream" or "Connection reset" error, i.e. the reused socket was dead before any response arrived. HTTP statuses are never retried. - Require HTTP::Tiny 0.049, the first release whose kept-alive socket is never reused across fork or threads. - Tests: new options stored, ua injection used, same HTTP::Tiny object for every request, POST retried once (and only once) on the three stale-socket errors, not on other 599s or HTTP statuses, GET not retried by the module. - Docs: CONNECTION REUSE POD section, README "Connection reuse" section and option rows, Changes entry for 1.1.0; $VERSION 1.0.0 -> 1.1.0. - CI: push.branches ['*'] could never match the default branch (it contains a slash) and pull_request targeted a non-existent "main", so the workflow never ran; drop both filters and run cpanm --installdeps so every Perl in the matrix tests the HTTP::Tiny the prerequisite promises. Release workflow unchanged. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 --- .github/workflows/ci.yml | 9 +-- Changes | 16 ++++ Makefile.PL | 2 +- README.md | 29 ++++++- lib/Net/IPData.pm | 164 +++++++++++++++++++++++++++++++++------ t/01-constructor.t | 71 ++++++++++++++++- t/03-mock-responses.t | 102 +++++++++++++++++++++++- 7 files changed, 359 insertions(+), 34 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9fbf2f6..2efc4fa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,10 +1,6 @@ name: CI -on: - push: - branches: ['*'] - pull_request: - branches: [main] +on: [push, pull_request] jobs: test: @@ -21,6 +17,9 @@ jobs: with: perl-version: ${{ matrix.perl-version }} + - name: Install dependencies + run: cpanm --installdeps . + - name: Build run: | perl Makefile.PL diff --git a/Changes b/Changes index b3977de..3600b23 100644 --- a/Changes +++ b/Changes @@ -1,5 +1,21 @@ Revision history for Net-IPData +1.1.0 2026-11-03 + - Connection reuse: the HTTP::Tiny user agent keeps one HTTPS connection + alive across requests (keep_alive => 1, now explicit and configurable); + HTTP/2 is not available with HTTP::Tiny, so the saving is the TLS + handshake, not multiplexing + - New constructor options keep_alive (default 1), keep_alive_timeout + (forwarded to HTTP::Tiny only when set) and ua (inject any object with + HTTP::Tiny-compatible get/post), with matching accessors + - bulk() retries the POST once when a kept-alive connection turns out to + be dead before any response arrived (HTTP::Tiny status 599 with Socket + closed / Unexpected end of stream / Connection reset); HTTP statuses are + never retried + - Require HTTP::Tiny 0.049 or later (fork- and thread-safe keep-alive) + - CI runs on every push and pull request and installs the declared + prerequisites before testing + 1.0.0 2026-02-23 - Initial release - Full ipdata.co API support: lookup, bulk, field filtering, single field diff --git a/Makefile.PL b/Makefile.PL index 00a6af6..852b46e 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -10,7 +10,7 @@ WriteMakefile( LICENSE => 'perl_5', MIN_PERL_VERSION => '5.010001', PREREQ_PM => { - 'HTTP::Tiny' => '0.034', # core since 5.14, need SSL support + 'HTTP::Tiny' => '0.049', # core since 5.14; 0.049+ for fork/thread-safe keep-alive 'JSON::PP' => '2.27', # core since 5.14 'Carp' => '0', 'Scalar::Util' => '0', diff --git a/README.md b/README.md index 480d0f2..cf26c1d 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ Perl client for the [ipdata.co](https://ipdata.co/) IP geolocation and threat in - **Rich data** -- geolocation, ASN, company, carrier, currency, timezone, and language info - **EU endpoint** -- route requests through EU-only servers for GDPR compliance - **Zero non-core dependencies** -- uses only `HTTP::Tiny` and `JSON::PP` (core since Perl 5.14) +- **Connection reuse** -- one persistent HTTPS connection per client object (HTTP/1.1 keep-alive), so only the first request pays the TLS handshake - **Production ready** -- input validation, structured error handling, configurable timeouts ## Installation @@ -169,6 +170,29 @@ Error messages include the HTTP status code and the API's error message: | 429 | Rate limit exceeded | | 599 | Network/connection failure | +## Connection reuse + +A `Net::IPData` object keeps one persistent HTTPS connection (HTTP/1.1 keep-alive) for as long as it lives. Create one client per process (or one per API key or endpoint), keep it, and reuse it for every lookup: each new client pays a fresh TLS handshake. HTTP/2 is not available with `HTTP::Tiny`, so the saving is the TLS handshake, not multiplexing. + +```perl +my $ipdata = Net::IPData->new(api_key => 'YOUR_API_KEY'); # one TLS handshake... + +for my $ip (@ips) { + my $result = $ipdata->lookup($ip); # ...reused by every lookup +} +``` + +The reuse is `HTTP::Tiny`'s own (`keep_alive => 1`, now explicit and configurable): a connection the server has already closed is detected and replaced transparently, a `GET` on a socket that died silently is retried once by `HTTP::Tiny` itself, and `bulk` (a non-idempotent `POST`) gets the same single retry from this module when the failure happened at the socket level before any response arrived (`HTTP::Tiny` status 599 with *Socket closed*, *Unexpected end of stream* or *Connection reset*). A request that received an HTTP status is never retried. + +A `Net::IPData` object is not shared between threads: use one per thread. It is fork-safe: `HTTP::Tiny` 0.049 and later never reuses a socket inherited from another process or thread, so a forked child opens its own connection on its first request. There is nothing to close explicitly; the connection is shut down when the object is destroyed, so letting it go out of scope (or `undef $ipdata`) at shutdown is all the cleanup needed. + +Pass `keep_alive => 0` to open a new connection per request, or supply your own user agent with `ua => $obj` (any object with `HTTP::Tiny`-compatible `get`/`post` methods returning `{status, content, headers, success}`), for example a pre-configured `HTTP::Tiny` or a mock in tests: + +```perl +my $ua = HTTP::Tiny->new(verify_SSL => 1, keep_alive => 1, timeout => 10); +my $ipdata = Net::IPData->new(api_key => 'YOUR_API_KEY', ua => $ua); +``` + ## Constructor Options | Parameter | Type | Default | Description | @@ -177,6 +201,9 @@ Error messages include the HTTP status code and the API's error message: | `eu` | bool | `0` | Use the EU endpoint (`eu-api.ipdata.co`) | | `base_url` | string | `https://api.ipdata.co` | Custom base URL (overrides `eu`) | | `timeout` | int | `30` | HTTP timeout in seconds | +| `keep_alive` | bool | `1` | Keep the HTTPS connection open between requests (HTTP/1.1 keep-alive); `0` opens a new connection per request | +| `keep_alive_timeout` | int | *unset* | Idle timeout in seconds, forwarded to `HTTP::Tiny->new` only when set; stock `HTTP::Tiny` (up to 0.086) has no such setting and ignores it | +| `ua` | object | new `HTTP::Tiny` | Your own user agent: any object with `HTTP::Tiny`-compatible `get`/`post` returning `{status, content, headers, success}` | ## Methods @@ -234,7 +261,7 @@ A full lookup returns a hashref with this shape: ## Requirements - Perl 5.10.1 or later -- Core modules only: [HTTP::Tiny](https://metacpan.org/pod/HTTP::Tiny), [JSON::PP](https://metacpan.org/pod/JSON::PP), [Carp](https://metacpan.org/pod/Carp), [Scalar::Util](https://metacpan.org/pod/Scalar::Util) +- Core modules only: [HTTP::Tiny](https://metacpan.org/pod/HTTP::Tiny) 0.049 or later (ships with Perl 5.22+; older Perls install it from CPAN), [JSON::PP](https://metacpan.org/pod/JSON::PP), [Carp](https://metacpan.org/pod/Carp), [Scalar::Util](https://metacpan.org/pod/Scalar::Util) - Recommended: [IO::Socket::SSL](https://metacpan.org/pod/IO::Socket::SSL) and [Net::SSLeay](https://metacpan.org/pod/Net::SSLeay) for HTTPS support ## Testing diff --git a/lib/Net/IPData.pm b/lib/Net/IPData.pm index c6f5e7b..31b05f4 100644 --- a/lib/Net/IPData.pm +++ b/lib/Net/IPData.pm @@ -4,7 +4,7 @@ use strict; use warnings; use 5.010001; -our $VERSION = '1.0.0'; +our $VERSION = '1.1.0'; use HTTP::Tiny; use JSON::PP qw(encode_json decode_json); @@ -49,22 +49,42 @@ sub new { $base_url //= BASE_URL; $base_url =~ s{/+$}{}; # strip trailing slashes - my $timeout = $args{timeout} // 30; - - my $ua = HTTP::Tiny->new( - agent => "Net-IPData-Perl/$VERSION", - timeout => $timeout, - default_headers => { - 'Accept' => 'application/json', - 'Content-Type' => 'application/json', - }, - ); + my $timeout = $args{timeout} // 30; + my $keep_alive = $args{keep_alive} // 1; + my $keep_alive_timeout = $args{keep_alive_timeout}; + + # One user agent per Net::IPData object: it owns the kept-alive HTTPS + # connection, so building it once here (and never per request) is what + # amortises the TCP+TLS handshake across lookups. + my $ua = $args{ua}; + if (defined $ua) { + croak 'Parameter "ua" must be an object with "get" and "post" methods' + unless blessed($ua) && $ua->can('get') && $ua->can('post'); + } + else { + $ua = HTTP::Tiny->new( + agent => "Net-IPData-Perl/$VERSION", + timeout => $timeout, + keep_alive => $keep_alive, + # HTTP::Tiny ignores constructor keys it does not know, so the + # idle timeout is only forwarded when the caller asked for one. + (defined $keep_alive_timeout + ? (keep_alive_timeout => $keep_alive_timeout) + : ()), + default_headers => { + 'Accept' => 'application/json', + 'Content-Type' => 'application/json', + }, + ); + } return bless { - api_key => $api_key, - base_url => $base_url, - ua => $ua, - timeout => $timeout, + api_key => $api_key, + base_url => $base_url, + ua => $ua, + timeout => $timeout, + keep_alive => $keep_alive, + keep_alive_timeout => $keep_alive_timeout, }, $class; } @@ -170,9 +190,12 @@ sub is_eu { # --- Accessors --- -sub api_key { return $_[0]->{api_key} } -sub base_url { return $_[0]->{base_url} } -sub timeout { return $_[0]->{timeout} } +sub api_key { return $_[0]->{api_key} } +sub base_url { return $_[0]->{base_url} } +sub timeout { return $_[0]->{timeout} } +sub keep_alive { return $_[0]->{keep_alive} } +sub keep_alive_timeout { return $_[0]->{keep_alive_timeout} } +sub ua { return $_[0]->{ua} } # --- Private Methods --- @@ -195,11 +218,31 @@ sub _get { sub _post { my ($self, $url, $data) = @_; - my $body = encode_json($data); + my $body = encode_json($data); my $response = $self->{ua}->post($url, { content => $body }); + + # A kept-alive connection can be closed by the server while idle. + # HTTP::Tiny retries idempotent methods on such a socket itself but never + # a POST, so retry exactly once when the failure was at the socket level + # (status 599) rather than an HTTP status. HTTP::Tiny has already dropped + # the dead connection by then, so the retry opens a fresh one. + if (_is_stale_connection_error($response)) { + $response = $self->{ua}->post($url, { content => $body }); + } + return $self->_handle_response($response); } +# True for the status-599 pseudo-responses HTTP::Tiny returns when a reused +# socket turned out to be dead; never true for a real HTTP status. +sub _is_stale_connection_error { + my ($response) = @_; + + return 0 unless ($response->{status} // 0) == 599; + my $content = $response->{content} // ''; + return $content =~ /Socket closed|Unexpected end of stream|Connection reset/ ? 1 : 0; +} + sub _handle_response { my ($self, $response) = @_; @@ -294,12 +337,74 @@ has no non-core dependencies. =head2 new my $ipdata = Net::IPData->new( - api_key => 'YOUR_API_KEY', # required - eu => 0, # use EU endpoint (default: 0) - base_url => '...', # custom base URL (overrides eu) - timeout => 30, # HTTP timeout in seconds (default: 30) + api_key => 'YOUR_API_KEY', # required + eu => 0, # use EU endpoint (default: 0) + base_url => '...', # custom base URL (overrides eu) + timeout => 30, # HTTP timeout in seconds (default: 30) + keep_alive => 1, # reuse the HTTPS connection (default: 1) + keep_alive_timeout => 60, # idle timeout in seconds (default: unset) + ua => $user_agent, # your own HTTP::Tiny-compatible object ); +=over 4 + +=item keep_alive + +Whether the underlying L keeps its connection to the API open +between requests (default 1). Set it to 0 to open a fresh connection for +every request. Not applied when C is supplied. + +=item keep_alive_timeout + +Idle timeout in seconds for the kept-alive connection. It is forwarded to +C<< HTTP::Tiny->new >> only when you set it. HTTP::Tiny itself (up to and +including 0.086) has no idle-timeout setting and ignores constructor keys it +does not know, so with the stock user agent the value is stored (see +L) but changes nothing: HTTP::Tiny detects a connection the +server has closed and opens a new one. Not applied when C is supplied. + +=item ua + +An object to use instead of a new L: anything with +HTTP::Tiny-compatible C and C methods that +return a hashref with C, C, C and C. Use +it to share one pre-configured HTTP::Tiny (proxy, C, +C) or to inject a mock in tests. C, C and +C are not applied to a supplied object; configure it +yourself. + +=back + +=head1 CONNECTION REUSE + +A C object keeps one persistent HTTPS connection (HTTP/1.1 +keep-alive) for as long as it lives. Create one client per process (or one +per API key or endpoint), keep it, and reuse it for every lookup: each new +client pays a fresh TLS handshake. HTTP/2 is not available with +L, so the saving is the TLS handshake, not multiplexing. + +The reuse is HTTP::Tiny's own (C<< keep_alive => 1 >>): the connection is +reused while the server keeps it open, a connection the server has already +closed is detected and replaced transparently, and a GET on a socket that +died silently is retried once by HTTP::Tiny itself. The non-idempotent +C request gets the same single retry from this module, but only +when the failure happened at the socket level before any response arrived +(HTTP::Tiny status 599 with a I, I +or I error). A request that received an HTTP status is +never retried. + +A C object is not shared between threads: use one object per +thread. It is fork-safe: HTTP::Tiny 0.049 and later records the process and +thread that opened the connection and never reuses a socket inherited from +another process or thread, so a forked child opens its own connection on +its first request. + +There is nothing to close explicitly. The connection is shut down when the +object is destroyed, so letting it go out of scope (or C) at +shutdown is all the cleanup needed. Pass C<< keep_alive => 0 >> to open a +new connection per request, or your own C to control the transport +completely. + =head1 METHODS =head2 lookup($ip, %opts) @@ -335,7 +440,9 @@ on the field. Look up multiple IP addresses in a single request (maximum 100). Sends a POST request to the C endpoint. Accepts the same C -option as C. +option as C. If the kept-alive connection turns out to be dead +before any response arrived, the request is retried once (see +L). Returns an arrayref of result hashrefs. @@ -375,6 +482,12 @@ The following methods call C for common fields: =item timeout - The HTTP timeout value +=item keep_alive - Whether the HTTPS connection is kept alive between requests (default 1) + +=item keep_alive_timeout - The idle timeout given to the constructor, or undef + +=item ua - The user agent object (an L unless one was injected) + =back =head1 ERROR HANDLING @@ -418,7 +531,8 @@ Perl 5.10.1 or later. Only core modules are required: =over 4 -=item * L (core since Perl 5.14) +=item * L 0.049 or later (core since Perl 5.14; 0.049 ships +with Perl 5.22 and later, older Perls install it from CPAN) =item * L (core since Perl 5.14) diff --git a/t/01-constructor.t b/t/01-constructor.t index 6d6c45c..71b5db5 100644 --- a/t/01-constructor.t +++ b/t/01-constructor.t @@ -1,7 +1,7 @@ #!/usr/bin/env perl use strict; use warnings; -use Test::More tests => 14; +use Test::More tests => 34; use_ok('Net::IPData'); @@ -48,3 +48,72 @@ like($@, qr/ip.*missing/i, 'lookup_field requires ip'); eval { $ipd->lookup_field('8.8.8.8', '') }; like($@, qr/field.*missing/i, 'lookup_field requires field'); + +# --- connection reuse options (defaults) --- +is($ipd->keep_alive, 1, 'keep_alive defaults to 1'); +ok(!defined $ipd->keep_alive_timeout, 'keep_alive_timeout is unset by default'); +isa_ok($ipd->ua, 'HTTP::Tiny', 'default user agent'); +is($ipd->ua->keep_alive, 1, 'HTTP::Tiny is built with keep_alive => 1'); +is($ipd->ua->timeout, 30, 'HTTP::Tiny is built with the timeout'); +is($ipd->ua->agent, "Net-IPData-Perl/$Net::IPData::VERSION", 'HTTP::Tiny carries the Net-IPData-Perl user agent'); +my $ssl_options = $ipd->ua->SSL_options || {}; +ok(!$ssl_options->{SSL_alpn_protocols}, 'no ALPN protocols (h2) are advertised'); + +# --- keep_alive => 0 --- +my $no_keep_alive = Net::IPData->new(api_key => 'test_key', keep_alive => 0); +is($no_keep_alive->keep_alive, 0, 'keep_alive => 0 is stored'); +is($no_keep_alive->ua->keep_alive, 0, 'HTTP::Tiny is built with keep_alive => 0'); + +# --- keep_alive_timeout --- +my $idle = Net::IPData->new(api_key => 'test_key', keep_alive_timeout => 60); +is($idle->keep_alive_timeout, 60, 'keep_alive_timeout is stored'); +is($idle->keep_alive, 1, 'keep_alive still defaults to 1 alongside keep_alive_timeout'); + +# --- ua injection --- +{ + package My::Test::UA; + + sub new { return bless { calls => [] }, shift } + + sub get { + my ($self, $url) = @_; + push @{ $self->{calls} }, ['get', $url]; + return { + status => 200, + success => 1, + headers => { 'content-type' => 'application/json' }, + content => '{"ip":"8.8.8.8","city":"Mountain View"}', + }; + } + + sub post { + my ($self, $url, $opts) = @_; + push @{ $self->{calls} }, ['post', $url, $opts]; + return { + status => 200, + success => 1, + headers => { 'content-type' => 'application/json' }, + content => '[{"ip":"8.8.8.8"}]', + }; + } +} + +my $mock_ua = My::Test::UA->new; +my $injected = Net::IPData->new(api_key => 'test_key', ua => $mock_ua); +is($injected->ua, $mock_ua, 'injected ua is used as-is'); +is($injected->keep_alive, 1, 'keep_alive is still recorded with an injected ua'); + +my $result = $injected->lookup('8.8.8.8'); +is($result->{city}, 'Mountain View', 'lookup goes through the injected ua'); +is($mock_ua->{calls}[0][0], 'get', 'injected ua received a GET'); +is($mock_ua->{calls}[0][1], 'https://api.ipdata.co/8.8.8.8?api-key=test_key', 'injected ua received the lookup URL'); + +$injected->bulk(['8.8.8.8']); +is($mock_ua->{calls}[1][0], 'post', 'bulk goes through the injected ua as a POST'); +is($mock_ua->{calls}[1][2]{content}, '["8.8.8.8"]', 'bulk body is the JSON array of IPs'); + +eval { Net::IPData->new(api_key => 'test_key', ua => 'not an object') }; +like($@, qr/"ua"/, 'rejects a ua that is not an object'); + +eval { Net::IPData->new(api_key => 'test_key', ua => {}) }; +like($@, qr/"ua"/, 'rejects an unblessed ua'); diff --git a/t/03-mock-responses.t b/t/03-mock-responses.t index def8546..4a3ca36 100644 --- a/t/03-mock-responses.t +++ b/t/03-mock-responses.t @@ -1,14 +1,16 @@ #!/usr/bin/env perl use strict; use warnings; -use Test::More tests => 27; +use Test::More tests => 50; use JSON::PP qw(encode_json decode_json); +use Scalar::Util qw(refaddr); use_ok('Net::IPData'); # --- Mock HTTP::Tiny to avoid real network calls --- my @mock_responses; +my @mock_calls; # [ method, HTTP::Tiny object, url, opts ] for every request made { no warnings 'redefine'; @@ -18,11 +20,13 @@ my @mock_responses; *HTTP::Tiny::get = sub { my ($self, $url) = @_; + push @mock_calls, ['get', $self, $url]; return _next_mock_response($url); }; *HTTP::Tiny::post = sub { my ($self, $url, $opts) = @_; + push @mock_calls, ['post', $self, $url, $opts]; return _next_mock_response($url, $opts); }; } @@ -225,3 +229,99 @@ my $ipd = Net::IPData->new(api_key => 'test_key_123'); like($@, qr/Network error/i, 'network failure raises error'); like($@, qr/timed out/i, 'network error includes reason'); } + +sub mock_network_error { + my ($text) = @_; + push @mock_responses, { + status => 599, + content => $text, + headers => { 'content-type' => 'text/plain' }, + success => 0, + }; +} + +# --- Test: every request on one object goes through the same HTTP::Tiny --- +{ + my %seen = map { refaddr($_->[1]) => 1 } @mock_calls; + is(scalar keys %seen, 1, 'all requests so far went through a single HTTP::Tiny object'); + is((keys %seen)[0], refaddr($ipd->ua), 'that object is the one the ua accessor returns'); + isa_ok($ipd->ua, 'HTTP::Tiny', 'ua'); + ok($ipd->ua->keep_alive, 'the HTTP::Tiny object has keep_alive enabled'); +} + +# --- Test: bulk (POST) is retried once when the kept-alive socket was dead --- +{ + @mock_calls = (); + mock_network_error("Socket closed by remote server: Broken pipe\n"); + mock_json_response(200, [ { ip => '8.8.8.8', country_name => 'United States' } ]); + + my $results = $ipd->bulk(['8.8.8.8']); + is(scalar @mock_calls, 2, 'bulk retried once after a "Socket closed" 599'); + is($mock_calls[0][0], 'post', 'first attempt was a POST'); + is($mock_calls[1][0], 'post', 'retry was a POST'); + is($mock_calls[1][2], $mock_calls[0][2], 'retry used the same URL'); + is($mock_calls[1][3]{content}, $mock_calls[0][3]{content}, 'retry sent the same body'); + is($results->[0]{ip}, '8.8.8.8', 'result of the retry is returned'); +} + +{ + @mock_calls = (); + mock_network_error("Unexpected end of stream\n"); + mock_json_response(200, [ { ip => '1.1.1.1' } ]); + + my $results = $ipd->bulk(['1.1.1.1']); + is(scalar @mock_calls, 2, 'bulk retried once after "Unexpected end of stream"'); + is($results->[0]{ip}, '1.1.1.1', 'result of the retry is returned'); +} + +{ + @mock_calls = (); + mock_network_error("Could not write to socket: 'Connection reset by peer'\n"); + mock_json_response(200, [ { ip => '9.9.9.9' } ]); + + my $results = $ipd->bulk(['9.9.9.9']); + is(scalar @mock_calls, 2, 'bulk retried once after "Connection reset"'); + is($results->[0]{ip}, '9.9.9.9', 'result of the retry is returned'); +} + +# --- Test: bulk is retried exactly once --- +{ + @mock_calls = (); + mock_network_error("Socket closed by remote server: Broken pipe\n"); + mock_network_error("Socket closed by remote server: Broken pipe\n"); + + eval { $ipd->bulk(['8.8.8.8']) }; + like($@, qr/Network error: Socket closed/, 'bulk fails when the single retry fails too'); + is(scalar @mock_calls, 2, 'bulk is retried exactly once'); + is(scalar @mock_responses, 0, 'no third attempt was made'); +} + +# --- Test: other network errors are not retried --- +{ + @mock_calls = (); + mock_network_error("Timed out while waiting for socket to become ready for reading\n"); + + eval { $ipd->bulk(['8.8.8.8']) }; + like($@, qr/Network error: Timed out/, 'timeout is reported as a network error'); + is(scalar @mock_calls, 1, 'a timeout 599 is not retried'); +} + +# --- Test: an HTTP status is never retried --- +{ + @mock_calls = (); + mock_json_response(500, { message => 'Internal server error' }); + + eval { $ipd->bulk(['8.8.8.8']) }; + like($@, qr/500/, 'bulk HTTP error is propagated'); + is(scalar @mock_calls, 1, 'an HTTP status is never retried'); +} + +# --- Test: GET is not retried by the module (HTTP::Tiny retries idempotent requests itself) --- +{ + @mock_calls = (); + mock_network_error("Socket closed by remote server: Broken pipe\n"); + + eval { $ipd->lookup('8.8.8.8') }; + like($@, qr/Network error: Socket closed/, 'GET stale-connection error is reported'); + is(scalar @mock_calls, 1, 'GET is not retried by the module'); +} From a5fae02882440affaeb669f18f2a417c1dc6b33e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 10:32:03 +0000 Subject: [PATCH 2/4] Add is_satellite, is_anycast and blocklist date fields The API now returns two more top-level booleans, is_satellite and is_anycast, siblings of is_eu that are present only when true, and each threat.blocklists entry may carry first_seen and last_seen (UTC calendar dates as YYYY-MM-DD strings) and days_seen (integer); some feeds, such as ipdata's own vpn feed, omit all three. Net::IPData passes the decoded JSON through unchanged, so the new keys already reach callers; this commit makes them first-class: - is_satellite and is_anycast join %VALID_FIELDS (fields => filtering) and %PATH_FIELDS (lookup_field), with is_satellite($ip) and is_anycast($ip) convenience methods beside is_eu($ip). - POD and README document the five names, the "present only when true" behaviour (a missing key reads as false; test with a plain boolean check, not exists), that the dates stay strings, and that the three blocklist fields are optional. The SYNOPSIS, README example and examples/basic.pl show the two flags. - t/05-response-fields.t exercises fixtures built from the API samples (t/fixtures: satellite, anycast with dated blocklist entries, VPN exit with an undated entry) through the ua => injection seam: typed values, false-when-absent, optional blocklist fields, bulk items with the same shape, and field-name validation accepting both new names. JSON::PP tolerates unknown keys, which the fixtures' extra keys (scores, is_vpn) confirm. - Changes entry for 1.1.0 mentions the new fields; MANIFEST lists the new test and fixtures. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 --- Changes | 6 ++ MANIFEST | 4 + README.md | 24 ++++- examples/basic.pl | 2 + lib/Net/IPData.pm | 61 +++++++++++- t/05-response-fields.t | 167 +++++++++++++++++++++++++++++++++ t/fixtures/anycast.json | 35 +++++++ t/fixtures/satellite.json | 10 ++ t/fixtures/vpn_blocklists.json | 18 ++++ 9 files changed, 322 insertions(+), 5 deletions(-) create mode 100644 t/05-response-fields.t create mode 100644 t/fixtures/anycast.json create mode 100644 t/fixtures/satellite.json create mode 100644 t/fixtures/vpn_blocklists.json diff --git a/Changes b/Changes index 3600b23..2ed358d 100644 --- a/Changes +++ b/Changes @@ -15,6 +15,12 @@ Revision history for Net-IPData - Require HTTP::Tiny 0.049 or later (fork- and thread-safe keep-alive) - CI runs on every push and pull request and installs the declared prerequisites before testing + - New API response fields: top-level is_satellite and is_anycast (present + only when true; a missing key reads as false) and first_seen, last_seen + and days_seen on threat.blocklists entries (optional; dates kept as + YYYY-MM-DD strings); the JSON passes through unchanged + - is_satellite and is_anycast are accepted in fields => and lookup_field, + with is_satellite($ip) and is_anycast($ip) convenience methods 1.0.0 2026-02-23 - Initial release diff --git a/MANIFEST b/MANIFEST index cc88610..81e3d00 100644 --- a/MANIFEST +++ b/MANIFEST @@ -7,4 +7,8 @@ t/01-constructor.t t/02-validation.t t/03-mock-responses.t t/04-live.t +t/05-response-fields.t +t/fixtures/anycast.json +t/fixtures/satellite.json +t/fixtures/vpn_blocklists.json examples/basic.pl diff --git a/README.md b/README.md index cf26c1d..f1f026c 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,8 @@ printf "Timezone: %s\n", $result->{time_zone}{name}; printf "Currency: %s (%s)\n", $result->{currency}{name}, $result->{currency}{code}; printf "EU: %s\n", $result->{is_eu} ? 'Yes' : 'No'; printf "Threat: %s\n", $result->{threat}{is_threat} ? 'Yes' : 'No'; +printf "Satellite: %s\n", $result->{is_satellite} ? 'Yes' : 'No'; # key present only when true +printf "Anycast: %s\n", $result->{is_anycast} ? 'Yes' : 'No'; # key present only when true ``` ### Look Up Your Own IP @@ -122,6 +124,8 @@ my $langs = $ipdata->languages('8.8.8.8'); # arrayref my $country = $ipdata->country_name('8.8.8.8'); # string my $code = $ipdata->country_code('8.8.8.8'); # string my $eu = $ipdata->is_eu('8.8.8.8'); # boolean +my $sat = $ipdata->is_satellite('8.8.8.8'); # boolean +my $anycast = $ipdata->is_anycast('8.8.8.8'); # boolean # Or use the generic method for any path-accessible field my $city = $ipdata->lookup_field('8.8.8.8', 'city'); @@ -222,6 +226,8 @@ my $ipdata = Net::IPData->new(api_key => 'YOUR_API_KEY', ua => $ua); | `country_name($ip)` | string | Country name | | `country_code($ip)` | string | ISO 3166-1 alpha-2 country code | | `is_eu($ip)` | boolean | EU member state | +| `is_satellite($ip)` | boolean | Satellite internet provider | +| `is_anycast($ip)` | boolean | Anycast address | ## API Response Structure @@ -231,6 +237,8 @@ A full lookup returns a hashref with this shape: { ip => "8.8.8.8", is_eu => false, + is_satellite => true, # present only when true + is_anycast => true, # present only when true city => "Mountain View", region => "California", region_code => "CA", @@ -254,10 +262,24 @@ A full lookup returns a hashref with this shape: time_zone => { name, abbr, offset, is_dst, current_time }, threat => { is_tor, is_icloud_relay, is_proxy, is_datacenter, is_anonymous, is_known_attacker, is_known_abuser, - is_threat, is_bogon }, + is_threat, is_bogon, + blocklists => [{ name, site, type, + first_seen, last_seen, days_seen }] }, } ``` +The JSON is passed through unchanged, so keys the API adds later simply appear in the hashref. Five fields deserve a note: + +| Field | Type | Description | +|-------|------|-------------| +| `is_satellite` | boolean | The address belongs to a satellite internet provider. The API includes the key only when it is true, so test it with a plain `if ($result->{is_satellite})`, never with `exists`; a missing key reads as false | +| `is_anycast` | boolean | The address is an anycast address. Present only when true, like `is_satellite` | +| `threat.blocklists[].first_seen` | string | First calendar date (UTC, `YYYY-MM-DD`) on which that blocklist listed the address; left as a string | +| `threat.blocklists[].last_seen` | string | Most recent such date (UTC, `YYYY-MM-DD`); left as a string | +| `threat.blocklists[].days_seen` | integer | Number of days that blocklist has listed the address | + +Some feeds, for example ipdata's own vpn feed, omit all three blocklist fields, so check for them with `exists` before use. Bulk results carry the same shape per item. + ## Requirements - Perl 5.10.1 or later diff --git a/examples/basic.pl b/examples/basic.pl index 5fd8d08..b0d33a9 100644 --- a/examples/basic.pl +++ b/examples/basic.pl @@ -21,6 +21,8 @@ printf "Currency: %s (%s)\n", $result->{currency}{name}, $result->{currency}{code}; printf "EU: %s\n", $result->{is_eu} ? 'Yes' : 'No'; printf "Threat: %s\n", $result->{threat}{is_threat} ? 'Yes' : 'No'; +printf "Satellite: %s\n", $result->{is_satellite} ? 'Yes' : 'No'; # key present only when true +printf "Anycast: %s\n", $result->{is_anycast} ? 'Yes' : 'No'; # key present only when true print "\n--- Bulk lookup ---\n"; my $bulk = $ipdata->bulk(['8.8.8.8', '1.1.1.1', '9.9.9.9']); diff --git a/lib/Net/IPData.pm b/lib/Net/IPData.pm index 31b05f4..3a50959 100644 --- a/lib/Net/IPData.pm +++ b/lib/Net/IPData.pm @@ -20,7 +20,7 @@ use constant { # Valid top-level fields for filtering my %VALID_FIELDS = map { $_ => 1 } qw( - ip is_eu city region region_code region_type + ip is_eu is_satellite is_anycast city region region_code region_type country_name country_code continent_name continent_code latitude longitude postal calling_code flag emoji_flag emoji_unicode asn company carrier @@ -29,7 +29,7 @@ my %VALID_FIELDS = map { $_ => 1 } qw( # Valid single-field path lookups (fields accessible via /{ip}/{field}) my %PATH_FIELDS = map { $_ => 1 } qw( - ip is_eu city region region_code region_type + ip is_eu is_satellite is_anycast city region region_code region_type country_name country_code continent_name continent_code latitude longitude postal calling_code flag emoji_flag emoji_unicode asn carrier languages @@ -188,6 +188,16 @@ sub is_eu { return $self->lookup_field($ip, 'is_eu'); } +sub is_satellite { + my ($self, $ip) = @_; + return $self->lookup_field($ip, 'is_satellite'); +} + +sub is_anycast { + my ($self, $ip) = @_; + return $self->lookup_field($ip, 'is_anycast'); +} + # --- Accessors --- sub api_key { return $_[0]->{api_key} } @@ -306,6 +316,10 @@ Net::IPData - Perl client for the ipdata.co IP geolocation and threat intelligen printf "IP: %s, Country: %s, City: %s\n", $result->{ip}, $result->{country_name}, $result->{city}; + # Flags the API includes only when true (a missing key reads as false) + print "satellite ISP\n" if $result->{is_satellite}; + print "anycast address\n" if $result->{is_anycast}; + # Look up your own IP address my $me = $ipdata->lookup_mine(); @@ -470,6 +484,10 @@ The following methods call C for common fields: =item is_eu($ip) - EU membership (boolean) +=item is_satellite($ip) - Satellite internet provider flag (boolean) + +=item is_anycast($ip) - Anycast address flag (boolean) + =back =head1 ACCESSORS @@ -510,7 +528,8 @@ when available. A full lookup returns a hashref with the following keys: - ip, is_eu, city, region, region_code, region_type, + ip, is_eu, is_satellite, is_anycast, + city, region, region_code, region_type, country_name, country_code, continent_name, continent_code, latitude, longitude, postal, calling_code, flag, emoji_flag, emoji_unicode, @@ -522,9 +541,43 @@ A full lookup returns a hashref with the following keys: time_zone => { name, abbr, offset, is_dst, current_time }, threat => { is_tor, is_icloud_relay, is_proxy, is_datacenter, is_anonymous, is_known_attacker, is_known_abuser, - is_threat, is_bogon }, + is_threat, is_bogon, + blocklists => [{ name, site, type, + first_seen, last_seen, days_seen }] }, count => N (for bulk lookups) +The JSON is passed through unchanged, so keys the API adds later simply +appear in the hashref. Five fields deserve a note: + +=over 4 + +=item is_satellite + +True when the address belongs to a satellite internet provider. The API +includes this key only when it is true and omits it otherwise, so test it +with a plain boolean check (C<< if ($result->{is_satellite}) >>), never +with C; a missing key reads as false. + +=item is_anycast + +True when the address is an anycast address. Present only when true, like +C. + +=item first_seen, last_seen (per C<< threat->{blocklists} >> entry) + +The first and the most recent calendar date (UTC, C) on which +that blocklist listed the address. The module leaves them as strings. + +=item days_seen (per C<< threat->{blocklists} >> entry) + +The number of days that blocklist has listed the address (integer). + +=back + +Some feeds, for example ipdata's own vpn feed, omit all three blocklist +fields, so check for them with C before use. Bulk results carry the +same shape per item. + =head1 REQUIREMENTS Perl 5.10.1 or later. Only core modules are required: diff --git a/t/05-response-fields.t b/t/05-response-fields.t new file mode 100644 index 0000000..b191ff2 --- /dev/null +++ b/t/05-response-fields.t @@ -0,0 +1,167 @@ +#!/usr/bin/env perl +use strict; +use warnings; +use Test::More tests => 60; +use File::Basename qw(dirname); +use File::Spec; + +use_ok('Net::IPData'); + +# --- A mock user agent injected through ua => serves queued raw JSON bodies +# exactly as the API would send them and records every request made. --- +{ + package Fixture::UA; + + sub new { return bless { queue => [], calls => [] }, shift } + + sub queue { + my ($self, $status, $body) = @_; + push @{ $self->{queue} }, [ $status, $body ]; + return $self; + } + + sub get { my ($self, $url) = @_; return $self->_respond('GET', $url) } + sub post { my ($self, $url, $opts) = @_; return $self->_respond('POST', $url, $opts) } + + sub _respond { + my ($self, $method, $url, $opts) = @_; + push @{ $self->{calls} }, [ $method, $url, $opts ]; + my $next = shift @{ $self->{queue} } or die "No fixture queued for $method $url"; + my ($status, $body) = @$next; + return { + status => $status, + success => ($status >= 200 && $status < 300) ? 1 : 0, + headers => { 'content-type' => 'application/json' }, + content => $body, + }; + } + + sub last_url { return $_[0]{calls}[-1][1] } +} + +my $fixture_dir = File::Spec->catdir(dirname(__FILE__), 'fixtures'); + +sub fixture { + my ($name) = @_; + my $path = File::Spec->catfile($fixture_dir, $name); + open my $fh, '<:raw', $path or die "Cannot read $path: $!"; + local $/; + return scalar <$fh>; +} + +my $ua = Fixture::UA->new; +my $ipd = Net::IPData->new(api_key => 'test_key', ua => $ua); + +# --- Sample A: satellite ISP; is_anycast absent --- +{ + $ua->queue(200, fixture('satellite.json')); + my $r = $ipd->lookup('98.97.34.0'); + + is($r->{ip}, '98.97.34.0', 'satellite: ip'); + ok($r->{is_satellite}, 'satellite: is_satellite is true'); + ok(!$r->{is_eu}, 'satellite: is_eu is false'); + ok(!exists $r->{is_anycast}, 'satellite: is_anycast key is absent (API omits false flags)'); + ok(!$r->{is_anycast}, 'satellite: absent is_anycast reads as false'); + is($r->{asn}{name}, 'Space Exploration Technologies Corporation', 'satellite: nested asn passes through'); + is(ref $r->{threat}{blocklists}, 'ARRAY', 'satellite: blocklists is an arrayref'); + is(scalar @{ $r->{threat}{blocklists} }, 0, 'satellite: blocklists is empty'); + is($r->{threat}{scores}{trust_score}, 97, 'satellite: keys the module does not know (scores) pass through'); + is($r->{count}, '0', 'satellite: count passes through'); +} + +# --- Sample B: anycast address with dated blocklist entries --- +{ + $ua->queue(200, fixture('anycast.json')); + my $r = $ipd->lookup('8.8.8.8'); + + ok($r->{is_anycast}, 'anycast: is_anycast is true'); + ok(!exists $r->{is_satellite}, 'anycast: is_satellite key is absent'); + ok(!$r->{is_satellite}, 'anycast: absent is_satellite reads as false'); + ok(exists $r->{city} && !defined $r->{city}, 'anycast: null city decodes to undef'); + is($r->{country_code}, 'US', 'anycast: country_code'); + + my $bl = $r->{threat}{blocklists}; + is(scalar @$bl, 2, 'anycast: two blocklist entries'); + is($bl->[0]{name}, 'ipdata', 'anycast: blocklist name'); + is($bl->[0]{type}, 'cloud', 'anycast: blocklist type'); + is($bl->[0]{first_seen}, '2026-11-02', 'anycast: first_seen'); + is($bl->[0]{last_seen}, '2026-11-02', 'anycast: last_seen'); + is($bl->[0]{days_seen}, 1, 'anycast: days_seen'); + is(ref $bl->[0]{first_seen}, '', 'anycast: first_seen stays a plain YYYY-MM-DD string, not a date object'); + like($bl->[0]{last_seen}, qr/^\d{4}-\d{2}-\d{2}$/, 'anycast: last_seen keeps the YYYY-MM-DD format'); + is($bl->[1]{type}, 'open_resolvers', 'anycast: second blocklist entry'); + is($bl->[1]{days_seen}, 1, 'anycast: second entry days_seen'); + ok($r->{threat}{is_datacenter}, 'anycast: threat flags pass through'); +} + +# --- Sample C: VPN exit; neither flag present, one blocklist entry without dates --- +{ + $ua->queue(200, fixture('vpn_blocklists.json')); + my $r = $ipd->lookup('187.13.250.1'); + + ok(!exists $r->{is_satellite} && !exists $r->{is_anycast}, 'vpn: neither top-level flag is present'); + ok(!$r->{is_satellite} && !$r->{is_anycast}, 'vpn: both flags read as false'); + ok($r->{threat}{is_vpn}, 'vpn: is_vpn is true'); + + my $bl = $r->{threat}{blocklists}; + is(scalar @$bl, 6, 'vpn: six blocklist entries'); + is($bl->[0]{name}, 'Gluetun', 'vpn: first entry name'); + is($bl->[0]{first_seen}, '2026-10-31', 'vpn: first entry first_seen'); + is($bl->[0]{last_seen}, '2026-11-02', 'vpn: first entry last_seen'); + is($bl->[0]{days_seen}, 2, 'vpn: first entry days_seen'); + + my $undated = $bl->[2]; + is($undated->{name}, 'ipdata', 'vpn: third entry is the ipdata feed'); + is($undated->{type}, 'vpn', 'vpn: third entry is the vpn feed'); + ok(!exists $undated->{first_seen}, 'vpn: undated entry has no first_seen'); + ok(!exists $undated->{last_seen}, 'vpn: undated entry has no last_seen'); + ok(!exists $undated->{days_seen}, 'vpn: undated entry has no days_seen'); + is(scalar(grep { exists $_->{days_seen} } @$bl), 5, 'vpn: the other five entries carry the dates'); +} + +# --- Bulk responses carry the same shape per item --- +{ + my $bulk_body = '[' . join(',', fixture('satellite.json'), fixture('anycast.json'), fixture('vpn_blocklists.json')) . ']'; + $ua->queue(200, $bulk_body); + my $results = $ipd->bulk(['98.97.34.0', '8.8.8.8', '187.13.250.1']); + + is(scalar @$results, 3, 'bulk: three results'); + ok($results->[0]{is_satellite} && !$results->[0]{is_anycast}, 'bulk: item 0 is satellite only'); + ok($results->[1]{is_anycast} && !$results->[1]{is_satellite}, 'bulk: item 1 is anycast only'); + ok(!$results->[2]{is_satellite} && !$results->[2]{is_anycast}, 'bulk: item 2 has neither flag'); + is($results->[1]{threat}{blocklists}[0]{first_seen}, '2026-11-02', 'bulk: dated blocklist entry'); + ok(!exists $results->[2]{threat}{blocklists}[2]{days_seen}, 'bulk: undated blocklist entry'); +} + +# --- Field-name validation accepts the two new names --- +{ + $ua->queue(200, '{"ip":"98.97.34.0","is_satellite":true}'); + my $r = $ipd->lookup('98.97.34.0', fields => [qw(ip is_satellite is_anycast)]); + like($ua->last_url, qr/[?&]fields=ip,is_satellite,is_anycast(?:&|$)/, 'lookup fields => accepts is_satellite and is_anycast'); + ok($r->{is_satellite}, 'filtered lookup returns is_satellite'); + + $ua->queue(200, '[{"ip":"8.8.8.8","is_anycast":true}]'); + $ipd->bulk(['8.8.8.8'], fields => 'is_anycast,is_satellite'); + like($ua->last_url, qr/[?&]fields=is_anycast,is_satellite(?:&|$)/, 'bulk fields => accepts both new names as a string'); + + $ua->queue(200, 'true'); + ok($ipd->lookup_field('98.97.34.0', 'is_satellite'), 'lookup_field accepts is_satellite'); + like($ua->last_url, qr{/98\.97\.34\.0/is_satellite\?}, 'lookup_field requests /{ip}/is_satellite'); + + $ua->queue(200, 'true'); + ok($ipd->lookup_field('8.8.8.8', 'is_anycast'), 'lookup_field accepts is_anycast'); + like($ua->last_url, qr{/8\.8\.8\.8/is_anycast\?}, 'lookup_field requests /{ip}/is_anycast'); + + $ua->queue(200, 'true'); + ok($ipd->is_satellite('98.97.34.0'), 'is_satellite convenience method'); + like($ua->last_url, qr{/98\.97\.34\.0/is_satellite\?}, 'is_satellite requests the is_satellite path field'); + + $ua->queue(200, 'true'); + ok($ipd->is_anycast('8.8.8.8'), 'is_anycast convenience method'); + like($ua->last_url, qr{/8\.8\.8\.8/is_anycast\?}, 'is_anycast requests the is_anycast path field'); + + eval { $ipd->lookup('8.8.8.8', fields => ['is_satelite']) }; + like($@, qr/Invalid field: is_satelite/, 'a misspelt field name is still rejected'); + + is(scalar @{ $ua->{queue} }, 0, 'every queued fixture was consumed'); +} diff --git a/t/fixtures/anycast.json b/t/fixtures/anycast.json new file mode 100644 index 0000000..67424bb --- /dev/null +++ b/t/fixtures/anycast.json @@ -0,0 +1,35 @@ +{ + "ip": "8.8.8.8", + "is_anycast": true, + "is_eu": false, + "city": null, + "region": null, + "region_code": null, + "region_type": null, + "country_name": "United States", + "country_code": "US", + "continent_name": "North America", + "continent_code": "NA", + "latitude": 39.7599983215332, + "longitude": -98.5, + "postal": null, + "calling_code": "1", + "flag": "https://ipdata.co/flags/us.png", + "emoji_flag": "🇺🇸", + "emoji_unicode": "U+1F1FA U+1F1F8", + "asn": { "asn": "AS15169", "name": "Google LLC", "domain": "about.google", "route": "8.8.8.0/24", "type": "hosting" }, + "company": { "name": "Google LLC", "domain": "about.google", "network": "8.8.8.0/24", "type": "hosting" }, + "languages": [ { "name": "English", "native": "English", "code": "en" } ], + "currency": { "name": "US Dollar", "code": "USD", "symbol": "$", "native": "$", "plural": "US dollars" }, + "time_zone": { "name": null, "abbr": null, "offset": null, "is_dst": null, "current_time": null }, + "threat": { + "is_tor": false, "is_vpn": false, "is_icloud_relay": false, "is_proxy": false, "is_datacenter": true, + "is_anonymous": false, "is_known_attacker": false, "is_known_abuser": false, "is_threat": false, "is_bogon": false, + "blocklists": [ + { "days_seen": 1, "first_seen": "2026-11-02", "last_seen": "2026-11-02", "name": "ipdata", "site": "https://ipdata.co", "type": "cloud" }, + { "days_seen": 1, "first_seen": "2026-11-02", "last_seen": "2026-11-02", "name": "ipdata", "site": "https://ipdata.co", "type": "open_resolvers" } + ], + "scores": { "vpn_score": 100, "proxy_score": 93, "threat_score": 100, "trust_score": 2 } + }, + "count": "0" +} diff --git a/t/fixtures/satellite.json b/t/fixtures/satellite.json new file mode 100644 index 0000000..c0abd6f --- /dev/null +++ b/t/fixtures/satellite.json @@ -0,0 +1,10 @@ +{ + "ip": "98.97.34.0", + "is_satellite": true, + "is_eu": false, + "city": "Seattle", + "country_code": "US", + "asn": { "asn": "AS14593", "name": "Space Exploration Technologies Corporation", "domain": "spacex.com", "route": "98.97.32.0/21", "type": "business" }, + "threat": { "is_tor": false, "is_vpn": false, "is_icloud_relay": false, "is_proxy": false, "is_datacenter": false, "is_anonymous": false, "is_known_attacker": false, "is_known_abuser": false, "is_threat": false, "is_bogon": false, "blocklists": [], "scores": { "vpn_score": 0, "proxy_score": 1, "threat_score": 8, "trust_score": 97 } }, + "count": "0" +} diff --git a/t/fixtures/vpn_blocklists.json b/t/fixtures/vpn_blocklists.json new file mode 100644 index 0000000..f4d1485 --- /dev/null +++ b/t/fixtures/vpn_blocklists.json @@ -0,0 +1,18 @@ +{ + "ip": "187.13.250.1", + "is_eu": false, + "threat": { + "is_tor": false, "is_vpn": true, "is_icloud_relay": false, "is_proxy": false, "is_datacenter": true, + "is_anonymous": true, "is_known_attacker": true, "is_known_abuser": true, "is_threat": true, "is_bogon": false, + "blocklists": [ + { "days_seen": 2, "first_seen": "2026-10-31", "last_seen": "2026-11-02", "name": "Gluetun", "site": "https://github.com/qdm12/gluetun", "type": "vpn" }, + { "days_seen": 1, "first_seen": "2026-11-02", "last_seen": "2026-11-02", "name": "ipdata", "site": "https://ipdata.co", "type": "cloud" }, + { "name": "ipdata", "site": "https://ipdata.co", "type": "vpn" }, + { "days_seen": 2, "first_seen": "2026-10-31", "last_seen": "2026-11-02", "name": "NordVPN", "site": "https://nordvpn.com", "type": "vpn" }, + { "days_seen": 2, "first_seen": "2026-10-31", "last_seen": "2026-11-02", "name": "Wikimedia Global Blocklist", "site": "https://wikimedia.org", "type": "general" }, + { "days_seen": 2, "first_seen": "2026-10-31", "last_seen": "2026-11-02", "name": "X4BNet", "site": "https://github.com/X4BNet/lists_vpn", "type": "vpn" } + ], + "scores": { "vpn_score": 100, "proxy_score": 99, "threat_score": 100, "trust_score": 0 } + }, + "count": "0" +} From 7e71d7368df24e75f6b9bebafdd686589d8aa01d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 10:39:17 +0000 Subject: [PATCH 3/4] Make Makefile.PL work with the MakeMaker bundled in old Perls CI on Perl 5.14 ran "cpanm --installdeps ." and installed nothing, then failed the new keep-alive tests against the core HTTP::Tiny 0.012 (no keep_alive accessor): ExtUtils::MakeMaker 6.57 does not know TEST_REQUIRES and cannot express a meta-spec 2 META_MERGE. It writes a MYMETA.yml that claims meta-spec 2 but keeps the 1.4 layout, so CPAN::Meta treats the top-level "requires" as an unknown key and cpanm sees no prerequisites at all. Build the WriteMakefile arguments in a hash and, on MakeMaker older than 6.64, fold TEST_REQUIRES into PREREQ_PM; older than 6.68, drop META_MERGE. The release tarball's META files are generated on Perl 5.38, whose MakeMaker takes the same arguments as before, so nothing changes for modern Perls; old ones now get a MYMETA that lists HTTP::Tiny 0.049 and cpanm installs it before the tests run. Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 --- Makefile.PL | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/Makefile.PL b/Makefile.PL index 852b46e..7207e83 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -2,7 +2,7 @@ use strict; use warnings; use ExtUtils::MakeMaker; -WriteMakefile( +my %WriteMakefileArgs = ( NAME => 'Net::IPData', VERSION_FROM => 'lib/Net/IPData.pm', ABSTRACT_FROM => 'lib/Net/IPData.pm', @@ -43,3 +43,18 @@ WriteMakefile( dist => { COMPRESS => 'gzip -9f', SUFFIX => 'gz' }, clean => { FILES => 'Net-IPData-*' }, ); + +# ExtUtils::MakeMaker before 6.64 does not know TEST_REQUIRES, and before +# 6.68 it cannot express a meta-spec 2 META_MERGE: it writes a MYMETA that +# claims version 2 but keeps the 1.4 layout, so CPAN::Meta drops the +# prerequisites and "cpanm --installdeps ." installs nothing (seen with the +# MakeMaker bundled with Perl 5.14). Fall back to what those versions know. +unless (eval { ExtUtils::MakeMaker->VERSION('6.64') }) { + my $test_requires = delete $WriteMakefileArgs{TEST_REQUIRES}; + $WriteMakefileArgs{PREREQ_PM} = { %{ $WriteMakefileArgs{PREREQ_PM} }, %$test_requires }; +} +unless (eval { ExtUtils::MakeMaker->VERSION('6.68') }) { + delete $WriteMakefileArgs{META_MERGE}; +} + +WriteMakefile(%WriteMakefileArgs); From 223aad35387d1043a1bbfbaf00c60a311564c02a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 10:49:00 +0000 Subject: [PATCH 4/4] Make keep_alive_timeout effective on HTTP::Tiny 0.092 and later HTTP::Tiny 0.092 added keep_alive_timeout (seconds a kept-alive connection may sit idle before the next request opens a fresh one) and documents it as a constructor attribute, but its constructor, checked up to 0.096, never copies the argument into the object, so forwarding the option to HTTP::Tiny->new alone leaves it inert. Keep forwarding it, as specified, and on HTTP::Tiny 0.092 or later also set the attribute directly; older HTTP::Tiny has no such setting and keeps ignoring it. Correct the POD, README and Changes, which described the option as having no effect with any stock HTTP::Tiny (true only up to 0.090), and recommend 60 seconds against api.ipdata.co, which keeps idle connections for roughly two minutes. t/01 checks that the value reaches the HTTP::Tiny object where the attribute exists (CI's Perl 5.14 and 5.20 install HTTP::Tiny 0.096; the check is skipped on older versions). Claude-Session: https://claude.ai/code/session_01VQiRp4zEJKUXHrKkFaeUke Co-Authored-By: Claude Fable 5.1 --- Changes | 3 ++- README.md | 4 ++-- lib/Net/IPData.pm | 32 ++++++++++++++++++++++---------- t/01-constructor.t | 9 ++++++++- 4 files changed, 34 insertions(+), 14 deletions(-) diff --git a/Changes b/Changes index 2ed358d..59848bf 100644 --- a/Changes +++ b/Changes @@ -6,7 +6,8 @@ Revision history for Net-IPData HTTP/2 is not available with HTTP::Tiny, so the saving is the TLS handshake, not multiplexing - New constructor options keep_alive (default 1), keep_alive_timeout - (forwarded to HTTP::Tiny only when set) and ua (inject any object with + (idle timeout in seconds, honoured by HTTP::Tiny 0.092 and later, + forwarded only when set) and ua (inject any object with HTTP::Tiny-compatible get/post), with matching accessors - bulk() retries the POST once when a kept-alive connection turns out to be dead before any response arrived (HTTP::Tiny status 599 with Socket diff --git a/README.md b/README.md index f1f026c..6d0163c 100644 --- a/README.md +++ b/README.md @@ -186,7 +186,7 @@ for my $ip (@ips) { } ``` -The reuse is `HTTP::Tiny`'s own (`keep_alive => 1`, now explicit and configurable): a connection the server has already closed is detected and replaced transparently, a `GET` on a socket that died silently is retried once by `HTTP::Tiny` itself, and `bulk` (a non-idempotent `POST`) gets the same single retry from this module when the failure happened at the socket level before any response arrived (`HTTP::Tiny` status 599 with *Socket closed*, *Unexpected end of stream* or *Connection reset*). A request that received an HTTP status is never retried. +The reuse is `HTTP::Tiny`'s own (`keep_alive => 1`, now explicit and configurable): a connection the server has already closed is detected and replaced transparently, a `GET` on a socket that died silently is retried once by `HTTP::Tiny` itself, and `bulk` (a non-idempotent `POST`) gets the same single retry from this module when the failure happened at the socket level before any response arrived (`HTTP::Tiny` status 599 with *Socket closed*, *Unexpected end of stream* or *Connection reset*). A request that received an HTTP status is never retried. With `HTTP::Tiny` 0.092 or later, `keep_alive_timeout => 60` additionally stops reusing a connection that has been idle for a minute, before the server (which keeps idle connections for roughly two minutes) can drop it silently. A `Net::IPData` object is not shared between threads: use one per thread. It is fork-safe: `HTTP::Tiny` 0.049 and later never reuses a socket inherited from another process or thread, so a forked child opens its own connection on its first request. There is nothing to close explicitly; the connection is shut down when the object is destroyed, so letting it go out of scope (or `undef $ipdata`) at shutdown is all the cleanup needed. @@ -206,7 +206,7 @@ my $ipdata = Net::IPData->new(api_key => 'YOUR_API_KEY', ua => $ua); | `base_url` | string | `https://api.ipdata.co` | Custom base URL (overrides `eu`) | | `timeout` | int | `30` | HTTP timeout in seconds | | `keep_alive` | bool | `1` | Keep the HTTPS connection open between requests (HTTP/1.1 keep-alive); `0` opens a new connection per request | -| `keep_alive_timeout` | int | *unset* | Idle timeout in seconds, forwarded to `HTTP::Tiny->new` only when set; stock `HTTP::Tiny` (up to 0.086) has no such setting and ignores it | +| `keep_alive_timeout` | int | *unset* | Seconds a kept-alive connection may sit idle before the next request opens a fresh one; honoured by `HTTP::Tiny` 0.092+ (older versions ignore it) and forwarded only when set; `60` suits api.ipdata.co | | `ua` | object | new `HTTP::Tiny` | Your own user agent: any object with `HTTP::Tiny`-compatible `get`/`post` returning `{status, content, headers, success}` | ## Methods diff --git a/lib/Net/IPData.pm b/lib/Net/IPData.pm index 3a50959..34780bb 100644 --- a/lib/Net/IPData.pm +++ b/lib/Net/IPData.pm @@ -66,8 +66,9 @@ sub new { agent => "Net-IPData-Perl/$VERSION", timeout => $timeout, keep_alive => $keep_alive, - # HTTP::Tiny ignores constructor keys it does not know, so the - # idle timeout is only forwarded when the caller asked for one. + # Forwarded only when the caller asked for one, so HTTP::Tiny's + # own default (no idle limit) applies otherwise; HTTP::Tiny + # before 0.092 has no such setting and ignores the key. (defined $keep_alive_timeout ? (keep_alive_timeout => $keep_alive_timeout) : ()), @@ -76,6 +77,13 @@ sub new { 'Content-Type' => 'application/json', }, ); + # HTTP::Tiny 0.092 and later document keep_alive_timeout as a + # constructor attribute, but their constructor (checked up to 0.096) + # never copies the argument into the object, so set the attribute + # directly where it exists. + if (defined $keep_alive_timeout && eval { HTTP::Tiny->VERSION('0.092') }) { + $ua->{keep_alive_timeout} = $keep_alive_timeout; + } } return bless { @@ -356,7 +364,7 @@ has no non-core dependencies. base_url => '...', # custom base URL (overrides eu) timeout => 30, # HTTP timeout in seconds (default: 30) keep_alive => 1, # reuse the HTTPS connection (default: 1) - keep_alive_timeout => 60, # idle timeout in seconds (default: unset) + keep_alive_timeout => 60, # idle timeout in seconds (default: none) ua => $user_agent, # your own HTTP::Tiny-compatible object ); @@ -370,12 +378,13 @@ every request. Not applied when C is supplied. =item keep_alive_timeout -Idle timeout in seconds for the kept-alive connection. It is forwarded to -C<< HTTP::Tiny->new >> only when you set it. HTTP::Tiny itself (up to and -including 0.086) has no idle-timeout setting and ignores constructor keys it -does not know, so with the stock user agent the value is stored (see -L) but changes nothing: HTTP::Tiny detects a connection the -server has closed and opens a new one. Not applied when C is supplied. +Idle timeout in seconds for the kept-alive connection: once it has gone +unused for longer than this, the next request opens a fresh connection +instead of reusing it. Honoured by L 0.092 and later, whose own +default (no limit) applies when you leave it unset; older HTTP::Tiny has no +such setting and ignores it, in which case the value is merely stored (see +L). api.ipdata.co keeps idle connections open for roughly two +minutes, so 60 is a sensible value. Not applied when C is supplied. =item ua @@ -405,7 +414,10 @@ C request gets the same single retry from this module, but only when the failure happened at the socket level before any response arrived (HTTP::Tiny status 599 with a I, I or I error). A request that received an HTTP status is -never retried. +never retried. With HTTP::Tiny 0.092 or later, C<< keep_alive_timeout => 60 >> +additionally stops reusing a connection that has been idle for a minute, +before the server (which keeps idle connections for roughly two minutes) can +drop it silently. A C object is not shared between threads: use one object per thread. It is fork-safe: HTTP::Tiny 0.049 and later records the process and diff --git a/t/01-constructor.t b/t/01-constructor.t index 71b5db5..bdb3b7f 100644 --- a/t/01-constructor.t +++ b/t/01-constructor.t @@ -1,7 +1,7 @@ #!/usr/bin/env perl use strict; use warnings; -use Test::More tests => 34; +use Test::More tests => 35; use_ok('Net::IPData'); @@ -68,6 +68,13 @@ is($no_keep_alive->ua->keep_alive, 0, 'HTTP::Tiny is built with keep_alive => 0' my $idle = Net::IPData->new(api_key => 'test_key', keep_alive_timeout => 60); is($idle->keep_alive_timeout, 60, 'keep_alive_timeout is stored'); is($idle->keep_alive, 1, 'keep_alive still defaults to 1 alongside keep_alive_timeout'); +SKIP: { + skip "HTTP::Tiny $HTTP::Tiny::VERSION predates keep_alive_timeout (0.092)", 1 + unless eval { HTTP::Tiny->VERSION('0.092') }; + # The attribute HTTP::Tiny::Handle reads to decide whether a kept-alive + # connection has sat idle for too long. + is($idle->ua->{keep_alive_timeout}, 60, 'keep_alive_timeout reaches the HTTP::Tiny object'); +} # --- ua injection --- {