[gsoc 25] WHOIS Information and IP Address-Based Geolocation

docs/developer/extending.rst

@@ -344,6 +344,7 @@ Once you have created the models, add the following to your
     CONFIG_VPNCLIENT_MODEL = "sample_config.VpnClient"
     CONFIG_ORGANIZATIONCONFIGSETTINGS_MODEL = "sample_config.OrganizationConfigSettings"
     CONFIG_ORGANIZATIONLIMITS_MODEL = "sample_config.OrganizationLimits"
+    CONFIG_WHOISINFO_MODEL = "sample_config.WHOISInfo"
     DJANGO_X509_CA_MODEL = "sample_pki.Ca"
     DJANGO_X509_CERT_MODEL = "sample_pki.Cert"
     GEO_LOCATION_MODEL = "sample_geo.Location"

docs/index.rst

@@ -48,6 +48,8 @@ the OpenWISP architecture.
     user/zerotier.rst
     user/openvpn.rst
     user/subnet-division-rules.rst
+    user/whois.rst
+    user/estimated-location.rst
     user/rest-api.rst
     user/settings.rst
 

docs/user/estimated-location.rst

@@ -0,0 +1,87 @@
+Estimated Location
+==================
+
+.. important::
+
+    The **Estimated Location** feature is **disabled by default**.
+
+    Before enabling it, the :doc:`WHOIS Lookup feature <whois>` must be
+    enabled. Then set
+    :ref:`OPENWISP_CONTROLLER_ESTIMATED_LOCATION_ENABLED` to ``True``
+
+.. contents:: **Table of contents**:
+    :depth: 1
+    :local:
+
+Overview
+--------
+
+This feature automatically creates or updates a device’s location based on
+latitude and longitude information retrieved from the WHOIS Lookup
+feature.
+
+Trigger Conditions
+------------------
+
+This feature is triggered when:
+
+- A **fresh WHOIS lookup** is performed for a device.
+- Or when a WHOIS record already exists for the device’s IP **and**:
+
+  - The device’s last IP address is **public**.
+  - WHOIS lookup and Estimated Location is **enabled** for the device’s
+    organization.
+
+Behavior
+--------
+
+The system will **attach the already existing matching location** of
+another device with same ip to the current device if:
+
+- Only one device is found with that IP and it has a location.
+- The current device **has no location** or that location is
+  **estimated**.
+
+If there are multiple devices with location for the same IP, the system
+will **not attach any location** to the current device and a notification
+will be sent suggesting the user to manually assign/create a location for
+the device.
+
+If there is **no matching location**, a new estimated location is created
+or the existing one is updated using coordinates from the WHOIS record,
+but only if the existing location is estimated.
+
+If two devices share the same IP address and are assigned to the same
+location, and the last IP of one of the devices is updated, the system
+will create a new estimated location for that device.
+
+Visibility of Estimated Status
+------------------------------
+
+The estimated status of a location is visible on the location page if the
+feature is enabled for the organization. The location admin page also
+includes indicators for the estimated status.
+
+- The name of the location will have suffix **(Estimated Location :
+  <ip_address>)**.
+- A warning on top of the page.
+- **Is Estimated** field.
+
+Changes to the ``coordinates`` and ``geometry`` of the estimated location
+will set the ``is_estimated`` field to ``False`` and remove the
+"(Estimated Location)" suffix with IP from the location name.
+
+In REST API, the field will be visible in the :ref:`Device Location
+<device_location_estimated>`, :ref:`Location list
+<location_list_estimated>`, :ref:`Location Detail
+<location_detail_estimated>` and :ref:`Location list (GeoJson)
+<location_geojson_estimated>` if the feature is **enabled**. The field can
+also be used for filtering in the location list (including geojson)
+endpoints and in the :ref:`Device List <device_list_estimated_filters>`.
+
+Managing Older Estimated Locations
+----------------------------------
+
+Whenever location related fields in WHOIS records are updated as per
+:ref:`Managing WHOIS Older Records <whois_older_records>`; the location
+will also be updated automatically.

docs/user/rest-api.rst

@@ -68,6 +68,22 @@ List Devices
 
     GET /api/v1/controller/device/
 
+.. _device_list_whois:
+
+**WHOIS Details**
+
+If :doc:`WHOIS Lookup feature <whois>` is enabled, each device in the list
+response will also include a ``whois_info`` field with related brief WHOIS
+information.
+
+.. _device_list_estimated_filters:
+
+**Estimated Location Filters**
+
+if :doc:`Estimated Location feature <estimated-location>` is enabled,
+devices can be filtered based on the estimated nature of their location
+using the ``geo_is_estimated``.
+
 **Available filters**
 
 You can filter a list of devices based on their configuration status using
@@ -145,6 +161,13 @@ Get Device Detail
 
     GET /api/v1/controller/device/{id}/
 
+.. _device_detail_whois:
+
+**WHOIS Details**
+
+If :doc:`WHOIS Lookup feature <whois>` is enabled, the response will also
+include a ``whois_info`` field with related detailed WHOIS information.
+
 Download Device Configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -529,13 +552,26 @@ of certificate's organization as show in the example below:
 
     GET /api/v1/controller/cert/{common_name}/group/?org={org1_slug},{org2_slug}
 
+.. |est_loc| replace:: Estimated Location feature
+
+.. _est_loc: estimated-location.html
+
+.. |estimated_details| replace:: If |est_loc|_ is enabled, the location
+    response will also include ``is_estimated`` status field.
+
 Get Device Location
 ~~~~~~~~~~~~~~~~~~~
 
 .. code-block:: text
 
     GET /api/v1/controller/device/{id}/location/
 
+.. _device_location_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
 .. _create_device_location:
 
 Create Device Location
@@ -772,6 +808,14 @@ List Locations
 
     GET /api/v1/controller/location/
 
+.. _location_list_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
+Locations can also be filtered using the ``is_estimated``.
+
 **Available filters**
 
 You can filter using ``organization_id`` or ``organization_slug`` to get
@@ -853,6 +897,12 @@ Get Location Details
 
     GET /api/v1/controller/location/{pk}/
 
+.. _location_detail_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
 Change Location Details
 ~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -895,6 +945,14 @@ List Locations with Devices Deployed (in GeoJSON Format)
 
     GET /api/v1/controller/location/geojson/
 
+.. _location_geojson_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
+Locations can also be filtered using the ``is_estimated``.
+
 **Available filters**
 
 You can filter using ``organization_id`` or ``organization_slug`` to get

docs/user/settings.rst

@@ -761,3 +761,84 @@ recoverable failures, improving the reliability of the system.
 For more information on these settings, you can refer to the `the celery
 documentation regarding automatic retries for known errors.
 <https://docs.celeryq.dev/en/stable/userguide/tasks.html#automatic-retry-for-known-exceptions>`_
+
+.. _openwisp_controller_whois_enabled:
+
+``OPENWISP_CONTROLLER_WHOIS_ENABLED``
+-------------------------------------
+
+============ =========
+**type**:    ``bool``
+**default**: ``False``
+============ =========
+
+Allows enabling the optional :doc:`WHOIS Lookup feature <whois>`.
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/whois-admin-setting.png
+    :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/whois-admin-setting.png
+    :alt: WHOIS admin setting
+
+After enabling this feature, you have to set
+:ref:`OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT
+<OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT>` and
+:ref:`OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY
+<OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY>`.
+
+.. warning::
+
+    If these three settings are not configured as expected, an
+    ``ImproperlyConfigured`` exception will be raised.
+
+.. _openwisp_controller_whois_geoip_account:
+
+``OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT``
+-------------------------------------------
+
+============ =======
+**type**:    ``str``
+**default**: None
+============ =======
+
+Maxmind Account ID required for the :doc:`WHOIS Lookup feature <whois>`.
+
+.. _openwisp_controller_whois_geoip_key:
+
+``OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY``
+---------------------------------------
+
+============ =======
+**type**:    ``str``
+**default**: None
+============ =======
+
+Maxmind License Key required for the :doc:`WHOIS Lookup feature <whois>`.
+
+.. _openwisp_controller_whois_estimated_location_enabled:
+
+``OPENWISP_CONTROLLER_WHOIS_ESTIMATED_LOCATION_ENABLED``
+--------------------------------------------------------
+
+============ =========
+**type**:    ``bool``
+**default**: ``False``
+============ =========
+
+Allows enabling the optional :doc:`Estimated Location feature
+<estimated-location>`.
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-location-setting.png
+    :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-location-setting.png
+    :alt: Estimated Location setting
+
+.. _openwisp_controller_whois_refresh_threshold_days:
+
+``OPENWISP_CONTROLLER_WHOIS_REFRESH_THRESHOLD_DAYS``
+----------------------------------------------------
+
+============ =======
+**type**:    ``int``
+**default**: ``14``
+============ =======
+
+Specifies the number of days after which the WHOIS information for a
+device is considered stale and eligible for refresh.