diff --git a/CHANGELOG.md b/CHANGELOG.md index 304fc378c..169ce55ce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ### :magic_wand: Added * Python 3.12 and 3.13 support ([PR #1052](https://github.com/aws/aws-advanced-python-wrapper/pull/1052)). -* [Simple Read/Write Splitting Plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) (`srw`). This plugin adds functionality to switch between endpoints via calls to the Connection#setReadOnly method. It does not rely on cluster topology. It relies purely on the provided endpoints and their DNS resolution ([PR #1048](https://github.com/aws/aws-advanced-python-wrapper/pull/1048)). +* [Simple Read/Write Splitting Plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) (`srw`). This plugin adds functionality to switch between endpoints via calls to the Connection#setReadOnly method. It does not rely on cluster topology. It relies purely on the provided endpoints and their DNS resolution ([PR #1048](https://github.com/aws/aws-advanced-python-wrapper/pull/1048)). * Wrapper resource cleanup method `aws_advanced_python_wrapper.release_resources()`. This method should be called at program exit to properly clean up background threads and resources ([PR #1066](https://github.com/aws/aws-advanced-python-wrapper/pull/1066)). ### :bug: Fixed @@ -28,7 +28,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ## [1.4.0] - 2025-10-17 ### :magic_wand: Added -* [EFM v2](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md#host-monitoring-plugin-v2), an improved alternate version of the `efm` plugin which addresses issues such as garbage collection and monitoring stability, is now live! +* [EFM v2](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md#host-monitoring-plugin-v2), an improved alternate version of the `efm` plugin which addresses issues such as garbage collection and monitoring stability, is now live! ### :bug: Fixed * Update subscribed methods to explicit methods ([PR #960](https://github.com/aws/aws-advanced-python-wrapper/pull/960)) @@ -41,7 +41,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ## [1.3.0] - 2025-07-28 ### :magic_wand: Added -* [Blue/Green Plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheBlueGreenPlugin.md), which adds support for blue/green deployments ([PR #911](https://github.com/aws/aws-advanced-python-wrapper/pull/911)). +* [Blue/Green Plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md), which adds support for blue/green deployments ([PR #911](https://github.com/aws/aws-advanced-python-wrapper/pull/911)). * Limitless Plugin, which adds support for limitless deployments ([PR #912](https://github.com/aws/aws-advanced-python-wrapper/pull/912)). * Add weighted random host selection strategy ([PR #907](https://github.com/aws/aws-advanced-python-wrapper/pull/907)). * Add expiration time for secrets cache in the Secrets Manager Plugin ([PR #906](https://github.com/aws/aws-advanced-python-wrapper/pull/906)). @@ -61,7 +61,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ## [1.2.0] - 2024-12-12 ### :magic_wand: Added -* [Custom endpoint plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheCustomEndpointPlugin.md), which adds support for RDS custom endpoints. +* [Custom endpoint plugin](https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheCustomEndpointPlugin.md), which adds support for RDS custom endpoints. ## [1.1.1] - 2024-10-18 ### :magic_wand: Added @@ -73,7 +73,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ## [1.1.0] - 2024-07-31 ### :magic_wand: Added -* Okta authentication support. See the [documentation](./docs/using-the-python-driver/using-plugins/UsingTheOktaAuthenticationPlugin.md) for more details and sample code. +* Okta authentication support. See the [documentation](docs/using-the-python-wrapper/using-plugins/UsingTheOktaAuthenticationPlugin.md) for more details and sample code. ## [1.0.0] - 2024-05-23 The Amazon Web Services (AWS) Advanced Python Wrapper allows an application to take advantage of the features of clustered Aurora databases. diff --git a/Maintenance.md b/Maintenance.md index f72f83033..7568c0e35 100644 --- a/Maintenance.md +++ b/Maintenance.md @@ -22,7 +22,7 @@ backlog of unreleased changes. In contrast, `aws-advanced-python-wrapper` releases new major versions only when there are a critical mass of breaking changes (e.g. changes that are incompatible with existing APIs). This tends to happen if we need to -change the way the driver is currently working. Fortunately, the JDBC API is fairly mature and has not changed, however +change the way the wrapper is currently working. Fortunately, the JDBC API is fairly mature and has not changed, however in the event that the API changes we will release a version to be compatible. Please note: Both the roadmap and the release dates reflect intentions rather than firm commitments and may change diff --git a/README.md b/README.md index 05a6b8213..fd0c32154 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Amazon Web Services (AWS) Advanced Python Driver +# Amazon Web Services (AWS) Advanced Python Wrapper [![Python Checks and Unit Tests](https://github.com/aws/aws-advanced-python-wrapper/actions/workflows/main.yml/badge.svg?branch=main)](https://github.com/aws/aws-advanced-python-wrapper/actions/workflows/main.yml) [![Integration Tests](https://github.com/aws/aws-advanced-python-wrapper/actions/workflows/integration_tests.yml/badge.svg)](https://github.com/aws/aws-advanced-python-wrapper/actions/workflows/integration_tests.yml) @@ -6,35 +6,35 @@ ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aws-advanced-python-wrapper) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) -## About the Driver +## About the Wrapper Hosting a database cluster in the cloud via Aurora is able to provide users with sets of features and configurations to obtain maximum performance and availability, such as database failover. However, at the moment, most existing drivers do not currently support those functionalities or are not able to entirely take advantage of it. -The main idea behind the AWS Advanced Python Driver is to add a software layer on top of an existing Python driver that would enable all the enhancements brought by Aurora, without requiring users to change their workflow with their databases and existing Python drivers. +The main idea behind the AWS Advanced Python Wrapper is to add a software layer on top of an existing Python driver that would enable all the enhancements brought by Aurora, without requiring users to change their workflow with their databases and existing Python drivers. ### What is Failover? -In an Amazon Aurora database cluster, **failover** is a mechanism by which Aurora automatically repairs the cluster status when a primary DB instance becomes unavailable. It achieves this goal by electing an Aurora Replica to become the new primary DB instance, so that the DB cluster can provide maximum availability to a primary read-write DB instance. The AWS Advanced Python Driver is designed to understand the situation and coordinate with the cluster in order to provide minimal downtime and allow connections to be very quickly restored in the event of a DB instance failure. +In an Amazon Aurora database cluster, **failover** is a mechanism by which Aurora automatically repairs the cluster status when a primary DB instance becomes unavailable. It achieves this goal by electing an Aurora Replica to become the new primary DB instance, so that the DB cluster can provide maximum availability to a primary read-write DB instance. The AWS Advanced Python Wrapper is designed to understand the situation and coordinate with the cluster in order to provide minimal downtime and allow connections to be very quickly restored in the event of a DB instance failure. -### Benefits of the AWS Advanced Python Driver +### Benefits of the AWS Advanced Python Wrapper -Although Aurora is able to provide maximum availability through the use of failover, existing client drivers do not currently support this functionality. This is partially due to the time required for the DNS of the new primary DB instance to be fully resolved in order to properly direct the connection. The AWS Advanced Python Driver allows customers to continue using their existing community drivers in addition to having the AWS Advanced Python Driver fully exploit failover behavior by maintaining a cache of the Aurora cluster topology and each DB instance's role (Aurora Replica or primary DB instance). This topology is provided via a direct query to the Aurora DB, essentially providing a shortcut to bypass the delays caused by DNS resolution. With this knowledge, the AWS Advanced Python Driver can more closely monitor the Aurora DB cluster status so that a connection to the new primary DB instance can be established as fast as possible. +Although Aurora is able to provide maximum availability through the use of failover, existing client drivers do not currently support this functionality. This is partially due to the time required for the DNS of the new primary DB instance to be fully resolved in order to properly direct the connection. The AWS Advanced Python Wrapper allows customers to continue using their existing community drivers in addition to having the AWS Advanced Python Wrapper fully exploit failover behavior by maintaining a cache of the Aurora cluster topology and each DB instance's role (Aurora Replica or primary DB instance). This topology is provided via a direct query to the Aurora DB, essentially providing a shortcut to bypass the delays caused by DNS resolution. With this knowledge, the AWS Advanced Python Wrapper can more closely monitor the Aurora DB cluster status so that a connection to the new primary DB instance can be established as fast as possible. ### Enhanced Failure Monitoring -Since a database failover is usually identified by reaching a network or a connection timeout, the AWS Advanced Python Driver introduces an enhanced and customizable manner to faster identify a database outage. +Since a database failover is usually identified by reaching a network or a connection timeout, the AWS Advanced Python Wrapper introduces an enhanced and customizable manner to faster identify a database outage. -Enhanced Failure Monitoring (EFM) is a feature available from the [Host Monitoring Connection Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md#enhanced-failure-monitoring) that periodically checks the connected database host's health and availability. If a database host is determined to be unhealthy, the connection is aborted (and potentially routed to another healthy host in the cluster). +Enhanced Failure Monitoring (EFM) is a feature available from the [Host Monitoring Connection Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md#enhanced-failure-monitoring) that periodically checks the connected database host's health and availability. If a database host is determined to be unhealthy, the connection is aborted (and potentially routed to another healthy host in the cluster). -### Using the AWS Advanced Python Driver with plain RDS databases +### Using the AWS Advanced Python Wrapper with plain RDS databases -The AWS Advanced Python Driver also works with RDS provided databases that are not Aurora. +The AWS Advanced Python Wrapper also works with RDS provided databases that are not Aurora. -Please visit [this page](./docs/using-the-python-driver/UsingThePythonDriver.md#using-the-aws-advanced-python-driver-with-plain-rds-databases) for more information. +Please visit [this page](docs/using-the-python-wrapper/UsingThePythonWrapper.md#using-the-aws-advanced-python-wrapper-with-plain-rds-databases) for more information. ## Getting Started -To start using the driver with Psycopg, you need to pass Psycopg's connect function to the `AwsWrapperConnection#connect` method as shown in the following example: +To start using the wrapper with Psycopg, you need to pass Psycopg's connect function to the `AwsWrapperConnection#connect` method as shown in the following example: ```python from aws_advanced_python_wrapper import AwsWrapperConnection @@ -56,7 +56,7 @@ The `AwsWrapperConnection#connect` method accepts the connection configuration t You can either pass the connection configuration entirely through the connection string, entirely though the keyword arguments, or through a mixture of both. -To use the driver with MySQL Connector/Python, see the following example: +To use the wrapper with MySQL Connector/Python, see the following example: ```python from aws_advanced_python_wrapper import AwsWrapperConnection @@ -75,95 +75,95 @@ with AwsWrapperConnection.connect( print(row) ``` -For more details on how to download the AWS Advanced Python Driver, minimum requirements to use it, +For more details on how to download the AWS Advanced Python Wrapper, minimum requirements to use it, and how to integrate it within your project and with your Python driver of choice, please visit the [Getting Started page](./docs/GettingStarted.md). ### Connection Properties The following table lists the connection properties used with the AWS Advanced Python Wrapper. -| Parameter | Documentation Link | -|----------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------:| -| `auxiliary_query_timeout_sec` | [Driver Parameters](./docs/using-the-python-driver/UsingThePythonDriver.md#aws-advanced-python-driver-parameters) | -| `topology_refresh_ms` | [Driver Parameters](./docs/using-the-python-driver/UsingThePythonDriver.md#aws-advanced-python-driver-parameters) | -| `cluster_id` | [Driver Parameters](./docs/using-the-python-driver/UsingThePythonDriver.md#aws-advanced-python-driver-parameters) | -| `cluster_instance_host_pattern` | [Driver Parameters](./docs/using-the-python-driver/UsingThePythonDriver.md#aws-advanced-python-driver-parameters) | -| `wrapper_dialect` | [Dialects](./docs/using-the-python-driver/DatabaseDialects.md), and whether you should include it. | -| `wrapper_driver_dialect` | [Driver Dialect](./docs/using-the-python-driver/DriverDialects.md), and whether you should include it. | -| `plugins` | [Connection Plugin Manager](./docs/using-the-python-driver/UsingThePythonDriver.md#connection-plugin-manager-parameters) | -| `auto_sort_wrapper_plugin_order` | [Connection Plugin Manager](./docs/using-the-python-driver/UsingThePythonDriver.md#connection-plugin-manager-parameters) | -| `profile_name` | [Connection Plugin Manager](./docs/using-the-python-driver/UsingThePythonDriver.md#connection-plugin-manager-parameters) | -| `connect_timeout` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `socket_timeout` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `tcp_keepalive` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `tcp_keepalive_time` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `tcp_keepalive_interval` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `tcp_keepalive_probes` | [Network Timeouts](./docs/using-the-python-driver/UsingThePythonDriver.md#network-timeouts) | -| `enable_failover` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failover_mode` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `cluster_instance_host_pattern` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failover_cluster_topology_refresh_rate_sec` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failover_reader_connect_timeout_sec` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failover_timeout_sec` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failover_writer_reconnect_interval_sec` | [Failover Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) | -| `failure_detection_count` | [Host Monitoring Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) | -| `failure_detection_enabled` | [Host Monitoring Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) | -| `failure_detection_interval_ms` | [Host Monitoring Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) | -| `failure_detection_time_ms` | [Host Monitoring Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) | -| `monitor_disposal_time_ms` | [Host Monitoring Plugin](./docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) | -| `iam_default_port` | [IAM Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) | -| `iam_host` | [IAM Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) | -| `iam_region` | [IAM Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) | -| `iam_expiration` | [IAM Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) | -| `secrets_manager_secret_id` | [Secrets Manager Plugin](./docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | -| `secrets_manager_region` | [Secrets Manager Plugin](./docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | -| `secrets_manager_endpoint` | [Secrets Manager Plugin](./docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | -| `secrets_manager_secret_username` | [Secrets Manager Plugin](./docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | -| `secrets_manager_secret_password` | [Secrets Manager Plugin](./docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | -| `reader_host_selector_strategy` | [Connection Strategy](./docs/using-the-python-driver/using-plugins/UsingTheReadWriteSplittingPlugin.md#connection-strategies) | -| `db_user` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `idp_username` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `idp_password` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `idp_endpoint` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_role_arn` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_idp_arn` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_region` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `idp_name` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `idp_port` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `rp_identifier` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_host` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_default_port` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `iam_token_expiration` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `http_request_connect_timeout` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | -| `ssl_secure` | [Federated Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | - -### Using the AWS Advanced Python Driver - -Technical documentation regarding the functionality of the AWS Advanced Python Driver will be maintained in this GitHub repository. Since the AWS Advanced Python Driver requires an underlying Python driver, please refer to the individual driver's documentation for driver-specific information. -To find all the documentation and concrete examples on how to use the AWS Advanced Python Driver, please refer to the [AWS Advanced Python Driver Documentation](./docs/README.md) page. +| Parameter | Documentation Link | +|----------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------:| +| `auxiliary_query_timeout_sec` | [Driver Parameters](docs/using-the-python-wrapper/UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) | +| `topology_refresh_ms` | [Driver Parameters](docs/using-the-python-wrapper/UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) | +| `cluster_id` | [Driver Parameters](docs/using-the-python-wrapper/UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) | +| `cluster_instance_host_pattern` | [Driver Parameters](docs/using-the-python-wrapper/UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) | +| `wrapper_dialect` | [Dialects](docs/using-the-python-wrapper/DatabaseDialects.md), and whether you should include it. | +| `wrapper_driver_dialect` | [Driver Dialect](./docs/using-the-python-wrapper/DriverDialects.md), and whether you should include it. | +| `plugins` | [Connection Plugin Manager](docs/using-the-python-wrapper/UsingThePythonWrapper.md#connection-plugin-manager-parameters) | +| `auto_sort_wrapper_plugin_order` | [Connection Plugin Manager](docs/using-the-python-wrapper/UsingThePythonWrapper.md#connection-plugin-manager-parameters) | +| `profile_name` | [Connection Plugin Manager](docs/using-the-python-wrapper/UsingThePythonWrapper.md#connection-plugin-manager-parameters) | +| `connect_timeout` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `socket_timeout` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `tcp_keepalive` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `tcp_keepalive_time` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `tcp_keepalive_interval` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `tcp_keepalive_probes` | [Network Timeouts](docs/using-the-python-wrapper/UsingThePythonWrapper.md#network-timeouts) | +| `enable_failover` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failover_mode` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `cluster_instance_host_pattern` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failover_cluster_topology_refresh_rate_sec` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failover_reader_connect_timeout_sec` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failover_timeout_sec` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failover_writer_reconnect_interval_sec` | [Failover Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) | +| `failure_detection_count` | [Host Monitoring Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) | +| `failure_detection_enabled` | [Host Monitoring Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) | +| `failure_detection_interval_ms` | [Host Monitoring Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) | +| `failure_detection_time_ms` | [Host Monitoring Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) | +| `monitor_disposal_time_ms` | [Host Monitoring Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) | +| `iam_default_port` | [IAM Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) | +| `iam_host` | [IAM Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) | +| `iam_region` | [IAM Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) | +| `iam_expiration` | [IAM Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) | +| `secrets_manager_secret_id` | [Secrets Manager Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | +| `secrets_manager_region` | [Secrets Manager Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | +| `secrets_manager_endpoint` | [Secrets Manager Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | +| `secrets_manager_secret_username` | [Secrets Manager Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | +| `secrets_manager_secret_password` | [Secrets Manager Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) | +| `reader_host_selector_strategy` | [Connection Strategy](docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md#connection-strategies) | +| `db_user` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `idp_username` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `idp_password` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `idp_endpoint` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_role_arn` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_idp_arn` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_region` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `idp_name` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `idp_port` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `rp_identifier` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_host` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_default_port` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `iam_token_expiration` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `http_request_connect_timeout` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | +| `ssl_secure` | [Federated Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) | + +### Using the AWS Advanced Python Wrapper + +Technical documentation regarding the functionality of the AWS Advanced Python Wrapper will be maintained in this GitHub repository. Since the AWS Advanced Python Wrapper requires an underlying Python driver, please refer to the individual driver's documentation for driver-specific information. +To find all the documentation and concrete examples on how to use the AWS Advanced Python Wrapper, please refer to the [AWS Advanced Python Wrapper Documentation](./docs/README.md) page. ### Known Limitations #### Amazon RDS Blue/Green Deployments -Support for Blue/Green deployments using the AWS Advanced Python Driver requires specific metadata tables. The following service versions provide support for Blue/Green Deployments: +Support for Blue/Green deployments using the AWS Advanced Python Wrapper requires specific metadata tables. The following service versions provide support for Blue/Green Deployments: - Supported RDS PostgreSQL Versions: `rds_tools v1.7 (17.1, 16.5, 15.9, 14.14, 13.17, 12.21)` and above. - Supported Aurora PostgreSQL Versions: Engine Release `17.5, 16.9, 15.13, 14.18, 13.21` and above. - Supported Aurora MySQL Versions: Engine Release `3.07` and above. -Please note that Aurora Global Database and RDS Multi-AZ clusters with Blue/Green deployments is currently not supported. For detailed information on supported database versions, refer to the [Blue/Green Deployment Plugin Documentation](./docs/using-the-python-driver/using-plugins/UsingTheBlueGreenPlugin.md). +Please note that Aurora Global Database and RDS Multi-AZ clusters with Blue/Green deployments is currently not supported. For detailed information on supported database versions, refer to the [Blue/Green Deployment Plugin Documentation](docs/using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md). #### MySQL Connector/Python C Extension When connecting to Aurora MySQL clusters, it is recommended to use the Python implementation of the MySQL Connector/Python driver by setting the `use_pure` connection argument to `True`. -The AWS Advanced Python Driver internally calls the MySQL Connector/Python's `is_connected` method to verify the connection. The [MySQL Connector/Python's C extension](https://dev.mysql.com/doc/connector-python/en/connector-python-cext.html) uses a network blocking implementation of the `is_connected` method. +The AWS Advanced Python Wrapper internally calls the MySQL Connector/Python's `is_connected` method to verify the connection. The [MySQL Connector/Python's C extension](https://dev.mysql.com/doc/connector-python/en/connector-python-cext.html) uses a network blocking implementation of the `is_connected` method. In the event of a network failure where the host can no longer be reached, the `is_connected` call may hang indefinitely and will require users to forcibly interrupt the application. #### MySQL Support for the IAM Authentication Plugin The official MySQL Connector/Python [offers a Python implementation and a C implementation](https://dev.mysql.com/doc/connector-python/en/connector-python-example-connecting.html#:~:text=Using%20the%20Connector/Python%20Python%20or%20C%20Extension) of the driver that can be toggled using the `use_pure` connection argument. -The [IAM Authentication Plugin](./docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) is incompatible with the Python implementation of the driver due to its 255-character password limit. +The [IAM Authentication Plugin](docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) is incompatible with the Python implementation of the driver due to its 255-character password limit. The IAM Authentication Plugin generates a temporary AWS IAM token to authenticate users. Passing this token to the Python implementation of the driver will result in error messages similar to the following: ``` Error occurred while opening a connection: int1store requires 0 <= i <= 255 @@ -178,23 +178,23 @@ Unfortunately, due to conflicting limitations, you will need to decide if using #### Toxiproxy network failure behavior -In the AWS Advanced Python Driver test suite, we have two methods of simulating database/network failure: +In the AWS Advanced Python Wrapper test suite, we have two methods of simulating database/network failure: - method 1: initiate failover using the AWS RDS SDK. - method 2: use a test dependency called Toxiproxy. -Toxiproxy creates proxy network containers that sit between the test and the actual database. The test code connects to the proxy containers instead of the database. Toxiproxy can then be used to disable network activity between the driver and the database via the proxy. This network failure simulation is stricter than scenario 1. It does not allow any communication between the driver and the database, and does not give the database a chance to break off any connections. +Toxiproxy creates proxy network containers that sit between the test and the actual database. The test code connects to the proxy containers instead of the database. Toxiproxy can then be used to disable network activity between the wrapper and the database via the proxy. This network failure simulation is stricter than scenario 1. It does not allow any communication between the wrapper and the database, and does not give the database a chance to break off any connections. We have observed different behavior when testing with method 1 vs method 2. With method 1, the server failover is detected without unexpected side effects. With method 2, we have observed some side effects. We have only observed these side effects when using Toxiproxy, and have never observed them in a real-world scenario. Psycopg and MySQL Connector/Python do not provide client-side query timeouts. When querying a host that has been disabled via Toxiproxy, the query will hang indefinitely until the host is re-enabled. Whether this behavior would ever occur in a real world scenario is uncertain. We reached out to the Psycopg team, who indicated they have not seen this issue (see full discussion [here](https://github.com/psycopg/psycopg/discussions/676)). They also believe that Toxiproxy tests for stricter conditions than would occur in a real-world scenario. However, we will list the side effects we have noticed during testing due to this behavior: -1. The EFM plugin and `socket_timeout` connection parameter use helper threads to execute queries used to detect if the network is working properly. These helper threads are executed with a timeout so that failure can be detected if the network is down. If the host was disabled using Toxiproxy, although the timeout will return control to the main thread, the helper thread will be stuck in a loop waiting on results from the server. It will be stuck until the host is re-enabled with Toxiproxy. There is no mechanism to cancel a running thread from another thread in Python, so the thread will consume resources until the host is re-enabled. Using an Event to signal the thread to stop is not an option, as the loop occurs inside the underlying driver code. Note that although the helper thread will be stuck, control will still be returned to the main thread so that the driver is still usable. +1. The EFM plugin and `socket_timeout` connection parameter use helper threads to execute queries used to detect if the network is working properly. These helper threads are executed with a timeout so that failure can be detected if the network is down. If the host was disabled using Toxiproxy, although the timeout will return control to the main thread, the helper thread will be stuck in a loop waiting on results from the server. It will be stuck until the host is re-enabled with Toxiproxy. There is no mechanism to cancel a running thread from another thread in Python, so the thread will consume resources until the host is re-enabled. Using an Event to signal the thread to stop is not an option, as the loop occurs inside the underlying driver code. Note that although the helper thread will be stuck, control will still be returned to the main thread so that the wrapper is still usable. 2. As a consequence of side effect 1, if a query is executed against a host that has been disabled with Toxiproxy, the Python program will not exit if the host is not re-enabled. The EFM helper threads mentioned in side effect 1 are run using a ThreadPoolExecutor. Although the ThreadPoolExecutor implementation uses daemon threads, it also joins all threads at Python exit. Because the helper thread is stuck in this scenario, the Python application will hang waiting the thread to join. This behavior has only been observed when using the MySQL Connector/Python driver. 3. As a consequence of side effect 1, if a query is executed against a host that has been disabled with Toxiproxy, and the host is still disabled when the Python program exits, a segfault may occur. This occurs because the helper thread is stuck in a loop attempting to read a connection pointer. When the program is exiting, the pointer is destroyed. The helper thread may try to read from the pointer after it is destroyed, leading to a segfault. This behavior has only been observed when using the Psycopg driver. ## Getting Help and Opening Issues -If you encounter a bug with the AWS Advanced Python Driver, we would like to hear about it. +If you encounter a bug with the AWS Advanced Python Wrapper, we would like to hear about it. Please search the [existing issues](https://github.com/aws/aws-advanced-python-wrapper/issues) to see if others are also experiencing the issue before reporting the problem in a new issue. GitHub issues are intended for bug reports and feature requests. When opening a new issue, please fill in all required fields in the issue template to help expedite the investigation process. diff --git a/docs/GettingStarted.md b/docs/GettingStarted.md index a8ce390b4..6310343e4 100644 --- a/docs/GettingStarted.md +++ b/docs/GettingStarted.md @@ -2,37 +2,37 @@ ## Minimum Requirements -Before using the AWS Advanced Python Driver, you must install: +Before using the AWS Advanced Python Wrapper, you must install: - Python 3.10 - 3.13 (inclusive). -- The AWS Advanced Python Driver. +- The AWS Advanced Python Wrapper. - Your choice of underlying Python driver. - To use the wrapper with Aurora with PostgreSQL compatibility, install [Psycopg](https://github.com/psycopg/psycopg). - To use the wrapper with Aurora with MySQL compatibility, install [MySQL Connector/Python](https://github.com/mysql/mysql-connector-python). > [!NOTE]\ -> The driver has been verified on Psycopg 3.1.12+ and MySQL Connector/Python 8.1.0+. Compatibility with prior versions have not been tested. +> The wrapper has been verified on Psycopg 3.1.12+ and MySQL Connector/Python 8.1.0+. Compatibility with prior versions have not been tested. -## Obtaining the AWS Advanced Python Driver +## Obtaining the AWS Advanced Python Wrapper -You can install the AWS Advanced Python Driver and the underlying Python drivers via [pip](https://pip.pypa.io/en/stable/). +You can install the AWS Advanced Python Wrapper and the underlying Python drivers via [pip](https://pip.pypa.io/en/stable/). The order of installation does not matter. -To use the AWS Advanced Python Driver with Psycopg for Aurora PostgreSQL, run: +To use the AWS Advanced Python Wrapper with Psycopg for Aurora PostgreSQL, run: ```shell pip install aws-advanced-python-wrapper pip install psycopg ``` -To use the AWS Advanced Python Driver with MySQL Connector/Python for Aurora MySQL, run: +To use the AWS Advanced Python Wrapper with MySQL Connector/Python for Aurora MySQL, run: ```shell pip install aws-advanced-python-wrapper pip install mysql-connector-python ``` -## Using the AWS Advanced Python Driver +## Using the AWS Advanced Python Wrapper -To start using the driver with Psycopg, you need to pass Psycopg's connect function to the `AwsWrapperConnection#connect` method as shown in the following example: +To start using the wrapper with Psycopg, you need to pass Psycopg's connect function to the `AwsWrapperConnection#connect` method as shown in the following example: ```python from aws_advanced_python_wrapper import AwsWrapperConnection @@ -46,7 +46,7 @@ awsconn = AwsWrapperConnection.connect( ) ``` -Similarly, to start using the driver with MySQL Connector/Python, you need to pass the connect function to the `AwsWrapperConnection#connect` method as shown in the following example: +Similarly, to start using the wrapper with MySQL Connector/Python, you need to pass the connect function to the `AwsWrapperConnection#connect` method as shown in the following example: ```python awsconn = AwsWrapperConnection.connect( mysql.connector.Connect, @@ -85,11 +85,11 @@ awsconn = AwsWrapperConnection.connect( > **NOTE**: If the same configuration is specified in both the connection string and the keyword arguments, the keyword argument takes precedence. -The AWS Advanced Python Driver implements the [PEP 249 Database API](https://peps.python.org/pep-0249/). +The AWS Advanced Python Wrapper implements the [PEP 249 Database API](https://peps.python.org/pep-0249/). After establishing a connection, you can use it in the same pattern as you would with the community Python drivers. -However, the driver introduces some custom errors for the Failover Plugin and the Read/Write Splitting Plugin that need to be explicitly handled. +However, the wrapper introduces some custom errors for the Failover Plugin and the Read/Write Splitting Plugin that need to be explicitly handled. -For instance, after a successful failover, some session states may not be transferred to the new connection, so the driver throws a `FailoverSuccessError` to notify the application that the connection may need to be reconfigured, or to create a new cursor object. +For instance, after a successful failover, some session states may not be transferred to the new connection, so the wrapper throws a `FailoverSuccessError` to notify the application that the connection may need to be reconfigured, or to create a new cursor object. See this simple PostgreSQL example: ```python @@ -106,7 +106,7 @@ with AwsWrapperConnection.connect( cursor.execute(sql) except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. # The old cursor is no longer reusable and the application needs to reconfigure sessions states. reconfigure_session_states(awsconn) @@ -114,8 +114,8 @@ with AwsWrapperConnection.connect( with awsconn.cursor() as cursor: cursor.execute(sql) ``` -A full PostgreSQL example is available at [PGFailover.py](./examples/PGFailover.py) and [MySQLFailover.py](./examples/MySQLFailover.py) +A full PostgreSQL example is available at [PGFailover.py](./examples/PGFailover.py). A full MySQL example, [MySQLFailover.py](./examples/MySQLFailover.py), is available as well. -You can learn more about the AWS Advanced Python Driver specific errors in the [Using the Failover Plugin](./using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#Failover-Errors) page. +You can learn more about the AWS Advanced Python Wrapper specific errors in the [Using the Failover Plugin](using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#Failover-Errors) page. -For more detailed information about how to use and configure the AWS Advanced Python Driver, please visit [this page](./using-the-python-driver/UsingThePythonDriver.md). +For more detailed information about how to use and configure the AWS Advanced Python Wrapper, please visit [this page](using-the-python-wrapper/UsingThePythonWrapper.md). diff --git a/docs/README.md b/docs/README.md index 24f9eeec5..2cfbd23bb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,29 +1,30 @@ # Documentation - [Getting Started](./GettingStarted.md) -- [Using the AWS Python Driver](./using-the-python-driver/UsingThePythonDriver.md) - - [Logging](./using-the-python-driver/UsingThePythonDriver.md#logging) - - [Python Wrapper Parameters](./using-the-python-driver/UsingThePythonDriver.md#aws-advanced-python-driver-parameters) - - [Support For RDS Multi-AZ Database Cluster](./using-the-python-driver/SupportForRDSMultiAzDBCluster.md) - - [Session State](./using-the-python-driver/SessionState.md) - - [Database Dialects](./using-the-python-driver/DatabaseDialects.md) - - [Driver Dialects](./using-the-python-driver/DriverDialects.md) - - [Telemetry](./using-the-python-driver/Telemetry.md) - - [Plugins](./using-the-python-driver/UsingThePythonDriver.md#plugins) - - [Failover Plugin](./using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) - - [Failover Configuration Guide](./using-the-python-driver/FailoverConfigurationGuide.md) - - [Host Monitoring Plugin](./using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) - - [Aurora Connection Tracker Plugin](./using-the-python-driver/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) - - [Aurora Initial Connection Strategy Plugin](./using-the-python-driver/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) - - [Host Availability Strategy](./using-the-python-driver/HostAvailabilityStrategy.md) - - [IAM Authentication Plugin](./using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md) - - [AWS Secrets Manager Plugin](./using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md) - - [Federated Authentication Plugin](./using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md) - - [Read Write Splitting Plugin](./using-the-python-driver/using-plugins/UsingTheReadWriteSplittingPlugin.md) - - [Reader Selection Strategies](./using-the-python-driver/ReaderSelectionStrategies.md) - - [Simple Read Write Splitting Plugin](./using-the-python-driver/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) - - [Fastest Response Strategy Plugin](./using-the-python-driver/using-plugins/UsingTheFastestResponseStrategyPlugin.md) - - [Host Availability Strategy](./using-the-python-driver/HostAvailabilityStrategy.md) +- [Using the AWS Python Wrapper](using-the-python-wrapper/UsingThePythonWrapper.md) + - [Logging](using-the-python-wrapper/UsingThePythonWrapper.md#logging) + - [Python Wrapper Parameters](using-the-python-wrapper/UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) + - [Support For RDS Multi-AZ Database Cluster](using-the-python-wrapper/SupportForRDSMultiAzDBCluster.md) + - [Session State](using-the-python-wrapper/SessionState.md) + - [Database Dialects](using-the-python-wrapper/DatabaseDialects.md) + - [Driver Dialects](using-the-python-wrapper/DriverDialects.md) + - [Telemetry](using-the-python-wrapper/Telemetry.md) + - [Plugins](using-the-python-wrapper/UsingThePythonWrapper.md#plugins) + - [Failover Plugin](using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) + - [Failover v2 Plugin](using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md) + - [Failover Configuration Guide](using-the-python-wrapper/FailoverConfigurationGuide.md) + - [Host Monitoring Plugin](using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) + - [Aurora Connection Tracker Plugin](using-the-python-wrapper/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) + - [Aurora Initial Connection Strategy Plugin](using-the-python-wrapper/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) + - [Host Availability Strategy](using-the-python-wrapper/HostAvailabilityStrategy.md) + - [IAM Authentication Plugin](using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) + - [AWS Secrets Manager Plugin](using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) + - [Federated Authentication Plugin](using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) + - [Read Write Splitting Plugin](using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md) + - [Reader Selection Strategies](using-the-python-wrapper/ReaderSelectionStrategies.md) + - [Simple Read Write Splitting Plugin](using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) + - [Fastest Response Strategy Plugin](using-the-python-wrapper/using-plugins/UsingTheFastestResponseStrategyPlugin.md) + - [Host Availability Strategy](using-the-python-wrapper/HostAvailabilityStrategy.md) - [Development Guide](./development-guide/DevelopmentGuide.md) - [Setup](./development-guide/DevelopmentGuide.md#setup) - [Testing Overview](./development-guide/DevelopmentGuide.md#testing-overview) diff --git a/docs/development-guide/LoadablePlugins.md b/docs/development-guide/LoadablePlugins.md index 51ebbe542..7ea35af0a 100644 --- a/docs/development-guide/LoadablePlugins.md +++ b/docs/development-guide/LoadablePlugins.md @@ -9,7 +9,7 @@ Plugins let users: - measure execution time - and more -The AWS Advanced Python Driver has several built-in plugins; you can [see the list here](../../docs/using-the-python-driver/UsingThePythonDriver.md#list-of-available-plugins). +The AWS Advanced Python Driver has several built-in plugins; you can [see the list here](../using-the-python-wrapper/UsingThePythonWrapper.md#list-of-available-plugins). ## Available Services @@ -68,7 +68,7 @@ To register a custom plugin, follow these steps: - Import and call `PluginManager.register_plugin(plugin_code: str, plugin_factory: Type[PluginFactory], weight: int = WEIGHT_RELATIVE_TO_PRIOR_PLUGIN)` with the appropriate arguments: - The first argument specifies a short name for the plugin that will be used when specifying the `plugins` connection parameter. The name should not contain spaces. In the example below, we will use `custom_plugin`. - The second argument should be the `PluginFactory` class you created for the custom plugin. Note that the class itself should be passed rather than an instance of the class. - - The third (optional) argument specifies a weight for the custom plugin. The weight will determine the plugin's ordering in the plugin chain if the `auto_sort_wrapper_plugin_order` property is enabled. All plugins with unspecified weight will be ordered according to the `plugins` parameter setting. More information on this property can be found [here](../../docs/using-the-python-driver/UsingThePythonDriver.md#connection-plugin-manager-parameters). + - The third (optional) argument specifies a weight for the custom plugin. The weight will determine the plugin's ordering in the plugin chain if the `auto_sort_wrapper_plugin_order` property is enabled. All plugins with unspecified weight will be ordered according to the `plugins` parameter setting. More information on this property can be found [here](../using-the-python-wrapper/UsingThePythonWrapper.md#connection-plugin-manager-parameters). - When creating a connection, in the `plugins` parameter, include the plugin name that you specified as the first argument to `register_plugin`. This will ensure that your plugin is included in the plugin chain. ### Example diff --git a/docs/development-guide/Pipelines.md b/docs/development-guide/Pipelines.md index 84a4714df..cc2fd17d5 100644 --- a/docs/development-guide/Pipelines.md +++ b/docs/development-guide/Pipelines.md @@ -22,7 +22,7 @@ The connect pipeline performs any additional setup or post connection steps requ The most common usage of the connect pipeline is to fetch extra credentials from external locations. -An example would be the [IAM connection plugin](../using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md). The IAM connection plugin generates an IAM authentication token to be used when establishing a connection. Since authentication is only required when establishing a connection and not required for any subsequent execution, the IAM authentication plugin only needs to implement the connect pipeline. +An example would be the [IAM connection plugin](../using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md). The IAM connection plugin generates an IAM authentication token to be used when establishing a connection. Since authentication is only required when establishing a connection and not required for any subsequent execution, the IAM authentication plugin only needs to implement the connect pipeline. ## Force Connect Pipeline @@ -60,7 +60,7 @@ For cases where keeping updated information on existing and available readers is such as during the failover procedure, it is important to have a host list provider that can re-fetch information once in a while, like the RDS host list provider. The RDS host list provider should be automatically used by the AWS Advanced Python Driver when the application is connecting to Aurora databases. -However, you can ensure that the provider is used by specifying a topology-aware dialect. For more information, see [Database Dialects](../using-the-python-driver/DatabaseDialects.md). +However, you can ensure that the provider is used by specifying a topology-aware dialect. For more information, see [Database Dialects](../using-the-python-wrapper/DatabaseDialects.md). ## Connection Changed Notification Pipeline diff --git a/docs/development-guide/PluginManager.md b/docs/development-guide/PluginManager.md index e005445a9..5fee4ec0e 100644 --- a/docs/development-guide/PluginManager.md +++ b/docs/development-guide/PluginManager.md @@ -6,7 +6,7 @@ The connection plugin manager has the following main functionalities: - Initiate pipelines ## Load and Initialize Plugins -The connection plugin manager initializes all plugins registered via the [`plugins`](../using-the-python-driver/UsingThePythonDriver.md#connection-plugin-manager-parameters) configuration parameter. +The connection plugin manager initializes all plugins registered via the [`plugins`](../using-the-python-wrapper/UsingThePythonWrapper.md#connection-plugin-manager-parameters) configuration parameter. ## Initiate Pipelines
diagram for the plugin service design
diff --git a/docs/examples/MySQLFailover.py b/docs/examples/MySQLFailover.py index d007567c2..ef433aaa2 100644 --- a/docs/examples/MySQLFailover.py +++ b/docs/examples/MySQLFailover.py @@ -39,8 +39,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -51,12 +51,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/examples/MySQLReadWriteSplitting.py b/docs/examples/MySQLReadWriteSplitting.py index a102d68da..52fbf7738 100644 --- a/docs/examples/MySQLReadWriteSplitting.py +++ b/docs/examples/MySQLReadWriteSplitting.py @@ -57,8 +57,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -69,12 +69,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/examples/MySQLSimpleReadWriteSplitting.py b/docs/examples/MySQLSimpleReadWriteSplitting.py index 170796648..20042a7a5 100644 --- a/docs/examples/MySQLSimpleReadWriteSplitting.py +++ b/docs/examples/MySQLSimpleReadWriteSplitting.py @@ -39,8 +39,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -51,12 +51,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/examples/PGFailover.py b/docs/examples/PGFailover.py index c15964fbe..82549d08d 100644 --- a/docs/examples/PGFailover.py +++ b/docs/examples/PGFailover.py @@ -39,8 +39,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -51,12 +51,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/examples/PGReadWriteSplitting.py b/docs/examples/PGReadWriteSplitting.py index 1e1b26a9e..2c81f5d51 100644 --- a/docs/examples/PGReadWriteSplitting.py +++ b/docs/examples/PGReadWriteSplitting.py @@ -57,8 +57,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -69,12 +69,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/examples/PGSimpleReadWriteSplitting.py b/docs/examples/PGSimpleReadWriteSplitting.py index 944cdee06..810c4af19 100644 --- a/docs/examples/PGSimpleReadWriteSplitting.py +++ b/docs/examples/PGSimpleReadWriteSplitting.py @@ -39,8 +39,8 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O return cursor except FailoverSuccessError: - # Query execution failed and AWS Advanced Python Driver successfully failed over to an available instance. - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror + # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror # The old cursor is no longer reusable and the application needs to reconfigure sessions states. configure_initial_session_states(conn) @@ -51,12 +51,12 @@ def execute_queries_with_failover_handling(conn: Connection, sql: str, params: O except FailoverFailedError as e: # User application should open a new connection, check the results of the failed transaction and re-run it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror raise e except TransactionResolutionUnknownError as e: # User application should check the status of the failed transaction and restart it if needed. See: - # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror + # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror raise e diff --git a/docs/images/failover_v1.jpg b/docs/images/failover_v1.jpg new file mode 100644 index 000000000..1945afc6a Binary files /dev/null and b/docs/images/failover_v1.jpg differ diff --git a/docs/images/failover_v2.jpg b/docs/images/failover_v2.jpg new file mode 100644 index 000000000..baebc22b9 Binary files /dev/null and b/docs/images/failover_v2.jpg differ diff --git a/docs/images/failover_v2_panic.jpg b/docs/images/failover_v2_panic.jpg new file mode 100644 index 000000000..a50e1a360 Binary files /dev/null and b/docs/images/failover_v2_panic.jpg differ diff --git a/docs/using-the-python-driver/SessionState.md b/docs/using-the-python-driver/SessionState.md deleted file mode 100644 index e0d42ee48..000000000 --- a/docs/using-the-python-driver/SessionState.md +++ /dev/null @@ -1,37 +0,0 @@ -# Session States - -## What is a session state? - -Every connection is associated with a connection session on the server and a group of related session settings like the autocommit flag or the read_only flag. The following session settings are tracked by the AWS Advanced Python Driver, and together they form a session state: -- autocommit -- read_only - -Since the AWS Advanced Python Driver can transparently switch physical connection to a server (for instance, during a cluster failover), it's important to re-apply the current session state to a new connection during such switch. - -## Tracking Session States Changes -
diagram for the session state transfer
- -The diagram above shows the process of switching one database connection `A` to a new connection `B`. After connection `A` is established, it's returned to the user application, and the user application may use this connection to query data from the database as well as to change some session settings. -For example, if the user application sets the `read_only` value on a connection, the AWS Advanced Python Driver intercepts this operation and stores a new session setting for the `read_only` setting. At the same time, the driver verifies if the original session setting is known or not. If the original setting is not known, the driver will fetch the value of `read_only` and store it as a pristine value in order to save the original session setting. Later, the driver may need the pristine value to restore the connection session state to its original state. - -## Restore to the Original Session State - -Before closing an existing connection, the AWS Advanced Python Driver may try to reset all changes to the session state made by the user application. Some application frameworks and connection pools may intercept calls that close connections and may perform additional connection configuration. Since the AWS Advanced Python Driver might change the internal physical connection to a server, a new physical connection's settings may become unexpected to the user application and may cause errors. It is also important to mention that calling `close()` on a connection while using connection pooling doesn't close the connection or stop communicating to a server. Instead, the connection is returned to a pool of available connections. Cleaning up a session state before returning a connection to a pool is necessary to avoid side effects and errors when a connection is retrieved from a pool to be reused. - -Before closing a connection, the AWS Advanced Python Driver sets its session state settings with the pristine values that have been previously stored in the driver. If a pristine value isn't available, it means that there have been no changes to that particular setting made by the user application, and that it's safe to assume that this setting is in its original/unchanged state. - -Session state reset is enabled by default, and can be disabled by using the `reset_session_state_on_close` configuration parameter. - -## Transfer Session State to a new Connection - -When the driver needs to switch to a new connection, it opens a new connection and transfers a session state to it. All current session state values are applied to the new connection. Pristine values for a new connection are also fetched and stored if needed. When a new connection is configured, it replaces the current internal connection. - -Session transfer is enabled by default, and can be disabled by using the `transfer_session_state_on_switch` configuration parameter. - -## Session State Custom handlers - -It is possible to extend or replace existing logic of resetting session state and transferring session state with custom handlers. Use the following methods on `aws_advanced_python_wrapper.states.SessionStateTransferHandlers` class to set and reset custom handlers: -- `set_reset_session_state_on_close_func` -- `clear_reset_session_state_on_close_func` -- `set_transfer_session_state_on_switch_func` -- `reset_transfer_session_state_on_switch_func` diff --git a/docs/using-the-python-driver/DatabaseDialects.md b/docs/using-the-python-wrapper/DatabaseDialects.md similarity index 75% rename from docs/using-the-python-driver/DatabaseDialects.md rename to docs/using-the-python-wrapper/DatabaseDialects.md index 03aaa52f8..ec77928ff 100644 --- a/docs/using-the-python-driver/DatabaseDialects.md +++ b/docs/using-the-python-wrapper/DatabaseDialects.md @@ -1,10 +1,10 @@ # Database Dialects -The AWS Advanced Python Driver uses two types of dialects: driver dialects and database dialects. This page is on database dialects. To find out more about driver dialects, see [Driver Dialects](./DriverDialects.md). +The AWS Advanced Python Wrapper uses two types of dialects: driver dialects and database dialects. This page is on database dialects. To find out more about driver dialects, see [Driver Dialects](./DriverDialects.md). ## What are database dialects? -The AWS Advanced Python Driver is a wrapper that requires an underlying driver, and it is meant to be compatible with any Python driver. Database dialects help the AWS Advanced Python Driver determine what kind of underlying database is being used. To function correctly, the AWS Advanced Python Driver requires details unique to specific databases such as the default port number or the method to get the current host from the database. These details can be defined and provided to the AWS Advanced Python Driver by using database dialects. +The AWS Advanced Python Wrapper requires an underlying driver, and it is meant to be compatible with any Python driver. Database dialects help the AWS Advanced Python Wrapper determine what kind of underlying database is being used. To function correctly, the AWS Advanced Python Wrapper requires details unique to specific databases such as the default port number or the method to get the current host from the database. These details can be defined and provided to the AWS Advanced Python Wrapper by using database dialects. ## Configuration Parameters @@ -14,7 +14,7 @@ The AWS Advanced Python Driver is a wrapper that requires an underlying driver, > **NOTES:** > -> The `wrapper_dialect` parameter is not required. When it is not provided by the user, the AWS Advanced Python Driver will attempt to determine which of the existing dialects to use based on other connection details. However, if the dialect is known by the user, it is preferable to set the `wrapper_dialect` parameter because it will take time to resolve the dialect. +> The `wrapper_dialect` parameter is not required. When it is not provided by the user, the AWS Advanced Python Wrapper will attempt to determine which of the existing dialects to use based on other connection details. However, if the dialect is known by the user, it is preferable to set the `wrapper_dialect` parameter because it will take time to resolve the dialect. ### List of Available Dialect Codes @@ -36,13 +36,13 @@ Dialect codes specify what kind of database any connections will be made to. ## Custom Dialects -If you are interested in using the AWS Advanced Python Driver but your desired database type is not currently supported, it is possible to create a custom dialect. +If you are interested in using the AWS Advanced Python Wrapper but your desired database type is not currently supported, it is possible to create a custom dialect. To create a custom dialect, implement the [`DatabaseDialect`](../../aws_advanced_python_wrapper/database_dialect.py) class. For databases clusters that are aware of their topology, the `TopologyAwareDatabaseDialect` interface should also be implemented. See the following classes in the [Database Dialect file](../../aws_advanced_python_wrapper/database_dialect.py) for examples: - PgDialect is a generic dialect that should work with any PostgreSQL database. - AuroraPgDialect is an extension of PgDialect, but also implements the `TopologyAwareDatabaseDialect` interface. -Once the custom dialect class has been created, tell the AWS Advanced Python Driver to use it by setting the `custom_dialect` attribute in the `DialectManager` class. It is not necessary to set the `wrapper_dialect` parameter. See below for an example: +Once the custom dialect class has been created, tell the AWS Advanced Python Wrapper to use it by setting the `custom_dialect` attribute in the `DialectManager` class. It is not necessary to set the `wrapper_dialect` parameter. See below for an example: ```python custom_dialect: Dialect = CustomDialect() diff --git a/docs/using-the-python-driver/DriverDialects.md b/docs/using-the-python-wrapper/DriverDialects.md similarity index 56% rename from docs/using-the-python-driver/DriverDialects.md rename to docs/using-the-python-wrapper/DriverDialects.md index 8fe3c25d1..a5f3dd98b 100644 --- a/docs/using-the-python-driver/DriverDialects.md +++ b/docs/using-the-python-wrapper/DriverDialects.md @@ -1,10 +1,10 @@ # Driver Dialects -The AWS Advanced Python Driver uses two types of dialects: driver dialects and database dialects. This page is on driver dialects. To find out more about database dialects, see [Database Dialects](./DatabaseDialects.md). +The AWS Advanced Python Wrapper uses two types of dialects: driver dialects and database dialects. This page is on driver dialects. To find out more about database dialects, see [Database Dialects](./DatabaseDialects.md). ## What are driver dialects? -The AWS Advanced Python Driver is a wrapper that requires an underlying driver, and it is meant to be compatible with any Python driver. Driver dialects help the AWS Advanced Python Driver to properly pass database calls to an underlying Python driver. To function correctly, the AWS Advanced Python Driver requires details unique to specific target driver such as a database name used by the driver or whether to include some specific configuration parameters to a list of properties. These details can be defined and provided to the AWS Advanced Python Driver by using driver dialects. +The AWS Advanced Python Wrapper is a wrapper that requires an underlying driver, and it is meant to be compatible with any Python driver. Driver dialects help the AWS Advanced Python Wrapper to properly pass database calls to an underlying Python driver. To function correctly, the AWS Advanced Python Wrapper requires details unique to specific target driver such as a database name used by the driver or whether to include some specific configuration parameters to a list of properties. These details can be defined and provided to the AWS Advanced Python Wrapper by using driver dialects. By default, the driver dialect is determined based on the Connection function passed to the `AwsWrapperConnection#connect` method. @@ -16,7 +16,7 @@ By default, the driver dialect is determined based on the Connection function pa > **NOTES:** > -> The `wrapper_driver_dialect` parameter is not required. When it is not provided by the user, the AWS Advanced Python Driver will determine which of the existing target driver dialects to use based on the Connect function passed to the driver. If target driver specific implementation is not found, the AWS Advanced Python Driver will use a generic target driver dialect. +> The `wrapper_driver_dialect` parameter is not required. When it is not provided by the user, the AWS Advanced Python Wrapper will determine which of the existing target driver dialects to use based on the Connect function passed to the wrapper. If target driver specific implementation is not found, the AWS Advanced Python Wrapper will use a generic target driver dialect. ### List of Available Driver Codes @@ -30,7 +30,7 @@ Driver Dialect codes specify which driver dialect class to use. ## Custom Driver Dialects -If you are interested in using the AWS Advanced Python Driver but your desired target driver has unique features incompatible with the generic dialect, it is possible to create a custom target driver dialect. +If you are interested in using the AWS Advanced Python Wrapper but your desired target driver has unique features incompatible with the generic dialect, it is possible to create a custom target driver dialect. To create a custom target driver dialect, implement the [`DriverDialect`](../../aws_advanced_python_wrapper/driver_dialect.py) interface. See the following classes for examples: @@ -39,7 +39,7 @@ To create a custom target driver dialect, implement the [`DriverDialect`](../../ - [MysqlDriverDialect](../../aws_advanced_python_wrapper/mysql_driver_dialect.py) - This is a dialect that should work with [MySQL Connector/Python](https://github.com/mysql/mysql-connector-python). -Once the custom driver dialect class has been created, tell the AWS Advanced Python Driver to use it by setting the `custom_dialect` attribute in the `DriverDialectManager` class. It is not necessary to set the `wrapper_driver_dialect` parameter. See below for an example: +Once the custom driver dialect class has been created, tell the AWS Advanced Python Wrapper to use it by setting the `custom_dialect` attribute in the `DriverDialectManager` class. It is not necessary to set the `wrapper_driver_dialect` parameter. See below for an example: ```python custom_driver_dialect: DriverDialect = CustomDriverDialect() diff --git a/docs/using-the-python-driver/FailoverConfigurationGuide.md b/docs/using-the-python-wrapper/FailoverConfigurationGuide.md similarity index 94% rename from docs/using-the-python-driver/FailoverConfigurationGuide.md rename to docs/using-the-python-wrapper/FailoverConfigurationGuide.md index a1e62be1f..df28c36fa 100644 --- a/docs/using-the-python-driver/FailoverConfigurationGuide.md +++ b/docs/using-the-python-wrapper/FailoverConfigurationGuide.md @@ -34,5 +34,5 @@ A common misconception about failover is the expectation that only one host will ### Monitor Failures and Investigate If you are experiencing difficulties with the failover plugin, try the following: -- Enable [logging](../using-the-python-driver/UsingThePythonDriver.md#logging) to find the cause of the failure. If it is a timeout, review the [failover time profiles](#failover-time-profiles) section and adjust the timeout values. +- Enable [logging](./UsingThePythonWrapper.md#logging) to find the cause of the failure. If it is a timeout, review the [failover time profiles](#failover-time-profiles) section and adjust the timeout values. - For additional assistance, visit the [getting help page](../../README.md#getting-help-and-opening-issues). diff --git a/docs/using-the-python-driver/HostAvailabilityStrategy.md b/docs/using-the-python-wrapper/HostAvailabilityStrategy.md similarity index 83% rename from docs/using-the-python-driver/HostAvailabilityStrategy.md rename to docs/using-the-python-wrapper/HostAvailabilityStrategy.md index b7dc129f9..84a9783b2 100644 --- a/docs/using-the-python-driver/HostAvailabilityStrategy.md +++ b/docs/using-the-python-wrapper/HostAvailabilityStrategy.md @@ -1,16 +1,16 @@ # Host Availability Strategy ## What is Host Availability Strategy -The AWS Advanced Python Driver keeps track of the availability of database hosts. A host's availability status can be either `AVAILABLE` or `NOT_AVAILABLE`. This information is used by the driver when deciding which hosts to connect to. +The AWS Advanced Python Wrapper keeps track of the availability of database hosts. A host's availability status can be either `AVAILABLE` or `NOT_AVAILABLE`. This information is used by the wrapper when deciding which hosts to connect to. -When database hosts fail, they are marked as `NOT_AVAILABLE` and will not be checked again to see if they have recovered. Host Availability Strategies can help the driver keep host health statuses up to date by checking whether unavailable hosts have become available. For example, the Exponential Backoff strategy will let the driver retry hosts that have had their availability set to `NOT_AVAILABLE` by setting the host availability to `AVAILABLE` after a backoff period. +When database hosts fail, they are marked as `NOT_AVAILABLE` and will not be checked again to see if they have recovered. Host Availability Strategies can help the wrapper keep host health statuses up to date by checking whether unavailable hosts have become available. For example, the Exponential Backoff strategy will let the wrapper retry hosts that have had their availability set to `NOT_AVAILABLE` by setting the host availability to `AVAILABLE` after a backoff period. -Different strategies can be swapped out for different behaviors. The default Host Availability Strategy the driver uses can be configured as specified in the [configuration parameters table](#configuration-parameters). +Different strategies can be swapped out for different behaviors. The default Host Availability Strategy the wrapper uses can be configured as specified in the [configuration parameters table](#configuration-parameters). ## Configuration Parameters | Parameter | Value | Required | Description | Default Value | |---------------------------------------------------|-----------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| `default_host_availability_strategy` | `String` | No | This overrides the driver's default host availability strategy. Possible values are listed in the [Host Availability Strategies](#host-availability-strategies) section. | `None` | +| `default_host_availability_strategy` | `String` | No | This overrides the wrapper's default host availability strategy. Possible values are listed in the [Host Availability Strategies](#host-availability-strategies) section. | `None` | | `host_availability_strategy_max_retries` | `Integer` | No | Maximum number of times a host availability strategy will retry a host that is not available. | `5` | | `host_availability_strategy_initial_backoff_time` | `Integer` | No | The initial wait time in seconds. This parameter is only applicable for host availability strategies employing a time-based backoff. | `30` | diff --git a/docs/using-the-python-driver/ReaderSelectionStrategies.md b/docs/using-the-python-wrapper/ReaderSelectionStrategies.md similarity index 100% rename from docs/using-the-python-driver/ReaderSelectionStrategies.md rename to docs/using-the-python-wrapper/ReaderSelectionStrategies.md diff --git a/docs/using-the-python-wrapper/SessionState.md b/docs/using-the-python-wrapper/SessionState.md new file mode 100644 index 000000000..50dd54bcf --- /dev/null +++ b/docs/using-the-python-wrapper/SessionState.md @@ -0,0 +1,37 @@ +# Session States + +## What is a session state? + +Every connection is associated with a connection session on the server and a group of related session settings like the autocommit flag or the read_only flag. The following session settings are tracked by the AWS Advanced Python Wrapper, and together they form a session state: +- autocommit +- read_only + +Since the AWS Advanced Python Wrapper can transparently switch physical connection to a server (for instance, during a cluster failover), it's important to re-apply the current session state to a new connection during such switch. + +## Tracking Session States Changes +
diagram for the session state transfer
+ +The diagram above shows the process of switching one database connection `A` to a new connection `B`. After connection `A` is established, it's returned to the user application, and the user application may use this connection to query data from the database as well as to change some session settings. +For example, if the user application sets the `read_only` value on a connection, the AWS Advanced Python Wrapper intercepts this operation and stores a new session setting for the `read_only` setting. At the same time, the wrapper verifies if the original session setting is known or not. If the original setting is not known, the wrapper will fetch the value of `read_only` and store it as a pristine value in order to save the original session setting. Later, the wrapper may need the pristine value to restore the connection session state to its original state. + +## Restore to the Original Session State + +Before closing an existing connection, the AWS Advanced Python Wrapper may try to reset all changes to the session state made by the user application. Some application frameworks and connection pools may intercept calls that close connections and may perform additional connection configuration. Since the AWS Advanced Python Wrapper might change the internal physical connection to a server, a new physical connection's settings may become unexpected to the user application and may cause errors. It is also important to mention that calling `close()` on a connection while using connection pooling doesn't close the connection or stop communicating to a server. Instead, the connection is returned to a pool of available connections. Cleaning up a session state before returning a connection to a pool is necessary to avoid side effects and errors when a connection is retrieved from a pool to be reused. + +Before closing a connection, the AWS Advanced Python Wrapper sets its session state settings with the pristine values that have been previously stored in the wrapper. If a pristine value isn't available, it means that there have been no changes to that particular setting made by the user application, and that it's safe to assume that this setting is in its original/unchanged state. + +Session state reset is enabled by default, and can be disabled by using the `reset_session_state_on_close` configuration parameter. + +## Transfer Session State to a new Connection + +When the wrapper needs to switch to a new connection, it opens a new connection and transfers a session state to it. All current session state values are applied to the new connection. Pristine values for a new connection are also fetched and stored if needed. When a new connection is configured, it replaces the current internal connection. + +Session transfer is enabled by default, and can be disabled by using the `transfer_session_state_on_switch` configuration parameter. + +## Session State Custom handlers + +It is possible to extend or replace existing logic of resetting session state and transferring session state with custom handlers. Use the following methods on `aws_advanced_python_wrapper.states.SessionStateTransferHandlers` class to set and reset custom handlers: +- `set_reset_session_state_on_close_func` +- `clear_reset_session_state_on_close_func` +- `set_transfer_session_state_on_switch_func` +- `reset_transfer_session_state_on_switch_func` diff --git a/docs/using-the-python-driver/SupportForRDSMultiAzDBCluster.md b/docs/using-the-python-wrapper/SupportForRDSMultiAzDBCluster.md similarity index 69% rename from docs/using-the-python-driver/SupportForRDSMultiAzDBCluster.md rename to docs/using-the-python-wrapper/SupportForRDSMultiAzDBCluster.md index cbe1f3dbd..77e331432 100644 --- a/docs/using-the-python-driver/SupportForRDSMultiAzDBCluster.md +++ b/docs/using-the-python-wrapper/SupportForRDSMultiAzDBCluster.md @@ -1,14 +1,14 @@ # Support for Amazon RDS Multi-AZ DB Cluster -In addition to Aurora database clusters, the AWS Advanced Python Driver supports the Amazon RDS Multi-AZ DB Cluster Deployment. By leveraging the topology information within the RDS Multi-AZ DB Cluster, the driver is capable of switching over the connection to a new writer host in approximately 1 second or less, given there is no replica lag during minor version upgrades or OS maintenance upgrades. +In addition to Aurora database clusters, the AWS Advanced Python Wrapper supports the Amazon RDS Multi-AZ DB Cluster Deployment. By leveraging the topology information within the RDS Multi-AZ DB Cluster, the wrapper is capable of switching over the connection to a new writer host in approximately 1 second or less, given there is no replica lag during minor version upgrades or OS maintenance upgrades. ## General Usage -The process of using the AWS Advanced Python Driver with RDS Multi-AZ DB Cluster is the same as using it with an RDS Aurora cluster. All properties, configurations, functions, etc., remain consistent. Instead of connecting to a generic database endpoint, simply replace the endpoint with the Cluster Writer Endpoint provided by the RDS Multi-AZ DB Cluster. +The process of using the AWS Advanced Python Wrapper with RDS Multi-AZ DB Cluster is the same as using it with an RDS Aurora cluster. All properties, configurations, functions, etc., remain consistent. Instead of connecting to a generic database endpoint, simply replace the endpoint with the Cluster Writer Endpoint provided by the RDS Multi-AZ DB Cluster. ### MySQL -There are permissions that must be granted to all non-administrative users who need database access. Without proper access, these users cannot utilize many of the driver's advanced features, including failover support. To grant the necessary permissions to non-administrative users, execute the following statement: +There are permissions that must be granted to all non-administrative users who need database access. Without proper access, these users cannot utilize many of the wrapper's advanced features, including failover support. To grant the necessary permissions to non-administrative users, execute the following statement: ```sql GRANT SELECT ON mysql.rds_topology TO 'non-admin-username'@'%' @@ -38,7 +38,7 @@ Per AWS documentation, the `rds_tools` extension must be manually installed usin CREATE EXTENSION rds_tools; ``` -The extension must be granted to all non-administrative users who need database access. Without access to `rds_tools`, non-admin users cannot utilize many of the driver's advanced features, including failover support. To grant the necessary permissions to non-administrative users, execute the following statement: +The extension must be granted to all non-administrative users who need database access. Without access to `rds_tools`, non-admin users cannot utilize many of the wrapper's advanced features, including failover support. To grant the necessary permissions to non-administrative users, execute the following statement: ```sql GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA rds_tools TO non-admin-username; @@ -67,15 +67,15 @@ For more details on the `failover` plugin configuration, refer to the [Failover ## Examples -We have created many examples in the [examples](../examples) folder demonstrating how to use the driver. +We have created many examples in the [examples](../examples) folder demonstrating how to use the wrapper. ## Limitations The following plugins have been tested and confirmed to work with Amazon RDS Multi-AZ DB Clusters: -* [Aurora Connection Tracker Plugin](../using-the-python-driver/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) -* [Failover Connection Plugin](../using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md) -* [Host Monitoring Connection Plugin](../using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md) +* [Aurora Connection Tracker Plugin](/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) +* [Failover Connection Plugin](/using-plugins/UsingTheFailoverPlugin.md) +* [Host Monitoring Connection Plugin](/using-plugins/UsingTheHostMonitoringPlugin.md) The compatibility of other plugins has not been tested at this time. They may function as expected or potentially result in unhandled behavior. Use at your own discretion. diff --git a/docs/using-the-python-driver/Telemetry.md b/docs/using-the-python-wrapper/Telemetry.md similarity index 80% rename from docs/using-the-python-driver/Telemetry.md rename to docs/using-the-python-wrapper/Telemetry.md index fef18713a..0014953b5 100644 --- a/docs/using-the-python-driver/Telemetry.md +++ b/docs/using-the-python-wrapper/Telemetry.md @@ -1,31 +1,31 @@ # Monitoring Monitoring is the ability to gather data and insights on the execution of an application. Users will also be able to inspect the gathered data and determine potential actions to take depending on the data collected. -The AWS Advanced Python Driver has a Telemetry feature allowing you to collect and visualize data of the AWS Advanced Python Driver execution at a global level and at plugin level. -You can now monitor the performance of the driver as a whole or within specific plugins with your configurations, and determine whether the driver's performance meets your expectations. +The AWS Advanced Python Wrapper has a Telemetry feature allowing you to collect and visualize data of the AWS Advanced Python Wrapper execution at a global level and at plugin level. +You can now monitor the performance of the wrapper as a whole or within specific plugins with your configurations, and determine whether the wrapper's performance meets your expectations. ## Terminology -The AWS Advanced Python Driver provides telemetry data through two different forms: **Traces** and **Metrics**. +The AWS Advanced Python Wrapper provides telemetry data through two different forms: **Traces** and **Metrics**. ### Traces Traces give an overview of what is happening in a specific section of the execution of an application. A trace is composed by a hierarchical sequence of segments, each of which contain basic information about the execution (e.g., duration), and whether that section was executed successfully or not. -In the AWS Advanced Python Driver, initially a trace will be generated for every method call made to the driver. Depending on whether the user application has already a trace open, it might be either nested into the opened trace or dropped. And then, for each enabled plugin, another segment will be created only for the plugin execution, linked to the +In the AWS Advanced Python Wrapper, initially a trace will be generated for every method call made to the wrapper. Depending on whether the user application has already a trace open, it might be either nested into the opened trace or dropped. And then, for each enabled plugin, another segment will be created only for the plugin execution, linked to the method call segment. -Traces from the AWS Advanced Python Driver are submitted to [**AWS X-Ray**](https://aws.amazon.com/xray/). +Traces from the AWS Advanced Python Wrapper are submitted to [**AWS X-Ray**](https://aws.amazon.com/xray/). ### Metrics Metrics are numeric data that were measured and collected through the execution of an application. Those metrics can give an insight on how many times some action (e.g., failover) has happened, and for actions that may happen multiple times, their success or failure rate (failover, cache hits, etc.), amongst other related information. -The AWS Advanced Python Driver will submit metrics data to [**Amazon Cloudwatch**](https://aws.amazon.com/cloudwatch/). +The AWS Advanced Python Wrapper will submit metrics data to [**Amazon Cloudwatch**](https://aws.amazon.com/cloudwatch/). -The list of available metrics for the AWS Advanced Python Driver and its plugins is available in the [List of Metrics](#List-Of-Metrics) section of this page. +The list of available metrics for the AWS Advanced Python Wrapper and its plugins is available in the [List of Metrics](#List-Of-Metrics) section of this page. ## Setting up the AWS Distro for OpenTelemetry Collector (ADOT Collector) @@ -37,7 +37,7 @@ Before enabling the Telemetry feature, a few setup steps are required to ensure to [AWS X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/security-iam.html) and [Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/auth-and-access-control-cw.html). -2. Download the [AWS Distro for OpenTelemetry Collector](https://aws-otel.github.io/docs/getting-started/collector) and set it up. The AWS Distro for OpenTelemetry Collector is responsible from receiving telemetry data from the application using the AWS Advanced Python Driver and forward it to AWS. Both of those connections happen via HTTP, therefore URLs and ports need to be correctly configured for the collector. +2. Download the [AWS Distro for OpenTelemetry Collector](https://aws-otel.github.io/docs/getting-started/collector) and set it up. The AWS Distro for OpenTelemetry Collector is responsible from receiving telemetry data from the application using the AWS Advanced Python Wrapper and forward it to AWS. Both of those connections happen via HTTP, therefore URLs and ports need to be correctly configured for the collector. > [!WARNING] > - The AWS Distro for OpenTelemetry Collector can be set up either locally or remotely. It is up to the user to decide @@ -48,16 +48,16 @@ Before enabling the Telemetry feature, a few setup steps are required to ensure ## Using Telemetry -Telemetry for the AWS Advanced Python Driver is a monitoring strategy that overlooks all plugins enabled in [`plugins`](./UsingThePythonDriver.md#connection-plugin-manager-parameters) and is not a plugin in itself. +Telemetry for the AWS Advanced Python Wrapper is a monitoring strategy that overlooks all plugins enabled in [`plugins`](UsingThePythonWrapper.md#connection-plugin-manager-parameters) and is not a plugin in itself. Therefore no changes are required in the `plugins` parameter to enable Telemetry. -In order to enable Telemetry in the AWS Advanced Python Driver, you need to: +In order to enable Telemetry in the AWS Advanced Python Wrapper, you need to: 1. Set the `enable_telemetry` property to `True`. You can either set it through Properties or directly in the connection string. 2. Set up the recorders that will export the telemetry data from the code to the ADOT Collector. -Setting up the recorders require to instantiate an `TracerProvider` and a `MeterProvider` in the application code prior to executing the driver. +Setting up the recorders require to instantiate an `TracerProvider` and a `MeterProvider` in the application code prior to executing the wrapper. Instantiating the `TracerProvider` requires you to configure the endpoints where traces are being forwarded to, whereas the `MeterProvider` allows you to configure how meters are exported. The code sample below shows a simple manner to instantiate trace recording in an application using OpenTelemetry. @@ -89,19 +89,19 @@ We also provide more complete sample application using telemetry in the examples In addition to the parameter that enables Telemetry, there are other parameters configuring how telemetry data will be forwarded. -| Parameter | Value | Required | Description | Default Value | -|------------------------------|:-------:|:--------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| `enable_telemetry` | Boolean | No | Telemetry will be enabled when this property is set to `True`, otherwise no telemetry data will be gathered during the execution of the wrapper. | `False` | -| `telemetry_traces_backend` | String | No | Determines to which backend the gathered tracing data will be forwarded to. Possible values include: `NONE`, `XRAY`, and `OTLP`.
`NONE` indicates that the application will collect tracing data but this data will not be forwarded anywhere.
`XRAY` indicates that the traces will be collected by the AWS XRay Daemon.
`OTLP` indicates that the traces will be collected by the AWS OTEL Collector. | `NONE` | -| `telemetry_metrics_backend` | String | No | Determines to which backend the gathered metrics data will be forwarded to. Possible values include: `NONE` and `OTLP`.
`NONE` indicates that the application will collect metrics data but this data will not be forwarded anywhere.
`OTLP` indicates that the metrics will be collected by the AWS OTEL COllector. | `NONE` | -| `telemetry_submit_top_level` | Boolean | No | By default the driver will look for open traces in the users application prior to record telemetry data. If there is a current open trace, the traces created will be attached to that open trace. If not, all telemetry traces created will be top level. Setting the parameter to `False` means that every method call to the driver will generate a trace with no direct parent trace attached to it. If there is already an open trace being recorded by the application, no driver traces will be created. See the [Nested tracing strategies section](#nested-tracing-strategies-using-amazon-xray) for more information. | `False` | +| Parameter | Value | Required | Description | Default Value | +|------------------------------|:-------:|:--------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| +| `enable_telemetry` | Boolean | No | Telemetry will be enabled when this property is set to `True`, otherwise no telemetry data will be gathered during the execution of the wrapper. | `False` | +| `telemetry_traces_backend` | String | No | Determines to which backend the gathered tracing data will be forwarded to. Possible values include: `NONE`, `XRAY`, and `OTLP`.
`NONE` indicates that the application will collect tracing data but this data will not be forwarded anywhere.
`XRAY` indicates that the traces will be collected by the AWS XRay Daemon.
`OTLP` indicates that the traces will be collected by the AWS OTEL Collector. | `NONE` | +| `telemetry_metrics_backend` | String | No | Determines to which backend the gathered metrics data will be forwarded to. Possible values include: `NONE` and `OTLP`.
`NONE` indicates that the application will collect metrics data but this data will not be forwarded anywhere.
`OTLP` indicates that the metrics will be collected by the AWS OTEL COllector. | `NONE` | +| `telemetry_submit_top_level` | Boolean | No | By default the wrapper will look for open traces in the users application prior to record telemetry data. If there is a current open trace, the traces created will be attached to that open trace. If not, all telemetry traces created will be top level. Setting the parameter to `False` means that every method call to the wrapper will generate a trace with no direct parent trace attached to it. If there is already an open trace being recorded by the application, no wrapper traces will be created. See the [Nested tracing strategies section](#nested-tracing-strategies-using-amazon-xray) for more information. | `False` | ## Nested tracing strategies using Amazon XRay -As you could see in the [Telemetry Parameters](#Telemetry-Parameters) section, the AWS Advanced Python Driver allows a user to determine which strategy for nested traces to use when using Telemetry. +As you could see in the [Telemetry Parameters](#Telemetry-Parameters) section, the AWS Advanced Python Wrapper allows a user to determine which strategy for nested traces to use when using Telemetry. -Traces are hierarchical entities, and it might be that the user application already has an open trace in a given sequence of code that connects to the AWS Advanced Python Driver. -In this case, the Telemetry feature allows users to determine which strategy to use for the Telemetry traces generated when using the driver. +Traces are hierarchical entities, and it might be that the user application already has an open trace in a given sequence of code that connects to the AWS Advanced Python Wrapper. +In this case, the Telemetry feature allows users to determine which strategy to use for the Telemetry traces generated when using the wrapper. A top level trace is a trace that has no link to any other parent trace, and is directly accessible from the list of submitted traces within XRay. In the following pictures, the top level traces of an application are displayed in AWS X-Ray. @@ -116,7 +116,7 @@ All the individual plugin traces are linked to a parent trace for the method cal ## List of Metrics -The AWS Advanced Python Driver also submits a set of metrics to Amazon Cloudwatch when the driver is used. +The AWS Advanced Python Wrapper also submits a set of metrics to Amazon Cloudwatch when the wrapper is used. These metrics are predefined, and they help give insight on what is happening inside the plugins when the plugins are used. Metrics can be one of 3 types: counters, gauges or histograms. diff --git a/docs/using-the-python-driver/UsingThePythonDriver.md b/docs/using-the-python-wrapper/UsingThePythonWrapper.md similarity index 89% rename from docs/using-the-python-driver/UsingThePythonDriver.md rename to docs/using-the-python-wrapper/UsingThePythonWrapper.md index bc753267d..8a17b458f 100644 --- a/docs/using-the-python-driver/UsingThePythonDriver.md +++ b/docs/using-the-python-wrapper/UsingThePythonWrapper.md @@ -1,25 +1,25 @@ -# Using the AWS Advanced Python Driver +# Using the AWS Advanced Python Wrapper -The AWS Advanced Python Driver leverages community Python drivers and enables support of AWS and Aurora functionalities. +The AWS Advanced Python Wrapper leverages community Python drivers and enables support of AWS and Aurora functionalities. Currently, [Psycopg](https://github.com/psycopg/psycopg) 3.1.12+ and [MySQL Connector/Python](https://github.com/mysql/mysql-connector-python) 8.1.0+ are supported. Compatibility with prior versions of these drivers has not been tested. -### Using the AWS Advanced Python Driver with RDS Multi-AZ database Clusters +### Using the AWS Advanced Python Wrapper with RDS Multi-AZ database Clusters The [AWS RDS Multi-AZ DB Clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) are capable of switching over the current writer host to another host in the cluster within approximately 1 second or less, in case of minor engine version upgrade or OS maintenance operations. -The AWS Advanced Python Driver has been optimized for such fast-failover when working with AWS RDS Multi-AZ DB Clusters. +The AWS Advanced Python Wrapper has been optimized for such fast-failover when working with AWS RDS Multi-AZ DB Clusters. With the Failover plugin, the downtime during certain DB cluster operations, such as engine minor version upgrades, can be reduced to one second or even less with finely tuned parameters. It supports both MySQL and PostgreSQL clusters. Visit [this page](./SupportForRDSMultiAzDBCluster.md) for more details. -## Using the AWS Advanced Python Driver with plain RDS databases +## Using the AWS Advanced Python Wrapper with plain RDS databases -It is possible to use the AWS Advanced Python Driver with plain RDS databases, but individual features may or may not be compatible. For example, failover handling and enhanced failure monitoring are not compatible with plain RDS databases and the relevant plugins must be disabled. Plugins can be enabled or disabled as seen in the [Connection Plugin Manager Parameters](#connection-plugin-manager-parameters) section. Please note that some plugins have been enabled by default. Plugin compatibility can be verified in the [plugins table](#list-of-available-plugins). +It is possible to use the AWS Advanced Python Wrapper with plain RDS databases, but individual features may or may not be compatible. For example, failover handling and enhanced failure monitoring are not compatible with plain RDS databases and the relevant plugins must be disabled. Plugins can be enabled or disabled as seen in the [Connection Plugin Manager Parameters](#connection-plugin-manager-parameters) section. Please note that some plugins have been enabled by default. Plugin compatibility can be verified in the [plugins table](#list-of-available-plugins). -## AWS Advanced Python Driver Parameters +## AWS Advanced Python Wrapper Parameters -These parameters are applicable to any instance of the AWS Advanced Python Driver. +These parameters are applicable to any instance of the AWS Advanced Python Wrapper. | Parameter | Description | Required | Default Value | |----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------| @@ -73,7 +73,7 @@ finally: ## Plugins -The AWS Advanced Python Driver uses plugins to execute database API calls. You can think of a plugin as an extensible code module that adds extra logic around any database API calls. The AWS Advanced Python Driver has a number of [built-in plugins](#list-of-available-plugins) available for use. +The AWS Advanced Python Wrapper uses plugins to execute database API calls. You can think of a plugin as an extensible code module that adds extra logic around any database API calls. The AWS Advanced Python Wrapper has a number of [built-in plugins](#list-of-available-plugins) available for use. Plugins are loaded and managed through the Connection Plugin Manager and may be identified by a `str` name in the form of plugin code. @@ -86,11 +86,12 @@ Plugins are loaded and managed through the Connection Plugin Manager and may be ### List of Available Plugins -The AWS Advanced Python Driver has several built-in plugins that are available to use. Please visit the individual plugin page for more details. +The AWS Advanced Python Wrapper has several built-in plugins that are available to use. Please visit the individual plugin page for more details. | Plugin name | Plugin Code | Database Compatibility | Description | Additional Required Dependencies | |--------------------------------------------------------------------------------------------------------|-------------------------------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------| | [Failover Connection Plugin](./using-plugins/UsingTheFailoverPlugin.md) | `failover` | Aurora | Enables the failover functionality supported by Amazon Aurora clusters. Prevents opening a wrong connection to an old writer host dues to stale DNS after failover event. This plugin is enabled by default. | None | +| [Failover Connection Plugin v2](./using-plugins/UsingTheFailover2Plugin.md) | `failover_v2` | Aurora | The next version (v2) of the [Failover Plugin](./using-plugins/UsingTheFailoverPlugin.md) | None | | [Host Monitoring Connection Plugin](./using-plugins/UsingTheHostMonitoringPlugin.md) | `host_monitoring_v2` or `host_monitoring` | Aurora | Enables enhanced host connection failure monitoring, allowing faster failure detection rates. This plugin is enabled by default. | None | | [IAM Authentication Connection Plugin](./using-plugins/UsingTheIamAuthenticationPlugin.md) | `iam` | Any database | Enables users to connect to their Amazon Aurora clusters using AWS Identity and Access Management (IAM). | [Boto3 - AWS SDK for Python](https://aws.amazon.com/sdk-for-python/) | | [AWS Secrets Manager Connection Plugin](./using-plugins/UsingTheAwsSecretsManagerPlugin.md) | `aws_secrets_manager` | Any database | Enables fetching database credentials from the AWS Secrets Manager service. | [Boto3 - AWS SDK for Python](https://aws.amazon.com/sdk-for-python/) | @@ -108,9 +109,9 @@ For more information, see [Custom Plugins](../development-guide/LoadablePlugins. ## Logging -The AWS Python Driver uses the standard [Python logging module](https://docs.python.org/3/library/logging.html) to log information. To configure logging for the AWS Python Driver, refer to [this section of the Python logging docs](https://docs.python.org/3/howto/logging.html#configuring-logging). Please note the following: +The AWS Advanced Python Wrapper uses the standard [Python logging module](https://docs.python.org/3/library/logging.html) to log information. To configure logging for the AWS Advanced Python Wrapper, refer to [this section of the Python logging docs](https://docs.python.org/3/howto/logging.html#configuring-logging). Please note the following: - As mentioned in a warning in [this section](https://docs.python.org/3/howto/logging.html#configuring-logging), existing loggers will be disabled by default when `fileConfig()` or `dictConfig()` is called. We recommend that you pass `disable_existing_loggers=False` when calling either of these functions. -- The AWS Python Driver uses module-level loggers, as recommended in the Python docs. You can configure a specific module's logger by referring to its module path. Here is an example `logging.conf` file that configures logging for the failover plugin: +- The AWS Advanced Python Wrapper uses module-level loggers, as recommended in the Python docs. You can configure a specific module's logger by referring to its module path. Here is an example `logging.conf` file that configures logging for the failover plugin: ``` [loggers] keys=root,failover diff --git a/docs/using-the-python-driver/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md similarity index 85% rename from docs/using-the-python-driver/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md index a108ae0d7..9c0068365 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md @@ -1,7 +1,7 @@ # Aurora Connection Tracker Plugin This plugin tracks all the opened connections. In the event of a cluster failover, this plugin will close all the impacted connections. -This plugin is enabled by default. It can also be explicitly included by adding the plugin code `aurora_connection_tracker` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value. +This plugin is enabled by default. It can also be explicitly included by adding the plugin code `aurora_connection_tracker` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value. ## Use Case User applications can have two types of connections: @@ -9,7 +9,7 @@ User applications can have two types of connections: 1. active connections that are used to execute statements or perform other types of database operations. 2. idle connections that the application holds references to but are not used for any operations. -For instance, a user application has an active connection and an idle connection to host A where host A is a writer instance. The user application executes DML statements against host A when a cluster failover occurs. A different host is promoted as the writer, so host A is now a reader. The driver will failover the active connection to the new writer, but it will not modify the idle connection. +For instance, a user application has an active connection and an idle connection to host A where host A is a writer instance. The user application executes DML statements against host A when a cluster failover occurs. A different host is promoted as the writer, so host A is now a reader. The wrapper will failover the active connection to the new writer, but it will not modify the idle connection. When the application tries to continue the workflow with the idle connection that is still pointing to a host that has changed roles, i.e. host A, users may get an error caused by unexpected behaviour, such as `cannot execute UPDATE in a read-only transaction`. diff --git a/docs/using-the-python-driver/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md similarity index 97% rename from docs/using-the-python-driver/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md index 5b762fa0f..81194b07e 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md @@ -15,7 +15,7 @@ This plugin helps by replacing the out of date endpoint if DNS is updating. ## Enabling the Aurora Initial Connection Strategy Plugin -To enable the Aurora Initial Connection Strategy Plugin, add `initial_connection` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value. +To enable the Aurora Initial Connection Strategy Plugin, add `initial_connection` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value. ## Aurora Initial Connection Strategy Connection Parameters diff --git a/docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md similarity index 89% rename from docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md index d068bd0e6..da905bf0e 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheAwsSecretsManagerPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md @@ -1,15 +1,15 @@ # AWS Secrets Manager Plugin -The AWS Advanced Python Driver supports usage of database credentials stored as secrets in the [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) through the AWS Secrets Manager Connection Plugin. When you create a new connection with this plugin enabled, the plugin will retrieve the secret and the connection will be created with the credentials inside that secret. +The AWS Advanced Python Wrapper supports usage of database credentials stored as secrets in the [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) through the AWS Secrets Manager Connection Plugin. When you create a new connection with this plugin enabled, the plugin will retrieve the secret and the connection will be created with the credentials inside that secret. ## Enabling the AWS Secrets Manager Connection Plugin > [!WARNING]\ -> To use this plugin, you must install [`boto3`](https://aws.amazon.com/sdk-for-python/) in your project. These parameters are required for the AWS Advanced Python Driver to pass database credentials to the underlying driver. +> To use this plugin, you must install [`boto3`](https://aws.amazon.com/sdk-for-python/) in your project. These parameters are required for the AWS Advanced Python Wrapper to pass database credentials to the underlying driver. > You can install it via `pip install boto3`. The AWS Secret Manager plugin requires authentication via AWS Credentials. These credentials can be defined in `~/.aws/credentials` or set as environment variables. All users must set `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Users who are using temporary security credentials will also need to additionally set `AWS_SESSION_TOKEN`. -To enable the AWS Secrets Manager Connection Plugin, add the plugin code `aws_secrets_manager` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value. +To enable the AWS Secrets Manager Connection Plugin, add the plugin code `aws_secrets_manager` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value. ## AWS Secrets Manager Connection Plugin Parameters The following properties are required for the AWS Secrets Manager Connection Plugin to retrieve database credentials from the AWS Secrets Manager. @@ -48,7 +48,7 @@ awsconn = AwsWrapperConnection.connect( ) ``` -If you specify a secret ARN as the `secrets_manager_secret_id`, the AWS Advanced Python Driver will parse the region from the ARN and set it as the `secrets_manager_region` value. +If you specify a secret ARN as the `secrets_manager_secret_id`, the AWS Advanced Python Wrapper will parse the region from the ARN and set it as the `secrets_manager_region` value. ```python awsconn = AwsWrapperConnection.connect( psycopg.Connection.connect, @@ -59,7 +59,7 @@ awsconn = AwsWrapperConnection.connect( ) ``` -If you specify `secrets_manager_secret_username` and `secrets_manager_secret_password`, the AWS Advanced Python Driver will parse the secret searching for those specified keys. +If you specify `secrets_manager_secret_username` and `secrets_manager_secret_password`, the AWS Advanced Python Wrapper will parse the secret searching for those specified keys. ```python awsconn = AwsWrapperConnection.connect( psycopg.Connection.connect, diff --git a/docs/using-the-python-driver/using-plugins/UsingTheBlueGreenPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md similarity index 90% rename from docs/using-the-python-driver/using-plugins/UsingTheBlueGreenPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md index ce637c524..bcdd69c9a 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheBlueGreenPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md @@ -4,7 +4,7 @@ The [Blue/Green Deployment](https://docs.aws.amazon.com/whitepapers/latest/blue-green-deployments/introduction.html) technique enables organizations to release applications by seamlessly shifting traffic between two identical environments running different versions of the application. This strategy effectively mitigates common risks associated with software deployment, such as downtime and limited rollback capability. -The AWS Python Driver leverages the Blue/Green Deployment approach by intelligently managing traffic distribution between blue and green hosts, minimizing the impact of stale DNS data and connectivity disruptions on user applications. +The AWS Advanced Python Wrapper leverages the Blue/Green Deployment approach by intelligently managing traffic distribution between blue and green hosts, minimizing the impact of stale DNS data and connectivity disruptions on user applications. ## Prerequisites @@ -28,7 +28,7 @@ The AWS Python Driver leverages the Blue/Green Deployment approach by intelligen > > **Blue/Green Support Behaviour and Version Compatibility:** > -> The AWS Advanced Python Driver now includes enhanced full support for Blue/Green Deployments. This support requires a minimum database version that includes a specific metadata table. The metadata will be accessible provided the green deployment satisfies the minimum version compatibility requirements. This constraint **does not** apply to RDS MySQL. +> The AWS Advanced Python Wrapper now includes enhanced full support for Blue/Green Deployments. This support requires a minimum database version that includes a specific metadata table. The metadata will be accessible provided the green deployment satisfies the minimum version compatibility requirements. This constraint **does not** apply to RDS MySQL. > > For RDS Postgres, you will also need to manually install the `rds_tools` extension using the following DDL so that the metadata required by the wrapper is available: > @@ -36,9 +36,9 @@ The AWS Python Driver leverages the Blue/Green Deployment approach by intelligen > CREATE EXTENSION rds_tools; > ``` > -> If your database version does **not** support this table, the driver will automatically detect its absence and fallback to its previous behaviour. In this fallback mode, Blue/Green handling is subject to the same limitations listed above. +> If your database version does **not** support this table, the wrapper will automatically detect its absence and fallback to its previous behaviour. In this fallback mode, Blue/Green handling is subject to the same limitations listed above. > -> **No action is required** if your database does not include the new metadata table -- the driver will continue to operate as before. If you have questions or encounter issues, please open an issue in this repository. +> **No action is required** if your database does not include the new metadata table -- the wrapper will continue to operate as before. If you have questions or encounter issues, please open an issue in this repository. > > Supported RDS PostgreSQL Versions: `rds_tools v1.7 (17.1, 16.5, 15.9, 14.14, 13.17, 12.21)` and above.
> Supported Aurora PostgreSQL Versions: Engine Release `17.5, 16.9, 15.13, 14.18, 13.21` and above.
@@ -55,7 +55,7 @@ During a [Blue/Green switchover](https://docs.aws.amazon.com/AmazonRDS/latest/Us - Internal security certificates are regenerated to accommodate the new host names -All factors mentioned above may cause application disruption. The AWS Advanced Python Driver aims to minimize application disruption during Blue/Green switchover by performing the following actions: +All factors mentioned above may cause application disruption. The AWS Advanced Python Wrapper aims to minimize application disruption during Blue/Green switchover by performing the following actions: - Actively monitors Blue/Green switchover status and implements appropriate measures to suspend, pass-through, or re-route database traffic - Prior to Blue/Green switchover initiation, compiles a comprehensive inventory of cluster and instance endpoints for both blue and green hosts along with their corresponding IP addresses - During the active switchover phase, temporarily suspends database traffic to connected blue hosts, which helps unload database hosts and reduces transaction lag for green hosts, thereby enhancing overall switchover performance @@ -65,9 +65,9 @@ All factors mentioned above may cause application disruption. The AWS Advanced P - Intelligently detects switchover failures and rollbacks to the original state, implementing appropriate connection handling measures to maintain application stability -## How do I use the Blue/Green Deployment Plugin with the AWS Python Driver? +## How do I use the Blue/Green Deployment Plugin with the AWS Python Wrapper? -To enable the Blue/Green Deployment functionality, add the plugin code `bg` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter value. +To enable the Blue/Green Deployment functionality, add the plugin code `bg` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter value. The Blue/Green Deployment Plugin supports the following configuration parameters: | Parameter | Value | Required | Description | Example Value | Default Value | @@ -77,7 +77,7 @@ The Blue/Green Deployment Plugin supports the following configuration parameters | `bg_interval_baseline_ms` | Integer | No | The baseline interval (ms) for checking the Blue/Green Deployment status. It's highly recommended to keep this parameter below 900000ms (15 minutes). | `60000` | `60000` | | `bg_interval_increased_ms` | Integer | No | The increased-frequency interval (ms) for checking the Blue/Green Deployment status. Configure this parameter within the range of 500-2000 milliseconds. | `1000` | `1000` | | `bg_interval_high_ms` | Integer | No | The high-frequency interval (ms) for checking the Blue/Green Deployment status. Configure this parameter within the range of 50-500 milliseconds. | `100` | `100` | -| `bg_switchover_timeout_ms` | Integer | No | Maximum duration (in milliseconds) allowed for switchover completion. If the switchover process stalls or exceeds this timeframe, the driver will automatically assume completion and resume normal operations. | `180000` | `180000` | +| `bg_switchover_timeout_ms` | Integer | No | Maximum duration (in milliseconds) allowed for switchover completion. If the switchover process stalls or exceeds this timeframe, the wrapper will automatically assume completion and resume normal operations. | `180000` | `180000` | | `bg_suspend_new_blue_connections` | Boolean | No | Enables Blue/Green Deployment switchover to suspend new blue connection requests while the switchover process is in progress. | `false` | `false` | The plugin establishes dedicated monitoring connections to track Blue/Green Deployment status. To apply specific configurations to these monitoring connections, add the `blue-green-monitoring-` prefix to any configuration parameter, as shown in the following example: @@ -107,7 +107,7 @@ props["blue-green-monitoring-connect_timeout"] = 10 ## Plan your Blue/Green switchover in advance -To optimize Blue/Green switchover support with the AWS Python Driver, advance planning is essential. Please follow these recommended steps: +To optimize Blue/Green switchover support with the AWS Advanced Python Wrapper, advance planning is essential. Please follow these recommended steps: 1. Create a Blue/Green Deployment for your database. 2. Configure your application by incorporating the `bg` plugin along with any additional parameters of your choice, then deploy your application to the corresponding environment. diff --git a/docs/using-the-python-driver/using-plugins/UsingTheCustomEndpointPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheCustomEndpointPlugin.md similarity index 89% rename from docs/using-the-python-driver/using-plugins/UsingTheCustomEndpointPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheCustomEndpointPlugin.md index 2b656e734..724ceb6de 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheCustomEndpointPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheCustomEndpointPlugin.md @@ -1,17 +1,17 @@ # Custom Endpoint Plugin -The Custom Endpoint Plugin adds support for RDS custom endpoints. When the Custom Endpoint Plugin is in use, the driver will analyse custom endpoint information to ensure instances used in connections are part of the custom endpoint being used. This includes connections used in failover and read-write splitting. +The Custom Endpoint Plugin adds support for RDS custom endpoints. When the Custom Endpoint Plugin is in use, the wrapper will analyse custom endpoint information to ensure instances used in connections are part of the custom endpoint being used. This includes connections used in failover and read-write splitting. ## Prerequisites - This plugin requires the AWS SDK for Python, [Boto3](https://pypi.org/project/boto3/). Boto3 is a runtime dependency and must be resolved. It can be installed via pip like so: `pip install boto3`. -## How to use the Custom Endpoint Plugin with the AWS Advanced Python Driver +## How to use the Custom Endpoint Plugin with the AWS Advanced Python Wrapper ### Enabling the Custom Endpoint Plugin 1. If needed, create a custom endpoint using the AWS RDS Console: - If needed, review the documentation about [creating a custom endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-custom-endpoint-creating.html). -2. Add the plugin code `custom_endpoint` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter value, or to the current [driver profile](../UsingThePythonDriver.md#connection-plugin-manager-parameters). +2. Add the plugin code `custom_endpoint` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter value, or to the current [driver profile](../UsingThePythonWrapper.md#connection-plugin-manager-parameters). 3. If you are using the failover plugin, set the failover parameter `failover_mode` according to the custom endpoint type. For example, if the custom endpoint you are using is of type `READER`, you can set `failover_mode` to `strict_reader`, or if it is of type `ANY`, you can set `failover_mode` to `reader_or_writer`. 4. Specify parameters that are required or specific to your case. diff --git a/docs/using-the-python-driver/using-plugins/UsingTheDeveloperPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheDeveloperPlugin.md similarity index 100% rename from docs/using-the-python-driver/using-plugins/UsingTheDeveloperPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheDeveloperPlugin.md diff --git a/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md new file mode 100644 index 000000000..e3ae6bd87 --- /dev/null +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md @@ -0,0 +1,75 @@ +# Failover Plugin v2 +The AWS Advanced Python Wrapper uses the Failover Plugin v2 to provide minimal downtime in the event of a DB instance failure. The plugin is the next version (v2) of the [Failover Plugin](./UsingTheFailoverPlugin.md) and unless explicitly stated otherwise, most of the information and suggestions for the Failover Plugin are applicable to the Failover Plugin v2. + +## Differences between the Failover Plugin and the Failover Plugin v2 + +The Failover Plugin performs a failover process for each DB connection. Each failover process is triggered independently and is unrelated to failover processes in other connections. While such independence between failover processes has some benefits, it also leads to additional resources like extra threads. If dozens of DB connections are failing over at the same time, it may cause significant load on a client environment. + +
+Picture 1. Each connection triggers its own failover process to detect a new writer. +

+ +If a connection needs to get the latest topology, it calls `RdsHostListProvider`. It should be noted that `RdsHostListProvider` runs in the same thread as a connection failover process. As shown in _Picture 1_ above, different connections start and end their failover processes independently. + +The Failover Plugin v2 uses an optimized approach where the process of detecting and confirming a cluster topology is delegated to a central topology monitoring component that runs in a separate thread. When the topology is confirmed and a new writer is detected, each waiting connection can resume and reconnect to a required host. This design helps minimize resources required for failover processing and scales better compared to the Failover Plugin. + +
+Picture 2. Connections call MonitoringRdsHostListProvider, which is responsible for detecting the new writer. While waiting for MonitoringRdsHostListProvider, connection threads suspend. +

+ +If two connections encounter communication issues with their internal (physical) DB connections, each connection may send a request to the topology monitoring component (`MonitoringRdsHostListProvider` in _Picture 2_) for updated topology information reflecting the new writer. Both connections are notified as soon as the latest topology is available. Connection threads can resume, continue with their suspended workflows, and reconnect to a reader or a writer host as needed. + +The topology monitoring component mentioned above (`MonitoringRdsHostListProvider`) updates topology periodically. Usually it uses a connection to a writer host to fetch a cluster topology. Using a connection to a writer host gathers topology first hand without a risk of getting stale data as is the case of fetching topology from a reader. In some exceptional cases the monitoring component may (temporarily) use a reader connection to fetch topology however it will switch back to a writer host as soon as possible. + +
+Picture 3. MonitoringRdsHostListProvider detects a new writer by establishing connections to hosts in separate threads. +

+ +When the cluster topology needs to be confirmed, the monitoring component opens new threads, one for each host (see _Picture 3_). Each of these threads tries to connect to a host and checks if the host is a writer. When Aurora failover occurs, the new writer host is the first host to reflect the true topology of the cluster. Other hosts connect to the new writer shortly after and update their local copies of the topology. Topology information acquired from a reader host may be outdated/inaccurate for a short period after failover. You can see a typical example of stale topology in the diagram above: thread `instance-3`, box `Topology`, to the right. The stale topology incorrectly shows that `instance-3` is still a writer. + +The threads monitoring the topology stop when a new writer is detected. For 30 seconds after a new writer is detected (and after all waiting connections have been notified), topology continues to be updated at an increased rate. This allows time for all readers to appear in the topology, since 30 seconds is usually enough time for cluster failover to complete and cluster topology to stabilize. + +All improvements mentioned above help the Failover Plugin v2 to operate with improved performance and less demand for resources. + +A summary of the key differences and between the `failover` and `failover_v2` plugins is outlined below. +With the `failover` plugin: +- Each connection performs its own failover process. +- Each connection fetches topology by calling the `RdsHostListProvider` in the same thread. +- Topology may be fetched from a reader host and it may be stale. + +With the `failover_v2` plugin: +- Each connection delegates detection of the new writer to the `MonitoringRdsHostListProvider` (which runs in its own thread) and suspends until the new writer is confirmed. +- The `MonitoringRdsHostListProvider` tries to connect to every cluster host in parallel. +- The `MonitoringRdsHostListProvider` uses an "Am I a writer?" approach to avoid reliance on stale topology. +- The `MonitoringRdsHostListProvider` continues topology monitoring at an increased rate to ensure all cluster hosts appear in the topology. + +## Using the Failover Plugin v2 +The Failover Plugin, not the Failover Plugin v2, will be enabled by default if the [`wrapperPlugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value is not specified. If you would like to override the default plugins, you can explicitly include the failover plugin v2 in your list of plugins by adding the plugin code `failover_v2` to the [`wrapperPlugins`](../UsingThePythonWrapper.md#aws-advanced-Python-wrapper-parameters) value, or by adding it to the current [driver profile](../UsingThePythonWrapper.md#connection-plugin-manager-parameters). After you load the plugin, the failover v2 feature will be enabled. + +Please refer to the [failover configuration guide](../FailoverConfigurationGuide.md) for tips to keep in mind when using the failover plugins. + +> [!WARNING] +> Do not use the `failover` and `failover_v2` plugins at the same time for the same connection! + +### Failover Plugin v2 Configuration Parameters +In addition to the parameters that you can configure for the underlying driver, you can pass the following parameters for the AWS Advanced Python Wrapper to specify additional failover behavior. + +| Parameter | Value | Required | Description | Default Value | +|------------------------------------------|:-------:|:--------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `failover_mode` | String | No | Defines a mode for failover process. Failover process may prioritize hosts with different roles and connect to them. Possible values:

- `strict-writer` - Failover process follows writer host and connects to a new writer when it changes.
- `reader-or-writer` - During failover, the wrapper tries to connect to any available/accessible reader host. If no reader is available, the wrapper will connect to a writer host. This logic mimics the logic of the Aurora read-only cluster endpoint.
- `strict-reader` - During failover, the wrapper tries to connect to any available reader host. If no reader is available, the wrapper raises an error. Reader failover to a writer host will only be allowed for single-host clusters. This logic mimics the logic of the Aurora read-only cluster endpoint. | Default value depends on connection url. For Aurora read-only cluster endpoint, it's set to `reader-or-writer`. Otherwise, it's `strict-writer`. | +| `cluster_instance_host_pattern` | String | If connecting using an IP address or custom domain URL: Yes

Otherwise: No | This parameter is not required unless connecting to an AWS RDS cluster via an IP address or custom domain URL. In those cases, this parameter specifies the cluster instance DNS pattern that will be used to build a complete instance endpoint. A "?" character in this pattern should be used as a placeholder for the DB instance identifiers of the instances in the cluster. See [here](./UsingTheFailoverPlugin.md#host-pattern) for more information.

Example: `?.my-domain.com`, `any-subdomain.?.my-domain.com:9999`

Use case Example: If your cluster instance endpoints follow this pattern:`instanceIdentifier1.customHost`, `instanceIdentifier2.customHost`, etc. and you want your initial connection to be to `customHost:1234`, then your connection parameters should look like this: `host=customHost:1234 cluster_instance_host_pattern=?.customHost` | If the provided connection string is not an IP address or custom domain, the AWS Advanced Python Wrapper will automatically acquire the cluster instance host pattern from the customer-provided connection string. | +| `failover_timeout_sec` | Integer | No | Maximum allowed time in seconds to attempt reconnecting to a new writer or reader instance after a cluster failover is initiated. | `300` | +| `topology_refresh_ms` | Integer | No | Cluster topology refresh rate in seconds when a cluster is not in failover. It refers to the regular, slow monitoring rate explained above. | `30000` | +| `cluster_topology_high_refresh_rate_ms` | Integer | No | Interval of time in milliseconds to wait between attempts to update cluster topology after the writer has come back online following a failover event. It corresponds to the increased monitoring rate described earlier. Usually, the topology monitoring component uses this increased monitoring rate for 30s after a new writer was detected. | `100` | +| `failover_reader_host_selector_strategy` | String | No | Strategy used to select a reader host during failover. For more information on the available reader selection strategies, see this [table](../ReaderSelectionStrategies.md). | `random` | +| `enable_connect_failover` | Boolean | No | Enable/disable cluster-aware failover if the initial connection to the database fails due to a network exception. Note that this may result in a connection to a different instance in the cluster than was specified by the URL. | `False` | + + +Please refer to the original [Failover Plugin](./UsingTheFailoverPlugin.md) for more details about error codes, configurations, connection pooling and sample codes. + +### Sample Code +[PostgreSQL Failover Sample Code](../../examples/PGFailover.py) +[MySQL Failover Sample Code](../../examples/MySQLFailover.py) + +This sample code uses the original `failover` plugin, but it can also be used with the `failover_v2` plugin. Configuration parameters should be adjusted in accordance with the table above. + diff --git a/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md similarity index 67% rename from docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md index aa4d90aa9..b0eb3d2ef 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md @@ -1,34 +1,34 @@ # Failover Plugin -In an Amazon Aurora database (DB) cluster, failover is a mechanism by which Aurora automatically repairs the DB cluster status when a primary DB instance becomes unavailable. It achieves this goal by electing an Aurora Reader to become the new primary DB instance, so that the DB cluster can provide maximum availability to a primary read-write DB instance. The AWS Advanced Python Driver uses the Failover Plugin to coordinate with this behavior in order to provide minimal downtime in the event of a DB instance failure. +In an Amazon Aurora database (DB) cluster, failover is a mechanism by which Aurora automatically repairs the DB cluster status when a primary DB instance becomes unavailable. It achieves this goal by electing an Aurora Reader to become the new primary DB instance, so that the DB cluster can provide maximum availability to a primary read-write DB instance. The AWS Advanced Python Wrapper uses the Failover Plugin to coordinate with this behavior in order to provide minimal downtime in the event of a DB instance failure. -## The AWS Advanced Python Driver Failover Process +## The AWS Advanced Python Wrapper Failover Process
-The figure above provides a simplified overview of how the AWS Advanced Python Driver handles an Aurora failover encounter. Starting at the top of the diagram, an application uses the AWS Advanced Python Driver to get a logical connection to an Aurora database. +The figure above provides a simplified overview of how the AWS Advanced Python Wrapper handles an Aurora failover encounter. Starting at the top of the diagram, an application uses the AWS Advanced Python Wrapper to get a logical connection to an Aurora database. In this example, the application requests a connection using the Aurora DB cluster endpoint and is returned with a logical connection that is physically connected to the primary DB instance in the DB cluster, DB instance C. By design, details about which specific DB instance the physical connection is connected to have been abstracted away. -Over the course of the application's lifetime, it executes various statements against the logical connection. If DB instance C is stable and active, these statements succeed and the application continues as normal. If DB instance C experiences a failure, Aurora will initiate failover to promote a new primary DB instance. At the same time, the AWS Advanced Python Driver will intercept the related communication exception and kick off its own internal failover process. +Over the course of the application's lifetime, it executes various statements against the logical connection. If DB instance C is stable and active, these statements succeed and the application continues as normal. If DB instance C experiences a failure, Aurora will initiate failover to promote a new primary DB instance. At the same time, the AWS Advanced Python Wrapper will intercept the related communication exception and kick off its own internal failover process. -If the primary DB instance has failed, the AWS Advanced Python Driver attempts to use its internal topology cache to temporarily connect to an active Aurora Reader. This Aurora Reader will be periodically queried for the DB cluster topology until the new primary DB instance is identified (DB instance A or B in this case). If the driver is unable to connect to an active Aurora Reader or the cluster is still being reconfigured, the driver will retry the connection until it is successful. +If the primary DB instance has failed, the AWS Advanced Python Wrapper attempts to use its internal topology cache to temporarily connect to an active Aurora Reader. This Aurora Reader will be periodically queried for the DB cluster topology until the new primary DB instance is identified (DB instance A or B in this case). If the wrapper is unable to connect to an active Aurora Reader or the cluster is still being reconfigured, the wrapper will retry the connection until it is successful. -At this point, the Python Driver will connect to the new primary DB instance and return control to the application by raising a FailoverSuccessError so you can reconfigure the session state as needed. Although the DNS endpoint for the DB cluster might not yet resolve to the new primary DB instance, the AWS Advanced Python Driver has already discovered this new DB instance during its failover process, and will be directly connected to it when the application continues executing statements. In this way the AWS Advanced Python Driver provides a faster way to reconnect to a newly promoted DB instance, thus increasing the availability of the DB cluster. +At this point, the AWS Advanced Python Wrapper will connect to the new primary DB instance and return control to the application by raising a FailoverSuccessError so you can reconfigure the session state as needed. Although the DNS endpoint for the DB cluster might not yet resolve to the new primary DB instance, the AWS Advanced Python Wrapper has already discovered this new DB instance during its failover process, and will be directly connected to it when the application continues executing statements. In this way the AWS Advanced Python Wrapper provides a faster way to reconnect to a newly promoted DB instance, thus increasing the availability of the DB cluster. ## Using the Failover Plugin -The failover plugin will be loaded by default if the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter is not specified. The failover plugin can also be explicitly loaded by adding the plugin code `failover` to the [`plugins`](../UsingThePythonDriver.md#aws-advanced-python-driver-parameters) parameter. After you load the plugin, the failover feature will be enabled by default and the `enable_failover` parameter will be set to True.

Please refer to the [failover configuration guide](../FailoverConfigurationGuide.md) for tips to keep in mind when using the failover plugin. +The failover plugin will be loaded by default if the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter is not specified. The failover plugin can also be explicitly loaded by adding the plugin code `failover` to the [`plugins`](../UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) parameter. After you load the plugin, the failover feature will be enabled by default and the `enable_failover` parameter will be set to True.

Please refer to the [failover configuration guide](../FailoverConfigurationGuide.md) for tips to keep in mind when using the failover plugin. ### Failover Parameters -In addition to the parameters that you can configure for the underlying driver, you can pass the following connection parameters to the AWS Advanced Python Driver to specify additional failover behavior. +In addition to the parameters that you can configure for the underlying driver, you can pass the following connection parameters to the AWS Advanced Python Wrapper to specify additional failover behavior. | Parameter | Value | Required | Description | Default Value | |----------------------------------------------|:-------:|:--------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `failover_mode` | String | No | Defines a mode for failover process. Failover process may prioritize hosts with different roles and connect to them. Possible values:

- `strict_writer` - Failover process follows writer host and connects to a new writer when it changes.
- `reader_or_writer` - During failover, the driver tries to connect to any available/accessible reader host. If no reader is available, the driver will connect to a writer host. This logic mimics the logic of the Aurora read-only cluster endpoint.
- `strict_reader` - During failover, the driver tries to connect to any available reader host. If no reader is available, the driver raises an error. Reader failover to a writer host will only be allowed for single-host clusters. This logic mimics the logic of the Aurora read-only cluster endpoint.

If this parameter is omitted, default value depends on connection URL. For Aurora read-only cluster endpoint, it's set to `reader_or_writer`. Otherwise, it's `strict_writer`. | Default value depends on connection URL. For Aurora read-only cluster endpoint, it's set to `reader_or_writer`. Otherwise, it's `strict_writer`. | -| `cluster_instance_host_pattern` | String | If connecting using an IP address or custom domain URL: Yes

Otherwise: No | This parameter is not required unless connecting to an AWS RDS cluster via an IP address or custom domain URL. In those cases, this parameter specifies the cluster instance DNS pattern that will be used to build a complete instance endpoint. A "?" character in this pattern should be used as a placeholder for the DB instance identifiers of the instances in the cluster. See [here](#host-pattern) for more information.

Example: `?.my-domain.com`, `any-subdomain.?.my-domain.com:9999`

Use case Example: If your cluster instance endpoints follow this pattern:`instanceIdentifier1.customHost`, `instanceIdentifier2.customHost`, etc. and you want your initial connection to be to `customHost:1234`, then your connection parameters should look like this: `host=customHost:1234 cluster_instance_host_pattern=?.customHost` | If the provided connection string is not an IP address or custom domain, the AWS Advanced Python Driver will automatically acquire the cluster instance host pattern from the customer-provided connection string. | -| `enable_failover` | Boolean | No | Set to `True` to enable the fast failover behavior offered by the AWS Advanced Python Driver. Set to `False` for simple database connections that do not require fast failover functionality. | `True` | +| `failover_mode` | String | No | Defines a mode for failover process. Failover process may prioritize hosts with different roles and connect to them. Possible values:

- `strict_writer` - Failover process follows writer host and connects to a new writer when it changes.
- `reader_or_writer` - During failover, the wrapper tries to connect to any available/accessible reader host. If no reader is available, the wrapper will connect to a writer host. This logic mimics the logic of the Aurora read-only cluster endpoint.
- `strict_reader` - During failover, the wrapper tries to connect to any available reader host. If no reader is available, the wrapper raises an error. Reader failover to a writer host will only be allowed for single-host clusters. This logic mimics the logic of the Aurora read-only cluster endpoint.

If this parameter is omitted, default value depends on connection URL. For Aurora read-only cluster endpoint, it's set to `reader_or_writer`. Otherwise, it's `strict_writer`. | Default value depends on connection URL. For Aurora read-only cluster endpoint, it's set to `reader_or_writer`. Otherwise, it's `strict_writer`. | +| `cluster_instance_host_pattern` | String | If connecting using an IP address or custom domain URL: Yes

Otherwise: No | This parameter is not required unless connecting to an AWS RDS cluster via an IP address or custom domain URL. In those cases, this parameter specifies the cluster instance DNS pattern that will be used to build a complete instance endpoint. A "?" character in this pattern should be used as a placeholder for the DB instance identifiers of the instances in the cluster. See [here](#host-pattern) for more information.

Example: `?.my-domain.com`, `any-subdomain.?.my-domain.com:9999`

Use case Example: If your cluster instance endpoints follow this pattern:`instanceIdentifier1.customHost`, `instanceIdentifier2.customHost`, etc. and you want your initial connection to be to `customHost:1234`, then your connection parameters should look like this: `host=customHost:1234 cluster_instance_host_pattern=?.customHost` | If the provided connection string is not an IP address or custom domain, the AWS Advanced Python Wrapper will automatically acquire the cluster instance host pattern from the customer-provided connection string. | +| `enable_failover` | Boolean | No | Set to `True` to enable the fast failover behavior offered by the AWS Advanced Python Wrapper. Set to `False` for simple database connections that do not require fast failover functionality. | `True` | | `failover_cluster_topology_refresh_rate_sec` | Integer | No | Cluster topology refresh rate in seconds during a writer failover process. During the writer failover process, cluster topology may be refreshed at a faster pace than normal to speed up discovery of the newly promoted writer. | `2` | | `failover_reader_connect_timeout_sec` | Integer | No | Maximum allowed time in seconds to attempt to connect to a reader instance during a reader failover process. | `30` | | `failover_timeout_sec` | Integer | No | Maximum allowed time in seconds to attempt reconnecting to a new writer or reader instance after a cluster failover is initiated. | `300` | @@ -36,7 +36,7 @@ In addition to the parameters that you can configure for the underlying driver, ## Host Pattern -When connecting to Aurora clusters, the [`cluster_instance_host_pattern`](#failover-parameters) parameter is required if the connection string does not provide enough information about the database cluster domain name. If the Aurora cluster endpoint is used directly, the AWS Advanced Python Driver will recognize the standard Aurora domain name and can re-build a proper Aurora instance name when needed. In cases where the connection string uses an IP address, a custom domain name, or localhost, the driver won't know how to build a proper domain name for a database instance endpoint. For example, if a custom domain was being used and the cluster instance endpoints followed a pattern of `instanceIdentifier1.customHost`, `instanceIdentifier2.customHost`, etc., the driver would need to know how to construct the instance endpoints using the specified custom domain. Since there isn't enough information from the custom domain alone to create the instance endpoints, you should set the `cluster_instance_host_pattern` to `?.customHost`, so that the connection parameters include `host=customHost cluster_instance_host_pattern=?.customHost`. Refer to [this diagram](../../images/failover_behavior.png) about AWS Advanced Python Driver behavior during failover for different connection URLs and more details and examples. +When connecting to Aurora clusters, the [`cluster_instance_host_pattern`](#failover-parameters) parameter is required if the connection string does not provide enough information about the database cluster domain name. If the Aurora cluster endpoint is used directly, the AWS Advanced Python Wrapper will recognize the standard Aurora domain name and can re-build a proper Aurora instance name when needed. In cases where the connection string uses an IP address, a custom domain name, or localhost, the wrapper won't know how to build a proper domain name for a database instance endpoint. For example, if a custom domain was being used and the cluster instance endpoints followed a pattern of `instanceIdentifier1.customHost`, `instanceIdentifier2.customHost`, etc., the wrapper would need to know how to construct the instance endpoints using the specified custom domain. Since there isn't enough information from the custom domain alone to create the instance endpoints, you should set the `cluster_instance_host_pattern` to `?.customHost`, so that the connection parameters include `host=customHost cluster_instance_host_pattern=?.customHost`. Refer to [this diagram](../../images/failover_behavior.png) about AWS Advanced Python Wrapper behavior during failover for different connection URLs and more details and examples. ## Failover Errors @@ -48,11 +48,11 @@ When connecting to Aurora clusters, the [`cluster_instance_host_pattern`](#failo ### FailoverFailedError -When the AWS Advanced Python Driver throws a `FailoverFailedError`, the original connection has failed, and the AWS Advanced Python Driver tried to failover to a new instance, but was unable to. There are various reasons this may happen: no hosts were available, a network failure occurred, and so on. In this scenario, please wait until the server is up or other problems are solved. (Exception will be thrown.) +When the AWS Advanced Python Wrapper throws a `FailoverFailedError`, the original connection has failed, and the AWS Advanced Python Wrapper tried to failover to a new instance, but was unable to. There are various reasons this may happen: no hosts were available, a network failure occurred, and so on. In this scenario, please wait until the server is up or other problems are solved. (Exception will be thrown.) ### FailoverSuccessError -When the AWS Advanced Python Driver raises a `FailoverSuccessError`, the original connection has failed while outside a transaction, and the AWS Advanced Python Driver successfully failed over to another available instance in the cluster. However, any session state configuration of the initial connection is now lost. In this scenario, you should: +When the AWS Advanced Python Wrapper raises a `FailoverSuccessError`, the original connection has failed while outside a transaction, and the AWS Advanced Python Wrapper successfully failed over to another available instance in the cluster. However, any session state configuration of the initial connection is now lost. In this scenario, you should: - Reuse and reconfigure the original connection (e.g., reconfigure session state to be the same as the original connection). - Recreate the `Cursor` object. @@ -60,7 +60,7 @@ When the AWS Advanced Python Driver raises a `FailoverSuccessError`, the origina ### TransactionResolutionUnknownError -When the AWS Advanced Python Driver throws a `TransactionResolutionUnknownError`, the original connection has failed within a transaction. In this scenario, the AWS Advanced Python Driver first attempts to rollback the transaction and then fails over to another available instance in the cluster. Note that the rollback might be unsuccessful as the initial connection may be broken at the time that the AWS Advanced Python Driver recognizes the problem. Note also that any session state configuration of the initial connection is now lost. In this scenario, you should: +When the AWS Advanced Python Wrapper throws a `TransactionResolutionUnknownError`, the original connection has failed within a transaction. In this scenario, the AWS Advanced Python Wrapper first attempts to rollback the transaction and then fails over to another available instance in the cluster. Note that the rollback might be unsuccessful as the initial connection may be broken at the time that the AWS Advanced Python Wrapper recognizes the problem. Note also that any session state configuration of the initial connection is now lost. In this scenario, you should: - Reuse and reconfigure the original connection (e.g: reconfigure session state to be the same as the original connection). - Recreate the `Cursor` object. @@ -73,7 +73,7 @@ When the AWS Advanced Python Driver throws a `TransactionResolutionUnknownError` [MySQL Failover Sample Code](../../examples/MySQLFailover.py) > [!WARNING]\ -> **Warnings About Proper Usage of the AWS Advanced Python Driver** -> 1. A common practice when using Python drivers is to wrap invocations against a Connection object in a try-except block, and dispose of the Connection object if an Exception is hit. If this practice is left unaltered, the application will lose the fast-failover functionality offered by the AWS Advanced Python Driver. When failover occurs, the AWS Advanced Python Driver internally establishes a ready-to-use connection inside the original Connection object before throwing an exception to the user. If this Connection object is disposed of, the newly established connection will be thrown away. The correct practice is to check the exception type for failover errors and reuse the Connection object if the error type indicates successful failover. The [PostgreSQL Failover Sample Code](../../examples/PGFailover.py) demonstrates this practice. See the section about [Failover Errors](#failover-errors) for more details. +> **Warnings About Proper Usage of the AWS Advanced Python Wrapper** +> 1. A common practice when using Python drivers is to wrap invocations against a Connection object in a try-except block, and dispose of the Connection object if an Exception is hit. If this practice is left unaltered, the application will lose the fast-failover functionality offered by the AWS Advanced Python Wrapper. When failover occurs, the AWS Advanced Python Wrapper internally establishes a ready-to-use connection inside the original Connection object before throwing an exception to the user. If this Connection object is disposed of, the newly established connection will be thrown away. The correct practice is to check the exception type for failover errors and reuse the Connection object if the error type indicates successful failover. The [PostgreSQL Failover Sample Code](../../examples/PGFailover.py) demonstrates this practice. See the section about [Failover Errors](#failover-errors) for more details.

-> 2. We highly recommended that you use the cluster and read-only cluster endpoints instead of the direct instance endpoints of your Aurora cluster, unless you are confident in your application's use of instance endpoints. Although the AWS Advanced Python Driver will correctly failover to the new writer instance when using instance endpoints, use of these endpoints is discouraged because individual instances can spontaneously change reader/writer status when failover occurs. the AWS Advanced Python Driver will always connect directly to the instance specified if an instance endpoint is provided, so a write-safe connection cannot be assumed if the application uses instance endpoints. +> 2. We highly recommended that you use the cluster and read-only cluster endpoints instead of the direct instance endpoints of your Aurora cluster, unless you are confident in your application's use of instance endpoints. Although the AWS Advanced Python Wrapper will correctly failover to the new writer instance when using instance endpoints, use of these endpoints is discouraged because individual instances can spontaneously change reader/writer status when failover occurs. the AWS Advanced Python Wrapper will always connect directly to the instance specified if an instance endpoint is provided, so a write-safe connection cannot be assumed if the application uses instance endpoints. diff --git a/docs/using-the-python-driver/using-plugins/UsingTheFastestResponseStrategyPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheFastestResponseStrategyPlugin.md similarity index 92% rename from docs/using-the-python-driver/using-plugins/UsingTheFastestResponseStrategyPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheFastestResponseStrategyPlugin.md index 55283968b..9106971a7 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheFastestResponseStrategyPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheFastestResponseStrategyPlugin.md @@ -5,7 +5,7 @@ The host response time is measured at an interval set by `response_measurement_i ## Using the Fastest Response Strategy Plugin -The plugin can be loaded by adding the plugin code `fastest_response_strategy` to the [`plugins`](../UsingThePythonDriver.md#aws-advanced-python-driver-parameters) parameter. The Fastest Response Strategy Plugin is not loaded by default, and must be loaded along with the [`read_write_splitting`](./UsingTheReadWriteSplittingPlugin.md) plugin. +The plugin can be loaded by adding the plugin code `fastest_response_strategy` to the [`plugins`](../UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) parameter. The Fastest Response Strategy Plugin is not loaded by default, and must be loaded along with the [`read_write_splitting`](./UsingTheReadWriteSplittingPlugin.md) plugin. > [!IMPORTANT]\ > **`reader_response_strategy` must be set to `fastest_reponse` when using this plugin. Otherwise an error will be thrown:** diff --git a/docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md similarity index 98% rename from docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md index 1419f4b68..9156b1451 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheFederatedAuthenticationPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md @@ -9,7 +9,7 @@ Federated Identity allows users to use the same set of credentials to access mul When a user wants access to a resource, it authenticates with the IdP. From this a security token generated and is passed to the SP then grants access to said resource. In the case of AD FS, the user signs into the AD FS sign in page. This generates a SAML Assertion which acts as a security token. The user then passes the SAML Assertion to the SP when requesting access to resources. The SP verifies the SAML Assertion and grants access to the user. -## How to use the Federated Authentication Plugin with the AWS Advanced Python Driver +## How to use the Federated Authentication Plugin with the AWS Advanced Python Wrapper ### Enabling the Federated Authentication Plugin > [!NOTE]\ @@ -21,7 +21,7 @@ In the case of AD FS, the user signs into the AD FS sign in page. This generates - If needed, review the documentation about [IAM authentication for MariaDB, MySQL, and PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html). 2. Set up an IAM Identity Provider and IAM role. The IAM role should be using the IAM policy set up in step 1. - If needed, review the documentation about [creating IAM identity providers](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create.html). For AD FS, see the documentation about [creating IAM SAML identity providers](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml.html). -3. Add the plugin code `federated_auth` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value, or to the current [driver profile](../UsingThePythonDriver.md#connection-plugin-manager-parameters). +3. Add the plugin code `federated_auth` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value, or to the current [driver profile](../UsingThePythonWrapper.md#connection-plugin-manager-parameters). 4. Specify parameters that are required or specific to your case. ### Federated Authentication Plugin Parameters diff --git a/docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md similarity index 84% rename from docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md index bb4833265..50b828d1d 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheHostMonitoringPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md @@ -10,11 +10,11 @@ The figure that follows shows a simplified Enhanced Failure Monitoring (EFM) wor Enhanced Failure Monitoring helps user applications detect failures earlier. When a user application executes a query, EFM may detect that the connected database host is unavailable. When this happens, the query is cancelled and the connection will be aborted. This allows queries to fail fast instead of waiting indefinitely or failing due to a timeout. -One use case is to pair EFM with the [Failover Connection Plugin](./UsingTheFailoverPlugin.md). When EFM discovers a database host failure, the connection will be aborted. Without the Failover Connection Plugin, the connection would be terminated up to the user application level. With the Failover Connection Plugin, the AWS Advanced Python Driver can attempt to failover to a different, healthy database host where the query can be executed. +One use case is to pair EFM with the [Failover Connection Plugin](./UsingTheFailoverPlugin.md). When EFM discovers a database host failure, the connection will be aborted. Without the Failover Connection Plugin, the connection would be terminated up to the user application level. With the Failover Connection Plugin, the AWS Advanced Python Wrapper can attempt to failover to a different, healthy database host where the query can be executed. ### Using the Host Monitoring Connection Plugin -The Host Monitoring Connection Plugin will be loaded by default if the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter is not specified. The Host Monitoring Connection Plugin can also be explicitly loaded by adding the plugin code `host_monitoring` to the [`plugins`](../UsingThePythonDriver.md#aws-advanced-python-driver-parameters) parameter. Enhanced Failure Monitoring is enabled by default when the Host Monitoring Connection Plugin is loaded, but it can be disabled by setting the `failure_detection_enabled` parameter to `False`. +The Host Monitoring Connection Plugin will be loaded by default if the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter is not specified. The Host Monitoring Connection Plugin can also be explicitly loaded by adding the plugin code `host_monitoring` to the [`plugins`](../UsingThePythonWrapper.md#aws-advanced-python-wrapper-parameters) parameter. Enhanced Failure Monitoring is enabled by default when the Host Monitoring Connection Plugin is loaded, but it can be disabled by setting the `failure_detection_enabled` parameter to `False`. This plugin only works with drivers that support aborting connections from a separate thread. At this moment, this plugin is incompatible with the MySQL Connector/Python driver. @@ -22,7 +22,7 @@ This plugin only works with drivers that support aborting connections from a sep > The Host Monitoring Plugin creates monitoring threads in the background to monitor all connections established to each cluster instance. The monitoring threads can be cleaned up in two ways: > 1. If there are no connections to the cluster instance the thread is monitoring for over a period of time, the Host Monitoring Plugin will automatically terminate the thread. This period of time is adjustable via the `monitor_disposal_time_ms` parameter. > 2. Client applications can manually call `aws_advanced_python_wrapper.release_resources()` to clean up any dangling resources. -> It is best practice to call `aws_advanced_python_wrapper.release_resources()` at the end of the application to ensure a graceful exit; otherwise, the application may wait until the `monitor_disposal_time_ms` has been passed before terminating. This is because the Python driver waits for all daemon threads to complete before exiting. +> It is best practice to call `aws_advanced_python_wrapper.release_resources()` at the end of the application to ensure a graceful exit; otherwise, the application may wait until the `monitor_disposal_time_ms` has been passed before terminating. This is because the Python wrapper waits for all daemon threads to complete before exiting. > See [PGFailover](../../examples/PGFailover.py) for an example. ### Enhanced Failure Monitoring Parameters @@ -78,10 +78,10 @@ finally: > **If specifying a monitoring- prefixed timeout, always ensure you provide a non-zero timeout value** >[!WARNING]\ -> Warnings About Usage of the AWS Advanced Python Driver with RDS Proxy +> Warnings About Usage of the AWS Advanced Python Wrapper with RDS Proxy > We recommend you either disable the Host Monitoring Connection Plugin or avoid using RDS Proxy endpoints when the Host Monitoring Connection Plugin is active. > -> Although using RDS Proxy endpoints with the AWS Advanced Python Driver with Enhanced Failure Monitoring doesn't cause any critical issues, we don't recommend this approach. The main reason is that RDS Proxy transparently re-routes requests to a single database instance. RDS Proxy decides which database instance is used based on many criteria (on a per-request basis). Switching between different instances makes the Host Monitoring Connection Plugin useless in terms of instance health monitoring because the plugin will be unable to identify which instance it's connected to, and which one it's monitoring. This could result in false positive failure detections. At the same time, the plugin will still proactively monitor network connectivity to RDS Proxy endpoints and report outages back to a user application if they occur. +> Although using RDS Proxy endpoints with the AWS Advanced Python Wrapper with Enhanced Failure Monitoring doesn't cause any critical issues, we don't recommend this approach. The main reason is that RDS Proxy transparently re-routes requests to a single database instance. RDS Proxy decides which database instance is used based on many criteria (on a per-request basis). Switching between different instances makes the Host Monitoring Connection Plugin useless in terms of instance health monitoring because the plugin will be unable to identify which instance it's connected to, and which one it's monitoring. This could result in false positive failure detections. At the same time, the plugin will still proactively monitor network connectivity to RDS Proxy endpoints and report outages back to a user application if they occur. # Host Monitoring Plugin v2 diff --git a/docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md similarity index 87% rename from docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md index e879e2a6a..780ccc479 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheIamAuthenticationPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md @@ -9,15 +9,15 @@ AWS Identity and Access Management (IAM) grants users access control across all The IAM Authentication plugin requires authentication via AWS Credentials. These credentials can be defined in `~/.aws/credentials` or set as environment variables. All users must set `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Users who are using temporary security credentials will also need to additionally set `AWS_SESSION_TOKEN`. -To enable the IAM Authentication Connection Plugin, add the plugin code `iam` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter. +To enable the IAM Authentication Connection Plugin, add the plugin code `iam` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter. ## AWS IAM Database Authentication -The AWS Python Driver supports Amazon AWS Identity and Access Management (IAM) authentication. When using AWS IAM database authentication, the host URL must be a valid Amazon endpoint, and not a custom domain or an IP address. +The AWS Advanced Python Wrapper supports Amazon AWS Identity and Access Management (IAM) authentication. When using AWS IAM database authentication, the host URL must be a valid Amazon endpoint, and not a custom domain or an IP address.
i.e. `db-identifier.cluster-XYZ.us-east-2.rds.amazonaws.com` IAM database authentication use is limited to certain database engines. For more information on limitations and recommendations, please [review the IAM documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html). -## How do I use IAM with the AWS Python Driver? +## How do I use IAM with the AWS Advanced Python Wrapper? 1. Enable AWS IAM database authentication on an existing database or create a new database with AWS IAM database authentication on the AWS RDS Console: 1. If needed, review the documentation about [creating a new database](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateDBInstance.html). 2. If needed, review the documentation about [modifying an existing database](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.DBInstance.Modifying.html). @@ -29,14 +29,14 @@ IAM database authentication use is limited to certain database engines. For more 2. For a PostgreSQL database, use the following command to create a new user:
`CREATE USER db_userx; GRANT rds_iam TO db_userx;` -4. Add the plugin code `iam` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) parameter value. +4. Add the plugin code `iam` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) parameter value. | Parameter | Value | Required | Description | Example Value | |--------------------|:-------:|:--------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------| | `iam_default_port` | String | No | This property will override the default port that is used to generate the IAM token. The default port is determined based on the underlying driver protocol. For now, there is support for PostgreSQL and MySQL. Target drivers with different protocols will require users to provide a default port. | `1234` | | `iam_host` | String | No | This property will override the default hostname that is used to generate the IAM token. The default hostname is derived from the connection string. This parameter is required when users are connecting with custom endpoints. | `database.cluster-hash.us-east-1.rds.amazonaws.com` | | `iam_region` | String | No | This property will override the default region that is used to generate the IAM token. The default region is parsed from the connection string. | `us-east-2` | -| `iam_expiration` | Integer | No | This property determines how long an IAM token is kept in the driver cache before a new one is generated. The default expiration time is set to 14 minutes and 30 seconds. Note that IAM database authentication tokens have a lifetime of 15 minutes. | `600` | +| `iam_expiration` | Integer | No | This property determines how long an IAM token is kept in the wrapper cache before a new one is generated. The default expiration time is set to 14 minutes and 30 seconds. Note that IAM database authentication tokens have a lifetime of 15 minutes. | `600` | ## Sample code diff --git a/docs/using-the-python-driver/using-plugins/UsingTheLimitlessPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheLimitlessPlugin.md similarity index 97% rename from docs/using-the-python-driver/using-plugins/UsingTheLimitlessPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheLimitlessPlugin.md index 48b21f444..b2dbead7e 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheLimitlessPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheLimitlessPlugin.md @@ -3,7 +3,7 @@ ## What is Amazon Aurora Limitless Database? Amazon Aurora Limitless Database is a new type of database that can horizontally scale to handle millions of write transactions per second and manage petabytes of data. -Users will be able to use the AWS Python Driver with Aurora Limitless Databases and optimize their experience using the Limitless Plugin. +Users will be able to use the AWS Advanced Python Wrapper with Aurora Limitless Databases and optimize their experience using the Limitless Plugin. To learn more about Aurora Limitless Database, see the [Amazon Aurora Limitless documentation](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/limitless.html). ## Why use the Limitless Plugin? @@ -17,8 +17,8 @@ The Limitless Plugin achieves this with a monitoring thread that periodically po When a new connection is made, the plugin directs the connection to a transaction router selected from the cache using a weighted random strategy (See [Reader Selection Strategies](../ReaderSelectionStrategies.md) for more details). Routers with a higher load are assigned a lower weight, and routers with a lower load are assigned a higher weight. -## How to use the Limitless Plugin with the AWS Python Driver -To enable the Limitless Plugin, add the plugin code `limitless` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value. +## How to use the Limitless Plugin with the AWS Advanced Python Wrapper +To enable the Limitless Plugin, add the plugin code `limitless` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value. The URL used to connect to a limitless database is the DB shard group URL. diff --git a/docs/using-the-python-driver/using-plugins/UsingTheOktaAuthenticationPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheOktaAuthenticationPlugin.md similarity index 98% rename from docs/using-the-python-driver/using-plugins/UsingTheOktaAuthenticationPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheOktaAuthenticationPlugin.md index 59429ae39..a6d7f105e 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheOktaAuthenticationPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheOktaAuthenticationPlugin.md @@ -8,7 +8,7 @@ Federated Identity allows users to use the same set of credentials to access mul When a user wants access to a resource, it authenticates with the IdP. From this a security token generated and is passed to the SP then grants access to said resource. In the case of AD FS, the user signs into the AD FS sign in page. This generates a SAML Assertion which acts as a security token. The user then passes the SAML Assertion to the SP when requesting access to resources. The SP verifies the SAML Assertion and grants access to the user. -## How to use the Okta Authentication Plugin with the AWS Advanced Python Driver +## How to use the Okta Authentication Plugin with the AWS Advanced Python Wrapper ### Enabling the Okta Authentication Plugin > [!NOTE]\ @@ -20,7 +20,7 @@ In the case of AD FS, the user signs into the AD FS sign in page. This generates - If needed, review the documentation about [IAM authentication for MariaDB, MySQL, and PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html). 2. Configure Okta as the AWS identity provider. - If needed, review the documentation about [Amazon Web Services Account Federation](https://help.okta.com/en-us/content/topics/deploymentguides/aws/aws-deployment.htm) on Okta's documentation. -3. Add the plugin code `okta` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value, or to the current [driver profile](../UsingThePythonDriver.md#connection-plugin-manager-parameters). +3. Add the plugin code `okta` to the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) value, or to the current [driver profile](../UsingThePythonWrapper.md#connection-plugin-manager-parameters). 4. Specify parameters that are required or specific to your case. ### Federated Authentication Plugin Parameters diff --git a/docs/using-the-python-driver/using-plugins/UsingTheReadWriteSplittingPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md similarity index 91% rename from docs/using-the-python-driver/using-plugins/UsingTheReadWriteSplittingPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md index 9d1bdc083..37fd8cead 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheReadWriteSplittingPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md @@ -4,7 +4,7 @@ The Read/Write Splitting Plugin adds functionality to switch between writer and ### Loading the Read/Write Splitting Plugin -The Read/Write Splitting Plugin is not loaded by default. To load the plugin, include `read_write_splitting` in the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) connection parameter: +The Read/Write Splitting Plugin is not loaded by default. To load the plugin, include `read_write_splitting` in the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) connection parameter: ```python params = { @@ -33,21 +33,21 @@ The Read/Write Splitting Plugin is not currently supported for non-Aurora cluste ### Internal connection pooling > [!WARNING]\ -> If internal connection pools are enabled, database passwords may not be verified with every connection request. The initial connection request for each database instance in the cluster will verify the password, but subsequent requests may return a cached pool connection without re-verifying the password. This behavior is inherent to the nature of connection pools and not a bug with the driver. `ConnectionProviderManager.release_resources` can be called to close all pools and remove all cached pool connections. Take a look at the [Postgres](../../examples/PGInternalConnectionPoolPasswordWarning.py) or [MySQL](../../examples/MySQLInternalConnectionPoolPasswordWarning.py) sample code for more details. +> If internal connection pools are enabled, database passwords may not be verified with every connection request. The initial connection request for each database instance in the cluster will verify the password, but subsequent requests may return a cached pool connection without re-verifying the password. This behavior is inherent to the nature of connection pools and not a bug with the wrapper. `ConnectionProviderManager.release_resources` can be called to close all pools and remove all cached pool connections. Take a look at the [Postgres](../../examples/PGInternalConnectionPoolPasswordWarning.py) or [MySQL](../../examples/MySQLInternalConnectionPoolPasswordWarning.py) sample code for more details. -When `read_only` is first set on an `AwsWrapperConnection` object, the Read/Write Splitting Plugin will internally open a physical connection to a reader. The reader connection will then be cached for the given `AwsWrapperConnection`. Future `read_only` settings for the same `AwsWrapperConnection` will not open a new physical connection. However, other `AwsWrapperConnection` objects will need to open their own reader connections when `read_only` is first set. If your application frequently sets `read_only` on many `AwsWrapperConnection` objects, you can enable internal connection pooling to improve performance. When enabled, the AWS Advanced Python Driver will maintain an internal connection pool for each instance in the cluster. This allows the Read/Write Splitting Plugin to reuse connections that were established by previous `AwsWrapperConnection` objects. +When `read_only` is first set on an `AwsWrapperConnection` object, the Read/Write Splitting Plugin will internally open a physical connection to a reader. The reader connection will then be cached for the given `AwsWrapperConnection`. Future `read_only` settings for the same `AwsWrapperConnection` will not open a new physical connection. However, other `AwsWrapperConnection` objects will need to open their own reader connections when `read_only` is first set. If your application frequently sets `read_only` on many `AwsWrapperConnection` objects, you can enable internal connection pooling to improve performance. When enabled, the AWS Advanced Python Wrapper will maintain an internal connection pool for each instance in the cluster. This allows the Read/Write Splitting Plugin to reuse connections that were established by previous `AwsWrapperConnection` objects. > [!NOTE]\ > Initial connections to a cluster URL will not be pooled because it can be problematic to pool a URL that resolves to different instances over time. The main benefit of internal connection pools occurs when `read_only` is set. When `read_only` is set, an internal pool will be created for the instance that the plugin switches to, regardless of the initial connection URL. Connections for that instance can then be reused in the future. -The AWS Advanced Python Driver currently uses [SqlAlchemy](https://docs.sqlalchemy.org/en/20/core/pooling.html) to create and maintain its internal connection pools. The [Postgres](../../examples/PGReadWriteSplitting.py) or [MySQL](../../examples/MySQLReadWriteSplitting.py) sample code provides a useful example of how to enable this feature. The steps are as follows: +The AWS Advanced Python Wrapper currently uses [SqlAlchemy](https://docs.sqlalchemy.org/en/20/core/pooling.html) to create and maintain its internal connection pools. The [Postgres](../../examples/PGReadWriteSplitting.py) or [MySQL](../../examples/MySQLReadWriteSplitting.py) sample code provides a useful example of how to enable this feature. The steps are as follows: 1. Create an instance of `SqlAlchemyPooledConnectionProvider`, passing in any optional arguments if desired: - - The first optional argument takes in a `Callable` that can be used to return custom parameter settings for the connection pool. See [here](https://docs.sqlalchemy.org/en/20/core/pooling.html#sqlalchemy.pool.QueuePool) for a list of available parameters. Note that you should not set the `creator` parameter - this parameter will be set automatically by the AWS Advanced Python Driver. This is done to follow desired behavior and ensure that the Read/Write Splitting Plugin can successfully establish connections to new instances. + - The first optional argument takes in a `Callable` that can be used to return custom parameter settings for the connection pool. See [here](https://docs.sqlalchemy.org/en/20/core/pooling.html#sqlalchemy.pool.QueuePool) for a list of available parameters. Note that you should not set the `creator` parameter - this parameter will be set automatically by the AWS Advanced Python Wrapper. This is done to follow desired behavior and ensure that the Read/Write Splitting Plugin can successfully establish connections to new instances. - The second optional argument takes in a `Callable` that can be used to define custom key generation for the internal connection pools. Key generation is used to define when new connection pools are created; a new pool will be created each time a connection is requested with a unique key. By default, a new pool will be created for each unique instance-user combination. If you would like to define a different key system, you should pass in a `Callable` defining this logic. The `Callable` should take in a `HostInfo` and `Dict` specifying the connection properties and return a string representing the desired connection pool key. A simple example is shown below. > [!WARNING]\ - > If you do not include the username in the connection pool key, connection pools may be shared between different users. As a result, an initial connection established with a privileged user may be returned to a connection request with a lower-privilege user without re-verifying credentials. This behavior is inherent to the nature of connection pools and not a bug with the driver. `ConnectionProviderManager.release_resources` can be called to close all pools and remove all cached pool connections. + > If you do not include the username in the connection pool key, connection pools may be shared between different users. As a result, an initial connection established with a privileged user may be returned to a connection request with a lower-privilege user without re-verifying credentials. This behavior is inherent to the nature of connection pools and not a bug with the wrapper. `ConnectionProviderManager.release_resources` can be called to close all pools and remove all cached pool connections. ```python def get_pool_key(host_info: HostInfo, props: Dict[str, Any]): @@ -73,7 +73,7 @@ The AWS Advanced Python Driver currently uses [SqlAlchemy](https://docs.sqlalche 5. When you are finished using all connections, call `ConnectionProviderManager.release_resources()`. > [!IMPORTANT]\ -> You must call `ConnectionProviderManager.release_resources` to close the internal connection pools when you are finished using all connections. Unless `ConnectionProviderManager.release_resources` is called, the AWS Advanced Python Driver will keep the pools open so that they can be shared between connections. +> You must call `ConnectionProviderManager.release_resources` to close the internal connection pools when you are finished using all connections. Unless `ConnectionProviderManager.release_resources` is called, the AWS Advanced Python Wrapper will keep the pools open so that they can be shared between connections. ### Sample Code [Postgres read/write splitting sample code](../../examples/PGReadWriteSplitting.py) diff --git a/docs/using-the-python-driver/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md similarity index 91% rename from docs/using-the-python-driver/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md rename to docs/using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md index fdf919c2c..a2b264dea 100644 --- a/docs/using-the-python-driver/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md @@ -8,7 +8,7 @@ The plugin does not rely on cluster topology. It relies purely on the provided e ## Loading the Simple Read/Write Splitting Plugin -The Simple Read/Write Splitting Plugin is not loaded by default. To load the plugin, include `srw` in the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) connection parameter: +The Simple Read/Write Splitting Plugin is not loaded by default. To load the plugin, include `srw` in the [`plugins`](../UsingThePythonWrapper.md#connection-plugin-manager-parameters) connection parameter: ```python params = { @@ -54,7 +54,7 @@ When connecting with custom endpoints and other non-standard URLs, role verifica ## Limitations When Verifying Connections #### Non-RDS clusters -The verification step determines the role of the connection by executing a query against it. The AWS Advanced Python Driver does not support gathering such information for databases that are not Aurora or RDS clusters. Thus, when connecting to non-RDS clusters `verifyNewSrwConnections` must be set to `false`. +The verification step determines the role of the connection by executing a query against it. The AWS Advanced Python Wrapper does not support gathering such information for databases that are not Aurora or RDS clusters. Thus, when connecting to non-RDS clusters `verifyNewSrwConnections` must be set to `false`. #### Autocommit The verification logic results in errors such as `Cannot change transaction read-only property in the middle of a transaction` from the underlying driver when: @@ -71,10 +71,10 @@ If autocommit is essential to a workflow, either ensure the plugin has connected RDS Proxy provides connection pooling and management that significantly improves application scalability by reducing database connection overhead and enabling thousands of concurrent connections through connection multiplexing. Connecting exclusively through the proxy endpoint ensures consistent connection management, automatic failover handling, and centralized monitoring, while protecting the underlying database from connection exhaustion -and providing a stable abstraction layer that remains consistent even when database topology changes. To take full advantage of the benefits of RDS Proxy, it is recommended to only connect through RDS Proxy endpoints. By providing the read/write endpoint and a read-only endpoint to the Simple Read/Write Splitting Plugin, the AWS Advanced Python Driver will connect using +and providing a stable abstraction layer that remains consistent even when database topology changes. To take full advantage of the benefits of RDS Proxy, it is recommended to only connect through RDS Proxy endpoints. By providing the read/write endpoint and a read-only endpoint to the Simple Read/Write Splitting Plugin, the AWS Advanced Python Wrapper will connect using these endpoints any time setReadOnly is called. -There are limitations with the AWS Advanced Python Driver and RDS Proxy. This is currently intended, by design, since RDS Proxy decides which database instance is used based on many criteria (on a per-request basis). Due to this, functionality like [Failover](./UsingTheFailoverPlugin.md), [Enhanced Failure Monitoring](./UsingTheHostMonitoringPlugin.md), and topology-based [Read/Write Splitting](./UsingTheReadWriteSplittingPlugin.md) is not compatible since the driver relies on cluster topology and RDS Proxy handles this automatically. However, the driver can still be used to handle authentication workflows and to perform the endpoint-based read/write splitting of the Simple Read/Write Splitting Plugin. +There are limitations with the AWS Advanced Python Wrapper and RDS Proxy. This is currently intended, by design, since RDS Proxy decides which database instance is used based on many criteria (on a per-request basis). Due to this, functionality like [Failover](./UsingTheFailoverPlugin.md), [Enhanced Failure Monitoring](./UsingTheHostMonitoringPlugin.md), and topology-based [Read/Write Splitting](./UsingTheReadWriteSplittingPlugin.md) is not compatible since the wrapper relies on cluster topology and RDS Proxy handles this automatically. However, the driver can still be used to handle authentication workflows and to perform the endpoint-based read/write splitting of the Simple Read/Write Splitting Plugin. ## Using the Simple Read/Write Splitting Plugin against non-RDS clusters