diff --git a/doc/api/datasets.rst b/doc/api/datasets.rst index a1ead85a..29b58c96 100644 --- a/doc/api/datasets.rst +++ b/doc/api/datasets.rst @@ -32,6 +32,7 @@ Dataset Generators plm.datasets.make_plr_CCDDHNR2018 plm.datasets.make_plr_turrell2018 plm.datasets.make_lplr_LZZ2020 + plm.datasets.make_plpr_CP2025 plm.datasets.make_pliv_CHS2015 plm.datasets.make_pliv_multiway_cluster_CKMS2021 plm.datasets.make_confounded_plr_data diff --git a/doc/api/dml_models.rst b/doc/api/dml_models.rst index 77e1104a..450eded7 100644 --- a/doc/api/dml_models.rst +++ b/doc/api/dml_models.rst @@ -17,6 +17,7 @@ doubleml.plm DoubleMLPLR DoubleMLLPLR + DoubleMLPLPR DoubleMLPLIV diff --git a/doc/conf.py b/doc/conf.py index b31f4ea0..4699fa26 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -275,7 +275,11 @@ # Valid DOI; Causes 403 Client Error: Forbidden for url:... "https://doi.org/10.3982/ECTA15732", # Valid DOI; Causes 403 Client Error: Forbidden for url:... - "https://doi.org/10.1093/ectj/utab019" + "https://doi.org/10.1093/ectj/utab019", + # Valid DOI; Causes 403 Client Error: Forbidden for url:... + "https://doi.org/10.1093/ectj/utaf011", + # Valid DOI; Causes 403 Client Error: Forbidden for url:... + "https://doi.org/10.2307/1913646" ] # To execute R code via jupyter-execute one needs to install the R kernel for jupyter diff --git a/doc/examples/index.rst b/doc/examples/index.rst index 03f4e713..394134d3 100644 --- a/doc/examples/index.rst +++ b/doc/examples/index.rst @@ -23,6 +23,7 @@ General Examples py_double_ml_apo.ipynb py_double_ml_irm_vs_apo.ipynb py_double_ml_lplr.ipynb + py_double_ml_plpr.ipynb py_double_ml_ssm.ipynb learners/py_optuna.ipynb learners/py_learner.ipynb diff --git a/doc/examples/py_double_ml_plpr.ipynb b/doc/examples/py_double_ml_plpr.ipynb new file mode 100644 index 00000000..17616ee2 --- /dev/null +++ b/doc/examples/py_double_ml_plpr.ipynb @@ -0,0 +1,3342 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Python: Static Panel Models with Fixed Effects" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, we illustrate how the [DoubleML](https://docs.doubleml.org/stable/index.html) package can be used to estimate treatment effects for static panel models with fixed effects in a partially linear panel regression [DoubleMLPLPR](https://docs.doubleml.org/stable/guide/models.html#partially-linear-models-plm) model. The model is based on [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import optuna\n", + "import numpy as np\n", + "import pandas as pd\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sns\n", + "\n", + "from sklearn.base import clone\n", + "from sklearn.preprocessing import StandardScaler\n", + "from sklearn.preprocessing import PolynomialFeatures\n", + "from sklearn.compose import ColumnTransformer\n", + "from sklearn.pipeline import make_pipeline\n", + "from sklearn.base import BaseEstimator, TransformerMixin\n", + "from sklearn.linear_model import LassoCV\n", + "from lightgbm import LGBMRegressor\n", + "\n", + "from doubleml.data import DoubleMLPanelData\n", + "from doubleml.plm.datasets import make_plpr_CP2025\n", + "from doubleml import DoubleMLPLPR\n", + "\n", + "import warnings\n", + "warnings.filterwarnings(\"ignore\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data\n", + "\n", + "We will use the implemented data generating process [make_plpr_CP2025](https://docs.doubleml.org/stable/api/datasets.html#dataset-generators) to generate data similar to the simulation in [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011). For exposition, we use the simple linear `dgp_type=\"dgp1\"`, with 150 units, 10 time periods per unit, and a true treatment effect of `theta=0.5`.\n", + "\n", + "We set `time_type=\"int\"` such that the time variable values will be integers. It's also possible to use `\"float\"` or `\"datetime\"` time variables with [DoubleMLPLPR](https://docs.doubleml.org/stable/api/dml_models.html#doubleml-plm)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idtimeydx1x2x3x4x5x6...x21x22x23x24x25x26x27x28x29x30
011-1.2904790.9083071.710715-1.853675-1.4739071.366514-0.3220242.944020...-1.828362-3.010547-0.840202-3.0851591.169952-0.954107-3.925198-0.779510-0.4307001.004298
112-2.850646-1.316777-0.3250434.178599-1.159857-0.139527-0.230115-0.631976...-0.724172-0.421045-2.012480-2.081784-2.734123-0.879470-2.1412184.598401-4.222797-2.523024
213-4.338502-1.756120-0.8975901.505972-0.9251891.511500-2.2065610.132579...1.766109-2.252858-2.919826-1.974066-0.7738810.244633-1.7275501.6654670.562291-1.553616
314-2.7132360.9348661.9878492.596228-0.220666-0.480717-3.966273-0.911226...0.8561240.727759-0.5015791.0775042.268052-3.8214221.629055-0.220834-1.185091-5.462884
415-5.782997-4.357881-3.0865593.796975-1.539641-2.425617-1.020599-1.666200...2.617215-1.231835-0.8913500.2469812.4896420.319735-2.8103660.5858263.6437490.147147
\n", + "

5 rows × 34 columns

\n", + "
" + ], + "text/plain": [ + " id time y d x1 x2 x3 x4 \\\n", + "0 1 1 -1.290479 0.908307 1.710715 -1.853675 -1.473907 1.366514 \n", + "1 1 2 -2.850646 -1.316777 -0.325043 4.178599 -1.159857 -0.139527 \n", + "2 1 3 -4.338502 -1.756120 -0.897590 1.505972 -0.925189 1.511500 \n", + "3 1 4 -2.713236 0.934866 1.987849 2.596228 -0.220666 -0.480717 \n", + "4 1 5 -5.782997 -4.357881 -3.086559 3.796975 -1.539641 -2.425617 \n", + "\n", + " x5 x6 ... x21 x22 x23 x24 x25 \\\n", + "0 -0.322024 2.944020 ... -1.828362 -3.010547 -0.840202 -3.085159 1.169952 \n", + "1 -0.230115 -0.631976 ... -0.724172 -0.421045 -2.012480 -2.081784 -2.734123 \n", + "2 -2.206561 0.132579 ... 1.766109 -2.252858 -2.919826 -1.974066 -0.773881 \n", + "3 -3.966273 -0.911226 ... 0.856124 0.727759 -0.501579 1.077504 2.268052 \n", + "4 -1.020599 -1.666200 ... 2.617215 -1.231835 -0.891350 0.246981 2.489642 \n", + "\n", + " x26 x27 x28 x29 x30 \n", + "0 -0.954107 -3.925198 -0.779510 -0.430700 1.004298 \n", + "1 -0.879470 -2.141218 4.598401 -4.222797 -2.523024 \n", + "2 0.244633 -1.727550 1.665467 0.562291 -1.553616 \n", + "3 -3.821422 1.629055 -0.220834 -1.185091 -5.462884 \n", + "4 0.319735 -2.810366 0.585826 3.643749 0.147147 \n", + "\n", + "[5 rows x 34 columns]" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "np.random.seed(123)\n", + "data = make_plpr_CP2025(num_id=150, num_t=10, dim_x=30, theta=0.5, dgp_type=\"dgp1\", time_type=\"int\")\n", + "data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create a corresponding [DoubleMLPanelData](https://docs.doubleml.org/stable/api/generated/doubleml.data.DoubleMLPanelData.html) object, we need to set `static_panel=True` and specify `id_col` and `time_col` columns." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "data_obj = DoubleMLPanelData(data, y_col=\"y\", d_cols=\"d\", t_col=\"time\", id_col=\"id\", static_panel=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The partially linear panel regression (PLPR) model extends the partially linear model to panel data by introducing fixed effects $\\alpha_i^*$.\n", + "\n", + "The PLPR model takes the form\n", + "\n", + "$$\n", + "\\begin{align*}\n", + " Y_{it} &= \\theta_0 D_{it} + g_1(X_{it}) + \\alpha_i^* + U_{it}, \\\\\n", + " D_{it} &= m_1(X_{it}) + \\gamma_i + V_{it},\n", + "\\end{align*}\n", + "$$\n", + "\n", + "where\n", + "- $Y_{it}$ outcome, $D_{it}$ treatment, $X_{it}$ covariates, $\\theta_0$ causal treatment effect\n", + "- $g_1$ and $m_1$ nuisance functions\n", + "- $\\alpha_i^*$, $\\gamma_i$ unobserved individual heterogeneity, correlated with covariates\n", + "- $U_{it}$, $V_{it}$ error terms\n", + "\n", + "Further note $\\mathbb{E}[U_{it} \\mid D_{it}, X_{it}, \\alpha_i^*] = 0$ and $\\mathbb{E}[V_{it} \\mid X_{it}, \\gamma_i]=0$, but $\\mathbb{E}[\\alpha_i^* \\mid D_{it}, X_{it}] \\neq 0$.\n", + "\n", + "Alternatively we can write the partialling-out PLPR as \n", + "\n", + "$$\n", + "\\begin{align*}\n", + " Y_{it} &= \\theta_0 V_{it} + \\ell_1(X_{it}) + \\alpha_i + U_{it}, \\\\\n", + " V_{it} &= D_{it} - m_1(X_{it}) - \\gamma_i,\n", + "\\end{align*}\n", + "$$\n", + "\n", + "with nuisance function $\\ell_1$ and fixed effect $\\alpha_i$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Assumptions\n", + "\n", + "Define $\\xi_i$ as time-invariant heterogeneity terms influencing outcome and treatment and $L_{t-1}(W_i) = \\{ W_{i1}, \\dots, W_{it-1} \\}$ as lags of a random variable $W_{it}$ at wave $t$.\n", + "\n", + "- *No feedback to predictors*\n", + "$$ X_{it} \\perp L_{t-1} (Y_i, D_i) \\mid L_{t-1} (X_i), \\xi_i $$\n", + "- *Static panel*\n", + "$$ Y_{it}, D_{it} \\perp L_{t-1} (Y_i, X_i, D_i) \\mid X_{it}, \\xi_i $$\n", + "- *Selection on observables and omitted time-invariant variables*\n", + "$$ Y_{it} (.) \\perp D_{it} \\mid X_{it}, \\xi_i $$\n", + "- *Homogeneity and linearity of the treatment effect*\n", + "$$ \\mathbb{E} [Y_{it}(d) - Y_{it}(0) \\mid X_{it}, \\xi_i] = d \\theta_0 $$\n", + "- *Additive Separability*\n", + "$$\n", + "\\begin{align*}\n", + "\\mathbb{E} [Y_{it}(0) \\mid X_{it}, \\xi_i] &= g_1(X_{it}) + \\alpha^*_i \\quad \\text{where } \\alpha^*_i = \\alpha^*(\\xi_i), \\\\\n", + "\\mathbb{E} [D_{it} \\mid X_{it}, \\xi_i] &= m_1(X_{it}) + \\gamma_i \\quad \\text{where } \\gamma_i = \\gamma(\\xi_i)\n", + "\\end{align*}\n", + "$$\n", + "\n", + "For more information, see [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To estimate the causal effect, we can create a [DoubleMLPLPR](https://docs.doubleml.org/stable/api/dml_models.html#doubleml-plm) object. The model described in [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011) uses block-k-fold cross-fitting, where the entire time series of the sampled unit is allocated to one fold to allow for possible serial correlation within each unit which is common with panel data. Furthermore, cluster robust standard error are employed. [DoubleMLPLPR](https://docs.doubleml.org/stable/guide/models.html#partially-linear-models-plm) implements both aspects by using `id_col` as the cluster variable." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Estimation Approaches\n", + "\n", + "[Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011) describes multiple estimation approaches, which can be set with the `approach` parameter. Depending on the type of `approach`, different data transformations are performed along the way." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Correlated Random Effects\n", + "\n", + "The correlated random effects (cre) approaches includes the a general approach (`cre_general`) and an approach relying on normality assumptions (`cre_normal`).\n", + "\n", + "#### `cre_general`\n", + "\n", + "The `cre_general` approach:\n", + "\n", + "- Learning $\\ell_1$ from $\\{ Y_{it}, X_{it}, \\bar{X}_i : t=1,\\dots, T \\}_{i=1}^N$.\n", + "- First learning $\\tilde{m}_1$ from $\\{ D_{it}, X_{it}, \\bar{X}_i : t=1,\\dots, T \\}_{i=1}^N$, with predictions $\\hat{m}_{1,it} = \\tilde{m}_1 (X_{it}, \\bar{X}_i)$\n", + " - Calculate $\\hat{\\bar{m}}_i = T^{-1} \\sum_{t=1}^T \\hat{m}_{1,it}$,\n", + " - Calculate final nuisance part as $\\hat{m}^*_1 (X_{it}, \\bar{X}_i, \\bar{D}_i) = \\hat{m}_{1,it} + \\bar{D}_i - \\hat{\\bar{m}}_i$." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "learner = LassoCV()\n", + "ml_l = clone(learner)\n", + "ml_m = clone(learner)\n", + "\n", + "dml_plpr_cre_general = DoubleMLPLPR(data_obj, ml_l=ml_l, ml_m=ml_m, approach=\"cre_general\", n_folds=5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can look at the the transformed data using the `data_transform` property after the `DoubleMLPLPR` object was created." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idtimeydx1x2x3x4x5x6...x21_meanx22_meanx23_meanx24_meanx25_meanx26_meanx27_meanx28_meanx29_meanx30_mean
011-1.2904790.9083071.710715-1.853675-1.4739071.366514-0.3220242.944020...1.24018-0.52821-0.7341450.2274941.1647630.412979-1.2726080.459816-0.829863-1.145189
112-2.850646-1.316777-0.3250434.178599-1.159857-0.139527-0.230115-0.631976...1.24018-0.52821-0.7341450.2274941.1647630.412979-1.2726080.459816-0.829863-1.145189
213-4.338502-1.756120-0.8975901.505972-0.9251891.511500-2.2065610.132579...1.24018-0.52821-0.7341450.2274941.1647630.412979-1.2726080.459816-0.829863-1.145189
314-2.7132360.9348661.9878492.596228-0.220666-0.480717-3.966273-0.911226...1.24018-0.52821-0.7341450.2274941.1647630.412979-1.2726080.459816-0.829863-1.145189
415-5.782997-4.357881-3.0865593.796975-1.539641-2.425617-1.020599-1.666200...1.24018-0.52821-0.7341450.2274941.1647630.412979-1.2726080.459816-0.829863-1.145189
\n", + "

5 rows × 64 columns

\n", + "
" + ], + "text/plain": [ + " id time y d x1 x2 x3 x4 \\\n", + "0 1 1 -1.290479 0.908307 1.710715 -1.853675 -1.473907 1.366514 \n", + "1 1 2 -2.850646 -1.316777 -0.325043 4.178599 -1.159857 -0.139527 \n", + "2 1 3 -4.338502 -1.756120 -0.897590 1.505972 -0.925189 1.511500 \n", + "3 1 4 -2.713236 0.934866 1.987849 2.596228 -0.220666 -0.480717 \n", + "4 1 5 -5.782997 -4.357881 -3.086559 3.796975 -1.539641 -2.425617 \n", + "\n", + " x5 x6 ... x21_mean x22_mean x23_mean x24_mean x25_mean \\\n", + "0 -0.322024 2.944020 ... 1.24018 -0.52821 -0.734145 0.227494 1.164763 \n", + "1 -0.230115 -0.631976 ... 1.24018 -0.52821 -0.734145 0.227494 1.164763 \n", + "2 -2.206561 0.132579 ... 1.24018 -0.52821 -0.734145 0.227494 1.164763 \n", + "3 -3.966273 -0.911226 ... 1.24018 -0.52821 -0.734145 0.227494 1.164763 \n", + "4 -1.020599 -1.666200 ... 1.24018 -0.52821 -0.734145 0.227494 1.164763 \n", + "\n", + " x26_mean x27_mean x28_mean x29_mean x30_mean \n", + "0 0.412979 -1.272608 0.459816 -0.829863 -1.145189 \n", + "1 0.412979 -1.272608 0.459816 -0.829863 -1.145189 \n", + "2 0.412979 -1.272608 0.459816 -0.829863 -1.145189 \n", + "3 0.412979 -1.272608 0.459816 -0.829863 -1.145189 \n", + "4 0.412979 -1.272608 0.459816 -0.829863 -1.145189 \n", + "\n", + "[5 rows x 64 columns]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dml_plpr_cre_general.data_transform.data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that the covariates inlcude the original $X_{it}$ and additionally the unit mean $\\bar{X}_i$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After fitting the model, we can print the `DoubleMLPLPR` object. The data summary corresponds to the transformed data. Additional Information at the end also includes a pre-transformation data summary." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================== DoubleMLPLPR Object ==================\n", + "\n", + "------------------ Data Summary ------------------\n", + "Outcome variable: y\n", + "Treatment variable(s): ['d']\n", + "Covariates: ['x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'x8', 'x9', 'x10', 'x11', 'x12', 'x13', 'x14', 'x15', 'x16', 'x17', 'x18', 'x19', 'x20', 'x21', 'x22', 'x23', 'x24', 'x25', 'x26', 'x27', 'x28', 'x29', 'x30', 'x1_mean', 'x2_mean', 'x3_mean', 'x4_mean', 'x5_mean', 'x6_mean', 'x7_mean', 'x8_mean', 'x9_mean', 'x10_mean', 'x11_mean', 'x12_mean', 'x13_mean', 'x14_mean', 'x15_mean', 'x16_mean', 'x17_mean', 'x18_mean', 'x19_mean', 'x20_mean', 'x21_mean', 'x22_mean', 'x23_mean', 'x24_mean', 'x25_mean', 'x26_mean', 'x27_mean', 'x28_mean', 'x29_mean', 'x30_mean']\n", + "Instrument variable(s): None\n", + "Time variable: time\n", + "Id variable: id\n", + "Static panel data: True\n", + "No. Unique Ids: 150\n", + "No. Observations: 1500\n", + "\n", + "------------------ Score & Algorithm ------------------\n", + "Score function: partialling out\n", + "Static panel model approach: cre_general\n", + "\n", + "------------------ Machine Learner ------------------\n", + "Learner ml_l: LassoCV()\n", + "Learner ml_m: LassoCV()\n", + "Out-of-sample Performance:\n", + "Regression:\n", + "Learner ml_l RMSE: [[1.8336022]]\n", + "Learner ml_m RMSE: [[0.99534683]]\n", + "\n", + "------------------ Resampling ------------------\n", + "No. folds per cluster: 5\n", + "No. folds: 5\n", + "No. repeated sample splits: 1\n", + "\n", + "------------------ Fit Summary ------------------\n", + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d 0.490173 0.026161 18.737077 2.468215e-78 0.438899 0.541447\n", + "\n", + "------------------ Additional Information -------------\n", + "Cluster variable(s): ['id']\n", + "\n", + "Pre-Transformation Data Summary: \n", + "Outcome variable: y\n", + "Treatment variable(s): ['d']\n", + "Covariates: ['x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7', 'x8', 'x9', 'x10', 'x11', 'x12', 'x13', 'x14', 'x15', 'x16', 'x17', 'x18', 'x19', 'x20', 'x21', 'x22', 'x23', 'x24', 'x25', 'x26', 'x27', 'x28', 'x29', 'x30']\n", + "No. Observations: 1500\n", + "\n" + ] + } + ], + "source": [ + "dml_plpr_cre_general.fit()\n", + "print(dml_plpr_cre_general)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### `cre_normal`\n", + "\n", + "The `cre_normal` approach:\n", + "\n", + "Under the assumption that the conditional distribution $D_{i1}, \\dots, D_{iT} \\mid X_{i1}, \\dots X_{iT}$ is multivariate normal (see [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011) for further details):\n", + "- Learn $\\ell_1$ from $\\{ Y_{it}, X_{it}, \\bar{X}_i : t=1,\\dots, T \\}_{i=1}^N$,\n", + "- Learn $m^*_{1}$ from $\\{ D_{it}, X_{it}, \\bar{X}_i, \\bar{D}_i: t=1,\\dots, T \\}_{i=1}^N$." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d 0.505807 0.027168 18.617927 2.299468e-77 0.45256 0.559055\n" + ] + } + ], + "source": [ + "dml_plpr_cre_normal = DoubleMLPLPR(data_obj, ml_l=ml_l, ml_m=ml_m, approach=\"cre_normal\", n_folds=5)\n", + "dml_plpr_cre_normal.fit()\n", + "print(dml_plpr_cre_normal.summary)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `cre_normal` approach uses additionally inlcudes $\\bar{D}_i$ in the treatment nuisance estimation. The corresponding data can be assesses by the `d_mean` property." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[-0.94278854],\n", + " [-0.94278854],\n", + " [-0.94278854],\n", + " ...,\n", + " [ 0.15320478],\n", + " [ 0.15320478],\n", + " [ 0.15320478]], shape=(1500, 1))" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dml_plpr_cre_normal.d_mean" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Transformation Approaches\n", + "\n", + "The transformation approaches include first differences (`fd_exact`) and within-group (`wg_approx`) transformations.\n", + "\n", + "#### `fd_exact`\n", + "\n", + "Consider first differences (FD) transformation (`fd_exact`) $Q(Y_{it})= Y_{it} - Y_{it-1}$, under the assumptions from above, [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011) show that $\\mathbb{E}[Y_{it}-Y_{it-1} | X_{it-1},X_{it}] =\\Delta \\ell_1 (X_{it-1}, X_{it})$ and $\\mathbb{E}[D_{it}-D_{it-1} | X_{it-1},X_{it}] =\\Delta m_1 (X_{it-1}, X_{it})$. Therefore, the transformed nuisance function can be learnt as\n", + "\n", + "- $\\Delta \\ell_1 (X_{it-1}, X_{it})$ from $\\{ Y_{it}-Y_{it-1}, X_{it-1}, X_{it} : t=2, \\dots , T \\}_{i=1}^N$,\n", + "- $\\Delta m_1 (X_{it-1}, X_{it})$ from $\\{ D_{it}-D_{it-1}, X_{it-1}, X_{it} : t=2, \\dots , T \\}_{i=1}^N$." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idtimey_diffd_diffx1x2x3x4x5x6...x21_lagx22_lagx23_lagx24_lagx25_lagx26_lagx27_lagx28_lagx29_lagx30_lag
012-1.560167-2.225084-0.3250434.178599-1.159857-0.139527-0.230115-0.631976...-1.828362-3.010547-0.840202-3.0851591.169952-0.954107-3.925198-0.779510-0.4307001.004298
113-1.487856-0.439343-0.8975901.505972-0.9251891.511500-2.2065610.132579...-0.724172-0.421045-2.012480-2.081784-2.734123-0.879470-2.1412184.598401-4.222797-2.523024
2141.6252662.6909861.9878492.596228-0.220666-0.480717-3.966273-0.911226...1.766109-2.252858-2.919826-1.974066-0.7738810.244633-1.7275501.6654670.562291-1.553616
315-3.069761-5.292747-3.0865593.796975-1.539641-2.425617-1.020599-1.666200...0.8561240.727759-0.5015791.0775042.268052-3.8214221.629055-0.220834-1.185091-5.462884
416-1.0947990.5510510.289315-2.823134-3.137179-1.425923-0.7301160.232687...2.617215-1.231835-0.8913500.2469812.4896420.319735-2.8103660.5858263.6437490.147147
\n", + "

5 rows × 64 columns

\n", + "
" + ], + "text/plain": [ + " id time y_diff d_diff x1 x2 x3 x4 \\\n", + "0 1 2 -1.560167 -2.225084 -0.325043 4.178599 -1.159857 -0.139527 \n", + "1 1 3 -1.487856 -0.439343 -0.897590 1.505972 -0.925189 1.511500 \n", + "2 1 4 1.625266 2.690986 1.987849 2.596228 -0.220666 -0.480717 \n", + "3 1 5 -3.069761 -5.292747 -3.086559 3.796975 -1.539641 -2.425617 \n", + "4 1 6 -1.094799 0.551051 0.289315 -2.823134 -3.137179 -1.425923 \n", + "\n", + " x5 x6 ... x21_lag x22_lag x23_lag x24_lag x25_lag \\\n", + "0 -0.230115 -0.631976 ... -1.828362 -3.010547 -0.840202 -3.085159 1.169952 \n", + "1 -2.206561 0.132579 ... -0.724172 -0.421045 -2.012480 -2.081784 -2.734123 \n", + "2 -3.966273 -0.911226 ... 1.766109 -2.252858 -2.919826 -1.974066 -0.773881 \n", + "3 -1.020599 -1.666200 ... 0.856124 0.727759 -0.501579 1.077504 2.268052 \n", + "4 -0.730116 0.232687 ... 2.617215 -1.231835 -0.891350 0.246981 2.489642 \n", + "\n", + " x26_lag x27_lag x28_lag x29_lag x30_lag \n", + "0 -0.954107 -3.925198 -0.779510 -0.430700 1.004298 \n", + "1 -0.879470 -2.141218 4.598401 -4.222797 -2.523024 \n", + "2 0.244633 -1.727550 1.665467 0.562291 -1.553616 \n", + "3 -3.821422 1.629055 -0.220834 -1.185091 -5.462884 \n", + "4 0.319735 -2.810366 0.585826 3.643749 0.147147 \n", + "\n", + "[5 rows x 64 columns]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dml_plpr_fd_exact = DoubleMLPLPR(data_obj, ml_l=ml_l, ml_m=ml_m, approach=\"fd_exact\", n_folds=5)\n", + "dml_plpr_fd_exact.data_transform.data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We see that the outcome and treatment variables are now labeled `y_diff` and `d_diff` to indicate the first-difference transformation. Moreover, lagged covariates $X_{it-1}$ are added and rows for the first time period are dropped." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_diff 0.511822 0.032746 15.630162 4.536510e-55 0.447641 0.576002\n" + ] + } + ], + "source": [ + "dml_plpr_fd_exact.fit()\n", + "print(dml_plpr_fd_exact.summary)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### `wg_approx`\n", + "\n", + "For within-group (WG) transformation (`wg_approx`) $Q(X_{it})= X_{it} - \\bar{X}_{i}$, where $\\bar{X}_{i} = T^{-1} \\sum_{t=1}^T X_{it}$, approximate the model as\n", + "\n", + "$$\n", + "\\begin{align*}\n", + " Q(Y_{it}) &\\approx \\theta_0 Q(D_{it}) + g_1 (Q(X_{it})) + Q(U_{it}), \\\\\n", + " Q(D_{it}) &\\approx m_1 (Q(X_{it})) + Q(V_{it}).\n", + "\\end{align*}\n", + "$$\n", + "\n", + "Similarly for the partialling-out PLPR\n", + "\n", + "$$\n", + "Q(Y_{it}) \\approx \\theta_0 Q(V_{it}) + \\ell_1 (Q(X_{it})) + Q(U_{it}).\n", + "$$\n", + "\n", + "- $\\ell_1$ can be learnt from transformed data $\\{ Q(Y_{it}), Q(X_{it}) : t=1,\\dots,T \\}_{i=1}^N$,\n", + "- $m_1$ can be learnt from transformed data $\\{ Q(D_{it}), Q(X_{it}) : t=1,\\dots,T \\}_{i=1}^N$." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idtimey_demeand_demeanx1_demeanx2_demeanx3_demeanx4_demeanx5_demeanx6_demean...x21_demeanx22_demeanx23_demeanx24_demeanx25_demeanx26_demeanx27_demeanx28_demeanx29_demeanx30_demean
0111.5435711.7606602.207607-2.039516-0.6428471.0142040.3841661.826013...-3.162933-2.4759420.000728-3.3442190.082829-1.351303-2.670511-1.2753440.4075962.187878
112-0.016596-0.4644240.1718493.992759-0.328797-0.4918370.476074-1.749982...-2.0587430.113560-1.171550-2.340844-3.821246-1.276666-0.8865314.102567-3.384501-1.339444
213-1.504452-0.903767-0.4006981.320131-0.0941291.159190-1.500371-0.985427...0.431538-1.718253-2.078895-2.233126-1.861004-0.152563-0.4728631.1696321.400587-0.370036
3140.1208141.7872192.4847412.4103870.610394-0.833027-3.260084-2.029232...-0.4784471.2623640.3393520.8184431.180930-4.2186182.883741-0.716668-0.346795-4.279304
415-2.948947-3.505528-2.5896673.611134-0.708582-2.777927-0.314410-2.784206...1.282645-0.697230-0.050420-0.0120801.402519-0.077461-1.5556790.0899914.4820451.330727
\n", + "

5 rows × 34 columns

\n", + "
" + ], + "text/plain": [ + " id time y_demean d_demean x1_demean x2_demean x3_demean x4_demean \\\n", + "0 1 1 1.543571 1.760660 2.207607 -2.039516 -0.642847 1.014204 \n", + "1 1 2 -0.016596 -0.464424 0.171849 3.992759 -0.328797 -0.491837 \n", + "2 1 3 -1.504452 -0.903767 -0.400698 1.320131 -0.094129 1.159190 \n", + "3 1 4 0.120814 1.787219 2.484741 2.410387 0.610394 -0.833027 \n", + "4 1 5 -2.948947 -3.505528 -2.589667 3.611134 -0.708582 -2.777927 \n", + "\n", + " x5_demean x6_demean ... x21_demean x22_demean x23_demean x24_demean \\\n", + "0 0.384166 1.826013 ... -3.162933 -2.475942 0.000728 -3.344219 \n", + "1 0.476074 -1.749982 ... -2.058743 0.113560 -1.171550 -2.340844 \n", + "2 -1.500371 -0.985427 ... 0.431538 -1.718253 -2.078895 -2.233126 \n", + "3 -3.260084 -2.029232 ... -0.478447 1.262364 0.339352 0.818443 \n", + "4 -0.314410 -2.784206 ... 1.282645 -0.697230 -0.050420 -0.012080 \n", + "\n", + " x25_demean x26_demean x27_demean x28_demean x29_demean x30_demean \n", + "0 0.082829 -1.351303 -2.670511 -1.275344 0.407596 2.187878 \n", + "1 -3.821246 -1.276666 -0.886531 4.102567 -3.384501 -1.339444 \n", + "2 -1.861004 -0.152563 -0.472863 1.169632 1.400587 -0.370036 \n", + "3 1.180930 -4.218618 2.883741 -0.716668 -0.346795 -4.279304 \n", + "4 1.402519 -0.077461 -1.555679 0.089991 4.482045 1.330727 \n", + "\n", + "[5 rows x 34 columns]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dml_plpr_wg_approx = DoubleMLPLPR(data_obj, ml_l=ml_l, ml_m=ml_m, approach=\"wg_approx\", n_folds=5)\n", + "dml_plpr_wg_approx.data_transform.data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We see that the outcome, treatment and covariate variables are now labeled `y_demean`, `d_demean`, `xi_demean` to indicate the within-group transformations." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_demean 0.495323 0.025841 19.167824 6.872435e-82 0.444675 0.545972\n" + ] + } + ], + "source": [ + "dml_plpr_wg_approx.fit()\n", + "print(dml_plpr_wg_approx.summary)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the simple linear data generating process `dgp_type=\"dgp1\"`, we can see that all approaches lead to estimated close the true effect of `theta=0.5`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `data_original` property additionally includes the original data before any transformation was applied." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idtimeydx1x2x3x4x5x6...x21x22x23x24x25x26x27x28x29x30
011-1.2904790.9083071.710715-1.853675-1.4739071.366514-0.3220242.944020...-1.828362-3.010547-0.840202-3.0851591.169952-0.954107-3.925198-0.779510-0.4307001.004298
112-2.850646-1.316777-0.3250434.178599-1.159857-0.139527-0.230115-0.631976...-0.724172-0.421045-2.012480-2.081784-2.734123-0.879470-2.1412184.598401-4.222797-2.523024
213-4.338502-1.756120-0.8975901.505972-0.9251891.511500-2.2065610.132579...1.766109-2.252858-2.919826-1.974066-0.7738810.244633-1.7275501.6654670.562291-1.553616
314-2.7132360.9348661.9878492.596228-0.220666-0.480717-3.966273-0.911226...0.8561240.727759-0.5015791.0775042.268052-3.8214221.629055-0.220834-1.185091-5.462884
415-5.782997-4.357881-3.0865593.796975-1.539641-2.425617-1.020599-1.666200...2.617215-1.231835-0.8913500.2469812.4896420.319735-2.8103660.5858263.6437490.147147
\n", + "

5 rows × 34 columns

\n", + "
" + ], + "text/plain": [ + " id time y d x1 x2 x3 x4 \\\n", + "0 1 1 -1.290479 0.908307 1.710715 -1.853675 -1.473907 1.366514 \n", + "1 1 2 -2.850646 -1.316777 -0.325043 4.178599 -1.159857 -0.139527 \n", + "2 1 3 -4.338502 -1.756120 -0.897590 1.505972 -0.925189 1.511500 \n", + "3 1 4 -2.713236 0.934866 1.987849 2.596228 -0.220666 -0.480717 \n", + "4 1 5 -5.782997 -4.357881 -3.086559 3.796975 -1.539641 -2.425617 \n", + "\n", + " x5 x6 ... x21 x22 x23 x24 x25 \\\n", + "0 -0.322024 2.944020 ... -1.828362 -3.010547 -0.840202 -3.085159 1.169952 \n", + "1 -0.230115 -0.631976 ... -0.724172 -0.421045 -2.012480 -2.081784 -2.734123 \n", + "2 -2.206561 0.132579 ... 1.766109 -2.252858 -2.919826 -1.974066 -0.773881 \n", + "3 -3.966273 -0.911226 ... 0.856124 0.727759 -0.501579 1.077504 2.268052 \n", + "4 -1.020599 -1.666200 ... 2.617215 -1.231835 -0.891350 0.246981 2.489642 \n", + "\n", + " x26 x27 x28 x29 x30 \n", + "0 -0.954107 -3.925198 -0.779510 -0.430700 1.004298 \n", + "1 -0.879470 -2.141218 4.598401 -4.222797 -2.523024 \n", + "2 0.244633 -1.727550 1.665467 0.562291 -1.553616 \n", + "3 -3.821422 1.629055 -0.220834 -1.185091 -5.462884 \n", + "4 0.319735 -2.810366 0.585826 3.643749 0.147147 \n", + "\n", + "[5 rows x 34 columns]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dml_plpr_wg_approx.data_original.data.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Feature preprocessing pipelines\n", + "\n", + "We can incorporate preprocessing pipelines. For example, when using Lasso, we may want to include polynomial and interaction terms. Here, we create a class that allows us to include, for example, polynomials of order 3 and interactions between all variables." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "class PolyPlus(BaseEstimator, TransformerMixin):\n", + " \"\"\"PolynomialFeatures(degree=k) and additional terms x_i^(k+1).\"\"\"\n", + "\n", + " def __init__(self, degree=2, interaction_only=False, include_bias=False):\n", + " self.degree = degree\n", + " self.extra_degree = degree + 1\n", + " self.interaction_only = interaction_only\n", + " self.include_bias = include_bias\n", + " self.poly = PolynomialFeatures(degree=degree, interaction_only=interaction_only, include_bias=include_bias)\n", + "\n", + " def fit(self, X, y=None):\n", + " self.poly.fit(X)\n", + " self.n_features_in_ = X.shape[1]\n", + " return self\n", + "\n", + " def transform(self, X):\n", + " X = np.asarray(X)\n", + " X_poly = self.poly.transform(X)\n", + " X_extra = X ** self.extra_degree\n", + " return np.hstack([X_poly, X_extra])\n", + "\n", + " def get_feature_names_out(self, input_features=None):\n", + " input_features = np.array(\n", + " input_features\n", + " if input_features is not None\n", + " else [f\"x{i}\" for i in range(self.n_features_in_)]\n", + " )\n", + " poly_names = self.poly.get_feature_names_out(input_features)\n", + " extra_names = [f\"{name}^{self.extra_degree}\" for name in input_features]\n", + " return np.concatenate([poly_names, extra_names])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For this example we use the non-linear and discontinuous `dgp_type=\"dgp3\"`, with 30 covariates and a true treatment effect `theta=0.5`." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "dim_x = 30\n", + "theta = 0.5\n", + "\n", + "np.random.seed(123)\n", + "data_dgp3 = make_plpr_CP2025(num_id=500, num_t=10, dim_x=dim_x, theta=theta, dgp_type=\"dgp3\")\n", + "dml_data_dgp3 = DoubleMLPanelData(data_dgp3, y_col=\"y\", d_cols=\"d\", t_col=\"time\", id_col=\"id\", static_panel=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can apply the polynomial and intercation transformation for specific sets of covariates. For example, for the `fd_exact` approach, we can apply it to the original $X_{it}$ and lags $X_{it-1}$ seperately using `ColumnTransformer`.\n", + "\n", + "To achieve this, we pass need to pass the corresponding indices for these two sets. `DoubleMLPLPR` stacks sets $X_{it}$ and $X_{it-1}$ column-wise. Given our example data has 30 covariates, this means that the first 30 features in the nuisance estimation correspond to the original $X_{it}$, and the last 30 correspond to lags $X_{it-1}$. Therefore we define the indices `indices_x` and `indices_x_tr` as below." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "indices_x = [x for x in range(dim_x)]\n", + "indices_x_tr = [x + dim_x for x in indices_x]\n", + "\n", + "preprocessor = ColumnTransformer(\n", + " [\n", + " (\n", + " \"poly_x\",\n", + " PolyPlus(degree=2, include_bias=False, interaction_only=False),\n", + " indices_x,\n", + " ),\n", + " (\n", + " \"poly_x_tr\",\n", + " PolyPlus(degree=2, include_bias=False, interaction_only=False),\n", + " indices_x_tr,\n", + " ),\n", + " ],\n", + " remainder=\"passthrough\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This preprocessor can be applied for approaches `cre_general` and `cre_normal` in the same fashion. In this case the two sets of covariates would be the original $X_{it}$ and the unit mean $\\bar{X}_i$.\n", + "\n", + "**Remark**: Note that we set `remainder=\"passthrough\"` such that all remaining features, not part of `indices_x` and `indices_x_tr`, would not be preprocessed but still included in the nuisance estimation. This is particularly important for the `cre_normal` approach, as $\\bar{D}_i$ is further added to $X_{it}$ and $\\bar{X}_i$ in the treatment nuisance model." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can create the learner using a pipeline and fit the model." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Pipeline(steps=[('columntransformer',\n",
+       "                 ColumnTransformer(remainder='passthrough',\n",
+       "                                   transformers=[('poly_x', PolyPlus(),\n",
+       "                                                  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,\n",
+       "                                                   10, 11, 12, 13, 14, 15, 16,\n",
+       "                                                   17, 18, 19, 20, 21, 22, 23,\n",
+       "                                                   24, 25, 26, 27, 28, 29]),\n",
+       "                                                 ('poly_x_tr', PolyPlus(),\n",
+       "                                                  [30, 31, 32, 33, 34, 35, 36,\n",
+       "                                                   37, 38, 39, 40, 41, 42, 43,\n",
+       "                                                   44, 45, 46, 47, 48, 49, 50,\n",
+       "                                                   51, 52, 53, 54, 55, 56, 57,\n",
+       "                                                   58, 59])])),\n",
+       "                ('standardscaler', StandardScaler()),\n",
+       "                ('lassocv', LassoCV(cv=2, n_jobs=5))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" + ], + "text/plain": [ + "Pipeline(steps=[('columntransformer',\n", + " ColumnTransformer(remainder='passthrough',\n", + " transformers=[('poly_x', PolyPlus(),\n", + " [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,\n", + " 10, 11, 12, 13, 14, 15, 16,\n", + " 17, 18, 19, 20, 21, 22, 23,\n", + " 24, 25, 26, 27, 28, 29]),\n", + " ('poly_x_tr', PolyPlus(),\n", + " [30, 31, 32, 33, 34, 35, 36,\n", + " 37, 38, 39, 40, 41, 42, 43,\n", + " 44, 45, 46, 47, 48, 49, 50,\n", + " 51, 52, 53, 54, 55, 56, 57,\n", + " 58, 59])])),\n", + " ('standardscaler', StandardScaler()),\n", + " ('lassocv', LassoCV(cv=2, n_jobs=5))])" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ml_lasso = make_pipeline(\n", + " preprocessor, StandardScaler(), LassoCV(cv=2, n_jobs=5)\n", + ")\n", + "\n", + "ml_lasso" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_diff 0.517696 0.018044 28.690074 5.074285e-181 0.482329 0.553062\n" + ] + } + ], + "source": [ + "plpr_lasso_fd = DoubleMLPLPR(dml_data_dgp3, clone(ml_lasso), clone(ml_lasso), approach=\"fd_exact\", n_folds=5)\n", + "plpr_lasso_fd.fit(store_models=True)\n", + "print(plpr_lasso_fd.summary)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given that we apply the polynomial and interactions preprossing to two sets of 30 columns each, the number of features is 1050." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1050" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_lasso_fd.models[\"ml_m\"][\"d_diff\"][0][0].named_steps[\"lassocv\"].n_features_in_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As describes above, for the `cre_normal` approach adds $\\bar{X}_i$ to the features used in the treatment nuisance estimation." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d 0.560037 0.028292 19.794714 3.306228e-87 0.504585 0.615488\n" + ] + }, + { + "data": { + "text/plain": [ + "1051" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_lasso_cre_normal = DoubleMLPLPR(dml_data_dgp3, clone(ml_lasso), clone(ml_lasso), approach=\"cre_normal\", n_folds=5)\n", + "plpr_lasso_cre_normal.fit(store_models=True)\n", + "print(plpr_lasso_cre_normal.summary)\n", + "plpr_lasso_cre_normal.models[\"ml_m\"][\"d\"][0][0].named_steps[\"lassocv\"].n_features_in_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the `wg_approx` approach, there is only one set of features. We can create a similar learner for this setting." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "preprocessor_wg = ColumnTransformer(\n", + " [\n", + " (\n", + " \"poly_x\",\n", + " PolyPlus(degree=2, include_bias=False, interaction_only=False),\n", + " indices_x,\n", + " )\n", + " ],\n", + " remainder=\"passthrough\",\n", + ")\n", + "\n", + "ml_lasso_wg = make_pipeline(\n", + " preprocessor_wg, StandardScaler(), LassoCV(cv=2, n_jobs=5)\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_demean 1.151822 0.014172 81.277037 0.0 1.124047 1.179598\n" + ] + }, + { + "data": { + "text/plain": [ + "525" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_lasso_wg = DoubleMLPLPR(dml_data_dgp3, clone(ml_lasso_wg), clone(ml_lasso_wg), approach=\"wg_approx\", n_folds=5)\n", + "plpr_lasso_wg.fit(store_models=True)\n", + "print(plpr_lasso_wg.summary)\n", + "plpr_lasso_wg.models[\"ml_l\"][\"d_demean\"][0][0].named_steps[\"lassocv\"].n_features_in_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that for the more complicated data generating process `dgp3`, the approximation approach performs worse compared to the other approaches." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As another example, below we should how to select a specific covariate subset for preprocessing. This can be useful in case of the data includes dummy covariates, where adding polynomials might not be appropriate." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "39" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "x_cols = dml_data_dgp3.x_cols \n", + "x_cols_to_pre = [\"x3\", \"x6\", \"x22\"]\n", + "\n", + "indices_x_pre = [i for i, c in enumerate(x_cols) if c in x_cols_to_pre]\n", + "\n", + "preprocessor_alt = ColumnTransformer(\n", + " [\n", + " (\n", + " \"poly_x\",\n", + " PolyPlus(degree=2, include_bias=False, interaction_only=False),\n", + " indices_x_pre,\n", + " )\n", + " ],\n", + " remainder=\"passthrough\",\n", + ")\n", + "ml_lasso_alt = make_pipeline(\n", + " preprocessor_alt, StandardScaler(), LassoCV(cv=2, n_jobs=5)\n", + ")\n", + "\n", + "plpr_lasso_wg.learner[\"ml_l\"] = ml_lasso_alt\n", + "plpr_lasso_wg.learner[\"ml_m\"] = ml_lasso_alt\n", + "\n", + "plpr_lasso_wg.fit(store_models=True)\n", + "plpr_lasso_wg.models[\"ml_l\"][\"d_demean\"][0][0].named_steps[\"lassocv\"].n_features_in_" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
ColumnTransformer(remainder='passthrough',\n",
+       "                  transformers=[('poly_x', PolyPlus(), [2, 5, 21])])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" + ], + "text/plain": [ + "ColumnTransformer(remainder='passthrough',\n", + " transformers=[('poly_x', PolyPlus(), [2, 5, 21])])" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_lasso_wg.models[\"ml_l\"][\"d_demean\"][0][0].named_steps['columntransformer']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also look at the resulting features.\n", + "\n", + "**Remark**: Note, however, that the feature names here refer only to the corresponding `x_cols` indices, not the column names from the `pd.DataFrame` because [DoubleML](https://docs.doubleml.org/stable/index.html) uses `np.array`'s for fitting the model. Therefore the difference to the names from `x_cols_to_pre`." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['poly_x__x2', 'poly_x__x5', 'poly_x__x21', 'poly_x__x2^2',\n", + " 'poly_x__x2 x5', 'poly_x__x2 x21', 'poly_x__x5^2',\n", + " 'poly_x__x5 x21', 'poly_x__x21^2', 'poly_x__x2^3', 'poly_x__x5^3',\n", + " 'poly_x__x21^3', 'remainder__x0', 'remainder__x1', 'remainder__x3',\n", + " 'remainder__x4', 'remainder__x6', 'remainder__x7', 'remainder__x8',\n", + " 'remainder__x9', 'remainder__x10', 'remainder__x11',\n", + " 'remainder__x12', 'remainder__x13', 'remainder__x14',\n", + " 'remainder__x15', 'remainder__x16', 'remainder__x17',\n", + " 'remainder__x18', 'remainder__x19', 'remainder__x20',\n", + " 'remainder__x22', 'remainder__x23', 'remainder__x24',\n", + " 'remainder__x25', 'remainder__x26', 'remainder__x27',\n", + " 'remainder__x28', 'remainder__x29'], dtype=object)" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_lasso_wg.models[\"ml_l\"][\"d_demean\"][0][0].named_steps['columntransformer'].get_feature_names_out()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Hyperparameter tuning\n", + "\n", + "In this section we will use the `tune_ml_models()` method to tune hyperparameters using the [Optuna](https://optuna.org/) package. More details can found in the [Python: Hyperparametertuning with Optuna](https://docs.doubleml.org/stable/examples/learners/py_optuna.html) example notebook.\n", + "\n", + "As an example, we use [LightGBM](https://lightgbm.readthedocs.io/en/stable/) regressors and compare the estimates for the different static panel model approaches, when applied to the non-linear and discontinuous `dgp3`." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "dim_x = 30\n", + "theta = 0.5\n", + "\n", + "np.random.seed(11)\n", + "data_tune = make_plpr_CP2025(num_id=4000, num_t=10, dim_x=dim_x, theta=theta, dgp_type=\"dgp3\")\n", + "dml_data_tune = DoubleMLPanelData(data_tune, y_col=\"y\", d_cols=\"d\", t_col=\"time\", id_col=\"id\", static_panel=True)\n", + "ml_boost = LGBMRegressor(random_state=314, verbose=-1)" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "# parameter space for both ml models\n", + "def ml_params(trial):\n", + " return {\n", + " \"n_estimators\": 100,\n", + " \"learning_rate\": trial.suggest_float(\"learning_rate\", 0.1, 0.4, log=True),\n", + " \"max_depth\": trial.suggest_int(\"max_depth\", 2, 10),\n", + " \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 1, 5),\n", + " \"reg_lambda\": trial.suggest_float(\"reg_lambda\", 1e-2, 5, log=True),\n", + " }\n", + "\n", + "param_space = {\n", + " \"ml_l\": ml_params,\n", + " \"ml_m\": ml_params\n", + "}\n", + "\n", + "optuna_settings = {\n", + " \"n_trials\": 100,\n", + " \"show_progress_bar\": True,\n", + " \"verbosity\": optuna.logging.WARNING, # Suppress Optuna logs\n", + "}" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0e715d42cedf4f24b082a3742b6ba7a1", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/100 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
coefstd errtP>|t|2.5 %97.5 %
d0.4950910.00749166.0884830.00.4804080.509774
\n", + "" + ], + "text/plain": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d 0.495091 0.007491 66.088483 0.0 0.480408 0.509774" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_tune_cre_general = DoubleMLPLPR(dml_data_tune, clone(ml_boost), clone(ml_boost), approach=\"cre_general\", n_folds=5)\n", + "\n", + "plpr_tune_cre_general.tune_ml_models(\n", + " ml_param_space=param_space,\n", + " optuna_settings=optuna_settings,\n", + ")\n", + "\n", + "plpr_tune_cre_general.fit()\n", + "plpr_tune_cre_general.summary" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "0.509102" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "c36de95d4af446f29d9f84c53133e9ed", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/100 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
coefstd errtP>|t|2.5 %97.5 %
d0.4897590.01000248.9682710.00.4701560.509362
\n", + "" + ], + "text/plain": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d 0.489759 0.010002 48.968271 0.0 0.470156 0.509362" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_tune_cre_normal = DoubleMLPLPR(dml_data_tune, clone(ml_boost), clone(ml_boost), approach=\"cre_normal\", n_folds=5)\n", + "\n", + "plpr_tune_cre_normal.tune_ml_models(\n", + " ml_param_space=param_space,\n", + " optuna_settings=optuna_settings,\n", + ")\n", + "\n", + "plpr_tune_cre_normal.fit()\n", + "plpr_tune_cre_normal.summary" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "39ffcc1fef014cdc8010286cf9fe469c", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/100 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
coefstd errtP>|t|2.5 %97.5 %
d_diff0.5492950.00873662.8788430.00.5321740.566417
\n", + "" + ], + "text/plain": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_diff 0.549295 0.008736 62.878843 0.0 0.532174 0.566417" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_tune_fd = DoubleMLPLPR(dml_data_tune, clone(ml_boost), clone(ml_boost), approach=\"fd_exact\", n_folds=5)\n", + "\n", + "plpr_tune_fd.tune_ml_models(\n", + " ml_param_space=param_space,\n", + " optuna_settings=optuna_settings,\n", + ")\n", + "\n", + "plpr_tune_fd.fit()\n", + "plpr_tune_fd.summary" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "072390e43ee2426caf19fa0ffbb35cc4", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/100 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
coefstd errtP>|t|2.5 %97.5 %
d_demean1.1339620.005109221.9708920.01.1239491.143975
\n", + "" + ], + "text/plain": [ + " coef std err t P>|t| 2.5 % 97.5 %\n", + "d_demean 1.133962 0.005109 221.970892 0.0 1.123949 1.143975" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plpr_tune_wg = DoubleMLPLPR(dml_data_tune, clone(ml_boost), clone(ml_boost), approach=\"wg_approx\", n_folds=5)\n", + "\n", + "plpr_tune_wg.tune_ml_models(\n", + " ml_param_space=param_space,\n", + " optuna_settings=optuna_settings,\n", + ")\n", + "\n", + "plpr_tune_wg.fit()\n", + "plpr_tune_wg.summary" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "True treatment effect: 0.5\n", + "\n", + " Model theta se ci_lower ci_upper\n", + "cre_general 0.495091 0.007491 0.480408 0.509774\n", + " cre_normal 0.489759 0.010002 0.470156 0.509362\n", + " fd_exact 0.549295 0.008736 0.532174 0.566417\n", + " wg_approx 1.133962 0.005109 1.123949 1.143975\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAAJOCAYAAABm7rQwAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAorhJREFUeJzs3Xd0VNXexvFnJmXSSEJiEkgEEkIRpAVQpAkqGIpcsAAiShMUBRECFlRA4AoqRVCsCIiKSrFdQVEEUUGuUkSkS/fSEko6qXPeP3gzMiSQSTKZkPD9rJW1MvvsPed3pmzlyTn7mAzDMAQAAAAAAAC4kLmsCwAAAAAAAMDVh1AKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgCAcsJkMun5558v6zJwBWjfvr0aNGhQaL9Dhw7JZDLpvffeK/2icNV57733ZDKZtGnTprIuBQBQThFKAQDKjf379+vhhx9WzZo15eXlJX9/f7Vu3VqzZ8/WuXPnyro8XAXyQp68Hw8PD11zzTVq1aqVnnnmGR05cqSsS3SatWvX2o7zww8/LLBP69atZTKZ8gVkkZGRuuOOOy77/AMGDLB7Lf39/dW4cWPNmDFDmZmZtn7PP/+8TCaTTp065VCtee9LzZo11a9fPx04cMDW7+L3z2w2KygoSJ07d9aGDRsceVkAAIATuZd1AQAAOGLFihXq2bOnLBaL+vXrpwYNGigrK0vr1q3TE088oR07duidd94p6zJL1blz5+Tuzn+6rwR9+vRRly5dZLVadfbsWW3cuFGzZs3S7NmzNW/ePN17771lXaLTeHl56aOPPtL9999v137o0CH98ssv8vLyKvZzWywWvfvuu5KkxMREffrppxozZow2btyoTz75pMjPN2LECN1www3Kzs7Wli1b9M4772jFihX6888/FR4ebuuX9/7l5uZq7969euONN3TLLbdo48aNatiwYbGPBwAAFA3/ZwsAuOIdPHhQ9957r2rUqKE1a9aoatWqtm3Dhg3Tvn37tGLFijKssPRYrVZlZWXJy8urRP/4L+/S0tLk6+tb1mXYNG3aNF9Ic/jwYd1+++3q37+/6tWrp8aNG5dRdc7VpUsX/ec//9GpU6d0zTXX2No/+ugjhYWFqXbt2jp79myxntvd3d3udXz00UfVokULLV68WDNnzrQLkhzRtm1b3XPPPZKkgQMHqk6dOhoxYoQWLlyosWPH2vpd/P61bdtWnTt31ptvvqk33nijWMdSVjIyMuTp6SmzmQsgAADlD//1AgBc8V5++WWlpqZq3rx5doFUnlq1aunxxx+3Pc7JydHkyZMVHR0ti8WiyMhIPfPMM3aXBEn/XGK0du1aNW/eXN7e3mrYsKHWrl0rSfrss8/UsGFDeXl5qVmzZvr999/txg8YMEB+fn46cOCAYmNj5evrq/DwcE2aNEmGYdj1nT59ulq1aqXg4GB5e3urWbNmWrZsWb5jMZlMGj58uBYtWqTrr79eFotFK1eutG27cE2plJQUjRw5UpGRkbJYLAoNDVXHjh21ZcsWu+dcunSpmjVrJm9vb11zzTW6//77dfTo0QKP5ejRo+rRo4f8/PwUEhKiMWPGKDc39xLvzD++/PJLde3aVeHh4bJYLIqOjtbkyZMLHPvrr7+qS5cuqly5snx9fdWoUSPNnj07Xy379+9Xly5dVKlSJfXt21fS+XBq9OjRqlatmiwWi+rWravp06fne71XrVqlNm3aKDAwUH5+fqpbt66eeeYZuz6vvfaarr/+evn4+Khy5cpq3ry5Pvroo0KP9VJq1Kih9957T1lZWXr55Zftth04cEA9e/ZUUFCQfHx8dNNNN+ULUvPW5zl06JBde96laXmfywtt3rxZrVq1kre3t6KiovTWW285VOvu3bt1zz33KCgoSF5eXmrevLn+85//FNi3e/fuslgsWrp0qV37Rx99pF69esnNzc2hfTrCbDarffv2kpTvdSiOW2+9VdL5YPty2rZtK+n8JcKFOXPmjMaMGaOGDRvKz89P/v7+6ty5s/744w+7fnnv2+LFi/XMM8+oSpUq8vX11b/+9S/9/fffdn3z1ggr7P3Me85PPvlEzz33nCIiIuTj46Pk5GRJjn3Xt23bpgEDBtgug65SpYoGDRqk06dP5zvWo0eP6sEHH7R9r6OiovTII48oKyvLrl9mZqbi4uIUEhIiX19f3XnnnUpISMj3fN98843atm0rX19fVapUSV27dtWOHTvs+pw4cUIDBw7UtddeK4vFoqpVq6p79+5O+TwAAK48nCkFALjiffXVV6pZs6ZatWrlUP/Bgwdr4cKFuueeezR69Gj9+uuvmjp1qnbt2qXPP//cru++fft033336eGHH9b999+v6dOnq1u3bnrrrbf0zDPP6NFHH5UkTZ06Vb169dKePXvszkjIzc1Vp06ddNNNN+nll1/WypUrNWHCBOXk5GjSpEm2frNnz9a//vUv9e3bV1lZWfrkk0/Us2dPLV++XF27drWrac2aNVqyZImGDx+ua665RpGRkQUe59ChQ7Vs2TINHz5c9evX1+nTp7Vu3Trt2rVLTZs2lXQ+6Bg4cKBuuOEGTZ06VSdPntTs2bO1fv16/f777woMDLQ7ltjYWLVo0ULTp0/X999/rxkzZig6OlqPPPLIZV/z9957T35+foqLi5Ofn5/WrFmj8ePHKzk5WdOmTbP1W7Vqle644w5VrVpVjz/+uKpUqaJdu3Zp+fLl+YLF2NhYtWnTRtOnT5ePj48Mw9C//vUv/fDDD3rwwQfVpEkTffvtt3riiSd09OhRvfLKK5KkHTt26I477lCjRo00adIkWSwW7du3T+vXr7c9/9y5czVixAjdc889evzxx5WRkaFt27bp119/1X333XfZY72cli1bKjo6WqtWrbK1nTx5Uq1atVJ6erpGjBih4OBgLVy4UP/617+0bNky3XnnncXa19mzZ9WlSxf16tVLffr00ZIlS/TII4/I09NTgwYNuuS4HTt2qHXr1oqIiNDTTz8tX19fLVmyRD169NCnn36arx4fHx91795dH3/8se1z8Mcff2jHjh169913tW3btmLVfyl5wVBwcLDLnisv8KhcuXKhz3ngwAF98cUX6tmzp6KionTy5Em9/fbbateunXbu3Jnv7K4XXnhBJpNJTz31lOLj4zVr1ix16NBBW7dulbe3t61fUd7PyZMny9PTU2PGjFFmZqY8PT0d/q6vWrVKBw4c0MCBA1WlShXbpc87duzQf//7X5lMJknSsWPHdOONNyoxMVEPPfSQrrvuOh09elTLli1Tenq6PD09bfU89thjqly5siZMmKBDhw5p1qxZGj58uBYvXmzr88EHH6h///6KjY3VSy+9pPT0dL355ptq06aNfv/9d9s8d/fdd2vHjh167LHHFBkZqfj4eK1atUpHjhy55FwIACjHDAAArmBJSUmGJKN79+4O9d+6dashyRg8eLBd+5gxYwxJxpo1a2xtNWrUMCQZv/zyi63t22+/NSQZ3t7exuHDh23tb7/9tiHJ+OGHH2xt/fv3NyQZjz32mK3NarUaXbt2NTw9PY2EhARbe3p6ul09WVlZRoMGDYxbb73Vrl2SYTabjR07duQ7NknGhAkTbI8DAgKMYcOGXfK1yMrKMkJDQ40GDRoY586ds7UvX77ckGSMHz8+37FMmjTJ7jliYmKMZs2aXXIflzo+wzCMhx9+2PDx8TEyMjIMwzCMnJwcIyoqyqhRo4Zx9uxZu75WqzVfLU8//bRdny+++MKQZPz73/+2a7/nnnsMk8lk7Nu3zzAMw3jllVcMSXav/8W6d+9uXH/99YUe18UOHjxoSDKmTZt22eeWZCQlJRmGYRgjR440JBk///yzrU9KSooRFRVlREZGGrm5uYZhGMaCBQsMScbBgwftnu+HH37I99lr166dIcmYMWOGrS0zM9No0qSJERoaamRlZdnVu2DBAlu/2267zWjYsKHtfTGM869/q1atjNq1a+fb79KlS43ly5cbJpPJOHLkiGEYhvHEE08YNWvWtNVy8WtZo0YNo2vXrpd8jQzj/Pvs6+trJCQkGAkJCca+ffuMKVOmGCaTyWjUqJGt34QJEwp9P/NqnT9/vpGQkGAcO3bMWLFihREZGWmYTCZj48aNdq/HxIkTjYSEBOPEiRPGzz//bNxwww22Yy1MRkaG7T3Lc/DgQcNisdh9f/JqioiIMJKTk23tS5YsMSQZs2fPtrU5+n7mPWfNmjXtvnNF+a4X9F39+OOPDUnGTz/9ZGvr16+fYTabba/dhfK+r3mf2Q4dOth9h0eNGmW4ubkZiYmJhmGc/7wHBgYaQ4YMsXueEydOGAEBAbb2s2fPFvr9AgBULFy+BwC4ouVdllKpUiWH+n/99deSpLi4OLv20aNHS1K+S6bq16+vli1b2h63aNFC0vnLfqpXr56v/cI7eeUZPny47fe8y++ysrL0/fff29ovPiMiKSlJbdu2zXepnSS1a9dO9evXL+RIpcDAQP366686duxYgds3bdqk+Ph4Pfroo3brUXXt2lXXXXddgetwDR061O5x27ZtCzzmi114fCkpKTp16pTatm2r9PR07d69W5L0+++/6+DBgxo5cqTdGVqSbGdnXOjis7O+/vprubm5acSIEXbto0ePlmEY+uabbyTJ9txffvmlrFZrgfUGBgbqf//7nzZu3FjosRWVn5+fpPOvQ17dN954o9q0aWPX56GHHtKhQ4e0c+fOYu3H3d1dDz/8sO2xp6enHn74YcXHx2vz5s0Fjjlz5ozWrFmjXr162d6nU6dO6fTp04qNjdVff/2V73IvSbr99tsVFBSkTz75RIZh6JNPPlGfPn2KVfeF0tLSFBISopCQENWqVUvPPPOMWrZsme+MRkcNGjRIISEhCg8PV9euXZWWlqaFCxeqefPmdv0mTJigkJAQValSRW3bttWuXbs0Y8YM23pUl2OxWGxnS+bm5ur06dO2S0QL+j7369fPbv665557VLVqVdtclaco72f//v3tvnNF+a5fOC4jI0OnTp3STTfdJEm2+q1Wq7744gt169Yt32sn5f++PvTQQ3Ztbdu2VW5urg4fPizp/NlZiYmJ6tOnj+0zd+rUKbm5ualFixb64YcfbLV5enpq7dq1xV6nDABQvhBKAQCuaP7+/pL++Qd+YQ4fPiyz2axatWrZtVepUkWBgYG2fyTluTB4kqSAgABJUrVq1Qpsv/gfSmazWTVr1rRrq1OnjiT7NXGWL1+um266SV5eXgoKClJISIjefPNNJSUl5TuGqKiowg5T0vm1trZv365q1arpxhtv1PPPP28XIOUda926dfONve666/K9Fl5eXgoJCbFrq1y5skP/ONyxY4fuvPNOBQQEyN/fXyEhIbaFpPOOMe9SqgYNGhT6fO7u7rr22mvt2g4fPqzw8PB8AWW9evVs2yWpd+/eat26tQYPHqywsDDde++9WrJkiV1A9dRTT8nPz0833nijateurWHDhtld3lcSqampkv4JUg8fPlzge3Bx3UUVHh6eb/H3gj57F9q3b58Mw9C4ceNsYVDez4QJEyRJ8fHx+cZ5eHioZ8+e+uijj/TTTz/p77//LtFljnm8vLy0atUqrVq1yva869evz/edctT48eO1atUqrVmzRtu2bdOxY8f0wAMP5Ov30EMPadWqVfrqq680atQonTt3zqG106Tzgc0rr7yi2rVry2Kx6JprrlFISIi2bdtW4Pe5du3ado9NJpNq1aqV7z0qyvt58RxRlO/6mTNn9PjjjyssLEze3t4KCQmxPV9e/QkJCUpOTnbouyrln0fzLoPMmzv++usvSefD/os/d999953tM2exWPTSSy/pm2++UVhYmG6++Wa9/PLLOnHihEN1AADKH9aUAgBc0fz9/RUeHq7t27cXaVxBZ94U5FKLNF+q3bhoQW1H/Pzzz/rXv/6lm2++WW+88YaqVq0qDw8PLViwoMCFtS88k+FyevXqpbZt2+rzzz/Xd999p2nTpumll17SZ599ps6dOxe5zuIuWJ2YmKh27drJ399fkyZNUnR0tLy8vLRlyxY99dRTlzxb6XIuPBulqLy9vfXTTz/phx9+0IoVK7Ry5UotXrxYt956q7777ju5ubmpXr162rNnj5YvX66VK1fq008/1RtvvKHx48dr4sSJxdpvnu3btys0NNQWqDrqUp9ZR8MSR+S9F2PGjFFsbGyBfS4OdPPcd999euutt/T888+rcePGDp3NVxg3Nzd16NChxM+Tp2HDhg49X+3atW397rjjDrm5uenpp5/WLbfcUuCZQReaMmWKxo0bp0GDBmny5MkKCgqS2WzWyJEji/VZLw5H54iC9OrVS7/88oueeOIJNWnSRH5+frJarerUqVOx6y9svsx73g8++EBVqlTJ18/d/Z9/kowcOVLdunXTF198oW+//Vbjxo3T1KlTtWbNGsXExBSrPgDAlYtQCgBwxbvjjjv0zjvvaMOGDXaX2hWkRo0aslqt+uuvv2xnokjnF5tOTExUjRo1nFqb1WrVgQMHbGc0SNLevXslybYo76effiovLy99++23slgstn4LFiwo8f6rVq2qRx99VI8++qji4+PVtGlTvfDCC+rcubPtWPfs2WO7C1mePXv2OO21WLt2rU6fPq3PPvtMN998s6394jueRUdHSzof2hQniKhRo4a+//57paSk2J0tlXd54IXHYzabddttt+m2227TzJkzNWXKFD377LP64YcfbPv29fVV79691bt3b2VlZemuu+7SCy+8oLFjx9pdAlUUGzZs0P79+21nieXVtWfPnnx9L6477+ySxMREu36XOpPq2LFjSktLszu75uLP3sXyzkDy8PAo8nvQpk0bVa9eXWvXrtVLL71UpLFXumeffVZz587Vc889Z7vb5aUsW7ZMt9xyi+bNm2fXnpiYqGuuuSZf/7yzhPIYhqF9+/apUaNGdu3FeT/zOPpdP3v2rFavXq2JEydq/Pjxl6wxJCRE/v7+Rf5jwKXkffdDQ0Md+txFR0dr9OjRGj16tP766y81adJEM2bM0IcffuiUegAAVw4u3wMAXPGefPJJ+fr6avDgwTp58mS+7fv379fs2bMlSV26dJEkzZo1y67PzJkzJSnfne6cYc6cObbfDcPQnDlz5OHhodtuu03S+bMITCaT3Rkvhw4d0hdffFHsfebm5ua7VCg0NFTh4eHKzMyUJDVv3lyhoaF66623bG3S+duy79q1y2mvRd5ZEheeRZaVlaU33njDrl/Tpk0VFRWlWbNm5QteHDkDrUuXLsrNzbV7vSXplVdekclksp0ddubMmXxjmzRpIkm21+H06dN22z09PVW/fn0ZhqHs7OxCaynI4cOHNWDAAHl6euqJJ56wq/u3337Thg0bbG1paWl65513FBkZaTvjKO8f7j/99JOtX25urt55550C95eTk6O3337b9jgrK0tvv/22QkJC1KxZswLHhIaGqn379nr77bd1/PjxfNsTEhIueXwmk0mvvvqqJkyYUOAlceVZYGCgHn74YX377bfaunXrZfu6ubnl+7wuXbq0wLW4JOn999+3u/x42bJlOn78eL6zGYvzfuZx9Lte0HdVyj9fms1m9ejRQ1999ZU2bdqUb39FPWM0NjZW/v7+mjJlSoHfr7zPXXp6ujIyMuy2RUdHq1KlSnbHBQCoODhTCgBwxYuOjtZHH32k3r17q169eurXr58aNGigrKws/fLLL1q6dKkGDBggSWrcuLH69++vd955x3ZZ2W+//aaFCxeqR48euuWWW5xam5eXl1auXKn+/furRYsW+uabb7RixQo988wztvWZunbtqpkzZ6pTp0667777FB8fr9dff121atXStm3birXflJQUXXvttbrnnnvUuHFj+fn56fvvv9fGjRs1Y8YMSefPhnnppZc0cOBAtWvXTn369LHdJj4yMlKjRo1yymvQqlUrVa5cWf3799eIESNkMpn0wQcf5PuHq9ls1ptvvqlu3bqpSZMmGjhwoKpWrardu3drx44d+vbbby+7n27duumWW27Rs88+q0OHDqlx48b67rvv9OWXX2rkyJG2UGfSpEn66aef1LVrV9WoUUPx8fF64403dO2119oWG7/99ttVpUoVtW7dWmFhYdq1a5fmzJmjrl27OrSo/pYtW/Thhx/KarUqMTFRGzdu1Keffmo79gvPgnn66af18ccfq3PnzhoxYoSCgoK0cOFCHTx4UJ9++qntMsXrr79eN910k8aOHaszZ87YFhbPyckpsIbw8HC99NJLOnTokOrUqaPFixdr69ateuedd+Th4XHJ2l9//XW1adNGDRs21JAhQ1SzZk2dPHlSGzZs0P/+9z/98ccflxzbvXt3de/evdDXRzq/ftW///3vfO0xMTHFCkRnzpwpHx8fuzaz2axnnnmmyM9VkMcff1yzZs3Siy++qE8++eSS/e644w5NmjRJAwcOVKtWrfTnn39q0aJFl1wHKygoSG3atNHAgQN18uRJzZo1S7Vq1dKQIUPs+hX3/ZQc/677+/vb1mnKzs5WRESEvvvuu3xnNUrnL1P87rvv1K5dOz300EOqV6+ejh8/rqVLl2rdunX5blZwOf7+/nrzzTf1wAMPqGnTprr33nsVEhKiI0eOaMWKFWrdurXmzJmjvXv36rbbblOvXr1Uv359ubu76/PPP9fJkyd17733Orw/AEA5Uha3/AMAoDj27t1rDBkyxIiMjDQ8PT2NSpUqGa1btzZee+01u9vbZ2dnGxMnTjSioqIMDw8Po1q1asbYsWPt+hjGpW9bL8kYNmyYXVvereQvvFV53i3t9+/fb9x+++2Gj4+PERYWZkyYMCHfLePnzZtn1K5d27BYLMZ1111nLFiwwHar+8L2feG2CRMmGIZx/nbxTzzxhNG4cWOjUqVKhq+vr9G4cWPjjTfeyDdu8eLFRkxMjGGxWIygoCCjb9++xv/+9z+7PnnHcrGCaizI+vXrjZtuusnw9vY2wsPDjSeffNL49ttvDUnGDz/8YNd33bp1RseOHW11N2rUyHjttdcKrcUwzt9aftSoUUZ4eLjh4eFh1K5d25g2bZrd7ehXr15tdO/e3QgPDzc8PT2N8PBwo0+fPsbevXttfd5++23j5ptvNoKDgw2LxWJER0cbTzzxhJGUlHTZ48z7HOT9uLu7G0FBQUaLFi2MsWPHGocPHy5w3P79+4177rnHCAwMNLy8vIwbb7zRWL58eYH9OnToYFgsFiMsLMx45plnjFWrVuV7Hdu1a2dcf/31xqZNm4yWLVsaXl5eRo0aNYw5c+YUWO+CBQvy7adfv35GlSpVDA8PDyMiIsK44447jGXLltn6/PDDD4YkY+nSpZd9TfJquVCNGjXsXqcLfx588EHDMC7/Pl8o7zNY0I+bm1uRai3oe3yhAQMGGG5ubsa+ffsu+RwZGRnG6NGjjapVqxre3t5G69atjQ0bNhjt2rUz2rVrZ+uXV9PHH39sjB071ggNDTW8vb2Nrl275vucOPp+FnacjnzX//e//xl33nmnERgYaAQEBBg9e/Y0jh07Zje/5Dl8+LDRr18/IyQkxLBYLEbNmjWNYcOGGZmZmYZhGMaCBQsMScbGjRsLrPPi7/4PP/xgxMbGGgEBAYaXl5cRHR1tDBgwwNi0aZNhGIZx6tQpY9iwYcZ1111n+Pr6GgEBAUaLFi2MJUuWXPL9AACUbybDKMaKrQAAQAMGDNCyZctsd1sDgDxr167VLbfcoqVLl+qee+65bN/27dvr1KlTTlvDCQCA8oI1pQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuBxrSgEAAAAAAMDlOFMKAAAAAAAALkcoBQAAAAAAAJdzL+sCXM1qterYsWOqVKmSTCZTWZcDAAAAAABQoRiGoZSUFIWHh8tsvvT5UFddKHXs2DFVq1atrMsAAAAAAACo0P7++29de+21l9x+1YVSlSpVknT+hfH39y/jakrGarUqISFBISEhl00eAQCXxlwKACXHXAoAJVeR5tLk5GRVq1bNlsFcylUXSuVdsufv718hQqmMjAz5+/uX+w8sAJQV5lIAKDnmUgAouYo4lxa2bFLFOEoAAAAAAACUK4RSAAAAAAAAcDlCKQAAAAAAALjcVbemlKNyc3OVnZ1d1mVcltVqVXZ2tjIyMirM9aZXCg8PD7m5uZV1GQAAAAAAVFiEUhcxDEMnTpxQYmJiWZdSKMMwZLValZKSUujiYSi6wMBAValShdcWAAAAAIBSQCh1kbxAKjQ0VD4+Pld0IGEYhnJycuTu7n5F11neGIah9PR0xcfHS5KqVq1axhUBAAAAAFDxEEpdIDc31xZIBQcHl3U5hSKUKj3e3t6SpPj4eIWGhnIpHwAAAAAATsZCRBfIW0PKx8enjCvBlSDvc3Clry0GAAAAAEB5RChVAM46gsTnAAAAAACA0kQoBQAAAAAAAJcjlILLffHFF6pVq5bc3Nw0cuTIS7YBAAAAAICKi1CqAjCbzTKZTJf8ef7558u6RDsPP/yw7rnnHv3999+aPHnyJdtKYu3atTKZTEpMTCzxcwEAAAAAAOfj7nsVwLFjx2zrHy1evFjjx4/Xnj17bNv9/PxsvxuGodzcXLm7l81bn5qaqvj4eMXGxio8PPySbQAAAAAAoGLjTKkKoEqVKrafgIAAmUwm2+Pdu3erUqVK+uabb9SsWTNZLBatW7dOAwYMUI8ePeyeZ+TIkWrfvr3tsdVq1dSpUxUVFSVvb281btxYy5Ytu2wtmZmZGjNmjCIiIuTr66sWLVpo7dq1ks6fvVSpUiVJ0q233iqTyXTJNklat26d2rZtK29vb1WrVk0jRoxQWlqa3b6eeuopVatWTRaLRbVq1dK8efN06NAh3XLLLZKkypUry2QyacCAAcV/gQEAAAAAgNMRSl0lnn76ab344ovatWuXGjVq5NCYqVOn6v3339dbb72lHTt2aNSoUbr//vv1448/XnLM8OHDtWHDBn3yySfatm2bevbsqU6dOumvv/5Sq1atbGdwffrppzp+/Pgl2/bv369OnTrp7rvv1rZt27R48WKtW7dOw4cPt+2rX79++vjjj/Xqq69q165devvtt+Xn56dq1arp008/lSTt2bNHx48f1+zZs4v70gEAAAAAgFLA5XsOOvnzEZ1cd6TQfj4RlVSrX2O7tn3v/6H0oymFjg1rU11hbasXu8bLmTRpkjp27Ohw/8zMTE2ZMkXff/+9WrZsKUmqWbOm1q1bp7ffflvt2rXLN+bIkSNasGCBjhw5YrsMb8yYMVq5cqUWLFigKVOmKDQ0VJIUFBSkKlWqSFKBbVOnTlXfvn1ti57Xrl1br776qtq1a6c333xTR44c0ZIlS7Rq1Sp16NDBVl+eoKAg23MHBgY6fNwAAAAAAMA1CKUclJuZo+zkzEL75QRY8relZjk0Njczp1i1OaJ58+ZF6r9v3z6lp6fnC7KysrIUExNT4Jg///xTubm5qlOnjl17ZmamgoODi7T/P/74Q9u2bdOiRYtsbYZhyGq16uDBg/rzzz/l5uZWYDgGAAAAAACufIRSDnKzuMvDP3/gdDF3P88C2xwZ62YpvbfD19fX7rHZbJZhGHZt2dnZtt9TU1MlSStWrFBERIRdP4ul4GNJTU2Vm5ubNm/eLDc3N7ttFy627ojU1FQ9/PDDGjFiRL5t1atX1759+4r0fAAAAAAA4MpCKOWgsLbFv7Tu4sv5rgQhISHavn27XdvWrVvl4eEhSapfv74sFouOHDni8NlIMTExys3NVXx8vNq2bVui+po2baqdO3eqVq1aBW5v2LChrFarfvzxR9vlexfy9DwfDubm5paoDgAAAAAAUDoIpa5St956q6ZNm6b3339fLVu21Icffqjt27fbLs2rVKmSxowZo1GjRslqtapNmzZKSkrS+vXr5e/vr/79++d7zjp16qhv377q16+fZsyYoZiYGCUkJGj16tVq1KiRunbt6nB9Tz31lG666SYNHz5cgwcPlq+vr3bu3KlVq1Zpzpw5ioyMVP/+/TVo0CC9+uqraty4sQ4fPqz4+Hj16tVLNWrUkMlk0vLly9WlSxd5e3sX+WwtAAAAAACc7fTKmTq9cqZdm2EYkmFVbm6uktzcJJNZJpPJrk9wpzgFd4pzZamljrvvXaViY2M1btw4Pfnkk7rhhhuUkpKifv362fWZPHmyxo0bp6lTp6pevXrq1KmTVqxYoaioqEs+74IFC9SvXz+NHj1adevWVY8ePbRx40ZVr160s8waNWqkH3/8UXv37lXbtm0VExOj8ePH2xZQl6Q333xT99xzjx599FFdd911GjJkiNLS0iRJERERmjhxop5++mmFhYXZ3bUPAAAAAICyknsuWTlnj9r95CYeU27SCSk1QblJJ5SbeCx/n3PJZV2605mMixcWquCSk5MVEBCgpKQk+fv7223LyMjQwYMHFRUVJS8vrzKq0HGGYSgnJ0fu7u75ElSUXHn7PAAoHqvVqvj4eIWGhsps5m81AFAczKUA4Li8M6Ws2Rmypp6RdLlYxiSzX5DMHl7l6kypy2UvF+LyPQAAAAAAABcJ7hQnS8T1OjKzq2QySZc7V8hkkjU9UdfGrZBfw1jXFeki/BkDAAAAAADARXLTEvX3a3efD6MM6+U7G1bJMPT3a3crNy3RJfW5EqEUAAAAAACAiySuXygjM73wQCqPYZWRla7E9e+XbmFlgFAKAAAAAADABQzD0JlVrxVr7JlVr6qiLQtOKAUAAAAAAOACuamnlR2/X5df3LwAhqHs+P3KTTtTKnWVFUIpAAAAAAAAF7BmpJZs/LkUJ1VyZSCUAgAAAAAAcAGzl1/JxntXclIlVwb3si6gIpj543698tMBuzbDMGQ1JEOGTDLJbJJMJpNdn1E311Rcu2hXlgoAAAAAAMqIm1+wPEKjlZ1w4Pzd9xxlMskjpKbcfINKr7gyQCjlBMkZOTqalFGscQAAAAAA4OpgMpkU1PExnVw0qshjgzqOyHeyS3nH5XtO4O/lrogALwX7eKiwj4dJUrCPhyICvOTvRSZYlg4dOiSTyaStW7eWdSkAAAAAgKtEYOv+Mll8JJODkYzJLJOnjwJb9yvdwsoAoZQTxLWL1ru9GivxXI4KCy1NJinxXI7e7dWYS/cAAAAAALjKuPkGqtpjn54PCAoLpkxmyWRStcc+k5tvoEvqcyVCKSdIPJetexZukqHz60hdTt46U/cs3KTEc9muKbAA2dllt+/SlpWVVdYlAAAAAABQoNMrZ+rYvAdl9gksfF0pw5DZJ1DH5g3S6ZUzXVKfKxFKOcHCTX8rPSu30EAqj9WQ0rNy9f6mv51ah9Vq1csvv6xatWrJYrGoevXqeuGFF2yXqS1evFjt2rWTl5eXFi1aJEl69913Va9ePXl5eem6667TG2+84fD+fvnlFzVp0kReXl5q3ry5vvjii3yXw23fvl2dO3eWn5+fwsLC9MADD+jUqVO27e3bt9eIESP05JNPKigoSFWqVNHzzz9vt5/ExEQNHjxYISEh8vf316233qo//vjDtv35559XkyZN9O677yoqKkpeXl6SpJUrV6pNmzYKDAxUcHCw7rjjDu3fv78YrywAAAAAAM6Rey5ZOWePypp6WlJhQYIha+pp5Zw9qtxzya4oz6UIpUrIMAzNWXewWGNfW3dQRlFW2y/E2LFj9eKLL2rcuHHauXOnPvroI4WFhdm2P/3003r88ce1a9cuxcbGatGiRRo/frxeeOEF7dq1S1OmTNG4ceO0cOHCQveVnJysbt26qWHDhtqyZYsmT56sp556yq5PYmKibr31VsXExGjTpk1auXKlTp48qV69etn1W7hwoXx9ffXrr7/q5Zdf1qRJk7Rq1Srb9p49eyo+Pl7ffPONNm/erKZNm+q2227TmTNnbH327dunTz/9VJ999pktFEtLS1NcXJw2bdqk1atXy2w2684775TVai3OywsAAAAAQIm5efvLvXKE3Y9bYLjcAqpIfiFyC6git8Dw/H28/cu6dKdjpe0SOp2epf2n04s8zpC0/3S6zqRnK9jXs8R1pKSkaPbs2ZozZ4769+8vSYqOjlabNm106NAhSdLIkSN111132cZMmDBBM2bMsLVFRUVp586devvtt23PcSkfffSRTCaT5s6dKy8vL9WvX19Hjx7VkCFDbH3mzJmjmJgYTZkyxdY2f/58VatWTXv37lWdOnUkSY0aNdKECRMkSbVr19acOXO0evVqdezYUevWrdNvv/2m+Ph4WSwWSdL06dP1xRdfaNmyZXrooYcknb9k7/3331dISIhtX3fffbddzfPnz1dISIh27typBg0aOP7iAgAAAADgJMGd4hTcKS5fu9VqVXx8vEJDQ2U2Xx3nEBFKlVBqZm6Jxqdk5jgllNq1a5cyMzN12223XbJP8+bNbb+npaVp//79evDBB+2CpJycHAUEBBS6vz179qhRo0a2S+Uk6cYbb7Tr88cff+iHH36Qn59fvvH79++3C6UuVLVqVcXHx9ueIzU1VcHBwXZ9zp07Z3cpXo0aNewCKUn666+/NH78eP366686deqU7QypI0eOEEoBAAAAAFDGCKVKyM/iVqLxlSzOeQu8vb0L7ePr62v7PTU1VZI0d+5ctWjRwq6fm1vJjunCfXTr1k0vvfRSvm1Vq1a1/e7h4WG3zWQy2QKk1NRUVa1aVWvXrs33HIGBgbbfLzy2PN26dVONGjU0d+5chYeHy2q1qkGDBiyEDgAAAADAFYBQqoSCfTwVHeyjA6fTC12e7EImSTWDfRTk41FoX0fUrl1b3t7eWr16tQYPHlxo/7CwMIWHh+vAgQPq27dvkfdXt25dffjhh8rMzLRdVrdx40a7Pk2bNtWnn36qyMhIubsX76PWtGlTnThxQu7u7oqMjHR43OnTp7Vnzx7NnTtXbdu2lSStW7euWDUAAAAAAADnuzouUixFJpNJw9tEFWvsY22iZDKZnFKHl5eXnnrqKT355JN6//33tX//fv33v//VvHnzLjlm4sSJmjp1ql599VXt3btXf/75pxYsWKCZMwu/zeR9990nq9Wqhx56SLt27dK3336r6dOnS5LtmIYNG6YzZ86oT58+2rhxo/bv369vv/1WAwcOVG6uY5c9dujQQS1btlSPHj303Xff6dChQ/rll1/07LPPatOmTZccV7lyZQUHB+udd97Rvn37tGbNGsXF5b9mFwAAAAAAlA1CKSfo37yafDzdZHYwXzKbJB9PN/VrXs2pdYwbN06jR4/W+PHjVa9ePfXu3du2NlNBBg8erHfffVcLFixQw4YN1a5dO7333nuKiio8ZPP399dXX32lrVu3qkmTJnr22Wc1fvx4SbKtMxUeHq7169crNzdXt99+uxo2bKiRI0cqMDDQ4UXbTCaTvv76a918880aOHCg6tSpo3vvvVeHDx+2u7Pgxcxmsz755BNt3rxZDRo00KhRozRt2jSH9gkAAAAAAEqfyTCMolx1Vu4lJycrICBASUlJ8ve3v51iRkaGDh48qKioKLsFvB3x7Z543fHubzJkyHqZV9RskkwyacXgG3V73dDiHIKNYRjKycmRu7u70864KolFixZp4MCBSkpKcmiNqytdST4PAMqPq/EuJwDgbMylAFByFWkuvVz2cqHyfZRXiJk/7tfgJX8o0NtdhUV8hiEFervrwSV/aOaP+y/f+Qr3/vvva926dTp48KC++OILPfXUU+rVq1eFCKQAAAAAAEDpIpRyguSMHB1NytDp9OxCFzs3JJ1Oz9bRpAwlZ+S4orximTJlivz8/Ar86dy5syTpxIkTuv/++1WvXj2NGjVKPXv21DvvvFPGlQMAAAAAgPKAu+85gb+XuyIC7C/vMozzl/EZMmSS6fxlexddYufvdeW+/EOHDlWvXr0K3JZ3JtSTTz6pJ5980pVlAQAAAACACqJMU5GffvpJ06ZN0+bNm3X8+HF9/vnn6tGjxyX7Hz9+XKNHj9amTZu0b98+jRgxQrNmzXJZvZcS1y5ace2iy7oMpwoKClJQUFBZlwEAAAAAACqoMr18Ly0tTY0bN9brr7/uUP/MzEyFhIToueeeU+PGjUu5OgAAAAAAAJSWMj1TqnPnzrb1iRwRGRmp2bNnS5Lmz59fWmUBAAAAAACglF25ixo5SWZmpjIzM22Pk5OTJZ2/1aLVarXra7VaZRiG7ac8yKuzvNRbnuR9Dgr6rACoOPLmfr7nAFB8zKUAUHIVaS519BgqfCg1depUTZw4MV97QkKCMjIy7Nqys7NltVqVk5OjnBzH74yXsvVVpW59za7tfEhklQxDMpkkmfMtdO7X5DFVajLC4f1czDAM5ebmSsq/iDpKLicnR1arVadPn5aHh0dZlwOglFitViUlJckwDJnN3JQWAIqDuRQASq4izaUpKSkO9avwodTYsWMVFxdne5ycnKxq1aopJCRE/v7+dn0zMjKUkpIid3d3ubsX4aXJSVNu2rGiF5eTVrT9XAKBSelwd3eX2WxWcHCwvLy8Ch8AoFyyWq0ymUwKCQkp9//xB4CywlwKACVXkeZSR/8NXeFDKYvFIovFkq/dbDbne5PN5vNnM+X9OMrN4i83vwgZORmyZpyRdLlL6UwyewXJ5O4lN4t/ic5wMgzDNp4zpQq3du1a3XLLLTp79qwCAwML7Z/3OSjoswKgYuG7DgAlx1wKACVXUeZSR+uv8KGUKwQ0HSmPoPo6+WV3SSYVFkpZMxMV1ulL+UTe7qIKAQAAAAAArixlGr2lpqZq69at2rp1qyTp4MGD2rp1q44cOSLp/KV3/fr1sxuT1z81NVUJCQnaunWrdu7c6erS7eRmJCp+Re/z60epsMW8zq8zFb+it3IzEl1QXcGys7PLbN+Xk5WVVdYlAAAAAAAAFyjTUGrTpk2KiYlRTEyMJCkuLk4xMTEaP368JOn48eO2gCpPXv/Nmzfro48+UkxMjLp06eLy2i+UuusDGdnpKjyQymOVkZ2u1F0fOrUOq9Wql19+WbVq1ZLFYlH16tX1wgsv6NChQzKZTFq8eLHatWsnLy8vLVq0SJL07rvvql69evLy8tJ1112nN954w6F95T3nZ599pltuuUU+Pj5q3LixNmzYYNfv008/1fXXXy+LxaLIyEjNmDHDbntkZKQmT56sfv36yd/fXw899JDee+89BQYGavny5apbt658fHx0zz33KD09XQsXLlRkZKQqV66sESNG2BZ6l6QPPvhAzZs3V6VKlVSlShXdd999io+PL+GrCgAAAAAASkOZXr7Xvn37/79LXcHee++9fG2X618WDMNQ8tbXizU2eesc+TcZ5rT1oMaOHau5c+fqlVdeUZs2bXT8+HHt3r3btv3pp5/WjBkzFBMTYwumxo8frzlz5igmJka///67hgwZIl9fX/Xv39+hfT777LOaPn26ateurWeffVZ9+vTRvn375O7urs2bN6tXr156/vnn1bt3b/3yyy969NFHFRwcrAEDBtieY/r06Ro/frwmTJggSfr555+Vnp6uV199VZ988olSUlJ011136c4771RgYKC+/vprHThwQHfffbdat26t3r17Szp/9tfkyZNVt25dxcfHKy4uTgMGDNDXX3/tlNcXAAAAAAA4D2tKlZA147Rykg4UY6ShnKQDsmackZt3cInrSElJ0ezZszVnzhxboBQdHa02bdro0KFDkqSRI0fqrrvuso2ZMGGCZsyYYWuLiorSzp079fbbbzscSo0ZM0Zdu3aVJE2cOFHXX3+99u3bp+uuu04zZ87UbbfdpnHjxkmS6tSpo507d2ratGl2odStt96q0aNH2x7//PPPys7O1ptvvqno6GhJ0j333KMPPvhAJ0+elJ+fn+rXr69bbrlFP/zwgy2UGjRokO05atasqVdffVU33HCDUlNT5efnV5SXEwAAAAAAlLLyvZz7FcCalVrC8SlOqWPXrl3KzMzUbbfddsk+zZs3t/2elpam/fv368EHH5Sfn5/t59///rf279/v8H4bNWpk+71q1aqSZLtkbteuXWrdurVd/9atW+uvv/6yu+zuwrry+Pj42AIpSQoLC1NkZKRduBQWFmZ3ed7mzZvVrVs3Va9eXZUqVVK7du0kKd8loAAAAAAAoOxxplQJmT1LdgaO2bOSU+rw9vYutI+vr6/t99TU82Ha3Llz1aJFC7t+bm5uDu/Xw8PD9nveZYhWq6Nra+Wvq6DnzXvugtry9pWWlqbY2FjFxsZq0aJFCgkJ0ZEjRxQbG8vi6QAAAAAAXIEIpUrI7BUs94Caykk6KKko612Z5B4QJbNXkFPqqF27try9vbV69WoNHjy40P5hYWEKDw/XgQMH1LdvX6fUcLF69epp/fr1dm3r169XnTp1ihR8OWL37t06ffq0XnzxRVWrVk3S+YX0AQAAAADAlYlQqoRMJpP8mwzTmR/HFHmsf5PhTlvk3MvLS0899ZSefPJJeXp6qnXr1kpISNCOHTsueUnfxIkTNWLECAUEBKhTp07KzMzUpk2bdPbsWcXFxZW4ptGjR+uGG27Q5MmT1bt3b23YsEFz5sxx+A5/RVG9enV5enrqtdde09ChQ7V9+3ZNnjzZ6fsBAAAAAADOwZpSTuBX7wGZPHzk+MtplsnDR3717ndqHePGjdPo0aM1fvx41atXT71797Zbc+ligwcP1rvvvqsFCxaoYcOGateund577z1FRUU5pZ6mTZtqyZIl+uSTT9SgQQONHz9ekyZNslvk3FlCQkL03nvvaenSpapfv75efPFFTZ8+3en7AQAAAAAAzmEyDKMo15yVe8nJyQoICFBSUpL8/f3ttmVkZOjgwYOKioqSl5dXkZ43/dB3Ovlld8kwJF1uTSWzZDIprMd/5FOjY9EP4AKGYSgnJ0fu7u5OO+MK/yjJ5wFA+WG1WhUfH6/Q0FCZzfytBgCKg7kUAEquIs2ll8teLlS+j/IKkbRllk59/7DMlkAVvq6UIbMlUKdWPaSkLbNKvzgAAAAAAIArEKGUE1gzk5WbelTWjNNyJJSyZpw+3z8z2RXlFcuUKVPk5+dX4E/nzp3LujwAAAAAAFDOsdC5E5gt/nLzi7BrM/Iu4zMMyWSSZM53iZ3ZculT2Mra0KFD1atXrwK3eXt7u7gaAAAAAABQ0RBKOUFA05EKaDqyrMtwqqCgIAUFBZV1GQAAAAAAoILi8j0AAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC7HmlJO8Mr2H/XKjp/s2gzDkFWGbZ1zs0z5Fjofdf3NGtWgnStLBQAAAAAAuCIQSjlBcnaGjqYnFWscAAAAAADA1YjL95zA38NLET4BCrb4yFRIX5OkYIuPInwC5O/h5bQaDMPQQw89pKCgIJlMJm3dujVfn/bt22vkyJFO2ycAAAAAAEBxEUo5wagG7TS3TS8lZp2TqZBYyiSTErPOaW6bXk69dG/lypV67733tHz5ch0/flwNGjRw2nOXhUOHDl0yXAMAAAAAAOUfoZQTJGaeU881C2UYklXGZfvmrTPVc81CJWaec1oN+/fvV9WqVdWqVStVqVJF7u5cmQkAAAAAAK5chFJO8P6+TUrPySo0kMpjlaH0nCx9sH+TU/Y/cOBAPfbYYzpy5IhMJpMiIyOVlpamfv36yc/PT1WrVtWMGTOK9JyZmZkaM2aMIiIi5OvrqxYtWmjt2rWSpIyMDF1//fV66KGHbP3379+vSpUqaf78+ZKk06dPq0+fPoqIiJCPj48aNmyojz/+2G4fVqtVL7/8smrVqiWLxaLq1avrhRdekCRFRUVJkmJiYmQymdS+fftivjoAAAAAAOBKRChVQoZhaM6udcUa+9rOdTIMx4Ksy5k1a5YmTZqka6+9VsePH9fGjRv1xBNP6Mcff9SXX36p7777TmvXrtWWLVscfs7hw4drw4YN+uSTT7Rt2zb17NlTnTp10l9//SUvLy8tWrRICxcu1Jdffqnc3Fzdf//96tixowYNGiTpfHDVrFkzrVixQtu3b9dDDz2kBx54QL/99pttH2PHjtWLL76ocePGaefOnfroo48UFhYmSbZ+33//vY4fP67PPvusxK8TAAAAAAC4cnCNVwmdzkzX/pTTRR5nSNqfclpnMtMV7OVbohoCAgJUqVIlubm5qUqVKkpNTdW8efP04Ycf6rbbbpMkLVy4UNdee61Dz3fkyBEtWLBAR44cUXh4uCRpzJgxWrlypRYsWKApU6aoSZMm+ve//63Bgwfr3nvv1eHDh7V8+XLbc0RERGjMmDG2x4899pi+/fZbLVmyRDfeeKNSUlI0e/ZszZkzR/3795ckRUdHq02bNpKkkJAQSVJwcLCqVKlSotcHAAAAAABceQilSig1O7NE41OyM0scSl1s//79ysrKUosWLWxtQUFBqlu3rkPj//zzT+Xm5qpOnTp27ZmZmQoODrY9Hj16tL744gvNmTNH33zzjd223NxcTZkyRUuWLNHRo0eVlZWlzMxM+fj4SJJ27dqlzMxMW2gGAAAAAACuLoRSJeTnYSnR+EolHF8aUlNT5ebmps2bN8vNzc1um5+fn+33+Ph47d27V25ubvrrr7/UqVMn27Zp06Zp9uzZmjVrlho2bChfX1+NHDlSWVlZkiRvb2/XHAwAAAAAALgisaZUCQVbfBRdKVimIo4zSYquFKwgi4/Ta4qOjpaHh4d+/fVXW9vZs2e1d+9eh8bHxMQoNzdX8fHxqlWrlt3PhZfSDRo0SA0bNtTChQv11FNPadeuXbZt69evV/fu3XX//fercePGqlmzpt3+a9euLW9vb61evbrAGjw9PSWdP+MKAAAAAABUPJwpVUImk0nD67VR3G9fFnnsY/XbyGQqapxVOD8/Pz344IN64oknFBwcrNDQUD377LMymx3LIOvUqaO+ffuqX79+mjFjhmJiYpSQkKDVq1erUaNG6tq1q15//XVt2LBB27ZtU7Vq1bRixQr17dtX//3vf+Xp6anatWtr2bJl+uWXX1S5cmXNnDlTJ0+eVP369SVJXl5eeuqpp/Tkk0/K09NTrVu3VkJCgnbs2KEHH3xQoaGh8vb21sqVK3XttdfKy8tLAQEBTn+tAAAAAABA2eBMKSfoV6u5fNw9ZXbwfCmzTPJx99QD0c1LraZp06apbdu26tatmzp06KA2bdqoWbNmDo9fsGCB+vXrp9GjR6tu3brq0aOHNm7cqOrVq2v37t164okn9MYbb6hatWqSpDfeeEOnTp3SuHHjJEnPPfecmjZtqtjYWLVv315VqlRRjx497PYxbtw4jR49WuPHj1e9evXUu3dvxcfHS5Lc3d316quv6u2331Z4eLi6d+/unBcGAAAAAABcEUyGYRhlXYQrJScnKyAgQElJSfL397fblpGRoYMHDyoqKkpeXl5Fet5vj+5Rt1XvyjAkqy79kpplkskkLe84WLdHOLbw+KUYhqGcnBy5u7uXyhlXV7uSfB4AlB9Wq1Xx8fEKDQ11+IxSAIA95lIAKLmKNJdeLnu5UPk+yivEK9t/1JB1SxTo6S3jMoGUJBkyFOjprcHrluiV7T+6qEIAAAAAAIArC2tKOUFydoaOpic51NeQdDoz3TauLPz888/q3LnzJbenpqa6sBoAAAAAAHA1IpRyAn8PL0X42C/CbRiGrDJkGJLJlHfZninfuLLQvHlzbd26tUz2DQAAAAAAIBFKOcWoBu00qkG7si7DYd7e3qpVq1ZZlwEAAAAAAK5irCkFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlWFPKCU6vnKnTK2fatRmGIRlW2VY6N5nzLXQe3ClOwZ3iXFkqAAAAAADAFYFQyglyzyUr5+zRYo0DAAAAAAC4GhFKOYGbt7/cK0fImp0ha+oZScZleptk9guS2cNLbt7+rioRAAAAAADgikIo5QTBneJkibheR2Z2PX+pnnGZUMpkkjU9UdfGrZBfw1jXFQkAAAAAAHAFYaFzJ8hNS9Tfr919PowyrJfv/P/rTP392t3KTUt0SX0VRXZ2dlmXAAAAAAAAnIRQygkS1y+UkZleeCCVx7DKyEpX4vr3nbL/5cuXKzAwULm5uZKkrVu3ymQy6emnn7b1GTx4sO6//35J0ty5c1WtWjX5+Pjozjvv1MyZMxUYGOjQvvbv36/u3bsrLCxMfn5+uuGGG/T999/b9YmMjNTkyZPVp08f+fr6KiIiQq+//rpdH5PJpDfffFOdO3eWt7e3atasqWXLltm2Hzp0SCaTSYsXL1a7du3k5eWlRYsWyWq1atKkSbr22mtlsVjUpEkTrVy5UtL5xeU7dOig2NjY8wvNSzpz5oyuvfZajR8/vmgvKgAAAAAAKFWEUiVkGIbOrHqtWGPPrHrVFp6URNu2bZWSkqLff/9dkvTjjz/qmmuu0dq1a219fvzxR7Vv317r16/X0KFD9fjjj2vr1q3q2LGjXnjhBYf3lZqaqi5dumj16tX6/fff1alTJ3Xr1k1Hjhyx6zdt2jQ1btxYv//+u55++mk9/vjjWrVqlV2fcePG6e6779Yff/yhvn376t5779WuXbvs+uSN3bVrl2JjYzV79mzNmDFD06dP17Zt2xQbG6t//etf+uuvv2QymbRw4UJt3LhRr776qiRp6NChioiIIJQCAAAAAOAKYzKckYqUI8nJyQoICFBSUpL8/e0XGs/IyNDBgwcVFRUlLy8vh54vJ+WU9g4PKXY9dV4/JXe/4GKNNQxDOTk5cnd3V/PmzdWnTx+NGTNGd955p2644QZNnDhRp0+fVlJSkq699lrt3btX48aNU2pqqpYvX257nvvvv1/Lly9XYmJisepo0KCBhg4dquHDh0s6f6ZUvXr19M0339j63HvvvUpOTtbXX38t6fyZUkOHDtWbb75p63PTTTepadOmeuONN3To0CFFRUVp1qxZevzxx219IiIiNGzYMD3zzDO2thtvvFE33HCD7WyspUuXql+/fho5cqRee+01/f7776pdu3aRj6s4nwcA5Y/ValV8fLxCQ0NlNvO3GgAoDuZSACi5ijSXXi57uVD5PsorgDUjtWTjz6U4pY527dpp7dq1MgxDP//8s+666y7Vq1dP69at048//qjw8HDVrl1be/bs0Y033mg39uLHl5OamqoxY8aoXr16CgwMlJ+fn3bt2pXvTKmWLVvme3zxWVCO9GnevLnt9+TkZB07dkytW7e269O6dWu7cT179tSdd96pF198UdOnTy9WIAUAAAAAAEoXd98rIbOXX8nGe1dySh3t27fX/Pnz9ccff8jDw0PXXXed2rdvr7Vr1+rs2bNq166dU/YzZswYrVq1StOnT1etWrXk7e2te+65R1lZWU55/ov5+voWeUx6ero2b94sNzc3/fXXX6VQFQAAAAAAKCnOlCohN79geYRGSyZT0QaaTPIIjZabb5BT6shbV+qVV16xBVB5odTatWvVvn17SVLdunW1ceNGu7EXP76c9evXa8CAAbrzzjvVsGFDValSRYcOHcrX77///W++x/Xq1Stynwv5+/srPDxc69evz1dT/fr1bY9Hjx4ts9msb775Rq+++qrWrFnj6OEBAAAAAAAX4UypEjKZTArq+JhOLhpV5LFBHUfIVNQw6xIqV66sRo0aadGiRZozZ44k6eabb1avXr2UnZ1tC6oee+wx3XzzzZo5c6a6deumNWvW6JtvvnG4jtq1a+uzzz5Tt27dZDKZNG7cOFmt+e86uH79er388svq0aOHVq1apaVLl2rFihV2fZYuXarmzZurTZs2WrRokX777TfNmzfvsvt/4oknNGHCBEVHR6tJkyZasGCBtm7dqkWLFkmSVqxYofnz52vDhg1q2rSpnnjiCfXv31/btm1T5cqVHTpGAAAAAABQ+jhTygkCW/eXyeIjmRx8OU1mmTx9FNi6n1PraNeunXJzc21nRQUFBal+/fqqUqWK6tatK+n8+ktvvfWWZs6cqcaNG2vlypUaNWqUwwt5z5w5U5UrV1arVq3UrVs3xcbGqmnTpvn6jR49Wps2bVJMTIz+/e9/a+bMmYqNjbXrM3HiRH3yySdq1KiR3n//fX388cd2ZzwVZMSIEYqLi9Po0aPVsGFDrVy5Uv/5z39Uu3ZtJSQk6MEHH9Tzzz9vq2nixIkKCwvT0KFDHTo+AAAAAADgGtx97wIludta6p/f6sjMrpJhSEb+M4dsTGbJZFL1uK/l1/D24hyCzYV33yvJGVdDhgzR7t279fPPP5eonjyRkZEaOXKkRo4ceck+JpNJn3/+uXr06OGUfZYG7r4HXB0q0l1OAKCsMJcCQMlVpLmUu++50OmVM3Vs3oMy+wSeD6UuxzBk9gnUsXmDdHrlTJfUd7Hp06frjz/+0L59+/Taa69p4cKF6t+/f5nUAgAAAAAArk6sKeUEueeSlXP2qIO9DVlTT8v6/+PKwm+//aaXX35ZKSkpqlmzpl599VUNHjxYknT99dfr8OHDBY57++231bdvX1eWCgAAAAAAKihCKSdw8/aXe+UIuzYj7zI+wzh/Zz6TOd8ldm7elz6FrTQtWbLkktu+/vprZWdnF7gtLCzMoecv6G58F7vKrhoFAAAAAAAXIZRyguBOcQruFFfWZThFjRo1yroEAAAAAABwFWBNKQAAAAAAALgcoVQBuLQMEp8DAAAAAABKE6HUBTw8PCRJ6enpZVwJrgR5n4O8zwUAAAAAAHCeMl1T6qefftK0adO0efNmHT9+XJ9//rl69Ohx2TFr165VXFycduzYoWrVqum5557TgAEDnFKPm5ubAgMDFR8fL0ny8fHJtzj5lcQwDOXk5Mjd3f2KrrO8MQxD6enpio+PV2BgoNzc3Mq6JAAAAAAAKpwyDaXS0tLUuHFjDRo0SHfddVeh/Q8ePKiuXbtq6NChWrRokVavXq3BgweratWqio2NdUpNVapUkSRbMHUlMwxDVqtVZnP+O/uh5AIDA22fBwAAAAAA4FxlGkp17txZnTt3drj/W2+9paioKM2YMUOSVK9ePa1bt06vvPKK00Ipk8mkqlWrKjQ0VNnZ2U55ztJitVp1+vRpBQcHy2zmSkxn8vDw4AwpAAAAAABKUZmGUkW1YcMGdejQwa4tNjZWI0eOdPq+3NzcrvhQwmq1ysPDQ15eXoRSAAAAAACgXClXodSJEycUFhZm1xYWFqbk5GSdO3dO3t7e+cZkZmYqMzPT9jg5OVnS+UDHarWWbsGlzGq12i7hAwAUD3MpAJQccykAlFxFmksdPYZyFUoVx9SpUzVx4sR87QkJCcrIyCiDipzHarUqKSlJhmFwphQAFBNzKQCUHHMpAJRcRZpLU1JSHOpXrkKpKlWq6OTJk3ZtJ0+elL+/f4FnSUnS2LFjFRcXZ3ucnJysatWqKSQkRP7+/qVab2mzWq0ymUwKCQkp9x9YACgrzKUAUHLMpQBQchVpLvXy8nKoX7kKpVq2bKmvv/7arm3VqlVq2bLlJcdYLBZZLJZ87Wazudy/ydL5hdkryrEAQFlhLgWAkmMuBYCSqyhzqaP1l+lRpqamauvWrdq6dask6eDBg9q6dauOHDki6fxZTv369bP1Hzp0qA4cOKAnn3xSu3fv1htvvKElS5Zo1KhRZVE+AAAAAAAAiqlMQ6lNmzYpJiZGMTExkqS4uDjFxMRo/PjxkqTjx4/bAipJioqK0ooVK7Rq1So1btxYM2bM0LvvvqvY2NgyqR8AAAAAAADFU6aX77Vv316GYVxy+3vvvVfgmN9//70UqwIAAAAAAEBpK98XKQIAAAAAAKBcIpQCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFyOUAoAAAAAAAAuRygFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlCKUAAAAAAADgcoRSAAAAAAAAcDlCKQAAAAAAALgcoRQAAAAAAABcjlAKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFyOUAoAAAAAAAAuRygFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlCKUAAAAAAADgcoRSAAAAAAAAcDlCKQAAAAAAALgcoRQAAAAAAABcjlAKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFyOUAoAAAAAAAAuRygFAAAAAAAAl7siQqnXX39dkZGR8vLyUosWLfTbb79dsm92drYmTZqk6OhoeXl5qXHjxlq5cqULqwUAAAAAAEBJlXkotXjxYsXFxWnChAnasmWLGjdurNjYWMXHxxfY/7nnntPbb7+t1157TTt37tTQoUN155136vfff3dx5QAAAAAAACiuMg+lZs6cqSFDhmjgwIGqX7++3nrrLfn4+Gj+/PkF9v/ggw/0zDPPqEuXLqpZs6YeeeQRdenSRTNmzHBx5QAAAAAAACiuMg2lsrKytHnzZnXo0MHWZjab1aFDB23YsKHAMZmZmfLy8rJr8/b21rp160q1VgAAAAAAADiPe1nu/NSpU8rNzVVYWJhde1hYmHbv3l3gmNjYWM2cOVM333yzoqOjtXr1an322WfKzc0tsH9mZqYyMzNtj5OTkyVJVqtVVqvVSUdSNqxWqwzDKPfHAQBlibkUAEqOuRQASq4izaWOHkOZhlLFMXv2bA0ZMkTXXXedTCaToqOjNXDgwEte7jd16lRNnDgxX3tCQoIyMjJKu9xSZbValZSUJMMwZDaX+ZWYAFAuMZcCQMkxlwJAyVWkuTQlJcWhfmUaSl1zzTVyc3PTyZMn7dpPnjypKlWqFDgmJCREX3zxhTIyMnT69GmFh4fr6aefVs2aNQvsP3bsWMXFxdkeJycnq1q1agoJCZG/v7/zDqYMWK1WmUwmhYSElPsPLACUFeZSACg55lIAKLmKNJdevOzSpZRpKOXp6almzZpp9erV6tGjh6Tzb8Lq1as1fPjwy4718vJSRESEsrOz9emnn6pXr14F9rNYLLJYLPnazWZzuX+TJclkMlWYYwGAssJcCgAlx1wKACVXUeZSR+sv88v34uLi1L9/fzVv3lw33nijZs2apbS0NA0cOFCS1K9fP0VERGjq1KmSpF9//VVHjx5VkyZNdPToUT3//POyWq168skny/IwAAAAAAAAUARlHkr17t1bCQkJGj9+vE6cOKEmTZpo5cqVtsXPjxw5YpewZWRk6LnnntOBAwfk5+enLl266IMPPlBgYGAZHQEAAAAAAACKymQYhlHWRbhScnKyAgIClJSUVCHWlIqPj1doaGi5P7UPAMoKcykAlBxzKQCUXEWaSx3NXop1lPv379dzzz2nPn36KD4+XpL0zTffaMeOHcWrFgAAAAAAAFeVIodSP/74oxo2bKhff/1Vn332mVJTUyVJf/zxhyZMmOD0AgEAAAAAAFDxFDmUevrpp/Xvf/9bq1atkqenp6391ltv1X//+1+nFgcAAAAAAICKqcih1J9//qk777wzX3toaKhOnTrllKIAAAAAAABQsRU5lAoMDNTx48fztf/++++KiIhwSlEAAAAAAACo2IocSt1777166qmndOLECZlMJlmtVq1fv15jxoxRv379SqNGAAAAAAAAVDBFDqWmTJmi6667TtWqVVNqaqrq16+vm2++Wa1atdJzzz1XGjUCAAAAAACggnEv6gBPT0/NnTtX48aN0/bt25WamqqYmBjVrl27NOoDAAAAAABABVTkUCpP9erVVb16dWfWAgAAAAAAgKtEkUOpQYMGXXb7/Pnzi10MAAAAAAAArg5FDqXOnj1r9zg7O1vbt29XYmKibr31VqcVBgAAAAAAgIqryKHU559/nq/NarXqkUceUXR0tFOKAgAAAAAAQMVW5LvvFfgkZrPi4uL0yiuvOOPpAAAAAAAAUME5JZSSpP379ysnJ8dZTwcAAAAAAIAKrMiX78XFxdk9NgxDx48f14oVK9S/f3+nFQYAAAAAAICKq8ih1O+//2732Gw2KyQkRDNmzCj0znwAAAAAAACAVIxQ6ocffiiNOgAAAAAAAHAVcdqaUgAAAAAAAICjHDpTKiYmRiaTyaEn3LJlS4kKAgAAAAAAQMXnUCjVo0ePUi4DAAAAAAAAVxOHQqkJEyaUdh0AAAAAAAC4irCmFAAAAAAAAFyuyHffy83N1SuvvKIlS5boyJEjysrKstt+5swZpxUHAAAAAACAiqnIZ0pNnDhRM2fOVO/evZWUlKS4uDjdddddMpvNev7550uhRAAAAAAAAFQ0RQ6lFi1apLlz52r06NFyd3dXnz599O6772r8+PH673//Wxo1AgAAAAAAoIIpcih14sQJNWzYUJLk5+enpKQkSdIdd9yhFStWOLc6AAAAAAAAVEhFDqWuvfZaHT9+XJIUHR2t7777TpK0ceNGWSwW51YHAAAAAACACqnIodSdd96p1atXS5Iee+wxjRs3TrVr11a/fv00aNAgpxcIAAAAAACAisfhu+/NmTNH999/v1588UVbW+/evVW9enVt2LBBtWvXVrdu3UqlSAAAAAAAAFQsDp8p9eyzzyo8PFx9+/bVmjVrbO0tW7ZUXFwcgRQAAAAAAAAc5nAodeLECb311ls6duyYOnbsqKioKE2ePFl///13adYHAAAAAACACsjhUMrb21v9+vXTDz/8oL/++ksPPPCA5s2bp6ioKHXq1ElLly5VdnZ2adYKAAAAAACACqLIC51LUs2aNTVp0iQdPHhQ33zzjYKDgzVgwABFREQ4uz4AAAAAAABUQMUKpfKYTCa5u7vLZDLJMAzOlAIAAAAAAIBDihVK/f3335o0aZJq1qypjh076tixY5o7d66OHz/u7PoAAAAAAABQAbk72jErK0ufffaZ5s+frzVr1qhq1arq37+/Bg0apJo1a5ZmjQAAAAAAAKhgHA6lqlSpovT0dN1xxx366quvFBsbK7O5RFf/AQAAAAAA4CrlcCj13HPP6YEHHlBISEhp1gMAAAAAAICrgMOhVFxcXGnWAQAAAAAAgKsI198BAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHC5IodSkyZNUnp6er72c+fOadKkSU4pCgAAAAAAABVbkUOpiRMnKjU1NV97enq6Jk6c6JSiAAAAAAAAULEVOZQyDEMmkylf+x9//KGgoCCnFAUAAAAAAICKzd3RjpUrV5bJZJLJZFKdOnXsgqnc3FylpqZq6NChpVIkAAAAAAAAKhaHQ6lZs2bJMAwNGjRIEydOVEBAgG2bp6enIiMj1bJly1IpEgAAAAAAABWLw6FU//79JUlRUVFq1aqVPDw8Sq0oAAAAAAAAVGwOh1J52rVrJ6vVqr179yo+Pl5Wq9Vu+8033+y04gAAAAAAAFAxFTmU+u9//6v77rtPhw8flmEYdttMJpNyc3OdVhwAAAAAAAAqpiKHUkOHDlXz5s21YsUKVa1atcA78QEAAAAAAACXU+RQ6q+//tKyZctUq1at0qgHAAAAAAAAVwFzUQe0aNFC+/btK41aAAAAAAAAcJUo8plSjz32mEaPHq0TJ06oYcOG+e7C16hRI6cVBwAAAAAAgIqpyKHU3XffLUkaNGiQrc1kMskwDBY6BwAAAAAAgEOKHEodPHiwNOoAAAAAAADAVaTIoVSNGjVKow4AAAAAAABcRYq80LkkffDBB2rdurXCw8N1+PBhSdKsWbP05ZdfFquI119/XZGRkfLy8lKLFi3022+/Xbb/rFmzVLduXXl7e6tatWoaNWqUMjIyirVvAAAAAAAAuF6RQ6k333xTcXFx6tKlixITE21rSAUGBmrWrFlFLmDx4sWKi4vThAkTtGXLFjVu3FixsbGKj48vsP9HH32kp59+WhMmTNCuXbs0b948LV68WM8880yR9w0AAAAAAICyUeRQ6rXXXtPcuXP17LPPys3NzdbevHlz/fnnn0UuYObMmRoyZIgGDhyo+vXr66233pKPj4/mz59fYP9ffvlFrVu31n333afIyEjdfvvt6tOnT6FnVwEAAAAAAODKUeRQ6uDBg4qJicnXbrFYlJaWVqTnysrK0ubNm9WhQ4d/CjKb1aFDB23YsKHAMa1atdLmzZttIdSBAwf09ddfq0uXLkXaNwAAAAAAAMpOkRc6j4qK0tatW/MteL5y5UrVq1evSM916tQp5ebmKiwszK49LCxMu3fvLnDMfffdp1OnTqlNmzYyDEM5OTkaOnToJS/fy8zMVGZmpu1xcnKyJMlqtcpqtRap3iuN1WqVYRjl/jgAoCwxlwJAyTGXAkDJVaS51NFjKHIoFRcXp2HDhikjI0OGYei3337Txx9/rKlTp+rdd98tcqFFtXbtWk2ZMkVvvPGGWrRooX379unxxx/X5MmTNW7cuHz9p06dqokTJ+ZrT0hIKPeLo1utViUlJckwDJnNxVqzHgCuesylAFByzKUAUHIVaS5NSUlxqJ/JMAyjqE++aNEiPf/889q/f78kKTw8XBMnTtSDDz5YpOfJysqSj4+Pli1bph49etja+/fvr8TExALv5te2bVvddNNNmjZtmq3tww8/1EMPPaTU1NR8b1xBZ0pVq1ZNZ8+elb+/f5HqvdJYrVYlJCQoJCSk3H9gAaCsMJcCQMkxlwJAyVWkuTQ5OVmVK1dWUlLSZbOXIp8pJUl9+/ZV3759lZ6ertTUVIWGhharSE9PTzVr1kyrV6+2hVJWq1WrV6/W8OHDCxyTnp6e783JW3C9oHzNYrHIYrHkazebzeX+TZYkk8lUYY4FAMoKcykAlBxzKQCUXEWZSx2tv1ihVB4fHx/5+PiU5CkUFxen/v37q3nz5rrxxhs1a9YspaWlaeDAgZKkfv36KSIiQlOnTpUkdevWTTNnzlRMTIzt8r1x48apW7dudncDBAAAAAAAwJXLoVCqadOmWr16tSpXrqyYmBiZTKZL9t2yZUuRCujdu7cSEhI0fvx4nThxQk2aNNHKlStti58fOXLELmF77rnnZDKZ9Nxzz+no0aMKCQlRt27d9MILLxRpvwAAAAAAACg7DoVS3bt3t10Cd+HaT84yfPjwS16ut3btWrvH7u7umjBhgiZMmOD0OgAAAAAAAOAaDoVSFwZAhEEAAAAAAAAoqSKvnLVx40b9+uuv+dp//fVXbdq0ySlFAQAAAAAAoGIrcig1bNgw/f333/najx49qmHDhjmlKAAAAAAAAFRsRQ6ldu7cqaZNm+Zrj4mJ0c6dO51SFAAAAAAAACq2IodSFotFJ0+ezNd+/Phxubs7tEQVAAAAAAAArnJFDqVuv/12jR07VklJSba2xMREPfPMM+rYsaNTiwMAAAAAAEDFVORTm6ZPn66bb75ZNWrUUExMjCRp69atCgsL0wcffOD0AgEAAAAAAFDxFDmUioiI0LZt27Ro0SL98ccf8vb21sCBA9WnTx95eHiURo0AAAAAAACoYIq1CJSvr68eeughZ9cCAAAAAACAq4RDodR//vMfde7cWR4eHvrPf/5z2b7/+te/nFIYAAAAAAAAKi6HQqkePXroxIkTCg0NVY8ePS7Zz2QyKTc311m1AQAAAAAAoIJyKJSyWq0F/g4AAAAAAAAUh9mRTkFBQTp16pQkadCgQUpJSSnVogAAAAAAAFCxORRKZWVlKTk5WZK0cOFCZWRklGpRAAAAAAAAqNgcunyvZcuW6tGjh5o1aybDMDRixAh5e3sX2Hf+/PlOLRAAAAAAAAAVj0Oh1IcffqhXXnlF+/fvlyQlJSVxthQAAAAAAACKzaFQKiwsTC+++KIkKSoqSh988IGCg4NLtTAAAAAAAABUXEVe6PyWW26Rp6dnqRYFAAAAAACAio2FzgEAAAAAAOByLHQOAAAAAAAAlyvyQucmk4mFzgEAAAAAAFAiLHQOAAAAAAAAl3MolLrQwYMHbb9nZGTIy8vLqQUBAAAAAACg4nNoofMLWa1WTZ48WREREfLz89OBAwckSePGjdO8efOcXiAAAAAAAAAqniKHUv/+97/13nvv6eWXX5anp6etvUGDBnr33XedWhwAAAAAAAAqpiKHUu+//77eeecd9e3bV25ubrb2xo0ba/fu3U4tDgAAAAAAABVTkUOpo0ePqlatWvnarVarsrOznVIUAAAAAAAAKrYih1L169fXzz//nK992bJliomJcUpRAAAAAAAAqNiKfPe98ePHq3///jp69KisVqs+++wz7dmzR++//76WL19eGjUCAAAAAACgginymVLdu3fXV199pe+//16+vr4aP368du3apa+++kodO3YsjRoBAAAAAABQwRT5TClJatu2rVatWuXsWgAAAAAAAHCVKFYoJUmbN2/Wrl27JEnXX38960kBAAAAAADAYUUOpeLj43Xvvfdq7dq1CgwMlCQlJibqlltu0SeffKKQkBBn1wgAAAAAAIAKpshrSj322GNKSUnRjh07dObMGZ05c0bbt29XcnKyRowYURo1AgAAAAAAoIIp8plSK1eu1Pfff6969erZ2urXr6/XX39dt99+u1OLAwAAAAAAQMVU5DOlrFarPDw88rV7eHjIarU6pSgAAAAAAABUbEUOpW699VY9/vjjOnbsmK3t6NGjGjVqlG677TanFgcAAAAAAICKqcih1Jw5c5ScnKzIyEhFR0crOjpaUVFRSk5O1muvvVYaNQIAAAAAAKCCKfKaUtWqVdOWLVv0/fffa/fu3ZKkevXqqUOHDk4vDgAAAAAAABVTkUMpSTKZTOrYsaM6duzo7HoAAAAAAABwFXD48r01a9aofv36Sk5OzrctKSlJ119/vX7++WenFgcAAAAAAICKyeFQatasWRoyZIj8/f3zbQsICNDDDz+smTNnOrU4AAAAAAAAVEwOh1J//PGHOnXqdMntt99+uzZv3uyUogAAAAAAAFCxORxKnTx5Uh4eHpfc7u7uroSEBKcUBQAAAAAAgIrN4VAqIiJC27dvv+T2bdu2qWrVqk4pCgAAAAAAABWbw6FUly5dNG7cOGVkZOTbdu7cOU2YMEF33HGHU4sDAAAAAABAxeTuaMfnnntOn332merUqaPhw4erbt26kqTdu3fr9ddfV25urp599tlSKxQAAAAAAAAVh8OhVFhYmH755Rc98sgjGjt2rAzDkCSZTCbFxsbq9ddfV1hYWKkVCgAAAAAAgIrD4VBKkmrUqKGvv/5aZ8+e1b59+2QYhmrXrq3KlSuXVn0AAAAAAACogIoUSuWpXLmybrjhBmfXAgAAAAAAgKuEwwudAwAAAAAAAM5CKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuNwVEUq9/vrrioyMlJeXl1q0aKHffvvtkn3bt28vk8mU76dr164urBgAAAAAAAAlUeah1OLFixUXF6cJEyZoy5Ytaty4sWJjYxUfH19g/88++0zHjx+3/Wzfvl1ubm7q2bOniysHAAAAAABAcZV5KDVz5kwNGTJEAwcOVP369fXWW2/Jx8dH8+fPL7B/UFCQqlSpYvtZtWqVfHx8CKUAAAAAAADKkTINpbKysrR582Z16NDB1mY2m9WhQwdt2LDBoeeYN2+e7r33Xvn6+pZWmQAAAAAAAHAy97Lc+alTp5Sbm6uwsDC79rCwMO3evbvQ8b/99pu2b9+uefPmXbJPZmamMjMzbY+Tk5MlSVarVVartZiVXxmsVqsMwyj3xwEAZYm5FABKjrkUAEquIs2ljh5DmYZSJTVv3jw1bNhQN9544yX7TJ06VRMnTszXnpCQoIyMjNIsr9RZrVYlJSXJMAyZzWV+JSYAlEvMpQBQcsylAFByFWkuTUlJcahfmYZS11xzjdzc3HTy5Em79pMnT6pKlSqXHZuWlqZPPvlEkyZNumy/sWPHKi4uzvY4OTlZ1apVU0hIiPz9/Ytf/BXAarXKZDIpJCSk3H9gAaCsMJcCQMkxlwJAyVWkudTLy8uhfmUaSnl6eqpZs2ZavXq1evToIen8m7B69WoNHz78smOXLl2qzMxM3X///ZftZ7FYZLFY8rWbzeZy/yZLkslkqjDHAgBlhbkUAEqOuRQASq6izKWO1l/ml+/FxcWpf//+at68uW688UbNmjVLaWlpGjhwoCSpX79+ioiI0NSpU+3GzZs3Tz169FBwcHBZlA0AAAAAAIASKPNQqnfv3kpISND48eN14sQJNWnSRCtXrrQtfn7kyJF8CduePXu0bt06fffdd2VRMgAAAAAAAErIZBiGUdZFuFJycrICAgKUlJRUIdaUio+PV2hoaLk/tQ8AygpzKQCUHHMpAJRcRZpLHc1eyvdRAgAAAAAAoFwilAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFyOUAoAAAAAAAAuRygFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlCKUAAAAAAADgcoRSAAAAAAAAcDlCKQAAAAAAALgcoRQAAAAAAABcjlAKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFyOUAoAAAAAAAAuRygFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlCKUAAAAAAADgcoRSAAAAAAAAcDlCKQAAAAAAALgcoRQAAAAAAABcjlAKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXuyJCqddff12RkZHy8vJSixYt9Ntvv122f2JiooYNG6aqVavKYrGoTp06+vrrr11ULQAAAAAAAErKvawLWLx4seLi4vTWW2+pRYsWmjVrlmJjY7Vnzx6Fhobm65+VlaWOHTsqNDRUy5YtU0REhA4fPqzAwEDXFw8AAAAAAIBiKfNQaubMmRoyZIgGDhwoSXrrrbe0YsUKzZ8/X08//XS+/vPnz9eZM2f0yy+/yMPDQ5IUGRnpypIBAAAAAABQQmV6+V5WVpY2b96sDh062NrMZrM6dOigDRs2FDjmP//5j1q2bKlhw4YpLCxMDRo00JQpU5Sbm+uqsgEAAAAAAFBCZXqm1KlTp5Sbm6uwsDC79rCwMO3evbvAMQcOHNCaNWvUt29fff3119q3b58effRRZWdna8KECfn6Z2ZmKjMz0/Y4OTlZkmS1WmW1Wp14NK5ntVplGEa5Pw4AKEvMpQBQcsylAFByFWkudfQYyvzyvaKyWq0KDQ3VO++8Izc3NzVr1kxHjx7VtGnTCgylpk6dqokTJ+ZrT0hIUEZGhitKLjVWq1VJSUkyDENm8xWxZj0AlDvMpQBQcsylAFByFWkuTUlJcahfmYZS11xzjdzc3HTy5Em79pMnT6pKlSoFjqlatao8PDzk5uZma6tXr55OnDihrKwseXp62vUfO3as4uLibI+Tk5NVrVo1hYSEyN/f34lH43pWq1Umk0khISHl/gMLAGWFuRQASo65FABKriLNpV5eXg71K9NQytPTU82aNdPq1avVo0cPSeffhNWrV2v48OEFjmndurU++ugjWa1W25u0d+9eVa1aNV8gJUkWi0UWiyVfu9lsLvdvsiSZTKYKcywAUFaYSwGg5JhLAaDkKspc6mj9ZX6UcXFxmjt3rhYuXKhdu3bpkUceUVpamu1ufP369dPYsWNt/R955BGdOXNGjz/+uPbu3asVK1ZoypQpGjZsWFkdAgAAAAAAAIqozNeU6t27txISEjR+/HidOHFCTZo00cqVK22Lnx85csQuYatWrZq+/fZbjRo1So0aNVJERIQef/xxPfXUU2V1CAAAAAAAACgik2EYRlkX4UrJyckKCAhQUlJShVhTKj4+XqGhoeX+1D4AKCvMpQBQcsylAFByFWkudTR7Kd9HCQAAAAAAgHKJUAoAAAAAAAAuRygFAAAAAAAAlyOUAgAAAAAAgMsRSgEAAAAAAMDlCKUAAAAAAADgcoRSAAAAAAAAcDlCKQAAAAAAALgcoRQAAAAAAABcjlAKAAAAAAAALkcoBQAAAAAAAJcjlAIAAAAAAIDLEUoBAAAAAADA5QilAAAAAAAA4HKEUgAAAAAAAHA597IuAAAAAED58Mr2H/XKjp8K3Jabmys3N7cCt426/maNatCuNEsDAJRDhFIAAAAAHJKcnaGj6UnFGgcAwMUIpQAAAAA4xN/DSxE+AXZthgwdS0+WJIV7+8tkMhU4DgCAixFKAQAAAHDIqAbt8l2Gl5adKf8Pn5Uk7bzzSVWyEEABABzDQucAAAAAAABwOUIpAAAAAMVmGIbt99MZaXaPAQC4HC7fAwAAAFBkiZnn9P6+TXp158+2tujPpiq6UrCG12ujfrWaK9DiXYYVAgCudIRSAAAAAIrk26N71HPNQqXnZOXbdiDltOJ++1LPbflGS2/tr9iIumVQIQCgPODyPQAAAAAO+/boHnVb9a7O5WTLkHTxxXp5bedystVt1bv69uge1xcJACgXCKUAAAAAOCQx85x6rlkow5Cs+eIoe1YZMgyp55qFSsw856IKAQDlCaEUAAAAAIe8v2+T0nOyCg2k8lhlKD0nSx/s31TKlQEAyiNCKQAAAACFMgxDc3atK9bY13au4658AIB8CKUAAAAAFOp0Zrr2p5x28BypfxiS9qec1pnM9NIoCwBQjhFKAQAAAChUanZmicanlHA8AKDiIZQCAAAAUCg/D0uJxlcq4XgAQMVDKAUAAACgUMEWH0VXCpapiONMkqIrBSvI4lMaZQEAyjFCKQAAAACFMplMGl6vTbHGPla/jUymosZZAICKjlAKAAAAgEP61WouH3dPmR08X8osk3zcPfVAdPNSrgwAUB4RSgEAAABwSKDFW0tv7S+TSYUGU2aZZDJJy27tr0CLt4sqBACUJ4RSAAAAABwWG1FXX3UcLG93D5mkfNFUXpu3u4eWdxys2yPqur5IAEC54F7WBQAAAAAoX2Ij6upIr3H6YP8mzd7xsw6mnrFtq1kpWI/Vb6N+tZorwJMzpAAAl0YoBQAAAKDIAi3eeqx+Ww2sdYMCFj0nSTpw9zOqXqkyi5oDABzC5XsAAAAAiu3CACrI4kMgBQBwGKEUAAAAAAAAXI7L9wAAAAA45JXtP+qVHT/ZtRkybL/X//zlAs+UGnX9zRrVoF2p1wcAKF8IpQAAAAA4JDk7Q0fTky65/di55EuOAwDgYoRSAAAAABzi7+GlCJ+AArfl5ubKzc3tkuMAALgYoRQAAAAAh4xq0K7Ay/CsVqvi4+MVGhoqs5llawEAjuG/GAAAAAAAAHA5QikAAAAAAAC4HKEUAAAAAAAAXI5QCgAAAAAAAC5HKAUAAAAAAACXI5QCAAAAAACAyxFKAQAAAAAAwOUIpQAAAAAAAOByhFIAAAAAAABwOUIpAAAAAAAAuByhFAAAAAAAAFzOvawLQOFm/rhfr/x0oMBt1txcmd3cCtw26uaaimsXXZqlAUC5kbRllpK2zM6/wZByrbn6n9lNMuXfHND0cQU0HVnq9QEAAABXG0KpciA5I0dHkzIu0yP7kuMAAOdZM5OVm3r0kttzLzMOAAAAgPMRSpUD/l7uigjwsmszDEPHkjMlSeH+FplM+f+87+/F2wsAecwWf7n5Rdg3GoZy045Jktx8q0qm/Fe1my3+rigPAAAAuOqYDMMwyrqI119/XdOmTdOJEyfUuHFjvfbaa7rxxhsL7Pvee+9p4MCBdm0Wi0UZGZc7k+gfycnJCggIUFJSkvz9y+8/NFIzsuX/3EpJ0oGxt6hGkG+BwRQA4NKs2Wk6/HplSVK1R07L3VKpjCsCgPLJarUqPj5eoaGhMptZthYAiqMizaWOZi9lfpSLFy9WXFycJkyYoC1btqhx48aKjY1VfHz8Jcf4+/vr+PHjtp/Dhw+7sOKylXguW7N/PqDGM3+ytdWc+oPqvLhGs38+oMRzBV/KBwAAAAAAcCUp81Bq5syZGjJkiAYOHKj69evrrbfeko+Pj+bPn3/JMSaTSVWqVLH9hIWFubDisvPtnnhVm7xKcV/u0KEz6XbbDpxOV9yXO1Rt8ip9u+fSgR4AAAAAAMCVoEwXHcrKytLmzZs1duxYW5vZbFaHDh20YcOGS45LTU1VjRo1ZLVa1bRpU02ZMkXXX399gX0zMzOVmZlpe5ycfH7B2j9n/CI/L9/L1ucTUUnR9zeya9v/4TalH00p9NjCWldXaJtqtse5mTnaOevXQsdJUs0HGso3/J/T25J2n9YnS//UYylpMiQVdL1lXtu57Fzd8e6v+s+gGxVbN0RHv9mnM9tOFrrPgOuCVb37dXZtu9/YqOyUrELHXtuplio3/icYzDiVpr/mbS10nCRd90hzefhbbI9PbTym42sOFjrO6xof1X4wxq7t0JIdSjmYWOjYa5qHq+ptUXZtf7603qF6I3tdr0pRgbbHKQcTdWjJDofGNnyqtd3j46sP6tSmY4WOqxQVqMhe9p/vv+b9roxT6ZcY8Y+qt0bpmhvCbY+zkzO1+81NDtVb+8Em8rrmn+/I2T9O6n8r9xU6zqOSp6579Aa7tiNf7lbS7tOFjg1qFKaIzrXs2nbM+q+smZdagvof1btfp4Drgm2P044l68AHfxY6TpLqj2whN8s/02H8ur91cv2RQsddSXPEkS93FzrObHHT9SNvsmtjjkiXv/f533bM/EZSFV3TPII5wgHMEcwRF6q4c0R+/H9EwXOEIUPn/LJ1InWvTDIxRzBH2GGOYI64GHNEwXPEhXOpb4R/uZ4jrFarQ/sr01Dq1KlTys3NzXemU1hYmHbvLvig69atq/nz56tRo0ZKSkrS9OnT1apVK+3YsUPXXnttvv5Tp07VxIkT87WnmtNlmC+/BlN2jvJdRpiUk6IMc+FfUI/0s1L8PxOgNStXKQ6Mk6RTZ04rzf2fNbKOnTqt0ZcJpC5kNc6f/tbz/U3a/HBjZWQmObRfa6aHvC461mQjTTnmwi8HPJ16Rtnx/7yWmYnnHD7W+NMJ8sjwtD1OTD3r0NjM/7/W9kKJ2clKd2CsOSNRbheNdfi9STylc/H//MczLTHZ8WO9aJ9nMhIdGpubbc7/ObSmKstc+DpqltSzssb/8zXPTstyuN6EM6dksabZHiennnForLuRna/es5nJSnNkv5lJ8rj4c6g0GebCJ7TTyaeVGf/Pf1DOnUlz/FgTEmT2dLM9PpPu2OfwSpkjUpMd+yyZlP+zdOYqnSNMSpWP2xr5mf9j2+7vfb9yrFWUcuJemf/3oEyeAbZtzBH5MUcwR1yoos0Rl8P/RxS8T0OGss1W5ZizZZKJOYI5wg5zBHPExZgjCp9Lc3JM5XqOSEkpPDyTyuHd91q2bKmWLVvaHrdq1Ur16tXT22+/rcmTJ+frP3bsWMXFxdkeJycnq1q1avKz+sjP6nPZffm4V1JoaKhdW4r7CXk4EPgF+VS2G5ubmaOEQvaX55qgYPmG/pNMvvdzvDJVeCCVxyrpXLZVK49k6i5LgGQtfEIJsPjnO9YzpsPKthb+14tgvyBVvmBshjlNidb/OVRraHCI3V8vzIdzlGlNLHScl9knX73pHglycyCNDfIKzDf2pKPvTeA1qhQaaHuckuapVGvhfx2SlG+fuV5psloL/3JX8sj/3iSZjyrDWvjVt0F+lXXNBWOzkzN1xurYGmwhQdfY/fXC47ihc9YzhY7zMHnmqzfDckZma+H/0xFkCcg3NkG+sloL/+tFsH+wAkIv+OtFTrKSrYX/dUiSQkJC7P56IZ9MZVsLn0SvlDnCcsZNadaEQseZ5Zav3mxL8lU3R7iZN8rH83lJmfnGuZlOyv30bGV+NVchXT6Rd42OkpgjCsIcwRxxoYo0RxSG/48oeI4wZOicNVveVg+ZZGKOYI6wwxzBHHEx5oiC54gL51Lfcj5HeHl5ObS/Mr37XlZWlnx8fLRs2TL16NHD1t6/f38lJibqyy+/dOh5evbsKXd3d3388ceF9i1vd98zDEN1XlyjA6fTHQ6lJMkkqWawj/Y+fSt35QOA/5d+6Dud/LK7ZBg6H+FfilkymRTW/Uv5RN7uqvIAoNyqSHeMAoCyUpHm0nJx9z1PT081a9ZMq1evtrVZrVatXr3a7myoy8nNzdWff/6pqlWrllaZZep0epb2FzGQks6fVbX/dLrOpHM3PgCQpNyMRMWv6O1AIKXz2w1D8St6Kzcj0QXVAQAAAFefMo/e4uLiNHfuXC1cuFC7du3SI488orS0NA0cOFCS1K9fP7uF0CdNmqTvvvtOBw4c0JYtW3T//ffr8OHDGjx4cFkdQqlKdWDRtctJycxxUiUAUL6l7vpARna6Cg+k8lhlZKcrddeHpVkWAAAAcNUq8zWlevfurYSEBI0fP14nTpxQkyZNtHLlStvi50eOHLE7be3s2bMaMmSITpw4ocqVK6tZs2b65ZdfVL9+/bI6hFLlZ3ErvNNlVLKU+VsMAGXOMAwlb329WGOTt86Rf5NhXAoNAAAAOFmZrilVFlhTCgCuPrnnTunI2+GFd7yE6g8fl5t3cOEdAeAqVZHWQQGAslKR5tJysaYUCmcymTS8TVSxxj7WJopACgAkWbNSSzjesVvaAgAAAHAcoVQ50L95Nfl4usnsYL5kNkk+nm7q17xa6RYGAOWE2dOvhOMrOakSAAAAAHkIpcqBQG8PLevfXCaZCg2mzCbJJJM+7d9cgd4erikQAK5wZq9guQfU1PmLm4vCJPf/a+/O46qq9v+Pvw6DjAIqoKIIzhOOqGCC13nIgRyyvNe0gbRUSC0zs7yVU5bpz6kSrbQ0S0sc8mqmZIpWKo5pyUUTJ9QQBRRBOOf8/uB79hWnurdkkPfzn3Kfs89Zm8d5fPZen/VZa3nWwM65/L1oloiIiIhIqaakVAnRta4vX0W2wsXRHhO3dqtsx1wc7Vkf2YoudX0Lv5EiIsWUyWTCo+mI/+lcj6YjNRVaREREROQeUFKqBOla15dTr3ZmVkRDAsu7FnitRgVXZkU05PSrnZWQEhG5Dff6j2FydOWP3/rsMDm64l5/0L1sloiIiIhIqeVQ1A2Q/46XiyPR4TV4sqU/Hq9sBODXl9tTrZybRvJFRO7C3tkL3x6fc35NBFjtAMtd3m0HJhO+PVdg7+xVSC0UERERESldlJQqAWZ+d4xZ244XOGa1Wo3/bzNv520TUqPb1mDM32re8/aJiJQUroFdqBixhgvrH8Gam/V/R603vCM/lpocXfDtuQLXgM6F3kYRERERkdJCSakSICM7jzPp2Xd8/WxGzh3PExGRglwDu+D/1K9c+XkpGfvmkpfxq/Gag2d1PJqOpGyDx7Bz8izCVoqIiIiI3P+UlCoBPJwdqOLpfNvXLGYzdvb2dzxPRERuZe/shWezkbg3fJyT7+bvrOf3xL8p41FNU6FFRERERAqJshYlwJi/1bztNDyLxcKFCxfw9fXFzk5r1ouI/LduTEDZO5dXQkpEREREpBApkyEiIiIiIiIiIoVOSSkRERERERERESl0SkqJiIiIiIiIiEih05pSIiJSKqTv/X+k751d8KDVavzv2Y+DwHTrWI1n8+fwbD7qHrdORERERKT0UVJKRERKBUtOBuYrZ+74uvlqyh3PExERERGRv56SUiIiUirYOXlg717l1hesYLaYsbezh9tsvmfn5HHvGyciIiIiUgopKSUiIqWCZ/NRt52GZ7FYuHDhAr6+vtjZaalFEREREZHCoqdvEREREREREREpdEpKiYiIiIiIiIhIoVNSSkRERERERERECp2SUiIiIiIiIiIiUuiUlBIRERERERERkUKnpJSIiIiIiIiIiBQ6JaVERERERERERKTQKSklIiIiIiIiIiKFTkkpEREREREREREpdEpKiYiIiIiIiIhIoVNSSkRERERERERECp2SUiIiIiIiIiIiUuiUlBIRERERERERkUKnpJSIiIiIiIiIiBQ6JaVERERERERERKTQKSklIiIiIiIiIiKFTkkpEREREREREREpdEpKiYiIiIiIiIhIoVNSSkRERERERERECp1DUTegsFmtVgAyMjKKuCV/nsViITMzE2dnZ+zslF8UEflfKJaKiPx5iqUiIn/e/RRLbTkXWw7mTkpdUiozMxMAf3//Im6JiIiIiIiIiMj9KzMzE09Pzzu+brL+XtrqPmOxWDh79ixly5bFZDIVdXP+lIyMDPz9/Tl16hQeHh5F3RwRkRJJsVRE5M9TLBUR+fPup1hqtVrJzMzEz8/vrlVfpa5Sys7OjqpVqxZ1M/5SHh4eJf4HKyJS1BRLRUT+PMVSEZE/736JpXerkLIp2ZMURURERERERESkRFJSSkRERERERERECp2SUiWYk5MT//znP3FycirqpoiIlFiKpSIif55iqYjIn1caY2mpW+hcRERERERERESKniqlRERERERERESk0CkpJSIiIiIiIiIihU5JKRERERERuSuLxVLUTRARkfuQklIlwIwZM+jVq1dRN0NEpET78MMPjViq5RRFRP47dnb/6TYohoqIyF9FSakSwNvbm2+//ZbffvutqJsiIlJi2dvbs379enJycjCZTEXdHBGREmXt2rU888wzWK1WxVAREfnLKClVzFgsllvKozt37sz169c5dOhQEbVKRKRksVqtt8TS1q1b4+LiQnx8fBG1SkSk5DCbzZjNZuPfbm5uxMTEkJqaSnx8PAsWLCjC1omIlHwnT55k9uzZHD16FCi9VahKShUTts6TnZ1dgfJogCpVqlC7dm02b95cFE0TESkxbLHUZDLdEksrV65Mw4YNWbt2LVB6b/wiIn+Evb099vb2xr+/++47TCYTVatWpVevXuzfv58rV64UYQtFREoG2zPn8ePHOX36tHH8ww8/ZM6cOVSvXh2LxVJqq1CVlCoiN448wX/m6X/33XfMmDGDbdu2kZeXZ7zepUsXvvnmmwLHRERKs9sllWyx9PDhw3z00Ufs3LnTeM3V1ZX27duzadOmQmujiEhxdbsYajuWnp7O4sWLGThwIKNGjQLyk/4eHh48+uijXLp0iffeew93d/fCbLKISIly9epVIH+w9NChQ/To0YNevXpx+vRp8vLy+PLLL3nuuecoU6bMLYOppUnpvfIiZht5OnfuHAAffPABtWrVYtCgQaxevZqhQ4cSGRlpvL979+4cOnSIs2fPFkl7RUSKC1un6ebRpNzcXFasWEFQUBBt27bl3Xff5amnniI6OhrIj7vt27cnMTGRixcvltrRKBEpvSwWizEwersYaDKZ+Pnnn+nduzdvv/023t7eVK5cmT179jB58mSee+459u/fT05OTmE3XUSkREhKSuKZZ56hWrVqdOjQgcmTJ2M2m2nUqBEJCQn4+voSHh7OO++8Q2pqqjY0Q0mpe8JqtZKXl3fbrXMzMjLYs2cPmzZtwtHRkYEDBwL5WdRJkyZx6tQp4uPjWbVqFR9//DE//PADACEhIdjZ2bF79+5CvRYRkaJktVoLVJbaFtjNzs4mLi6OhISEAu//9ddfGTNmDKdPn2b37t0sWLCABQsWsG/fPgDq16+Ph4cHW7ZsKdTrEBEpChaLpUBFlJ2dnTEwum/fPuLj48nKyjJez8rKYurUqWRnZ7N9+3bmzp3L888/T4sWLQBo164dP//8M2fOnCncCxERKaYOHjzIiy++yPr167FarUycOJFjx44xY8YMBg8ezBtvvMH48eP57bffcHV15csvv6RPnz5MmTKF0NBQypUrB5TuZSWUlLoHTCYTDg4O2NnZFehMZWVl8eKLL9K9e3cWLVrE6tWr+fTTTwHo378/AwcO5PTp08yePZvx48cDsGHDBrKysvD09KRZs2Z8/fXXRXJNIiKF4eZkvslkKlBZajKZ+Oabb6hfvz5DhgyhX79+fPHFF+Tl5eHo6Ej//v158sknyc3N5fPPP+eDDz4gNzeXVatWAVCxYkVatGjB6tWrgdL9ACAi95+bY5qdnV2BiqhTp04RFRVFxYoV6dy5M9HR0fTt25fU1FQALl++zIYNG3jhhRcoX748AA4ODsb5wcHBODs7G4OmiqEiUhplZ2fTu3dvPv30U15//XUOHDiAl5cXGzZsYOvWrYwcOZIBAwYwYsQIYmJi2Lx5s7F8hLu7O4MGDeLKlSscOnSIKVOmALevXi0tlJT6H928I8mNkpOTmTBhAs2aNaNLly4sWrSInJwcXF1dadKkCbm5udSqVYsePXpQuXJlrFYrfn5+LFmyhC5durBixQrCw8Pp06cP69atIzMzE4CuXbuybds2rl27VpiXKiJyT90YS2+cT3/y5EkuXrzIU089hbu7O3/7299YtGgRS5Ys4b333iM5OZnmzZvz7rvvcvDgQQBq1qzJ6tWrCQ0NZcqUKXh6etK3b1++/PJLrl+/jrOzM506deK7774DSvcDgIjcf26Oab/99hvDhw9nx44dAKSlpZGamsry5cs5f/48q1atIi0tjenTp3P9+nU8PT3JyMjAzc0NuHWgoGzZsoSGhvLVV1/d9vtERO43Fy9eZOXKlezatYvc3FwAnJ2dOX36NIMGDaJly5Z8/fXXtGnThl9++YXy5cvzwAMPGOe3bduWGjVqsH37duNYQkIC1atXZ8GCBSxatIjHH3/8trOsSgslpf4LN3acbt6RBPJHi6xWK6+++io7d+5k+PDh9OnTh1dffdXIgDZv3hwnJyf8/PwAyMvLw2Qy8dNPPzF58mQefvhhvvnmG1544QUeeughjh07xqlTpwB48MEHSUxMJCkpqZCuWETkr2WLkzeyxdJr166xbNkyXn75ZdasWUOLFi14/PHHqVq1KvHx8bRt25bo6GicnZ3p1q0bdnZ2PP/88+Tm5rJ161Ygfx7/2LFj6dOnD9u2bWPOnDn069eP48eP8+uvvwL5DwcpKSkkJiYW6rWLiPwVblwX6mY7duxg165dxr/j4uJYvnw5devWBSAwMJAZM2bQoUMHzp07x969e8nIyGDz5s0kJibi5uZGUFCQMaJvZ2dnxGzbTnt///vfWbduHS+++CKDBw9mzJgx9/JyRUQK3Z49e1i/fj3z58+nUaNGjB49mr///e+MHz/eiL99+/albNmyBdaEaty4MSdOnODatWtG7KxRowZOTk5cvnzZOHfu3LkMGzaMjh07snHjRnbs2EFISAhHjhwp/IstBpSUugOr1XpLttLWccrJyWHlypUMGDCAcePGcfz4cSB/tOi9994jKSmJ9evX8/TTTzNy5EiGDh3KwoUL2bx5M7Vr16ZFixYFHhgg/6b/66+/0r9/f1xdXUlLS2Px4sVkZmaSkJCAxWKhcePG1KtXzyixFhEpKW5cWPfmkfVPPvmE8ePHM2XKFObOnYubmxsVKlSgTZs27N69m379+tG0aVNeffVVWrduzW+//WacW69ePXx8fIw1owCOHTvG448/jpeXF1lZWXzyySdcv37d2ImvevXq1KhRg6NHjxbClYuI/LVuXBfqxIkTZGdnAxhrmfTs2ZO1a9cCMGfOHIYNG4a3tzdWqxVPT0/c3NwYOnQoLVq0YOrUqdSrVw+z2WzEyEGDBrFs2TLWrFljrOP3448/snTpUiA/KTV16lR2796Ns7MzjzzySBH8FURE/jopKSkFdrmPjY2lV69erFu3jtjYWI4dO8bzzz/PypUreeuttwAIDw8vsCYfQFhYGHl5ecTFxRV43k1ISKBu3brY29vz7bffkpqaSps2bYD8taO/+uorFi5cSIMGDUrltGiH339L6WKxWIz59zd3nL766ivi4uJwd3dn3759VK5cmY0bNxIfH8/ixYupXbs2x48fx8HBgbi4OGbPns3Bgwe5fv06PXr0oGLFinh4eNCoUSM2b94M/GeefoMGDfD29mbkyJF0796d3bt3M3DgQHJyckhPT8disVCmTJlSmz0VkZLN1oFKSEjg3LlzhIaGUqFCBQCuX7/O9OnTadGiBZ988gl169bl2rVr1KxZk927d9O4cWMA/Pz8CA0N5euvvyYvLw8HBwcqVKhgvO/s2bPUrFkTPz8/hg8fTv/+/UlISCAkJISsrCwjCVWpUiVVnIpIsbF3717MZjMtW7Y0kkB3k5SUxOTJk1m7di1eXl40bdqUqKgo2rdvz5YtW3jhhRcYO3YsixcvxmKx0KlTJ+A/G0V88MEHbN++nVWrVtG6dWsyMzMJCgoypkEPGzaMI0eO8NRTT9GpUyeSkpI4d+4cI0aM4Pr165QpU4aoqCiioqLu+d9GROSvZIuDqampXLp0iVmzZvHRRx9Rs2ZNunbtymuvvUbZsmUZMmQI06ZNo3LlyoSEhADw7LPPcuTIEf71r38RFRVFaGgozs7OHDhwgIYNG2KxWHB2dubRRx9lxowZXL16lcGDB7N8+XIAIwl19OhRHn/8cYKDg4122apZoXROiy7VlVJms/m2C0JCfsnewoULC4y+5+TksHjxYmJjY5k0aRIxMTEsXLiQzMxMYmNjAfD29mbHjh289tprBAcHs2LFCpKTk1m6dCmNGjXCwcGBBg0acO3aNfbu3Qvkd8gAVq9eTZ06dfjss8/w9/cnIiKC+Ph4XnzxxQKLTN6pZFtEpKjcrrr0RqtXr6ZKlSp07tyZCRMm0LNnT2NUvnv37lSoUIGGDRsaN2UXFxdCQkK4dOkS58+fB/KT+PXr1+fKlSsFdt1r2rQp169f54cffsBkMrFkyRJ8fX2ZNm0a165dY+DAgcTFxfHmm28WaJNiqYgUtatXrzJo0CAWLlwI3L0zYoux8+fP5/z583z11Vds3boVLy8vpk2bxrZt2wCYOnUq06ZNY8uWLZw/f55GjRoB+c+4WVlZ7Nixg1atWtG6dWsAtm7dSkZGBj/99BMXL17Ezc2NmJgYlixZQpUqVRg8eDA7d+5k/PjxlClTxmjP3XabFhEpjkwmE4cPH6Z27dq8/PLL5OXlsWnTJl544QUWL15sHKtTpw5ly5bFz8+vQAVVq1atuHr1KgcOHMDJyYkWLVoYG5HZ4vfUqVPp0aMH7777LoGBgYwfP56hQ4caAwTPPPMMU6dOxdnZuUDbSmOFlE2pqpSybS1uZ2dXoPT5Rvv37+fxxx/nzJkz1KlTh3nz5tG2bVvmzp1L27ZtCQgIoEqVKjRt2hTI/2HWr1+fPXv2cP36dfz9/fHx8eHjjz+mQYMGxueePHmS5ORkwsPDqVu3Lrm5uaxbt47mzZsb7QgNDSUkJOSWBxKz2Vygrbdrt4hIUbJVl6anp3Py5EkaNmxoJPnPnj3LSy+9RGRkJGPHjuXEiROMHTuWZ599lvj4ePz8/AgICMDR0ZGMjAw8PDwAqF27Nt7e3mzcuJEhQ4YA+SNJvr6+bN261Ri5qlevHtnZ2cYaUR07diQ8PLxA5wn+Uwlro1gqIoUhIyODpUuXsnTpUubOnUtwcLCxYY6bmxsPPPAAFy9e5PTp01StWvWO1VJ2dnZs376d+Ph4NmzYgLe3N9nZ2TRu3JgVK1YQGxtL27ZtKVOmDL1798be3p7s7GyefvppPvjgA3x8fHB1dcXX15ctW7awdOlSnJycWLduHR06dODSpUskJydToUIF7Ozs6NGjBz169Ljjddl2mxYRKY7i4uKIjY3l2rVrDBo0iHbt2gHQsGFDqlatyr/+9S82b95M69atCQ8Px2q1MnnyZCIiIujUqRMhISEcOHCA9PR0o7q/UqVK5ObmGkUlvXr1Yu7cuVy5cgV3d3cgv7J/+vTpDBw4EGdnZ+rVq3dL22w5iRtjfWmskLIpVZVStpunnZ0dOTk5nD59mhdeeMHY3e7q1auMGTOGTp06kZKSwo4dO/j4449ZsGABW7duxcfHhzp16uDk5MSFCxeMz23UqBFnz54lMTGRXr16ERAQQHR0NHFxcVy7do2DBw8yZcoUVq9ejcVioUaNGowcOZL27dsDBTtGJpMJi8VCXl6ekS1Vx0lEirNdu3aRkpJCREQElStXplevXowYMcKoRDp48CBpaWkMHjwYd3d3goKCePPNNzl37pxRZdqyZUuSkpIKrBfl5+dHkyZN2Lhxo3EsICAADw8PY+cnyK+UWr9+PS+99JJxrEyZMreM4t+YkBIRKSyTJk1i/vz5hIWFERAQAOQ/29kS582aNeO3337j559/Bu4+Wn79+nUSEhKYOnUqtWrVwsfHh4ULFzJy5EgjeQ/wxRdfUKlSJT755BNSU1OJiIgwdn4aP3483bp146WXXmLMmDGEhIQwf/58tmzZQvPmzQt8n21AtzSP4ItIyXLhwgUeffRRhg0bxrlz5zCbzXTt2pXFixeTk5MDQHBwMAEBAVSsWNE4LywsDH9/f+O5s1+/fmzZsqXArnl79+7lzJkzhIaGAtCuXTuSk5NJSUkp0Aar1UrTpk2NhNTNcdTe3r5UJ6FuVqqe0LOzs42beNeuXZk3bx4zZ8401hm5cOECqampvPTSS1y5coX333+fGTNmkJeXZ2wfHhwczIULFzh27JjxuS1btsRsNrNr1y48PDx49913cXBwYNSoUdSrV4/Q0FDOnTtHjx49MJlM+Pj4EB0dTdu2bW/bTjs7OxwcHPRDFZFi7/z584SGhvL0008biaXXX3+dxYsXs2jRIiD/Bm6rELWpUaMGLVq0MJJLXbp04cSJE8YOeQDlypUjODiY1atXG8d8fX0ZO3Ys8+bNM47Z29tTuXLlW9p240CEiEhRSE9PJz4+noiICN566y28vb0B+Prrr+nUqRMREREcP36czMxMIyl1t5jl4uJCuXLlSExM5M033+TQoUMcPnyYN998k6ZNmxqDAZ9//jnNmzenY8eOfP7559StW5d+/frx1VdfERAQwNtvv80vv/zCqVOnGDFiBJUqVcJkMt2SfDKZTOo8iUiJkpubS4cOHdiyZQsrV67ko48+Yvz48SxevNhYPicsLAyz2czp06eN8ypVqkSlSpU4c+YMABEREeTk5BAVFcWsWbN4+eWXef/995kwYQIuLi4ANG/enCtXrlC7du0CbbDFzBuLTBRH76xUPamvXbuWDz/8kJEjR/LOO+9w4sQJAGNdkx9//JGsrCyCgoIICAhgwYIF+Pr6snHjRkaPHg1A69atuXr1aoEFx5s1a4a9vT07duwA8hNXGzduZN68eXz22WdkZWWxZs0aOnToUODHqDn4IlKcmM3mAnHpj4yMV6xYkc6dO/Pjjz/So0cP/Pz8GDJkCI899hhLliwB8sukMzIySE5ONs5zc3OjatWqxhbjHTp0wGq1Foitjo6OhIeH8/TTT3P16lXj+N/+9jeaNGnyp69XRORe8/T0pEaNGsTFxfHYY4/x6KOPkpaWRlRUFIGBgTz11FOcPn2aw4cPc+TIkQLJ+9upWLEi5cqVo2XLlvTv35/AwEAATp8+zbJlyzhx4gT79u0jPj6egQMHAuDv78+CBQtISEigZ8+eADg7O+Pu7n5Ldb46TSJSXGVnZ5ORkQHc/RnVx8eHyMhIypYty+zZs+nevTvTp0/n6NGj7N+/H8hf6iE9PZ24uDjjPKvVys6dO2nTpg0Wi4VKlSpRrVo1unbtysmTJ9m/fz/jxo3j2WefLfB9rq6ud2yPYuofUyIngpvNZkwmkzGS9Hs7ldjWEZk3bx7BwcGMGjUKgJiYGDIzM1m3bh3R0dH4+/vj6OhI7969mTVrFi4uLsZ3XLt2DcjPhpYtW5affvrJ+HxfX19at25NYGAgubm5ODo6YrVaC1RC2UaubpyKp9F7ESkqtoXJb7deXVZWFqmpqVSrVu0PfVaLFi1ITk6mTp06xrE+ffqwdOlSTp06RXh4OI6OjqxatYquXbsCcO7cOVavXs0rr7yC2WzG09OTcuXKkZSUxNWrV3FzcwPyk1UdOnS4bft1oxeR4i46Oprly5djb29PnTp1GDZsGNOmTcPFxYU33ngDPz8/evfuTa9evThy5AjJycnUqlXrjjGuZs2aREVFMWXKFC5cuEBkZCRJSUl88cUXWK1W2rRpg729PcOHD6djx47GeY6Ojvj7+9/yebZ1VkVEiqOtW7eya9cuateuzWuvvcaDDz7ItGnTMJvNd1zTrkyZMmRkZPDkk09y8eJF2rVrx4QJE4iKiuLIkSPk5eVRvXp1AgMDeeeddyhXrhzdunVjxYoVODs7ExYWZsTFNm3akJyczKpVqyhbtuwd26ln0j+n2N+FbHPZb2Rvb2/sIHLy5Mnf/RHY2dlx4cKFAjdki8WCh4cHAwYMYPfu3aSnpxMUFESNGjU4ceIEbm5uxo/x+PHjREZGkpKSgpubGz4+Ply4cIGLFy8a3/H2228zYsQIHB0dgduX7GltKBEparZ4apuSYWO1Wvn0009p2LAhVapU4R//+AcTJ0404tzdRqS6dOnCr7/+WmCtvbZt25KXl8cPP/xA+fLlGTduHJ9++in/+Mc/WLBgAc8++ywNGjSgT58+Rju+/vpr5syZYySkbCwWyy2Vpbr5i0hJEB0dzeLFi6lSpQrDhw+nffv27N+/n+DgYPz8/IyY/PDDD5ORkfGH1pUaMWIEU6dO5fjx4/Tp04fRo0fj7e3NuHHjCAwMpHHjxrz++usFdnZSzBSR4swW886cOWMs5fDUU08RERFBQkICX375JYcOHTIq6n8vmR4TE8PBgweZOXMmb7zxBmFhYdjb2xvJf8ifwufu7s7+/fvp06cPMTExvPzyyzRq1Mh47nzkkUfYu3evMc3v96pZ5X9TbCulbDvO3a7jtHz5cqZMmcLZs2cJCgqiffv2PPfcc1SoUOGOI0tlypTBz8/PWITM9kMOCgoiMzOTPXv20LFjRyZOnEifPn0ICQnhwQcfJDExkX379lG7dm1j+sjChQspV67cHdt8Iz0EiEhRsSVzbpzHbotRGRkZzJ8/n2PHjvHPf/6TtLQ0Zs6cSWRkJA899BD79u1j3rx5XLp0iblz5971e5o3b06ZMmX44YcfqF69OlarFTc3N5o3b87WrVvp27cvDz/8MC4uLnzxxRfMnj2bsLAwJk+eTNWqVY3P8fHxue3naxRfREqqWrVqYTab8fPzIzY2ltDQUOrVq8euXbsAjBgdHh7O6NGj+eWXX+jVq9dd4569vT2RkZFGUt/Ly+uW91gsFmNXVBGR4iotLQ2z2YyPjw///ve/GTx4MP379yc8PJxPPvmEzZs307ZtW06dOsW1a9f49ttvycvLu2OVlG2G1MmTJ6lduzaenp4AfPbZZ6SkpJCens7u3bupWbMmoaGhbNy4kYcffpiYmBijuORGnTt35tKlSxw+fJj69evf9j3y5xWLJ/2b57NDwY7TtGnTiIyM5NSpU0bGMzIykr179zJ69Gji4+N57bXX7vodXl5e1KtXj/379xcYzf/++++xWq3s3r0bgJCQEDZt2sTAgQPZsWMHrq6uzJ49m9WrVxvl1LaE1M2jWKqEEpHi5MZNEy5fvkxaWhobN26kZcuWLFu2jM2bN9OkSROsViufffYZrVq1YvTo0VSvXp2aNWvi5OTE/PnzuXz58l07NmXLliUkJITNmzcXON66dWvWrl1rrBvVs2dPFi1axJEjR4iJiaFRo0b39PpFRIqDSpUq0bhxY2Pt0a5du5KQkEBycrLRwbl27Ro5OTkcPnzY2BX6RhaLBbPZbCxhYTabqVChAl5eXrfsNArcstW4iEhxcOMMqG+++QZvb2/efPNNAKpVq0ZKSgpBQUHExcXRvHlzY5c7f39/IiMjMZlMbN26Fbj9+sy2/vmDDz7IqVOnGDx4ML169SImJoaJEyfSv39/6tatC0CnTp24evUqJ0+eNJbfycvLMz7LYrHg4uLCtm3beOihh+7Fn0P+T7FISt3rjpPtx/nQQw/h6OjIwIED+eGHH9izZw+HDx8mODjY2AHKYrEQFBTEqFGj2LRpEwsXLqRz587GjiQ3fr5u9iJSnJ04cYLo6GiqVq1KaGgoH330EVWqVCEhIYFZs2bx3nvvERUVRbVq1YiNjeXSpUs8+OCDVKhQgc6dO1OmTBnmzZt3x9GoG7Vr147ly5dz5coVIzZOmDCB+Ph4Y5QKwMHB4bYdKBGR+5WnpydNmzYlOTmZ1NRUevbsSc2aNXnmmWfYuXMnly9fZv78+fj4+PDTTz+RmJgI5E8TsXXg7OzsjKUgcnNzsbe3N2KodhoVkZJgw4YN+Pj4GLOP0tPTCQwMZNmyZSQmJuLk5ERWVhYmk4msrCy8vLwK7I5Xq1Yt/P392bBhA3D7pJStSKRr1658/vnnNG7cmPr16/P2228ba/o1a9YMq9WKl5cXnp6exMXFkZaWZsRSGzs7O6xWK2FhYX/oWVj+d8Xi7nWvO062DlLDhg2ZP38+Tk5OdOvWjfbt2+Pv78+oUaM4dOgQUHCayM0dJyWhRKSkMJvNTJw4kaNHjzJz5kxWrFhBkyZNCAgIIDAwkPbt2xdYmLxx48bExsYSHBzMunXrSEpKYs2aNQwfPhx3d/ff/b5HH32UV155pUAc9vb2JiAg4Jb3qgMlIqVN3bp1cXNz49tvvwXg/fffx9HRkQEDBlChQgVyc3P54IMPePfdd2nWrBmQvzi5rYP1/fffM27cOBo2bGhMe1YMFZGSpFOnTri7uzNjxgwAzp8/zyOPPEL16tWJiYkhJSWFRo0acfLkScLCwjh79ixJSUnG+eXKlSMvL8+oOr3TLCVbMj8oKIgFCxbw1ltvERwcDPxnaQvbe6Kjo+nXr98dFzFX/79wFHnKz9ZxOn/+PDNnzqRevXpcuHDhdztOY8eO5ZVXXiEoKAgPD48//H1t2rRh3bp1HDt2zPjcJ554gho1apCenl5gRP/mbKmISEmxbNkyNm7cyJIlS+jevXuBSs9mzZpx6tQpzGazMcUjNDSUnTt38vTTTxfYde/7778nMTGRIUOG3PIdZrMZq9WKg4MDtWrV4uWXXy606xMRKUkCAwMpX748K1eu5OGHH6Z9+/YEBweze/du6tatW2B9Pcifzjdv3jzWrVvHoUOHcHFxoVWrVkRHRxMeHl5EVyEi8r9zdHRk1KhRrFq1ip49e+Lj48OmTZuYMWMGb7/9NitWrMDX15esrCxatmyJu7s7H330EWFhYbi6urJjxw4uX77MiRMnSEtLo3z58lit1gI7St+8HjVAXl6esdOoLZlv+++TTz5Z6H8HuVWRZ1wKu+MEkJycjLOzMydOnGDNmjV8//33vPbaa3h6emqbcRG5L+zbt48aNWrQvXt3ID/Jblv8sXv37kyYMIGMjAxjjbwnnniCZcuW0bNnT15//XX8/PzYtGkT27ZtIyIigtzcXBwcHO64CcWVK1dwd3e/6+KTIiKlVaVKlRgyZAiurq7GMQ8PDzp27Aj8Z6kJq9Vq7Bp94MAB2rVrxzvvvENQUBAuLi5F0nYRkb/KgAED2LdvH9OmTWPSpEn89NNPtGnThu3bt7Np0yaOHz9O9+7dKVeuHFOnTqVv375ERETg6OjI5cuXGTt2LDExMcTHx9O7d29jQwdbkuncuXPExsby888/M3ToUIKCgvRcWgIUed3vnTpOAN27dychIYGMjAwjUfTEE09QsWJFevbsSWxsLD/++COTJk1i4sSJZGZmkpuba0y7s93g7e3tC/wYL168yDPPPEOjRo1YsGABUVFR9OrVy/h+EZGSLjAw0JiHf+OaJAAdOnQgNTXV2Hoc8kuiP/30U0JDQ5kxYwZdu3Y1diQZNGgQjo6ORvWoyWQiJSWFRYsW0aNHD6pUqcKcOXMAdOMXEbkNFxcXhg0bxmOPPVbguO1Z9eaOVUBAAEuXLuWNN96gZcuWSkiJyH2hatWqDB06lHXr1vHFF1/QoEED0tLSiIyMJDMzk6NHjxrvbdeuHVu2bKFVq1YEBgby4YcfEhwcTF5eHjk5OQBkZWWxdu1annjiCapXr069evV4//33KVu2LBUrViyqy5T/UpH3HgIDA1m5ciWAMQJ/u47TAw88APyn4/TOO+8wY8YMDh8+TMOGDRkyZAgDBgwwdjGxdYxSUlJYv349sbGx7Nu3j9GjRzN27FimTp1KzZo17zh/VESkJGvfvj2jR49m+/btBaZ6HD9+nAoVKlCvXj02bNhgxFar1UrdunVZsGABZ8+epUqVKrd85p49e5g5cyY//vgjqamp1KpViw4dOjBmzBhat25daNcmIlJS2Z51bTQYKiKlidVqJTw8nN69ezN16lR69uxJdnY2fn5+9O7dm/j4eNLS0oz3N2vWjKZNmxqxcty4cZjNZsLCwsjKyqJJkyYAhIWFMX36dMLDw6lcuXKRXJv870xW2xBNETl48CBNmzblu+++u23H6YEHHqBv375MmjQJwJheZ7Va/+uOU7du3WjdunWB0mkRkftVly5dSEtL49lnnyUiIoIDBw6watUqIiMjiYmJITU11RgUuJ0bpz5bLBbmzJnDwYMH6datm276IiIiIvJfsfXlt27dSqdOnahTpw6HDh3C3t6eK1eukJ2djbe3t/G+3Nxcxo8fT2ZmJt999x15eXlMnz6dfv36AXDmzBkqV66sjR9KuCJPSoE6TiIi90JSUhKzZs1i27ZtpKSkkJeXx2OPPcZLL72Er6+vUVkqIiIiIlJYcnJy2LZtG4GBgdSuXbvAazcWoZhMJt566y0SExMJDw8nIiICLy+vomm03DPFIimljpOIyL2zb98+nJ2dqV+/foHj2thBRERERESKUrFIStmo4yQi8te6OX7euJupiIiIiEhRUT9foBglpdRxEhG5d3TTFxERERGR4qbYJKVs1HESEREREREREbn/Fbtl6pWQEhERERERERG5/xW7pJSIiIiIiIiIiNz/lJQSEREREREREZFCp6SUiIiIiIiIiIgUOiWlRERERERERESk0CkpJSIiIiIiIiIihU5JKRERERERERERKXRKSomIiIiIiIiISKFTUkpERERERERERAqdklIiIiIiIiIiIlLolJQSEREREREREZFC9/8Bmfj7vlbpKVMAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "palette = sns.color_palette(\"colorblind\")\n", + "\n", + "ci_cre_general = plpr_tune_cre_general.confint()\n", + "ci_cre_normal = plpr_tune_cre_normal.confint()\n", + "ci_fd = plpr_tune_fd.confint()\n", + "ci_wg = plpr_tune_wg.confint()\n", + "\n", + "comparison_data = {\n", + " \"Model\": [\"cre_general\", \"cre_normal\", \"fd_exact\", \"wg_approx\"],\n", + " \"theta\": [plpr_tune_cre_general.coef[0], plpr_tune_cre_normal.coef[0], plpr_tune_fd.coef[0], plpr_tune_wg.coef[0]],\n", + " \"se\": [plpr_tune_cre_general.se[0], plpr_tune_cre_normal.se[0], plpr_tune_fd.se[0], plpr_tune_wg.se[0]],\n", + " \"ci_lower\": [ci_cre_general.iloc[0, 0], ci_cre_normal.iloc[0, 0], ci_fd.iloc[0, 0], ci_wg.iloc[0, 0]],\n", + " \"ci_upper\": [ci_cre_general.iloc[0, 1], ci_cre_normal.iloc[0, 1], ci_fd.iloc[0, 1], ci_wg.iloc[0, 1]]\n", + "}\n", + "df_comparison = pd.DataFrame(comparison_data)\n", + "\n", + "print(f\"True treatment effect: {theta}\\n\")\n", + "print(df_comparison.to_string(index=False))\n", + "\n", + "# Create comparison plot \n", + "plt.figure(figsize=(12, 6))\n", + "\n", + "for i in range(len(df_comparison)):\n", + " plt.errorbar(i, df_comparison.loc[i, \"theta\"],\n", + " yerr=[[df_comparison.loc[i, \"theta\"] - df_comparison.loc[i, \"ci_lower\"]],\n", + " [df_comparison.loc[i, \"ci_upper\"] - df_comparison.loc[i, \"theta\"]]],\n", + " fmt='o', capsize=5, capthick=2, ecolor=palette[i], color=palette[i],\n", + " label=df_comparison.loc[i, \"Model\"], markersize=10, zorder=2)\n", + "plt.axhline(y=theta, color=palette[4], linestyle='--',\n", + " linewidth=2, label=\"True effect\", zorder=1)\n", + "\n", + "plt.title(\"Comparison across DoubleMLPLPR approaches\")\n", + "plt.ylabel(\"Coefficient Value\")\n", + "plt.xticks(range(4), df_comparison[\"Model\"], rotation=15, ha=\"right\")\n", + "plt.legend()\n", + "plt.grid(True, alpha=0.3)\n", + "plt.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We again see that the `wg_approx` leads to a biased estimate in the non-linear and discontinuous `dgp3` setting. The approaches `cre_general`, `cre_normal`, `fd_exact`, in combination with [LightGBM](https://lightgbm.readthedocs.io/en/stable/) regressors, tuned using the [Optuna](https://optuna.org/) package, lead to estimate close to the true treatment effect.\n", + "\n", + "This is line with the simulation results in [Clarke and Polselli (2025)](https://doi.org/10.1093/ectj/utaf011), albeit only for only one dataset in this example." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + }, + "orig_nbformat": 4 + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/doc/examples/py_double_ml_rdflex.ipynb b/doc/examples/py_double_ml_rdflex.ipynb index 0d7b09e6..06abf4a0 100644 --- a/doc/examples/py_double_ml_rdflex.ipynb +++ b/doc/examples/py_double_ml_rdflex.ipynb @@ -24,7 +24,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -70,9 +70,102 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ydscorex0x1x2
01.5630671.00.471435-0.198503-0.193849-0.493426
11.3486220.0-1.1909760.013677-0.630880-0.885832
22.2659291.01.432707-0.2661470.439675-0.051651
37.4773570.0-0.3126520.845241-0.6597550.436764
413.1851300.0-0.7205890.739595-0.7413800.948112
\n", + "
" + ], + "text/plain": [ + " y d score x0 x1 x2\n", + "0 1.563067 1.0 0.471435 -0.198503 -0.193849 -0.493426\n", + "1 1.348622 0.0 -1.190976 0.013677 -0.630880 -0.885832\n", + "2 2.265929 1.0 1.432707 -0.266147 0.439675 -0.051651\n", + "3 7.477357 0.0 -0.312652 0.845241 -0.659755 0.436764\n", + "4 13.185130 0.0 -0.720589 0.739595 -0.741380 0.948112" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "np.random.seed(1234)\n", "\n", @@ -96,9 +189,2964 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + " \n", + " " + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "application/vnd.plotly.v1+json": { + "config": { + "plotlyServerURL": "https://plot.ly" + }, + "data": [ + { + "hovertemplate": "Treatment=True
Score=%{x}
Outcome=%{y}", + "legendgroup": "True", + "marker": { + "color": "#636efa", + "symbol": "circle" + }, + "mode": "markers", + "name": "True", + "orientation": "v", + "showlegend": true, + "type": "scatter", + "x": [ + 0.47143516373249306, + 1.4327069684260973, + 0.8871629403077386, + 0.8595884137174165, + 0.015696372114428918, + 1.150035724719818, + 0.9919460223426778, + 0.9533241281124304, + 0.002118364683486495, + 0.405453411570191, + 0.2890919409800353, + 1.3211581921293856, + 0.19342137647035826, + 0.5534389109567419, + 1.3181515541801367, + 0.6755540851223808, + 1.0589691875711504, + 0.3374376536139724, + 1.0475785728927218, + 1.0459382556276653, + 0.8637172916848387, + 0.12471295376821585, + 0.8416747129961416, + 2.390960515463033, + 0.07619958783723642, + 0.036141936684072715, + 0.24779219974854666, + 0.018289191349219306, + 0.7554139823981354, + 0.2152685809694434, + 0.841008794931391, + 0.3540203321992379, + 0.5657383060625951, + 1.5456588046255575, + 0.30796885521603423, + 1.0338007325554992, + 2.0306036208387996, + 0.21188338677770105, + 0.7047206243171088, + 0.4620597371620487, + 0.7042282254621743, + 0.5235079678938094, + 2.0078429507780005, + 0.2269625418708953, + 0.6319794458091295, + 0.0395126866933667, + 0.46439232505089606, + 1.3211056154702059, + 0.15263055220453448, + 0.16452954293239852, + 0.7673687357524115, + 0.9849198419098969, + 0.270835848826804, + 1.3919861934464073, + 0.07984231300862901, + 0.8165939265478418, + 0.5282881452973941, + 0.2912053597430635, + 0.5665336963535724, + 0.503591759111203, + 0.2852956847818571, + 0.48428811274975, + 1.3634815124261457, + 1.2245743551261743, + 0.8754755042743244, + 0.7491638059190651, + 0.6806560043814565, + 0.047071635325711084, + 0.3948442093272043, + 0.4362576043409168, + 0.3937105991386652, + 0.6941032876787643, + 0.6786296737098565, + 0.2395559950038969, + 0.15122662929444983, + 0.8161272333600409, + 1.8935344675962007, + 0.6396327631937027, + 1.9302467674655757, + 1.2103837049045143, + 0.7974354194278735, + 0.7025622240016004, + 1.1768124501049289, + 0.7009077309156047, + 0.9841880707224154, + 2.365768628840039, + 0.4961429262475947, + 0.7965948666649523, + 1.3577972581070583, + 0.33419793089900884, + 0.5367838249027502, + 0.22598548673375538, + 0.6287758265382943, + 0.18649434877307466, + 0.9524783451091219, + 0.9881375825933089, + 0.13968327403129469, + 2.123691888593553, + 0.12227343425882888, + 1.4229859527722433, + 0.3635645568066484, + 1.2723950785462097, + 0.2093947875359238, + 1.1043515698603126, + 0.8891574940728737, + 0.2883768477152654, + 0.156998376446689, + 1.0576331832046142, + 0.07187803937036874, + 1.9107594272771828, + 0.7879646831376144, + 0.5130821444669919, + 1.0439446554953231, + 2.10778515279337, + 1.4599274815852936, + 1.0154054578125205, + 0.7491846009241975, + 0.4402663916687296, + 0.6889718563269422, + 1.92453332001529, + 0.4112040207072239, + 0.8907649561731171, + 0.22636322725170696, + 1.126385822137273, + 0.2471117161577363, + 0.1211718053315409, + 0.2989839355420466, + 0.24945521515244942, + 0.5810733206206065, + 2.7638440798495663, + 0.3993254369795763, + 0.6684881692058056, + 0.5004829001543722, + 0.8630648605906245, + 1.1539222406122254, + 1.181944105999232, + 0.39137144842193117, + 0.2950799597862399, + 1.8638010621325043, + 0.12678118632697175, + 0.0037598477954959237, + 0.5539212514397202, + 1.5294006744839312, + 0.20545485392597962, + 0.3130131289808603, + 0.8665207121682993, + 0.2990705295943764, + 1.076540643846122, + 0.3631773475444523, + 1.8936798255518426, + 0.3876744557775985, + 0.023557610392553562, + 0.17738671938459905, + 0.9835127293431644, + 0.0235045858853947, + 0.553776977863366, + 0.3537692939966397, + 1.5410299832779386, + 0.06308453895730506, + 1.7879796324557498, + 0.3862536891890525, + 0.8227753737653734, + 1.0572033312141293, + 0.03187997356903155, + 1.3431824575289413, + 0.5270464509549642, + 0.7111123993648228, + 2.6377912107038637, + 1.4311837522545638, + 0.5927584465506688, + 0.170296895326288, + 0.2885813223432078, + 0.17160238198599498, + 0.9828178292431048, + 0.9244428673433661, + 0.9959929702457525, + 0.955557932627455, + 0.7138407711950433, + 0.13337096219412198, + 1.9087796580911864, + 0.15592326145312668, + 0.7596525320929627, + 0.18167274532368163, + 0.27449257074640343, + 0.06791235738075722, + 0.26697323153986774, + 1.3829972488529256, + 0.13139536722591508, + 0.4344371851022324, + 0.2645340683169988, + 0.5656583442738247, + 0.5850842795017246, + 0.4151005583815808, + 0.4255305913978583, + 1.498569955138537, + 0.8210479561079149, + 2.0978007807716916, + 1.2829332468745234, + 0.2703379257393719, + 1.0031403620449315, + 1.0786735513204067, + 0.34075268512976414, + 2.4814584228756535, + 1.3852548761892063, + 0.6078621862094797, + 0.10203511229233127, + 0.21071705250565526, + 1.7923391966469424, + 0.555877426385625, + 1.5727427457558043, + 1.850397801609901, + 0.1934642338198724, + 1.0327232157358608, + 0.3938922629126754, + 0.33250662216224913, + 0.5289444088560467, + 0.04826420692565666, + 0.06198845052177757, + 1.9321781695813465, + 0.35583899747282033, + 0.9894148472858507, + 1.388647381631717, + 1.0877137365241412, + 0.43880126303439065, + 1.2456099371478462, + 0.5029255176706321, + 0.5855993109298576, + 0.35280056010840655, + 0.27477883705216777, + 0.23734976450942513, + 0.44850974251240683, + 0.3495667716276656, + 1.4061761394942678, + 1.5573976506490226, + 1.6726268084547062, + 0.599962085981993, + 0.3315265183521529, + 0.26623740682783636, + 0.2794280245327373, + 0.27166078437402447, + 0.09911201694909029, + 0.025941713524132055, + 1.1055869627971613, + 2.0751854067317823, + 0.4190598246739658, + 0.4134052764244962, + 0.7175769325882829, + 0.26172483967990484, + 1.384580160839074, + 0.5000695850282261, + 1.951103236540061, + 0.03678625863071046, + 0.44585250626100126, + 0.4500013006954749, + 0.5154795307938983, + 0.09988207650262723, + 0.7257141531219552, + 0.9169762830692352, + 0.11911398259834126, + 0.523546018063799, + 0.8215173089989222, + 1.7303155195063402, + 0.22892725550405343, + 1.0927410572908394, + 1.7114259970115762, + 1.4353870862526215, + 0.040843424766001724, + 0.36201819567044535, + 0.5725921910432455, + 1.1044602367331493, + 0.75568884284869, + 0.010281457155327522, + 0.22458577076489658, + 0.028944498401123394, + 2.1040773666293053, + 1.5675141434637165, + 0.16495455703336395, + 1.2073482032456735, + 1.7119870129859922, + 0.6575858924331783, + 0.20007988155239426, + 1.091259859221752, + 1.368220262225958, + 0.8916251175268084, + 0.8375662124501372, + 0.5141447133892467, + 0.7439528461648938, + 0.0899015732727284, + 0.3227395618852, + 1.0501136937792215, + 0.7978764964211627, + 0.5525243131579977, + 1.6415521890288047, + 2.0277395138175196, + 0.07214886239228502, + 0.6131516784246442, + 1.4891664262942452, + 0.22129657503462105, + 0.801888214216484, + 1.118854972135864, + 1.569548064582847, + 1.4277317710676356, + 0.7792145652142417, + 0.24025265459504974, + 0.1911699128232047, + 1.470996579505006, + 0.7189049215428981, + 0.7646112966882201, + 1.0213556589900556, + 1.7096560949798731, + 0.029561951937617355, + 0.12384201643823771, + 0.28943005670297534, + 0.4144350112798109, + 0.9498579011377821, + 0.5218485421095229, + 0.502810027757784, + 1.0557521639658634, + 0.5397352373427955, + 0.3469912503733692, + 0.8741928933305493, + 1.699458930621629, + 1.2828372246866984, + 0.3938001271158281, + 0.4257406105857961, + 0.17196425471545257, + 0.8649983085249567, + 0.7978708863196312, + 0.6769776126657561, + 0.4872101346078452, + 0.765665358162811, + 0.4756748381165133, + 1.5528072078665365, + 0.18853062859763994, + 0.9889637830869817, + 0.21112708032840474, + 1.6117222924558978, + 0.5667304418132302, + 1.5387739411600052, + 0.3096639858949537, + 1.769100570312561, + 0.2682004879637708, + 1.850810168972202, + 1.700769505670963, + 0.3533910559483136, + 0.3364890211021921, + 1.0571054344422781, + 1.1986787224676672, + 0.8043460416389878, + 1.1916553458718568, + 1.4482691251839777, + 0.20145146638225445, + 0.5017557989907293, + 1.6133349220114617, + 1.2990825737762788, + 0.5531844259723022, + 0.5456699950930499, + 0.26228661897316985, + 0.21907020968038537, + 0.9323529322684813, + 1.7430524278808484, + 0.9540316671351478, + 0.7120509492248905, + 1.6615021287437646, + 2.547691702457717, + 2.6519991181342113, + 0.10374933997096626, + 0.11543222568465579, + 0.2586704644560386, + 1.4877433319614572, + 1.4543434763906329, + 0.7043082981680985, + 1.4663513758566715, + 0.3536221825367628, + 0.36320976100296887, + 0.4328745920492785, + 0.29563065498089497, + 2.0798442368337016, + 1.0627364666653065, + 0.01805510965628009, + 0.02279839968937127, + 0.8124230006825046, + 0.615135458353951, + 0.3246098059252541, + 0.6705693906174532, + 0.1491613231433755, + 0.3270638038828972, + 1.6761854255764614, + 0.6809960077267706, + 0.1020797487098907, + 1.1950561396844794, + 0.353683601249814, + 1.0540450911893136, + 0.35207889140437837, + 0.6659818403106037, + 1.0655015019548215, + 0.3644839042458271, + 2.1158507289245674, + 0.6855615854791441, + 1.3812877674130073, + 0.06842655874188228, + 0.31253629248338644, + 0.45425318515036217, + 0.6931185215037656, + 2.337598482195056, + 0.0800997455382507, + 0.4134509638251659, + 0.20445196734865428, + 0.47845329452970037, + 0.02958097097523698, + 0.01746667513481011, + 2.054289530002812, + 0.4238146840382535, + 1.2499025106911574, + 0.5711487534631029, + 0.2970484291574556, + 0.8718652175016266, + 0.6895231987879346, + 1.9845899031898866, + 0.9551299189232728, + 1.4704739392964385, + 0.7108614710694785, + 0.6708259593192185, + 0.06338342659156378, + 0.20024796080692478, + 2.6429226847057117, + 0.8933687076816909, + 1.288554738453981, + 0.33119818801148354, + 1.8560568664485682, + 2.1225899596034625, + 0.020305520730884315, + 1.591464114097166, + 0.0005873380366202194, + 0.15487036534807186, + 0.010685782511001917, + 0.6754553436625048, + 0.8183481023464054, + 1.0389554757038926, + 0.981271848028589, + 0.6166208035641588, + 0.020856736754465456, + 0.5993399050363709, + 1.983488054493826, + 0.07788802802787388, + 0.4045844144487453, + 1.1069498460085399, + 0.2685148227050607, + 0.488360812855723, + 0.5868386021149908, + 0.043672011407345696, + 0.23058049991323532, + 0.3285638932885436, + 0.8380485839532836, + 0.10816425581928157, + 2.039686037158919, + 0.866452901845807, + 0.7084741210544878, + 1.0771702093251387, + 0.06543853933530658, + 1.3120365067494253, + 0.21537592656166116, + 0.2597174736789681, + 0.46954942686078716, + 0.4922331098465613, + 0.51217052768379, + 1.265974917971162, + 0.19091532051019103, + 1.001555629245316, + 1.122267866057918, + 2.0118420191461803, + 0.30201074703431263, + 2.2035145562945293, + 1.3460983256762604, + 0.626820481067724, + 0.10997254269648227, + 2.0065611269776644, + 1.0159978401232488, + 0.2923665381528412, + 1.3370321440963355, + 0.3186160154286117, + 1.3970387291415345, + 0.11182047780156927, + 0.4315681264740078, + 1.6137850971757342, + 1.638962314523427, + 0.13409659216509714, + 0.17203736635766856, + 0.8806401486464327, + 0.7577097291811645, + 0.8896634154552829, + 1.1341571652425653, + 1.7316301950572073, + 0.7007847957258879, + 0.8811405230322151, + 0.16270221119589887, + 0.6697599912207146, + 0.7436871197925536, + 0.6167126196705511, + 0.9718668340412515, + 0.45913770322630687, + 0.6562382063047018, + 0.7515034769028561, + 0.2721639657384405, + 0.9315457527732457, + 0.32753233118239405, + 0.7408138005017622 + ], + "xaxis": "x", + "y": [ + 1.5630670750871647, + 2.265929471529643, + 10.748742710522935, + 4.464563054300227, + 5.720577430325107, + 1.590658838647819, + 2.0944572743407113, + 3.8561356439738455, + 3.6652752947892124, + 4.258380402564378, + 3.4246531645049862, + 12.32396548183264, + 8.460567186562127, + 9.617960664434255, + 8.104539653403087, + 8.484138990719682, + 11.263973781556944, + 8.462833084556626, + 10.939603495455666, + 7.399661064524305, + 2.620400631064451, + 4.010865464260581, + 3.381320486974768, + 1.8308852277275207, + 4.55686741680098, + 2.948719095440292, + 9.600952713627429, + 3.1942833111391407, + 4.894615192702758, + 5.635183887917489, + 13.036012082261806, + 6.637853691092357, + 13.221400040211847, + 2.7991157129239728, + 5.835865097197437, + 15.518747141216663, + 0.9412421119534005, + 7.3429859357301615, + 7.044358420828881, + 4.279050919030776, + 4.661836726142834, + 7.027239323766707, + 6.2983880594659505, + 3.020013027089809, + 5.057610475845288, + 1.9338547248594324, + 4.132263189536221, + 2.5448605324559037, + 3.8272043421838253, + 3.6461745154987355, + 3.943333161825477, + 1.2567470937311578, + 1.9564278707373952, + 1.5677485344907052, + 16.513337028875252, + 10.184765679426906, + 3.8731537783093266, + 13.740174353522601, + 1.6039867353606896, + 6.255971423541705, + 10.861840580482127, + 3.482434591178435, + 4.160683319935916, + 11.758696194494975, + 5.0114814973324, + 2.3060695154765902, + 17.628049600932172, + 1.5396465065237206, + 2.987991789023849, + 12.819285172419374, + 1.910701078233155, + 6.44432760117919, + 2.7231523458302638, + 13.654564132534047, + 2.6867551361260875, + 1.4447510137167534, + 4.184193644033716, + 2.3123531867413947, + 5.907226338887424, + 2.207973811762361, + 2.297649078380153, + 4.0875966628748115, + 1.7662042158113374, + 4.390268193041852, + 2.4763266658009067, + 2.677127307459024, + 4.521164016793937, + 3.9790615945018963, + 3.8699549823091317, + 5.007461426415795, + 3.338940416336114, + 9.869888400801477, + 6.307738209931988, + 3.903746274695888, + 3.0856592384183767, + 3.670106787790936, + 8.294504633413677, + 2.928135017125198, + 17.485468447411897, + 18.184456808988877, + 2.1441758155970394, + 19.011448878519992, + 3.544852768481243, + 14.147775772480772, + 7.308070628222553, + 6.950585492608079, + 3.7891548279849387, + 2.2020926001030663, + 10.16406881190855, + 8.976670952867433, + 2.4702041620939954, + 5.134771415941695, + 10.618572668452142, + 12.154706893256707, + 2.959084173607903, + 2.5765816250505438, + 3.2033905280772075, + 3.500852809076819, + 18.77166976434441, + 4.433431324550289, + 12.468809201993697, + 11.611429868963665, + 3.863768591178403, + 7.001435101935673, + 8.369174521358374, + 15.217050945628085, + 13.603820025465769, + 4.348878276341836, + 4.213142860639733, + 14.91323011942871, + 7.9492001745411365, + 3.408500471687419, + 3.528396784394658, + 11.496296186991868, + 7.899498460269936, + 15.511067704191701, + 2.560866588137461, + 11.84023714538772, + 0.30347550852825705, + 4.877125463354467, + 2.695630017314183, + 10.231373497595118, + 3.7324465944061935, + 1.572300015216746, + 2.731916387579277, + 8.17374189569183, + 9.72395855428605, + 2.4875526365435787, + 4.978081921739523, + 2.2771763059744146, + 8.140580358652747, + 4.640616627035953, + 4.263313964754971, + 8.04651848454491, + 6.8025687454395705, + 4.3657037465282595, + 3.6977943889433376, + 11.208286605796419, + 3.5840621161906934, + 2.39730823672077, + 10.141716452962367, + 16.010875697096775, + 10.716622199946908, + 11.861472590823427, + 1.8039268941832942, + 3.0747743564141383, + 13.012572481931103, + -0.7598574267134771, + 4.283313579755587, + 4.8146064377234845, + 1.4098298865217036, + 1.9945304991518857, + 6.487369383152597, + 16.256389831013333, + 9.199846271814256, + 3.025112405354149, + 1.6760337811953563, + 9.270906507890658, + 10.851786931249737, + 10.658459208235294, + 8.902449122129982, + 10.958092102214838, + 9.806679996607661, + 8.301841904208958, + 2.8959520678532464, + 2.4492962315955906, + 11.095742608750829, + 2.512564357382991, + 4.4968525225538745, + 17.201070405665643, + 11.655181721121407, + 8.458668947695022, + 7.182585257102322, + 1.8381018734017056, + 6.954448421794118, + 10.874116064766108, + 7.44274037363752, + 8.225006373432192, + 10.410356026338688, + 8.092928912418811, + 7.2279781627067194, + 3.115857209972452, + 4.282548417419812, + 14.506884332057322, + 4.224684124764206, + 3.2777888910561184, + 14.656807361191017, + 0.7123243327064984, + 7.476384536900452, + 2.4078354924710546, + 1.4229833078605425, + 6.2566415961307165, + 2.211029639410973, + 4.23489477868798, + 6.258356272014977, + 3.379380966693951, + 1.6472462881020162, + 11.264043112573113, + 2.042943303573523, + 3.220181807764813, + 14.4752145699751, + 5.145156625312843, + 2.654358669444195, + 13.879051236600745, + 7.45682004663257, + 5.347351369089084, + 2.139848712789145, + 9.752660083201123, + 5.02936560189911, + 10.322939743368408, + 3.8839303549256052, + 4.652898997689499, + 9.23135357094443, + 8.509909377619739, + 2.3943898694921013, + 2.154134614445248, + 3.568654362308105, + 2.6317679540729095, + 10.558916997021942, + 12.347672582990297, + 4.282706463206077, + 9.290154652133618, + 3.5201550487419118, + 3.082022967226665, + 14.394201595285786, + 8.12396657748174, + 10.788541878471024, + 1.8422694533861999, + 4.5510943644870325, + 1.8698190374996018, + 6.289082660734713, + 3.6330481899665306, + 6.677261752418803, + 13.645123193561433, + 5.4817511509321575, + 10.73135266063922, + 4.755103062805348, + 3.148671470811746, + 1.6236894726215128, + 10.17983249026187, + 6.4106108238588435, + 12.186651231469241, + 4.539902667427349, + 3.856974473907083, + 5.112886606965867, + 9.704875635951035, + 4.702400931420402, + 6.127085315063862, + 5.895177962193958, + 3.805952128761161, + 3.0980465678831446, + 6.34465527074354, + 4.492431190958236, + 2.6299896045787308, + 1.2657470922349159, + 6.942637664055524, + 3.348774115542101, + 5.402157484675704, + 10.116117235217182, + 4.461354743256478, + 7.531671178472615, + 1.582390736608868, + 1.9596143879565644, + 15.489472594917014, + 5.822830547230848, + 7.499318869571259, + 10.03270304973028, + 8.190469660155882, + 2.9821676416870058, + 4.729218380691532, + 1.1470163880823563, + 3.593338126565563, + 4.513639508012342, + 0.5105259430515321, + 12.05679949241901, + 7.8535085774225974, + 3.8655532959460177, + 3.2977205841009107, + 9.993921914076, + 13.662304944150975, + 3.730133386249415, + 6.930689831586816, + 7.843459588109377, + 6.9954149907923355, + 6.506224292866517, + 1.6342150915008669, + 4.3174584761957835, + 2.2745894543814305, + 4.156564963752937, + 5.681035169088436, + 1.868230527485405, + 12.792242512055923, + 2.9604221983391765, + 2.0462505871181245, + 6.711606376048099, + 4.068431613494066, + 3.8490213170428658, + 4.4039593393058425, + 3.264179673326793, + 2.3879789571191288, + 3.0930389945578867, + 2.020899416110505, + 10.207668213067246, + 4.860619789530659, + 2.6754162996313244, + 6.374779432437629, + 9.937184334004515, + 4.587884587512182, + 3.0462423536527674, + 2.221756539602425, + 6.228476809017635, + 3.4441544833538806, + 3.272093636010206, + 7.872279730973945, + 3.8243847518725254, + 4.927058093891117, + 5.30569605379481, + 1.926392403294464, + 2.982656548934253, + 1.4127318427374114, + 4.7461574093585215, + 2.8483087547703185, + 0.8933631641278355, + 3.4880649403573276, + 11.732943752874197, + 6.9813129049237475, + 3.2384593212065633, + 1.9256466820771712, + 8.418832968509168, + 2.752751478600281, + 6.284160109950633, + 2.163444486136517, + 1.9692343390160645, + 0.7311217267201264, + 10.787539601288069, + 12.895746374184167, + 10.965974552589072, + 10.062736049997376, + 4.195976793683249, + 7.098242051931191, + 0.8145235790787874, + 4.005491424762978, + 2.5337226893140006, + 1.818136863872882, + 7.744568721567811, + 5.590598935913978, + 17.320957993486164, + 8.091359695868857, + 3.056244034539619, + 11.46875965833262, + 1.0895965516554755, + 2.409659052296855, + 5.310917476297208, + 5.0474420794264265, + 8.399995484071185, + 16.68462394752922, + 7.319113255288522, + -0.07188241454945812, + 2.7809199910271896, + 11.597484092134568, + 6.473066959556584, + 6.362604379349993, + 1.9203431272301326, + 3.294921420620403, + 2.8302969183486257, + 10.377576340374638, + 12.695367528603342, + 14.31462998390479, + 6.229088516361773, + 2.1612625683247133, + 8.573328912072396, + 14.068835777185974, + 7.116304508486813, + 4.478951681107789, + 13.201334815401736, + 2.2535400933557805, + 10.917238694141234, + 6.201274413655002, + 4.620054753231987, + 1.7236331316200588, + 7.381441868078257, + 3.0983349286181285, + 4.8533996369185095, + 11.3952442006207, + 1.2327216484910748, + 3.5448227286349776, + 6.393261931006413, + 3.7689547564428887, + 8.039514583666621, + 3.7843861480103103, + 2.3747919230315953, + 0.8808992404321346, + 16.19267979954547, + 6.800721179390649, + 2.480441393050849, + 10.909191504815636, + 1.729538434904268, + 1.8707473024376742, + 0.6245858936561355, + 3.3411429792009937, + 11.066438850385445, + 13.650547610477856, + 8.896877285673233, + 3.838830559941911, + 1.7326646640660939, + 1.5785057335161181, + 5.465699169142491, + 4.433684814922784, + 3.8177536189397165, + 10.325611392051856, + 5.069830743631792, + 7.058062632929908, + 2.915932092366684, + 3.3253092292052147, + 4.67475873038095, + 6.37197924529851, + 16.56232837237978, + 5.727536425813394, + 2.4067812717509782, + 3.7479395372103377, + 4.357959643164212, + 3.115714363474604, + 4.554665308156477, + 4.381187068919068, + 2.0932624617665345, + 5.842986192059503, + 13.775377927488867, + 10.52804827793297, + 3.0287108774118243, + 3.9188253218804037, + 10.103443000619611, + 4.2469082705562595, + 3.1962885130444167, + 5.631753203632753, + 13.114802646740193, + 5.088701749783405, + 7.733015419190123, + 2.0922488953952154, + 4.301228811426256, + 10.84119574001116, + 9.642091012688123, + 3.14344292302663, + 9.285082770663648, + 10.636217269201904, + 1.9518078384347444, + 3.6790083186093123, + 5.481767881878449, + 1.7560951420608684, + 7.57750145845892, + 2.348768957826344, + 2.758855452517031, + 3.6692564484876717, + 17.34422736518518, + 0.8589880967324383, + 12.449212352595664, + 4.8875616041211565, + 8.846735256556087, + 16.957494350302664, + 4.805564824787897, + 8.964989170975489, + 2.1676659305603234, + 0.9233793256699923, + 6.9371029470728205, + 6.768823511173058, + 2.04085544817848, + 0.3963597695790085, + 3.6025896704276534, + 9.584272616464322, + 13.280705656341217, + 9.155549718844648, + 1.6382341393030238, + 2.8484947110270165, + 3.785737569930755, + 1.3657386439237735, + 5.057159291263697, + 4.299912064495803, + 3.412464976813795, + 9.636493224325857, + 11.415592200763612, + 2.91287781057633, + 8.420632293781814, + 7.114443764932338, + 3.6829951311847617, + 2.3490980265992176, + 8.965626018208413, + 10.961085374384197, + 7.950627443018606 + ], + "yaxis": "y" + }, + { + "hovertemplate": "Treatment=False
Score=%{x}
Outcome=%{y}", + "legendgroup": "False", + "marker": { + "color": "#EF553B", + "symbol": "circle" + }, + "mode": "markers", + "name": "False", + "orientation": "v", + "showlegend": true, + "type": "scatter", + "x": [ + -1.1909756947064645, + -0.3126518960917129, + -0.7205887333650116, + -0.6365235044173491, + -2.2426849541854055, + -2.0212548201949705, + -0.334077365808097, + -1.5469055532292402, + -0.2026463246291819, + -0.6559693441389339, + -0.4693052847058996, + -1.8170272265901968, + -0.1831085401789987, + -0.3978402281999914, + -0.12209157484767426, + -0.32279480560829565, + -0.5664459304649568, + -2.0749776006900293, + -0.8971567844396987, + -0.1367948332613474, + -1.4458100770443063, + -1.4019732815008439, + -0.10091819994891389, + -0.5482424491868549, + -0.14461950836938436, + -0.0355130252781402, + -0.9742363337673154, + -0.07034487710410242, + -0.20849876310587975, + -2.4004536338122957, + -1.1426312890227635, + -0.785435211763197, + -0.9262543135302259, + -1.1526591092509524, + -3.5635166606247353, + -0.4300956908764876, + -0.3999645806965225, + -1.0278505586819058, + -0.5847182112607883, + -0.08194705182666534, + -0.3447660142546443, + -1.0689887834801322, + -0.5118813091268151, + -0.781105283625392, + -0.4680176663374855, + -1.2811082751440426, + -1.710715324029529, + -0.4507651031362744, + -0.20393286610125122, + -0.18217541166573417, + -1.8184989903916142, + -0.24843205438084665, + -0.6177066479970167, + -0.682883996449334, + -1.703012774113238, + -0.47932400357549726, + -0.2990162929660804, + -0.9620288319051914, + -2.0852656421201097, + -1.7353488744703927, + -0.379810784047379, + -0.850346271655115, + -0.524336102632456, + -0.12172840866682022, + -0.4740208901256851, + -0.05669571649092953, + -0.8048337241667037, + -2.12362024909293, + -0.33350244043324745, + -0.886719352485378, + -0.7438303679362013, + -0.3202038822408791, + -0.9161988612665409, + -0.8596682999817407, + -0.07260831396267549, + -0.5506029235634193, + -0.9381526140931042, + -1.2390715625947926, + -0.22301898188247601, + -1.4094317399227687, + -2.14785503764361, + -1.347532513457755, + -0.014752111804594866, + -1.4495666088633876, + -1.1955237416669708, + -0.591862973147249, + -0.4145048435321895, + -1.4257947334380214, + -0.5928860038389254, + -1.4731164134656474, + -0.896580615301871, + -0.43154951551604476, + -0.16113690824405888, + -1.0515389375700381, + -0.31956139984023146, + -0.6199930877212022, + -0.5714553510797893, + -0.7914888094778855, + -0.5246273459284702, + -0.5464162802735939, + -0.6755214784366882, + -0.2766461272769873, + -2.0786178937021753, + -0.387886231967176, + -0.08710696996399898, + -0.1570991368466517, + -0.7404690227807179, + -1.2476529230512328, + -0.2757737286563052, + -1.0516278594652122, + -1.392054077493087, + -0.8810470922015694, + -1.7122739244712557, + -1.4070845457432255, + -1.2689942560733918, + -0.8388429214202899, + -0.5040427180819637, + -0.7884401845178787, + -0.48575274428280835, + -0.6602295020442799, + -0.31868492957801425, + -0.7226621936743727, + -0.27559363376768226, + -0.4904835914786814, + -0.37443660019278063, + -2.397503976138816, + -0.28444068392629274, + -1.2656005376012107, + -0.35450870806937423, + -0.10476112306710425, + -0.6837898224310078, + -0.0505396895487724, + -0.3640099563071088, + -1.5533423394310608, + -0.3192978018201611, + -0.21754548049047615, + -1.7421376305949654, + -0.09443480774144415, + -1.7517059477920245, + -0.5425795169067793, + -0.025348902062768783, + -0.28755244928410806, + -0.06124629986631457, + -0.7310333425852061, + -1.0227737046951428, + -0.8070383084728134, + -0.3420107818792151, + -0.42300622841924834, + -0.036982814875919444, + -0.1708010306976224, + -0.022460873511426515, + -0.17429815275208735, + -1.0713686714550896, + -0.04846050255238326, + -0.8452903895662435, + -0.9807243575005858, + -0.42782610997202053, + -0.36015657086902797, + -0.45824038933749284, + -1.3379684019509706, + -0.04138653961487197, + -0.1980749596235683, + -1.154600965389669, + -1.2680690503770815, + -1.080096483157591, + -0.6112816747957622, + -1.4365739367721186, + -0.8969041706975123, + -1.7243930789918331, + -1.3127127065922315, + -0.6818813054911307, + -0.10465157484776595, + -0.3333504267494006, + -0.49674493786490903, + -1.7398037975225593, + -0.7558621756229088, + -0.8811117326479355, + -0.9500255347962833, + -1.120520993992725, + -1.0275156732178674, + -0.23833536546732603, + -0.22663229367683593, + -0.923830720176923, + -1.2700634783862885, + -0.19547241269675866, + -0.4634193992174632, + -0.961602240881236, + -0.6313987783785193, + -0.70425204708826, + -0.24048253675834233, + -1.8155576924320858, + -0.9101238456951419, + -0.7485279533390595, + -0.10812492648026306, + -0.2598436943633885, + -0.31244217715639994, + -0.05953196470440051, + -0.47001481339778667, + -1.0274688471736129, + -0.3066384838696108, + -1.2496967727943529, + -1.5601488888852328, + -0.5933563949764882, + -0.17028851232950049, + -0.5407998076498752, + -0.6841283287665846, + -0.2171991251446381, + -0.38336458015082797, + -0.781816088073111, + -1.21962569517377, + -0.7239489167555044, + -1.7347783075439347, + -0.3630899064372582, + -0.5882267282885189, + -0.5723023507451587, + -1.2080155928296474, + -1.3397010399749156, + -1.4894097013310104, + -1.1509530656259641, + -0.2427712711003623, + -0.5012681327516669, + -0.6981289789688551, + -0.2864254139357557, + -0.2783674615930086, + -1.4103507576806693, + -1.1540839403826029, + -1.3801733950718382, + -0.4958330052131627, + -0.41327907241887124, + -1.7193166679275491, + -0.029560902753084674, + -0.7660398835228289, + -0.7130239566297442, + -2.226117620977587, + -0.5638902404112554, + -1.5221795568231706, + -0.014278788627562308, + -0.24672080681888428, + -0.1653294765172792, + -2.0749804861700474, + -1.002755217016917, + -0.14874184933286258, + -0.045370411959291965, + -0.704492925529761, + -0.35357152666142744, + -0.03065886493678412, + -0.2001718506206763, + -0.48131023935727413, + -1.167520333241006, + -0.4256720265205931, + -1.1347082265682895, + -0.9515209412315562, + -0.12713045607576656, + -0.6242296327908734, + -0.40064420205997625, + -1.1370615382091245, + -0.9031590369968799, + -1.0972780624190317, + -0.07130920963109931, + -1.3193390866508534, + -0.5600929458770713, + -0.6315857879244524, + -0.8670697795627686, + -1.0025626707106647, + -0.6245894793017125, + -0.21214580398595856, + -1.1935641822743448, + -0.48902205827370954, + -0.050049692409833806, + -1.7240502035381267, + -0.37714942561155657, + -0.0002208983015993693, + -0.1686991122175147, + -1.153093261804181, + -1.4389778344742428, + -1.365317321173664, + -1.530892108386614, + -0.42446671286333926, + -1.3718378731166938, + -0.26641789264219634, + -0.10281429468501402, + -0.5864646217583874, + -0.9980556587715391, + -0.9528030215809088, + -0.1665595772769085, + -1.1894434708478612, + -0.5348572637202641, + -0.02159784664665048, + -1.1051915914797859, + -1.0559463385249355, + -0.3014922439465465, + -1.9692628740518392, + -0.35442243933376033, + -0.44699048181139484, + -1.1169783859357703, + -0.7882663825810968, + -1.3045412108969867, + -1.5751793626207562, + -0.15998365617495094, + -0.643333236166963, + -0.7153708866312951, + -0.45255766631820826, + -0.13004743219159512, + -0.3937331661815619, + -0.6410597741081536, + -0.8401490192933458, + -0.2048613320814621, + -0.1070733343325384, + -0.24210668506471908, + -0.9024827749371498, + -0.289095787657148, + -0.34318403393053737, + -2.200564435744119, + -2.5960623967791756, + -1.594885645232233, + -1.1100767316660405, + -0.9984064955184256, + -2.275806602546662, + -1.149298768674316, + -1.5861363032929552, + -0.4723808755437636, + -0.18884634787574245, + -0.6411772534653856, + -0.44884868035202796, + -1.4622250619252337, + -2.594654269661883, + -0.7147453741565917, + -0.12736538411633605, + -1.0749752659886933, + -0.27835427971563803, + -1.0368186639409402, + -1.8977106282505025, + -0.20036229600666147, + -0.12470367976984963, + -0.5793366572156923, + -0.7618194555160586, + -0.038912324029113265, + -0.356409207238252, + -1.8625686534341959, + -0.1362303703563351, + -1.0855234635679416, + -1.813277132405328, + -0.19791075971583402, + -0.5405657877076904, + -1.8424329199610465, + -0.5111660166205718, + -0.58093512839474, + -1.6797914849149362, + -0.625888706466102, + -1.2502708184687266, + -0.12139461101379515, + -1.2808952713375565, + -0.35296169014182177, + -0.9055276404436707, + -0.7037176105934078, + -0.2779593857816089, + -0.052165291629840356, + -0.40831076211055395, + -1.8104296677951879, + -0.9033080622801077, + -0.9808768767289832, + -0.22843361310270258, + -1.514846657215433, + -0.07526128723722576, + -0.29705808175737614, + -0.7274143817732619, + -0.24587534038557923, + -0.12794894211963254, + -0.3693106199094617, + -2.6230042844775308, + -0.15744398471175033, + -0.7884718753018505, + -0.06584033431115952, + -1.418479846109691, + -0.1397533402109211, + -0.5553548562130987, + -0.5019177788520144, + -0.2453604968501647, + -1.2694318590858544, + -0.2623238551840711, + -0.7817174376760302, + -1.3394111207653079, + -0.1824419843989386, + -2.5318833133900305, + -0.955683515590636, + -0.1336742103217792, + -1.077403324709461, + -0.16371119657423003, + -0.11828188509401265, + -1.1805528861977703, + -0.021557883623027695, + -0.8118350520747238, + -2.251213065289845, + -1.12610018599184, + -0.3078667459168953, + -0.9618845356398223, + -0.6567958804642327, + -0.2883631824527201, + -0.21979349574451643, + -0.8455002410306656, + -0.3337447706635678, + -0.14550836017250823, + -0.07536789810631976, + -0.061147655522904056, + -0.7530412824561289, + -0.34022586976460656, + -0.5509960393063006, + -0.6040373262522097, + -0.866260054447336, + -0.35588838632283426, + -0.9543423107689742, + -0.4640317318179717, + -1.5237107863092993, + -0.8406493439809086, + -0.5437403904555086, + -0.9733422132310651, + -0.11601728545379246, + -0.6908582143577912, + -0.7874870323351537, + -0.3781952626733465, + -0.05546876687944298, + -0.8490095099068151, + -0.0780897990873757, + -0.5273122406197597, + -0.24268793363602895, + -0.03552991507180546, + -0.9599149837120845, + -0.6289509748030468, + -0.17539274109078976, + -0.433426256842158, + -2.368892931279029, + -1.5927211354658712, + -2.441670740242832, + -0.2627338282699097, + -0.822618318710863, + -0.042268338331824626, + -0.39127020083745107, + -0.9078816127611445, + -0.04174551326063229, + -1.9943356434051311, + -0.5892856890062176, + -0.9849802336244897, + -1.0361773997113255, + -0.6587570839399224, + -0.6079835611701924, + -0.7414694894279271, + -1.470561473204019, + -0.12851239742110507, + -0.16063982952151423, + -0.6133974021549207, + -0.711763631312299, + -0.42147083331107177, + -0.341216883674434, + -1.8229324878347437, + -0.6707133283158744, + -1.1420202563413573, + -0.11489386424404562, + -0.9292300251800608, + -0.09742149039769883, + -2.119596982319414, + -2.1969217399884924, + -0.48966164024190245, + -0.5135490947661565, + -1.0739210599237325, + -0.29882845731838414, + -0.06581075463628752, + -1.8438652086995353, + -0.7578505995039082, + -1.59249315299401, + -0.24325392379441643, + -0.23102356637828622, + -0.3389292822418378, + -0.8798130487131584, + -0.2973845710076038, + -1.3650171542843021, + -0.4212656827400207, + -1.47256655193283, + -0.6366050127557136, + -0.015838160674877966, + -0.3878147409362679, + -0.7055198448961673, + -1.2858104335854346, + -0.6382100575533082, + -0.6532038099890327, + -1.31996590252498, + -0.8187126044465817, + -0.5740804187646116, + -0.7119035172923535, + -0.7356927851492516, + -0.9634637827705722, + -1.8721836112668646, + -0.5562800612192723, + -0.8019050477600198 + ], + "xaxis": "x", + "y": [ + 1.3486218401039956, + 7.477357302978893, + 13.185130441195385, + 3.213884670828087, + 4.851025691238358, + 13.709625572413577, + 0.9095215968919781, + 0.4585860711409128, + 6.330611091218783, + -0.3899856239234457, + 15.159272440031835, + 10.159416656466759, + 1.1910906009059457, + 1.2031646458756038, + 3.7025147848367004, + 4.5153386414094765, + -0.5561128481502722, + 7.565566495800468, + 0.8914961577507419, + 4.784164959023863, + 4.477655236806118, + -0.41651759242783837, + 0.08806035597773143, + 0.5297637724001851, + 1.7908939267943453, + 6.416756279757693, + 3.3053798974431836, + 1.9901027948435226, + 1.9163785856840159, + 2.01801601787926, + 3.3679245111731215, + -0.14581384197929406, + 2.8463253251114113, + 1.7735415229957858, + 13.65848649643465, + 1.3891269452447803, + 9.120307212105697, + 0.6460634901698075, + 3.1896832136455227, + 2.4476007802679893, + 2.440691459852534, + 8.340491663558723, + 6.922115043013113, + 7.022901571093229, + 0.06724489894951725, + 0.20282808173671818, + 0.821493754710078, + 8.041734745886036, + -0.2992694879926443, + 3.5192039335414838, + 2.3208762968492294, + 1.5946545977070559, + 4.908132773248832, + 1.2154220956546324, + 2.508549845342969, + 6.842041176943584, + 13.919276932471254, + 0.7130649555967261, + 2.7789795394321213, + 9.200972090131284, + 7.472999443980326, + 1.7287378459171878, + 2.3855982259100093, + 9.427352114241756, + 1.3364200530683075, + 1.3210536706273739, + 4.707067484257473, + 6.92580071128665, + 2.36880536392164, + 1.4228427608646224, + 2.6813583321436183, + 1.1048547692079531, + -0.07502604670082638, + 2.250307023533512, + 4.018691995722657, + 3.0817250359340385, + 4.73533281061189, + 5.262460946495654, + -0.030953077101010695, + 6.655120482443857, + 10.887575486792901, + 9.881541668780825, + 0.3624775790859004, + 7.771131601508578, + 12.993615872678404, + 4.46180962563332, + 9.046233596815545, + 0.6153874717064401, + 0.06209084779627111, + 9.02540580710607, + 11.130776201020911, + 0.7583313584682627, + 4.167519345670676, + 1.2958887040965574, + 0.3231021368979874, + 2.4968047294816027, + 5.4849946541383945, + 8.851689212669179, + 0.22497260167637093, + -0.679854495220918, + 1.0178674945952164, + 10.06259122175227, + 7.49361519727407, + 0.7039466740923791, + 0.1923276664283496, + 4.309096888839158, + 0.9005263052843315, + 10.7904567286213, + 3.9516058179371756, + 2.044976597218969, + 4.291426997822883, + 8.136593182364422, + 0.21645849306063691, + 2.2875031906526178, + 1.0004089437269108, + 0.36111171265387776, + 0.44487698164791045, + 5.727982001110966, + 5.389075708870314, + 0.2591187071686343, + 4.349196807521143, + 8.847499888688484, + 17.086010675852076, + 2.357682600073091, + 0.11936442692347213, + 2.6745891379239874, + 9.510315808240478, + 10.660321663718435, + 12.513661621748446, + 6.560471867036691, + 3.9335568742082114, + 8.082556021092136, + 1.6394456261191244, + 0.6787831612420692, + 3.7778571560668484, + 1.8008966196476315, + 11.974289287917196, + 0.6789358057649202, + 3.479487236837873, + 4.558638359694437, + 1.4779720094871518, + 2.8765581705606706, + 1.618984651084493, + -0.5967617898546517, + 8.557701640861922, + 15.273736946209622, + 1.540555906600884, + 17.37779663412622, + 2.905320796685039, + 1.6520725885832626, + -0.4424743006174689, + 4.802259517550979, + 8.050338971448395, + 5.927871152344614, + 11.152073205328248, + 1.3549755284738203, + 5.554899749469977, + 0.7331132023276788, + 2.752146771893159, + 1.0359179974705544, + 4.345536900055321, + 2.6233719765050934, + 1.7590213928451695, + 1.0450913975811487, + 7.998289415441801, + 0.1183953918038539, + 4.708576775768007, + 1.6006440834766547, + 4.692783712058903, + 1.0360663945550246, + 4.549213238167896, + 1.9233745949318444, + 1.9797244398997378, + 9.128279660356313, + 3.269725036219773, + 0.6211616233500779, + 0.37517543140685394, + 5.043483592849441, + 1.1108138951977866, + 0.34558384540343884, + 1.109807387231308, + 11.397328743843131, + 2.311139343857397, + 7.325764630305342, + 5.864583657218799, + -0.12756523321898525, + 3.145222040875022, + 7.651476911828151, + 6.177626722731482, + 3.3784508875812698, + 4.338054049855501, + 5.753557164164235, + 0.5226603330804618, + -0.060170859100088875, + 2.2899617239460897, + 3.0045118892692733, + 6.894214243659627, + 1.7029680839076393, + 16.453126153055717, + 2.5241162657869958, + 5.086393411792524, + 2.067446896541806, + 0.2037421228824528, + 3.788566728440795, + 0.8433810912004589, + 8.73398022259693, + 6.164455561396544, + 5.798025255516479, + 15.633069587432637, + 10.748226336308578, + 1.3749827233735594, + 4.621400375274052, + 0.14514464126031956, + 0.5142775354371905, + -0.30679187046944933, + 10.365028176603392, + 4.229053755115862, + 6.838719423247291, + 1.6735109145218159, + 0.09525147204741025, + 5.633274169859272, + 1.3874135513253338, + 6.051750045442512, + 3.0353469770448553, + 0.973018917455266, + 8.896447542504344, + 1.1384522783195905, + 2.6157789831409017, + -0.618778332438777, + 1.0481937278251565, + 7.527241486465692, + 0.8439946021650253, + 5.68636813453995, + 1.4273143209321062, + 1.7294988055854636, + 6.401381203699718, + -0.432290068850227, + 1.246765100289748, + 3.8145575498127813, + 14.981223733938906, + 4.3547117742037145, + 9.944575127516327, + 1.0349323282462257, + 2.250865405991169, + 0.3433784534174193, + 2.163676222610197, + 0.14744314604060216, + -0.40229619520916193, + 9.63400358141513, + 2.24932445591205, + 4.057333469835328, + 6.781982108778782, + 10.759073146618778, + 1.3471386356430683, + 0.7646496195873962, + 0.5658228293303318, + 2.2951524342113, + 7.356610396655633, + 5.5637627509143, + 2.9546347896691243, + 8.408626718705081, + 2.720189725548378, + 10.322106818447857, + 0.05575704045468888, + 13.593412969522282, + 9.987294315572047, + 6.039899082718677, + 8.181240813066784, + 7.502746435960094, + 4.064052627696925, + 7.825621809088959, + 7.644107124755398, + 4.713550018087741, + 0.31933532074385657, + 9.292805500080178, + 14.740556336698535, + 3.6480216912856616, + 1.1014840130123558, + 0.8500635875769066, + 2.021668269770731, + 7.69610149606448, + 1.904009068981885, + 2.104071728640031, + 1.4282422725463955, + -0.1542287184063889, + 0.14638362545597935, + 5.600240737714463, + 1.489300874371906, + 1.4954795451515095, + 4.362376402733272, + 0.6213348206282769, + 7.709307418869114, + 2.4904908001412, + 5.555281863940814, + 9.422003092737722, + 2.4254330580864294, + 1.338207062398862, + 17.309342449174437, + 4.700038239212424, + 4.707541074441268, + 3.9882370260199664, + 0.5659620963183319, + 1.5533344686194606, + 0.9200400636616757, + 4.545408300690253, + 9.932720415705845, + 0.6306143094905791, + 4.088178795937608, + 6.679938991496159, + 10.174251422781808, + 11.050052132691514, + 10.79419395285209, + 1.0932824737816762, + 1.1123793072914967, + 10.634001853069048, + 3.342639023519304, + 2.1131210563570484, + 2.4123423916134765, + 3.2635831872750387, + 9.865134367880087, + 15.170521775956992, + 0.9864638070484877, + 5.952277871232277, + -0.22429698968552159, + 10.653426281895236, + 7.078660084260667, + 0.026152308571340682, + 2.4163115686018695, + 6.81506264593537, + 1.753500032982672, + 6.13418577972205, + 7.027036730809288, + 7.239424806847834, + 2.67964625593345, + -0.29242847000262817, + 7.346906185677314, + 7.08512293401148, + 1.1187963156683463, + 5.184519286167036, + 11.681597391550724, + 8.598933157024868, + 6.0189098680206605, + 9.089152432868591, + 8.587613013227351, + 3.6528693345703385, + 1.7021195783648564, + 12.4428409274409, + 6.630439258776994, + 4.7151269165494964, + 0.7293393249272646, + 1.2932899904926258, + 3.9902807973338073, + 5.949867349778937, + 7.200662450178622, + 2.319123516352471, + 1.0725604118217094, + 0.0826444359382778, + 17.617140053944556, + 2.587835772105568, + 9.6534920237235, + 10.266156167544603, + 15.534887826327623, + 4.585755041419343, + 0.36043911405584644, + 9.223885330219982, + 0.2538609094114361, + 5.513051565329957, + 5.3767398542711105, + 1.2188733275402446, + 0.27227752465740784, + 5.957509410035419, + 7.980442801090926, + 1.211856035154864, + 5.842941177649649, + 2.34455830212435, + 1.1150603903826617, + 0.5233138627951399, + 0.014215309483299951, + 4.540614289611321, + 10.09604969691086, + 7.567435602565755, + 9.819981825729077, + 5.1909970978463535, + 1.4552681482097538, + 4.321063824763318, + 4.66780249434247, + 6.498074761940801, + -0.20374752217212186, + 8.654007293669366, + 0.2674051948706437, + 3.6929968178840227, + -0.30886086104336674, + 1.2340808958358562, + 1.4145487256673344, + 2.2274011885945777, + 9.744818442988178, + 0.785235196851721, + 5.865549775035278, + 6.894842119981693, + 7.54866454467063, + 4.28335825443005, + -0.09534686814461035, + 0.34810649511538816, + 2.9850848814314466, + 6.100195736648981, + 1.0072378541569067, + 1.7853915517559302, + 0.2346367486240703, + -0.3610901011561299, + 1.632631753231078, + 3.324277613747201, + 11.038362053887168, + 0.623055074210312, + 2.4529519164172675, + 0.8928873362688777, + 1.0205389396139006, + 0.3147848495708494, + 1.4150212059382248, + 5.297619157667393, + 3.2507613327370932, + 0.3143949242786218, + 19.04654958147302, + 0.9358132860305264, + 6.865095541781097, + 2.0022066056655534, + 3.550165232686205, + 3.6148860820759716, + 0.2893192391838322, + 2.6870295469965573, + 7.6771207410376086, + 2.5997293556522543, + 9.723843473476052, + 19.89695246719818, + 0.7525531876775715, + 2.099322031098568, + 3.640862762636694, + 10.126588712300522, + 1.8965559795552247, + 0.4837995815931926, + 0.38883834760001845, + 3.128238530472426, + 3.0287181157971594, + 5.145528983088213, + 2.9839938774044934, + 5.251659075522024, + 5.981752248331269, + 9.905469616628542, + 8.113039292587432, + 2.389183056858123, + 1.5281421008725755, + 0.7251112401444999, + 5.511232338873529, + 2.5722549478657353, + 2.7598012680987805, + 3.7086949801888873, + 5.028506415538427, + 1.0273216027632337, + 7.909008494881561, + 15.239083103886744, + 12.336183290192066, + 2.6448770290342143, + 2.2983019526159363, + 4.26966015537506, + 19.360679032521865, + 7.620029935403366, + 7.608947235750834, + 0.4578952696239318, + 2.5636303602200265, + 1.2361366208392117, + 2.522278900393948, + -0.07875447360853444, + -0.09820607356546696, + 0.5192230455272994, + 2.0760441107813534, + 10.078478898125562, + 4.026487709074533, + 1.4687740141322216, + 1.6736114150139414, + 0.3284413774117222, + 4.364665831733724, + 8.057701278992917, + 0.6144847987536664, + 2.7175702943333575, + 2.597155363410655, + 0.9061489054095168, + 12.503389183146409, + 4.587884045549208, + 2.149740301840303, + 7.815182602979085, + 10.466311998993257, + 0.05084904692638065, + 0.23193727300348288, + 2.1533483673451492 + ], + "yaxis": "y" + } + ], + "layout": { + "legend": { + "title": { + "text": "Treatment" + }, + "tracegroupgap": 0 + }, + "template": { + "data": { + "bar": [ + { + "error_x": { + "color": "#2a3f5f" + }, + "error_y": { + "color": "#2a3f5f" + }, + "marker": { + "line": { + "color": "#E5ECF6", + "width": 0.5 + }, + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "bar" + } + ], + "barpolar": [ + { + "marker": { + "line": { + "color": "#E5ECF6", + "width": 0.5 + }, + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "barpolar" + } + ], + "carpet": [ + { + "aaxis": { + "endlinecolor": "#2a3f5f", + "gridcolor": "white", + "linecolor": "white", + "minorgridcolor": "white", + "startlinecolor": "#2a3f5f" + }, + "baxis": { + "endlinecolor": "#2a3f5f", + "gridcolor": "white", + "linecolor": "white", + "minorgridcolor": "white", + "startlinecolor": "#2a3f5f" + }, + "type": "carpet" + } + ], + "choropleth": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "choropleth" + } + ], + "contour": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "contour" + } + ], + "contourcarpet": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "contourcarpet" + } + ], + "heatmap": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "heatmap" + } + ], + "heatmapgl": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "heatmapgl" + } + ], + "histogram": [ + { + "marker": { + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "histogram" + } + ], + "histogram2d": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "histogram2d" + } + ], + "histogram2dcontour": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "histogram2dcontour" + } + ], + "mesh3d": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "mesh3d" + } + ], + "parcoords": [ + { + "line": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "parcoords" + } + ], + "pie": [ + { + "automargin": true, + "type": "pie" + } + ], + "scatter": [ + { + "fillpattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + }, + "type": "scatter" + } + ], + "scatter3d": [ + { + "line": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatter3d" + } + ], + "scattercarpet": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattercarpet" + } + ], + "scattergeo": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattergeo" + } + ], + "scattergl": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattergl" + } + ], + "scattermapbox": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattermapbox" + } + ], + "scatterpolar": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterpolar" + } + ], + "scatterpolargl": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterpolargl" + } + ], + "scatterternary": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterternary" + } + ], + "surface": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "surface" + } + ], + "table": [ + { + "cells": { + "fill": { + "color": "#EBF0F8" + }, + "line": { + "color": "white" + } + }, + "header": { + "fill": { + "color": "#C8D4E3" + }, + "line": { + "color": "white" + } + }, + "type": "table" + } + ] + }, + "layout": { + "annotationdefaults": { + "arrowcolor": "#2a3f5f", + "arrowhead": 0, + "arrowwidth": 1 + }, + "autotypenumbers": "strict", + "coloraxis": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "colorscale": { + "diverging": [ + [ + 0, + "#8e0152" + ], + [ + 0.1, + "#c51b7d" + ], + [ + 0.2, + "#de77ae" + ], + [ + 0.3, + "#f1b6da" + ], + [ + 0.4, + "#fde0ef" + ], + [ + 0.5, + "#f7f7f7" + ], + [ + 0.6, + "#e6f5d0" + ], + [ + 0.7, + "#b8e186" + ], + [ + 0.8, + "#7fbc41" + ], + [ + 0.9, + "#4d9221" + ], + [ + 1, + "#276419" + ] + ], + "sequential": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "sequentialminus": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ] + }, + "colorway": [ + "#636efa", + "#EF553B", + "#00cc96", + "#ab63fa", + "#FFA15A", + "#19d3f3", + "#FF6692", + "#B6E880", + "#FF97FF", + "#FECB52" + ], + "font": { + "color": "#2a3f5f" + }, + "geo": { + "bgcolor": "white", + "lakecolor": "white", + "landcolor": "#E5ECF6", + "showlakes": true, + "showland": true, + "subunitcolor": "white" + }, + "hoverlabel": { + "align": "left" + }, + "hovermode": "closest", + "mapbox": { + "style": "light" + }, + "paper_bgcolor": "white", + "plot_bgcolor": "#E5ECF6", + "polar": { + "angularaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "bgcolor": "#E5ECF6", + "radialaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + } + }, + "scene": { + "xaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + }, + "yaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + }, + "zaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + } + }, + "shapedefaults": { + "line": { + "color": "#2a3f5f" + } + }, + "ternary": { + "aaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "baxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "bgcolor": "#E5ECF6", + "caxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + } + }, + "title": { + "x": 0.05 + }, + "xaxis": { + "automargin": true, + "gridcolor": "white", + "linecolor": "white", + "ticks": "", + "title": { + "standoff": 15 + }, + "zerolinecolor": "white", + "zerolinewidth": 2 + }, + "yaxis": { + "automargin": true, + "gridcolor": "white", + "linecolor": "white", + "ticks": "", + "title": { + "standoff": 15 + }, + "zerolinecolor": "white", + "zerolinewidth": 2 + } + } + }, + "title": { + "text": "Scatter Plot of Outcome vs. Score by Treatment Status" + }, + "xaxis": { + "anchor": "y", + "domain": [ + 0, + 1 + ], + "title": { + "text": "Score" + } + }, + "yaxis": { + "anchor": "x", + "domain": [ + 0, + 1 + ], + "title": { + "text": "Outcome" + } + } + } + }, + "text/html": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], "source": [ "fig = px.scatter(\n", " x=df['score'],\n", @@ -136,9 +3184,45 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Call: rdrobust\n", + "Number of Observations: 1000\n", + "Polynomial Order Est. (p): 1\n", + "Polynomial Order Bias (q): 2\n", + "Kernel: Triangular\n", + "Bandwidth Selection: mserd\n", + "Var-Cov Estimator: NN\n", + "\n", + " Left Right\n", + "------------------------------------------------\n", + "Number of Observations 493 507\n", + "Number of Unique Obs. 493 507\n", + "Number of Effective Obs. 291 299\n", + "Bandwidth Estimation 0.78 0.78\n", + "Bandwidth Bias 1.282 1.282\n", + "rho (h/b) 0.608 0.608\n", + "\n", + "Method Coef. S.E. t-stat P>|t| 95% CI \n", + "-------------------------------------------------------------------------\n", + "Conventional 2.407 0.634 3.795 1.475e-04 [1.164, 3.65]\n", + "Robust - - 3.122 1.796e-03 [0.869, 3.802]\n" + ] + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "rdrobust_sharp_noadj = rdrobust(y=df['y'], x=df['score'], fuzzy=df['d'], c=0.0)\n", "rdrobust_sharp_noadj" @@ -165,9 +3249,45 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Call: rdrobust\n", + "Number of Observations: 1000\n", + "Polynomial Order Est. (p): 1\n", + "Polynomial Order Bias (q): 2\n", + "Kernel: Triangular\n", + "Bandwidth Selection: mserd\n", + "Var-Cov Estimator: NN\n", + "\n", + " Left Right\n", + "------------------------------------------------\n", + "Number of Observations 493 507\n", + "Number of Unique Obs. 493 507\n", + "Number of Effective Obs. 281 285\n", + "Bandwidth Estimation 0.732 0.732\n", + "Bandwidth Bias 1.22 1.22\n", + "rho (h/b) 0.6 0.6\n", + "\n", + "Method Coef. S.E. t-stat P>|t| 95% CI \n", + "-------------------------------------------------------------------------\n", + "Conventional 2.207 0.433 5.099 3.416e-07 [1.359, 3.056]\n", + "Robust - - 4.311 1.629e-05 [1.189, 3.172]\n" + ] + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "rdrobust_sharp = rdrobust(y=df['y'], x=df['score'], fuzzy=df['d'], covs=df[cov_names], c=0.0)\n", "rdrobust_sharp" @@ -195,7 +3315,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ @@ -211,9 +3331,116 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Method Coef. S.E. t-stat P>|t| 95% CI\n", + "-------------------------------------------------------------------------\n", + "Conventional 2.014 0.103 19.494 1.231e-84 [1.812, 2.217]\n", + "Robust - - 16.872 7.173e-64 [1.779, 2.247]\n", + "Design Type: Sharp\n", + "Cutoff: 0\n", + "First Stage Kernel: triangular\n", + "Final Bandwidth: [0.6607402]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n" + ] + } + ], "source": [ "ml_g = LGBMRegressor(n_estimators=500, learning_rate=0.01, verbose=-1)\n", "\n", @@ -261,9 +3488,102 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ydscorex0x1x2
0-0.1835530.00.471435-0.198503-0.193849-0.493426
11.3486220.0-1.1909760.013677-0.630880-0.885832
22.2659291.01.432707-0.2661470.439675-0.051651
39.6945611.0-0.3126520.845241-0.6597550.436764
415.0014031.0-0.7205890.739595-0.7413800.948112
\n", + "
" + ], + "text/plain": [ + " y d score x0 x1 x2\n", + "0 -0.183553 0.0 0.471435 -0.198503 -0.193849 -0.493426\n", + "1 1.348622 0.0 -1.190976 0.013677 -0.630880 -0.885832\n", + "2 2.265929 1.0 1.432707 -0.266147 0.439675 -0.051651\n", + "3 9.694561 1.0 -0.312652 0.845241 -0.659755 0.436764\n", + "4 15.001403 1.0 -0.720589 0.739595 -0.741380 0.948112" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "np.random.seed(1234)\n", "\n", @@ -286,9 +3606,2935 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "application/vnd.plotly.v1+json": { + "config": { + "plotlyServerURL": "https://plot.ly" + }, + "data": [ + { + "hovertemplate": "Treatment=False
Score=%{x}
Outcome=%{y}", + "legendgroup": "False", + "marker": { + "color": "#636efa", + "symbol": "circle" + }, + "mode": "markers", + "name": "False", + "orientation": "v", + "showlegend": true, + "type": "scatter", + "x": [ + 0.47143516373249306, + -1.1909756947064645, + 0.9919460223426778, + -1.5469055532292402, + -0.6559693441389339, + -0.8971567844396987, + -1.4019732815008439, + -0.10091819994891389, + -0.14461950836938436, + -0.785435211763197, + 0.9849198419098969, + 0.270835848826804, + -0.3447660142546443, + 0.5665336963535724, + -1.2811082751440426, + -1.710715324029529, + -0.20393286610125122, + 0.3937105991386652, + -0.4740208901256851, + -0.05669571649092953, + -0.9161988612665409, + -0.8596682999817407, + -0.22301898188247601, + -0.014752111804594866, + -1.4257947334380214, + -0.5928860038389254, + -0.31956139984023146, + -0.5246273459284702, + 0.7879646831376144, + -0.387886231967176, + -0.08710696996399898, + 1.8638010621325043, + -1.7122739244712557, + -1.2689942560733918, + -0.5040427180819637, + -0.37443660019278063, + -1.5533423394310608, + -0.09443480774144415, + -0.06124629986631457, + -0.7310333425852061, + -0.1708010306976224, + 0.26697323153986774, + -0.36015657086902797, + -1.3379684019509706, + -1.154600965389669, + -1.2680690503770815, + -0.6112816747957622, + -1.3127127065922315, + 1.5727427457558043, + -0.10465157484776595, + -0.7558621756229088, + -0.8811117326479355, + -0.4634193992174632, + -0.7485279533390595, + -0.2598436943633885, + -0.47001481339778667, + -1.5601488888852328, + -0.5407998076498752, + -0.5882267282885189, + 0.26172483967990484, + -0.41327907241887124, + -1.7193166679275491, + -0.5638902404112554, + -0.24672080681888428, + -0.045370411959291965, + -0.704492925529761, + -0.2001718506206763, + -0.48131023935727413, + -0.4256720265205931, + -0.40064420205997625, + -0.42446671286333926, + -1.3718378731166938, + 1.470996579505006, + -0.1665595772769085, + -1.1051915914797859, + 0.029561951937617355, + -0.3014922439465465, + -0.45255766631820826, + -0.3937331661815619, + -0.2048613320814621, + -2.200564435744119, + -0.9984064955184256, + 0.3096639858949537, + 1.850810168972202, + -0.7147453741565917, + 0.20145146638225445, + 1.7430524278808484, + -0.356409207238252, + -0.12139461101379515, + 0.1020797487098907, + -0.2779593857816089, + -0.052165291629840356, + -0.7274143817732619, + -0.5553548562130987, + -0.5019177788520144, + -0.2453604968501647, + -2.5318833133900305, + -0.16371119657423003, + -1.1805528861977703, + -0.8118350520747238, + -2.251213065289845, + -0.14550836017250823, + -0.061147655522904056, + -0.866260054447336, + -1.5237107863092993, + -0.5437403904555086, + -0.11601728545379246, + -0.6908582143577912, + -0.05546876687944298, + -0.24268793363602895, + -0.6289509748030468, + -0.2627338282699097, + -1.9943356434051311, + -0.6587570839399224, + 0.7084741210544878, + 0.4922331098465613, + -0.11489386424404562, + 0.19091532051019103, + -1.8438652086995353, + -0.7578505995039082, + -1.59249315299401, + -0.23102356637828622, + -1.47256655193283, + -0.015838160674877966, + -1.2858104335854346, + 0.7007847957258879, + 0.6697599912207146, + -0.7119035172923535, + -1.8721836112668646, + -0.5562800612192723, + 0.2721639657384405 + ], + "xaxis": "x", + "y": [ + -0.1835525489058031, + 1.3486218401039956, + 0.4411676938396249, + 0.4585860711409128, + -0.3899856239234457, + 0.8914961577507419, + -0.41651759242783837, + 0.08806035597773143, + 1.7908939267943453, + -0.14581384197929406, + -0.5136153229766942, + -0.14978357398012626, + 2.440691459852534, + 0.13616926531694526, + 0.20282808173671818, + 0.821493754710078, + -0.2992694879926443, + -0.3409880513396322, + 1.3364200530683075, + 1.3210536706273739, + -0.07502604670082638, + 2.250307023533512, + -0.030953077101010695, + 0.3624775790859004, + 0.6153874717064401, + 0.06209084779627111, + 0.3231021368979874, + 0.22497260167637093, + 0.4916082218068387, + 0.7039466740923791, + 0.1923276664283496, + 0.27940335629258184, + 0.21645849306063691, + 1.0004089437269108, + 0.44487698164791045, + 0.11936442692347213, + 0.6787831612420692, + 0.6789358057649202, + 1.618984651084493, + -0.5967617898546517, + 1.6520725885832626, + 0.6200601403132633, + 0.7331132023276788, + 1.0359179974705544, + 1.7590213928451695, + 1.0450913975811487, + 0.1183953918038539, + 1.0360663945550246, + 1.2510063032700756, + 1.9233745949318444, + 0.6211616233500779, + 0.37517543140685394, + -0.12756523321898525, + 0.5226603330804618, + 2.2899617239460897, + 1.7029680839076393, + 2.067446896541806, + 0.8433810912004589, + 0.5142775354371905, + -0.17912038210257414, + -0.618778332438777, + 1.0481937278251565, + 1.7294988055854636, + 1.246765100289748, + 1.0349323282462257, + 2.250865405991169, + 0.14744314604060216, + -0.40229619520916193, + 2.24932445591205, + 0.7646496195873962, + 1.1014840130123558, + 0.8500635875769066, + 0.6782697328395315, + -0.1542287184063889, + 1.4954795451515095, + 0.008527384457970084, + 0.6213348206282769, + 0.5659620963183319, + 0.9200400636616757, + 0.6306143094905791, + 1.0932824737816762, + 2.1131210563570484, + -0.49198150408242947, + 0.6459814131494103, + 0.026152308571340682, + 0.5245791129371666, + 0.16988689972629495, + 1.1187963156683463, + 1.2932899904926258, + 0.5085681260628232, + 1.0725604118217094, + 0.0826444359382778, + 0.2538609094114361, + 1.1150603903826617, + 0.5233138627951399, + 0.014215309483299951, + 1.4552681482097538, + -0.20374752217212186, + 0.2674051948706437, + -0.30886086104336674, + 1.2340808958358562, + -0.09534686814461035, + 2.9850848814314466, + -0.3610901011561299, + 0.623055074210312, + 0.8928873362688777, + 0.3147848495708494, + 1.4150212059382248, + 0.3143949242786218, + 2.0022066056655534, + 0.2893192391838322, + 0.7525531876775715, + 0.38883834760001845, + 2.9839938774044934, + 0.1486403718262942, + 0.4219006710464874, + 1.0273216027632337, + -0.14896148702487677, + 0.4578952696239318, + 2.5636303602200265, + 1.2361366208392117, + -0.07875447360853444, + 1.4687740141322216, + 0.3284413774117222, + 0.6144847987536664, + -0.12478320649983757, + 1.0844978616963237, + 2.149740301840303, + 0.05084904692638065, + 0.23193727300348288, + 0.2787166720903948 + ], + "yaxis": "y" + }, + { + "hovertemplate": "Treatment=True
Score=%{x}
Outcome=%{y}", + "legendgroup": "True", + "marker": { + "color": "#EF553B", + "symbol": "circle" + }, + "mode": "markers", + "name": "True", + "orientation": "v", + "showlegend": true, + "type": "scatter", + "x": [ + 1.4327069684260973, + -0.3126518960917129, + -0.7205887333650116, + 0.8871629403077386, + 0.8595884137174165, + -0.6365235044173491, + 0.015696372114428918, + -2.2426849541854055, + 1.150035724719818, + 0.9533241281124304, + -2.0212548201949705, + -0.334077365808097, + 0.002118364683486495, + 0.405453411570191, + 0.2890919409800353, + 1.3211581921293856, + -0.2026463246291819, + 0.19342137647035826, + 0.5534389109567419, + 1.3181515541801367, + -0.4693052847058996, + 0.6755540851223808, + -1.8170272265901968, + -0.1831085401789987, + 1.0589691875711504, + -0.3978402281999914, + 0.3374376536139724, + 1.0475785728927218, + 1.0459382556276653, + 0.8637172916848387, + -0.12209157484767426, + 0.12471295376821585, + -0.32279480560829565, + 0.8416747129961416, + 2.390960515463033, + 0.07619958783723642, + -0.5664459304649568, + 0.036141936684072715, + -2.0749776006900293, + 0.24779219974854666, + -0.1367948332613474, + 0.018289191349219306, + 0.7554139823981354, + 0.2152685809694434, + 0.841008794931391, + -1.4458100770443063, + -0.5482424491868549, + 0.3540203321992379, + -0.0355130252781402, + 0.5657383060625951, + 1.5456588046255575, + -0.9742363337673154, + -0.07034487710410242, + 0.30796885521603423, + -0.20849876310587975, + 1.0338007325554992, + -2.4004536338122957, + 2.0306036208387996, + -1.1426312890227635, + 0.21188338677770105, + 0.7047206243171088, + 0.4620597371620487, + 0.7042282254621743, + 0.5235079678938094, + -0.9262543135302259, + 2.0078429507780005, + 0.2269625418708953, + -1.1526591092509524, + 0.6319794458091295, + 0.0395126866933667, + 0.46439232505089606, + -3.5635166606247353, + 1.3211056154702059, + 0.15263055220453448, + 0.16452954293239852, + -0.4300956908764876, + 0.7673687357524115, + 1.3919861934464073, + 0.07984231300862901, + -0.3999645806965225, + -1.0278505586819058, + -0.5847182112607883, + 0.8165939265478418, + -0.08194705182666534, + 0.5282881452973941, + -1.0689887834801322, + -0.5118813091268151, + 0.2912053597430635, + 0.503591759111203, + 0.2852956847818571, + 0.48428811274975, + 1.3634815124261457, + -0.781105283625392, + -0.4680176663374855, + 1.2245743551261743, + 0.8754755042743244, + -0.4507651031362744, + 0.7491638059190651, + -0.18217541166573417, + 0.6806560043814565, + -1.8184989903916142, + 0.047071635325711084, + 0.3948442093272043, + -0.24843205438084665, + -0.6177066479970167, + -0.682883996449334, + 0.4362576043409168, + -1.703012774113238, + -0.47932400357549726, + -0.2990162929660804, + 0.6941032876787643, + 0.6786296737098565, + 0.2395559950038969, + 0.15122662929444983, + 0.8161272333600409, + 1.8935344675962007, + 0.6396327631937027, + -0.9620288319051914, + -2.0852656421201097, + 1.9302467674655757, + -1.7353488744703927, + 1.2103837049045143, + 0.7974354194278735, + -0.379810784047379, + 0.7025622240016004, + -0.850346271655115, + 1.1768124501049289, + -0.524336102632456, + 0.7009077309156047, + 0.9841880707224154, + -0.12172840866682022, + 2.365768628840039, + 0.4961429262475947, + 0.7965948666649523, + 1.3577972581070583, + -0.8048337241667037, + -2.12362024909293, + -0.33350244043324745, + -0.886719352485378, + 0.33419793089900884, + 0.5367838249027502, + -0.7438303679362013, + -0.3202038822408791, + 0.22598548673375538, + 0.6287758265382943, + 0.18649434877307466, + 0.9524783451091219, + 0.9881375825933089, + -0.07260831396267549, + -0.5506029235634193, + -0.9381526140931042, + -1.2390715625947926, + 0.13968327403129469, + 2.123691888593553, + 0.12227343425882888, + -1.4094317399227687, + 1.4229859527722433, + -2.14785503764361, + -1.347532513457755, + 0.3635645568066484, + 1.2723950785462097, + -1.4495666088633876, + -1.1955237416669708, + -0.591862973147249, + -0.4145048435321895, + 0.2093947875359238, + -1.4731164134656474, + -0.896580615301871, + 1.1043515698603126, + -0.43154951551604476, + -0.16113690824405888, + 0.8891574940728737, + 0.2883768477152654, + -1.0515389375700381, + -0.6199930877212022, + 0.156998376446689, + -0.5714553510797893, + 1.0576331832046142, + -0.7914888094778855, + 0.07187803937036874, + 1.9107594272771828, + 0.5130821444669919, + -0.5464162802735939, + 1.0439446554953231, + 2.10778515279337, + 1.4599274815852936, + 1.0154054578125205, + 0.7491846009241975, + -0.6755214784366882, + 0.4402663916687296, + 0.6889718563269422, + -0.2766461272769873, + 1.92453332001529, + 0.4112040207072239, + 0.8907649561731171, + 0.22636322725170696, + -2.0786178937021753, + 1.126385822137273, + 0.2471117161577363, + 0.1211718053315409, + 0.2989839355420466, + -0.1570991368466517, + -0.7404690227807179, + -1.2476529230512328, + 0.24945521515244942, + 0.5810733206206065, + 2.7638440798495663, + 0.3993254369795763, + 0.6684881692058056, + -0.2757737286563052, + 0.5004829001543722, + 0.8630648605906245, + -1.0516278594652122, + -1.392054077493087, + 1.1539222406122254, + 1.181944105999232, + 0.39137144842193117, + -0.8810470922015694, + 0.2950799597862399, + -1.4070845457432255, + 0.12678118632697175, + 0.0037598477954959237, + -0.8388429214202899, + 0.5539212514397202, + -0.7884401845178787, + 1.5294006744839312, + 0.20545485392597962, + 0.3130131289808603, + 0.8665207121682993, + 0.2990705295943764, + 1.076540643846122, + 0.3631773475444523, + 1.8936798255518426, + -0.48575274428280835, + 0.3876744557775985, + 0.023557610392553562, + -0.6602295020442799, + -0.31868492957801425, + -0.7226621936743727, + 0.17738671938459905, + 0.9835127293431644, + 0.0235045858853947, + 0.553776977863366, + 0.3537692939966397, + -0.27559363376768226, + -0.4904835914786814, + -2.397503976138816, + 1.5410299832779386, + 0.06308453895730506, + -0.28444068392629274, + -1.2656005376012107, + 1.7879796324557498, + -0.35450870806937423, + -0.10476112306710425, + 0.3862536891890525, + 0.8227753737653734, + -0.6837898224310078, + 1.0572033312141293, + 0.03187997356903155, + 1.3431824575289413, + -0.0505396895487724, + -0.3640099563071088, + -0.3192978018201611, + 0.5270464509549642, + 0.7111123993648228, + -0.21754548049047615, + 2.6377912107038637, + -1.7421376305949654, + 1.4311837522545638, + 0.5927584465506688, + 0.170296895326288, + -1.7517059477920245, + 0.2885813223432078, + -0.5425795169067793, + 0.17160238198599498, + 0.9828178292431048, + -0.025348902062768783, + -0.28755244928410806, + 0.9244428673433661, + -1.0227737046951428, + 0.9959929702457525, + 0.955557932627455, + 0.7138407711950433, + 0.13337096219412198, + -0.8070383084728134, + -0.3420107818792151, + 1.9087796580911864, + 0.15592326145312668, + 0.7596525320929627, + -0.42300622841924834, + 0.18167274532368163, + 0.27449257074640343, + 0.06791235738075722, + -0.036982814875919444, + 1.3829972488529256, + -0.022460873511426515, + 0.13139536722591508, + 0.4344371851022324, + 0.2645340683169988, + 0.5656583442738247, + 0.5850842795017246, + -0.17429815275208735, + -1.0713686714550896, + -0.04846050255238326, + -0.8452903895662435, + 0.4151005583815808, + 0.4255305913978583, + -0.9807243575005858, + -0.42782610997202053, + 1.498569955138537, + -0.45824038933749284, + -0.04138653961487197, + 0.8210479561079149, + 2.0978007807716916, + 1.2829332468745234, + 0.2703379257393719, + 1.0031403620449315, + 1.0786735513204067, + 0.34075268512976414, + -0.1980749596235683, + 2.4814584228756535, + 1.3852548761892063, + 0.6078621862094797, + -1.080096483157591, + 0.10203511229233127, + -1.4365739367721186, + 0.21071705250565526, + -0.8969041706975123, + -1.7243930789918331, + 1.7923391966469424, + 0.555877426385625, + -0.6818813054911307, + 1.850397801609901, + -0.3333504267494006, + 0.1934642338198724, + -0.49674493786490903, + 1.0327232157358608, + -1.7398037975225593, + 0.3938922629126754, + -0.9500255347962833, + 0.33250662216224913, + 0.5289444088560467, + -1.120520993992725, + 0.04826420692565666, + 0.06198845052177757, + -1.0275156732178674, + -0.23833536546732603, + 1.9321781695813465, + -0.22663229367683593, + -0.923830720176923, + 0.35583899747282033, + -1.2700634783862885, + -0.19547241269675866, + 0.9894148472858507, + 1.388647381631717, + 1.0877137365241412, + 0.43880126303439065, + -0.961602240881236, + 1.2456099371478462, + 0.5029255176706321, + -0.6313987783785193, + -0.70425204708826, + -0.24048253675834233, + 0.5855993109298576, + 0.35280056010840655, + -1.8155576924320858, + -0.9101238456951419, + -0.10812492648026306, + 0.27477883705216777, + 0.23734976450942513, + -0.31244217715639994, + 0.44850974251240683, + -0.05953196470440051, + 0.3495667716276656, + -1.0274688471736129, + -0.3066384838696108, + 1.4061761394942678, + -1.2496967727943529, + 1.5573976506490226, + -0.5933563949764882, + -0.17028851232950049, + 1.6726268084547062, + -0.6841283287665846, + 0.599962085981993, + -0.2171991251446381, + 0.3315265183521529, + -0.38336458015082797, + -0.781816088073111, + 0.26623740682783636, + 0.2794280245327373, + -1.21962569517377, + -0.7239489167555044, + -1.7347783075439347, + 0.27166078437402447, + 0.09911201694909029, + -0.3630899064372582, + 0.025941713524132055, + -0.5723023507451587, + 1.1055869627971613, + -1.2080155928296474, + -1.3397010399749156, + 2.0751854067317823, + -1.4894097013310104, + -1.1509530656259641, + 0.4190598246739658, + 0.4134052764244962, + 0.7175769325882829, + -0.2427712711003623, + -0.5012681327516669, + -0.6981289789688551, + 1.384580160839074, + -0.2864254139357557, + 0.5000695850282261, + 1.951103236540061, + -0.2783674615930086, + 0.03678625863071046, + 0.44585250626100126, + -1.4103507576806693, + 0.4500013006954749, + 0.5154795307938983, + -1.1540839403826029, + -1.3801733950718382, + -0.4958330052131627, + -0.029560902753084674, + -0.7660398835228289, + 0.09988207650262723, + -0.7130239566297442, + -2.226117620977587, + 0.7257141531219552, + 0.9169762830692352, + -1.5221795568231706, + -0.014278788627562308, + -0.1653294765172792, + 0.11911398259834126, + -2.0749804861700474, + -1.002755217016917, + 0.523546018063799, + 0.8215173089989222, + -0.14874184933286258, + 1.7303155195063402, + 0.22892725550405343, + 1.0927410572908394, + -0.35357152666142744, + -0.03065886493678412, + 1.7114259970115762, + 1.4353870862526215, + -1.167520333241006, + 0.040843424766001724, + 0.36201819567044535, + 0.5725921910432455, + 1.1044602367331493, + 0.75568884284869, + 0.010281457155327522, + 0.22458577076489658, + -1.1347082265682895, + -0.9515209412315562, + -0.12713045607576656, + 0.028944498401123394, + 2.1040773666293053, + 1.5675141434637165, + -0.6242296327908734, + -1.1370615382091245, + -0.9031590369968799, + -1.0972780624190317, + -0.07130920963109931, + -1.3193390866508534, + -0.5600929458770713, + 0.16495455703336395, + -0.6315857879244524, + -0.8670697795627686, + -1.0025626707106647, + 1.2073482032456735, + 1.7119870129859922, + -0.6245894793017125, + -0.21214580398595856, + -1.1935641822743448, + 0.6575858924331783, + 0.20007988155239426, + 1.091259859221752, + -0.48902205827370954, + 1.368220262225958, + 0.8916251175268084, + 0.8375662124501372, + 0.5141447133892467, + -0.050049692409833806, + 0.7439528461648938, + -1.7240502035381267, + 0.0899015732727284, + 0.3227395618852, + -0.37714942561155657, + -0.0002208983015993693, + 1.0501136937792215, + -0.1686991122175147, + 0.7978764964211627, + 0.5525243131579977, + -1.153093261804181, + 1.6415521890288047, + -1.4389778344742428, + 2.0277395138175196, + -1.365317321173664, + 0.07214886239228502, + 0.6131516784246442, + 1.4891664262942452, + 0.22129657503462105, + -1.530892108386614, + 0.801888214216484, + 1.118854972135864, + 1.569548064582847, + 1.4277317710676356, + -0.26641789264219634, + 0.7792145652142417, + -0.10281429468501402, + -0.5864646217583874, + 0.24025265459504974, + -0.9980556587715391, + 0.1911699128232047, + -0.9528030215809088, + 0.7189049215428981, + 0.7646112966882201, + -1.1894434708478612, + -0.5348572637202641, + -0.02159784664665048, + -1.0559463385249355, + 1.0213556589900556, + 1.7096560949798731, + 0.12384201643823771, + 0.28943005670297534, + 0.4144350112798109, + 0.9498579011377821, + 0.5218485421095229, + -1.9692628740518392, + 0.502810027757784, + -0.35442243933376033, + -0.44699048181139484, + -1.1169783859357703, + -0.7882663825810968, + 1.0557521639658634, + -1.3045412108969867, + 0.5397352373427955, + 0.3469912503733692, + 0.8741928933305493, + -1.5751793626207562, + -0.15998365617495094, + -0.643333236166963, + -0.7153708866312951, + 1.699458930621629, + 1.2828372246866984, + 0.3938001271158281, + 0.4257406105857961, + -0.13004743219159512, + -0.6410597741081536, + 0.17196425471545257, + -0.8401490192933458, + 0.8649983085249567, + -0.1070733343325384, + -0.24210668506471908, + -0.9024827749371498, + 0.7978708863196312, + -0.289095787657148, + 0.6769776126657561, + -0.34318403393053737, + 0.4872101346078452, + 0.765665358162811, + -2.5960623967791756, + 0.4756748381165133, + -1.594885645232233, + -1.1100767316660405, + -2.275806602546662, + 1.5528072078665365, + -1.149298768674316, + 0.18853062859763994, + 0.9889637830869817, + -1.5861363032929552, + -0.4723808755437636, + -0.18884634787574245, + 0.21112708032840474, + 1.6117222924558978, + 0.5667304418132302, + -0.6411772534653856, + 1.5387739411600052, + -0.44884868035202796, + 1.769100570312561, + 0.2682004879637708, + -1.4622250619252337, + 1.700769505670963, + 0.3533910559483136, + 0.3364890211021921, + 1.0571054344422781, + 1.1986787224676672, + -2.594654269661883, + 0.8043460416389878, + 1.1916553458718568, + -0.12736538411633605, + -1.0749752659886933, + 1.4482691251839777, + -0.27835427971563803, + -1.0368186639409402, + 0.5017557989907293, + 1.6133349220114617, + -1.8977106282505025, + 1.2990825737762788, + 0.5531844259723022, + -0.20036229600666147, + 0.5456699950930499, + 0.26228661897316985, + 0.21907020968038537, + 0.9323529322684813, + -0.12470367976984963, + 0.9540316671351478, + 0.7120509492248905, + -0.5793366572156923, + -0.7618194555160586, + 1.6615021287437646, + 2.547691702457717, + -0.038912324029113265, + -1.8625686534341959, + 2.6519991181342113, + -0.1362303703563351, + 0.10374933997096626, + 0.11543222568465579, + 0.2586704644560386, + 1.4877433319614572, + 1.4543434763906329, + 0.7043082981680985, + 1.4663513758566715, + 0.3536221825367628, + -1.0855234635679416, + 0.36320976100296887, + -1.813277132405328, + -0.19791075971583402, + 0.4328745920492785, + -0.5405657877076904, + 0.29563065498089497, + 2.0798442368337016, + -1.8424329199610465, + -0.5111660166205718, + -0.58093512839474, + 1.0627364666653065, + 0.01805510965628009, + 0.02279839968937127, + -1.6797914849149362, + 0.8124230006825046, + -0.625888706466102, + -1.2502708184687266, + 0.615135458353951, + -1.2808952713375565, + 0.3246098059252541, + 0.6705693906174532, + 0.1491613231433755, + 0.3270638038828972, + -0.35296169014182177, + -0.9055276404436707, + 1.6761854255764614, + 0.6809960077267706, + -0.7037176105934078, + 1.1950561396844794, + 0.353683601249814, + -0.40831076211055395, + -1.8104296677951879, + -0.9033080622801077, + 1.0540450911893136, + -0.9808768767289832, + 0.35207889140437837, + 0.6659818403106037, + 1.0655015019548215, + -0.22843361310270258, + 0.3644839042458271, + -1.514846657215433, + -0.07526128723722576, + -0.29705808175737614, + -0.24587534038557923, + -0.12794894211963254, + -0.3693106199094617, + -2.6230042844775308, + 2.1158507289245674, + 0.6855615854791441, + -0.15744398471175033, + 1.3812877674130073, + 0.06842655874188228, + 0.31253629248338644, + 0.45425318515036217, + -0.7884718753018505, + -0.06584033431115952, + -1.418479846109691, + -0.1397533402109211, + 0.6931185215037656, + -1.2694318590858544, + -0.2623238551840711, + 2.337598482195056, + -0.7817174376760302, + 0.0800997455382507, + -1.3394111207653079, + 0.4134509638251659, + 0.20445196734865428, + 0.47845329452970037, + 0.02958097097523698, + -0.1824419843989386, + 0.01746667513481011, + 2.054289530002812, + 0.4238146840382535, + -0.955683515590636, + -0.1336742103217792, + -1.077403324709461, + 1.2499025106911574, + 0.5711487534631029, + 0.2970484291574556, + 0.8718652175016266, + 0.6895231987879346, + 1.9845899031898866, + -0.11828188509401265, + 0.9551299189232728, + 1.4704739392964385, + -0.021557883623027695, + 0.7108614710694785, + -1.12610018599184, + -0.3078667459168953, + -0.9618845356398223, + 0.6708259593192185, + 0.06338342659156378, + -0.6567958804642327, + -0.2883631824527201, + -0.21979349574451643, + 0.20024796080692478, + -0.8455002410306656, + 2.6429226847057117, + -0.3337447706635678, + 0.8933687076816909, + 1.288554738453981, + 0.33119818801148354, + -0.07536789810631976, + 1.8560568664485682, + -0.7530412824561289, + -0.34022586976460656, + -0.5509960393063006, + -0.6040373262522097, + 2.1225899596034625, + 0.020305520730884315, + -0.35588838632283426, + -0.9543423107689742, + 1.591464114097166, + 0.0005873380366202194, + -0.4640317318179717, + 0.15487036534807186, + -0.8406493439809086, + 0.010685782511001917, + 0.6754553436625048, + -0.9733422132310651, + 0.8183481023464054, + 1.0389554757038926, + 0.981271848028589, + 0.6166208035641588, + -0.7874870323351537, + 0.020856736754465456, + 0.5993399050363709, + -0.3781952626733465, + 1.983488054493826, + 0.07788802802787388, + 0.4045844144487453, + -0.8490095099068151, + -0.0780897990873757, + -0.5273122406197597, + 1.1069498460085399, + -0.03552991507180546, + -0.9599149837120845, + 0.2685148227050607, + 0.488360812855723, + 0.5868386021149908, + -0.17539274109078976, + -0.433426256842158, + 0.043672011407345696, + -2.368892931279029, + -1.5927211354658712, + -2.441670740242832, + -0.822618318710863, + -0.042268338331824626, + -0.39127020083745107, + 0.23058049991323532, + 0.3285638932885436, + 0.8380485839532836, + -0.9078816127611445, + -0.04174551326063229, + 0.10816425581928157, + -0.5892856890062176, + -0.9849802336244897, + -1.0361773997113255, + 2.039686037158919, + -0.6079835611701924, + 0.866452901845807, + -0.7414694894279271, + -1.470561473204019, + -0.12851239742110507, + -0.16063982952151423, + 1.0771702093251387, + -0.6133974021549207, + -0.711763631312299, + 0.06543853933530658, + -0.42147083331107177, + -0.341216883674434, + 1.3120365067494253, + -1.8229324878347437, + 0.21537592656166116, + -0.6707133283158744, + 0.2597174736789681, + 0.46954942686078716, + 0.51217052768379, + -1.1420202563413573, + 1.265974917971162, + -0.9292300251800608, + 1.001555629245316, + -0.09742149039769883, + -2.119596982319414, + -2.1969217399884924, + 1.122267866057918, + 2.0118420191461803, + 0.30201074703431263, + -0.48966164024190245, + -0.5135490947661565, + -1.0739210599237325, + 2.2035145562945293, + -0.29882845731838414, + 1.3460983256762604, + 0.626820481067724, + -0.06581075463628752, + 0.10997254269648227, + 2.0065611269776644, + 1.0159978401232488, + 0.2923665381528412, + 1.3370321440963355, + -0.24325392379441643, + -0.3389292822418378, + 0.3186160154286117, + 1.3970387291415345, + -0.8798130487131584, + 0.11182047780156927, + 0.4315681264740078, + 1.6137850971757342, + -0.2973845710076038, + 1.638962314523427, + 0.13409659216509714, + 0.17203736635766856, + -1.3650171542843021, + -0.4212656827400207, + 0.8806401486464327, + 0.7577097291811645, + -0.6366050127557136, + -0.3878147409362679, + 0.8896634154552829, + 1.1341571652425653, + 1.7316301950572073, + -0.7055198448961673, + -0.6382100575533082, + 0.8811405230322151, + -0.6532038099890327, + -1.31996590252498, + -0.8187126044465817, + 0.16270221119589887, + -0.5740804187646116, + 0.7436871197925536, + 0.6167126196705511, + 0.9718668340412515, + -0.7356927851492516, + -0.9634637827705722, + 0.45913770322630687, + 0.6562382063047018, + 0.7515034769028561, + 0.9315457527732457, + 0.32753233118239405, + 0.7408138005017622, + -0.8019050477600198 + ], + "xaxis": "x", + "y": [ + 2.265929471529643, + 9.694561155437166, + 15.001403007791444, + 10.748742710522935, + 4.464563054300227, + 4.458606814787848, + 5.720577430325107, + 4.07067774720589, + 1.590658838647819, + 3.8561356439738455, + 13.810615341379208, + 2.7698934669732473, + 3.6652752947892124, + 4.258380402564378, + 3.4246531645049862, + 12.32396548183264, + 8.236964297743414, + 8.460567186562127, + 9.617960664434255, + 8.104539653403087, + 17.09901330545235, + 8.484138990719682, + 10.681717563115276, + 3.001393280247692, + 11.263973781556944, + 3.3413201248723268, + 8.462833084556626, + 10.939603495455666, + 7.399661064524305, + 2.620400631064451, + 5.242739862271615, + 4.010865464260581, + 7.248533468937351, + 3.381320486974768, + 1.8308852277275207, + 4.55686741680098, + 1.51630276636483, + 2.948719095440292, + 7.30264204621881, + 9.600952713627429, + 6.601195519451783, + 3.1942833111391407, + 4.894615192702758, + 5.635183887917489, + 13.036012082261806, + 4.828822090660765, + 2.4365815271131694, + 6.637853691092357, + 8.53838768331626, + 13.221400040211847, + 2.7991157129239728, + 4.397073336490484, + 4.2005707454675845, + 5.835865097197437, + 4.100982661677124, + 15.518747141216663, + 1.3614330070922058, + 0.9412421119534005, + 4.84558565236586, + 7.3429859357301615, + 7.044358420828881, + 4.279050919030776, + 4.661836726142834, + 7.027239323766707, + 4.691120570711789, + 6.2983880594659505, + 3.020013027089809, + 3.238771401281311, + 5.057610475845288, + 1.9338547248594324, + 4.132263189536221, + 8.978379801118601, + 2.5448605324559037, + 3.8272043421838253, + 3.6461745154987355, + 3.7141268554733924, + 3.943333161825477, + 1.5677485344907052, + 16.513337028875252, + 11.516513554272976, + 1.8749519970640258, + 5.245026374790678, + 10.184765679426906, + 5.0282521403687825, + 3.8731537783093266, + 9.526111855670262, + 8.930290760456417, + 13.740174353522601, + 6.255971423541705, + 10.861840580482127, + 3.482434591178435, + 4.160683319935916, + 8.337209995273493, + 2.525660823074181, + 11.758696194494975, + 5.0114814973324, + 9.964967051348433, + 2.3060695154765902, + 5.563853505384101, + 17.628049600932172, + 2.6871766020875216, + 1.5396465065237206, + 2.987991789023849, + 3.839128804311066, + 6.914663671850868, + 3.0532649901547106, + 12.819285172419374, + 3.510958942199201, + 8.90246101383686, + 15.9873797131138, + 6.44432760117919, + 2.7231523458302638, + 13.654564132534047, + 2.6867551361260875, + 1.4447510137167534, + 4.184193644033716, + 2.3123531867413947, + 2.219716094903136, + 2.3552451215543204, + 5.907226338887424, + 9.755776884038891, + 2.207973811762361, + 2.297649078380153, + 9.667867403640276, + 4.0875966628748115, + 3.621094634839108, + 1.7662042158113374, + 3.6172598935433316, + 4.390268193041852, + 2.4763266658009067, + 11.076447482487703, + 2.677127307459024, + 4.521164016793937, + 3.9790615945018963, + 3.8699549823091317, + 6.269105752749612, + 6.681577465665945, + 4.48780277684491, + 2.908206085100257, + 5.007461426415795, + 3.338940416336114, + 4.640072016011472, + 3.1218223722428795, + 9.869888400801477, + 6.307738209931988, + 3.903746274695888, + 3.0856592384183767, + 3.670106787790936, + 6.122887292442154, + 4.952098165720181, + 6.421845648941789, + 7.00091245254906, + 8.294504633413677, + 2.928135017125198, + 17.485468447411897, + 7.531354353045948, + 18.184456808988877, + 10.813496038550449, + 11.227216864967803, + 2.1441758155970394, + 19.011448878519992, + 8.190505279202194, + 14.509762242844465, + 6.046114345093918, + 11.091717008532877, + 3.544852768481243, + 9.866525617443084, + 12.551249053912088, + 14.147775772480772, + 2.654741728524375, + 6.2650647639237524, + 7.308070628222553, + 6.950585492608079, + 2.573817644256974, + 4.423405333839632, + 3.7891548279849387, + 7.628517620276847, + 2.2020926001030663, + 10.418298407825423, + 10.16406881190855, + 8.976670952867433, + 5.134771415941695, + 1.0265758626467236, + 10.618572668452142, + 12.154706893256707, + 2.959084173607903, + 2.5765816250505438, + 3.2033905280772075, + 2.3635302870548887, + 3.500852809076819, + 18.77166976434441, + 11.956913263566697, + 4.433431324550289, + 12.468809201993697, + 11.611429868963665, + 3.863768591178403, + 7.1068830221501, + 7.001435101935673, + 8.369174521358374, + 15.217050945628085, + 13.603820025465769, + 6.658213969381527, + 2.970389713710068, + 12.394596021181313, + 4.348878276341836, + 4.213142860639733, + 14.91323011942871, + 7.9492001745411365, + 3.408500471687419, + 6.2663586180489155, + 3.528396784394658, + 11.496296186991868, + 3.6321862632085935, + 5.607037732401054, + 7.899498460269936, + 15.511067704191701, + 2.560866588137461, + 9.731651246426889, + 11.84023714538772, + 2.8732879832270277, + 4.877125463354467, + 2.695630017314183, + 1.2133795212480423, + 10.231373497595118, + 7.091572264058836, + 3.7324465944061935, + 1.572300015216746, + 2.731916387579277, + 8.17374189569183, + 9.72395855428605, + 2.4875526365435787, + 4.978081921739523, + 2.2771763059744146, + 6.6339700137369, + 8.140580358652747, + 4.640616627035953, + 1.8377549423898605, + 6.342573084839112, + 10.855949418171228, + 4.263313964754971, + 8.04651848454491, + 6.8025687454395705, + 4.3657037465282595, + 3.6977943889433376, + 19.085287142725683, + 3.972984645526237, + 1.642655075425628, + 11.208286605796419, + 3.5840621161906934, + 11.546673655801067, + 11.952633574770253, + 2.39730823672077, + 14.671836195629043, + 8.530659137003743, + 10.141716452962367, + 16.010875697096775, + 5.188713804683356, + 10.716622199946908, + 11.861472590823427, + 1.8039268941832942, + 10.40288304422664, + 3.5002087345455655, + 5.297098100050996, + 3.0747743564141383, + 13.012572481931103, + 3.7846499729219025, + -0.7598574267134771, + 12.424871923769345, + 4.283313579755587, + 4.8146064377234845, + 1.4098298865217036, + 3.947546558909086, + 1.9945304991518857, + 6.546133143596889, + 6.487369383152597, + 16.256389831013333, + 3.7995193097475877, + 4.963357564324481, + 9.199846271814256, + 9.816627223980333, + 3.025112405354149, + 1.6760337811953563, + 9.270906507890658, + 10.851786931249737, + 17.044317060137793, + 3.1073986708814494, + 10.658459208235294, + 8.902449122129982, + 10.958092102214838, + 19.54071816474041, + 9.806679996607661, + 8.301841904208958, + 2.8959520678532464, + 4.833954303426042, + 11.095742608750829, + 1.1760851881807914, + 2.512564357382991, + 4.4968525225538745, + 17.201070405665643, + 11.655181721121407, + 8.458668947695022, + 6.789125404106795, + 9.187651303946337, + 8.090996981895236, + 12.70118691914138, + 7.182585257102322, + 1.8381018734017056, + 3.062039859234129, + 7.095788044645277, + 6.954448421794118, + 4.385438408594497, + 6.590367019049524, + 10.874116064766108, + 7.44274037363752, + 8.225006373432192, + 10.410356026338688, + 8.092928912418811, + 7.2279781627067194, + 3.115857209972452, + 4.695954776025797, + 4.282548417419812, + 14.506884332057322, + 4.224684124764206, + 8.921462959631633, + 3.2777888910561184, + 5.572925949158813, + 14.656807361191017, + 3.218124759032054, + 5.506818285663842, + 0.7123243327064984, + 7.476384536900452, + 6.663232799363023, + 1.4229833078605425, + 4.0366851227981435, + 6.2566415961307165, + 10.72206032966684, + 2.211029639410973, + 3.6637708586232267, + 4.23489477868798, + 6.084959782991081, + 6.258356272014977, + 3.379380966693951, + 3.0993170207247682, + 1.6472462881020162, + 11.264043112573113, + 2.163243668231, + 3.528084599887016, + 2.042943303573523, + 12.68556134281856, + 4.484170364411881, + 3.220181807764813, + 8.142758404508227, + 7.661445440033579, + 14.4752145699751, + 5.145156625312843, + 2.654358669444195, + 13.879051236600745, + 5.099931116909535, + 7.45682004663257, + 5.347351369089084, + 9.141745153929424, + 8.300043464836314, + 5.31576547034449, + 2.139848712789145, + 9.752660083201123, + 4.527300075464674, + 6.897328712791664, + 2.338330865206532, + 5.02936560189911, + 10.322939743368408, + 5.392449167791411, + 3.8839303549256052, + 8.68167101482884, + 4.652898997689499, + 17.917205292427912, + 4.297245511251533, + 9.23135357094443, + 6.493175338396726, + 8.509909377619739, + 1.875745309646252, + 5.531646458900818, + 2.3943898694921013, + 10.781361555333927, + 2.154134614445248, + 8.013136736112866, + 3.568654362308105, + 7.4422908265758725, + 17.227895549922852, + 2.6317679540729095, + 10.558916997021942, + 12.453784287346103, + 3.3062936042446442, + 4.947169977729538, + 12.347672582990297, + 4.282706463206077, + 2.291754437791348, + 9.290154652133618, + 1.5717542687400683, + 3.5201550487419118, + 11.832308202021874, + 5.099030109770507, + 3.082022967226665, + 7.26008928972419, + 3.233861162032418, + 14.394201595285786, + 8.12396657748174, + 10.788541878471024, + 2.439550018809831, + 7.385012991957169, + 3.2595052918257976, + 4.5510943644870325, + 8.683203534724592, + 1.8698190374996018, + 6.289082660734713, + 4.788223873186884, + 3.6330481899665306, + 6.677261752418803, + 1.846964087038559, + 13.645123193561433, + 5.4817511509321575, + 10.313632395816132, + 1.8366444709316732, + 4.140511100668412, + 9.202789285333695, + 2.447011946793957, + 10.73135266063922, + 7.574194175314418, + 1.3505635513390526, + 4.755103062805348, + 3.148671470811746, + 7.496146584414306, + 1.8908363200219356, + 6.133489598125723, + 1.6236894726215128, + 14.997907268596853, + 5.904399986686302, + 10.17983249026187, + 6.4106108238588435, + 11.97078302652149, + 12.186651231469241, + 4.539902667427349, + 3.856974473907083, + 2.0881366955685468, + 3.889966063019122, + 5.112886606965867, + 9.704875635951035, + 10.799089667366115, + 4.702400931420402, + 6.127085315063862, + 5.895177962193958, + 3.805952128761161, + 3.0980465678831446, + 6.34465527074354, + 4.492431190958236, + 5.977163204512254, + 8.562762070357183, + 12.635811069085616, + 2.6299896045787308, + 1.2657470922349159, + 6.942637664055524, + 3.2373690084810773, + 1.9488870521864206, + 3.9231622333029783, + 8.431169321816414, + 7.476758283389479, + 3.6965208088994967, + 10.375372377939263, + 3.348774115542101, + 4.889123861091899, + 11.74636802375234, + 1.6184490045922735, + 5.402157484675704, + 10.116117235217182, + 16.019602135417088, + 11.705763971904398, + 7.299053052365139, + 4.461354743256478, + 7.531671178472615, + 1.582390736608868, + 9.534542269263813, + 1.9596143879565644, + 15.489472594917014, + 5.822830547230848, + 7.499318869571259, + 9.985216608333925, + 10.03270304973028, + 4.083188020683909, + 8.190469660155882, + 2.9821676416870058, + 9.242591839213503, + 9.998569948100528, + 4.729218380691532, + 7.593892036137035, + 1.1470163880823563, + 3.593338126565563, + 1.5426314453879986, + 4.513639508012342, + 10.118913464563223, + 0.5105259430515321, + 15.775099197228867, + 12.05679949241901, + 7.8535085774225974, + 3.8655532959460177, + 3.2977205841009107, + 3.8546112806651625, + 9.993921914076, + 13.662304944150975, + 3.730133386249415, + 6.930689831586816, + 3.833114948129331, + 7.843459588109377, + 9.651382866988321, + 4.32487240334703, + 6.9954149907923355, + 3.6732751826267678, + 6.506224292866517, + 2.836830831794468, + 4.3174584761957835, + 2.2745894543814305, + 1.3672812791732438, + 7.07526394073255, + 4.003455411724395, + 6.252880061722262, + 4.156564963752937, + 5.681035169088436, + 12.792242512055923, + 2.9604221983391765, + 2.0462505871181245, + 6.711606376048099, + 4.068431613494066, + 7.858383748838975, + 3.8490213170428658, + 4.580154835089165, + 7.501941096065977, + 10.7780201322668, + 4.067694705027642, + 4.4039593393058425, + 2.6901058328850302, + 3.264179673326793, + 2.3879789571191288, + 3.0930389945578867, + 18.30115733900274, + 6.471707350070127, + 6.680905291971551, + 5.742366537822184, + 2.020899416110505, + 10.207668213067246, + 4.860619789530659, + 2.6754162996313244, + 3.585152771900879, + 5.644030250207612, + 6.374779432437629, + 11.720995085266173, + 9.937184334004515, + 5.590718960449072, + 8.701223834117853, + 11.128463987492674, + 4.587884587512182, + 12.894044239488057, + 3.0462423536527674, + 12.477953973674333, + 2.221756539602425, + 6.228476809017635, + 0.18358528357621517, + 3.4441544833538806, + 10.965190200526736, + 4.980845003123529, + 1.3932772724703333, + 3.272093636010206, + 4.266746562229977, + 7.872279730973945, + 3.8243847518725254, + 10.670256218445862, + 17.284779647497007, + 2.644376105310677, + 4.927058093891117, + 5.30569605379481, + 1.926392403294464, + 7.845102420161203, + 2.982656548934253, + 1.4997244182188925, + 4.7461574093585215, + 2.8483087547703185, + 12.047151352512428, + 3.4880649403573276, + 11.732943752874197, + 6.9813129049237475, + 3.2384593212065633, + 1.9256466820771712, + 5.7124412355837615, + 8.418832968509168, + 2.752751478600281, + 4.171197545881602, + 8.616904364240284, + 6.284160109950633, + 4.1810477059181155, + 7.4242033037320105, + 1.9692343390160645, + 0.7311217267201264, + 6.935912510464833, + 10.787539601288069, + 12.895746374184167, + 9.301229136757724, + 10.965974552589072, + 10.062736049997376, + 4.195976793683249, + 7.098242051931191, + 5.019460485585374, + 4.005491424762978, + 2.5337226893140006, + 1.3647916549187198, + 9.121321684423284, + 1.818136863872882, + 7.744568721567811, + 9.401422362119996, + 5.36004998386494, + 5.590598935913978, + 13.76757142388392, + 17.320957993486164, + 8.091359695868857, + 3.056244034539619, + 11.46875965833262, + 1.0895965516554755, + 2.409659052296855, + 5.310917476297208, + 5.0474420794264265, + 10.065747029572861, + 8.399995484071185, + 5.999303540119944, + 10.570025627681863, + 16.68462394752922, + 10.523661692873938, + 7.319113255288522, + -0.07188241454945812, + 4.120672217814852, + 3.850016635488781, + 14.561860000216683, + 2.7809199910271896, + 11.597484092134568, + 6.473066959556584, + 7.602216633263769, + 6.362604379349993, + 6.265031461356174, + 2.1327429654368624, + 1.9203431272301326, + 5.387341322372937, + 3.294921420620403, + 2.8302969183486257, + 10.377576340374638, + 12.695367528603342, + 8.089813178847363, + 8.618589782571492, + 14.31462998390479, + 6.229088516361773, + 3.6173224162494, + 8.573328912072396, + 14.068835777185974, + 19.519186032954707, + 2.645542358212573, + 11.057659219487391, + 7.116304508486813, + 12.203057597332762, + 4.478951681107789, + 13.201334815401736, + 2.2535400933557805, + 17.589185003221452, + 10.917238694141234, + 5.944176212265241, + 2.914926419776223, + 10.427571072431162, + 7.305542662732612, + 7.136485673917394, + 3.473006149253745, + -0.8948036838159846, + 6.201274413655002, + 4.620054753231987, + 8.522818690465863, + 1.7236331316200588, + 7.381441868078257, + 3.0983349286181285, + 4.8533996369185095, + 9.404158761029409, + 3.199212957368143, + 6.869418523046383, + 4.075155979302653, + 11.3952442006207, + 5.9819054694834, + 11.939881048529529, + 1.2327216484910748, + 9.520996289253404, + 3.5448227286349776, + 11.366415461847787, + 6.393261931006413, + 3.7689547564428887, + 8.039514583666621, + 3.7843861480103103, + 7.333510258233163, + 2.3747919230315953, + 0.8808992404321346, + 16.19267979954547, + 6.045912993968392, + 6.976194046073316, + 7.562195090630139, + 6.800721179390649, + 2.480441393050849, + 10.909191504815636, + 1.729538434904268, + 1.8707473024376742, + 0.6245858936561355, + 10.40459783669121, + 3.3411429792009937, + 11.066438850385445, + 5.808740731181803, + 13.650547610477856, + 2.967127570033277, + 3.816920031069716, + 11.49549251788382, + 8.896877285673233, + 3.838830559941911, + 2.8170253237765004, + 7.634432948206113, + 8.981233283470715, + 1.7326646640660939, + 9.386419189514582, + 1.5785057335161181, + 5.879091645985078, + 5.465699169142491, + 4.433684814922784, + 3.8177536189397165, + 2.4048289956555164, + 10.325611392051856, + 7.6074803442300984, + 3.0375230618436166, + 3.4273699234055215, + 2.572549951305657, + 5.069830743631792, + 7.058062632929908, + 3.4135514271495384, + 4.966608680610329, + 2.915932092366684, + 3.3253092292052147, + 12.436111994293643, + 4.67475873038095, + 3.65708240545015, + 6.37197924529851, + 16.56232837237978, + 2.4184801906818314, + 5.727536425813394, + 2.4067812717509782, + 3.7479395372103377, + 4.357959643164212, + 7.165247823105613, + 3.115714363474604, + 4.554665308156477, + 5.376398009703256, + 4.381187068919068, + 2.0932624617665345, + 5.842986192059503, + 20.390985540233196, + 2.979502069102285, + 8.754246686289934, + 13.775377927488867, + 5.559315982479618, + 5.057133610748745, + 10.52804827793297, + 3.0287108774118243, + 3.9188253218804037, + 4.249590936570091, + 9.05470715769896, + 10.103443000619611, + 1.4257378401352638, + 10.234732460655913, + 18.51017035543307, + 3.742531403422764, + 5.682302063107566, + 12.052165270043073, + 4.2469082705562595, + 3.1962885130444167, + 5.631753203632753, + 3.828289353783408, + 2.3368296936625397, + 13.114802646740193, + 4.6819579463944265, + 4.4347380132450684, + 6.819121063304491, + 5.088701749783405, + 7.159051819363544, + 7.733015419190123, + 7.742955730548548, + 10.542708203338568, + 9.712064503322658, + 4.373533894658112, + 4.301228811426256, + 3.12170971164312, + 2.2179477278952398, + 10.84119574001116, + 7.114318566169389, + 4.096600466653108, + 9.642091012688123, + 3.0947128735013534, + 3.14344292302663, + 5.680906971592573, + 9.285082770663648, + 10.636217269201904, + 3.6790083186093123, + 6.420833606247791, + 5.481767881878449, + 9.402526274613027, + 7.57750145845892, + 17.423540501737804, + 12.111981679281225, + 2.115230798899664, + 2.348768957826344, + 2.758855452517031, + 3.6692564484876717, + 4.077931749222289, + 5.881245329861467, + 20.71532564835613, + 17.34422736518518, + 9.611104553639624, + 0.8589880967324383, + 12.449212352595664, + 8.980197224567005, + 4.8875616041211565, + 8.846735256556087, + 16.957494350302664, + 4.805564824787897, + 8.964989170975489, + 4.2911895405364895, + 1.930967355657544, + 2.1676659305603234, + 0.9233793256699923, + 2.112532122381978, + 6.9371029470728205, + 6.768823511173058, + 2.04085544817848, + 3.670871516056362, + 0.3963597695790085, + 3.6025896704276534, + 9.584272616464322, + 11.15412511310908, + 5.738400719657727, + 13.280705656341217, + 9.155549718844648, + 3.47788305538066, + 6.184206312746015, + 1.6382341393030238, + 2.8484947110270165, + 3.785737569930755, + 9.57791145430953, + 4.21649053438879, + 5.057159291263697, + 4.587668407155297, + 2.229396331836379, + 14.030088066682668, + 4.299912064495803, + 6.197549969436092, + 9.636493224325857, + 11.415592200763612, + 2.91287781057633, + 9.473267442479868, + 12.181693147978782, + 8.420632293781814, + 7.114443764932338, + 3.6829951311847617, + 8.965626018208413, + 10.961085374384197, + 7.950627443018606, + 4.221227283607655 + ], + "yaxis": "y" + } + ], + "layout": { + "legend": { + "title": { + "text": "Treatment" + }, + "tracegroupgap": 0 + }, + "template": { + "data": { + "bar": [ + { + "error_x": { + "color": "#2a3f5f" + }, + "error_y": { + "color": "#2a3f5f" + }, + "marker": { + "line": { + "color": "#E5ECF6", + "width": 0.5 + }, + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "bar" + } + ], + "barpolar": [ + { + "marker": { + "line": { + "color": "#E5ECF6", + "width": 0.5 + }, + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "barpolar" + } + ], + "carpet": [ + { + "aaxis": { + "endlinecolor": "#2a3f5f", + "gridcolor": "white", + "linecolor": "white", + "minorgridcolor": "white", + "startlinecolor": "#2a3f5f" + }, + "baxis": { + "endlinecolor": "#2a3f5f", + "gridcolor": "white", + "linecolor": "white", + "minorgridcolor": "white", + "startlinecolor": "#2a3f5f" + }, + "type": "carpet" + } + ], + "choropleth": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "choropleth" + } + ], + "contour": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "contour" + } + ], + "contourcarpet": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "contourcarpet" + } + ], + "heatmap": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "heatmap" + } + ], + "heatmapgl": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "heatmapgl" + } + ], + "histogram": [ + { + "marker": { + "pattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + } + }, + "type": "histogram" + } + ], + "histogram2d": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "histogram2d" + } + ], + "histogram2dcontour": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "histogram2dcontour" + } + ], + "mesh3d": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "type": "mesh3d" + } + ], + "parcoords": [ + { + "line": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "parcoords" + } + ], + "pie": [ + { + "automargin": true, + "type": "pie" + } + ], + "scatter": [ + { + "fillpattern": { + "fillmode": "overlay", + "size": 10, + "solidity": 0.2 + }, + "type": "scatter" + } + ], + "scatter3d": [ + { + "line": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatter3d" + } + ], + "scattercarpet": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattercarpet" + } + ], + "scattergeo": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattergeo" + } + ], + "scattergl": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattergl" + } + ], + "scattermapbox": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scattermapbox" + } + ], + "scatterpolar": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterpolar" + } + ], + "scatterpolargl": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterpolargl" + } + ], + "scatterternary": [ + { + "marker": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "type": "scatterternary" + } + ], + "surface": [ + { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + }, + "colorscale": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "type": "surface" + } + ], + "table": [ + { + "cells": { + "fill": { + "color": "#EBF0F8" + }, + "line": { + "color": "white" + } + }, + "header": { + "fill": { + "color": "#C8D4E3" + }, + "line": { + "color": "white" + } + }, + "type": "table" + } + ] + }, + "layout": { + "annotationdefaults": { + "arrowcolor": "#2a3f5f", + "arrowhead": 0, + "arrowwidth": 1 + }, + "autotypenumbers": "strict", + "coloraxis": { + "colorbar": { + "outlinewidth": 0, + "ticks": "" + } + }, + "colorscale": { + "diverging": [ + [ + 0, + "#8e0152" + ], + [ + 0.1, + "#c51b7d" + ], + [ + 0.2, + "#de77ae" + ], + [ + 0.3, + "#f1b6da" + ], + [ + 0.4, + "#fde0ef" + ], + [ + 0.5, + "#f7f7f7" + ], + [ + 0.6, + "#e6f5d0" + ], + [ + 0.7, + "#b8e186" + ], + [ + 0.8, + "#7fbc41" + ], + [ + 0.9, + "#4d9221" + ], + [ + 1, + "#276419" + ] + ], + "sequential": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ], + "sequentialminus": [ + [ + 0, + "#0d0887" + ], + [ + 0.1111111111111111, + "#46039f" + ], + [ + 0.2222222222222222, + "#7201a8" + ], + [ + 0.3333333333333333, + "#9c179e" + ], + [ + 0.4444444444444444, + "#bd3786" + ], + [ + 0.5555555555555556, + "#d8576b" + ], + [ + 0.6666666666666666, + "#ed7953" + ], + [ + 0.7777777777777778, + "#fb9f3a" + ], + [ + 0.8888888888888888, + "#fdca26" + ], + [ + 1, + "#f0f921" + ] + ] + }, + "colorway": [ + "#636efa", + "#EF553B", + "#00cc96", + "#ab63fa", + "#FFA15A", + "#19d3f3", + "#FF6692", + "#B6E880", + "#FF97FF", + "#FECB52" + ], + "font": { + "color": "#2a3f5f" + }, + "geo": { + "bgcolor": "white", + "lakecolor": "white", + "landcolor": "#E5ECF6", + "showlakes": true, + "showland": true, + "subunitcolor": "white" + }, + "hoverlabel": { + "align": "left" + }, + "hovermode": "closest", + "mapbox": { + "style": "light" + }, + "paper_bgcolor": "white", + "plot_bgcolor": "#E5ECF6", + "polar": { + "angularaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "bgcolor": "#E5ECF6", + "radialaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + } + }, + "scene": { + "xaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + }, + "yaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + }, + "zaxis": { + "backgroundcolor": "#E5ECF6", + "gridcolor": "white", + "gridwidth": 2, + "linecolor": "white", + "showbackground": true, + "ticks": "", + "zerolinecolor": "white" + } + }, + "shapedefaults": { + "line": { + "color": "#2a3f5f" + } + }, + "ternary": { + "aaxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "baxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + }, + "bgcolor": "#E5ECF6", + "caxis": { + "gridcolor": "white", + "linecolor": "white", + "ticks": "" + } + }, + "title": { + "x": 0.05 + }, + "xaxis": { + "automargin": true, + "gridcolor": "white", + "linecolor": "white", + "ticks": "", + "title": { + "standoff": 15 + }, + "zerolinecolor": "white", + "zerolinewidth": 2 + }, + "yaxis": { + "automargin": true, + "gridcolor": "white", + "linecolor": "white", + "ticks": "", + "title": { + "standoff": 15 + }, + "zerolinecolor": "white", + "zerolinewidth": 2 + } + } + }, + "title": { + "text": "Scatter Plot of Outcome vs. Score by Treatment Status" + }, + "xaxis": { + "anchor": "y", + "domain": [ + 0, + 1 + ], + "title": { + "text": "Score" + } + }, + "yaxis": { + "anchor": "x", + "domain": [ + 0, + 1 + ], + "title": { + "text": "Outcome" + } + } + } + }, + "text/html": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], "source": [ "fig = px.scatter(\n", " x=df['score'],\n", @@ -324,9 +6570,45 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Call: rdrobust\n", + "Number of Observations: 1000\n", + "Polynomial Order Est. (p): 1\n", + "Polynomial Order Bias (q): 2\n", + "Kernel: Triangular\n", + "Bandwidth Selection: mserd\n", + "Var-Cov Estimator: NN\n", + "\n", + " Left Right\n", + "------------------------------------------------\n", + "Number of Observations 493 507\n", + "Number of Unique Obs. 493 507\n", + "Number of Effective Obs. 213 224\n", + "Bandwidth Estimation 0.542 0.542\n", + "Bandwidth Bias 0.864 0.864\n", + "rho (h/b) 0.628 0.628\n", + "\n", + "Method Coef. S.E. t-stat P>|t| 95% CI \n", + "-------------------------------------------------------------------------\n", + "Conventional 4.64 3.052 1.52 1.285e-01 [-1.343, 10.622]\n", + "Robust - - 1.414 1.575e-01 [-1.952, 12.054]\n" + ] + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "rdrobust_fuzzy_noadj = rdrobust(y=df['y'], x=df['score'], fuzzy=df['d'], c=0.0)\n", "rdrobust_fuzzy_noadj" @@ -347,9 +6629,45 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Call: rdrobust\n", + "Number of Observations: 1000\n", + "Polynomial Order Est. (p): 1\n", + "Polynomial Order Bias (q): 2\n", + "Kernel: Triangular\n", + "Bandwidth Selection: mserd\n", + "Var-Cov Estimator: NN\n", + "\n", + " Left Right\n", + "------------------------------------------------\n", + "Number of Observations 493 507\n", + "Number of Unique Obs. 493 507\n", + "Number of Effective Obs. 211 223\n", + "Bandwidth Estimation 0.54 0.54\n", + "Bandwidth Bias 0.88 0.88\n", + "rho (h/b) 0.614 0.614\n", + "\n", + "Method Coef. S.E. t-stat P>|t| 95% CI \n", + "-------------------------------------------------------------------------\n", + "Conventional 2.802 2.326 1.205 2.283e-01 [-1.757, 7.361]\n", + "Robust - - 1.097 2.728e-01 [-2.367, 8.381]\n" + ] + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "rdrobust_fuzzy = rdrobust(y=df['y'], x=df['score'], fuzzy=df['d'], covs=df[cov_names], c=0.0)\n", "rdrobust_fuzzy" @@ -385,7 +6703,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 12, "metadata": {}, "outputs": [], "source": [ @@ -401,9 +6719,196 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMRegressor was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Method Coef. S.E. t-stat P>|t| 95% CI\n", + "-------------------------------------------------------------------------\n", + "Conventional 0.979 0.726 1.348 1.777e-01 [-0.445, 2.403]\n", + "Robust - - 0.994 3.201e-01 [-0.831, 2.541]\n", + "Design Type: Fuzzy\n", + "Cutoff: 0\n", + "First Stage Kernel: triangular\n", + "Final Bandwidth: [0.61170069]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n", + "/home/ubuntu/.venv/lib/python3.12/site-packages/sklearn/utils/validation.py:2739: UserWarning:\n", + "\n", + "X does not have valid feature names, but LGBMClassifier was fitted with feature names\n", + "\n" + ] + } + ], "source": [ "ml_g = LGBMRegressor(n_estimators=500, learning_rate=0.01, verbose=-1)\n", "ml_m = LGBMClassifier(n_estimators=500, learning_rate=0.01, verbose=-1)\n", @@ -442,7 +6947,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 14, "metadata": {}, "outputs": [], "source": [ @@ -455,7 +6960,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -493,9 +6998,24 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Method Coef. S.E. t-stat P>|t| 95% CI\n", + "-------------------------------------------------------------------------\n", + "Conventional 1.876 0.101 18.619 2.241e-77 [1.678, 2.073]\n", + "Robust - - 15.992 1.443e-57 [1.612, 2.062]\n", + "Design Type: Sharp\n", + "Cutoff: 0\n", + "First Stage Kernel: triangular\n", + "Final Bandwidth: [0.61771229]\n" + ] + } + ], "source": [ "rdflex_sharp_stack = RDFlex(dml_data_sharp,\n", " ml_g,\n", @@ -516,9 +7036,24 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 17, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Method Coef. S.E. t-stat P>|t| 95% CI\n", + "-------------------------------------------------------------------------\n", + "Conventional 1.726 0.609 2.835 4.577e-03 [0.533, 2.918]\n", + "Robust - - 2.642 8.243e-03 [0.483, 3.261]\n", + "Design Type: Fuzzy\n", + "Cutoff: 0\n", + "First Stage Kernel: triangular\n", + "Final Bandwidth: [0.54550506]\n" + ] + } + ], "source": [ "rdflex_fuzzy_stack = RDFlex(dml_data_fuzzy,\n", " ml_g,\n", @@ -540,7 +7075,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, "outputs": [], "source": [ @@ -556,9 +7091,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 19, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAGGCAYAAABmGOKbAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjYsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvq6yFwwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAd+BJREFUeJzt3XdYFFfbBvB7AQEbKAqiiIglKCoW1NgLUcFu7MZuNLH3Eo2KHWOaxvgaS6zYe4u9G7vGglgjNgSxICBVdp/vDz5WV0ooW2C5f9e1F+zMmdmzDDvPPmfOnKMQEQERERERERERaZ2JoStAREREREREZKyYdBMRERERERHpCJNuIiIiIiIiIh1h0k1ERERERESkI0y6iYiIiIiIiHSESTcRERERERGRjjDpJiIiIiIiItIRJt1EREREREREOsKkm4iIiIiIiEhHmHQTpYNCocDQoUMNXY1soVGjRmjUqJGhq0FEREQpYKwm0g8m3UQAbt68iY4dO8LJyQmWlpZwcHBA06ZNsXDhQkNXLcP69OkDhUKhfuTLlw+lSpVCx44dsW3bNqhUKkNXkYiIKF1WrVqlEds+fnz33XeGrl66MVYT5Qxmhq4AkaGdPXsWjRs3RokSJTBgwADY29vj6dOnOH/+PBYsWIBhw4YZuooZZmFhgeXLlwMAoqOj8fjxY+zZswcdO3ZEo0aNsGvXLlhZWenktQ8dOqST/RIREc2YMQPOzs4ayypWrGig2mQOYzWR8WPSTTne7NmzYW1tjUuXLqFAgQIa60JCQvRen8jISOTNm1cr+zIzM0OPHj00ls2aNQtz587FxIkTMWDAAGzatEkrr/Upc3NzneyXiIioefPmqF69uqGroRWM1UTGj93LKcf7999/UaFChSQJNwDY2dklu83OnTtRsWJFWFhYoEKFCjhw4IDG+sePH2Pw4MFwcXFB7ty5UahQIXTq1AmPHj3SKJfYTe7kyZMYPHgw7OzsULx4cQDAtGnToFAocOfOHXTu3BlWVlYoVKgQRowYgZiYmEy95++++w7NmjXDli1bcO/ePY11+/fvR/369ZE3b17kz58fLVu2xK1btzTKBAcHo2/fvihevDgsLCxQtGhRtG3bVuP9JXef2OPHj9GmTRvkzZsXdnZ2GDVqFA4ePAiFQoETJ05obFuxYkX4+/ujcePGyJMnDxwcHDBv3rxMvW8iIjJ+CoUC06ZNS7K8ZMmS6NOnj0a5lB6PHj3CiRMnUlxfsmRJAEDv3r1RuHBhvH//PsnrNWvWDC4uLhl+H4zVRMaDV7opx3NycsK5c+fg5+eXpq5pZ86cwfbt2zF48GDkz58fv/32Gzp06IAnT56gUKFCAIBLly7h7Nmz6Nq1K4oXL45Hjx5h8eLFaNSoEfz9/ZEnTx6NfQ4ePBi2traYOnUqIiMjNdZ17twZJUuWhI+PD86fP4/ffvsNoaGhWLNmTabed8+ePXHo0CEcPnwYn332GQBg7dq16N27Nzw9PfHDDz8gKioKixcvRr169fDPP/+ov2R06NABt27dwrBhw1CyZEmEhITg8OHDePLkibrMpyIjI+Hh4YGgoCCMGDEC9vb2WL9+PY4fP55s+dDQUHh5eaF9+/bo3Lkztm7digkTJqBSpUpo3rx5pt47ERFlb2FhYXj16pXGssKFC6drH2vXrk2ybPLkyQgJCUG+fPlQvnz5JGXevn2L0aNHqxvle/bsiTVr1uDgwYNo1aqVulxwcDCOHTsGb2/vdNXpU4zVREZCiHK4Q4cOiampqZiamkrt2rVl/PjxcvDgQYmLi0tSFoCYm5vLgwcP1MuuX78uAGThwoXqZVFRUUm2PXfunACQNWvWqJetXLlSAEi9evUkPj5eo7y3t7cAkDZt2mgsHzx4sACQ69evp/q+evfuLXnz5k1x/T///CMAZNSoUSIiEhERIQUKFJABAwZolAsODhZra2v18tDQUAEgP/74Y6qv37BhQ2nYsKH6+c8//ywAZOfOnepl0dHRUq5cOQEgx48f19j2079VbGys2NvbS4cOHVJ9XSIiMl6JcTO5RyIA4u3tnWRbJycn6d27d4r7njdvXpLY8zGVSiWtWrWSfPnyya1bt0RERKlUSvHixaVLly4aZX/55RdRKBTy8OHDVN8PYzVRzsDu5ZTjNW3aFOfOnUObNm1w/fp1zJs3D56ennBwcMDu3buTlG/SpAlKly6tfu7m5gYrKys8fPhQvSx37tzq39+/f4/Xr1+jTJkyKFCgAK5evZpknwMGDICpqWmy9RsyZIjG88SB3f7666/0vdFP5MuXDwAQEREBADh8+DDevn2Lbt264dWrV+qHqakpPv/8c3Urd+7cuWFubo4TJ04gNDQ0za934MABODg4oE2bNupllpaWGDBgQIr1+/geN3Nzc9SsWVPj70xERDnTokWLcPjwYY1HZhw/fhwTJ07EsGHD0LNnz2TLzJw5E3v37sWqVavg6uoKADAxMUH37t2xe/dudTwFgHXr1qFOnTpJBntLL8ZqIuPApJsIQI0aNbB9+3aEhobi4sWLmDhxIiIiItCxY0f4+/trlC1RokSS7QsWLKgR1KKjozF16lQ4OjrCwsIChQsXhq2tLd6+fYuwsLAk26cWlMuWLavxvHTp0jAxMUlyf3h6vXv3DgCQP39+AMD9+/cBAB4eHrC1tdV4HDp0SD2onIWFBX744Qfs378fRYoUQYMGDTBv3jwEBwen+nqPHz9G6dKloVAoNJaXKVMm2fLFixdPUvbTvzMREeVMNWvWRJMmTTQeGfXs2TN06dIFdevWxS+//JJsmQMHDmD69OmYOHEiOnTooLGuV69eiI6Oxo4dOwAAd+/exZUrV1JM3tODsZrIOPCebqKPmJubo0aNGqhRowY+++wz9O3bF1u2bNG4JyulK9Iiov592LBhWLlyJUaOHInatWvD2toaCoUCXbt2TXbOzY+vjP+XT4NbRvn5+QH4EEgT67V27VrY29snKW9m9uF0MXLkSLRu3Ro7d+7EwYMHMWXKFPj4+ODYsWOoWrWqVuqXlr8zERFRWimVyiTL4uLi0LFjR1hYWGDz5s0asS5RQEAAunfvjqZNm2LWrFlJ1ru6usLd3R2+vr7o1asXfH19YW5ujs6dO2e6zozVRMaBSTdRChKnIgkKCkr3tlu3bkXv3r3x888/q5fFxMTg7du36d7X/fv3Na6EP3jwACqVKsVBUNJq7dq1UCgUaNq0KQCou8zb2dml6YpB6dKlMWbMGIwZMwb3799HlSpV8PPPP8PX1zfZ8k5OTvD394eIaDQcPHjwIFPvg4iI6GMFCxZMEm/j4uKSjefDhw/HtWvXcOrUKRQpUiTJ+ujoaLRv3x4FChTAhg0bYGKSfCfRXr16YfTo0QgKCsL69evRsmVLFCxYMNPvhbGayDiweznleMePH0+2RTbxnumMTPdhamqaZJ8LFy5MtpX9vyxatCjJfgBkalTQuXPn4tChQ+jSpYu6+7qnpyesrKwwZ86cZKc+efnyJQAgKioqyZRlpUuXRv78+REbG5via3p6eiIwMFDjPvmYmBgsW7Ysw++DiIjoU6VLl8apU6c0li1dujRJDF65ciWWLFmCRYsWoWbNmsnua+DAgbh37x527NiRahLdrVs3KBQKjBgxAg8fPkwy73ZGMFYTGQ9e6aYcb9iwYYiKisKXX36JcuXKIS4uDmfPnsWmTZtQsmRJ9O3bN937bNWqFdauXQtra2u4urri3LlzOHLkiHpKsfQICAhAmzZt4OXlhXPnzsHX1xdfffUVKleu/J/bxsfHq1uzY2Ji8PjxY+zevRs3btxA48aNsXTpUnVZKysrLF68GD179kS1atXQtWtX2Nra4smTJ9i3bx/q1q2L33//Hffu3cMXX3yBzp07w9XVFWZmZtixYwdevHiBrl27pliXb7/9Fr///ju6deuGESNGoGjRoli3bh0sLS0BaK/bPBER5Wz9+/fHwIED0aFDBzRt2hTXr1/HwYMHNaYUe/XqFQYPHgxXV1dYWFgkufL75Zdf4sSJE1izZg06dOiAGzdu4MaNG+r1+fLlQ7t27dTPbW1t4eXlhS1btqBAgQJo2bJlmuvLWE1k/Jh0U473008/YcuWLfjrr7+wdOlSxMXFoUSJEhg8eDAmT56MAgUKpHufCxYsgKmpKdatW4eYmBjUrVsXR44cgaenZ7r3tWnTJkydOhXfffcdzMzMMHToUPz4449p2jY2NlY9kEuePHlgZ2cHd3d3TJ06FV9++WWSbnJfffUVihUrhrlz5+LHH39EbGwsHBwcUL9+fXXjg6OjI7p164ajR49i7dq1MDMzQ7ly5bB58+Ykg8t8LF++fDh27BiGDRuGBQsWIF++fOjVqxfq1KmDDh06qAM6ERFRZgwYMAABAQH4888/ceDAAdSvXx+HDx/GF198oS7z7t07xMTEwN/fP9kBzwICAtRXjbdt24Zt27ZprHdyctJIuoGELuZ79+5F586dYWFhkeb6MlYTGT+FcKQDoixp2rRpmD59Ol6+fKnROm9s5s+fj1GjRuHZs2dwcHAwdHWIiIgyZNeuXWjXrh1OnTqF+vXrG7o6WsVYTZQ5vKebiPQmOjpa43lMTAyWLFmCsmXLMogTEVG2tmzZMpQqVQr16tUzdFUyhbGaSPvYvZyI9KZ9+/YoUaIEqlSpgrCwMPj6+uLOnTtYt26doatGRESUIRs3bsSNGzewb98+LFiwINvf98xYTaR9TLqJSG88PT2xfPlyrFu3DkqlEq6urti4cSO6dOli6KoRERFlSLdu3ZAvXz58/fXXGDx4sKGrk2mM1UTax3u6iYiIiIiIiHSE93QTERERERER6QiTbiIiIiIiIiIdyXH3dKtUKjx//hz58+fP9gNdEBHldCKCiIgIFCtWLMlctpS9MV4TERkHxuocmHQ/f/4cjo6Ohq4GERFp0dOnT1G8eHFDV4O0iPGaiMi45ORYneOS7vz58wNIOOhWVlYGrg0REWVGeHg4HB0d1ed2Mh6M10RExoGxOgcm3Yld1KysrBjEiYiMBLsfGx/GayIi45KTY3XO7FRPREREREREpAdMuomIiIiIiIh0hEk3ERERERERkY4w6SYiIiIiIiLSESbdRERERERERDrCpJuIiIiIiIhIR5h0ExEREREREelIjpunOz2Wn36I5acD0r1d//rO6F+/lA5qRERERJ9ivCYioqyMSXcqImLiERwek6HtiIiISD8Yr4mIKCtj0p2K/JZmsLey1FgmELwIjwUAFLGygAKKZLcjIiIi/WC8JiKirEwhImLoSuhTeHg4rK2tERYWBisrq3RvHxUXD9epBwEA/jM8kcecAZuIyFAye06nrIvxmojIODBWcyA1IiIiIiIiIp1h0k1ERERERESkI0y6iYiIiIiIiHSESTcRERGly6lTp9C6dWsUK1YMCoUCO3fu1FgvIpg6dSqKFi2K3Llzo0mTJrh//75hKktERGRgTLqJiIgoXSIjI1G5cmUsWrQo2fXz5s3Db7/9hj/++AMXLlxA3rx54enpiZiY9E/rRURElN0ZNOlevHgx3NzcYGVlBSsrK9SuXRv79+9PsfyqVaugUCg0HpaWlimWJyIiIu1r3rw5Zs2ahS+//DLJOhHB/PnzMXnyZLRt2xZubm5Ys2YNnj9/nuSKOBERUU5g0KS7ePHimDt3Lq5cuYLLly/Dw8MDbdu2xa1bt1LcxsrKCkFBQerH48eP9VhjIiIiSk1AQACCg4PRpEkT9TJra2t8/vnnOHfuXIrbxcbGIjw8XONBRERkDAw6aWXr1q01ns+ePRuLFy/G+fPnUaFChWS3USgUsLe310f1iIiIKJ2Cg4MBAEWKFNFYXqRIEfW65Pj4+GD69Ok6rRsREZEhZJl7upVKJTZu3IjIyEjUrl07xXLv3r2Dk5MTHB0d//OqOMCWcyIiouxg4sSJCAsLUz+ePn1q6CoRERFphcGT7ps3byJfvnywsLDAwIEDsWPHDri6uiZb1sXFBStWrMCuXbvg6+sLlUqFOnXq4NmzZynu38fHB9bW1uqHo6Ojrt4KERFRjpfYG+3Fixcay1+8eJFqTzULCwv1GC+JDyIiImNg8KTbxcUF165dw4ULFzBo0CD07t0b/v7+yZatXbs2evXqhSpVqqBhw4bYvn07bG1tsWTJkhT3z5ZzIiIi/XF2doa9vT2OHj2qXhYeHo4LFy6k2pONiIjIWBn0nm4AMDc3R5kyZQAA7u7uuHTpEhYsWJBqIp0oV65cqFq1Kh48eJBiGQsLC1hYWGitvkRERDndu3fvNGJvQEAArl27BhsbG5QoUQIjR47ErFmzULZsWTg7O2PKlCkoVqwY2rVrZ7hKExERGYjBk+5PqVQqxMbGpqmsUqnEzZs30aJFCx3XioiIiBJdvnwZjRs3Vj8fPXo0AKB3795YtWoVxo8fj8jISHzzzTd4+/Yt6tWrhwMHDnCaTyIiypEMmnRPnDgRzZs3R4kSJRAREYH169fjxIkTOHjwIACgV69ecHBwgI+PDwBgxowZqFWrFsqUKYO3b9/ixx9/xOPHj9G/f39Dvg0iIqIcpVGjRhCRFNcrFArMmDEDM2bM0GOtiIiIsiaDJt0hISHo1asXgoKCYG1tDTc3Nxw8eBBNmzYFADx58gQmJh9uOw8NDcWAAQMQHByMggULwt3dHWfPnk1x4DUiIiIiIiIiQzJo0v3nn3+muv7EiRMaz3/99Vf8+uuvOqwRERERERERkfYYfPRyIiIiIiIiImPFpJuIiIiIiIhIR5h0ExEREREREekIk24iIiIiIiIiHWHSTURERERERKQjTLqJiIiIiIiIdIRJNxEREREREZGOMOkmIiIiIiIi0hEm3UREREREREQ6wqSbiIiIiIiISEeYdBMRERERERHpCJNuIiIiIiIiIh1h0k1ERERERESkI0y6iYiIiIiIiHSESTcRERERERGRjjDpJiIiIiIiItIRJt1EREREREREOsKkm4iIiIiIiEhHmHQTERERERER6QiTbiIiIiIiIiIdYdJNREREWqVUKjFlyhQ4Ozsjd+7cKF26NGbOnAkRMXTViIiI9M7M0BUgIiIi4/LDDz9g8eLFWL16NSpUqIDLly+jb9++sLa2xvDhww1dPSIiIr1i0k1ERERadfbsWbRt2xYtW7YEAJQsWRIbNmzAxYsXDVwzIiIi/WP3ciIiItKqOnXq4OjRo7h37x4A4Pr16zhz5gyaN29u4JoRERHpH690ExERkVZ99913CA8PR7ly5WBqagqlUonZs2eje/fuKW4TGxuL2NhY9fPw8HB9VJWIiEjneKWbiIiItGrz5s1Yt24d1q9fj6tXr2L16tX46aefsHr16hS38fHxgbW1tfrh6OioxxoTERHpDpNuIiIi0qpx48bhu+++Q9euXVGpUiX07NkTo0aNgo+PT4rbTJw4EWFhYerH06dP9VhjIiIi3WH3ciIiItKqqKgomJhotuubmppCpVKluI2FhQUsLCx0XTUiIiK9Y9JNREREWtW6dWvMnj0bJUqUQIUKFfDPP//gl19+Qb9+/QxdNSIiIr1j0k1Gbfnph1h+OiDd2/Wv74z+9UvpoEZERMZv4cKFmDJlCgYPHoyQkBAUK1YM3377LaZOnWroqhEREekdk24yahEx8QgOj8nQdkRElDH58+fH/PnzMX/+fENXhYiIyOAMOpDa4sWL4ebmBisrK1hZWaF27drYv39/qtts2bIF5cqVg6WlJSpVqoS//vpLT7Wl7Ci/pRnsrSw1HkWsPtwzWMTKIsl6eytL5LdkexQREREREWWeQTOL4sWLY+7cuShbtixEBKtXr0bbtm3xzz//oEKFCknKnz17Ft26dYOPjw9atWqF9evXo127drh69SoqVqxogHdAWV3/+qWSdBOPiouH69SDAIDjYxshjzkTbCIiIiIi0g2DXulu3bo1WrRogbJly+Kzzz7D7NmzkS9fPpw/fz7Z8gsWLICXlxfGjRuH8uXLY+bMmahWrRp+//13PdeciIiIiIiI6L9lmUt8SqUSW7ZsQWRkJGrXrp1smXPnzmH06NEayzw9PbFz584U9xsbG4vY2Fj18/DwcK3Ul4iIiIi0gwOfEpExM3jSffPmTdSuXRsxMTHIly8fduzYAVdX12TLBgcHo0iRIhrLihQpguDg4BT37+Pjg+nTp2u1zkRERESkPRz4lIiMmcGTbhcXF1y7dg1hYWHYunUrevfujZMnT6aYeKfXxIkTNa6Oh4eHw9HRUSv7JiIiIqLMSxz49GMCwYvwhN6KRawsoIAi2e2IiLI6g5+pzM3NUaZMGQCAu7s7Ll26hAULFmDJkiVJytrb2+PFixcay168eAF7e/sU929hYQELC4sU1xMRERGRYXHgUyIyZgYdSC05KpVK4x7sj9WuXRtHjx7VWHb48OEU7wEnIiIiIiIiMiSDNhlOnDgRzZs3R4kSJRAREYH169fjxIkTOHgwoVWzV69ecHBwgI+PDwBgxIgRaNiwIX7++We0bNkSGzduxOXLl7F06VJDvg0iIiIiIiKiZBk06Q4JCUGvXr0QFBQEa2truLm54eDBg2jatCkA4MmTJzAx+XAxvk6dOli/fj0mT56MSZMmoWzZsti5cyfn6CYiIiIiIqIsyaBJ959//pnq+hMnTiRZ1qlTJ3Tq1ElHNSIiIiIiIiLSnix3TzcRERERERGRsWDSTURERERERKQjTLqJiIiIiIiIdIRJNxEREREREZGOMOkmIiIiIiIi0hEm3UREREREREQ6wqSbiIiIiIiISEcylHTPmDEDUVFRSZZHR0djxowZma4UERERaQdjNhERkWFlKOmePn063r17l2R5VFQUpk+fnulKERERkXYwZhMRERlWhpJuEYFCoUiy/Pr167Cxscl0pYiIiEg7GLOJiIgMyyw9hQsWLAiFQgGFQoHPPvtMI4grlUq8e/cOAwcO1HoliYiIKH0Ys4mIiLKGdCXd8+fPh4igX79+mD59OqytrdXrzM3NUbJkSdSuXVvrlSQiIqL0YcwmIiLKGtKVdPfu3RsA4OzsjDp16iBXrlw6qRQRERFljqFjdmBgICZMmID9+/cjKioKZcqUwcqVK1G9enW91oOIiMjQ0pV0J2rYsCFUKhXu3buHkJAQqFQqjfUNGjTQSuWIiIgocwwRs0NDQ1G3bl00btwY+/fvh62tLe7fv4+CBQtq/bWIiIiyugwl3efPn8dXX32Fx48fQ0Q01ikUCiiVSq1UjoiIiDLHEDH7hx9+gKOjI1auXKle5uzsrPXXISIiyg4yNHr5wIEDUb16dfj5+eHNmzcIDQ1VP968eaPtOhIREVEGGSJm7969G9WrV0enTp1gZ2eHqlWrYtmyZaluExsbi/DwcI0HERGRMcjQle779+9j69atKFOmjLbrQ0RERFpkiJj98OFDLF68GKNHj8akSZNw6dIlDB8+HObm5up7zT/l4+PDecOJiMgoZehK9+eff44HDx5ouy5ERESkZYaI2SqVCtWqVcOcOXNQtWpVfPPNNxgwYAD++OOPFLeZOHEiwsLC1I+nT5/qscZERES6k6Er3cOGDcOYMWMQHByMSpUqJRkR1c3NTSuVIyIioswxRMwuWrQoXF1dNZaVL18e27ZtS3EbCwsLWFhYaL0uREREhpahpLtDhw4AgH79+qmXKRQKiAgHUiMiIspCDBGz69ati7t372osu3fvHpycnLT+WkRERFldhpLugIAAbdeDiIiIdMAQMXvUqFGoU6cO5syZg86dO+PixYtYunQpli5dqve6EBERGVqGkm62VBMREWUPhojZNWrUwI4dOzBx4kTMmDEDzs7OmD9/Prp37673uhARERlahgZSA4C1a9eibt26KFasGB4/fgwAmD9/Pnbt2qW1yhEREVHmGSJmt2rVCjdv3kRMTAxu376NAQMG6Oy1iIiIsrIMJd2J04C0aNECb9++Vd8PVqBAAcyfP1+b9SMiIqJMYMwmIiIyrAwl3QsXLsSyZcvw/fffw9TUVL28evXquHnzptYqR0RERJnDmE1ERGRYGUq6AwICULVq1STLLSwsEBkZmelKERERkXYwZhMRERlWhpJuZ2dnXLt2LcnyAwcOoHz58pmtExEREWkJYzYREZFhZWj08tGjR2PIkCGIiYmBiODixYvYsGEDfHx8sHz5cm3XkYiIiDKIMZuIiMiwMpR09+/fH7lz58bkyZMRFRWFr776CsWKFcOCBQvQtWtXbdeRiIiIMogxm4iIyLAylHQDQPfu3dG9e3dERUXh3bt3sLOz02a9iIiISEsYs4mIiAwnw/N0J8qTJ0+Gg7ePjw9q1KiB/Pnzw87ODu3atcPdu3dT3WbVqlVQKBQaD0tLywy9PhERUU6SmZhNREREGZPmK93VqlXD0aNHUbBgQVStWhUKhSLFslevXk3TPk+ePIkhQ4agRo0aiI+Px6RJk9CsWTP4+/sjb968KW5nZWWlkZynVhciIqKcRhcxm4iIiDImzUl327ZtYWFhAQBo166dVl78wIEDGs9XrVoFOzs7XLlyBQ0aNEhxO4VCAXt7e63UgYiIyNjoImYTERFRxqQ56fb29k72d20KCwsDANjY2KRa7t27d3BycoJKpUK1atUwZ84cVKhQQSd1IiIiym70EbOJiIgobTJ0T/elS5dw4cKFJMsvXLiAy5cvZ6giKpUKI0eORN26dVGxYsUUy7m4uGDFihXYtWsXfH19oVKpUKdOHTx79izZ8rGxsQgPD9d4EBER5RS6iNnZgVIl6t8vBrzReE5ERKRPGUq6hwwZgqdPnyZZHhgYiCFDhmSoIkOGDIGfnx82btyYarnatWujV69eqFKlCho2bIjt27fD1tYWS5YsSba8j48PrK2t1Q9HR8cM1Y+IiCg70kXMzuoO+AWhyS8n1c/7rLyEej8cwwG/IAPWioiIcqoMJd3+/v6oVq1akuVVq1aFv79/uvc3dOhQ7N27F8ePH0fx4sXTtW2uXLlQtWpVPHjwINn1EydORFhYmPqR3BcPIiIiY6XtmJ3VHfALwiDfq3gRHquxPDgsBoN8rzLxJiIivctQ0m1hYYEXL14kWR4UFAQzs7RP/S0iGDp0KHbs2IFjx47B2dk53XVRKpW4efMmihYtmmJdraysNB5EREQ5hbZidnagVAmm7/FHch3JE5dN3+PPruZERKRXGUq6mzVrpr6CnOjt27eYNGkSmjZtmub9DBkyBL6+vli/fj3y58+P4OBgBAcHIzo6Wl2mV69emDhxovr5jBkzcOjQITx8+BBXr15Fjx498PjxY/Tv3z8jbyXdeI8YERFlJ9qK2dnBxYA3CAqLSXG9AAgKi8HFgDf6qxQREeV4GWri/umnn9CgQQM4OTmhatWqAIBr166hSJEiWLt2bZr3s3jxYgBAo0aNNJavXLkSffr0AQA8efIEJiYf2gZCQ0MxYMAABAcHo2DBgnB3d8fZs2fh6uqakbeSLgf8guC9+5b6eZ+Vl1DU2hLerV3hVTH5K+1ERESGpK2YnR2ERKSccGekHBERkTZkKOl2cHDAjRs3sG7dOly/fh25c+dG37590a1bN+TKlSvN+xH576vEJ06c0Hj+66+/4tdff01vlTMt8R6xT2uceI/Y4h7VmHgTEVGWo62YnR3Y5bfUajkiIiJtyPDNXHnz5sU333yjzbpkWf91j5gCCfeINXW1h6mJQs+1IyIiSl1Oidk1nW1Q1NoSwWExycZsBQB7a0vUdLbRd9WIiCgHS3PSvXv3bjRv3hy5cuXC7t27Uy3bpk2bTFcsK0nPPWK1SxfSX8WIiIiSkVNjtqmJAt6tXTHI9yoUgEbindgk7t3alQ3kRESkV2lOutu1a4fg4GDY2dmhXbt2KZZTKBRQKpXaqFuWwXvEiIgoO8nJMdurYlEs7lEN3rtvaUwbZs8xWIiIyEDSnHSrVKpkf88JeI8YERFlJzk5ZgMJiXfdMoVRadohAMCqvjVQv6wtr3ATEZFBpHnKMBsbG7x69QoA0K9fP0REROisUllN4j1iKYVqBYCivEeMiIiyiKwWs+fOnQuFQoGRI0fq7TU/TrBrOtsw4SYiIoNJc9IdFxeH8PBwAMDq1asRE5NzulIn3iMGIEnizXvEiIgoq8lKMfvSpUtYsmQJ3NzcDFYHIiIiQ0pz9/LatWujXbt2cHd3h4hg+PDhyJ07d7JlV6xYobUKZhW8R4yIiLKLrBKz3717h+7du2PZsmWYNWuWzl6HiIgoK0vzlW5fX1+0aNEC7969AwCEhYUhNDQ02Yex8qpYFEdGN1Q/X9W3Bs5M8GDCTUREWUpWidlDhgxBy5Yt0aRJE52+DhERUVaW5ivdRYoUwdy5cwEAzs7OWLt2LQoVynnTY/EeMSIiyuqyQszeuHEjrl69ikuXLqWpfGxsLGJjP/QkS+weT0RElN1laCC1xo0bw9zcXGeVIiIioowzdMx++vQpRowYgXXr1sHSMm0ze/j4+MDa2lr9cHR01HEtiYiI9IMDqRERERkZQ8fsK1euICQkBNWqVYOZmRnMzMxw8uRJ/PbbbzAzM0t2bvCJEyciLCxM/Xj69Kle60xERKQrHEiNiIjIyBg6Zn/xxRe4efOmxrK+ffuiXLlymDBhAkxNTZNsY2FhAQsLC63XhYiIyNDSnHT7+vri119/xb///guFQoGwsDBe7SYiIsqCDB2z8+fPj4oVK2osy5s3LwoVKpRkORERkbHjQGqU4yhVov79YsAb1C9rywHxiMioMGYTERFlHWlOuj8WEBCg/j0mJibNg6QQGdoBvyB4776lft5n5SUU5VzrRGTEskrMPnHihEFel4iIyNDSPJDax1QqFWbOnAkHBwfky5cPDx8+BABMmTIFf/75p1YrSKQtB/yCMMj3Kl6Ex2osDw6LwSDfqzjgF2SgmhER6Q5jNhERkWFlKOmeNWsWVq1ahXnz5mlMQ1KxYkUsX75ca5Uj0halSjB9jz8kmXWJy6bv8dfoek5EZAwYs4mIiAwrQ93L16xZg6VLl+KLL77AwIED1csrV66MO3fuaK1yRNpyMeANgsJSHkRIAASFxeBiwBvULs37HrOS5acfYvnpgP8u+In+9Z3Rv34pHdSIKHthzCYiIjKsDCXdgYGBKFOmTJLlKpUK79+/z3SliLQtJCJto/amtRzpT0RMPILD039cImLidVAbouyHMZuIiMiwMpR0u7q64vTp03ByctJYvnXrVlStWlUrFSPSJrv8aRs4KK3lSH/yW5rB3krzuAhEfW9+ESsLKJB09Pn8lhk6vREZHcZsIiIiw8rQt9KpU6eid+/eCAwMhEqlwvbt23H37l2sWbMGe/fu1XYdiTKtprMNilpbIjgsJtn7uhUA7K0tUdPZRt9Vo//Qv36pJN3Eo+Li4Tr1IADg+NhGyGPOBJsoJYzZREREhpWhgdTatm2LPXv24MiRI8ibNy+mTp2K27dvY8+ePWjatKm260iUaaYmCni3dgWAJNdEE597t3blfN1EZHQYs4mIiAwrw5eH6tevj8OHD2uzLkQ65VWxKBb3qAbv3bc0pg2z5zzdRGTkGLOJiIgMJ1N9Mq9cuYLbt28DACpUqMB7wyjL86pYFHXLFEalaYcAAKv61kD9sra8wk1ERo8xm4iIyDAylHSHhISga9euOHHiBAoUKAAAePv2LRo3boyNGzfC1tZWm3Uk0qqPE+yazjZMuInIqDFmU3alVH0YheViwBs2khNRtpWhe7qHDRuGiIgI3Lp1C2/evMGbN2/g5+eH8PBwDB8+XNt1JCIiogxizKbs6IBfEJr8clL9vM/KS6j3wzEc8AsyYK2IiDImQ1e6Dxw4gCNHjqB8+fLqZa6urli0aBGaNWumtcoRERFR5jBmU3ZzwC8Ig3yvJpltJDgsBoN8r2Jxj2och4VIy5affojlpwPSvV3/+s5JZpmhpDKUdKtUKuTKlSvJ8ly5ckGlUmW6UkRERKQdjNmUnShVgul7/JOd3lOQMOPI9D3+aOpqz67mRFoUEROP4PCYDG1H/y1D3cs9PDwwYsQIPH/+XL0sMDAQo0aNwhdffKG1yhEREVHmMGZTdnIx4A2CwlL+4i8AgsJicDHgjf4qRZQD5Lc0g72VpcajiJWFen0RK4sk6+2tLJHfMlPjcucYGfor/f7772jTpg1KliwJR0dHAMDTp09RsWJF+Pr6arWCRESUvbHLmmExZlN2EhKRtittaS1HRGnTv36pJDE3Ki4erlMPAgCOj22EPOZMsDMqQ385R0dHXL16FUeOHMGdO3cAAOXLl0eTJk20WjkiouRwRNvshV3WDIsxm7ITu/yWWi1HRJQVpKt7+bFjx+Dq6orw8HAoFAo0bdoUw4YNw7Bhw1CjRg1UqFABp0+fTvP+fHx8UKNGDeTPnx92dnZo164d7t69+5/bbdmyBeXKlYOlpSUqVaqEv/76Kz1vg4iyMY5om/2wy5phaDtmE+lDTWcbFLW2RErNqAoARa0tUdPZRp/VIiLKlHR9o5k/fz4GDBgAKyurJOusra3x7bff4pdffkH9+vXTtL+TJ09iyJAhqFGjBuLj4zFp0iQ0a9YM/v7+yJs3b7LbnD17Ft26dYOPjw9atWqF9evXo127drh69SoqVqyYnrdDRNkMR7TNnthlzTC0HbOJ9MHURAHv1q4Y5HsVCkDjfJ+YiHu3dmXvJiLKVtJ1pfv69evw8vJKcX2zZs1w5cqVNO/vwIED6NOnDypUqIDKlStj1apVePLkSar7WLBgAby8vDBu3DiUL18eM2fORLVq1fD777+n560QUTbzXyPaAgkj2n7c9ZwoJ9N2zCbSF6+KRbG4RzXYfdQjBgDsrS3ZuEpE2VK6Li28ePEi2WlH1DszM8PLly8zXJmwsDAAgI1Nyl2Gzp07h9GjR2ss8/T0xM6dO9P3YpGRgKlp0uWmpoClpWa5j8XFI3fc/9+bGB0NmOdPuezHTEyA3Lk/PI+KAiSF5EChAPLkyVjZ6GggtSlgPu5BkJ6yMTGAUqmdsnnyJNQbAGJjgfhU7ttMT9ncuRP+zgAQFwe8f598ubh4KEQFUaShLJDw/5D4v5Kesu/fJ5RPiYUFYGaW/rLx8Ql/i5SYmwOJn9P0lFUqE45dSnLlSiif3rIqVcL/WibLXnwUmrYRbW8HonbJggl/L4v//8ImkvA5Ssl/fe4zWvbTzz3PER/KRsd+OJdGRgLvPwpH6fncp/S+Secxm0iXvCoWRd0yhVFp2iEAwKq+NTh+BxFlW+lKuh0cHODn54cyZcoku/7GjRsoWjRjrY8qlQojR45E3bp1U+0mHhwcjCJFimgsK1KkCIKDg5MtHxsbi9iPko7w8PCEX4oVS/4FWrQA9u378NzOTuPLeh4At///d+WVBsDJD/eWomRJ4NWr5PdbvTpw6dKH566uwOPHyZd1dQVu3frwvEYNwN8/+bJOTsCjRx+eN2gAXL6cfNnChYGPv2A1b65Z/4/lyaOZIHToAKR27/zHX3x79gS2bk257Lt3H76Af/stsHp1ymVDQgBb24TfR48G/ve/lMsGBCQcAwD4/nvgp5+SLZYHQJl+i3Df1ilhwZw5wPTpKe/34sWEYwAACxYA48enXPb4caBRo4Tfly4Fhg5NuezevUDLlgm/r1sH9O2bctnNm4FOnRJ+37ED6Nw55bIrVwJ9+iT8fvAg0KpVymV//x0YMiTh99OngcaNUy47bx4wblzC71evAjVrplzW2xuYNi3h99u3gdRu/Rg7Fvjxx4TfnzwBnJ2TLRZSvgHQJpW/fWK5Tt2B26eA3r2BVasSFkZFAfnypbxRx47Ali0fnqdW9j/OERoaNgROnPjwnOeIBD17Is/WrepzKX79pGx6zhH//pvyuhxOlzGbSB8+TrBrOtsw4c7iOFMFUcrSlXS3aNECU6ZMgZeXFywtNUeNjI6Ohre3N1ql9gU/FUOGDIGfnx/OnDmToe1T4uPjg+mpJVRElC3YvQvVajkiY6fLmE1E9CnOVEGUMoVI2vvmvXjxAtWqVYOpqSmGDh0KFxcXAMCdO3ewaNEiKJVKXL16NcmV6P8ydOhQ7Nq1C6dOnYJzCle5EpUoUQKjR4/GyJEj1cu8vb2xc+dOXL9+PUn55K50Ozo6Iuz582QHl/mvrqNRcfFwn3kEAHDFuxnyWLN7ebrLGrh7eVRcPCrMPQVRmMB/hifyQMXu5UCW716uVAnqLTyP4PDYZO/rVgCwt7LAmWG1Eq6GsHt58mWzyDkiKjr2w7l0ShPNgdTS8bkPj4+HdYECCAsLS/6cnoPpKmbrS3h4OKytrTN8bD8erM9/hicH68uGeAyzl+SudAsEL8ITvocUsbKAIplx6XmlO+vS1mcws+dzY5Cuv1yRIkVw9uxZDBo0CBMnTkRivq5QKODp6YlFixalK3iLCIYNG4YdO3bgxIkT/5lwA0Dt2rVx9OhRjaT78OHDqF27drLlLSwsYGFhkXRF3ryaXwJT8mmZXPGINv//L9wff0FOrmxqPv4SrM2yn9ZJW2Ut0zEfZnrKWlh8SIy0Wdbc/EMi96lc8R/u5/6vsunZb5LXyfUhodVmWTOzDwm4Nsuamqb9fzg9ZU1MtFLWFIB3mwqpj2jbpgJM8yfTNVyhSN/nMyuUNfZzhInZh3Np3rxASoH8vz73ibcMURLajtnp4ePjg+3bt+POnTvInTs36tSpgx9++EGd+BOR8eFMFUQpS/d/vpOTE/766y+EhobiwYMHEBGULVsWBQsWTPeLDxkyBOvXr8euXbuQP39+9X3Z1tbWyP3/X/Z69eoFBwcH+Pj4AABGjBiBhg0b4ueff0bLli2xceNGXL58GUuXLk336xNR9pI4oq337lvqlnMgYURb79auHNGW6BPajNnpkZEpQYmIiIxVhpubChYsiBqJg0tl0OLFiwEAjRIHnvp/K1euRJ//HwjqyZMnMDH5cFWyTp06WL9+PSZPnoxJkyahbNmy2LlzJ+foJsohOKItUfppI2anx4EDBzSer1q1CnZ2drhy5QoaNGigt3oQERFlBQbt45GW28lPfDzy7//r1KkTOiWO5ExEOQ5HtM3+Pp5P/WLAGzacGLm0TAmaIk7xmWPHYOEUnymUzeJjsABIMq5K7vexSaeHTKEsx2D5f1nkHJHi9J5A2j/3qf09cwrJYcLCwgSAhIWFZWj7yNj34jRhrzhN2CuRse+1XDvSBx7D7I/HMHvbf/O51Jx9WH0MnSbslVpzjsj+m8/Tva/MntNJ95RKpbRs2VLq1q2barmYmBgJCwtTP54+fZpwbBO+oiZ9tGihuYM8eZIvB0h8gwaaZQsXTrGsVK+uWdbJKeWyrq6aZV1dUy7r5KRZtnr1lMsWLqxZtmHDlMvmyaNZtkWLlMt++rWvY8fUy75796Fs796plw0J+VB28ODUywYEfCg7dmyqZZv0W/ThXO/tnfp+L178sN9581Ive/z4h7K//5562b17P5RduTL1sps3fyi7eXPqZVeu/FB2797Uy/7++4eyx4+nXnbevA9lL15Mvay394eyfn6plx079kPZgIDUyw4eLCIJ8brqsHWpl+3d+8N+371LvWzHjhr/wqmWTcc5Qho21CzLc4SIiMS07yhnHSvJzvIN5KxjJYlXmGiWTeM5IgyQnB6rOZoBERHpzQG/IAzyvZpkBPrgsBgM8r2KxT2q8d58I5PWKUE5xScRUdZxwC8I00p2QnDZD1e+i4a/hPfRpfC6d86ANcue0jVlmDHgFCTEY5j98RhmT0qVoN4PxxAUlnxXRwUSBsU7M8EjzV3NOQ1J1paeKUE5xecn2L2cU3ymVDYbdS+PiouH65QDyP0+Nun0kJ+UBZDw+WH38gQGPEek1ECeGJkXd6wAr/K2af7ch4eHw7pYsRwdq/lNlYiI9OJiwJsUE24goe9ZUFgMLga8Qe3ShfRXMdI6kfRPCcopPrVUllN8pr8sp/hMoKUpPpNQKBI+i6lND/lR2SwxbWcOPkcoVYLpe/yTJNxAQpxWAJh++F80reYEU8VHDeSpfe5TS/JzCJP/LkJERJR5IRGpXG3JQDnKuoYMGQJfX1+sX79ePSVocHAwolO7MkZERAaXngZySjsm3UREpBd2+dN2VS2t5SjrWrx4McLCwtCoUSMULVpU/di0aZOhq0ZERKlgA7lusHs5ERHpRU1nGxS1tkRwWEyy3dYS7+mu6ZyBaaUoS8lhw8UQERkNNpDrBq90ExGRXpiaKODd2hXAh8FYEiU+927tyvm6iYiIDCSxgTylSKwAUJQN5OnGK91ElKUtP/0Qy08HaCyTj66TNv7pBBTJhIb+9Z3Rv34pndeP0serYlEs7lEN3rtv4UX4h5F67a0t4d3aldOFERERGVBiA/kg36tQABo909hAnnFMuokoS4uIiUdweMr3DX2cuH26HWVNXhWLom6Zwqg07RAAYFXfGqhf1pYBnIiIKAtgA7n2Mekmoiwtv6UZ7K3Sf99Qfkue3rKyjxPsms42TLiJiIiyEDaQaxe/lRJRlta/fil2EyciIiLSMzaQaw8HUiMiIiIiIiLSESbdRERERESkdUrVh2G4Lga80XhOlJMw6SYiIiIiIq064BeEJr+cVD/vs/IS6v1wDAf8ggxYKyLDYNJNRERERERac8AvCIN8ryaZYSQ4LAaDfK8y8aYch0k3ERERERFphVIlmL7HH8l1JE9cNn2PP7uaU47CpJuIiIiIiLTiYsAbBIXFpLheAASFxeBiwBv9VYrIwJh0ExERERGRVoREpJxwZ6QckTFg0k1ERERERFphl99Sq+WIjIGZoStARETGbfnph1h+OkBjmXx0t1/jn05AAUWS7frXd0b/+qV0Xj8iItKems42KGptieCwmGTv61YAsLe2RE1nG31XjchgmHQTEZFORcTEIzg85W6En45u+/F2RESUvZiaKODd2hWDfK9CAWgk3onNq96tXWFqkrSxlchYMekmIiKdym9pBnur9HcjzG/JEEVElB15VSyKxT2qwXv3LY2GVXtrS3i3doVXxaIGrB2R/vEbDRER6VT/+qXYTZyIKIfxqlgUdcsURqVphwAAq/rWQP2ytrzCTTkSk24iIiIiItK6jxPsms42TLizMI6/oltMuomIiIiIiHIwjr+iW0y6iYiIiIiIcjCOv6Jb/CsRERERERHlYBx/RbdMDF0BIiIiIiIiImPFpJuIiIiIiIhIR5h0ExERkU4sWrQIJUuWhKWlJT7//HNcvHjR0FUiIiLSO4Mm3adOnULr1q1RrFgxKBQK7Ny5M9XyJ06cgEKhSPIIDg7WT4WJiIgoTTZt2oTRo0fD29sbV69eReXKleHp6YmQkBBDV42IiEivDJp0R0ZGonLlyli0aFG6trt79y6CgoLUDzs7Ox3VkIiIiDLil19+wYABA9C3b1+4urrijz/+QJ48ebBixQpDV42IiEivDDp6efPmzdG8efN0b2dnZ4cCBQpov0JERESUaXFxcbhy5QomTpyoXmZiYoImTZrg3LlzyW4TGxuL2NgP88CGh4frvJ5ERET6kC3v6a5SpQqKFi2Kpk2b4u+//zZ0dYiIiOgjr169glKpRJEiRTSWFylSJMVbwnx8fGBtba1+ODo66qOqREREOpetku6iRYvijz/+wLZt27Bt2zY4OjqiUaNGuHr1aorbxMbGIjw8XONBREREWcvEiRMRFhamfjx9+tTQVSIiItIKg3YvTy8XFxe4uLion9epUwf//vsvfv31V6xduzbZbXx8fDB9+nR9VZGIiCjHK1y4MExNTfHixQuN5S9evIC9vX2y21hYWMDCwkIf1SMiItKrbHWlOzk1a9bEgwcPUlzPlnMiIiL9Mjc3h7u7O44ePapeplKpcPToUdSuXduANSMiItK/bHWlOznXrl1D0aJFU1zPlnMiIiL9Gz16NHr37o3q1aujZs2amD9/PiIjI9G3b19DV42IiEivDJp0v3v3TuMqdUBAAK5duwYbGxuUKFECEydORGBgINasWQMAmD9/PpydnVGhQgXExMRg+fLlOHbsGA4dOmSot0BERETJ6NKlC16+fImpU6ciODgYVapUwYEDB5IMrkZERGTsDJp0X758GY0bN1Y/Hz16NACgd+/eWLVqFYKCgvDkyRP1+ri4OIwZMwaBgYHIkycP3NzccOTIEY19EBERUdYwdOhQDB061NDVICIiMiiDJt2NGjWCiKS4ftWqVRrPx48fj/Hjx+u4VkRERERElB7LTz/E8tMBGssEH77nN/7pBBRQJNmuf31n9K9fSuf1IzKkbH9PNxERERERGVZETDyCw2NSXP8iPDbF7YiMHZNuIiIiIiLKlPyWZrC3sszQdkTGjv/lRERERESUKf3rl2I3caIUZPt5uomIiIiIiIiyKl7pJqPGQT2IiIiIiMiQmHSTUeOgHkREREREZEhMusmocVAPIiIiIiIyJGYWZNQ4qAcRERERERkSB1IjIiIiIiIi0hEm3UREREREREQ6wqSbiIiIiIiISEeYdBMRERERERHpCJNuIiIiIiIiIh1h0k1ERERERESkI5wyjIiIiLK15acfYvnpAI1lAlH/3vinE1BAkWS7/vWdOa0kERHpHJNuIiIiytYiYuIRHB6T4voX4bEpbkdERKRrTLqJiIgoW8tvaQZ7K8sMbUdERKRrjDZERESUrfWvX4rdxImIKMviQGpEREREREREOsKkm4iIiIiIiEhHmHQTERERERER6Qjv6U4FpyAhIiJKn0ePHmHmzJk4duwYgoODUaxYMfTo0QPff/89zM3NDV09yqL4nYuIjBmT7lRwChIiIqL0uXPnDlQqFZYsWYIyZcrAz88PAwYMQGRkJH766SdDV4+yKH7nIiJjxqQ7FZyChIiIKH28vLzg5eWlfl6qVCncvXsXixcvZtJNKeJ3LiIyZjxTpYJTkBAREWVeWFgYbGxsDF0NysL4nYuIjBmTbiIiItKZBw8eYOHChf95lTs2NhaxsR+6EIeHh+u6akRERHrB0cuJiIjoP3333XdQKBSpPu7cuaOxTWBgILy8vNCpUycMGDAg1f37+PjA2tpa/XB0dNTl2yEiItIbhYjIfxczHuHh4bC2tkZYWBisrKwMXR0iIsoEntP15+XLl3j9+nWqZUqVKqUeofz58+do1KgRatWqhVWrVsHEJPV2/uSudDs6OvLYEhFlc4zV7F5OREREaWBrawtbW9s0lQ0MDETjxo3h7u6OlStX/mfCDQAWFhawsLDIbDWJiIiyHCbdREREpDWBgYFo1KgRnJyc8NNPP+Hly5fqdfb29gasGRERkWEw6SYiIiKtOXz4MB48eIAHDx6gePHiGuty2B1tREREAHJg0p0Y8DkqKhFR9pd4Lmcyl3X06dMHffr0yfR+GK+JiIwDY3UOTLojIiIAgKOiEhEZkYiICFhbWxu6GqRFjNdERMYlJ8fqHDd6uUqlwvPnz5E/f34oFIoM7SNxRNWnT5/m2BH4sjsew+yPxzD708YxFBFERESgWLFiaRqsi7IPxmsCeAyNAY9h9sZYrR057kq3iYlJknvMMsrKyoonj2yOxzD74zHM/jJ7DHNqq7mxY7ymj/EYZn88htkbY3Xm5MymBiIiIiIiIiI9YNJNREREREREpCNMujPAwsIC3t7esLCwMHRVKIN4DLM/HsPsj8eQdI3/Y9kfj2H2x2OYvfH4aUeOG0iNiIiIiIiISF94pZuIiIiIiIhIR5h0ExEREREREekIk24iIiIiIiIiHWHSTURERERERKQjTLqJiIj0YObMmfj6669x7NgxQ1eFiIiIUqCLeM2kW89UKpWhq0A6Fh4ejnfv3hm6GqRl169fN3QVKJt6+fIlAKBDhw4oU6YMunXrhi1btiAqKsrANaPUMF4bP8Zr48R4TRmly3jNpFsPzp07hxIlSiAuLg4mJvyTG7Njx46hRo0auHv3rqGrQlq0cuVKDBs2jMeV0iUsLAy1atXC//73P0RFRcHV1RUTJ07E2LFjMXfuXMydO9fQVaRPMF7nHIzXxonxmjJCH/GaEUUPihUrhp9//hnm5ubgtOjG6cWLFwAADw8PbNiwAe7u7up1PObZk4jgzp07AIBmzZrhr7/+gouLi8Z6otRYW1ujfPny2L9/P65du6ZePm7cOHTt2hV79uzB4sWLDVdBSoLx2vgxXhsfxmvKLH3Ea4XwP1HrVCoVRo8eDTMzM/z0009QqVQwMTHBq1evoFKpYGdnBxGBQqEwdFVJC/r374/w8HD89NNPKFGiBADg9u3bePz4MTw9PXmcs6Hw8HAMGjQIL1++xK5du5A7d24AwF9//YXnz5+jf//+/AxTqmJiYmBpaYmIiAhUrVoVX375Jb777jsUKlQIABASEoIff/wRGzduxI0bN1CwYEED1zhnYrzOWRivjQ/jNWWWvuI1r3TrgEKhgJ2dHX777TcEBgbCxMQEUVFR6NChA1q1amXo6pGWJLZX1alTB5cuXcLff/+tXjdy5EhMmTIFt27dMlT1KBOsrKzg7u6OyMhIrFu3DkBCYF+6dClWrlyJf//9FwqFgvd8UoosLS0BAK9fv0br1q1x6NAhnDlzRr3ezs4O3bp1Q5EiReDj42OoauZ4jNc5A+O18WK8pszSV7xm0q1lKpUKCoUCgwYNgpubG4YOHQoAyJ07NyZOnIgbN27gwIEDGicAngiyp/j4eABAv3794OLigvXr1+PGjRsAgHnz5uHVq1fYt28foqOjAfA4ZxexsbEAgEGDBqFUqVLYvn07Hj58CCsrK/Tt2xe5cuXCsmXLAAAmJibstkbJOnXqFIoVK4a+ffvin3/+gZ+fH9avX49nz56py5QvXx7NmjWDn5+fussr6Q/jdc7BeG2cGK9JG/QVr5l0a1FitzQAiIyMxOTJk7Fr1y6cPXsWCoUC9erVQ6dOnTBy5EgAUJdN7Mr2/v17Q1Wd0klEkCtXLgDA48eP0bNnT1y/fh3Hjh1DVFQUKleujNatW2PTpk24dOkSgA/H++7du3j8+LHB6k6ps7CwAJBwEnZ0dERQUBDWr18PAGjbti2qVauGM2fO4NSpUwCg7rK2e/du9RcAojlz5qBp06Y4fvw41q5di0WLFmHPnj04dOiQ+otf7ty5UaFCBbx69Qrh4eEGrnHOwnidczBeGy/Ga9IGvcVrIa2KjIyUr776SsqUKSM9evQQhUIh9evXV6+/fv26WFtby6+//ioiIkqlUu7cuSPFixeXnTt3GqjWlBHPnj2TBg0aSIkSJaRbt25ibm4ubm5ucvbsWRERefv2rZQtW1ZGjBghr169EhGRmzdvSrVq1WTgwIGGrHqOplKpUl1/8uRJsbGxkYYNG0r79u0lX758UqlSJblw4YKIiJw7d06aNm0qffr0Ue/rzJkzYm9vLyNGjNB19SkLef/+fbLL7969K05OTrJx40aN5V5eXlK3bl3x8/NTL4uNjRVra2v5+++/ReS//z9Jexivcw7G6+yJ8Zq0JSvEa17p1rLZs2fj2rVr2Lp1K8aNG4fJkyfj77//xtKlSwEA5cqVw8iRI+Ht7Y3w8HCYmJjAxcUFJUqUYGtqFiJp6IK0ZMkShIeH4/r165g/fz6OHz+Of//9F9u3b8fLly9hbW2NYcOGYc+ePTh58iQAoGLFiqhZsyYaNmyo67dAyZCPBlNJqbvookWL4OXlhRMnTmDbtm1Yt24dlEol/vzzTwBArVq14OHhgdu3b2PDhg0AABcXF4wdOxZdu3bV47shQzMzMwMA7Nu3DwcPHsSjR48AJIyA/erVK5ibmwNIuJIKAD4+Pjh79iz27t2rnvPTzMwMX3zxBe7fvw8AHOxHjxivjQPjtXFivCZtyhLxOl0pOomIyKFDh+T+/fvq54ktHdHR0VK6dGmZOXOmel1ERISMHDlSChcuLFFRUSIi8uTJEylbtqz0799fXS4mJkZPtaf0+OOPP2THjh3y7t07EUm40qFSqSQqKko8PDxk+PDh6uUiIrNnzxYHBwfZs2ePeh916tSRzp07y927d0Uk5dY20o/IyEgZOnSo+upVIqVSKW/evJHq1avLjBkzNNZNnjxZqlSpIjt27BARkYcPH0r79u2lTZs2EhISoqeaU1Zz5swZ+eyzz6R8+fJSvnx5qVSpkvj6+oqISPfu3aVy5coa5W/duiVWVlbi5uYm+/fvF5GE/7uvvvpK3XJO2sV4nXMwXhsfxmvSlqwQr3mlO502bNgAT09PDBo0CJs3bwaQ0NKhVCphYWEBBwcHdSsJAOTNmxe9evVCXFwcZs+eDSChVWXKlCn4888/1fMKmpmZQUQ4eEcW8tdff2HQoEFYvXo1Bg0aBCDhPi+FQoHcuXMjLi5O3foVFxcHAJg0aRLev3+PjRs34uHDhwCAyZMn4+zZszh37hxEBGZmZjzOBrR3715s27YNc+bMwYwZM/D8+XMACcc2T548CA0NVbd4KpVKAEDv3r3x5MkTbNiwAeHh4XB2dkarVq3w5s0bbNq0Sb1v4SAtRivxfyFRREQEZsyYgZYtW8Lf3x83b95Es2bN0LNnT9y/fx8DBgzAo0ePMHPmTISFhQEALl68iK5du8La2hr29vYAEv7vRo0aBUdHR72/J2PHeJ1zMF4bJ8ZryogsG68z2GCQY12+fFk8PT1l9OjR0rJlS/nuu+/U66Kjo6V3797Srl07dSupSMK9RMWKFZNcuXLJnTt3RCTh/qEGDRrIl19+qff3QGnj5+cn1tbW4ufnJy4uLjJq1Cj1fUIiIkuWLJHcuXPLy5cvRUQkPj5eoqOjpXLlymJvby8LFy5Ut6i3bNlSvLy8JD4+3iDvhT5Yt26ddOjQQa5cuSKenp7SsWNHefTokXr9uHHjxMnJSZ4+fape9vr1aylTpowULVpUZs+eLSIi7969k5YtW0qfPn3UV1Yo+0u8ErpixQp5/vx5kvWnT58WEZE9e/aIvb29evmYMWOkQIEC0rVrV3n79q0olUpZsWKFWFtbS5UqVaRRo0ZiaWkpZ86c0c8bIcbrHITx2jgxXlNqslu85pXudAoNDUXevHkxcuRITJ48GStWrMCkSZNw9+5dWFpaomXLlggODlZPUQAAL168QN26dVGmTBns3bsXAGBtbY1atWpBqVSqp6igrENE4OTkhGrVquHdu3fYvn07IiIi0KtXL9y+fRsA0Lx5c5QvXx5fffUV3rx5A1NTU/j7+8Pd3R3ly5eHra2ten+enp4ICQlhi7kByf+3ant5eeHIkSMoVKiQ+nPauXNnHDt2DAAwa9YsREdHY+rUqeqRbPfs2QN3d3f0798fzZo1A5BwVSxfvnwQEeTNm9cA74h0QaFQ4PHjx/j666+xY8cOdYu5UqlEt27dsGzZMogILC0tUbt2bSxYsADFixfH6dOnsX37dmzYsAHW1taIj49H3759sXPnTvTu3RuVK1fG3bt3UbduXfX+SLcYr3MGxmvjw3hNaZHt4rVWU/gcIDIyUqytreX48eMikjAy4qBBg6RKlSrqlra5c+dK0aJFpUGDBjJ69GhxcnKS77//XqN17c6dO2JjYyOzZs0yxNugNHj58qV4eHhojFI7ePBg+eyzz2TlypUiInLlyhUpVqyYlC1bVry8vMTc3Fzmz58vsbGx6m1u374tFStWlEmTJqlb0slwgoKCpF27drJ582b1sp49e4qzs7OsWbNGRESOHTsmNWrUkMKFC4u7u7tYWFjIrl27NPZz48YNsba2lvXr1+u1/qRbifdwDhs2TEqXLi23b98WEZGQkBApWbKkXLp0SUQSWs6LFy8utra2smTJEo1RTFevXi0LFixIdv+8eqY/jNc5B+O1cWK8ptRkt3jNpDsdEk/AXbt2lV9++UVERB4/fiyff/65KBQKadGihfrEcOrUKenfv780bNhQli1bpt5H4oF+/fq1bN26Vc/vgD6mUqlSHO4/cbmXl5eMHj1aREQuXLgg5cqVkxIlSkihQoXkxx9/FBGRp0+fyvLly2XgwIHqL3ciH/5fXr16Jffu3dPhO6H0qly5suzdu1dERH7//XextLQUd3d3yZcvn/z0008SGRkp4eHhsmnTJvnxxx/l9evX6m0Tj+v79+/ZTc3IfBxg4+PjxcrKSsaPHy8iIvv375eqVatqlG/evLk0adJEbt68qV52+vRpadKkicyePVvi4uI0yvNLvP4wXhsXxuuci/GakpMd4zWT7nRSKpXy5ZdfyrRp02To0KFiZmYmXbp0kaNHj8rEiRPF0tJSfvrpJ4mIiEh2WxHOw5oVfHwMoqOjU1y/aNEiadKkiTRt2lTMzc1l+PDhEhQUJGvXrhU7Oztp0KCBhIaGJtmWX671L7UvZYkST9KjRo2SUqVKiYuLixQvXlz9RfuXX36RSpUqibu7e5JtOYqt8fq0NfvFixciIvLbb79J7ty55Z9//pF+/frJyJEjRUTUV8ZOnToljRs3FltbW/n666+lQ4cOYmlpKWPGjOEV7SyA8do4MF4bH8ZryqjsHK/NtNNJPXtLvG/HxOTDLe7y0fyAH5czMTFBzZo1MWnSJHz++efYvXs3PD09YWJiAg8PD8THx+PevXvq+eCAhHsBTE1N1fvnPKyGk3hcE4/ByJEjERkZiR9//BEFChRQl0tcr1AocPToUfTs2RN///03qlevDgDo0aMHVCoVXr9+DWtra/X/RuJPHmP9io+PV3/mkvvsJjI1NQUAODo6Ijw8HN9++y1GjRqFQoUKAQBGjBiBatWq4dWrVxr7kv8fxZaMU+L/xYYNG/C///0PtWvXxrx58zBs2DAsXLgQ48aNw5MnT+Du7o5nz57Bzs4OAFC/fn1s27YNy5Ytw5s3b2BiYoLr16/js88+A/AhZpD2MF7nHIzXxonxmjIjO8drhUjOHjP/4z/y7du38ebNG1SqVAlWVlZJ1ifavn075s2bhz/++ANVqlTRd5VJC65evYrIyEiMGTMGs2bNgoeHh8ZJOvHkff/+fVSuXBlXrlxB+fLlIQm9Q/hFOgv49LMpIpg6dSpiY2NRrFgxdO/eXWNwnMQyCoUCW7ZsweDBgxEYGKieboRyroiICHzzzTc4cuQIJk2ahNKlS6NevXqwsbHB2bNnUa9ePTRo0ACvXr3Cu3fvUKxYMVSqVAleXl5wdXWFi4uLxv6USiUUCgXPE1rGeJ0zMV5nf4zXpC3ZOV7n2DPRx63lYWFh6NixI+rWrYs+ffrAw8MDCxcuVK//VL58+RASEoLcuXMnWZfYhsGRabOuc+fOoXr16vjmm28wduxYNGvWLEmr6Mctr1WrVlWPipncBzOHt1vp1bFjxzBx4kQAUF+pAIBTp07B0dERBw4cgJmZGf744w8MHjxYfdwSyyUe1/r168PU1BQnT54EwGOY0124cAG3b9/GwYMHMWrUKLRp0wY2NjZQqVSoU6cO2rZti6ioKOzatQv79+/H8OHDER4ejqFDh6rn902kUqk0rpRS5jFe51yM19kX4zXpQnaO1znuW8Eff/wBIOEEICJQKpWYPHkyQkJC8Pfff2PDhg3w8PDA5MmTsX//fgBJA/IXX3yBR48e4eLFi0n2n3iSSOz+QIYTHx+f7PJy5crh22+/RUBAACpWrJjqPuzt7XH58mW8fv06xRM9u6bpz759+7Bv3z71VD6JJ8oVK1agc+fOuHTpEubMmYPRo0dj586d2LZtm0a5RCEhIVAqlQgODgbAY5gTJJdYJS47ffo0lEolXF1d1etUKpX6M79o0SLcuHEDe/bsgYuLC7p27YoNGzbg0aNHaN68ucY+mWxrD+N1zsF4bXwYrymjjDZe6/qm8azk1q1bolAoxNvbW0QSBnJ48eKF2NjYqKeUEBEJDQ2V/v37S+nSpZPdT2BgoMycOVOePXumh1pTen06OMeGDRtky5Yt6hELVSqVXLp0SfLnzy/z588XkeRHKUxctmnTJomKitJxrSk1icfi7t270rZtW+nUqZN6JNKnT59KnTp15NatW/LixQvp0KGDWFlZyffffy9v375NcZ+HDx/WS93J8D7+fP/zzz9y6dIljQGZvvnmG6lXr16y/y+RkZEiIjJkyBAxNzeXhw8faqzngGm6wXidMzBeGx/Ga8oMY47XRpl0pzQiYkxMjPj4+EiePHnUB+vp06fi5uamEcRVKpWcPXtWihYtKnv27MnQa5HhHThwQEqVKiXly5eXChUqiIODg2zZskVEROLi4mTChAlSqFChNAdojnBqGJ9+xhYvXiyff/65LFq0SEQS5mPMnz+/9OzZUwoWLCht27YVf39/dXk/Pz+NE+2n++NxNR6pnY///fdfadSokTg6OkqJEiWkVatWsnv3bhEROXHihCgUCtm/f7/GNnfu3JHFixern3+6njKP8ZpEGK+NBeM1pVVOjNdG2Q9OoVAgOjoac+bMwfr169XLLSws0KlTJ5QqVQrffvutelm+fPlw/fp1vHz5Ur19wYIFkStXrlS7Hkgqoy6SYZ06dQqzZs1C37594e/vDz8/P7Rp0wajR49GQEAAcuXKhYEDByJv3rz4/vvvAXy4jygl7DaqH5cvXwaQ8PmKj49P8hnr1KkTypYti507d+L+/fuwtbVFly5d4Ovri40bN2Lnzp0oX748AODw4cNYu3YtAgMD1dt/uj8eV+OR0vn41KlTaNeuHUqXLo0zZ87g8OHDKFy4MObOnYvAwEA0bNgQ7du3x6hRo7B582a8ePECd+7cwffff48jR44gJCQEAODl5aXPt5MjMF4T43X2xXhNGZUj47Vhc37deP/+vQwePFgUCoUoFAoZPXq0nD59Wr3O19dXFAqFnD9/XkREfHx8pEqVKuq5/0QSJkwvXbq0xiTqlPWkNBfjhQsXZMWKFSKS0Eo+ZMgQsba2FjMzM/n666/V5ZYtWyYmJiZy+/ZtvdSXkpfY4rlz506xtbWVjRs3qtfduHFDfv/9d7lx44aEhYWJiMjWrVulTp06MmnSJBEROXv2rOTKlUvmzJkjfn5+EhkZKdu3b5eKFSvK4MGD1duRcdu3b5+MHj1a/fzEiRMiknD1ZMSIEerla9asEWtra7GxsZHBgweLiMi7d++ka9euYmNjI+7u7mJlZSVt27aV169f6/U95DSM1zkH47VxYLwmbciJ8dook24RkYMHD4qHh4d8/vnn0rx5cylcuLD8+eef8vLlS3n//r14enqKm5ubiCR0V+nRo4c4OTnJl19+KZMmTZLChQtLv3791PcHUNa2d+9eOXDggAQEBKiXxcbGyq1bt8Td3V0aN24sFy9elHXr1kmuXLnk7NmzIiLy6tUrqVKlivz666+GqThJeHi4nDp1SkREbt++Ld27d5fGjRtLbGyseHt7i6mpqZQrV04cHBykTZs2IpLwmR0xYoTUqVNHfSwXL14sLi4uYm9vL3Xr1pV8+fLJvHnzDPa+SLc+7maY+CVw3rx5UqVKFZkwYYI4ODhIyZIl1UE4LCxMgoKCxMPDQ8qWLSvLli2T8ePHi5OTk/p/KCIiQvz8/GTnzp1y7dq1ZF+LtI/xOmdhvM6+GK8pIxivExht0i0iMmbMGPHy8pIdO3bI4sWLxdXVVSpXriz//POPHD16VKytreXPP/8UEZGXL1/K5s2bpWfPntKkSRPx9fU1cO0pLc6cOSOfffaZlC9fXlxdXaVSpUqydu1a9fopU6ZIixYtJDAwUERENm/eLAqFQpo2baoemCE8PNwgdSeRqKgoGThwoHTv3l29bNOmTVKtWjUZNmyY9O3bV27evClv3ryRw4cPi7W1tQwfPlxERC5evCgeHh4aV0IePHggR48elfXr16sHbhHJ2idhSp+Pj2VQUJAsXrxYDh06JCIiT548EXt7e1EoFDJq1Kgk206dOlVatmwpDx48EBGRtWvXiomJibRt2zbF1+L/jn4wXhs/xuvsjfGa0ovxWpNRJ91Xr16VZs2aSZ8+fUREJDg4WJo0aSJVqlSR9u3bS9euXcXJyUljm09HtjP0SHeUsoiICPH09JSRI0eKSMKxGjdunCgUCrl3756IiLi7u0vfvn3V28ydO1cGDRokJUqU0GhlV6lUWf7DaqzevHmj8Tw4OFhGjRoltra20rp1a1GpVOqW0U2bNomFhYXcuHFDRERmz54tderUkTVr1iS775S6M1L2N2bMGLG1tZVWrVrJrFmzJDo6WrZv3y6dOnWSChUqyG+//aYuq1QqJS4uTpo2bSpdunRRL581a5bUq1dP3Nzc5OrVqxr756Bb+sV4bdwYr40D4zVlBON1AqNOukVEfv75Z6lRo4Z6tNOIiAg5evSolCxZUhwdHUWhUMisWbOSbMfgnXWkdCz27dsn9vb26udjx46VAgUKSNeuXSU4OFhERBYuXKi+T7Bdu3bi6OjI+8GyiI+P65s3b6Rr167y9OlTEUmYHqR8+fLSvHlzEUk4CSeWd3FxkXHjxomIyL1796RBgwYyaNCgJAE7u5yEKW0Sj+eTJ0/Ew8NDatasKceOHZPnz59LRESERtk+ffpIkyZN5O+//1Yvi42NlVGjRknFihVl2bJlMn36dPn8889lz549SbYnw2C8zv4Yr40T4zWlB+N18ow+6Q4MDJROnTpJy5Yt5fnz5+rlDx48kF9++UXc3Nzk8uXLBqwhpebjE/G9e/c0PmxHjhyRL7/8UubPny8ODg7qD3WixJP61KlTpV27dtK+fXt58uRJkvVkOIndge7evSvOzs7qVs3o6GiZPXu2ODg4qO8fi4+Pl9jYWGnevLn0799ffaWDgycZt5s3b2rMx7lx40apXLmyugtqcs6fPy/u7u4yduxYjfk9r1y5IgMGDJCyZctKxYoVNeZ+5ZUzw2O8zt4Yr40b4zX9F8br1Bl90i0ism7dOqldu7b88MMPhq4KpSI0NFT9+8etqleuXJEqVapI8eLFpUaNGrJ3714RSWg5d3BwEFtbW/njjz80Av7q1atlwYIF6ucfz+3JqyKG8WlL9rlz56RatWoSGBgosbGxsnr1arGyslKPYHn58mVp1qyZ1KpVS33MQkJCpEyZMrJq1aok++dxNT7x8fHi6empvpdXqVTK6NGjxdPTUx3YIyIi5Pjx43Lw4EHZt2+fOmhPnDhR6tevL1u3bhWRhMQt8Yrav//+q36Nj7tDkuExXmcPjNfGjfGa0ovx+r/liAnvOnToADc3N6xduxbXrl1Lsj4+Pl7/lSINY8eORZcuXRAZGQkAMDU1RWhoKG7fvo3FixejVatWWLp0KUqVKoWePXvi3r17aNGiBSpXroxKlSqhTp066jn//v77b6xduxYRERGIi4sDAOTOnRsAoFQqYWpqapg3mYOpVKokczIGBQUhLCwMhQoVgrm5OTw8PODh4YFx48YBANzd3dG5c2f4+fnBxcUFI0aMgJubG4oUKYImTZokeQ0eV+MTGxuL0NBQVKhQAUDCHK3Ozs64d+8eJk2ahAkTJqBBgwaYNm0aOnXqhO7du6Nz584AgOHDhyNPnjyYNWsWOnfujLJly2L//v0AgFKlSgFIOB8oFArO35yFMF5nfYzXxo3xmjKC8ToNDJ3168uhQ4fE29tbo9sDGV5ii9WjR4/k0aNHGuu6du0qCoVCWrVqpTGyZYUKFaRDhw4iktAtpVGjRlK4cGH5+uuvpWPHjmJpaSmjR49mS2oW8/z5c1m9erU8e/ZMRBIGxahZs6ZGmb/++ksKFy4sS5YsERGRgIAA6dWrlxQsWFA2bdoku3fv1nu9yTASu4+1aNFChg4dqrFu7ty50qBBA6lWrZrMmzdPdu3aJQ8ePJCTJ0+KQqGQHTt2iEjCiLk+Pj7Sp08f8fPz0/dboAxivM6aGK9zDsZrSg/G67RRiIgYOvEnUqlUMDExwdGjR1GmTBk4OTnh33//Rb169VCvXj1s2rQJJiYJHTOOHj2Kpk2bYs+ePWjZsiVev36NP//8E2/fvsXr168xZswYfPbZZxr7Jf1K7grF999/j02bNiE2NhZdunRBbGws3r17h+XLl6vLhoaGwtvbG3v37sW9e/dgZmaGLVu2ICIiAv369Ut1/2R8VCoVZs6ciTt37uD3339HoUKF1OuioqKQJ08ejfKvXr1CtWrV0Lt3b8ycOTPJ/pRKJUxMTLJ3SzmRgTFeGxfGa9IGxus0MHTWr2/Z9eZ7Y/DpfRifPo+JiRGFQiHjxo1T39M1ZcoUKVy4sDx+/Fhjm7Zt20rt2rUlJCQk2deKj4/nsTaAT++3uXPnjnqKkcTlv/32m3h4eIhCoZAGDRokmfrh3LlzUqhQIRk4cKDGdp/+TjnDH3/8IdWqVZMzZ85oLE/u871jxw6pUqVKsq3kPB9kPzxmhsN4bfwYr0nbGK9Tl+OSbjKsmJgYjRFLEyWOTPrzzz9LoUKF5OLFiyIi8u7dOylevLiMGDFCo3xAQIAoFApZunRpkn0Z64c1KwoMDFQfu48DbOKgK5UqVRInJyeZO3euPHz4UL3+7du3Ym9vL8WLF5cyZcrI119/LUePHlXvZ/Xq1XL+/HmN12IAz1kSj3dcXJwUL15cJkyYoNFtNdH9+/fln3/+keHDh0vBggVl+vTp7KpKpAWM18aF8Zp0hfE6bZh0k968f/9eBg0aJG5ubuq5N5MLuM7OztKjRw/16KirV68Wc3NzdQtr4jY7d+6UyMhI/VSektiyZYtUrVpVtm/fLiIfTronTpyQsmXLyrhx4+Thw4eydetWqV69uvTo0UPev38v8fHxEh8fL/Xq1ZP58+fLoUOHpEOHDqJQKGTAgAEaU0aQ8bl27VqayyYG44ULF0qhQoVk//79GutXr14tTZo0ETc3N6lXr55cunRJq3UlyqkYr40L4zVlBOO1djHpJr3au3evNG7cWMaNG5dkXWIL7F9//SUmJiayd+9edWCoX7++NGjQQOLi4pJsxxZV/fj07xwQECB16tSRgQMHanQbnD17trRr1079/NdffxUTExMZM2aMhIeHi0jCVCI2NjZy5coVdbkzZ85wLlYjt2LFCqlfv77cuXMn3dt+8cUX0rp1a425OqOiomT37t3y999/q5clziVLRJnDeJ19MV5TZjFeax+TbtK78ePHS506deTIkSMiotl6nhgovvjiC6lXr54EBgaKiMjx48elVKlSEhQUpP8K53AqlUrjGK1evVp9Il24cKG4u7vLihUr1Os9PT3F19dXjhw5Io6OjuLm5qaeqzXRs2fPxNHRUS5fvpzk9XLSCTgnUKlU6itlz549k4iIiCTrU5PYen7r1i3p0aOHuLu7J5m389OyRKQdjNfZC+M1ZQbjtW4x6Sa9++eff8TT01N69uypbilNPHEn/ky8B+zHH39MtrWc9OPjE+TVq1elevXqolAo1F3UYmNjpUWLFtKhQwe5deuWiIiMGjVKFAqFODg4yC+//KIeZCcqKkq2b98u/v7+cvz4cSlXrpy8fv1a/2+K9CYsLEy++uoradq0qUbX0n379qnv70zPla+goCDp1auX1KxZUxYsWKBezqtnRLrBeJ19MF5TZjBe6x7nZiC9q1KlCpo1a4Z79+5h/fr1AKCeEsDExARBQUEoWbIkRowYgefPn2tMNREfH2+QOudUCoUCMTEx6NWrFz7//HPUqlUL4eHh+PLLL6FSqWBubo5vv/0WDx8+xJ49ewAAXbp0QdGiRTFkyBCMGjUKuXPnBgCcPn0aS5cuxatXr9CgQQPcvn0bNjY2hnx7pGNWVlZwd3dHZGSk+rMeHh6OpUuXYuXKlXjw4AEUCgVUKlWa9mdvb4/Vq1dj7NixeP78Ofz9/QHAuKYUIcpCGK+zD8ZrygzGaz0wdNZPOVNgYKB06tRJWrZsKcHBwSKSMELmzz//LI6OjrJkyRID15BEEu7bGzx4sJiamsq+fftEJPnuZH369BEPDw/1KLbff/+9mJmZyahRo8TX11cGDx4s1tbW8t1332lcCeE9YcYrJiZGRBKumPTo0UOaN2+u7ma2c+dOadiwoUyYMEFd/r+mKPp0WWxsrC6qTUSfYLzOHhivKaMYr/WDV7rJIIoVK4Z27dohNDQUK1euxIkTJ1CvXj1Mnz4dEyZMwDfffKMum9ZWNdI+MzMztGzZEg0aNMCpU6cAJFzd2LNnD5o2bYqTJ08CAEaMGIHw8HBs3rwZ0dHRmDVrFqZPn47bt29j6dKluHv3Lo4ePQofHx/kypVLY/9knCwsLAAAp06dgqOjI4KCgtSt523btkW1atVw5swZ9f9VYuv37t27ERsbm2yL+sct5B//HxGR7jBeZw+M15RRjNd6Yuisn3KumJgY+fbbb8Xc3FwUCoUMGTJEYz0H6Mg6xo8fLw0aNJBVq1ZJy5YtpWDBgjJ37lx166iIyOTJk6V27dqybds2jW1fvHih/j2njVRpzP7rvqyTJ0+KjY2NNGzYUNq3by/58uWTSpUqyYULF0QkYW7Ypk2bSp8+fdT7OnPmjNjb28vIkSN1Xn8iSjvG6+yD8Zo+xXidNfBKNxmMhYUFOnfujFGjRuHRo0f4/fffAXy4D8zEhP+eWUW3bt2QK1cu9OvXD4UKFYK/vz8mTJgACwsLdevm8OHDkStXLqxZswZBQUHqbe3s7AAASqUSJiYmPK5GQETUrdiJx//TVu5FixbB09MTJ06cwLZt27Bu3ToolUr8+eefAIBatWrBw8MDt2/fVreou7i4YOzYsejSpYse3w0R/RfG6+yD8Zo+xniddfDTRAbl4eGBuXPnokSJElAqlRARdmHKgqpUqYLWrVvDzc0NzZo1g729vfqkbWJiApVKBVtbW3Tp0gVVq1ZNdsCVjwfYoexNoVAgKioKw4YNw2+//Qbgw5dulUqF0NBQPHz4EOXKlVNv06ZNG7Rv3x4XL17Ezp07ASQM4uPg4IDNmzfj5cuXKFy4MMaMGYNatWrp/T0RUeoYr7MHxmv6GON11sGkm7IElUoFU1PTnD2qYRbXqVMnlC1bFhs2bEBwcLA6eAMf7t0ZPHgwvL291fcHkfHau3cvtm3bhjlz5mDGjBl4/vw5gIRgnidPHoSGhqr/D5RKJQCgd+/eePLkCTZs2IDw8HA4OzujVatWePPmDTZt2qTet4jo/w0RUZowXmd9jNf0McbrrIFJN2UJ7MKU9SUOpvPmzRusWbMGwIfj9umXLw6mY/zi4+NRp04dHDhwAGfPnsWIESPw+PFjAAldUdu3b4/Fixfj2bNn6qsmNjY2sLGxwenTp9XdUzt37gxra2tcuXIFkZGRAHL4lCJEWRzjddbHeE0fY7zOGnjmJKI069ChA9zc3LB27Vpcu3YtxXL8Uma8Elu1vby8cOTIERQqVAjLli0DkBCQjx07BgCYNWsWoqOjMXXqVFy6dAkAsGfPHri7u6N///5o1qwZACBv3rzIly8fRAR58+Y1wDsiIjI+jNfEeJ218JNGRGlmYWGBDh06oEOHDnB2djZ0dcgAElu14+Li0LhxY1y8eBGOjo7YsmULXFxc0L9/f6xduxbm5ubYuHEj/Pz80KJFC1SvXh3ffvstvvrqK8yYMQPVq1cHANy8eRMHDhyAp6enId8WEZFRYbwmxuusRSHsjE9ERBlQpUoVzJ49Gy1btsSiRYswduxYVKhQAXfv3sW0adMwaNAgKJVK7N+/H0+ePEG/fv3Ug/aoVCqYmJggPj4esbGxbDUnIiLSEcZrw2PSTUQZkngSJuOSGBJSu09LqVTC1NQUo0ePxq5du5ArVy5ERkbC29sb/fv3x6+//oqVK1fC3Nwcly9f1tg2Pj6eIx4TEekR47VxYrzOXviXJKIMYQA3Ph8H2I/n9vxU4kArjo6OCA8Px7fffotRo0ahUKFCAIARI0agWrVqePXqlca+OMUQEZH+MV4bH8br7IdXuomIcrBPr4CICKZOnYrY2FgUK1YM3bt3h62trcY2iUF5y5YtGDx4MAIDA2Fubq7vqhMREeUYjNfZG5u+iIhymGPHjmHixIkAoDF/66lTp+Do6IgDBw7AzMwMf/zxBwYPHqwezfTTeV7r168PU1NTnDx5EgDn6yQiItImxmvjwX4DREQ5zL59+3D48GHUrVsXrVq1Urecr1ixAp07d8Yvv/wCAChZsiSGDBmC0qVLo0aNGkm6KIaEhECpVCI4OBgA5+skIiLSJsZr48Hu5UREOURi17R79+5h/PjxMDc3x8qVK5E3b148e/YMXbp0wbJly1C4cGEMHjwYhw8fxrBhwzBu3DhYW1snu88jR46gSZMmen4nRERExovx2viwezkRUQ4gIuqW788++wxeXl548uQJVq9eDSBhTtebN29i7ty5KFeuHOLj43H+/HnMmjUL1tbWuHXrFpRKpcb+AKgDeGJXNiIiIso4xmvjxKSbiMgIJU79ISKIj49P0pWsU6dOKFu2LHbu3In79+/D1tYWXbp0ga+vLzZu3IidO3eifPnyAIDDhw9j7dq1CAwMVG//6f44Oi4REVH6MV7nDPyrExEZicTW7F27dqFFixbYtGkTFAoFzMzMcPPmTSxatAg3b95EeHg4ChUqhHbt2iEyMhKrVq0CAPTr1w9mZma4cuUKbt26haioKOzYsQOjR49GREQEChQoYLg3R0REZCQYr3MeJt1EREYgIiICZ86cAQC4uLigWbNmWLJkCeLi4jBt2jRUrVoVv//+O5o3b46ePXsCAL788kvUqFEDJ06cwLlz51C7dm389ttvWL16NZo0aYJmzZqhV69e6NWrFxYtWgQrKytDvkUiIqJsj/E6Z+JAakRE2Vx0dLS6ddvX1xcAsHnzZvzwww+oW7cu3r17h9GjR8PBwQFXrlxBx44d0bt3byxYsACXLl3Cd999B2dnZyxfvhwA8O+//+Lx48d48eIF2rRpg7x58wJIOkcoERERpR3jdc7FpJuIyAiEhoaiYMGC6ucvXrzADz/8AF9fX9SqVQu7du0CkHBv1+bNm9GrVy9cunQJlSpVwpw5c7Bv3z4MHDhQ3ar+sfj4eJiZcYZJIiKizGK8zpnYBEJElI0ljlBasGBBhIaGolu3bnj27BmKFCmCFi1aoHDhwuqBWUQESqUSnTt3RsmSJbF27VoACYO0mJmZ4dy5c4iPj9fYv4gwgBMREWUS43XOxqSbiCgbMzU1hUqlgkqlwsuXL3HhwgWMHTsWAFCvXj306NEDN27cwOnTp9VdzeLi4lCqVCmEhoZCpVKhbNmyWLRoEf73v/8lCdifjnpKRERE6cd4nbMx6SYiykY+vSPo/PnzqFGjBoKDg1GyZElMmzYN+/fvx8mTJ2FpaQlPT09UqFAB48ePh1KphKmpKcLCwnD//n3Uq1dPHdgrVqwIABpzexIREVHGMF7Tx5h0ExFlEyqVKklLdlBQEMLCwlCoUCGYm5vDw8MDHh4eGDduHADA3d0dnTt3hp+fH1xcXDBixAi4ubmhSJEiaNKkSZLXMDU11ct7ISIiMlaM1/QpJt1ERNmEiYkJgoKCsGbNGgQGBgIA/P39UahQIVhYWAAAihcvjm+++QYBAQFYunQpAOCLL75A+/bt8ebNG9StWxdLly7FmTNn4ODgYLD3QkREZKwYr+lTHL2ciCiLSuxe9rHvv/8emzZtQmxsLLp06YLY2Fi8e/cOy5cvV5cNDQ2Ft7c39u7di3v37sHMzAxbtmxBREQE+vXrl+r+iYiIKH0Yr+m/MOkmIspiEk/LiV3T7t69Czs7OxQsWBAiAoVCgYULF2Lnzp04fvw46tevj/nz56Nq1arqfZw/fx6tWrVCp06dsHjxYvV2ifvngCtERESZw3hNacWkm4jIwJ4/fw47OzuYmZlpBNjz589jyJAheP/+PcLDwzFo0CB07twZzs7OAICwsDCUK1cOZmZmsLS0RMOGDfHVV1/Bw8MDIoK1a9fCxcUFn3/+ufq1GMCJiIgyhvGaMor3dBMRGdDWrVvRqlUr7NmzR2P5yZMn0atXL3zxxRfYtWsXfv75Z2zduhVTp05FfHw8lEol8uXLhzJlymDs2LH43//+h7dv36JJkyb45ptvEBsbi169emkEcIBTihAREWUE4zVlBpNuIiI9+rRzUfXq1ZE7d24cOnQIL1++VAfZv//+GxUqVMC8efPg7OyMp0+f4urVqyhSpAiio6NhamqKN2/ewN/fH/Xr10fTpk2xdetWnD59Gv/73/9gaWlpiLdHRERkFBivSZuYdBMR6YGIaEwhsmbNGhw5cgQlS5ZEt27dcOnSJezdu1dd/tSpU+jYsSOOHj2KEiVKYOXKldi9ezd++ukn5M+fHwAQFxeHvHnzanwxqFu3LszMzKBSqfT7BomIiIwA4zXpApNuIiIdS7wvy8TEBP/88w9q1KiBPn36ICIiAgDwzTffoEiRIti3bx/8/f0BAK6urujZsyd69+6NUaNG4fz582jZsiWio6OxY8cO3L59G/fv30fevHnV94x9zMSEp3ciIqL0YLwmXeFRJiLSMYVCgZiYGPU9W7Vq1UJ4eDi+/PJLqFQqmJub49tvv8XDhw/V94p16dIFRYsWxZAhQzBq1Cjkzp0bAHD69GksXboUr169QoMGDXD79m3Y2NgY8u0REREZBcZr0hUzQ1eAiMjYxcfHY8yYMVi/fj12796NFi1aqLuTJbZwt2nTBjt27MChQ4fg4eGBzz//HH379sXUqVPx8uVLuLu74+zZs1i3bh0GDRqEWrVqqbeNj4+HmRlP50RERJnBeE26wivdREQ6ZmZmhpYtW6JBgwY4deoUgITgvWfPHjRt2hQnT54EAIwYMQLh4eHYvHkzoqOjMWvWLEyfPh23b9/G0qVLcffuXRw9ehQ+Pj7IlSuXxv6JiIgocxivSVc4TzcRkZ5MmDAB58+fR79+/bBlyxacPXsWEyZMwMiRI2FhYQEAmDJlCo4ePYqxY8eiffv26m1DQkJgZ2cHAEla3YmIiEh7GK9J2/gfQESkJ926dUOuXLnQr18/FCpUCP7+/pgwYQIsLCzUgXn48OHIlSsX1qxZg6CgIPW2iQFcqVTCxMSEAZyIiEhHGK9J29jHgYhIT6pUqYLWrVvj9evXaNasGezt7aFSqdRBWaVSwdbWFl26dMHLly+THXDF1NTUADUnIiLKORivSdvYvZyISI+eP3+OkSNHIioqCsuXL9cI5IlTlRAREZFhMV6TNrG/AxGRHhUrVgzt2rXDmzdvsGbNGgAf7vX6NIAndmEjIiIi/WK8Jm1i0k1EpGcdOnSAm5sb1q5di2vXrqVYjveBERERGQ7jNWkL7+kmItIzCwsLdOjQAfb29nB2djZ0dYiIiCgZjNekLbynm4iIiIiIiEhH2BeCiMiAeB8YERFR1sd4TZnBK91EREREREREOsIr3UREREREREQ6wqSbiIiIiIiISEeYdBMRERERERHpCJNuIiIiIiIiIh1h0k1ERERERESkI0y6iYiIiIiIiHSESTeREWvUqBFGjhyp9f1OmzYNVapU0fp+iYiIchrGaiLjx6SbyED69OkDhUKBgQMHJlk3ZMgQKBQK9OnTJ037OnHiBBQKBd6+favdShIREeVgjNVEpA1MuokMyNHRERs3bkR0dLR6WUxMDNavX48SJUoYsGZEREQEMFYTUeYx6SYyoGrVqsHR0RHbt29XL9u+fTtKlCiBqlWrqpepVCr4+PjA2dkZuXPnRuXKlbF161YAwKNHj9C4cWMAQMGCBZO0uqtUKowfPx42Njawt7fHtGnTNOrw5MkTtG3bFvny5YOVlRU6d+6MFy9eaJSZO3cuihQpgvz58+Prr79GTEyMlv8SREREWRNjNRFlFpNuIgPr168fVq5cqX6+YsUK9O3bV6OMj48P1qxZgz/++AO3bt3CqFGj0KNHD5w8eRKOjo7Ytm0bAODu3bsICgrCggUL1NuuXr0aefPmxYULFzBv3jzMmDEDhw8fBpAQ5Nu2bYs3b97g5MmTOHz4MB4+fIguXbqot9+8eTOmTZuGOXPm4PLlyyhatCj+97//6fJPQkRElKUwVhNRZihERAxdCaKcqE+fPnj79i2WLVsGR0dH3L17FwBQrlw5PH36FP3790eBAgWwZMkS2NjY4MiRI6hdu7Z6+/79+yMqKgrr16/HiRMn0LhxY4SGhqJAgQLqMo0aNYJSqcTp06fVy2rWrAkPDw/MnTsXhw8fRvPmzREQEABHR0cAgL+/PypUqICLFy+iRo0aqFOnDqpWrYpFixap91GrVi3ExMTg2rVruv0jERERGRBjNRFpg5mhK0CU09na2qJly5ZYtWoVRAQtW7ZE4cKF1esfPHiAqKgoNG3aVGO7uLg4jW5tKXFzc9N4XrRoUYSEhAAAbt++DUdHR3UQBwBXV1cUKFAAt2/fRo0aNXD79u0kA8jUrl0bx48fT/d7JSIiyo4Yq4koM5h0E2UB/fr1w9ChQwFAo5UaAN69ewcA2LdvHxwcHDTWWVhY/Oe+c+XKpfFcoVBApVJlprpEREQ5DmM1EWUU7+kmygK8vLwQFxeH9+/fw9PTU2Odq6srLCws8OTJE5QpU0bjkdjqbW5uDgBQKpXpet3y5cvj6dOnePr0qXqZv78/3r59C1dXV3WZCxcuaGx3/vz5dL9HIiKi7Iyxmogyile6ibIAU1NT3L59W/37x/Lnz4+xY8di1KhRUKlUqFevHsLCwvD333/DysoKvXv3hpOTExQKBfbu3YsWLVogd+7cyJcv33++bpMmTVCpUiV0794d8+fPR3x8PAYPHoyGDRuievXqAIARI0agT58+qF69OurWrYt169bh1q1bKFWqlPb/EERERFkUYzURZRSvdBNlEVZWVrCyskp23cyZMzFlyhT4+PigfPny8PLywr59++Ds7AwAcHBwwPTp0/Hdd9+hSJEi6u5v/0WhUGDXrl0oWLAgGjRogCZNmqBUqVLYtGmTukyXLl0wZcoUjB8/Hu7u7nj8+DEGDRqU+TdMRESUzTBWE1FGcPRyIiIiIiIiIh3hlW4iIiIiIiIiHWHSTURERERERKQjTLqJiIiIiIiIdIRJNxEREREREZGOMOkmIiIiIiIi0hEm3UREREREREQ6wqSbiIiIiIiISEeYdBMRERERERHpCJNuIiIiIiIiIh1h0k1ERERERESkI0y6iYiIiIiIiHSESTcRERERERGRjvwfYJn18pBUGEcAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], "source": [ "import matplotlib.pyplot as plt\n", "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(10, 4))\n", diff --git a/doc/guide/data/panel_data.rst b/doc/guide/data/panel_data.rst index 6c69a0f4..e1125570 100644 --- a/doc/guide/data/panel_data.rst +++ b/doc/guide/data/panel_data.rst @@ -1,4 +1,4 @@ -The ``DoubleMLPanelData`` class serves as data-backend for :ref:`DiD models ` and can be initialized from a dataframe. +The ``DoubleMLPanelData`` class serves as data-backend for :ref:`DiD models `, as well as the :ref:`DoubleMLPLPR model `, and can be initialized from a dataframe. The class is a subclass of :ref:`DoubleMLData ` and inherits all methods and attributes. Furthermore, it provides additional methods and attributes to handle panel data. @@ -7,6 +7,7 @@ Key arguments * ``id_col``: column to with unique identifiers for each unit * ``t_col``: column to specify the time periods of the observation +* ``static_panel``: Indicates whether the data model corresponds to a static panel data approach (``True``, used for the ``DoubleMLPLPR`` model) or to staggered adoption panel data (``False``, for :ref:`DiD models `) which is the default option. * ``datetime_unit``: unit of the time periods (e.g. 'Y', 'M', 'D', 'h', 'm', 's') .. note:: @@ -39,3 +40,29 @@ Example usage ) print(dml_data) + + +.. tab-set:: + + .. tab-item:: Python + :sync: py + + .. ipython:: python + + import numpy as np + import doubleml as dml + from doubleml.plm.datasets import make_plpr_CP2025 + + np.random.seed(42) + df = make_plpr_CP2025(num_id=100, num_t=5, dim_x=5) + dml_data = dml.data.DoubleMLPanelData( + df, + y_col="y", + d_cols="d", + id_col="id", + t_col="time", + x_cols=["x1", "x2", "x3", "x4", "x5"], + static_panel=True + ) + + print(dml_data) diff --git a/doc/guide/models/plm/lplr.rst b/doc/guide/models/plm/lplr.rst index 76073e99..6e3a00cd 100644 --- a/doc/guide/models/plm/lplr.rst +++ b/doc/guide/models/plm/lplr.rst @@ -9,5 +9,5 @@ The high-dimensional vector :math:`X = (X_1, \ldots, X_p)` consists of confoundi :math:`\text{expit}` is the logistic link function .. math:: - \text{expit} ( X ) = \frac{1}{1 + e^{-x}} + \text{expit} ( X ) = \frac{1}{1 + e^{-x}}. diff --git a/doc/guide/models/plm/plm_models.inc b/doc/guide/models/plm/plm_models.inc index 32fceeec..853b29cf 100644 --- a/doc/guide/models/plm/plm_models.inc +++ b/doc/guide/models/plm/plm_models.inc @@ -99,6 +99,38 @@ Logistic partially linear regression model (LPLR) dml_lplr_obj.fit().summary +.. _plpr-model: + +Partially linear panel regression model (PLPR) +********************************************** + +.. include:: /guide/models/plm/plpr.rst + +``DoubleMLPLPR`` implements PLPR models. Estimation is conducted via its ``fit()`` method. + +.. tab-set:: + + .. tab-item:: Python + :sync: py + + .. ipython:: python + + import numpy as np + import doubleml as dml + from doubleml.plm.datasets import make_plpr_CP2025 + from sklearn.linear_model import LassoCV + from sklearn.base import clone + np.random.seed(3142) + learner = LassoCV() + ml_l = clone(learner) + ml_m = clone(learner) + data = make_plpr_CP2025(num_id=250, num_t=10, dim_x=30, theta=0.5, dgp_type='dgp1') + obj_dml_data = dml.data.DoubleMLPanelData(data, 'y', 'd', 'time', 'id', static_panel=True) + dml_plpr_obj = dml.DoubleMLPLPR(obj_dml_data, ml_l, ml_m) + dml_plpr_obj.fit() + print(dml_plpr_obj) + + .. _pliv-model: Partially linear IV regression model (PLIV) diff --git a/doc/guide/models/plm/plpr.rst b/doc/guide/models/plm/plpr.rst new file mode 100644 index 00000000..11cb2f38 --- /dev/null +++ b/doc/guide/models/plm/plpr.rst @@ -0,0 +1,185 @@ +Suppose a panel study observes each of :math:`N` individuals over :math:`T` time periods (or waves). +For each individual :math:`i=1,\dots,N` and each period :math:`t=1,\dots,T`, the data consists of the +triple :math:`(Y_{it}, D_{it}, X_{it})`. Let :math:`\{(Y_{it}, D_{it}, X_{it}) : t = 1, \dots , T\}_{i=1}^N` +denote :math:`N` independent and identically distributed (iid) random vectors, where each vector +corresponds to individual :math:`i` observed across all T waves. + +.. note:: + The notation and identifying assumptions are based on `Clarke and Polselli (2025) `_, with some small adjustments to better fit into the general package documentation conventions, sometimes slightly abusing notation. + See also the R package `xtdml `_ implementation and corresponding reference `Polselli (2025) `_ for further details. + +**Partially linear panel regression (PLPR)** models `(Clarke and Polselli, 2025) `_ take the form + +.. math:: + + Y_{it} = \theta_0 D_{it} + g_1(X_{it}) + \alpha_i^* + U_{it}, & &\mathbb{E}(U_{it} | D_{it}, X_{it}, \alpha_i^*) = 0, + + D_{it} = m_1(X_{it}) + \gamma_i + V_{it}, & &\mathbb{E}(V_{it} | X_{it}, \gamma_i) = 0, + +where :math:`Y_{it}` is the outcome variable and :math:`D_{it}` is the policy variable of interest. +Further note that :math:`\mathbb{E}[\alpha_i^* | D_{it}, X_{it}] \neq 0`. The high-dimensional +vector :math:`X_{it} = (X_{it,1}, \ldots, X_{it,p})` consists of other confounding covariates. +:math:`\alpha_i^*` and :math:`\gamma_i` represent unobserved individual heterogeneity terms, +correlated with the covariates. :math:`U_{it}` and :math:`V_{it}` are stochastic errors. + +Alternatively one can write the *partialling-out* PLPR model as + +.. math:: + + Y_{it} = \theta_0 V_{it} + \ell_1(X_{it}) + \alpha_i + U_{it}, + + V_{it} = D_{it} - m_1(X_{it}) - \gamma_i, + +where :math:`\alpha_i` is a fixed effect. + +To account for the presence of unobserved heterogeneity, `Clarke and Polselli (2025) `_ +use different panel data approaches, under the following assumptions. + +Define potential outcomes :math:`Y_{it}(d)` for individual :math:`i` at wave :math:`t`, where realizations are +linked to the observed outcome by the consistency assumption :math:`Y_{it}(d_{it}) = Y_{it}`. + +:math:`\xi_i` represent time-invariant heterogeneity terms influencing outcome and treatment. +Further, define :math:`L_{t-1}(W_i) = \{W_{i1}, \dots, W_{it-1}\}` as lags of a random variable :math:`W_{it}` +at wave :math:`t`. + +Assumptions `(Clarke and Polselli, 2025) `_: + +- **No feedback to predictors** + :math:`X_{it} \perp L_{t-1} (Y_i, D_i) | L_{t-1} (X_i), \xi_i` + +- **Static panel** + :math:`Y_{it}, D_{it} \perp L_{t-1} (Y_i, X_i, D_i) | X_{it}, \xi_i` + +- **Selection on observables and omitted time-invariant variables** + :math:`Y_{it} (.) \perp D_{it} | X_{it}, \xi_i` + +- **Homogeneity and linearity of the treatment effect** + :math:`\mathbb{E} [Y_{it}(d) - Y_{it}(0) | X_{it}, \xi_i] = d \theta_0` + +- **Additive Separability** + .. math:: + + &\mathbb{E} [Y_{it}(0) | X_{it}, \xi_i] = g_1(X_{it}) + \alpha^*_i, & &\alpha^*_i = \alpha^*(\xi_i), + + &\mathbb{E} [D_{it} | X_{it}, \xi_i] = m_1(X_{it}) + \gamma_i, & &\gamma_i = \gamma(\xi_i) + + +**Correlated Random Effect (CRE) Approaches** + +These approaches convert the fixed-effects model into a random-effects specification using the +Mundlak device `(Mundlak, 1978) `_. + +Given the set of assumptions, the PLPR model under the CRE approaches take the form + +.. math:: + + Y_{it} = \theta_0 D_{it} + \tilde{g}_1 (X_{it}, \bar{X}_i) + a_i + U_{it}, + + D_{it} = \tilde{m}_1(X_{it}, \bar{X}_i) + c_i + V_{it}. + +For the *partialling-out* PLPR + +.. math:: + + Y_{it} = \theta_0 V_{it} + \tilde{\ell}_1(X_{it}, \bar{X}_i) + a_i + U_{it}, + + V_{it} = D_{it} - \tilde{m}_1(X_{it}, \bar{X}_i) - c_i, + +where :math:`a_i`, :math:`c_i` are random effects and covariate unit means +:math:`\bar{X}_i = T^{-1} \sum_{t=1}^{T} X_{it}`. + +**Transformation Approaches** + +These approaches remove individual heterogeneity from the model by transforming the data. +For some random variable :math:`W_{it}`, define the First-Difference (FD) transformation +:math:`Q(W_{it}) = W_{it} - W_{it-1}` (for :math:`t=2, \dots, T`), and the Within-Group (WG) +transformation :math:`Q(W_{it}) = X_{it} - \bar{X}_{i}`, where :math:`\bar{W}_{i} = T^{-1} \sum_{t=1}^T W_{it}`. + +The PLPR model under the transformation approaches takes the form + +.. math:: + + Q(Y_{it}) = \theta_0 Q(D_{it}) + Q(g_1(X_{it})) + Q(U_{it}), + + Q(D_{it}) = Q(m_1(X_{it})) + Q(V_{it}). + +For the *partialling-out* PLPR + +.. math:: + + Q(Y_{it}) = \theta_0 Q(V_{it}) + Q(\ell_1(X_{it})) + Q(U_{it}), + + Q(V_{it}) = Q(D_{it}) - Q(m_1(X_{it})). + +These transformations remove the fixed effect terms, as :math:`Q(\alpha_i^*)=Q(\alpha_i)=Q(\gamma_i)=0`. + +**Implementation** + +``DoubleMLPLPR`` implements the estimation and requires :ref:`DoubleMLPanelData ` with parameter ``static_panel=True`` as input. +Unit identifier and time period columns are set with ``id_col`` and ``t_col`` in :ref:`DoubleMLPanelData `. + +The model described in `Clarke and Polselli (2025) `_ uses block-k-fold sample splitting, where the entire time series +of the sampled unit is allocated to one fold to allow for possible serial correlation within each unit, which is often the case for panel data. Furthermore, +cluster robust standard error are employed. ``DoubleMLPLPR`` implements both of these aspects by using ``id_col`` as the cluster variable internally, see the example notebook +`Python: Cluster Robust Double Machine Learning `_. + +The ``DoubleMLPLPR`` model inlcudes four different estimation approaches. The first two are correlated random effects (CRE) variants, the latter +two are transformation approaches. This can be selected with the ``approach`` parameter. + +``approach='cre_general'``: + +- Learn :math:`\tilde{\ell}_1 (X_{it}, \bar{X}_i)` from :math:`\{ Y_{it}, X_{it}, \bar{X}_i : t=1,\dots, T \}_{i=1}^N`, + +- First learn :math:`\tilde{m}_1(X_{it}, \bar{X}_i)` from :math:`\{ D_{it}, X_{it}, \bar{X}_i : t=1,\dots, T \}_{i=1}^N`, with predictions :math:`\hat{m}_{1,it} = \tilde{m}_1 (X_{it}, \bar{X}_i)` + + - Calculate :math:`\hat{\bar{m}}_i = T^{-1} \sum_{t=1}^T \hat{m}_{1,it}`, + + - Calculate final nuisance part as :math:`\hat{m}^*_1 (X_{it}, \bar{X}_i, \bar{D}_i) = \hat{m}_{1,it} + \bar{D}_i - \hat{\bar{m}}_i`, + + where :math:`\hat{m}^*_1 (X_{it}, \bar{X}_i, \bar{D}_i) = \mathbb{E}[D_{it} | X_{it}, \bar{X}_i] + c_i`. + +- :math:`g_1` can be learnt iteratively from :math:`\{ Y_{it}, X_{it}, \bar{X}_i : t=1,\dots, T \}_{i=1}^N` using estimates for :math:`\tilde{\ell}_1, \tilde{m}_1`. + +``approach='cre_normal'`` + +Under the assumption that the conditional distribution :math:`D_{i1}, \dots, D_{iT} | X_{i1}, \dots X_{iT}` is multivariate normal (see `Clarke and Polselli (2025) `_ for further details): + +- Learn :math:`\tilde{\ell}_1 (X_{it}, \bar{X}_i)` from :math:`\{ Y_{it}, X_{it}, \bar{X}_i : t=1,\dots, T \}_{i=1}^N`, + +- Learn :math:`m^*_{1}` from :math:`\{ D_{it}, X_{it}, \bar{X}_i, \bar{D}_i: t=1,\dots, T \}_{i=1}^N`, + +- :math:`g_1` can be learnt iteratively from :math:`\{ Y_{it}, X_{it}, \bar{X}_i : t=1,\dots, T \}_{i=1}^N` using estimates for :math:`\tilde{\ell}_1, \tilde{m}_1`. + +``approach='fd_exact'`` + +Consider First-Difference (FD) transformation :math:`Q(W_{it})= W_{it} - W_{it-1}`, under the assumptions from above, +`Clarke and Polselli (2025) `_ show that :math:`\mathbb{E}[Y_{it}-Y_{it-1} | X_{it-1},X_{it}] =\Delta \ell_1 (X_{it-1}, X_{it})` and +:math:`\mathbb{E}[D_{it}-D_{it-1} | X_{it-1},X_{it}] =\Delta m_1 (X_{it-1}, X_{it})`. Therefore, the transformed nuisance function can be learnt as + +- Learn :math:`\Delta \ell_1 (X_{it-1}, X_{it})` from :math:`\{ Y_{it}-Y_{it-1}, X_{it-1}, X_{it} : t=2, \dots , T \}_{i=1}^N`, + +- Learn :math:`\Delta m_1 (X_{it-1}, X_{it})` from :math:`\{ D_{it}-D_{it-1}, X_{it-1}, X_{it} : t=2, \dots , T \}_{i=1}^N`, + +- :math:`\Delta g_1 (X_{it-1}, X_{it})` can be learnt iteratively from :math:`\{ Y_{it}-Y_{it-1}, X_{it-1}, X_{it} : t=2, \dots , T \}_{i=1}^N` using estimates for :math:`\Delta \ell_1, \Delta m_1`. + +``approach='wg_approx'`` + +For the Within-Group (WG) transformation :math:`Q(W_{it})= W_{it} - \bar{W}_{i}`, where :math:`\bar{W}_{i} = T^{-1} \sum_{t=1}^T W_{it}`. +Approximating the model gives + +.. math:: + \begin{align*} + Q(Y_{it}) &\approx \theta_0 Q(D_{it}) + g_1 (Q(X_{it})) + Q(U_{it}), \\ + Q(D_{it}) &\approx m_1 (Q(X_{it})) + Q(V_{it}). + \end{align*} + +Similarly, + +.. math:: + Q(Y_{it}) \approx \theta_0 Q(V_{it}) + \ell_1 (Q(X_{it})) + Q(U_{it}). + +- Learn :math:`\ell_1` from transformed data :math:`\{ Q(Y_{it}), Q(X_{it}) : t=1,\dots,T \}_{i=1}^N`, + +- Learn :math:`m_1` from transformed data :math:`\{ Q(D_{it}), Q(X_{it}) : t=1,\dots,T \}_{i=1}^N`, + +- :math:`g_1` can be learnt iteratively from :math:`\{ Q(Y_{it}), Q(X_{it}) : t=1,\dots,T \}_{i=1}^N`, using estimates for :math:`\ell_1, m_1`. diff --git a/doc/guide/scores/plm/plm_scores.inc b/doc/guide/scores/plm/plm_scores.inc index 7b6ad0c8..a1122ef1 100644 --- a/doc/guide/scores/plm/plm_scores.inc +++ b/doc/guide/scores/plm/plm_scores.inc @@ -14,6 +14,13 @@ Logistic partial linear regression (LPLR) .. include:: /guide/scores/plm/lplr_score.rst +.. _plpr-score: + +Partially linear panel regression (PLPR) +======================================== + +.. include:: /guide/scores/plm/plpr_score.rst + .. _pliv-score: Partially linear IV regression model (PLIV) diff --git a/doc/guide/scores/plm/plpr_score.rst b/doc/guide/scores/plm/plpr_score.rst new file mode 100644 index 00000000..b9b0c2c5 --- /dev/null +++ b/doc/guide/scores/plm/plpr_score.rst @@ -0,0 +1,111 @@ +For the PLPR model implemented in ``DoubleMLPLPR`` one can choose between +``score='partialling out'`` and ``score='IV-type'``. + +``score='partialling out'`` implements the score function: + +For correlated random effect (cre) approaches ``approach='cre_general'`` and ``approach='cre_normal'`` + +.. math:: + + \psi(W_{it}; \theta, \eta) &:= [Y_{it} - \tilde{\ell}(X_{it},\bar{X}_i) - \theta (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i)] [D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i] + + &= - (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) \theta + (Y_{it} - \tilde{\ell}(X_{it},\bar{X}_i)) (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) + + &= \psi_a(W_{it}; \eta) \theta + \psi_b(W_{it}; \eta) + +with :math:`\eta=(\tilde{\ell},\tilde{m})`, where + +.. math:: + + \tilde{\ell}_0(X_{it},\bar{X}_i) &:= \mathbb{E}[Y_{it} \mid X_{it}, \bar{X}_i] = \theta_0\mathbb{E}[D_{it} \mid X_{it}, \bar{X}_i] + g(X_{it}, \bar{X}_i), + + \tilde{m}_0(X_{it},\bar{X}_i) + c_i &:= \mathbb{E}[D_{it} \mid X_{it}, \bar{X}_i]. + +The components of the linear score are + +.. math:: + + \psi_a(W_{it}; \eta) &= - (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i), + + \psi_b(_{it}W; \eta) &= (Y_{it} - \tilde{\ell}(X_{it},\bar{X}_i)) (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i). + + +For transformation approaches ``approach='fd_exact'`` and ``approach='wg_approx'``, where :math:`Q(W_{it})` indicates a transformated variable :math:`W_{it}` + +.. math:: + + \psi(Q(W_{it}); \theta, \eta) &:= [Q(Y_{it}) - Q(\ell(X_{it})) - \theta (Q(D_{it}) - Q(m(X_{it})))] [Q(D_{it}) - Q(m(X_{it}))] + + &= - (Q(D_{it}) - Q(m(X_{it}))) (Q(D_{it}) - Q(m(X_{it}))) \theta + (Q(Y_{it}) - Q(\ell(X_{it}))) (Q(D_{it}) - Q(m(X_{it}))) + + &= \psi_a(Q(W_{it}); \eta) \theta + \psi_b(Q(W_{it}); \eta) + +with :math:`\eta=(\ell,m)`, where + +.. math:: + + Q(\ell_0(X)) &:= \mathbb{E}[Q(Y_{it}) \mid Q(X_{it})] = \theta_0\mathbb{E}[Q(D_{it}) \mid Q(X_{it})] + Q(g(X_{it})), + + Q(m_0(X)) &:= \mathbb{E}[Q(D_{it}) \mid Q(X_{it})]. + +The components of the linear score are + +.. math:: + + \psi_a(Q(W_{it}); \eta) &= - (Q(D_{it}) - Q(m(X_{it}))) (Q(D_{it}) - Q(m(X_{it}))), + + \psi_b(Q(W_{it}); \eta) &= Q(Y_{it}) - Q(\ell(X_{it})) (Q(D_{it}) - Q(m(X_{it}))). + +``score='IV-type'`` implements the score function: + +For correlated random effect (cre) approaches ``approach='cre_general'`` and ``approach='cre_normal'`` + +.. math:: + + \psi(W_{it}; \theta, \eta) &:= [Y_{it} - D_{it} \theta - \tilde{g}(X_{it},\bar{X}_i)] [D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i] + + &= - D_{it} (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) \theta + (Y_{it} - \tilde{g}(X_{it},\bar{X}_i)) (D_{it} - \tilde{m}(X_{it},\bar{X}_i) - c_i) + + &= \psi_a(W_{it}; \eta) \theta + \psi_b(W_{it}; \eta) + +with :math:`\eta=(\tilde{g},\tilde{m})`, where + +.. math:: + + \tilde{g}_0(X_{it},\bar{X}_i) &:= \mathbb{E}[Y_{it} - D_{it} \theta_0 \mid X_{it},\bar{X}_i], + + \tilde{m}_0(X_{it},\bar{X}_i) + c_i &:= \mathbb{E}[D_{it} \mid X_{it}, \bar{X}_i]. + +The components of the linear score are + +.. math:: + + \psi_a(W_{it}; \eta) &= - D_{it} (D_{it} - \tilde{m}(X_{it},\bar{X}_i)), + + \psi_b(W_{it}; \eta) &= (Y_{it} - \tilde{g}(X_{it},\bar{X}_i)) (D_{it} - \tilde{m}(X_{it},\bar{X}_i)). + +For transformation scores ``approach='fd_exact'`` and ``approach='wg_approx'``, where :math:`Q(W_{it})` indicates a transformated variable :math:`W_{it}` + +.. math:: + + \psi(Q(W_{it}); \theta, \eta) &:= [Q(Y_{it}) - Q(D_{it}) \theta - Q(g(X_{it}))] [Q(D_{it}) - Q(m(X_{it}))] + + &= - Q(D_{it}) (Q(D_{it}) - Q(m(X_{it}))) \theta + (Q(Y_{it}) - Q(g(X_{it}))) (Q(D_{it}) - Q(m(X_{it}))) + + &= \psi_a(Q(W_{it}); \eta) \theta + \psi_b(Q(W_{it}); \eta) + +with :math:`\eta=(g,m)`, where + +.. math:: + + Q(g_0(X_{it})) &:= \mathbb{E}[Q(Y_{it}) - Q(D_{it}) \theta_0 \mid Q(X_{it})], + + Q(m_0(X_{it})) &:= \mathbb{E}[Q(D_{it}) \mid Q(X_{it})]. + +The components of the linear score are + +.. math:: + + \psi_a(Q(W_{it}); \eta) &= - Q(D_{it}) (Q(D_{it}) - Q(m(X_{it}))), + + \psi_b(Q(W_{it}); \eta) &= (Q(Y_{it}) - Q(g(X_{it}))) (Q(D_{it}) - Q(m(X_{it}))). \ No newline at end of file diff --git a/doc/index.rst b/doc/index.rst index 742a9cf7..bf226f2f 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -263,7 +263,7 @@ Acknowledgements ---------------- Funding by the Deutsche Forschungsgemeinschaft (DFG, German Research -Foundation) is acknowledged – Project Number 431701914 and Grant GRK 2805/1. +Foundation) is acknowledged – Project Number 431701914, Grant GRK 2805/1 and Project Number 530859036. References ---------- diff --git a/doc/literature/literature.rst b/doc/literature/literature.rst index 4869223e..ecde52e7 100644 --- a/doc/literature/literature.rst +++ b/doc/literature/literature.rst @@ -142,6 +142,12 @@ Double Machine Learning Literature :octicon:`link` :bdg-link-dark:`URL ` |hr| + - Paul S Clarke, Annalivia Polselli |br| + **Double machine learning for static panel models with fixed effects** |br| + *The Econometrics Journal, utaf011, 2025* |br| + :octicon:`link` :bdg-link-dark:`URL ` + |hr| + - Yusuke Narita, Shota Yasui, Kohei Yata |br| **Debiased Off-Policy Evaluation for Recommendation Systems** |br| *RecSys '21: Fifteenth ACM Conference on Recommender Systems, 372–379, 2021* |br| diff --git a/doc/release/release.rst b/doc/release/release.rst index ccfa60f4..6890e5ea 100644 --- a/doc/release/release.rst +++ b/doc/release/release.rst @@ -7,6 +7,39 @@ Release Notes .. tab-item:: Python + .. dropdown:: DoubleML 0.11.2 + :class-title: sd-bg-primary sd-font-weight-bold + :open: + + - **Release highlight:** DoubleML for static panel models with fixed effects (by `Julian Diefenbacher `_) + + - Implementation via ``DoubleMLPLPR`` class + `Py #378 `_ + - Extended User Guide and Example Gallery + `Docs #267 `_ + + - Refactoring of the ``DoubleMLFramework`` class + `Py #376 `_ + + - Added tuning to workflow + `Docs #268 `_ + `Docs #269 `_ + + - Maintenance package + `Py #380 `_ + `Py #381 `_ + `Py #382 `_ + `Py #384 `_ + + - Maintenance documentation + `Docs #263 `_ + `Docs #264 `_ + `Docs #265 `_ + `Docs #266 `_ + `Docs #270 `_ + `Docs #272 `_ + `Docs #273 `_ + .. dropdown:: DoubleML 0.11.1 :class-title: sd-bg-primary sd-font-weight-bold :open: diff --git a/doc/workflow/workflow.rst b/doc/workflow/workflow.rst index c9acec0e..aa5516f1 100644 --- a/doc/workflow/workflow.rst +++ b/doc/workflow/workflow.rst @@ -431,7 +431,7 @@ relies on conditional exogeneity, which requires to control for confounding vari implements :ref:`sensitivity` with respect to omitted confounders. Analyzing the sensitivity of the intent-to-treat effect in the 401(k) example, we find that the effect remains positive even after adjusting for -omitted confounders with a lower bound of :math:`$4,611` for the point estimate and :math:`$2,359` including statistical uncertainty. +omitted confounders with a lower bound of :math:`$4,353` for the point estimate and :math:`$2,096` including statistical uncertainty. .. tab-set::