modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Setup Environment"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark.ml.feature import VectorAssembler\n",
    "\n",
    "os.environ['PYSPARK_PYTHON'] = sys.executable\n",
    "os.environ['PYSPARK_DRIVER_PYTHON'] = sys.executable"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "def init_spark():\n",
    "    return SparkSession \\\n",
    "        .builder \\\n",
    "        .master(\"local[*]\") \\\n",
    "        .appName(\"Chess Predict\") \\\n",
    "        .config(\"spark.some.config.option\", \"some-value\") \\\n",
    "        .getOrCreate()\n",
    "\n",
    "spark = init_spark()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [],
   "source": [
    "seed = 42"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Load Dataset"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [],
   "source": [
    "df_train = spark.read.parquet('data/train.parquet')\n",
    "df_test = spark.read.parquet('data/test.parquet', header=True, inferSchema=True)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Classification Problem\n",
    " In a new match between two players, who is most likely to win?"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Feature Selection"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "data": {
      "text/plain": "('rated',\n 'turns',\n 'match_status',\n 'clock',\n 'increment',\n 'white_rating',\n 'black_rating',\n 'open_cat',\n 'opening_ply',\n 'moves_vec',\n 'winner')"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = 'rated', 'turns', 'match_status', 'clock', 'increment', 'white_rating', 'black_rating', 'open_cat', 'opening_ply', 'moves_vec'\n",
    "target = 'winner'\n",
    "inputColumns = features + (target,)\n",
    "inputColumns"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [],
   "source": [
    "training_assembler = VectorAssembler(handleInvalid='error', inputCols=features, outputCol='features')\n",
    "test_assembler = VectorAssembler(handleInvalid='error', inputCols=features, outputCol='features')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [],
   "source": [
    "clf_train_df = training_assembler.transform(df_train.select(*inputColumns)).select('features', target)\n",
    "clf_test_df = test_assembler.transform(df_test.select(*inputColumns)).select('features', target)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Model Selection"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [],
   "source": [
    "from pyspark.ml.classification import LogisticRegression, DecisionTreeClassifier\n",
    "from pyspark.ml.tuning import ParamGridBuilder, CrossValidator\n",
    "from pyspark.ml.evaluation import MulticlassClassificationEvaluator\n",
    "from pyspark.mllib.evaluation import MulticlassMetrics\n",
    "from cf_matrix import make_confusion_matrix"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [],
   "source": [
    "skipGridSearch = True"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### *Decision Tree* for Classification"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [],
   "source": [
    "dt = DecisionTreeClassifier(labelCol=target, featuresCol=\"features\", seed=seed)\n",
    "\n",
    "dtParamGrid = (ParamGridBuilder()\n",
    "               .addGrid(dt.maxDepth, list(range(2, 8)))\n",
    "               .addGrid(dt.maxBins, [4, 8, 16, 32, 64])\n",
    "               .addGrid(dt.minInstancesPerNode, list(range(1, 4)))\n",
    "               .addGrid(dt.impurity, ['gini', 'entropy'])\n",
    "               .build())\n",
    "\n",
    "dtEvaluator = MulticlassClassificationEvaluator(labelCol=target, metricName='f1')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "From [Spark Docs](https://spark.apache.org/docs/latest/mllib-decision-tree.html#tunable-parameters)\n",
    "- `maxDepth`: Maximum depth of a tree. Deeper trees are more expressive (potentially allowing higher accuracy), but they are also more costly to train and are more likely to overfit.\n",
    "- `minInstancesPerNode`: For a node to be split further, each of its children must receive at least this number of training instances. This is commonly used with RandomForest since those are often trained deeper than individual trees.\n",
    "- `maxBins`: Number of bins used when discretizing continuous features.\n",
    "- `impurity`: Measure of the homogeneity of the labels at the node. (Choice of `gini` and `entropy` for classification)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Hyperparameter Tuning (Grid Search)  20-30 minutes"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "outputs": [],
   "source": [
    "if not skipGridSearch:\n",
    "    dtCv = CrossValidator(estimator = dt,\n",
    "                          estimatorParamMaps = dtParamGrid,\n",
    "                          evaluator = dtEvaluator,\n",
    "                          numFolds = 5)\n",
    "\n",
    "    dtCvModel = dtCv.fit(clf_train_df)\n",
    "\n",
    "    # Best Parameters found\n",
    "    bestParamMap = dtCvModel.bestModel.extractParamMap()\n",
    "    print( ','.join([''.join((param.name, '=', repr(value))) for param, value in bestParamMap.items()]) )"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Train"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [],
   "source": [
    "# Use Best Parameters to initialize and train a separate classifier model\n",
    "dtBest = DecisionTreeClassifier(cacheNodeIds=False,checkpointInterval=10,featuresCol='features',impurity='gini',labelCol='winner',leafCol='',maxBins=8,maxDepth=6,maxMemoryInMB=256,minInfoGain=0.0,minInstancesPerNode=1,minWeightFractionPerNode=0.0,predictionCol='prediction',probabilityCol='probability',rawPredictionCol='rawPrediction',seed=42)\n",
    "# assert dtBest.extractParamMap() == bestParamMap, 'Check if the input parameters are the best found. '\n",
    "bestModel = dtBest.fit(clf_train_df)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Evaluation"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train f1 score 0.706133786987661\n",
      "test f1 score 0.6720110224218644\n"
     ]
    }
   ],
   "source": [
    "trainPredictionAndLabels = bestModel.transform(clf_train_df).select('prediction', target)\n",
    "testPredictionAndLabels = bestModel.transform(clf_test_df).select('prediction', target)\n",
    "print('train f1 score', dtEvaluator.evaluate(trainPredictionAndLabels))\n",
    "print('test f1 score', dtEvaluator.evaluate(testPredictionAndLabels))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\tools\\miniconda3\\envs\\bigdata\\lib\\site-packages\\pyspark\\sql\\context.py:125: FutureWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "confusionMatrix = MulticlassMetrics(testPredictionAndLabels.rdd.map(tuple)).confusionMatrix().toArray()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 2 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEGCAYAAACaSwWnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA7HElEQVR4nO3dd3xUVfr48c+TSegdBWlSNCpFinQRCx1Eg64FBSki4CKw4LKKoqyui8IPLGBBEFlBRESRL4gizQIKSy9LE6JRCB3EUAMkeX5/3JtxAkkYQiYTLs/79ZoXM+eec++5l+SZk3PPOVdUFWOMMd4QEe4KGGOMyT4W1I0xxkMsqBtjjIdYUDfGGA+xoG6MMR4SGe4KZMKG5YTYyy+/HO4qXBaGDBkS7ip4nqpKduzmAvJmx/FCIjcHdWOMyTEXMrxbJNfGdAvqxhgDFtSNMcZTvDIR04K6McbgnaBuo1+MMQZISUkJ+nU+IjJRRPaLyMaAtJEislVENojITBEpFrDtGRGJFZGfRKR1QHpdEfmfu22MBNHvY0HdGGNwWurBvoLwAdDmrLQFQA1VrQlsA54BEJFqQEegulvmHRHxuWXGAr2AaPd19j7PYUHdGGPI3qCuqouB389Km6+qSe7H/wLl3fcxwDRVPaWqcUAs0EBEygBFVHWZOgedDHQ437EtqBtjDBcW1EWkl4isCnj1usDDPQrMdd+XA3YGbIt308q5789Oz5TdKDXGGC7sRqmqjgfGZ+U4IjIESAI+Sk1K7xCZpGfKgroxxpAzo19EpCvQHmiufx4wHqgQkK08sNtNL59Oeqas+8UYY8je0S/pEZE2wNPA3ap6ImDTbKCjiOQVkco4N0RXqOoe4KiINHJHvXQBZp3vONZSN8YYsrelLiIfA7cDV4hIPPBPnNEueYEF7sjE/6rq46q6SUSmA5txumWeUNVkd1d/xRlJkx+nD34u52FB3RhjyN6grqoPpZP8fib5hwHD0klfBdS4kGNbUDfGGLwzo9SCujHGYEHdGGM8Jas3QHMbC+rGGIO11I0xxlMsqBtjjIdYUDfGGA+xoG6MMR5iQd0YYzzERr8YY4yHWEvdGGM8xIK6McZ4iFeCekiX3hWR0iLyvojMdT9XE5EeoTymMcZkRTY/ozRsQr2e+gfAPKCs+3kbMCDEx8w2zzzzDI0bN6Z9+/b+tBEjRtCmTRvuuusunnjiCY4cOZKmzO7du6lTpw7vv+8syHby5El69epFmzZtuPPOOxk1alSOnsOlJm/evNx777307t2bXr16Ua7cn0/vatiwIc8++yz58+cHICIigvbt2/PYY4/Rq1cvGjduHK5qe0br1q3ZunUr27dv5+mnnw53dXKUBfXgXKGq04EUAPehq8mZF8k97r33XiZMmJAmrUmTJsyZM4cvvviCSpUqMW7cuDTbX3nlFZo2bZom7dFHH+Xrr79m5syZrFmzhu+//z7kdb9UtWzZkp9//plx48YxYcIEDh48CEDhwoWpXLkyCQkJ/rw33HADkZGRTJgwgYkTJ1KnTh2KFi0arqpf8iIiInj77bdp27Yt1apV46GHHqJq1arhrlaOCfVDMnJKqIP6cREpiftcPRFpBCRkXiT3qF+//jlB4pZbbiEy0rkVUbt2bfbu3evftnDhQsqXL090dLQ/LX/+/DRq1AiAPHnyUK1aNfbt25cDtb/05MmTh6uvvpr169cDzi/ZqVOnACfYf/PNN+e0kqKiohARoqKiSE5O9uc3F65BgwbExsYSFxfHmTNnmDZtGjExMeGuVo7xSks91DdKn8R5VNM1IvIjcCVwf4iPmWNmzJhB27ZtAThx4gTvvfceEydOZOLEienmP3LkCN9++y1du3bNyWpeMooVK8aJEydo3749pUqVYu/evSxYsIBKlSpx9OhR9u/fnyb/1q1bue666/jb3/5GZGQkCxcuJDExMUy1v/SVK1eOnTv/fKh9fHw8DRs2DGONclZuD9bBCnVQ3wTcBlyP82Tsn/DIc1HHjh2Lz+fj7rvvBuDNN9+ka9euFCxYMN38SUlJPPnkkzzyyCNUqFAh3TyXu4iICK666irmz5/P7t27admyJU2bNqVChQpMmzbtnPxly5YlJSWFMWPGkC9fPh555BF+/fVX/vjjj5yvvAe4j1hLwyuBLhheOddQB/VlqnoTTnAHQETWADeF+LghNXPmTL777js++OAD/y/C+vXrmTdvHqNGjeLIkSNERESQN29eOnfuDMDzzz9PpUqV6NatWxhrnrsdPXqUI0eOsHu388D0rVu30rRpU4oVK0aPHs6gqSJFivDoo4/ywQcfUL16dX755RdSUlI4ceIE8fHxlClTxoJ6FsXHx6dpcJQvX97/f3E5sKCeCRG5CigH5BeROjitdIAiQIFQHDOnLF68mPfee48pU6b4R2EATJ061f/+zTffpECBAv6A/vrrr3Ps2DGGDTvnEYQmwPHjxzl69CglSpTg999/p1KlSuzduzfNte3Tpw//+c9/OHnyJAkJCVSsWJGNGzcSFRVFuXLlWLFiRRjP4NK2cuVKoqOjqVSpErt27aJjx448/PDD4a5WjsntN0CDFaqWemugG1AeeC0g/SjwbIiOme2efPJJVqxYweHDh7n11lvp168f48eP5/Tp03Tv3h2AWrVq8a9//SvDfezdu5d3332XKlWqcM899wDQuXNn7r/fM7cWstW8efOIiYnB5/Nx+PBhvvzyywzzrl69mvbt29OzZ09EhPXr13PgwIEcrK23JCcn07dvX+bNm4fP52PixIls3rw53NXKMV5pqUsoT0RE/qKqM7JY3BtXOBd7+eWXw12Fy8KQIUPCXQXPU9VzbwhcoC1btgQdc6pWrXrRxwuVUHW/dFbVKUAlEXny7O2q+lo6xYwxJmy80lIP1UiU1CEghYDC6bzSJSK9RGSViKwaP358iKpmjDHnsnHqmVDVce6/L15gufFAajTPsSt36tQpOnXqxOnTp0lOTqZ169b079+fN954g0WLFhEREUHJkiV55ZVXKF26dFBlgQzLr169mhdeeIE8efLw2muvUbFiRY4cOcLAgQOZMGFCukPLLnV33nkn1157rX88P0CHDh0oWbIk4CwPcOrUKf/yCql8Ph+PPPIIPp+PiIgItm7dypIlSwAoVaoUbdq0IU+ePCQkJDBr1ixOnz5N+fLladOmDUlJScyaNYvDhw+TN29e7rnnnnSHRnpV69atGT16ND6fjwkTJjBixIhz8owePZp27dpx4sQJunXrxtq1azMtO3z4cNq2bcu6dev88y06d+5MiRIlGDNmTM6dXAjk9mAdrFD3qV8J9AQqEfAFoqqPBlE8x66wqnLixAkKFizImTNnePjhhxkyZAjXXnsthQoVAmDy5MnExsaec1M0o7K1a9fm2LFj6Zbv27cvgwYNYteuXSxZsoTBgwczfPhwmjVrRoMGDXLqtHO0T71ChQqcPn2au+++2x/UAzVv3pxTp07xww8/nLMtKiqKM2fOEBERwSOPPMKCBQvYvXs33bp145tvvmHHjh3UrFmTYsWKsXjxYv7yl7/wzTffULRoUa655hoWLVpE8+bN2b59Ozt27MiJ000jHH3qERERbNu2jZYtWxIfH8/KlSt56KGH2LJliz9P27Zt6devH+3ataNhw4aMHj2aRo0aZVh2165dzJkzh1tvvZUpU6YwfPhwYmNjmTNnjv9LNFyyo099w4YNQcecmjVr5tqWV6gnAs0CigILgS8DXrmKiPgnDSUlJZGUlISI+AMyOAtzpdeCzqgskGH5yMhIEhMTOXnyJJGRkezYsYN9+/blaEDPaTt37sx0tmfVqlXZtGlTutvOnDkDOIHK5/P500uWLOkP0nFxcdxwww2AM4ojMjLSv3RAsWLFKFy4cFgCergEM+U/JiaGyZMnA7B8+XKKFSvGVVddlWHZlJQU8uTJAzjLX5w5c4Z//OMfjBkzJqwBPbtkZ/eLiEwUkf0isjEgrYSILBCR7e6/xQO2PSMisSLyk4i0DkivKyL/c7eNkSD+jA/15KMCqnpJLPWWnJzMvffey44dO3j44YepVasW4Iwx/7//+z8KFy7s/wUItmxG5Xv37s3QoUPJmzcvI0eOZMSIEfztb38L/UnmUhUqVOD48eMcPnw43e0iwqOPPkrx4sVZvXq1f0LMgQMHiI6OZvv27VStWpXChZ3bNUuXLqVdu3YkJSUxe/ZsmjdvftktohbMlP/08pQrVy7DsseOHWPGjBmsXbuWRYsWkZCQQP369XnppZdCf0I5IJt7LT4A3gICg8ZgYJGqDheRwe7np0WkGtARqI6zou1CEblOVZOBsUAv4L/AV0AbYG5mBw51S32OiLQL8TGyhc/nY9asWXz//fds2LCBbdu2ATBw4EC+//577rrrLqZMmXJBZTMqX7VqVaZPn86HH37Izp07KVWqFKrKgAEDGDRokH9lwstF9erVM2ylg/PL9v777/Pmm29StmxZrrzySgC+/PJL6tatS/fu3cmTJw/Jyc4CoPv372fSpEl89NFHFCtWjKNHjyIidOjQgbvvvjvDpRy8JJgp/xnlyazsyJEjqVOnDoMGDeKll15i6NCh9OjRg08++eSSH7qZnS11VV0M/H5WcgwwyX0/CegQkD5NVU+pahwQCzQQkTJAEVVdps5BJweUyVBIgrqIHBWRI0B/nMB+0k1LTc+1ihQpQsOGDf0341K1b9+e+fPnZ6lsRuVVlbFjx9KnTx/eeust+vXrx913382HH3548SdyiRARrr/++jR9vRk5deoUv/32G1WqVAHg0KFDTJs2jf/85z9s3rw53eUBmjRpwo8//sgtt9zCkiVL2LhxI/Xq1cvu08h1gpnyn1GeYMrWrl0bgG3bttGlSxcefPBBatSowbXXXhuCs8kZFxLUA0fqua9eQRyitKrucY+1ByjlppcDdgbki3fTyrnvz07PVEiCuqoWVtUiOH3qjwM3uWmp6bnK77//7n/YRWJiIkuXLqVKlSr8+uuv/jzffPONP5gEUxY4b/mZM2dy2223UbRoURITE4mIiCAiIoKTJ09m8xnmXpUrV+bQoUMcPXo03e0FChQgb968gHMvIjV/6rZUTZo0Yc2aNWnK3njjjfz8888kJiYSFRXl/4WMiooK0dnkHoFT/qOioujYsSOzZ89Ok2f27Nl06dIFcB5AkpCQwN69e4Mqm9pKj4qK8t/nSElJSfN/cqm5kKCuquNVtV7A62LGYKfXT66ZpGcq1H3q/wFuAcaISBVgLbBEVUeH+LgXZP/+/QwePJjk5GRUlTZt2nDHHXfQr18/4uLiEBHKlSvHiy86IzT37dvHc889x3vvvZdhWYBXX3013fLg3DidOXOmf5ne7t27079/f6Kionj11Vdz/iKEWExMDBUrViR//vz07duXJUuWsH79eqpVq3ZO10uhQoVo164d06dPp2DBgtx1111EREQgImzZsoXY2FjA6ba56SZnbbiffvqJDRs2+PcRGRlJzZo1+fjjjwFYsWIF9957L8nJycyaNSuHzjp8Mpry37t3bwDGjRvHV199Rbt27YiNjeXEiRP+pS/Ot1xATEwMK1euZM+ePQAsW7aMDRs2+F+XqhxY+2WfiJRR1T1u10rqWtLxQODSreWB3W56+XTSMxXSIY0AIuID6gN34LTaT6rqDUEU9cag0VzMlgnIGZd6X/OlIDuGNK5YsSLomNOgQYPzHk9EKgFzVLWG+3kkcCjgRmkJVX1KRKoDU4EGODdKFwHRqposIiuBfsBynBulb6rqV5kdN6QtdRFZhDO7dBmwBKivqvszL2WMMTkvOxu4IvIxcDtwhYjEA/8EhgPTRaQHsAP3gUGquklEpgObgSTgCXfkC8BfcUbS5McZ9ZLpyBcIfffLBqAuUAPnMXZ/iMgyVb18Oo2NMZeE7AzqqvpQBpuaZ5B/GHDO2tyqugonfgYtpEFdVQcCiEghoDtOH/tVQN5QHtcYYy6UV5YJCHX3S1+gKU5r/TdgIk43jDHG5Cr2kIzg5Md5SMZqVb305xEbYzzLWupBUNWRody/McZkFwvqxhjjIRbUjTHGQyyoG2OMh1hQN8YYD7HRL8YY4yHWUjfGGA+xoG6MMR5iQd0YYzzEgroxxniIBXVjjPEQG/1ijDEeYi11Y4zxEAvqxhjjIRbUjTHGQyyoG2OMh9iNUmOM8RBrqRtjjIdYUDfGGA+xoG6MMR5iQd0YYzzEgnqIPfXUU+Guguf169cv3FW4LAwZMiTcVTBBsNEvxhjjIV5pqUeEuwLGGJMbqGrQr/MRkYEisklENorIxyKST0RKiMgCEdnu/ls8IP8zIhIrIj+JSOuLOQ8L6sYYQ/YFdREpB/QH6qlqDcAHdAQGA4tUNRpY5H5GRKq526sDbYB3RMSX1fOwoG6MMWRvSx2nazu/iEQCBYDdQAwwyd0+Cejgvo8BpqnqKVWNA2KBBlk9DwvqxhjDhQV1EeklIqsCXr0C9rMLGAXsAPYACao6HyitqnvcPHuAUm6RcsDOgKrEu2lZYjdKjTGGCxv9oqrjgfHpbXP7ymOAysAfwKci0jmT3Ul6hwi6MmexoG6MMWTr6JcWQJyqHgAQkc+Bm4F9IlJGVfeISBlgv5s/HqgQUL48TndNllj3izHGkK196juARiJSQEQEaA5sAWYDXd08XYFZ7vvZQEcRySsilYFoYEVWz8Na6sYYQ/a11FV1uYh8BqwBkoC1OF01hYDpItIDJ/Df7+bfJCLTgc1u/idUNTmrx7egbowxZO/kI1X9J/DPs5JP4bTa08s/DBiWHce2oG6MMdgyAcYY4yleWSbAgroxxnAZBHUReZNMxkqqav+Q1MgYY8LA80EdWJVjtTDGmDDzfFBX1UmBn0WkoKoeD32VjDEm53klqJ938pGINBaRzTiD5xGRWiLyTshrZowxOSglJSXoV24WzIzSN4DWwCEAVV0P3BrCOhljTI7L5lUawyao0S+qutOZ7eqX5dlOxhiTG+X2YB2sYIL6ThG5GVARyYOz+PuW0FbLGGNy1uUU1B8HRuOs77sLmAc8EcpKGWNMTrtsgrqqHgQ65UBdjDEmbHL7DdBgBTP6pYqIfCEiB0Rkv4jMEpEqOVE5Y4zJKV65URrM6JepwHSgDFAW+BT4OJSVMsaYnHY5BXVR1Q9VNcl9TSHIRy2JSIl00ipfaCWNMSbUvBLUM1v7JTUgfysig4FpOMH8QeDLIPf/hYi0VdUj7j6r4bT6a2S9yuEzePBgTp06haqSkpLCmDFj6NSpE1deeSUA+fLlIzExkTfeeIMCBQrwyCOPUL58eVatWsWsWbPOs/fL08iRI1m+fDnFihVjwoQJABw5coR///vf7Nu3j9KlS/P8889TuHBhzpw5wxtvvMFPP/1EREQEffr0oXbt2pw4cYKBAwf693ngwAFatGhBnz59wnVal6zWrVszevRofD4fEyZMYMSIEeGuUo7J7cE6WJndKF2NE8RTB6j3DtimwEtB7P9lnMB+J3A9MJlL/KbruHHjOHHihP/zRx995H/fvn17EhMTAThz5gzz5s3jqquuonTp0jlez0tF69at6dChQ5rgMW3aNOrUqcNDDz3Exx9/zLRp0+jZsydfffUVABMmTODw4cM8++yzvP322xQoUIBx48b5y//1r3/llltuyfFzudRFRETw9ttv07JlS+Lj41m5ciWzZ89my5bLYwSzV4J6ht0vqlpZVau4/579CupGqap+CbwOzAc+ADqo6rrsqHhuVLNmTdatWwc4Qf3XX3/lzJkz4a1ULlezZk0KFy6cJm3p0qW0atUKgFatWvHjjz8C8Ntvv1GnTh0AihcvTqFChdi2bVuasvHx8fzxxx/ceOONOVB7b2nQoAGxsbHExcVx5swZpk2bRkxMTLirlWO8skxAUDNKRaQGUA3Il5qmqpMzyX/2sr1FgF+AfiJySS/b27NnT1SV5cuXs3z5cn965cqVOXbsGAcPHgxj7bzh8OHDlCxZEoCSJUvyxx9/AFClShWWLl3KHXfcwf79+9m2bRv79+/nhhtu8Jf99ttvuf322zlrBrQJQrly5di5c6f/c3x8PA0bNgxjjXKWV1rq5w3qIvJP4HacoP4V0Bb4AacrJSNnL9u7Oov1y1Xeeecdjhw5QsGCBenZsyf79+8nLi4OgNq1a/tb6SY02rZty44dO+jTpw+lSpWievXq+Hy+NHm+/fZbBg8eHKYaXtrS+yL0SqALhlfONZiW+n1ALWCtqnYXkdLAhMwKpC7bKyIFgcTUJ2OLiA/Ie3FVDp8jR44AcPz4cTZt2kSFChWIi4sjIiKCGjVqMGbMmDDX0BuKFy/OoUOHKFmyJIcOHaJYsWIA+Hy+NDc/+/fvT/ny5f2ff/75Z5KTk7nuuutyusqeEB8fT4UKFfyfy5cvz+7du8NYo5zllaAezJDGk6qaAiSJSBFgPxDs5KNFQP6Az/mBhRdWxdwhKiqKvHnz+t9HR0ezd+9eAK699loOHDhAQkJCOKvoGY0bN2b+/PkAzJ8/n5tvvhmAxMRETp48CcDq1avx+XxUrFjRX+6bb76hWbNmOV9hj1i5ciXR0dFUqlSJqKgoOnbsyOzZs8NdrRzj+SGNAVaJSDHgPZxulGPAiiD3n09Vj6V+UNVjIlLggmuZCxQuXJguXboAziiBdevW+W/SZdT1MnjwYPLly4fP56N69epMmDCB/fv352S1c71hw4axfv16EhIS6NixI127dqVjx478+9//5uuvv6ZUqVI8//zzAPzxxx8MHjyYiIgISpYseU43y/fff8/LL78cjtPwhOTkZPr27cu8efPw+XxMnDiRzZs3h7taOSa3B+tgyYWciIhUAoqo6oYg8/8I9FPVNe7nusBbqtr4fGWfeuopb1zhXKxfv37hrsJl4eqrrw53FTxPVS/6zvi//vWvoGPO0KFDc+2d+MwmH92U2bbUQH0eA4BPRSS1Y64MzuQlY4zJVbzSUs+s++XVTLYpcN7OS1VdKSI34Ew8EmCrqmY4cFtEegG9wBmfXKtWrfMdwhhjsoXng7qq3pFNx7ieP8e413HHqac7HFJVxwPjwbpfjDE5KzuDunsfcgLOkigKPAr8BHwCVAJ+BR5Q1cNu/meAHjhPleuvqvOyeuxgRr9kmTvG/U33dQfw/4C7Q3nMrLj//vsZOnQoTz75pD+tZcuWDBkyhAEDBjBgwIA0E1xSRUZG0rdvXwYMGMCTTz5Jy5Yt/dvuvPNOBg0axMCBA+nSpQv58jnztipWrMjAgQPp16+ff4JNvnz56NGjR4jPMrz279/P3//+dx599FF69OjB559/DjjDEPv168djjz3Gc889x/HjxzPcR3JyMr1792bIkCHnbJs+fTotWrTwj0DauHEjPXv2pE+fPuzatQuAY8eO8fTTT3umRXY+rVu3ZuvWrWzfvp2nn3463TyjR49m+/btrF+/3j9bN7Oyw4cPZ/369UyaNMmf1rlzZ/r3v2TnE/pl8+iX0cDXqnoDzpDwLcBgYJGqRuOMDBwM/jWxOgLVgTbAO+7w7ywJakbpRbjgMe7hsGrVKpYuXcqDD6bt7l+yZAmLFy/OsFxSUhLjx4/n9OnT/gWmfvrpJ3bs2MG2bduYO3cuKSkptG3bljvuuIO5c+dy66238uGHH1K8eHEaN27MnDlzaNGiBd9++22oTzOsfD4fjz/+ONHR0Zw4cYK//vWv1K1bl1dffZXevXtTq1Yt5s6dy/Tp0+nevXu6+5g5cyZXX311mrV3wPnCWL16NaVKlfKnffbZZ/zzn/9k3759fPHFFzz++ONMmTKFhx9++LKYbRrMOi5t27YlOjqa6OhoGjZsyNixY2nUqFGGZXft2sXNN99MrVq1mDJlCjVq1CA2NpZu3brRpk2bMJ5t9siu6f/u0O9bgW4AqnoaOC0iMTgTOQEmAd8BTwMxwDRVPQXEiUgs0ABYlpXjh7SlzsWNcc8xcXFx5wSKYJ0+fRpwgpbP5/N/i2/fvt3/Q7Jjxw7/BJqUlBQiIyOJiooiOTmZEiVKUKRIEX755ZeLP5FcrGTJkkRHRwNQoEABrr76ag4ePEh8fDw1a9YEoG7duixZsiTd8gcOHGD58uW0a9funG1jx46lV69eaYK1z+fj9OnTJCYm4vP52L17NwcPHrxs7tMEs45LTEwMkyc7PaGpK2VeddVVGZZNSUkhT548AOTPn58zZ87wj3/8gzFjxpCUlJTj55jdLqSlLiK9RGRVwKtXwK6qAAeA/4jIWhGZ4E7ELK2qe9xj7QFSWyHlgJ0B5ePdtCwJ5slHIiKdRWSo+/lqEWkQ5P7PHuO+huDHuIfdzTffzMCBA7n//vvJnz9/unlEhAEDBjB06FC2bduWZu2MVPXr12fr1q2AM0HmL3/5C02bNmXp0qW0adPGP9HmcrF3715iY2O54YYbqFSpEkuXLgVg8eLFHDhwIN0y77zzDj179jynlb106VKuuOIKrrnmmjTpDz30EK+99hqff/45HTp0YOLEiXTr1i0k55MbpbeOS7ly5YLKk1H6sWPHmDFjBmvXriUuLo6EhATq16/vmQlKFxLUVXW8qtYLeI0P2FUkcBMwVlXrAMdxu1oykN6fjlnuIwym++UdIAVntMu/gKPADKD++Qqqauqc7ndF5GsuYIx7uC1btoyFC53Jr61ataJ9+/Z8+umn5+RTVd544w3y5ctH165dKV26NPv27fNvb9asGSkpKaxduxaAPXv28PbbbwPOImCpSw906tSJ5ORk5syZw7Fjx845jlecPHmSF198kT59+lCwYEEGDRrE22+/zZQpU2jcuDGRkef+SP73v/+lWLFiXHfddWkmeSUmJjJ16lSGDx9+Tplrr72Wt956C4ANGzb471+89NJLREZG8vjjj1O8ePHQnGQuEMw6LhnlyazsyJEjGTlyJADvvfceQ4cOpUePHrRq1YoNGzYwbNiw7Kh+WGTjvZZ4IF5VU1f8+wwnqO8TkTKqukdEyuD0XKTmrxBQvjyQ5fUZgul+aaiqTwCJAO7d2jyZFRCRm85+ASWAyMzGv+cmx44d838rr1ixIs2aGOlJTEzk559/5vrrr/en1a1bl6pVq/Lxx+k//a958+YsWrSIli1bMn/+fNasWUOTJk2y9Txyk6SkJF544QWaN29O06ZNAWdizogRIxg7dizNmjWjbNmy55TbuHEjy5Yto1OnTgwbNox169bxyiuvsHv3bvbu3Uvv3r3p1KkTBw4c4PHHH+f333/3l1VVPvroIzp37szkyZPp2rUrzZs3Z+bMmTl23uEQzDouGeUJpmzt2rUB2LZtG126dOHBBx+kRo0aXHvttSE4m5yRXTdKVXUvsFNEUoNBc2AzMBvo6qZ1BVKfnDMb6Cgied0nw0VzET0awbTUz7h3YhVARK7EablnJnCMe+AVEIIc4x5uhQsX5ujRowDUqFHDv85LoIIFC5KcnExiYiKRkZFER0fz3XffAXDddddx++238+6776a7pnrdunXZsmULJ0+eJCoqyv/Dktpn6TWqyqhRo6hYsSL33XefP/3w4cMUL16clJQUpkyZQvv27c8p+9hjj/HYY48BsG7dOj799FOeeeYZwLkhmqpTp0688847FC1a1J82f/58GjZsSOHChTl16hQRERFERET4H2biVYHruOzatYuOHTvy8MMPp8kze/Zs+vbty7Rp02jYsCEJCQns3buXAwcOnLfsSy+9RK9evYiKivKvlJmSkkKBApfkKiBAto9T7wd8JCJ5cJYd747TiJ4uIj2AHcD97nE3ich0nMCfBDyRughiVgQT1McAM4FSIjIMZ0TLc5kVSB3jLiL5gT7ALTjBfAkwNquVDZWHH36YKlWqULBgQZ599lkWLFhAlSpV/K3Gw4cPM2PGDACKFCnCfffdx8SJEylcuDAPPvggERERiAgbNmzwjy7o0KEDkZGR9OzZE3BulqYO44uKiqJevXq89957gNOX/Mgjj5CcnMzUqVNz+vRzxMaNG1m4cCGVK1emd2/nIVqPPvoou3bt8j/q75ZbbvGPojh48CCvvfbaRa3lkpiYyPz58/1PVbrvvvt44YUXiIqKSndYpJdktI5L6rUfN24cX331Fe3atSM2NpYTJ074Rx2dbw2YmJgYVq5cyZ49ewCnq3LDhg3+16UqOx9+4T4MqF46m5pnkH8YkC19V0Gt/eLOCm2O09JepKpBPd/K/fY5AqQ+8+0hoJiqPnC+sjb5KPRs7ZecYWu/hF52rP0yaNCgoGPOqFGjcu242GAeknE1cAL4IjBNVXcEsf/rVTVwDNm3IrL+wqtpjDGh5ZVJacF0v3zJnw+gzgdUxpnuWj2IsmtFpJGq/hdARBoCP2axrsYYEzKXTVBX1TRP8HVHr/TOrIyI/A/niyAK6CIiO9zPFXFuBhhjTK5y2QT1s6nqGhE53xj1c4cwGGNMLnbZBHUReTLgYwTOTKn0p/25VPW3i6yXMcbkqOwc/RJOwbTUCwe8T8LpY58RmuoYY0x4XBYtdXfSUSFV/UcO1ccYY8LC80FdRCJVNelSmdZvjDEXw/NBHWftgZuAdSIyG/gUZ7UxAFT18xDXzRhjcszlENRTlQAO4azXkjpeXQEL6sYYz7gcbpSWcke+bOTPYJ7KG19pxhjjuhxa6j6gENm8gLsxxuRGl0NQ36Oq/8qxmhhjTBhdDkE9165CZowx2e1yCOrprvtrjDFe5Pmgrqq/Z7TNGGO85nIY/WKMMZcNz7fUjTHmcmJB3RhjPMSCujHGeIgFdWOM8RAL6sYY4yE2+sUYYzzEWurGGOMhFtSNMcZDvBLUJbeeiIjkzooZc4EqV64c7ip43i+//HLRa1U98MADQcec6dOnn/d47uNAVwG7VLW9iJQAPgEqAb8CD6jqYTfvM0APIBnor6rzLvgEXBFZLWiMMV6SkpIS9CtIfwO2BHweDCxS1WhgkfsZEakGdASqA22Ad9wvhCyxoG6MMTjdL8G+zkdEygN3AhMCkmOASe77SUCHgPRpqnpKVeOAWKBBVs/DgroxxnBhQV1EeonIqoBXr7N29wbwFBDYrC+tqnvcY+0BSrnp5YCdAfni3bQssRulxhjDhd0oVdXxwPj0tolIe2C/qq4WkduD2F22Pl3OgroxxpCto1+aAHeLSDsgH1BERKYA+0SkjKruEZEywH43fzxQIaB8eWB3Vg9u3S/GGEP29amr6jOqWl5VK+HcAP1GVTsDs4GubrauwCz3/Wygo4jkFZHKQDSwIqvnYS11Y4whR5YJGA5MF5EewA7gfgBV3SQi04HNQBLwhKomZ/UgFtSNMYbQTD5S1e+A79z3h8jgMaGqOgwYlh3HtKBujDF4Z0apBXVjjMGCujHGeIoFdWOM8RBbT90YYzzEWurGGOMhFtSNMcZDLKgbY4yHWFA3xhgPsaBujDEeYqNfjDHGQ6ylbowxHmJB3RhjPMSCujHGeIgFdWOM8RAL6sYY4yE2+sUYYzzEWurGGOMhXgnqIX3wtIg0E5ECoTxGuJUvX55vvvmGzZs3s3HjRvr37x/uKnlS3rx5Wb58OevWrWPjxo288MIL4a7SJWvEiBGsWLGCuXPnpknv0qULCxcu5Ouvv+bpp58GoGbNmsyZM4c5c+bw5Zdf0qpVq3BUOUdk14Onwy3ULfVuwLsicghY4r5+UNXDIT5ujklKSuLvf/87a9eupVChQqxevZoFCxawZcuWcFfNU06dOkWzZs04fvw4kZGR/PDDD8ydO5fly5eHu2qXnM8++4zJkyczatQof1qjRo1o2bIl7dq14/Tp05QsWRKAbdu2ERMTQ3JyMldeeSVffvklixYtIjk5y89FzrVye7AOVkiDuqp2ARCRssB9wNtA2VAfNyft3buXvXv3AnDs2DG2bNlCuXLlLKiHwPHjxwGIiooiKirKM7+EOW3lypWUK1cuTVqnTp149913OX36NACHDh0CIDEx0Z8nb968OVfJMPDKjdJQd790FpFxwGdAC+AtoGkojxlOFStWpE6dOtZ6DJGIiAjWrl3L/v37WbBgAStWrAh3lTyjcuXK1K9fn88//5yPP/6YmjVr+rfVqlWLr7/+mrlz5/Lcc895spUO1v0SrDeAn4F3gW9V9dcQHy9sChYsyIwZMxgwYABHjx4Nd3U8KSUlhTp16lC0aFFmzpxJ9erV2bRpU7ir5Qk+n48iRYpw7733UrNmTd58801uu+02ANavX0+bNm245pprGDVqFN99952/Re8luT1YByukLXVVvQJ4FMgHDBORFSLyYSiPGQ6RkZHMmDGDjz76iJkzZ4a7Op6XkJDAd999R5s2bcJdFc/Yu3cv8+bNA2DDhg2kpKRQokSJNHl+/vlnTpw4wfXXXx+OKoacV1rqoe5+KQJcDVQEKgFFAW90XAV4//332bJlC6+//nq4q+JZV1xxBUWLFgUgX758tGjRgq1bt4a5Vt6xYMECGjduDDhdMVFRUfz++++UL18en88HQNmyZalSpQrx8fHhrGrIeCWoh7r75YeA11uq6rmfhiZNmtClSxc2bNjA2rVrAXj22WfPGS5mLk6ZMmWYNGkSPp+PiIgIpk+fzpdffhnual2SRo8eTcOGDSlevDg//vgjo0eP5tNPP2XEiBHMnTuXM2fO8I9//AOAevXq8fjjj5OUlERKSgpDhw7l8GHPDF5LI7cH62BJbj0REcmdFTPmAlWuXDncVfC8X375RS52HzVq1Ag65mzcuDHD44lIBWAycBVOz8R4VR0tIiWAT3B6LX4FHkgd3i0izwA9gGSgv6rOy+JphLalLiJXAk8B1XH61QFQ1WahPK4xxlyobGzgJgF/V9U1IlIYWC0iC3Dm7SxS1eEiMhgYDDwtItWAjjhxsiywUESuU9UsDTMKaZ868BGwFagMvIjz7bQyo8wi0ktEVonIqhDXyxhj0siuPnVV3aOqa9z3R4EtQDkgBpjkZpsEdHDfxwDTVPWUqsYBsUCDrJ5HqIN6SVV9Hzijqt+r6qNAo4wyq+p4Va2nqvVCXK9MtW7dmq1bt7J9+3b/dOmzjR49mu3bt7N+/Xrq1KnjT4+Li/P3r69c+ef31/Dhw1m/fj2TJk3yp3Xu3PmyXVbgYq7xgAED2LhxI//73/+YOnWqf1KMXeNzRURE8MUXXzBhwoRztrVo0YKvvvqKOXPmMGvWLOrVc37tKleu7F8aYM6cOaxfv57u3bsD8PTTT/PVV1+lmY3aoUMHunXrliPnE0oXEtQDG6Duq1d6+xSRSkAdYDlQWlX3uMfaA5Rys5UDdgYUi3fTsiTUQf2M++8eEblTROoA5UN8zIsSERHB22+/Tdu2balWrRoPPfQQVatWTZOnbdu2REdHEx0dTa9evRg7dmya7XfccQd16tShfv36ABQpUoSbb76ZWrVq4fP5qFGjBvny5aNbt2688847OXZuucXFXOOyZcvSv39/6tWrx4033ojP56Njx452jTPQvXt3fv7553S3LV26lHbt2tG+fXuefvppXnnlFcBpmLRv35727dtz9913k5iYyLx58yhcuDA33XQT7dq1w+fzcf3115M3b17uu+8+pkyZkpOnFRIXEtQDG6Dua/zZ+xORQsAMYICqHsnk0On1z2e5LyjUQf3fIlIU+DswCJgADAzxMS9KgwYNiI2NJS4ujjNnzjBt2jRiYmLS5ImJiWHy5MkALF++nGLFinHVVVdluM+UlBTy5MkDQP78+f2jC8aMGUNSUlLoTiaXuthrHBkZSf78+fH5fBQoUIDdu3fbNU7HVVddxR133MEnn3yS7vYTJ0743+fPnz/dboWbb76Z3377zX+No6KiAGdY6ZkzZ+jVqxcffPCBJ65xdg5pFJEonID+kap+7ibvE5Ey7vYywH43PR6oEFC8PLA7q+cRsqAuIj4gWlUTVHWjqt6hqnVVdXaojpkdypUrx86df/4lFB8ff846GZnlUVXmz5/PqlWr6NmzJ+CsCTNjxgzWrl1LXFwcCQkJ1K9fn9mzc/WlCJmLuca7d+9m1KhR7Nixgz179pCQkMCCBQvsGqfj+eefZ/jw4ZmuadKqVSsWLFjA+++/n2432F133cUXX3wBOGvvfP3118yZM4edO3dy9OhRatasycKFC0N2DjkpJSUl6FdmRESA94EtqvpawKbZQFf3fVdgVkB6RxHJKyKVgWggy2tghCyou3du7w7V/kPF+f9I6+xv5szyNGnShLp169K2bVueeOIJmjZ1lroZOXIkderUYdCgQbz00ksMHTqUHj168MknnzBkyJAQnEnudTHXuFixYsTExFC5cmXKli1LwYIF6dSpE2DXOFCzZs04dOgQGzduzDTf/PnzadmyJb179+bJJ59Msy0qKormzZunmXMxfvx42rdvz8svv8yTTz7J66+/zgMPPMCbb77JE088EZJzySnZ2FJvAjwCNBORde6rHTAcaCki24GW7mdUdRMwHdgMfA08kdWRLxD67pelIvKWiDQVkZtSXyE+5kWJj4+nQoU//xIqX748u3fvDjrPnj17ADhw4AAzZ86kQYO0N7Fr164NOEuadunShQcffJAaNWpw7bXXhuJ0cqWLucYtWrQgLi6OgwcPkpSUxOeff87NN9+cpqxdY6hbty7Nmzdn8eLFjBkzhsaNG/Paa69lmH/lypVcffXVFC9e3J922223sWnTJg4ePHhO/mrVqgFO//u9995Lv379uO6666hUqVK2n0tOycbRLz+oqqhqTVWt7b6+UtVDqtpcVaPdf38PKDNMVa9R1etV9aJmLoY6qN+MM/byRWAU8Kr7b661cuVKoqOjqVSpElFRUXTs2PGcP+Fnz55Nly5dAGjYsCEJCQns3buXAgUKUKhQIQAKFChAq1atzmkppbYgo6Ki/NOvU1JSKFDA088SSeNirvGOHTto1KgR+fPnB6B58+bnLHNs19j5q6VJkybceuut9O/fn2XLlp3TEq9YsaL/ffXq1YmKikozWzSw6+Vsqa30yMhIIiKcMKKq5MuXL938l4Ls7FMPp5BMPhKR1J+eOTh3cQP/ls7VVyQ5OZm+ffsyb948fD4fEydOZPPmzfTu3RuAcePG8dVXX9GuXTtiY2M5ceKEf7hX6dKl/Qt6RUZGMnXqVP8iSeDc/Fu5cqW/Nb9s2TI2bNjgf10uLuYar1ixgs8++4w1a9aQlJTE2rVrGT/+z4EHdo0z9/DDDwMwdepU2rRpwz333ENSUhKJiYlphn7my5ePW265heeee+6cfbRs2ZINGzawf79zn2/t2rXMnTuXrVu3XtLr8eT2YB2skCwTICL/dN9eD9THuSEgwF3AYlV9LIh9eOMKm8ueLRMQetmxTEDFihWDjjm//fbbRR8vVELSUlfVFwFEZD5wkzurChF5Afg0FMc0xpiL4ZWWeqhXabwaCFxN/zTOYjbGGJOrWFAPzofAChGZidOXfg9/rn1gjDG5hgX1IKjqMBGZy5/PJe2uqmtDeUxjjMkKC+pBclcrWxPq4xhjzMWwoG6MMR5yvun/lwoL6sYYg7XUjTHGUyyoG2OMh1hQN8YYD7GgbowxHmJB3RhjPMRGvxhjjIdYS90YYzzEgroxxniIBXVjjPEQC+rGGOMhdqPUGGM8xFrqxhjjIRbUjTHGQyyoG2OMh1hQN8YYD7GgbowxHuKV0S8R4a6AMcbkBqoa9Ot8RKSNiPwkIrEiMjgHqu9nLXVjjCH7ul9ExAe8DbQE4oGVIjJbVTdnywHOw1rqxhhDtrbUGwCxqvqLqp4GpgExIT8BV65tqauqhLsOF0pEeqnq+HDXw8vsGofe5XqNLyTmiEgvoFdA0viAa1YO2BmwLR5oePE1DI611LNXr/NnMRfJrnHo2TU+D1Udr6r1Al6BX4LpfTnk2NAaC+rGGJO94oEKAZ/LA7tz6uAW1I0xJnutBKJFpLKI5AE6ArNz6uC5tk/9EnXZ9UOGgV3j0LNrfBFUNUlE+gLzAB8wUVU35dTxxSuzqIwxxlj3izHGeIoFdWOM8RAL6lkkIscySH9cRLq477uJSNmcrdmlRUQqicjGdNK/E5F6WdhfNxF5K3tq510i8oKIDAp3PUz2sxul2UxV3w342A3YSA4OZzImq0QkUlWTwl0Pc3GspZ4BEXlKRPq7718XkW/c981FZIr7fpiIrBeR/4pIaTftBREZJCL3AfWAj0RknYjkF5G6IvK9iKwWkXkiUiZc55fLRIrIJBHZICKfiUiBwI0iMlZEVonIJhF5MSC9vogsdf8PVohI4bPK3Skiy0Tkipw6kdxMRIa4i0wtBK53074TkZdF5HvgbyJyl4gsF5G1IrIw4Of6fyJSTByHAv4a/VBEWoTvrMzZLKhnbDHQ1H1fDygkIlHALcASoCDwX1Wt5ebtGVhYVT8DVgGdVLU2kAS8CdynqnWBicCwHDiPS8H1ONOsawJHgD5nbR+iqvWAmsBtIlLTHf/7CfA39/+gBXAytYCI3AMMBtqp6sGcOIncTETq4oyXrgPcC9QP2FxMVW9T1VeBH4BGqloHZ82Sp9w8PwJNgOrAL/z5u9EI+G/oz8AEy7pfMrYaqOu2/k4Ba3CCe1OgP3AamBOQt+V59nc9UANYICLgjF/dk/3VviTtVNUf3fdTcK5voAfctTYigTJANZxp13tUdSWAqh4BcK/tHTj/V61S0w1NgZmqegJARAInw3wS8L488In7V2QeIM5NXwLcCvwGjAV6iUg54HdVTff+kgkPa6lnQFXPAL8C3YGlOD/UdwDXAFuAM/rnIP9kzv8FKcAmVa3tvm5U1VYhqfyl5+zJEv7PIlIZGAQ0d1vyXwL5cK5nRpMsfgEKA9dlf1UvaRldr+MB798E3lLVG4HeONca/vzLtSnwHXAAuA/n98LkIhbUM7cYJ6AsxvnhfRxYFxDMz+coTnAB+Am4UkQaA4hIlIhUz+b6XqquTr0uwEM4XQCpiuAEnQS3f7etm74VKCsi9QFEpLCIpH6x/obTxTDZrrHfYuAe995OYeCuDPIVBXa577umJqrqTuAKIFpVf8H5PxqEBfVcx4J65pbg/Lm/TFX3AYlc2A/xB8C7IrIOp7vlPmCEiKwH1gE3Z2dlL2FbgK4isgEogfPnPQCquh5YC2zCuQ/xo5t+GngQeNO9ngv4s1WJqv4EdAI+FZFrcug8ci1VXYPTzbIOmEHGP8cv4FyzJcDZ9yKWA9vc90twlpj9AZOr2DIBxhjjIdZSN8YYD7GgbowxHmJB3RhjPMSCujHGeIgFdWOM8RAL6iZTIpLsrl2zUUQ+PXtdlgvc1wfumjiIyAQRqZZJ3ttF5IKHfIrIr+mt9ZJR+ll5LmhmpK10aHIjC+rmfE66M2Br4CyN8HjgRhHxZWWnqvqYqm7OJMvt2Dh+Yy6YBXVzIZYA17qt6G9FZCrwPxHxichIEVnprrTYG8Bd0e8tEdksIl8CpVJ3JAHrpYtIGxFZ4662uEhEKuF8eQx0/0poKiJXisgM9xgrRaSJW7akiMx3VxUch7N8QKZE5P/EWSlzk7umTOC2V926LBKRK920a0Tka7fMEhG5IVuupjEhYAt6maC4U/DbAl+7SQ2AGqoa5wbGBFWtLyJ5gR9FZD7OioDXAzcCpYHNOLNCA/d7JfAecKu7rxKq+ruIvAscU9VRbr6pwOuq+oOIXI3zUN+qwD+BH1T1XyJyJ5AmSGfgUfcY+YGVIjJDVQ/hrLy5RlX/LiJD3X33xXkQ8+Oqul1EGgLvAM2ycBmNCTkL6uZ88rvLHIDTUn8fp1tkhaqmruDXCqiZ2l+Os35INM6qfh+rajKwW9w16c/SCFicui9V/T2DerQAqrmrMAIUcdcwuRVnnRdU9UsRORzEOfV3l+YFqODW9RCQwp8rFk4BPheRQu75fhpw7LxBHMOYsLCgbs7npLsevJ8b3AJX9hOgn6rOOytfOzJeGTCwbDBrVUQAjVX1ZGCiW5eg17oQkdtxviAaq+oJEfmOgDVjzqLucf84+xoYk1tZn7rJDvOAv7oPEUFErhORgjgrA3Z0+9zL4CxdfLZlOA++qOyWLeGmB65wCTAfpysEN19t9+1inIW7EJG2QPHz1LUocNgN6Dfg/KWQKgJn0TWAh3G6dY4AcSJyv3sMEZFa5zmGMWFjQd1khwk4/eVrxHmI9DicvwJnAtuB/+GsvPj92QVV9QBOP/jn7mqLqd0fX+AsFbtORFIfTFLPvRG7mT9H4bwI3Coia3C6gXacp65f4zw+bwPwEmmf2nMcqC4iq3H6zP/lpncCerj12wTEBHFNjAkLW6XRGGM8xFrqxhjjIRbUjTHGQyyoG2OMh1hQN8YYD7GgbowxHmJB3RhjPMSCujHGeMj/B/crqghcxGTfAAAAAElFTkSuQmCC\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "categories = ['white', 'black', 'draw']\n",
    "make_confusion_matrix(confusionMatrix, categories=categories, sum_stats=False, cmap='gray')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Because the classes are imbalanced and that we don't favor one class over the other, we adopted the F1 score as the main metric to evaluate our model. Given that there are 3 output classes, this result is acceptable for a simple decision tree classifier. For better performance, we could use Ensembling methods such as Random Forest."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### *Logistic Regression*"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "outputs": [],
   "source": [
    "lr = LogisticRegression(labelCol=target, featuresCol=\"features\")\n",
    "\n",
    "lrParamGrid = (ParamGridBuilder()\n",
    "               .addGrid(lr.regParam, [0.01, 0.1, 0.2, 0.8, 2.0])\n",
    "               .addGrid(lr.elasticNetParam, [0.0, 0.1, 0.2, 0.3, 0.4])\n",
    "               .addGrid(lr.fitIntercept, [True, False])\n",
    "               .addGrid(lr.maxIter, [10, 30, 50, 70, 100])\n",
    "               .build())\n",
    "\n",
    "lrEvaluator = MulticlassClassificationEvaluator(labelCol=target, metricName='f1')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "From [Medium Article by Dhiraj Rai](https://dhiraj-p-rai.medium.com/logistic-regression-in-spark-ml-8a95b5f5434c)\n",
    "- `regParam` the regularization parameter (>= 0). (default: 0.0)\n",
    "- `elasticNetParam` the ElasticNet mixing parameter, in range [0, 1]. For alpha = 0, the penalty is an L2 penalty. For alpha = 1, it is an L1 penalty. (default: 0.0)\n",
    "- `fitIntercept` whether to fit an intercept term. (default: True)\n",
    "- `maxIter` maximum iteration number.\n",
    "- `standardization` whether to standardize the training features before fitting the model. The coefficients of models will always be returned on the original scale, so it will be transparent for users. (default: True)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Hyperparameter Tuning (Grid Search) ~50 minutes"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "outputs": [],
   "source": [
    "if not skipGridSearch:\n",
    "    lrCv = CrossValidator(estimator = lr,\n",
    "                          estimatorParamMaps = lrParamGrid,\n",
    "                          evaluator = lrEvaluator,\n",
    "                          numFolds = 5)\n",
    "\n",
    "    lrCvModel = lrCv.fit(clf_train_df)\n",
    "\n",
    "    # Best Parameters found\n",
    "    bestParamMap = lrCvModel.bestModel.extractParamMap()\n",
    "    print( ','.join([''.join((param.name, '=', repr(value))) for param, value in bestParamMap.items()]) )"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Train"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "outputs": [],
   "source": [
    "lrBest = LogisticRegression(aggregationDepth=2,elasticNetParam=0.2,family='auto',featuresCol='features',fitIntercept=False,labelCol='winner',maxBlockSizeInMB=0.0,maxIter=100,predictionCol='prediction',probabilityCol='probability',rawPredictionCol='rawPrediction',regParam=0.01,standardization=True,threshold=0.5,tol=1e-06)\n",
    "# assert lrBest.extractParamMap() == bestParamMap, 'Check if the input parameters are the best found. '\n",
    "bestModel = lrBest.fit(clf_train_df)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Evaluation"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train f1 score 0.7636212881018776\n",
      "test f1 score 0.7470659930946674\n"
     ]
    }
   ],
   "source": [
    "trainPredictionAndLabels = bestModel.transform(clf_train_df).select('prediction', target)\n",
    "testPredictionAndLabels = bestModel.transform(clf_test_df).select('prediction', target)\n",
    "print('train f1 score', lrEvaluator.evaluate(trainPredictionAndLabels))\n",
    "print('test f1 score', lrEvaluator.evaluate(testPredictionAndLabels))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "outputs": [],
   "source": [
    "confusionMatrix = MulticlassMetrics(testPredictionAndLabels.rdd.map(tuple)).confusionMatrix().toArray()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 2 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEHCAYAAABRF9YCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA7uUlEQVR4nO3deZxP9f7A8dd7ViORqSyhqCZZsmSNSpu1ZaSUSrLciyKyleUm5bpxKSEUElJJXL9UtqJukjVbpJjoMrIUGsky2/v3xzkzfYdZvjPmO8vxfj4e38d8v5+zfD7nGO/5fD/nc95HVBVjjDHeEJTfDTDGGJN7LKgbY4yHWFA3xhgPsaBujDEeYkHdGGM8xIK6McZ4SEh+NyATNtcywNq2bZvfTbggzJs3L7+b4HmqKrmxm2ysm2l9IjIduAc4rKrVz1rWHxgNXK6qv7llg4AuQBLQS1WXuuV1gBlABLAI6K1ZzEO3nroxxgCq6vfLDzOAFmcXikgFoCmw16esKtAOqOZuM0lEgt3Fk4GuQJT7OmefZ7Ogbowx5G5QV9WvgKPpLBoLPEvabwXRwBxVPaOqe4AYoL6IlAWKq+pqt3c+C2idVd0FefjFGGPyTKDvrheR+4D9qrpFJM3oTTlgjc/nWLcswX1/dnmmLKgbYwzZC+oi0hVnWCTFFFWdksn6RYEhQLP0FqfXnEzKM2VB3RhjgOTkZL/XdQN4hkE8HdcAlYCUXnp5YKOI1MfpgVfwWbc88ItbXj6d8kzZmLoxxpDrF0rP3vd3qlpKVSuqakWcgH2jqh4EFgLtRCRcRCrhXBBdp6oHgD9EpKE4fwk6AB9lVZcFdWOMIXeDuoi8D6wGKotIrIh0yaTe7cBc4HtgCdBDVZPcxU8C03Aunv4ELM6qbht+McYYcvdCqao+ksXyimd9HgGMSGe9DUD1s8szY0HdGGMI/OyXvGJB3RhjsKBujDGekp3ZLwWZBXVjjMF66sYY4ykW1I0xxkMsqBtjjIdYUDfGGA+xC6XGGOMh1lM3xhgPsaBujDEeYkHdGGM8xIK6McZ4iAV1Y4zxEJv9YowxHmI9dWOM8RAL6sYY4yFeCeoBfZydiJQWkbdEZLH7uWpmj3UqaAYNGsRNN93EPffcc86yt956i8qVK3P06FEAVq1aRZs2bbj33ntp06YNq1evBuDUqVN07dqVFi1acPfddzNmzJg8PYbCKCgoiH//+98MHDgwtaxFixaMGzeOV199lfbt2wMQHBxMjx49eOWVVxg7diytW7fOpxZ7R/Pmzfnhhx/YtWsXzz33XH43J08F8hmleSnQPfUZwNvAEPfzTuAD4K0A15sr2rRpQ/v27c/55T5w4ADffPMNV1xxRWpZyZIlmTx5MqVLl2bnzp106dKFlStXAtC5c2caNmxIfHw8HTt25L///S9NmjTJ02MpTFq1asX+/fuJiIgAoFq1atSrV49+/fqRmJhI8eLFAbjpppsIDQ2lX79+hIWFMXbsWFatWsWvv/6an80vtIKCgpg4cSJNmzYlNjaW9evXs3DhQnbs2JHfTcsTBT1Y+yvQD56+TFXnAskAqpoIJGW+ScFRr149SpQocU75yy+/zIABA3Ae8O2oWrUqpUuXBiAqKor4+Hji4+OJiIigYcOGAISFhVG1alUOHTqUNwdQCEVGRnLjjTeyfPny1LJmzZrxf//3fyQmJgJw/PhxwPlPGB4eTlBQEGFhYSQmJnLq1Kl8abcX1K9fn5iYGPbs2UNCQgJz5swhOjo6v5uVZ5KTk/1+FWSB7qn/KSKXAgogIg2BuADXGVDLly+nVKlSXH/99Rmus3TpUqpUqUJYWFia8uPHj/PFF1/wxBNPBLqZhVanTp2YPXs2RYoUSS274oorqFKlCo888ggJCQnMmjWLn376iTVr1lCvXj2mTp1KWFgYM2fO5MSJE/nY+sKtXLly7Nu3L/VzbGwsDRo0yMcW5S3rqfunL7AQuEZEVgGzgF4BrjNgTp06xRtvvEHv3r0zXGfXrl2MGTOGl156KU15YmIiffv25fHHH6dChQqBbmqhdOONNxIXF8fu3bvTlAcFBXHRRRcxePBg3nnnHfr27QvAtddeS3JyMl27dqVHjx7ce++9lCpVKj+a7gm+3zxTeCXQ+cPG1P2zHWgCVAYE+JHA/yEJmL179xIbG5v6lfTgwYO0adOGDz/8kMsvv5yDBw/Ss2dPRo0axZVXXplm2+eff56KFSvSsWPHfGh54XD99ddTt25dateuTVhYGBERETz99NMcPXqUtWvXAhATE0NycjLFixfn5ptvZvPmzSQlJXH8+HF++OEHrrnmGg4fPpzPR1I4xcbGpulwlC9fnl9++SUfW5S3cjNYi8h04B7gsKpWd8tGA/cC8cBPQCdV/d1dNgjogjM83UtVl7rldXCuTUYAi4DemkVDAx1gV6tqoqpuV9VtqpoArA5wnQFTuXJlVq9ezYoVK1ixYgVlypThP//5D5dffjnHjx+na9eu9O3blzp16qTZbuzYsZw4cYLBgwfnU8sLh/fee4/u3bvTo0cPxo4dy7Zt25gwYQLr1q3jhhtuAKBs2bKEhIRw/PhxfvvtN6pXrw5AeHg41113Hfv378/PQyjU1q9fT1RUFBUrViQ0NJR27dqxcOHC/G5WnsnlnvoMoMVZZZ8B1VW1Bs6kkUHgzAoE2gHV3G0miUiwu81koCsQ5b7O3uc5AtJTF5EyQDkgQkRq4/TSAYoDRQNRZyD07duXdevWcezYMW699Vaefvpp2rZtm+66s2fPZu/evUyaNIlJkyYBMH36dBISEnjjjTe4+uqruf/++wFo3759hvsx5/riiy948skneeWVV0hMTGTixImAc+3iqaee4tVXX0VE+OKLL9i7d28+t7bwSkpKomfPnixdupTg4GCmT5/O999/n9/NyjO5eQFUVb8SkYpnlS3z+bgGeNB9Hw3MUdUzwB4RiQHqi8jPQHFVXQ0gIrOA1sDizOqWQIwPicgTQEegLrDBZ9EfwAxV/Y8fuynYA1ceYH9Y8sa8efPyuwmep6rnXhDIph9++MHvmFOlSpVuOD3oFFNUdYrvOm5Q/yRl+OWsZR8DH6jqbBF5HVijqrPdZW/hBO6fgZGqepdbfgvwnKqee+OMj4D01FV1JjBTRB5Q1fmBqMMYY3JTdjq4bgCfkuWK6RCRIUAi8G5KUXpVZFKeqUANv7R3/+pUFJG+57RK9dVA1GuMMTmVF7Na3FGMe4A7fS54xgK+U+LKA7+45eXTKc9UoC6UXuT+LAZcnM4rXSLSVUQ2iMiGKVNy9EfQGGNyJNBTGkWkBfAccJ+qnvRZtBBoJyLhIlIJ54LoOlU9APwhIg3FmW/aAfgoq3oCNfzypvvzxWxu5/uVJs/G1M+cOcNjjz1GfHw8SUlJNG/enF69erFjxw5eeOEFzpw5Q3BwMMOGDaNGjRrnbD9o0CC+/PJLLr30Uj755JM0y9555x1mz55NSEgITZo04dlnn+Xbb79l2LBhhIWF8eqrr3LVVVdx/Phx+vTpw7Rp09KdL1zYPfnkk9SpU4e4uDj69esHwOOPP06dOnVITEzk0KFDTJw4kZMnT56z7cSJEzl9+jTJyckkJSWdkxOmZcuWJCUlsXHjRmbPnk3lypX5+9//TkJCAuPGjePgwYMULVqUPn36MGLEiDw75vzWvHlzxo0bR3BwMNOmTWPUqFHnrDNu3DhatWrFyZMn6dixI5s2bcp025EjR9KyZUs2b96cehNd+/btiYyMZPz48Xl3cAGQy1Ma3wduAy4TkVjgBZzZLuHAZ+7/8TWq2l1Vt4vIXOB7nGGZHqqacuf9k/w1pXExWVwkhQDPUxeRy4G/AxV961LVzoGsN7tS7ka86KKLSEhI4NFHH+XWW29l/Pjx9OjRgyZNmvDf//6X0aNH884775yzfUY5YtasWcPy5cv5+OOPCQsL48iRIwC8/fbbTJgwgf379/P+++8zcOBAJk2aRLdu3TwZ0AG+/PJLlixZQs+ePVPLtmzZwrvvvktycjKPPfYY999/P++++2662w8bNow//vgjTVlGOWHuvfdexowZQ6lSpWjWrBmzZs3iwQcfZMGCBYE7wALGnzwuLVu2JCoqiqioKBo0aMDkyZNp2LBhhtvu37+fRo0aUbNmTWbPnk316tWJiYmhY8eOtGiR5Uy7Ai+XZ788kk5xhjmvVHUEcE6PQ1U3AOdcaM1MoOepfwSUAD4HPvV5FSgiwkUXOSNGiYmJJCYmIiKICH/++ScAf/zxR4Z3K2aUI+b999+na9euqekCLr30UgBCQkI4ffo0p06dIiQkhL1793Lo0CHq168fiMMrEHbs2HHOLfxbt25N/Y+0a9eu1PPjr4xywiQlJREWFkZYWBhJSUmULl2ayMjIC2p6nj95XKKjo5k1axYAa9eu5ZJLLqFMmTIZbpucnJz6uxwREUFCQgIDBgxg/Pjxqf8GhZndUeqfoqpaKPJ3JiUl0aZNG/bu3cujjz5KzZo1GTx4MF26dGHUqFEkJyczZ86cbO3z559/ZsOGDYwdO5bw8HCeffZZatSoQbdu3Rg6dCjh4eGMHj2aUaNGZZp64EJw++23880332S4/B//+AcAn332GZ9//jmQcU6YBQsW0K1bN+Lj45kwYQIdOnTI9r9dYedPHpf01ilXrlyG2544cYL58+ezadMmli9fTlxcHPXq1WP48OGBP6A8UNCDtb8CHdQ/EZFWqroowPWct+DgYD766COOHz9Ojx492LlzJ3PnzmXQoEE0b96cRYsWMWTIEGbMmOH3PlNuX587dy7fffcdzzzzDMuXL6dKlSrMnTsXcO7iK1WqFKrKM888Q0hICAMHDuSyyy4L0JEWPG3atCE5OTk1VfHZ/vGPf3Ds2DGKFy/O888/z/79+9mxY0eanDDXXnstffv2pUePHvz8888MGeJke65SpQrHjh1DROjTpw+JiYnMmjWLuLhCnVcuS/7kccloncy2HT16NKNHjwZg6tSpDB06lC5dutCsWTO2bt1aqK9ZeCWoB2T4RUT+EJHjOMm7PhGRU25ZSnmBVbx4cRo0aMDKlStZsGABzZo1A5zxx61bt2ZrX6VLl6Zp06aICDVq1CAoKIhjx46lLldVJk+ezFNPPcXrr7/O008/zX333ZfuuL1XNWnShDp16jBu3LgM10k5Z8ePH2fdunVce+21ABnmhPH1wAMPMG/ePNq2bcsHH3zAypUradWqVYCOpuDwJ49LRuv4s22tWrUA2LlzJx06dODhhx+mevXqqf82hZFXhl8CEtRV9WJVLY4zpt4duNEtSykvUI4ePZo6Hnv69Gm++eYbrr76akqVKsW6desA56JnxYoVs7Xfu+66izVr1gCkjk+WLFkydfmCBQto0qQJJUqU4PTp0wQFBREUFHTB5ASvVasWrVu3ZtSoUcTHx6e7Tnh4eGoa3vDwcGrWrJk6NJBRTpgUt912Gxs3buTPP/8kPDwcVU0zLuxl/uRxWbhwIR06dACgQYMGxMXFcfDgQb+2HT58OEOHDiU0NJTgYCdNSXJyMkWLFposIOfwSlAP9PDL28DNwHgRuRrYBKxU1Yy7Zfng8OHDDBw4kKSkJFSVFi1acPvtt3PxxRfzr3/9i8TERMLDw1PT6R46dIh//OMfTJ06Fcg4R8wDDzzA4MGDueeeewgNDWXkyJGpX21PnTrFggULmD59OuDkEe/VqxehoaG88sor+XMiAqh3795Uq1aNiy++mDfeeIO5c+dy//33ExISwvPPPw84vb6pU6dSsmRJunfvzssvv0yJEiUYMGAA4AyRff3112zevBnIOCcMODOamjRpwj//+U8APv74Y/r3709iYiKvvfZanh57fsgoj0u3bt0AePPNN1m0aBGtWrUiJiaGkydP0qlTp0y3TREdHc369es5cOAAAKtXr2br1q2pr8KqoD/8wl8Byf2SpgIn21g94HacXvspVc34CRN/Kdh/Dj3Acr/kDcv9Eni5kftl3bp1fsec+vXrF9i5x4Gep74c5+7S1cBKoJ6qWrJrY0yBU9CHVfwV6HnqW3ESwlcHagDVRSQiwHUaY0y22Zi6H1S1D4CIFAM64Yyxl8G5VdYYYwqMgh6s/RXo4ZeewC1AHeB/wHScYRhjjClQvHKhNNCzXyKAV4FvVbXw30dsjPEs66n7QVVHB3L/xhiTWyyoG2OMh1hQN8YYD7GgbowxHmJB3RhjPMRmvxhjjIdYT90YYzzEgroxxniIBXVjjPEQrwT1QCf0MsaYQiE3E3qJyHQROSwi23zKIkXkMxHZ5f4s6bNskIjEiMiPItLcp7yOiHznLhsv6T1r8CwW1I0xBmf2i78vP8wAWpxVNhBYrqpRwHL3MyJSFWgHVHO3meQ+hwJgMtAViHJfZ+/zHBbUjTGG3O2pq+pXwNGziqOBme77mUBrn/I5qnpGVfcAMUB9ESkLFFfV1epUOstnmwzZmLoxxpAnY+qlVfWAW9cBESnllpcD1visF+uWJbjvzy7PlPXUjTGG7PXURaSriGzweXU9j6rTGyfXTMozZT11Y4whez11VZ0CTMlmFYdEpKzbSy8LpDzaMxao4LNeeeAXt7x8OuWZsp66McaQ6xdK07MQeMJ9/wTwkU95OxEJF5FKOBdE17lDNX+ISEN31ksHn20yZD11Y4whd8fUReR94DbgMhGJBV4ARgJzRaQLsBdo69a7XUTmAt8DiUAPVU1yd/UkzkyaCGCx+8qUBXVjjCF3g7qqPpLBojszWH8EMCKd8g1A9ezUbUHdGGPwzh2lFtSNMQYL6sYY4ykW1AOsbdu2+d0Ezxs+fHh+N+GCMG/evPxugvGDPSTDGGM8xHrqxhjjIRbUjTHGQyyoG2OMh1hQN8YYD7GgbowxHmKzX4wxxkOsp26MMR5iQd0YYzzEgroxxniIBXVjjPEQu1BqjDEeYj11Y4zxEM8HdRGZQCZPrlbVXgFpkTHG5APPB3VgQ561whhj8pnng7qqzvT9LCIXqeqfgW+SMcbkPa8E9aCsVhCRm0Tke2CH+7mmiEwKeMuMMSYPJScn+/0qyLIM6sBrQHPgCICqbgFuDWCbjDEmz6mq36+CzJ+gjqruO6soKQBtMcaYfJObQV1E+ojIdhHZJiLvi0gREYkUkc9EZJf7s6TP+oNEJEZEfhSR5udzHP4E9X0i0ghQEQkTkf64QzHGGOMVuRXURaQc0Auoq6rVgWCgHTAQWK6qUcBy9zMiUtVdXg1oAUwSkeCcHoc/Qb070AMoB+wHarmfjTHGM3J5+CUEiBCREKAo8AsQDaRMQJkJtHbfRwNzVPWMqu4BYoD6OT2OLG8+UtXfgMdyWoExxhQG2bkAKiJdga4+RVNUdQqAqu4XkTHAXuAUsExVl4lIaVU94K5zQERKuduWA9b47CvWLcuRLIO6iFwNjAMa4tyMtBroo6q7c1qpMcYUNNm5AOoG8CnpLXPHyqOBSsDvwIci0j6T3Ul6VfjdmLP4M/zyHjAXKAtcAXwIvJ/TCo0xpiDKxeGXu4A9qvqrqiYA/wEaAYdEpCyA+/Owu34sUMFn+/I4wzU54k9QF1V9R1UT3dds/PwrIiKR6ZRVym4jjTEm0HIxqO8FGopIURER4E6cySULgSfcdZ4APnLfLwTaiUi4Gx+jgHU5PY7Mcr+kBOQvRGQgMAcnmD8MfOrn/j8WkZaqetzdZ1WcXn/1nDY4vwUFBTFy5EiOHj3KyJEj6dOnD1dccQUARYsW5eTJkwwYMACA1q1bc+edd5KcnMz06dPZsmVLfja9QBo/fjwbNmygRIkSTJgwAYBVq1bx/vvvExsby+jRo4mKigIgISGBSZMm8dNPPyEi/O1vf+OGG24A4KuvvmLevHkAREZG0rdvX4oXL54/B1WINW/enHHjxhEcHMy0adMYNWpUfjcpz+TW/HNVXSsi84CNQCKwCWeophgwV0S64AT+tu7620VkLvC9u34PVc3xtPHMxtS/xQniKeM93XzbDQz3Y///wgnsdwOVgVkU8ouurVq1Yv/+/URERAAwduzY1GUdOnTg5MmTAJQvX57GjRvTp08fIiMjef755+ndu3eBvxstr915553cfffdvPbaa6llV155JQMHDmTy5Mlp1l22bBng/CH4/fffeemllxgzZgyqyrRp03j99dcpXrw4M2bM4NNPP+WRRx7Jy0Mp9IKCgpg4cSJNmzYlNjaW9evXs3DhQnbsuDBmMOfmTUWq+gLwwlnFZ3B67emtPwIYkRt1Zzj8oqqVVPVq9+fZr6v92bmqfgqMBZYBM4DWqro5NxqeHyIjI7nxxhtZvnx5ustvuukmvv76awDq1q3LqlWrSExM5PDhwxw8eJBrr702L5tbKFSrVo1ixYqlKatQoQLly5c/Z919+/ZRs2ZNAC655BIuuugiYmJiUr8Snz59GlXl5MmTREaeM/JnslC/fn1iYmLYs2cPCQkJzJkzh+jo6PxuVp7xSpoAv/Kpi0h1oCpQJKVMVWdlsv7ZaXuLA7uBp0Wk0Kbt7dSpE7Nnz6ZIkSLnLKtSpQpxcXEcPHgQgEsvvZSdO3emLj969KgFmvNUqVIl1q5dyy233MJvv/3GTz/9xG+//cZ1111H9+7d6dWrF0WKFKFs2bJ069Yt6x2aNMqVK8e+fX/dPB4bG0uDBg3ysUV5q6Df/u8vf6Y0vgDchhPUFwEtga9xhlIycnba3m9z2L4C48YbbyQuLo7du3dTtWrVc5bffPPNqb30jHjllya/3HXXXezbt49+/fpx+eWXc/311xMcHExiYiJLlixh7NixlClThilTpjB//nweeuih/G5yoeJc00vrQvqd9cqx+tNTfxCoCWxS1U4iUhqYltkGKWl7ReQi4HTKoL9762v4+TU5f1x//fXUrVuX2rVrExYWRkREBE8//TQTJkwgKCiI+vXr89xzz6Wuf+TIES699NLUz5GRkRw7diw/mu4ZwcHB/O1vf0v9/Oyzz1K2bFn27NkDQNmyZQHnD+z8+fPzpY2FWWxsLBUq/DWzrnz58vzyS45n1hU6Xgnq/kxpPKWqyUCiiBTHmVvp15g6Tn6DCJ/PEcDn2WtiwfDee+/RvXt3evTowdixY9m2bVvqbI0aNWrwyy+/cPTo0dT1N2zYQOPGjQkJCaFUqVKULVuWmJiY/Gq+J5w5c4bTp08DsHnzZoKDg7nyyiuJjIxk3759xMXFpS5Lb0zeZG79+vVERUVRsWJFQkNDadeuHQsXLszvZuUZr2Rp9KenvkFELgGm4gyjnMD/OZRFVPVEygdVPSEiRbPdygKucePG5wy9xMbGsnr1asaOHUtycjLTpk0r8BdY8sOYMWPYtm0bx48fp3PnzjzyyCMUK1aMqVOnEhcXx/Dhw6lUqRIvvvgiv//+O8OGDSMoKIjIyEj69OkDONcvHn74YQYPHkxwcDClSpWiV69CedkmXyUlJdGzZ0+WLl1KcHAw06dP5/vvv8/vZuWZgh6s/SXZORARqQgUV9Wtfq6/CnhaVTe6n+sAr6vqTVlt27ZtW2+c4QJs+HB/ZqWa81WlSpX8boLnqWp6t9pny0svveR3zBk6dOh51xcomd18dGNmy1ICdRaewcl7kDIwVxbn5iVjjClQvNJTz2z45ZVMlilwR1Y7V9X1InI9zo1HAvzg5kJIl2/msxtvvJGrr/Z36N4YY86P54O6qt6eS3VU5q857rXdeerpTof0zXyWl8MvTz75JHXq1CEuLo5+/foB8Pjjj1OnTh0SExM5dOgQEydOTL1b1FfRokV58sknqVChAqrK5MmT2blzJw0bNuShhx6iXLlyDBo0iN27naSWlStX5u9//zsJCQmMGzeOgwcPUrRoUfr06cOIEblyQ1mBFB8fz+DBg0lISCApKYlGjRrx6KOP8vbbb7N+/XpCQkIoU6YMvXr1OudmJEg/nQDA+++/z7JlyyhRogQA7du3p27duuzYsYPJkycTGhpK//79KVu2LCdOnGD06NEMGzYs3el7XuPPLf/jxo2jVatWnDx5ko4dO7Jp06ZMtx05ciQtW7Zk8+bNPPGEk8akffv2REZGMn78+Lw7uADwfFDPDTmc457nvvzyS5YsWULPnj1Ty7Zs2cK7775LcnIyjz32GPfffz/vvvvuOdt26tSJTZs28corrxASEkJYWBjg3P04ZswYunbtmmb9e++9lzFjxlCqVCmaNWvGrFmzePDBB1mwYEFgDzKfhYaGMnz4cCIiIkhMTGTgwIHUqVOHWrVq0aFDB4KDg5k5cybz589PDRa+0ksnkOK+++7j/vvvT1P2f//3fwwcOJDDhw+zePFiOnfuzNy5c2nbtu0FEdD9ueW/ZcuWREVFERUVRYMGDZg8eTINGzbMcNv9+/fTqFEjatasyezZs6levToxMTF07NiRFi1a5OPR5g6vBHW/nlF6Hh7EyXVwUFU74cx3L3Dz1Hfs2MGJEyfSlG3dujV1tsquXbvSzDlPERERQdWqVVmxYgUAiYmJqb35/fv3pzvHNykpibCwMMLCwkhKSqJ06dJERkZ6fpaBiKTmy0lKSiIpyclXVLt2bYKDnSd3XXfddfz222/pbp9eOoHMBAcHEx8fz5kzZwgODubAgQMcOXKE6tULbS65bPHnlv/o6GhmzXL6V2vXruWSSy6hTJkyGW6bnJyc2mmJiIggISGBAQMGMH78eBITE/P8GHPbBZUm4DycUtVkEcnJHPcC4/bbb+ebb745p7x06dIcP36cHj16cNVVV7F7927efvttzpw5k+G+FixYQLdu3YiPj2fChAl06NCBOXPmBLL5BUZSUhL9+vXjwIEDtGrVisqVK6dZvnz5cm6++eZs73fRokV88cUXXHvttXTu3JlixYrx4IMPMnHiRMLCwujTpw9vv/02jz1WqHPJZYs/t/ynt065cuUy3PbEiRPMnz+fTZs2sXz5cuLi4qhXr55nZlF5pafuT5oAwcmseLWqviQiVwJlVNWfuernM8e9QGjTpg3JycmsXLnynGVBQUFUqlSJt956i5iYGDp16kTr1q354IMPMtzfzz//zJAhQwBnqtuxY8cQEfr06UNiYiKzZs1KvYnGa4KDg3nttdc4ceIEL7/8Mv/73/+46qqrAJg7dy5BQUE0adIkW/ts2bIlDz30ECLCu+++y/Tp0+nVqxdXX301o0ePBmD79u1ERkaiqvz73/8mJCSEzp07c8kll+T2IRYY/tzyn9E6mW07evTo1PM6depUhg4dSpcuXWjWrBlbt24t1NeFvBLU/Rl+mQTcBKTkMf0DmOjPzlX1KVX9XVXfAJoCT7jDMIVCkyZNqFOnDuPGjUt3+dGjRzly5EjqnaKrV6/O1oydBx54gHnz5tG2bVs++OADVq5cSatWrXKl7QVZsWLFuOGGG9i40ZkVu2LFCjZs2EC/fv2yPd59ySWXEBwcTFBQEM2aNWPXrl1plqsqc+fO5eGHH2bOnDk8+uij3HbbbXz88ce5djwFkT+3/Ge0jj/b1qpVC4CdO3fSoUMHHn74YapXr16oM5F65Y5Sf4J6A1XtAZwGUNVjQFhmG4jIjWe/gEggJLP57wVJrVq1aN26NaNGjSI+Pj7ddX7//XeOHDmS+pCMG264gdjYWL/2f9ttt7Fx40b+/PNPwsPDUdU0Y5ZeExcXl3rd4syZM2zZsoXy5cuzceNG5s+fz5AhQwgPz/7lFt/UDGvWrOHKK69Ms3zFihXUrVuXYsWKcebMGUQEEcl0iMwL/Lnlf+HChXTo0AGABg0apGYZ9Wfb4cOHM3ToUEJDQ1OviSQnJ1O0aOG9YdwrQd2fMfUENxGXAojI5UBWVwp857j7ngHBzznueal3795Uq1aNiy++mDfeeIO5c+dy//33ExISwvPPPw84PZKpU6dSsmRJunfvzssvvwyQ+nU/JCSEQ4cOMWnSJMC5UNW5c2eKFy/OoEGD+Pnnn1O/moaFhdGkSRP++c9/AvDxxx/Tv39/EhMT053d4QXHjh3jtddeIzk5GVWlcePG1KtXj27dupGQkMALLzjPE7juuut46qmnOHLkCBMnTmTo0KFA+ukEmjZtysyZM1MTepUqVYqnnnoqtc4zZ86wYsUKXnzxRcC5MDhq1ChCQkJSp656VUa3/KekJH7zzTdZtGgRrVq1IiYmhpMnT9KpU6dMt00RHR3N+vXrOXDgAOB8Q926dWvqq7Aq6MHaX1mmCRCRx3DuAr0RmIkzo+UfqvphljsXiQCeAm7GCeYrgcmqejqrbS1NQOB55QJXQWdpAgIvN9IEDBgwwO+YM3r06AI7LzbLnrqqvisi3+JMTRScpxf5+3yrmcBxIOWuhEdw5qhbomtjTIHilZ66P7NfrgROAh/7lqnqXj/2X1lVa/p8/kJE7OnLxpgC54IJ6sCn/PUA6iJAJeBHoJof224SkYaqugZARBoAq3LYVmOMCZgLJqir6g2+n93ZK5k+AFJEvsP5QxAKdBCRve7nqwBv3zppjCmULpigfjZV3Sgi9bJY7Z4ctscYY/JFbgZ196bLaUB1nA5tZ5wRjg+AisDPwEPuFHFEZBDQBUgCeqnq0pzW7c+Yel+fj0E4s2B+zWwbVf1fThtkjDH5IZdzuowDlqjqgyISBhQFBgPLVXWkiAwEBgLPiUhVoB3OkPYVwOcicl3Ks52zy5+bjy72eYXjjLFHZ7qFMcYUMrl185Gb5+pW4C13v/Gq+jtO3JzprjYTaO2+jwbmqOoZVd0DxAD1c3ocmfbU3ZuOiqnqgJxWYIwxhUE2H+2Z+kAf1xT3eRDgJC38FXhbRGri5L3qDZRW1QNuXQdEpJS7fjlgjc++Yt2yHMnscXYhqppYWG7rN8aY85GdoO77QJ90hOAMUz+tqmtFZBzOUEtG0ruRKccD/Jn11Ne5DdssIguBD4E/U2tU/U9OKzXGmIImFy+UxgKxqrrW/TwPJ6gfEpGybi+9LE4q8pT1K/hsXx4492EMfvJnTD0SOIKTr+Ue4F5sdosxxmNy6yEZqnoQ2CciKQ8MuBNnKvdCIOWxXk8AH7nvFwLtRCRcRCoBUZxHivLMeuql3Jkv2/jr5qPUdue0QmOMKYhyeZ7608C77syX3UAnnE70XBHpAuwF2rr1bheRuTiBPxHokdOZL5B5UA8GipHL4z3GGFMQ5WZQV9XNQN10Ft2ZwfojgFx5wkhmQf2Aqr6UG5UYY0xBdyHcUVpgU0saY0xuuxCCerpfE4wxxos8H9RV9WhGy4wxxmtyOU1Avsl2Qi9jjPEiz/fUjTHmQmJB3RhjPMSCujHGeIgFdWOM8RAL6sYY4yE2+8UYYzzEeurGGOMhFtSNMcZDLKgH2Lx58/K7CZ5n5zhvVKhQIeuVTL6zoG6MMR5iF0qNMcZDrKdujDEeYkHdGGM8xIK6McZ4iAV1Y4zxEAvqxhjjIV6Z/RKU3w0wxpiCQFX9fvlDRIJFZJOIfOJ+jhSRz0Rkl/uzpM+6g0QkRkR+FJHm53McFtSNMYbcD+pAb2CHz+eBwHJVjQKWu58RkapAO6Aa0AKYJCLBOT0OC+rGGEPuBnURKQ/cDUzzKY4GZrrvZwKtfcrnqOoZVd0DxAD1c3ocFtSNMYZc76m/BjwL+A7Ul1bVA25dB4BSbnk5YJ/PerFuWY5YUDfGGJwLpf6+RKSriGzweXVN2Y+I3AMcVtVv/axa0inL8VQcm/1ijDFkb0qjqk4BpmSwuDFwn4i0AooAxUVkNnBIRMqq6gERKQscdtePBXyzvpUHfslu+1NYT90YY8i94RdVHaSq5VW1Is4F0BWq2h5YCDzhrvYE8JH7fiHQTkTCRaQSEAWsy+lxWE/dGGPIk5uPRgJzRaQLsBdo69a7XUTmAt8DiUAPVU3KaSUW1I0xhsAEdVX9EvjSfX8EuDOD9UYAI3KjTgvqxhiDpQkwxhhP8UqaAAvqxhiD9dSNMcZTLKgbY4yHWFA3xhgPsaBujDEeYkHdGGM8xGa/GGOMh3ilpx7Q3C8icoeIFA1kHQVFUFAQGzdu5OOPP87vpnjSW2+9xaFDh/juu+/yuymF3ujRo9m4cSOfffZZmvKOHTvyxRdf8PnnnzN48GAAatasyeLFi1m8eDFLliyhefPzeihPgRaAh2Tki0D31DsCb4jIEWCl+/paVY8FuN4817t3b3bs2EHx4sXzuymeNGPGDF5//XVmzZqV300p9D788ENmzpzJ2LFjU8tuuukmmjVrRvPmzYmPj+fSSy8F4Mcff+See+4hKSmJUqVKsWTJEj7//HOSknKcmqTAKujB2l8B7amragdVvQ54ACe95ETg10DWmR/KlSvH3XffzbRp07Je2eTIypUrOXr0aH43wxPWrVvH77//nqbs8ccfZ9KkScTHxwNw5MgRAE6fPp0awMPDwz0T+NLjlZ56oIdf2ovIm8A84C7gdeCWQNaZH1577TWeffZZz1xoMReeSpUqUb9+fT766CPmzp1LjRo1UpfVqlWLzz//nGXLljF48GBP9tIhew/JKMgCnU/9NaAWMBXopar/VtXVAa4zT919990cPnyYjRs35ndTjMmxkJAQSpQoQXR0NCNGjGDSpEmpyzZv3sxdd93FvffeS48ePQgPD8/HlgaO9dT9oKqXAZ1xnv4xQkTWicg7gawzrzVu3Jj77ruPPXv2MGfOHO644w7eecdTh2guAAcOHGDx4sUAbNmyBVUlMjIyzToxMTGcPHmSypUr50cTA86Cuh9EpDhwJXAVUBEoQdoHsRZ6gwcPpkKFClSqVIl27dqxYsUKHn/88fxuljHZsmzZMho1agQ4QzGhoaEcPXqUChUqEBwcDDjXjq655hr27duX2a4KLa8E9UDPfvna5/W6qsYGuD7jUe+99x633XYbl112Gfv27eOFF15g+vTp+d2sQmnChAncdNNNlCxZkrVr1/Lqq6/ywQcfMHr0aD777DPi4+Pp27cvAPXq1eOpp54iISGB5ORkhgwZwrFjnpu8Bnhn9osU1AMRkYLZMGOyqUKFClmvZM7L3r175Xz3UaNGDb9jztatW8+7vkAJaE9dRC4HngWq4YyrA6CqdwSyXmOMya6CPqvFX4Ge/fIu8ANQCXgR+BlYH+A6jTEm27wyph7ooH6pqr4FJKjqf1W1M9Awo5VFpKuIbBCRDQFulzHGpGFB3T8J7s8DInK3iNQGyme0sqpOUdW6qlo3wO3KVPPmzfnhhx/YtWsXzz333DnLK1euzDfffMPp06fp169fmmUZ5SgZOXIkW7ZsYebMmall7du3p1evXoE5iAIuq3MMMG7cOHbt2sWWLVuoXbs24NzVuHbtWjZv3sy2bdsYNmxY6vp2js8VFBTEokWLePvtt89Z1rRpU5YuXcrixYv55JNPqFevHgBXX311ar6XxYsXs337drp06QLAoEGDWLp0aZoUA23atKFz5855c0ABZEHdP/8UkRJAP6A/MA3oE+A6z0tQUBATJ06kZcuWVK1alUceeYQqVaqkWefo0aP06tWLMWPGnLP9jBkzaNGiRZqy4sWL06hRI2rWrElwcDDVq1enSJEidOzYMc1NHhcKf85xy5YtiYqKIioqiq5duzJ58mQAzpw5wx133EGtWrWoVasWLVq0oEGDBnaOM9C5c2diYmLSXbZq1SqaN29Oy5Yt6d+/P6NGjQJg9+7dtGzZkpYtW3L33Xdz6tQplixZwsUXX0ydOnVo3rw5QUFBVK5cmfDwcNq2beuJnDy5FdRFpIKIfCEiO0Rku4j0dssjReQzEdnl/izps80gEYkRkR9F5LyypgUsqItIMBClqnGquk1Vb1fVOqq6MFB15ob69esTExPDnj17SEhIYM6cOURHR6dZ59dff2XDhg0kJCScs316OUqSk5MJCwsDICIigoSEBAYMGMD48eNJTEwM3MEUUP6c4+jo6NRAsXbtWi655BLKlCkDwJ9//glAaGgooaGhqKqd43SUKVOGO++8kzlz5qS7/OTJk6nvixYtmm6waty4MXv37mX//v0kJycTGhoKQJEiRUhMTKR79+5Mnz7dE+c4F3vqiUA/Va2CM9zcQ0SqAgOB5aoaBSx3P+Mua4czoaQFMMmNnzkSsKCuqknAfYHaf6CUK1cuzc0VsbGxlCtX7rz2eeLECebPn8+mTZvYs2cPcXFx1KtXj4ULC/Tft4Dx5xxntk5QUBCbNm3i8OHDfPbZZ6xbt87OcTqGDRvGv/71r0xndTRv3pwVK1YwY8YMBgwYcM7y++67j48++ghw/pimDMns27ePP/74g5o1a56Twrewyq3cL6p6QFU3uu//AHYA5YBoIGVscCbQ2n0fDcxR1TOqugeIAern9DgCPfzyjYi8LiK3iMiNKa8A13leRM6dfpobY2ijR4+mdu3a9O/fn+HDhzN06FC6dOnCBx98wJAhQ857/4WJP+c4s3WSk5OpXbs25cuXp379+lSrVg2wc+zrzjvv5Lfffssy//zSpUu54447+Nvf/kb//v3TLAsNDaVp06Z8+umnqWVvvPEGLVu25J///Cf9+vXjlVdeoV27dkyaNImnn346IMeSV7LTU/ed1OG+uqa3TxGpCNQG1gKlVfWAW9cBoJS7WjnA9zbdWLcsRwId1BvhfKV4ERgDvOL+LLBiY2PT3CxSvnx5fvnll1zbf61atQDYuXMnHTp04OGHH6Z69epce+21uVZHQefPOfZnnbi4OL788stzrmHYOYa6devStGlTVq1axeuvv06jRo147bXXMlx/3bp1XHnllZQsmTrMy2233ca2bdv47bffzlk/5Q/p7t27eeCBB3jqqaeoXLkyFStWzO1DyTPZCeq+kzrc15Sz9ycixYD5wDOqejyTqtO7kSnHPcmABHUR6SsifYFPgI+BT93XJ+6rwFq/fj1RUVFUrFiR0NBQ2rVrl6tf4VN6kKGhoak5NZKTkyla9IJ4QBTg3zleuHAhHTp0AKBBgwbExcVx8OBBLrvsMkqUKAE447p33XUXP/zwQ5pt7RzDqFGjaNCgAY0bN6Znz5588803PPPMM2nWueqqq1LfV69enbCwsDQpAKKjo1OHXs7Wv39/Xn311XPOcURERO4fTB7JzdkvIhKKE9DfVdX/uMWHRKSsu7wscNgtjwV8bzsuD+S4JxmoO0ovdn9WBuoBH+H8NboX+CpAdeaKpKQkevbsydKlSwkODmb69Ol8//33dOvWDYA333yT0qVLs2HDBooXL05ycjLPPPMMVatW5Y8//sg0R0l0dDTr16/nwIEDAKxevZqtW7emvi4U/pzjRYsW0apVq9TMgJ06dQKgbNmyzJw5k+DgYIKCgpg7d26a4QE7x5lr3749ALNnz6ZVq1Y88MADJCQkcPr0aXr06JG6XpEiRbjlllsYNGjQOfto1qwZW7Zs4dChQwB8++23LFu2jB07drBjx468OZAAyK2piuKMHb4F7FDVV30WLQSeAEa6Pz/yKX9PRF4FrgCigHU5rj+Qcy5FZBnwgHuxABG5GPhQVVtkvqXlfjHeYblfAi83cr9cddVVfsec//3vfxnWJyI34zy68zv+yko7GGdcfS5O5tq9QFtVPepuMwQnTXkiznDN4pwcAwQ+S+OVQLzP53icFLzGGFOg5FYHV1W/Jv1xcoA7M9hmBDAiN+oPdFB/B1gnIgtwBv7v568pPcYYU2AU9DtF/RXQoK6qI0RkMX89l7STqm4KZJ3GGJMTFtT95E7Ctwd4GmMKNAvqxhjjIRbUjTHGQ7zykAwL6sYYg/XUjTHGUyyoG2OMh1hQN8YYD7GgbowxHmJB3RhjPMRmvxhjjIdYT90YYzzEgroxxniIBXVjjPEQC+rGGOMhdqHUGGM8xHrqxhjjIRbUjTHGQyyoG2OMh1hQN8YYD/FKUA/K7wYYY0xBkJyc7PcrKyLSQkR+FJEYERmYB81PZT11Y4wh93rqIhIMTASaArHAehFZqKrf50oFWbCeujHG4AR1f19ZqA/EqOpuVY0H5gDRAT8Al/XUjTGGXB1TLwfs8/kcCzTIrZ1npcAGdVWV/G5DdolIV1Wdkt/t8DI7x4F3oZ7j7MQcEekKdPUpmuJzztLbT55dhbXhl9zVNetVzHmycxx4do6zoKpTVLWuz8v3j2AsUMHnc3ngl7xqmwV1Y4zJXeuBKBGpJCJhQDtgYV5VXmCHX4wxpjBS1UQR6QksBYKB6aq6Pa/qt6Ceuy64cch8YOc48OwcnydVXQQsyo+6xSt3URljjLExdWOM8RQL6jkkIicyKO8uIh3c9x1F5Iq8bVnhIiIVRWRbOuVfikjdHOyvo4i8njut8y4RGSYi/fO7HSb32Zh6LlPVN3w+dgS2kYfTmYzJKREJUdXE/G6HOT/WU8+AiDwrIr3c92NFZIX7/k4Rme2+HyEiW0RkjYiUdsuGiUh/EXkQqAu8KyKbRSRCROqIyH9F5FsRWSoiZfPr+AqYEBGZKSJbRWSeiBT1XSgik0Vkg4hsF5EXfcrricg37r/BOhG5+Kzt7haR1SJyWV4dSEEmIkPcJFOfA5Xdsi9F5F8i8l+gt4jcKyJrRWSTiHzu83v9nYhcIo4jPt9G3xGRu/LvqMzZLKhn7CvgFvd9XaCYiIQCNwMrgYuANapa0133774bq+o8YAPwmKrWAhKBCcCDqloHmA6MyIPjKAwq49yRVwM4Djx11vIhqloXqAE0EZEa7vzfD4De7r/BXcCplA1E5H5gINBKVX/Li4MoyESkDs586dpAG6Cez+JLVLWJqr4CfA00VNXaODlLnnXXWQU0BqoBu/nr/0ZDYE3gj8D4y4ZfMvYtUMft/Z0BNuIE91uAXkA88InPuk2z2F9loDrwmYiAM3/1QO43u1Dap6qr3Pezcc6vr4fc27JDgLJAVZzbrg+o6noAVT0O4J7b23H+rZqllBtuARao6kkAEfG9GeYDn/flgQ/cb5FhwB63fCVwK/A/YDLQVUTKAUdVNd3rSyZ/WE89A6qaAPwMdAK+wfmlvh24BtgBJOhf80GTyPoPpADbVbWW+7pBVZsFpPGFz9nzalM/i0gloD9wp9uT/xQognM+M5qPuxu4GLgu95taqGV0vv70eT8BeF1VbwC64Zxr+Oub6y3Al8CvwIM4/y9MAWJBPXNf4QSUr3B+ebsDm32CeVb+wAkuAD8Cl4vITQAiEioi1XK5vYXVlSnnBXgEZwggRXGcoBPnju+2dMt/AK4QkXoAInKxiKT8Yf0fzhDDLDvHqb4C7nev7VwM3JvBeiWA/e77J1IKVXUfcBkQpaq7cf6N+mNBvcCxoJ65lThf91er6iHgNNn7JZ4BvCEim3GGWx4ERonIFmAz0Cg3G1uI7QCeEJGtQCTO13sAVHULsAnYjnMdYpVbHg88DExwz+dn/NWrRFV/BB4DPhSRa/LoOAosVd2IM8yyGZhPxr/Hw3DO2Urg7GsRa4Gd7vuVOClmv8YUKHZHqTHGeIj11I0xxkMsqBtjjIdYUDfGGA+xoG6MMR5iQd0YYzzEgrrJlIgkublrtonIh2fnZcnmvma4OXEQkWkiUjWTdW8TkWxP+RSRn9PL9ZJR+VnrZOvOSMt0aAoiC+omK6fcO2Cr46RG6O67UESCc7JTVf2bqn6fySq3YfP4jck2C+omO1YC17q96C9E5D3gOxEJFpHRIrLezbTYDcDN6Pe6iHwvIp8CpVJ2JD750kWkhYhsdLMtLheRijh/PPq43xJuEZHLRWS+W8d6EWnsbnupiCxzswq+iZM+IFMi8n/iZMrc7uaU8V32ituW5SJyuVt2jYgscbdZKSLX58rZNCYALKGX8Yt7C35LYIlbVB+orqp73MAYp6r1RCQcWCUiy3AyAlYGbgBKA9/j3BXqu9/LganAre6+IlX1qIi8AZxQ1THueu8BY1X1axG5EuehvlWAF4CvVfUlEbkbSBOkM9DZrSMCWC8i81X1CE7mzY2q2k9Ehrr77onzzM7uqrpLRBoAk4A7cnAajQk4C+omKxFumgNweupv4QyLrFPVlAx+zYAaKePlOPlDonCy+r2vqknAL+LmpD9LQ+CrlH2p6tEM2nEXUNXNwghQ3M1hcitOnhdU9VMROebHMfVyU/MCVHDbegRI5q+MhbOB/4hIMfd4P/SpO9yPOozJFxbUTVZOufngU7nBzTeznwBPq+rSs9ZrRcaZAX239SdXRRBwk6qe8i102+J3rgsRuQ3nD8RNqnpSRL7EJ2fMWdSt9/ezz4ExBZWNqZvcsBR40n2ICCJynYhchJMZsJ075l4WJ3Xx2VbjPPiikrttpFvum+ESYBnOUAjuerXct1/hJO5CRFoCJbNoawngmBvQr8f5ppAiCCfpGsCjOMM6x4E9ItLWrUNEpGYWdRiTbyyom9wwDWe8fKM4D5F+E+db4AJgF/AdTubF/569oar+ijMO/h8322LK8MfHOKliN4tIyoNJ6roXYr/nr1k4LwK3ishGnGGgvVm0dQnO4/O2AsNJ+9SeP4FqIvItzpj5S275Y0AXt33bgWg/zokx+cKyNBpjjIdYT90YYzzEgroxxniIBXVjjPEQC+rGGOMhFtSNMcZDLKgbY4yHWFA3xhgPsaBujDEe8v8CL2Ksi4SxOAAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "categories = ['white', 'black', 'draw']\n",
    "make_confusion_matrix(confusionMatrix, categories=categories, sum_stats=False, cmap='gray')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "The LogisticRegression performs significantly better than the DecisionTreeRegressor."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Regression Problem\n",
    " Can a player's rating be determined from a single match record with another player?"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Feature Selection"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "outputs": [
    {
     "data": {
      "text/plain": "('rated',\n 'turns',\n 'match_status',\n 'clock',\n 'increment',\n 'open_cat',\n 'opening_ply',\n 'moves_vec',\n 'winner',\n 'black_rating')"
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = 'rated', 'turns', 'match_status', 'clock', 'increment', 'open_cat', 'opening_ply', 'moves_vec', 'winner', #'white_rating',\n",
    "target = 'black_rating'\n",
    "inputColumns = features + (target,)\n",
    "inputColumns"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "outputs": [],
   "source": [
    "training_assembler = VectorAssembler(handleInvalid='error', inputCols=features, outputCol='features')\n",
    "test_assembler = VectorAssembler(handleInvalid='error', inputCols=features, outputCol='features')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "outputs": [],
   "source": [
    "reg_train_df = training_assembler.transform(df_train.select(*inputColumns)).select('features', target)\n",
    "reg_test_df = test_assembler.transform(df_test.select(*inputColumns)).select('features', target)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Model Selection"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "outputs": [],
   "source": [
    "from pyspark.ml.regression import DecisionTreeRegressor, LinearRegression\n",
    "from pyspark.ml.tuning import ParamGridBuilder, CrossValidator\n",
    "from pyspark.ml.evaluation import RegressionEvaluator\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### *Decision Tree* for Regression"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "outputs": [],
   "source": [
    "dt = DecisionTreeRegressor(labelCol=target, featuresCol=\"features\", seed=seed)\n",
    "\n",
    "dtParamGrid = (ParamGridBuilder()\n",
    "               .addGrid(dt.maxDepth, list(range(2, 8)))\n",
    "               .addGrid(dt.maxBins, [4, 8, 16, 32, 64])\n",
    "               .addGrid(dt.minInstancesPerNode, list(range(1, 4)))\n",
    "               .build())\n",
    "\n",
    "dtEvaluator = RegressionEvaluator(labelCol=target, metricName='rmse')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "From [Spark Docs](https://spark.apache.org/docs/latest/mllib-decision-tree.html#tunable-parameters)\n",
    "- `maxDepth`: Maximum depth of a tree. Deeper trees are more expressive (potentially allowing higher accuracy), but they are also more costly to train and are more likely to overfit.\n",
    "- `minInstancesPerNode`: For a node to be split further, each of its children must receive at least this number of training instances. This is commonly used with RandomForest since those are often trained deeper than individual trees.\n",
    "- `maxBins`: Number of bins used when discretizing continuous features.\n",
    "- `impurity`: Impurity measure used to choose between candidate splits. (Only `variance` for regression)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Hyperparameter Tuning (Grid Search) ~18 min"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "outputs": [],
   "source": [
    "if not skipGridSearch:\n",
    "    dtCv = CrossValidator(estimator = dt,\n",
    "                          estimatorParamMaps = dtParamGrid,\n",
    "                          evaluator = dtEvaluator,\n",
    "                          numFolds = 5)\n",
    "\n",
    "    dtCvModel = dtCv.fit(reg_train_df)\n",
    "\n",
    "    # Best Parameters found\n",
    "    bestParamMap = dtCvModel.bestModel.extractParamMap()\n",
    "    print( ','.join([''.join((param.name, '=', repr(value))) for param, value in bestParamMap.items()]) )"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Train"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "outputs": [],
   "source": [
    "# Use Best Parameters to initialize and train a separate regressor model\n",
    "dtBest = DecisionTreeRegressor(cacheNodeIds=False,checkpointInterval=10,featuresCol='features',impurity='variance',labelCol='black_rating',leafCol='',maxBins=8,maxDepth=6,maxMemoryInMB=256,minInfoGain=0.0,minInstancesPerNode=3,minWeightFractionPerNode=0.0,predictionCol='prediction',seed=42)\n",
    "# assert dtBest.extractParamMap() == bestParamMap, 'Check if the input parameters are the best found. '\n",
    "bestModel = dtBest.fit(reg_train_df)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Evaluation"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train rmse 259.3537297282463\n",
      "test rmse 275.8599898782995\n"
     ]
    }
   ],
   "source": [
    "trainPredAndLabels = bestModel.transform(reg_train_df).select('prediction', target)\n",
    "testPredAndLabels = bestModel.transform(reg_test_df).select('prediction', target)\n",
    "print('train rmse', dtEvaluator.evaluate(trainPredAndLabels))\n",
    "print('test rmse', dtEvaluator.evaluate(testPredAndLabels))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\tools\\miniconda3\\envs\\bigdata\\lib\\site-packages\\seaborn\\distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).\n",
      "  warnings.warn(msg, FutureWarning)\n",
      "C:\\tools\\miniconda3\\envs\\bigdata\\lib\\site-packages\\seaborn\\distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).\n",
      "  warnings.warn(msg, FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEWCAYAAACufwpNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABJZElEQVR4nO3deXhU5dn48e+dyZ5A2BEESVBEEUExJChLXaoCUnFDwSJSF0TFWtta5ddWeWut+mJ9tbWK0bpQQEFxwQJqsajsQhTZkUU2CWsk+zYzz++Pc2acJJNkEmYymeT+XNdcZM55njP3mQxz51nOc8QYg1JKKRUMUeEOQCmlVPOhSUUppVTQaFJRSikVNJpUlFJKBY0mFaWUUkGjSUUppVTQaFJRTYqIvC4if7Z/Hioi2xt4nBki8sfgRqdqIyKLReTWcMehwkuTiqo3EdkjIiUiUigih0XkNRFJDvbrGGOWGWN6BxDPRBFZXqXuZGPMY8GOSUSmiUiFfe6ex4lgv05jEBEjIkX2OXwvIs+IiCPAutNEZJbvNmPMCGPMGyGKtZUd3x475n0i8o6IZITi9VTDaVJRDfUzY0wyMAAYCPyhagERiW70qBrHXGNMss+jjb9C/s6/vu9JI7yH/e3f40+Am4DbQvx69SYiccB/gXOBUUBr4GzgLWBkGENTfmhSUSfFGPM9sBjoC96/fu8VkR3ADnvbKBFZLyInRGSliPTz1BeR80XkKxEpEJG5QLzPvotF5IDP8+4i8q6IHBWR4yLyvIicDcwALvRtNfh2o9nP7xSRnSKSKyILRKSrzz4jIpNFZIeI/CAi/xARacj7UfX8PecgIg+JyCHgNRGJE5FnReSg/XjW/uLEX/kqx4+z38e+Pts62i3HTiLSQUT+bZfJFZFlIlLn/3NjzE5gBXCez3GfE5H9IpIvItkiMtTePhz4f8BN9nv+jb39MxG5w/55oogsF5Gn7ff0OxEZ4XPsNBH5wv69L7Hf80otHx+3AN2Aa4wxm4wxLmNMkTHmHWPMtLritfdNs1s2c+3X/EpE+tf1vqj606SiToqIdMf6a/Frn83XAJlAHxEZALwK3AW0B14CFthfjrHA+8C/gHbA28D1NbyOA/g3sBdIBU4F3jLGbAUmA6tqajWIyKXAE8CNQBf7GG9VKTYKq8XV3y53ZeDvQjXXYJ+//fwUrPPrAUwCfg8MwvoC7w9kULmlV7W8lzGmDHgXGOez+Ubgc2PMEeA3wAGgI9AZ68u/zrWYROQsYCiw02fzWjvGdsAc4G0RiTfGfAT8hR9bbDV9OWcC24EOwP8C//RJ1nOAL7E+E9OwEkdNfgp8bIwpquM0/Mbrs3801mfMs/99EYmp45iqvowx+tBHvR7AHqAQOIH1Bf0CkGDvM8ClPmVfBB6rUn87VnfLMOAgID77VgJ/tn++GDhg/3whcBSI9hPPRGB5lW2v+xznn8D/+uxLBiqAVJ+Yh/jsnwc8XMO5TwPK7XP3PJb67K96/hfb5eN9tu0CRvo8vxLYU1N5PzH8FNjt83wFMMH++U/AB8AZAfweDZAPFNk/vwnE1VL+B6zuMs/7MKvK/s+AO3x+Jzt99iXar3EKcBrgBBJ99s+qejyffUuAJ32en2e/7/nA9nrEu9pnXxSQAwwN9/+n5vbQlopqqGuMMW2MMT2MMfcYY0p89u33+bkH8Bu7O+aE3T3VHehqP7439v9y294aXq87sNcY42xArF19j2uMKQSOY7V2PA75/FyMlXhqMs8+d8/jkir791d5ftQYU1pTPPbPXWspX9V/gQQRyRSRHlhfsu/Z+6ZjtTY+EZHdIvJwLccBa0wsGWs8JRNI8uwQkd+IyFYRybN/bylYrY5Aed9TY0yx/WMy1rnm+myD6u+Zr+NYLUzPsdYbq0V6HRBXj3j3+xzDjdWi833fVRBoUlGh4Jsk9gOPV/kSTjTGvIn1l+KpVcYvTqvhmPuB08T/wHVd3TsHsZIbACKShNXt8n1dJ9JAVeOp+rxSPFjnfLCW8pUPZn0hzsPqArsZ+LcxpsDeV2CM+Y0xpifwM+DXInJZHcczxph5wCrgEbCmcwMPYXWttbW/xPMAz+/qZJY3zwHaiUiiz7butZT/FLjC/r35FUC8lV7DHmfqRuX3XQWBJhUVai8Dk+2/qkVEkkTkKhFphfUl5gR+KSLRInId1viCP19ifRk9aR8jXkQG2/sOA93sMRp/5gC/EJHz7AHxvwBrjDF7gnSO9fUm8Ad7gL0D1hd5TYPUNZmD1br4uf0z4J0UcYadqPMBl/0IxJPAJBE5BWiF9bs5CkSLyCNYs648DgOpgUwCqMoYsxdYB0wTkVgRuRArAdZkJtbv/j0R6SsiDnusJN2nTF3xAlwgItfZf5j8CigDVtc3flU7TSoqpIwx64A7geex+rh3YvW3Y4wpx+rCmGjvuwlrENrfcVxYXzxnAPuwui5usnf/F9gMHBKRY37qfgr8EZiP9eV0OjD2JE7LM+vJ99GpHvX/jPWlugHYCHxlbwuYMWYN1lhIV6zZdx69sMYgCrGS9gvGmM8CPOZG4HPgQeBj+7jfYnXPlVK5i+pt+9/jIvJVfWK3/RxrnOw41rnPxfqS9xdXKXAJsAVYiD2WgjWx4ka7WF3xgjXWdBPWZ+0W4DpjTEUDYle1kMrd2Uop1fjEmk6+zRjzaIiOPw1r8sL4UBxf/UhbKkqpRiciA0XkdBGJsq97GY01vVxFuOZ6xbNSqmk7Baursz1WV+bdxpiva6+iIoF2fymllAqakHZ/ichwEdku1vIY1ebL27OB/mbv32BffV1rXRF5zC67XkQ+kcrLbUy1y28XkZO5IloppVQDhKylYi+r8S1wOVbzdi0wzhizxafMSOA+rGU+MoHnjDGZtdUVkdbGmHy7/i+BPsaYySLSB2uqZgbWjJglwJn2rCG/OnToYFJTU4N85kop1bxlZ2cfM8Z09LcvlGMqGVjLNOwGEJG3sAbjtviUGQ3MtK+oXi0ibUSkC9baTn7rehKKLYkfL8IajbUWVBnwnYjstGNYVVOAqamprFu37uTPVCmlWhARqWnli5B2f51K5XniB6i8LEZtZWqtKyKPi8h+rLnuj9Tj9RCRSSKyTkTWHT16tF4npJRSqnahTCr+lg6v2tdWU5la6xpjfm+M6Q7MBqbU4/UwxmQZY9KNMekdO/ptvSmllGqgUCaVA1Rez8ffOjs1lQmkLljLU3iWSg+0jlJKqRAJ5ZjKWqCXiKRhLdw3FmvxO18LgCn2mEkmkGeMyRGRozXVFZFexpgddv2rgW0+x5ojIs9gDdT3wlovSqkWpaKiggMHDlBaWttCx0rVLT4+nm7duhETE/htZ0KWVIwxThGZgrUmjwN41RizWUQm2/tnAIuwZn7txFpu/Be11bUP/aSI9AbcWGv8eI63WUTmYU0EcAL31jbzS6nm6sCBA7Rq1YrU1FSkYTewVApjDMePH+fAgQOkpaUFXK9FX/yYnp5udPaXam62bt3KWWedpQlFnTRjDNu2bePss8+utF1Eso0x6f7q6NpfSjVDmlBUMDTkc6RJRSmlVNDogpJKNXPZ2dlBPd4FF1xQ6/49e/YwatQoNm3aVG3fxRdfzNNPP016ut+ekxq9/vrrrFu3jueff75e9QI9xsiRI5kzx7rX2Zw5c7jnnnsa/Dp1xXDFFVfQtau1utQdd9zBr3/9a/r06ROS1wsHTSpKhVlNX/p1fXmr4Fm0aBFgJcQXXnjhpJKKy+XC4XD43ff666/Tt29fb1J55ZVXGvw6TZV2fymlgs7pdHLrrbfSr18/brjhBoqLi6uVufvuu0lPT+ecc87h0Ud/vDfX2rVrueiii+jfvz8ZGRkUFBRUqrdw4UIuvPBCjh2rfJPPc889lxMnTmCMoX379sycOROAW265hSVLlgBw8OBBhg8fTq9evfjd737nrZuamsqxY8d4+OGH2bVrF+eddx4PPvggANOnT2fgwIH069evUpy+kpOTeeSRR8jMzGTVqlX86U9/YuDAgfTt25dJkyZhjOGdd95h3bp1/PznP+e8886jpKSEiy++2LtUVHJyMr///e/p378/gwYN4vDhwwDs2rWLQYMGMXDgQB555BGSk5P9xvDYY49x1llncfnllzNu3DiefvppAF5++WUGDhxI//79uf76672/i4kTJzJ58mSGDh3KmWeeyb///W+/x60vTSpKqaDbvn07kyZNYsOGDbRu3ZoXXnihWpnHH3+cdevWsWHDBj7//HM2bNhAeXk5N910E8899xzffPMNS5YsISEhwVvnvffe48knn2TRokV06NCh0vEGDx7MihUr2Lx5Mz179mTZsmUArF69mkGDBgGwfv165s6dy8aNG5k7dy7791e+4/CTTz7J6aefzvr165k+fTqffPIJO3bs4Msvv2T9+vVkZ2fzxRdfVDuXoqIi+vbty5o1axgyZAhTpkxh7dq1bNq0iZKSEv79739zww03kJ6ezuzZs1m/fn2l8/IcY9CgQXzzzTcMGzaMl19+GYD777+f+++/n7Vr13pbOFWtW7eO+fPn8/XXX/Puu+9WWtPwuuuuY+3atXzzzTecffbZ/POf//Tu27NnD59//jkLFy5k8uTJQbm2SZOKUirounfvzuDBgwEYP348y5cvr1Zm3rx5DBgwgPPPP5/NmzezZcsWtm/fTpcuXRg4cCAArVu3Jjra6qVfunQpTz31FAsXLqRt27bVjjd06FC++OILvvjiC+6++242btzI999/T7t27bx/3V922WWkpKQQHx9Pnz592Lu3xnURAfjkk0/45JNPOP/88xkwYADbtm1jx44d1co5HA6uv/567/OlS5eSmZnJueeey3//+182b95crU5VsbGxjBo1CrC6Pvfs2QPAqlWrGDNmDAA331z1+nHL8uXLGT16NAkJCbRq1Yqf/exn3n2bNm1i6NChnHvuucyePbtSLDfeeCNRUVH06tWLnj17sm3bNn+HrxdNKkqpoKs6FbXq8++++46nn36aTz/9lA0bNnDVVVdRWlqKMabGaaw9e/akoKCAb7/91u/+YcOGsWzZMpYtW8bFF19Mx44deeeddxg6dKi3TFxcnPdnh8OB0+ms9TyMMUydOpX169ezfv16du7cye23316tXHx8vHccpbS0lHvuuYd33nmHjRs3cueddwbUAoiJifGeeyCxVY2zJhMnTuT5559n48aNPProo5Viqev31BCaVJRSQbdv3z5WrbLuOvHmm28yZMiQSvvz8/NJSkoiJSWFw4cPs3jxYgDOOussDh48yNq1awEoKCjwfrn26NGDd999lwkTJvj9y7979+4cO3aMHTt20LNnT4YMGcLTTz9dKanUpVWrVpXGcK688kpeffVVCgsLAfj+++85cuRIrcfwfGl36NCBwsJC3nnnnRqPH4hBgwYxf/58AN566y2/ZYYMGcKHH35IaWkphYWFLFy40LuvoKCALl26UFFRwezZsyvVe/vtt3G73ezatYvdu3fTu3fvesXmj87+UqqZC8cssrPPPps33niDu+66i169enH33XdX2t+/f3/OP/98zjnnHHr27OntKouNjWXu3Lncd999lJSUkJCQ4B1kB+jduzezZ89mzJgxfPjhh5x++umVjpuZmYnLZa3ONHToUKZOnVotodWmffv2DB48mL59+zJixAimT5/O1q1bufDCCwFrMH3WrFl06tSpxmO0adOGO++8k3PPPZfU1FRvVx78ODiekJDgTbp1efbZZxk/fjx//etfueqqq0hJSalWZuDAgVx99dX079+fHj16kJ6e7i332GOPkZmZSY8ePTj33HMrJbXevXvzk5/8hMOHDzNjxgzi4+MDiqk2ukyLLtOiwizYU4q3bt1abVkNFbmKi4tJSEhARHjrrbd48803+eCDD6qVKywsJDk5meLiYoYNG0ZWVhYDBgzwc0TLxIkTGTVqFDfccEOtr+/v81TbMi3aUlFKqSYsOzubKVOmYIyhTZs2vPrqq37LTZo0iS1btlBaWsqtt95aa0IJJU0qSinVhA0dOpRvvvmmznKeFQEC9frrrzcwotrpQL1SSqmg0aSiVBgdOnSIVatWkZeXF+5QlAoKTSpKhdHmzZs5fvw4q1at4vjx4+EOR6mTpklFqTDKyckhMTERh8PBrl27wh2OUidNB+qVChNjDAcPHqRDhw44nU7y8/ND8jpZWVlBPd6kSZNq3X/ixImTWj7+2WefZdKkSSQmJjaofn0kJydTWFjIwYMH+eUvf1npQsW64vIsl9+mTZuQxxlJtKWiVJjk5eVRUlJCSkoKSUlJFBcX43a7wx3WSTtx4oTfBSQD9eyzz/pd1ThQ9VnexKNr1661JhSoHteiRYsaLaFUPadAz7Eh78XJ0paKUmFy8OBBAFJSUoiOjsYYQ3FxcY1Lm0cK3+XjL7/8cqZPn8706dOZN28eZWVlXHvttfzP//wPRUVF3HjjjRw4cACXy8Uf//hHDh8+zMGDB7nkkkvo0KEDS5curXTs1NRUbrrpJu/2OXPmcMYZZzBx4kTatWvH119/zYABA7jnnnu49957OXr0KImJibz88sucddZZfPfdd9x88804nU6GDx/uPa7vjcVcLhcPPfQQH3/8MSLCnXfe6W1V+saVmprKunXr6NChA88884z3+pE77riDX/3qV+zZs4cRI0YwZMgQVq5cyamnnsoHH3xQbXXio0ePMnnyZPbt2wdYyWvw4MFMmzaNgwcPsmfPHjp06MCZZ55Z6fkTTzzBbbfdxtGjR+nYsSOvvfYap512WrX34q9//Wsof93VaFJRKkxycnKIioqidevW3m1FRUURn1SefPJJNm3axPr16wEqLR9vjOHqq6/miy++4OjRo3Tt2tW7TlVeXh4pKSk888wzLF26tNrS9h6tW7fmyy+/ZObMmfzqV7/y3gfk22+/ZcmSJTgcDi677DJmzJhBr169WLNmDffccw///e9/uf/++7n77ruZMGEC//jHP/wePysri++++46vv/6a6OhocnNzadeuXY1xZWdn89prr7FmzRqMMWRmZvKTn/yEtm3bsmPHDt58801efvllbrzxRubPn8/48eMr1b///vt54IEHGDJkCPv27ePKK69k69at3mMvX76chIQEpk2bVun5z372MyZMmMCtt97Kq6++yi9/+Uvef//9au9FY9OkolSY5OTk0KlTJxwOB0lJSYC11Ebnzp3DHFlw+S4fD9Y57tixg6FDh/Lb3/6Whx56iFGjRgW88OO4ceO8/z7wwAPe7WPGjMHhcFBYWMjKlSu9y8UDlJWVAbBixQrv4oy33HILDz30ULXjL1myhMmTJ3uX3G/Xrl2t8Sxfvpxrr73W+zu87rrrWLZsGVdffTVpaWmcd955QOXl7Ku+3pYtW7zP8/PzvetzXX311ZVaNr7PV61axbvvvus9F9+bjnnei3DQpKJUmBw6dIhevXoB1kKKsbGxFBUVhTmq4PMsH3/XXXdV25ednc2iRYuYOnUqV1xxBY888kidx/Ndnt33Z8+Xutvtpk2bNt6WUm31a4q3PkvA17Z+YtWl9ktKSqqVcbvdrFq1qlq3GPx4TjU99+XvvQgHHahXKgycTidFRUWVBnqTkpK8S6xHskCXjz948CCJiYmMHz+e3/72t3z11Vd+61c1d+5c77+e1YN9tW7dmrS0NN5++23A+tL3LHMyePBg7/LxVZeB97jiiiuYMWOGd5A7Nze31riGDRvG+++/T3FxMUVFRbz33nv1Wm7/iiuu4Pnnn/c+rykZVnXRRRdVOpf6rMYcStpSUSoMPNOHU1JSvEu1Jycn13mvjoaoawpwsAW6fPzOnTt58MEHiYqKIiYmhhdffNEb74gRI+jSpUu1gXqwurIyMzNxu928+eabfmOYPXs2d999N3/+85+pqKhg7Nix9O/fn+eee46bb76Z5557rtKdGn3dcccdfPvtt/Tr14+YmBjuvPNOpkyZUmNcAwYMYOLEiWRkZHjrn3/++X67uvz529/+xr333ku/fv1wOp0MGzaMGTNmBFTvtttuY/r06d6B+qZAl77Xpe9VGOzZs4c33niDW265hR9++AGAnTt3sm3bNq688kpiYmJ06Xs/fGdcqcZR36XvtftLqTDwtFR8Z355+sFP5hoNpcItpElFRIaLyHYR2SkiD/vZLyLyN3v/BhEZUFddEZkuItvs8u+JSBt7e6qIlIjIevtRd/tRqTDxl1Q8d93zzFRS1Xmu0VBNV8iSiog4gH8AI4A+wDgR6VOl2Aigl/2YBLwYQN3/AH2NMf2Ab4GpPsfbZYw5z35MDs2ZKXXy8vPziY+PJzY21rvNM1PIc4/zk9GSu7VV8DTkcxTKlkoGsNMYs9sYUw68BYyuUmY0MNNYVgNtRKRLbXWNMZ8YYzxrD6wGuoXwHJQKifz8/EqtFPgxqZxsSyU+Pp7jx49rYlEnxRjD8ePH633f+lDO/joV2O/z/ACQGUCZUwOsC3AbMNfneZqIfA3kA38wxiyrWkFEJmG1ijjttNMCOhGlgs1fUnE4HMTExJx0UunWrRsHDhzg6NGjJ3UcpeLj4+nWrX5/t4cyqfi7eqjqn041lamzroj8HnACnsnmOcBpxpjjInIB8L6InGOMqbT0qzEmC8gCa/ZXnWehVAjk5eXRpUuXatvj4uJOOqnExMSQlpZ2UsdQqqFCmVQOAN19nncDDgZYJra2uiJyKzAKuMzYbXxjTBlQZv+cLSK7gDMBnTOsmhSn00lxcXG1lgoEJ6koFU6hHFNZC/QSkTQRiQXGAguqlFkATLBngQ0C8owxObXVFZHhwEPA1cYY79xLEeloD/AjIj2xBv93h/D8lGoQz1XZKSkp1fbFxcUFZaBeqXAJWUvFGOMUkSnAx4ADeNUYs1lEJtv7ZwCLgJHATqAY+EVtde1DPw/EAf+x17pZbc/0Ggb8SUScgAuYbIzJDdX5KdVQnvvRa0tFNUchXabFGLMIK3H4bpvh87MB7g20rr39jBrKzwfmn0y8SjUGf9eoeMTFxeFyucJycyWlgkGvqFeqkdWWVPQCSBXpNKko1cgKCwu9S91XFaxrVZQKF00qSjWywsLCGu/uGMyr6pUKB00qSjWyQJKKtlRUpNKkolQjKygooFWrVn73xcbGIiKaVFTE0qSiVCOrraUiIjqtWEU0vfOjUo0kOzsbp9NJeXk5eXl5ZGdn+y2nF0CqSKYtFaUakacF4hk78Sc2Npby8vLGCkmpoNKkolQj8rRAaltOPC4uTpOKiliaVJRqRIG2VMrKyvR+KCoiaVJRqhF5kkpdLRW3201FRUVjhaVU0GhSUaoRlZaWIiLExMTUWMZzpX1RUVFjhaVU0GhSUaoRlZWVERcXh73Ctl+erjFNKioSaVJRqhF5kkpttKWiIpkmFaUaUWlpaa3jKaAtFRXZNKko1Yi0paKaO00qSjUSt9tNeXl5nUnF4XAQHR2tSUVFJE0qSjWSQKYTe8TGxlJcXBzqkJQKOk0qSjWSQC589IiLi9OWiopImlSUaiT1SSqxsbGaVFRE0qSiVCMJZN0vD00qKlJpUlGqkdS3+6u4uFjX/1IRR5OKUo2krKyMmJgYoqLq/m8XGxuL2+3W+6qoiKNJRalGEsiFjx56AaSKVJpUlGokgVz46KFJRUUqTSpKNZKysrKAWyp6Vb2KVJpUlGoExph6tVQ0qahIFdKkIiLDRWS7iOwUkYf97BcR+Zu9f4OIDKirrohMF5Ftdvn3RKSNz76pdvntInJlKM9NqfooKSnB7XZrUlHNXsiSiog4gH8AI4A+wDgR6VOl2Aigl/2YBLwYQN3/AH2NMf2Ab4Gpdp0+wFjgHGA48IJ9HKXCrrCwEAjsGhWAqKgoEhISNKmoiBPKlkoGsNMYs9sYUw68BYyuUmY0MNNYVgNtRKRLbXWNMZ8YY5x2/dVAN59jvWWMKTPGfAfstI+jVNh5kkqgLRWApKQkXf9LRZxQJpVTgf0+zw/Y2wIpE0hdgNuAxfV4PURkkoisE5F1R48eDeA0lDp5BQUFQP2TirZUVKQJZVLxd7/UqpcH11Smzroi8nvACcyux+thjMkyxqQbY9I7duzop4pSwVff7i/QpKIiU3QIj30A6O7zvBtwMMAysbXVFZFbgVHAZebHdSwCeT2lwqKgoMB7n5RAJSYmalJRESeULZW1QC8RSRORWKxB9AVVyiwAJtizwAYBecaYnNrqishw4CHgamNMcZVjjRWROBFJwxr8/zKE56dUwPLz80lISKhXnaSkJO+sMaUiRchaKsYYp4hMAT4GHMCrxpjNIjLZ3j8DWASMxBpULwZ+UVtd+9DPA3HAf0QEYLUxZrJ97HnAFqxusXuNMa5QnZ9S9ZGfn1+vri+wkgpAcXExycnJoQhLqaALZfcXxphFWInDd9sMn58NcG+gde3tZ9Tyeo8Djzc0XqVCpaCggNatW9erjiepFBUVaVJREUOvqFcqxNxuNwUFBQ3q/gK9AFJFFk0qSoVYYWEhxpgGd39pUlGRRJOKUiGWn58P1G86MWhSUZFJk4pSIeZJKvXt/oqPj0dENKmoiKJJRakQa2hLRUR0qRYVcTSpKBVi+fn5REdHExMTU++6elW9ijSaVJQKsYKCAlq1aoV9XVW9aFJRkUaTilIhlp+fX+9rVDySk5O9i1EqFQk0qSgVYiebVDxTkpWKBJpUlAohY0yDrqb3aNWqFS6Xi9LS0iBHplRoaFJRKoSKiopwuVwn1VIBtAtMRQxNKkqF0IkTJwBo06ZNg+q3atUK+PF+LEo1dZpUlAqhvLw8AFJSUhpUX1sqKtIElFREZL6IXCUimoSUqodgtVQ0qahIEWiSeBG4GdghIk+KyFkhjEmpZuPEiRPEx8fX6970vmJjY4mNjdXuLxUxAkoqxpglxpifAwOAPVg3yFopIr8QkfpfJqxUC5GXl9fgrq/s7Gyys7OJiYlh3759ZGdnBzk6pYIv4O4sEWkPTATuAL4GnsNKMv8JSWRKNQN5eXkN7vryiIuLo6ysLDgBKRVigY6pvAssAxKBnxljrjbGzDXG3AfoLemU8sMYw4kTJxrcUvGIj4/X61RUxAj0dsKv2Lf39RKROGNMmTEmPQRxKRXxSktLKS8vP+mkoi0VFUkC7f76s59tq4IZiFLNjWc68cl2f8XHx+NyuXA6nUGISqnQqrWlIiKnAKcCCSJyPuBZZrU1VleYUsqP7OxsDh06BEBOTg4lJSUNPpZn5ph2galIUFf315VYg/PdgGd8thcA/y9EMSnVLHgSSX3v+FiV5+ZemlRUJKg1qRhj3gDeEJHrjTHzGykmpZqFkpISoqKiiI2NPanjaFJRkaSu7q/xxphZQKqI/LrqfmPMM36qKaWwFpNMSkpq0M25fHlaOppUVCSoq/sryf5Xpw0rVU/FxcUkJp780KPD4SAmJuakxmWUaix1dX+9ZP/7P40TjlLNgzGG4uJiOnToEJTjJSQkaFJRESHQix//V0Rai0iMiHwqIsdEZHyog1MqUpWXl+NyuYLSUgFNKipyBHqdyhXGmHxgFHAAOBN4sK5KIjJcRLaLyE4RedjPfhGRv9n7N4jIgLrqisgYEdksIm4RSffZnioiJSKy3n7MCPDclAq64uJigKAlFb2qXkWKQK+o9ywaORJ40xiTW9fgo4g4gH8Al2MlorUissAYs8Wn2Aigl/3IxFoNObOOupuA64CX/LzsLmPMeQGek1IhE+ykkpCQQEVFBWVlZQ1e8VipxhBoS+VDEdkGpAOfikhHoK4/mzKAncaY3caYcuAtYHSVMqOBmcayGmgjIl1qq2uM2WqM2R5g3EqFRSiSCkB+fn5QjqdUqAS69P3DwIVAujGmAiiieoKo6lRgv8/zA/a2QMoEUtefNBH5WkQ+F5Gh/gqIyCQRWSci644ePRrAIZWqv6KiIuLi4nA4HEE5niepeJZ+UaqpCrT7C+BsrOtVfOvMrKW8v/4xE2CZQOpWlQOcZow5LiIXAO+LyDn2WNCPBzEmC8gCSE9Pr+uYSjVIcXExSUlJdRcMkCYVFSkCSioi8i/gdGA94LI3G2pPKgeA7j7PuwEHAywTG0DdSowxZUCZ/XO2iOzCmlCwrrZ6SoVCoNOJ33333WrbrrvuumrbPOMomlRUUxdoSyUd6GOMqc9f9muBXiKSBnwPjMW6JbGvBcAUEXkLa6A+zxiTIyJHA6hbiT3Ok2uMcYlIT6zB/931iFepoHA6nZSWlvodT/GXRAIRFRVFfHy8jqmoJi/QpLIJOAWriykgxhiniEwBPgYcwKvGmM0iMtnePwNYhDWjbCdQDPyitroAInIt8HegI7BQRNYbY64EhgF/EhEnVmtqsjEmN9B4lQqWH374Aah7kL68vJwtW7awa9cu8vPzSUhIIC0tjSuuuILk5OqLWCQkJGhLRTV5gSaVDsAWEfkSu4sJwBhzdW2V7Bt7LaqybYbPzwa4N9C69vb3gPf8bJ8P6KKXKuyOHz8OUOOYijGGb775hs8++4zi4mJSUlJo164d+fn5fPrpp6xcuZLhw4dz9tlnAz92h2lSUZEg0KQyLZRBKNWc1JZUSktL+fDDD9mxYwfdu3fn+uuvp1u3bt5FJw8dOsTixYt57733OHbsGEOGDPHWTUxMJCcnB5fLFbRZZUoFW0BJxRjzuYj0AHoZY5aISCJWt5RSqorjx48TGxtbbcn7PXv28Prrr3PixAkuv/xy0tPTq61gfMoppzBhwgQWL17MsmXLKCsr47rrrkNESExMxBhDXl4e7dq1a8xTUipgga79dSfwDj9exX4q8H6IYlIqouXm5lZrpezevZtJkyZRWlrKuHHjGDhwYI1L4jscDq666irS09P58ssvmT59OsYY7zE9YzZKNUWBdn/di3WV+xoAY8wOEekUsqiUimDHjx+nbdu23ud79uzh7rvvRkS45ZZbaN++fZ3HEBEuv/xyHA4H8+bNIyEhgdtvvx2ABQsWVEtakyZNCu5JKNVAgSaVMmNMuecvK/sCSL1wUKkqysrKKCwspFu3boDVqpgyZQoAL730El999VXAxxIRLr30Urp168Ybb7xBYmIiXbt2xeVy1V1ZqTAJdO2vz0Xk/wEJInI58DbwYejCUioy+Q7SO51Opk6dSm5uLs8++yypqan1Pp6I8NBDDzFy5EhefPFF1q9fj9PpDHLUSgVPoC2Vh4HbgY3AXVhTfV8JVVBKRSrfpPL888+zbt06pk2b5p0e3BBRUVE88sgjlJaWMn/+fJxOJyNHjgxWyEoFVaCzv9wi8j7wvjFGV2FUyo+srCwKCgoAePXVV5k1axYDBgygvLy8wVfSe0RHR/P4449z3333sWDBAjp06EBGRkYwwlYqqGrt/rJvojVNRI4B24DtInJURB5pnPCUiixOpxO3282iRYto06YNl156adCOHRMTw9NPP81pp53GzJkz2bdvX9COrVSw1DWm8itgMDDQGNPeGNMOa42uwSLyQKiDUyrSOJ1OvvjiC3JzcxkxYkS1a1VOVpcuXRgzZgxJSUnMmDGDoqKioB5fqZNVV1KZAIwzxnzn2WCM2Q2Mt/cppWzGGI4cOcKyZcs499xzSUtLC/prtGvXjuTkZCZOnEheXh6vvfYaxhiysrIqPZQKl7qSSowx5ljVjfa4Soyf8kq1WC6Xi6VLlxIVFcUll1wSktfwXP9y6qmnct1117Fx40a+/PLLkLyWUg1RV1Ipb+A+pVqcPXv2sHHjRi6++GK/qwwHg8PhIDo6GqfTySWXXEJaWhpvv/02JSUlIXk9peqrrtlf/UXE3w0cBIgPQTxKRawPP/yQ+Ph4hg8f7p1aHAy+M8eys7OJjo6moqKCqKgoxo4dyxNPPMFHH33EtddeG7TXVKqham2pGGMcxpjWfh6tjDHa/aWUbd26dWzbto2hQ4eGrJXiER0djcvlwhhDamoqGRkZfPrpp97pzEqFU6BX1CulajF9+nTi4uK46KKLQv5a0dFWB4PnyvqRI0dSUVHB0qVLQ/7aStVFk4pSJ+m7777jnXfeIT09PeStFKieVLp06UL//v357LPPKCsrq62qUiGnSUWpk/Tss8/icDjIzMz0fuGHUtWkAnDllVdSVFTE8uXLQ/76StVGk4pSJ+GHH37gn//8JyNGjKB169bExIR+qFFEcDgclZLK6aefzhlnnMGSJUt0FWMVVppUlDoJ//rXvygqKmLEiBEAjdJS8bxO1dWKf/rTn5Kbm8uWLVsaJQal/NGkolQDGWN46aWXyMjIIDk5mejo6Brv5hhsnqRizI+3NerXrx+tWrXSLjAVVppUlGqgFStWsGXLFiZNmkROTk6jdH15eF7Lt7XicDi48MIL2bBhA4cOHWq0WJTypUlFqQZ68MEHiY+Pp6CggOLi4kZNKp5utoqKikrbhwwZgtvt5o033mi0WJTypUlFqQbIzc1l3bp1ZGZmEhVl/TcKR1KpOq7SuXNnzjjjDF555ZVKXWNKNRZNKko1wMyZM3E6nZxxxhkcPWrdty4nJ4e9e/eyd+/ekL2u5/j79u3D7XZTUFBQ7fWGDBnCzp07WblyZcjiUKommlSUqifPUvNpaWl07tyZqKiosLQKjDF+Jwacf/75xMfH8+abbzZ6TEppUlGqnpYvX87WrVsZOnQoYF034na7Gz2OmpJKfHw8V199NXPnzq025qJUqIU0qYjIcBHZLiI7ReRhP/tFRP5m798gIgPqqisiY0Rks4i4RSS9yvGm2uW3i8iVoTw31XJlZWXRunVr0tOtj5+IhK2l4nn9qm6++WaOHTvGkiVLGjss1cKFLKmIiAP4BzAC6AOME5E+VYqNAHrZj0nAiwHU3QRcB3xR5fX6AGOBc4DhwAv2cZQKmtzcXN5++23Gjx9PXFyc9ws9XEnFGOOdKOBrxIgRtG3bljlz5jR6XKplC2VLJQPYaYzZbYwpB94CRlcpMxqYaSyrgTYi0qW2usaYrcaY7X5ebzTwljGmzL798U77OEoFzaxZsygrK+POO+8E8I6nhGumVU1dYLGxsdxwww289957eh971ahCmVROBfb7PD9gbwukTCB1G/J6iMgkEVknIus8s3aUqktWVhYvvfQSTz31FKmpqd5b+Iar68vDk1T8xXDzzTdTVFTEhx9+GIbIVEsVyqTib72Kqp/8msoEUrchr4cxJssYk26MSe/YsWMdh1TqR7t37+bgwYPeAfravtAbi9vtRkT8DsgPGzaMU089VbvAVKMKZVI5AHT3ed4NOBhgmUDqNuT1lGqwZcuWERcX5x2gdzqdYU8qntf2l1SioqIYN24cixcvDurtjZWqTSiTylqgl4ikiUgs1iD6giplFgAT7Flgg4A8Y0xOgHWrWgCMFZE4EUnDGvz/MpgnpFqukpIS1q1bR0ZGBvHx8QCUl5cDhGU6sS9jjDeWqm6++WacTifvvPNOI0elWqqQJRVjjBOYAnwMbAXmGWM2i8hkEZlsF1sE7MYaVH8ZuKe2ugAicq2IHAAuBBaKyMd2nc3APGAL8BFwrzFGbyyhgmL16tVUVFR4u77ASipNYSkUt9tdYyznnXceZ599NrNnzw5DZKolkqbwnyJc0tPTzbp168IdhmriXC4XXbp0ITk5mYcf/vFyq8OHD1NRURH2m2JFRUURHR1Nx44d/d7PZeHChSxYsIC9e/dy2mmnhSFC1dyISLYxJt3fPr2iXqk6LFiwgKNHj3L55Zd7tzmdTtxud5NpqQA13p8+I8OaWa/LtqjGoElFqTr89a9/pUOHDpx//vnebZ4xjKaQVMC6l0pN4yodO3akZ8+ezJo1q5GjUi2RJhWlarF69WpWrFjBZZddVunK9YqKirDP/PIVGxtLWVlZjfFkZmayadMmNmzY0MiRqZZGk4pStfjzn/9MmzZtuOiiiyptLysrIzY2NkxRVRcXF4cxpsYFJNPT04mOjtbWigo5TSpK1eDzzz9n4cKFTJ061TuNGKyBe5fL1aSSiieWmsZVkpOTGT58OHPmzAn7FGjVvGlSUcoPYwy/+93v6NatG/fdd1+lfZ4v7ri4uHCE5pfD4SAmJqbGpAIwfvx4vv/+ez7//PNGjEy1NJpUlPJj/vz5fPnll/zpT38iISGh0r7y8nLvNN6mJD4+vtYpzkePHiUuLo4//vGPZGVlNXJ0qqXQpKJUFXl5eTzwwAP07duXCRMmVNpnjPGOp/hbHTicPF10paWlfvfHxsYyYMAAsrOz9eZdKmSa1p9aSjUBv/nNbzh48CDz58/H4ah8Sx6Xy4Xb7W5SXV+A9z71MTExnDhxgmPHjgHQo0ePSuUyMzNZtWqVzgJTIaNJRSkfixcv5p///CfDhw9n/fr1rF+/vtJ+TyugKQ3S+3K73X5v2uXRu3dvUlJSWLNmTSNGpVoS7f5SynbixAnuvPNOzjnnHEaNGuW3TFlZGdHR0U1uPMXDsxR+TYklKiqKgQMHsmnTJnJzcxs5OtUSaFJRyvbAAw9w6NAhXn/9dWJiYqrt9yzc2NS6vnzVdothj8zMTFwuF2+//XYjRqZaCk0qSmEtuvj666/z8MMPe++X4mvv3r0cOHAAsFo0e/fu9Y5jNDWe1kpNunfvTpcuXXTlYhUSmlRUi5eXl8f48ePp2rUrXbp0qXG6bbjvRx8oT1KpKbGICBkZGSxbtow9e/Y0bnCq2dOkolq8qVOnkpeXx6233uq328sjKioqIq5GD6QLzLNysbZWVLBpUlEt2ooVK3jxxRe59NJLSU1NrbGc5y//SEgq8OMssJpaVR06dGDo0KHMmjWrybe8VGTRpKJarIqKCu666y569OjB1VdfXWvZSOn68vB0gdV0ISTAhAkT2LZtG3qjOhVMmlRUi5OVlUVWVhYTJ05k8+bNjBw5stKCkVV5upIipZUCP3aB1ZZUxowZQ1xcHDNnzmzEyFRzp0lFtUglJSV8+OGHnHnmmfTv37/WsmVlZRHV9eXhdrspLS2tMe6UlBRGjx7Nm2++WeMNvpSqL00qqkX6+OOPKSgo4Prrr69zDa/i4uKI6vry8CSTkpISv/uzsrLo1KkTx48f54EHHmjM0FQzpklFtTh5eXksWbKEjIyMWgfnwVrrq6ysLOJaKWB1gUVHR9eYVAD69OlDq1atWL16dSNGppozTSqqxfnoo49wuVx1Ds7Dj3/l17ScfFOXmJhIRUVFjasSOxwOMjIy2LBhgy7booJCk4pqUXJycli2bBmZmZl07Nix1rLGGIqLi2u9dqWp89wLpri4uMYygwYNwul0Mm/evMYKSzVjmlRUizJ9+nRcLhcjR46ss2x5eTkul4vExMRGiCw0oqKiSEhIoKSkpMYuvO7du9O1a1edBaaCQpOKajEOHz7MjBkzyMjIoFOnTnWWLyoq8n4pR7KkpCRvq8sfEWHQoEGsWrWKnTt3NnJ0qrnRpKJajOnTp1NWVhZQK8XpdFJWVkZCQkKTu8NjfcXExBAbG+udxeZPRkYGIsK//vWvRo5ONTeaVFSLcOTIEV544QVuvvlmOnfuXGd5z1/1SUlJoQ4tpDyrKRcVFeFyubwrLVfVtm1bfvrTnzJz5syInOmmmo6QJhURGS4i20Vkp4g87Ge/iMjf7P0bRGRAXXVFpJ2I/EdEdtj/trW3p4pIiYistx8zQnluKrI8/fTTlJWV8Yc//KHOsnv37qWwsBC3282BAwea7BL39WGMwe1243A4amyt3HLLLezZs4cVK1Y0cnSqOQlZUhERB/APYATQBxgnIn2qFBsB9LIfk4AXA6j7MPCpMaYX8Kn93GOXMeY8+zE5NGemIs3Ro0f5xz/+wdixY+ndu3ed5aOiohCRiJ1GXJO61gO79tprSUpK0i4wdVJCeU/UDGCnMWY3gIi8BYwGtviUGQ3MNNafTqtFpI2IdAFSa6k7GrjYrv8G8BnwUAjPQ0W48ePHU1JSQu/evWu8V4qH7zpfkXYFfV0851RYWEh8fHy1saI5c+Zw7rnnMmvWLPr160dsbCyTJk0KU7QqUoWy++tUYL/P8wP2tkDK1Fa3szEmB8D+13caT5qIfC0in4vIUH9BicgkEVknIuuOHj1a33NSEebw4cMsXbqUjIwMTjnllDrLl5eXR9zikfXhcrlwOp01tlYGDRpESUkJGzZsaOTIVHMRyqTib8pM1T/9aioTSN2qcoDTjDHnA78G5ohI62oHMSbLGJNujEmv6+I3FfmeeuopXC4XV111VUDlCwsLveMPzZFnXMVznlX17t2bNm3asGrVqjBEp5qDUCaVA0B3n+fdgIMBlqmt7mG7iwz73yMAxpgyY8xx++dsYBdwZlDOREWkgwcP8uKLL5KZmRnQjK/y8nLvBY/NWXJyco2tlaioKAYNGsTmzZv54YcfwhCdinShTCprgV4ikiYiscBYYEGVMguACfYssEFAnt2lVVvdBcCt9s+3Ah8AiEhHe4AfEemJNfi/O3Snp5q6J554AqfTWa9WSiQucV9fCQkJREdHU1BQ4Le1MmTIEIwxrFy5MgzRqUgXsoF6Y4xTRKYAHwMO4FVjzGYRmWzvnwEsAkYCO4Fi4Be11bUP/SQwT0RuB/YBY+ztw4A/iYgTcAGTjTG6Ql4LtX//frKysvjFL35R5xpfYN0FsqysjOTkZMrKyhohwvAREVq1asUPP/xAcXFxtWtxOnbsyNlnn82KFStwuVw4HI4wRaoikTS3GS71kZ6ebvRWqs3T7bffzr/+9S927NjBxx9/XGf5EydOUFJSQqdOnWq8QLC56NGjB8YYcnNzcTqddOzYkaioyp0W2dnZZGVlsXjxYoYPHx6mSFVTJSLZxph0f/v0inrV7GzcuJHXXnuNKVOm0KNHjzrLO51OSkpKSExMbDF/lXtaK263m6Kiomr7+/fvT6tWrXj55ZfDEJ2KZJpUVLOSlZXFuHHjSEhIoHv37nVelwLWWApYA9gtSWxsLPHx8RQWFlabnBAdHc2FF17IBx98wL59+8IUoYpEmlRUs7J161Y2b97MiBEjAlq3q6KigpKSEpKSklpMK8WzHtjevXvJz8/HGENOTk61chdffDEAzz//fCNHqCKZJhXVbLjdbubPn0/79u255JJL6iy/d+9ejhw5gjGGH374wftF29J4BuOrTlBo3749N9xwA1lZWRQUFIQpOhVpNKmoZmP27Nns37+fa665JqC7NUZFRREVFdXsr0upi2f5lry8vGpTjB944AHy8vJ47bXXwhSdijSaVFSzkJeXx+9+9ztSU1NJT/c7KaUSp9OJw+HA7XY3++tSAuF0OnG5XN7xJY/MzEwuuuginn32WZxOZ5iiU5FEk4pqFh599FEOHz7MuHHjqk2PrcrtdnuvFtcvSosxxjtoX/U9efDBB/nuu++YNWtWmKJTkUSTiop469ev5+9//zuTJ08mNTW11rLGGE6cOIHT6dSEUkXr1q0RkWrdYKNHjyY9PZ1p06Y1+wtD1cnTpKIi2osvvsg111xDUlISvXr1qrWsy+Xi+PHjlJWVkZKS0uyWtj9ZDoeDVq1aUV5eXmldMBHh8ccfZ+/evbzyyithjFBFAr2iXq+oj2ijRo1i4cKF3HXXXQwYMKDGcqWlpZw4cQJjDG3atCEhIaFFzvQKRHR0NCJCRUUF8OMV+M888wx5eXns3LmzxV3ToyrTK+pVs7RmzRoWL17MoEGDakwonlsC//DDD7hcLsrLyzly5IgmlFp4ZsP5XrcjIlx77bUcPnyYadOmhSkyFQk0qaiIdPToUcaNG0dKSgpjx471W6aiooJjx455pw3rGEpgjDHea1d8Jz307NmTu+66i//7v//jq6++CmOEqinTpKIiTllZGddeey05OTncddddJCQkVNrvuWXusWPHMMZ4p8uqwHmmWjscDm83GMCTTz5Jp06duOOOOzRJK780qaiI4nK5uO2221ixYgVvvPEGaWlplfbv3buXgwcPUlBQgMvlorS0VAfkG8iTNDxdhwBt2rTh73//O19//bV2gym/NKmoiFFRUcH48eOZM2cOf/nLX7jxxhsr7S8vLycmJgYR0dZJkDidTtxuN8ePH8fpdJKVlUVubi6DBw/m8ccfZ/HixeEOUTUxmlRURCgoKOD666/nrbfe4qmnnmLq1KmV9hcXF3P8+HHgxy9CdfKMMbRr1w63282xY8coKirCGMPYsWPp1q0b48ePZ8+ePeEOUzUhmlRUk7d161YyMjJYuHAh48aNo02bNmRlZZGVlYUxhvz8fPLy8oiLi6OiokK7u4IsNjaWjh07EhMTQ35+PkeOHKGsrIw77rgDl8vFyJEjyc3Vm6wqiyYV1aTNmzePgQMHkpuby5IlS7zLscOPy60UFRWRmJhI27ZtwxdoM7Z3714OHDhAYWEhFRUVOJ1O7xphv/vd79i1axfXXHNNpQsmVculSUU1SQUFBQwZMoSbbrqJzp078+tf/5odO3Z49+/du5dDhw5RWlqK0+nkxIkTejOpRuCZTVdeXo7L5cLlcjF69GiWLVvGZZddpsu4KE0qqulZvnw5/fv3Z+XKlQwfPpzf/OY3lVohJSUl3qXtdfwkfDyz68466yxGjRrFypUrueCCCzSxtHCaVFSTUVpaytSpUxk2bBgiwm9/+1uuvfZaoqOjAau7Ky8vz7vcio6fNA0ul4v+/fszatQoNm/ezODBg72rQKuWJzrcASgF8PHHHzNhwgSOHDnCkCFDGDNmDPHx8YDV5VJSUkJBQQFut5ukpCT90mpijDH069ePhIQE5s+fT79+/Vi8eDF9+/YNd2iqkWlLRYXV7t27uf766xk+fDgiwv33388tt9xCfHw8xhj2799PTk4OeXl5OJ1OKioqNKE0YZdddhnPP/88R44c4cILL+SJJ57g8OHD4Q5LNSJtqaiw2L59O3/5y1+YPXs2sbGxPP7446SkpBAdHU1FRQUlJSWUlJQQHR2N2+3G6XRqV1cEEBHuvvtuLrjgAq6//np+//vf88knnzBmzBjS09M555xzSEpKCneYKoR06Xtd+r7RuFwu/vOf/5CVlcUHH3xAXFwcd911F3feeSdut5uPPvqIsrIy7/IgcXFxFBYWajKJUOXl5Xz00Uds2rSJzp07M2LECNLS0ujQoQNdu3alQ4cODBkyJNxhqgaobel7TSqaVELK7XaTnZ3No48+ypo1a8jNzSUpKYmMjAwuvPBC4uLiKiWN2NhY4uPjSUhIICoqSpeobwY6derEU089RU5ODn369OGiiy6ia9euREVF0bZtW3r06EGnTp1o37497du3p02bNpWW3VdNjyaVGmhSCb6ioiI2btzImjVrWL58OUuXLuX48eOICGlpaVxwwQX07t0bh8OBMabaQzVPFRUVfP3116xatYqioiI6depEZmYmaWlptGvXrlJZT7LxJJiUlJRKj+TkZEQkTGeiIIxJRUSGA88BDuAVY8yTVfaLvX8kUAxMNMZ8VVtdEWkHzAVSgT3AjcaYH+x9U4HbARfwS2PMx7XFp0mlsrKyMg4fPkxOTg779+/n0KFD5ObmUlRURElJCeXl5ZSVlVFaWkpxcbF3eRTPNN+CgoJKy6QnJiaSmppKv3796NatGwkJCbjdbk0eLZjT6WT79u1s2rSJ7777DrfbjYjQo0cPevToQZcuXUhNTSUhIcH7h0fVhUFFhISEBBITE2ndujUDBgyga9eupKSkVLr/C1gtZc91M3FxcdX2q4YJS1IREQfwLXA5cABYC4wzxmzxKTMSuA8rqWQCzxljMmurKyL/C+QaY54UkYeBtsaYh0SkD/AmkAF0BZYAZxpjalyqNlhJxfNXttvtxuVyee9F4VnSwvOIiooiOjq60sPhcHgfnr++PP+RPMfzffhu8/zse4WzbwxOp5OysjJKSkooLS31Dn6XlpZSWFjIoUOHOHLkCIcPH+bw4cPk5uZSWFjoXTben7i4OOLi4oiPj6/2s6fbqnPnzqSmpmryULUqLy9n//797N+/n5iYGHbt2sX3339f6XPjcDjo2LEjnTp1om3btiQnJ5OYmEh8fDyxsbEYY4iOjiY2NpbY2FiSk5OJjo6u1Pr1bdXExcV5k5Hn0apVK+Li4nA4HJX+T4qI9xEVFVXpeU2PQMoFeqymrLakEsrZXxnATmPMbjuIt4DRwBafMqOBmcb6FK0WkTYi0gWrFVJT3dHAxXb9N4DPgIfs7W8ZY8qA70Rkpx3DqmCf2MGDB3n99de9ySNYX55RUVF1dgM98cQTuFwubxILxmsnJCSQlJREcnIy3bp1IykpiVatWnm3tWrVisTERO9fev4+8FXj1oSi6hIbG8vpp5/O6aefDsDgwYOpqKio1AL2PHJzc9m/fz/FxcX1vmLf94vakyyuvPJKLr30UvLz85vsigyhTjR9+vThmmuuCeoxIbRJ5VRgv8/zA1itkbrKnFpH3c7GmBwAY0yOiHTyOdZqP8eqREQmAZPsp4Uisj3QE2qgDsCxEL/GSfG0YI4dqxRmk4+7Bhp342rycfv+weNpgS9YsKDDggULmnTcfjSl97pHTTtCmVT8pdWqf77WVCaQug15PYwxWUBWHccKGhFZV1MzsSnTuBuXxt24IjHuSIk5lKNWB4DuPs+7AQcDLFNb3cN2Fxn2v0fq8XpKKaVCKJRJZS3QS0TSRCQWGAssqFJmATBBLIOAPLtrq7a6C4Bb7Z9vBT7w2T5WROJEJA3oBXwZqpNTSilVXci6v4wxThGZAnyMNS34VWPMZhGZbO+fASzCmvm1E2tK8S9qq2sf+klgnojcDuwDxth1NovIPKzBfCdwb20zvxpRo3W1BZnG3bg07sYViXFHRMwt+uJHpZRSwaVXAimllAoaTSpKKaWCRpPKSRKRMSKyWUTcIpLusz1VREpEZL39mOGz7wIR2SgiO0Xkb/ZyNdiTDOba29eISGpjxmzvm2q//nYRubKpxOznHKaJyPc+7+/Ihp5DOInIcDvOnWKtENGkiMge+z1bLyLr7G3tROQ/IrLD/retT3m/730jxPmqiBwRkU0+2+odZ2N/RmqIO7I/2/4W9dNH4A/gbKA31pX96T7bU4FNNdT5ErgQ69qaxcAIe/s9wAz757HA3EaOuQ/wDRAHpAG7AEdTiNnPOUwDfutne73PIYyfHYcdX08g1o67T7g/01Vi3AN0qLLtf4GH7Z8fBp6q671vhDiHAQN8/881JM7G/ozUEHdEf7a1pXKSjDFbjTEBX5VvX1vT2hizylifhpnANfbu0VhLzwC8A1wWir84aonZu9SNMeY7rFl5GU0h5npoyDmEi3cpI2NMOeBZjqip8/2dv0Hlz0K1974xAjLGfAHknkyc4fiM1BB3TZpM3LXRpBJaaSLytYh8LiJD7W2nYl2o6eG7nIx3eRpjjBPIA9o3VrDUvmxOU4x5iohssLsQPF0bDTmHcKkp1qbEAJ+ISLZYSxxBlaWSAN+lkprS+dQ3zqb0GYnYz7beTjgAIrIEOMXPrt8bYz7wsx0gBzjNGHNcRC4A3heRc6h9OZmGLE/jVwNjbsiyOUGLuVowtZwD8CLwmP1ajwF/BW6rJZ6QxXkSmmJMVQ02xhwUa429/4jItlrKRsL5QNP/jET0Z1uTSgCMMT9tQJ0yoMz+OVtEdgFnYv0V0c2nqO9yMp6lZg6ISDSQQuBN45OOmdqXzQl5zFUFeg4i8jLw7yrxVI21tnMIlya/tJAx5qD97xEReQ+rO+uwiHQx1oKuTXmppPrG2SQ+I8aYw56fI/Gzrd1fISIiHcW6Lwwi0hNr2ZjddjO8QEQG2WMPE6i81IxnCZobgP/afaSNxe9SN00xZvtLwuNawDN7piHnEC6BLGUUNiKSJCKtPD8DV2C9z5GyVFK94mwqn5GI/2yHa4ZAc3lg/dIPYLVKDgMf29uvBzZjzdb4CviZT510rA/KLuB5flzZIB54G2sA7kugZ2PGbO/7vR3XdnxmkIQ7Zj/n8C9gI7AB6z9bl4aeQ5g/PyOxbki3C6trMuyfaZ/Yetqf32/sz/Lv7e3tgU+BHfa/7ep67xsh1jexupwr7M/27Q2Js7E/IzXEHdGfbV2mRSmlVNBo95dSSqmg0aSilFIqaDSpKKWUChpNKkoppYJGk4pSSqmg0aSiWhQRcdkrv24SkQ9FpE0d5c+rskrs1RKk1YRF5HUR+U5EvhGRb0Vkpoic6rN/kSc+EfmliGwVkdn2dQpL7PO4KRixKBUsmlRUS1NijDnPGNMX68r/e+sofx7WtSQAGGMWGGOeDGI8Dxpj+mOtGv01sNS+EBJjzEhjzAm73D3ASGPMz4HzgRj7POYG8iL2agdKhZwmFdWSrcJeeE9EMkRkpb0A6EoR6W1/uf8JuMnTKhCRiSLyvF3ndfveFStFZLeI3GBvjxKRF8S6Z82/7RbHDbUFYiz/BxwCRtjH2SMiHcS6F09PYIGIPATMAs6zYzpdrHtpfG4v+vix54psEflMRP4iIp8D99dR7ikR+dJuMQ21tztE5Gmx7tOxQUTus7f7PY5SoElFtVD2EjqX8eOyKNuAYcaY84FHgL8Yazn6R7DuEVNTq6ALMAQYBXhaMNdh3U/nXOAOrPtcBOor4CzfDcaYyVhrOV1ijHnKPuYyY8x5wD7g78ANxpgLgFeBx32qtzHG/AT4Wx3loo0xGcCvgEftbZOw7ttxvjGmHzBbRGLqOI5q4bRJrFqaBBFZj/Wlnw38x96eArwhIr2wVniNCfB47xtj3MAWEelsbxsCvG1vPyQiS+sRX33vRdMb6Iu1gjBYN/7K8dk/N8By79r/ZmO9NwA/xboBmxPAGJMrIn3rOI5q4TSpqJamxBhznoikYK3+ei/WX/GPAUuNMdeKdUvkzwI8XpnPz1Ll34Y4H2udqkAJsNkYU1NrqCjAcp7zcPHj94JQfQn1uo6jWjjt/lItkjEmD/gl8Fu7SycF+N7ePdGnaAHQqp6HXw5cb4+tdAYurquCWH6J1Z32UT1eazvQUUQutI8TI9Z9expaztcnwGTPIL+ItGvgcVQLoklFtVjGmK+xVuAdi3U/8ydEZAVWl47HUqBPPafvzsdacXYT8BKwBuuOmP5MF5FvsFYqHog1blJej3Mox7rlwFP2cdYDFzW0XBWvYI3ZbLDr3NzA46gWRFcpVioERCTZGFMoIu2xbgkw2BhzKNxxKRVqOqaiVGj8275wMRZ4TBOKaim0paKUUipodExFKaVU0GhSUUopFTSaVJRSSgWNJhWllFJBo0lFKaVU0Px/13PVbvN9b8sAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df_train.selectExpr('black_rating - white_rating').toPandas(), label='black white rating gap', color='gray')\n",
    "sns.distplot(testPredAndLabels.selectExpr(f'prediction - {target}').toPandas(), label='test prediction error', color='k')\n",
    "plt.title('Prediction Error vs Rating Gap')\n",
    "plt.xlabel('Rating Difference')\n",
    "plt.legend()\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "The RMSE score is very close to the standard deviation of the difference between `black_rating` and `white_rating`. If the model is given the `white_rating` feature, the RMSE would see a decrease of about 40, further improving the model prediction."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### *Linear Regression*"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "outputs": [],
   "source": [
    "lr = LinearRegression(labelCol=target, featuresCol=\"features\")\n",
    "\n",
    "lrParamGrid = (ParamGridBuilder()\n",
    "               .addGrid(lr.regParam, [0.01, 0.1, 0.2, 0.8, 2.0])\n",
    "               .addGrid(lr.elasticNetParam, [0.0, 0.1, 0.2, 0.3, 0.4])\n",
    "               .addGrid(lr.fitIntercept, [True, False])\n",
    "               .addGrid(lr.maxIter, [10, 30, 50, 70, 100])\n",
    "               .addGrid(lr.solver, ['auto', 'normal', 'l-bfgs'])\n",
    "               .build())\n",
    "\n",
    "lrEvaluator = RegressionEvaluator(labelCol=target, metricName='rmse')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "From [Medium Article by Dhiraj Rai](https://dhiraj-p-rai.medium.com/logistic-regression-in-spark-ml-8a95b5f5434c)\n",
    "- `regParam` the regularization parameter (>= 0). (default: 0.0)\n",
    "- `elasticNetParam` the ElasticNet mixing parameter, in range [0, 1]. For alpha = 0, the penalty is an L2 penalty. For alpha = 1, it is an L1 penalty. (default: 0.0)\n",
    "- `fitIntercept` whether to fit an intercept term. (default: True)\n",
    "- `maxIter` maximum iteration number.\n",
    "- `standardization` whether to standardize the training features before fitting the model. The coefficients of models will always be returned on the original scale, so it will be transparent for users. (default: True)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Hyperparameter Tuning (Grid Search)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "outputs": [],
   "source": [
    "if not skipGridSearch:\n",
    "    lrCv = CrossValidator(estimator = lr,\n",
    "                          estimatorParamMaps = lrParamGrid,\n",
    "                          evaluator = lrEvaluator,\n",
    "                          numFolds = 5)\n",
    "\n",
    "    lrCvModel = lrCv.fit(reg_train_df)\n",
    "\n",
    "    # Best Parameters found\n",
    "    bestParamMap = lrCvModel.bestModel.extractParamMap()\n",
    "    print( ','.join([''.join((param.name, '=', repr(value))) for param, value in bestParamMap.items()]) )"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Train"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "outputs": [],
   "source": [
    "lrBest = LinearRegression(aggregationDepth=2,elasticNetParam=0.0,epsilon=1.35,featuresCol='features',fitIntercept=True,labelCol='black_rating',loss='squaredError',maxBlockSizeInMB=0.0,maxIter=30,predictionCol='prediction',regParam=0.1,solver='auto',standardization=True,tol=1e-06)\n",
    "# assert lrBest.extractParamMap() == bestParamMap, 'Check if the input parameters are the best found. '\n",
    "bestModel = lrBest.fit(reg_train_df)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Model Evaluation"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train rmse 250.66053890029065\n",
      "test rmse 258.24699917173854\n"
     ]
    }
   ],
   "source": [
    "trainPredAndLabels = bestModel.transform(reg_train_df).select('prediction', target)\n",
    "testPredAndLabels = bestModel.transform(reg_test_df).select('prediction', target)\n",
    "print('train rmse', lrEvaluator.evaluate(trainPredAndLabels))\n",
    "print('test rmse', lrEvaluator.evaluate(testPredAndLabels))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\tools\\miniconda3\\envs\\bigdata\\lib\\site-packages\\seaborn\\distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).\n",
      "  warnings.warn(msg, FutureWarning)\n",
      "C:\\tools\\miniconda3\\envs\\bigdata\\lib\\site-packages\\seaborn\\distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).\n",
      "  warnings.warn(msg, FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEWCAYAAACufwpNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABJmUlEQVR4nO3deXxU1d348c83kz2QBAgoKKsii+yGrQG0WkFUwH1faB9FRFu1T31c2sfaWq1bfaQ/q4hWBUXEBQUVK+IGCGKIIjuyCBJ2AtlIMsnMnN8f9844SSbJJMxkMsn3/XrNKzP3nnPv904m880599xzxRiDUkopFQoxkQ5AKaVU86FJRSmlVMhoUlFKKRUymlSUUkqFjCYVpZRSIaNJRSmlVMhoUlFNioi8IiJ/s5+PFpEtDdzODBH539BGp2ojIh+JyI2RjkNFliYVVW8islNESkWkWEQOiMjLItIq1PsxxiwzxvQKIp7JIrK8St2pxpiHQh2TiDwoIhX2sXsf+aHeT2MQESMix+xj2CMiT4mII8i6D4rIa/7LjDHjjTGzwhRrazu+nXbMP4nI2yIyLBz7Uw2nSUU11ARjTCtgCDAU+FPVAiIS2+hRNY55xphWfo/0QIUCHX9935NGeA8H2r/HM4Ergd+EeX/1JiIJwGdAf+BCIBXoA7wBnB/B0FQAmlTUcTHG7AE+AvqB77/f20RkK7DVXnahiKwRkXwRWSEiA7z1RWSwiHwrIkUiMg9I9Ft3lojk+r3uLCLzReSQiOSJyDMi0geYAYz0bzX4d6PZr28WkW0ickREFopIJ791RkSmishWETkqIv8SEWnI+1H1+L3HICL3iMh+4GURSRCRp0Vkr/142v7iJFD5KttPsN/Hfn7L2tstxw4ikiEiH9hljojIMhGp8+/cGLMN+AoY5Lfd6SKyW0QKRSRHREbby88D7geutN/z7+3lX4jITfbzySKyXESetN/TH0VkvN+2u4vIUvv3vsR+zyu1fPxcD5wMXGSMWW+McRtjjhlj3jbGPFhXvPa6B+2WzTx7n9+KyMC63hdVf5pU1HERkc5Y/y1+57f4ImA40FdEhgAvAbcA7YDngYX2l2M88B7wKtAWeAu4tIb9OIAPgF1AN+Ak4A1jzCZgKrCyplaDiJwN/B24Auhob+ONKsUuxGpxDbTLjQv+XajmIuzjt1+fiHV8XYEpwB+BEVhf4AOBYVRu6VUt72OMcQLzgav9Fl8BfGmMOQj8N5ALtAdOwPryr3MuJhHpDYwGtvktzrZjbAu8DrwlIonGmP8Aj/Bzi62mL+fhwBYgA3gc+Ldfsn4d+AbrM/EgVuKoya+Aj40xx+o4jIDx+q2fhPUZ865/T0Ti6timqi9jjD70Ua8HsBMoBvKxvqCfBZLsdQY426/sc8BDVepvwepuGQPsBcRv3Qrgb/bzs4Bc+/lI4BAQGyCeycDyKste8dvOv4HH/da1AiqAbn4xj/Jb/yZwbw3H/iBQbh+79/G53/qqx3+WXT7Rb9l24Hy/1+OAnTWVDxDDr4Adfq+/Am6wn/8VWACcGsTv0QCFwDH7+VwgoZbyR7G6y7zvw2tV1n8B3OT3O9nmty7Z3seJQBfABST7rX+t6vb81i0BHvV7Pch+3wuBLfWI92u/dTHAPmB0pP+emttDWyqqoS4yxqQbY7oaY6YZY0r91u32e94V+G+7Oybf7p7qDHSyH3uM/Vdu21XD/joDu4wxrgbE2sl/u8aYYiAPq7Xjtd/veQlW4qnJm/axex+/rLJ+d5XXh4wxZTXFYz/vVEv5qj4DkkRkuIh0xfqSfdde9wRWa2OxiOwQkXtr2Q5Y58RaYZ1PGQ6keFeIyH+LyCYRKbB/b2lYrY5g+d5TY0yJ/bQV1rEe8VsG1d8zf3lYLUzvttYYq0V6CZBQj3h3+23Dg9Wi83/fVQhoUlHh4J8kdgMPV/kSTjbGzMX6T/GkKucvutSwzd1AFwl84rqu7p29WMkNABFJwep22VPXgTRQ1Xiqvq4UD9Yx762lfOWNWV+Ib2J1gV0DfGCMKbLXFRlj/tsY0wOYAPxeRM6pY3vGGPMmsBJ4AKzh3MA9WF1rbewv8QLA+7s6nunN9wFtRSTZb1nnWsp/Coy1f28BBRFvpX3Y55lOpvL7rkJAk4oKtxeAqfZ/1SIiKSJygYi0xvoScwG/E5FYEbkE6/xCIN9gfRk9am8jUUSy7HUHgJPtczSBvA78WkQG2SfEHwFWGWN2hugY62su8Cf7BHsG1hd5TSepa/I6VuviWvs54BsUcaqdqAsBt/0IxqPAFBE5EWiN9bs5BMSKyANYo668DgDdghkEUJUxZhewGnhQROJFZCRWAqzJbKzf/bsi0k9EHPa5kky/MnXFC3CGiFxi/2NyJ+AEvq5v/Kp2mlRUWBljVgM3A89g9XFvw+pvxxhTjtWFMdledyXWSehA23FjffGcCvyE1XVxpb36M2ADsF9EDgeo+ynwv8A7WF9OpwBXHcdheUc9+T861KP+37C+VNcC64Bv7WVBM8aswjoX0glr9J1XT6xzEMVYSftZY8wXQW5zHfAlcDfwsb3dH7C658qo3EX1lv0zT0S+rU/stmuxzpPlYR37PKwv+UBxlQG/BDYCH2KfS8EaWHGFXayueME613Ql1mfteuASY0xFA2JXtZDK3dlKKdX4xBpOvtkY8+cwbf9BrMEL14Vj++pn2lJRSjU6ERkqIqeISIx93cskrOHlKso11yuelVJN24lYXZ3tsLoybzXGfFd7FRUNtPtLKaVUyIS1+0tEzhORLWJNj1FtvLw9Guif9vq19tXXtdYVkYfssmtEZLFUnm7jPrv8FhE5niuilVJKNUDYWir2tBo/AOdiNW+zgauNMRv9ypwP/BZrmo/hwHRjzPDa6opIqjGm0K7/O6CvMWaqiPTFGqo5DGtEzBLgNHvUUEAZGRmmW7duIT5ypZRq3nJycg4bY9oHWhfOcyrDsKZp2AEgIm9gnYzb6FdmEjDbvqL6axFJF5GOWHM7BazrTSi2FH6+CGsS1lxQTuBHEdlmx7CypgC7devG6tWrj/9IlVKqBRGRmma+CGv310lUHieeS+VpMWorU2tdEXlYRHZjjXV/oB77Q0SmiMhqEVl96NCheh2QUkqp2oUzqQSaOrxqX1tNZWqta4z5ozGmMzAHuL0e+8MYM9MYk2mMyWzfPmDrTSmlVAOFM6nkUnk+n0Dz7NRUJpi6YE1P4Z0qPdg6SimlwiSc51SygZ4i0h1r4r6rsCa/87cQuN0+ZzIcKDDG7BORQzXVFZGexpitdv2JwGa/bb0uIk9hnajviTVflFItSkVFBbm5uZSV1TbRsVJ1S0xM5OSTTyYuLvjbzoQtqRhjXCJyO9acPA7gJWPMBhGZaq+fASzCGvm1DWu68V/XVtfe9KMi0gvwYM3x493eBhF5E2sggAu4rbaRX0o1V7m5ubRu3Zpu3bohDbuBpVIYY8jLyyM3N5fu3bsHXa9FX/yYmZlpdPSXam42bdpE7969NaGo42aMYfPmzfTp06fSchHJMcZkBqqjc38p1QxpQlGh0JDPkSYVpZRSIaMTSirVzOXk5IR0e2eccUat63fu3MmFF17I+vXrq60766yzePLJJ8nMDNhzUqNXXnmF1atX88wzz9SrXrDbOP/883n9deteZ6+//jrTpk1r8H7qimHs2LF06mTNLnXTTTfx+9//nr59+4Zlf5GgSUWpCKvpS7+uL28VOosWLQKshPjss88eV1Jxu904HI6A61555RX69evnSyovvvhig/fTVGn3l1Iq5FwuFzfeeCMDBgzgsssuo6SkpFqZW2+9lczMTE4//XT+/Oef782VnZ3NL37xCwYOHMiwYcMoKiqqVO/DDz9k5MiRHD5c+Saf/fv3Jz8/H2MM7dq1Y/bs2QBcf/31LFmyBIC9e/dy3nnn0bNnT/7nf/7HV7dbt24cPnyYe++9l+3btzNo0CDuvvtuAJ544gmGDh3KgAEDKsXpr1WrVjzwwAMMHz6clStX8te//pWhQ4fSr18/pkyZgjGGt99+m9WrV3PttdcyaNAgSktLOeuss3xTRbVq1Yo//vGPDBw4kBEjRnDgwAEAtm/fzogRIxg6dCgPPPAArVq1ChjDQw89RO/evTn33HO5+uqrefLJJwF44YUXGDp0KAMHDuTSSy/1/S4mT57M1KlTGT16NKeddhoffPBBwO3WlyYVpVTIbdmyhSlTprB27VpSU1N59tlnq5V5+OGHWb16NWvXruXLL79k7dq1lJeXc+WVVzJ9+nS+//57lixZQlJSkq/Ou+++y6OPPsqiRYvIyMiotL2srCy++uorNmzYQI8ePVi2bBkAX3/9NSNGjABgzZo1zJs3j3Xr1jFv3jx27658x+FHH32UU045hTVr1vDEE0+wePFitm7dyjfffMOaNWvIyclh6dKl1Y7l2LFj9OvXj1WrVjFq1Chuv/12srOzWb9+PaWlpXzwwQdcdtllZGZmMmfOHNasWVPpuLzbGDFiBN9//z1jxozhhRdeAOCOO+7gjjvuIDs729fCqWr16tW88847fPfdd8yfP7/SnIaXXHIJ2dnZfP/99/Tp04d///vfvnU7d+7kyy+/5MMPP2Tq1KkhubZJk4pSKuQ6d+5MVlYWANdddx3Lly+vVubNN99kyJAhDB48mA0bNrBx40a2bNlCx44dGTp0KACpqanExlq99J9//jmPPfYYH374IW3atKm2vdGjR7N06VKWLl3Krbfeyrp169izZw9t27b1/Xd/zjnnkJaWRmJiIn379mXXrhrnRQRg8eLFLF68mMGDBzNkyBA2b97M1q1bq5VzOBxceumlvteff/45w4cPp3///nz22Wds2LChWp2q4uPjufDCCwGr63Pnzp0ArFy5kssvvxyAa66pev24Zfny5UyaNImkpCRat27NhAkTfOvWr1/P6NGj6d+/P3PmzKkUyxVXXEFMTAw9e/akR48ebN68OdDm60WTilIq5KoORa36+scff+TJJ5/k008/Ze3atVxwwQWUlZVhjKlxGGuPHj0oKirihx9+CLh+zJgxLFu2jGXLlnHWWWfRvn173n77bUaPHu0rk5CQ4HvucDhwuVy1Hocxhvvuu481a9awZs0atm3bxn/9139VK5eYmOg7j1JWVsa0adN4++23WbduHTfffHNQLYC4uDjfsQcTW9U4azJ58mSeeeYZ1q1bx5///OdKsdT1e2oITSpKqZD76aefWLnSuuvE3LlzGTVqVKX1hYWFpKSkkJaWxoEDB/joo48A6N27N3v37iU7OxuAoqIi35dr165dmT9/PjfccEPA//w7d+7M4cOH2bp1Kz169GDUqFE8+eSTlZJKXVq3bl3pHM64ceN46aWXKC4uBmDPnj0cPHiw1m14v7QzMjIoLi7m7bffrnH7wRgxYgTvvPMOAG+88UbAMqNGjeL999+nrKyM4uJiPvzwQ9+6oqIiOnbsSEVFBXPmzKlU76233sLj8bB9+3Z27NhBr1696hVbIDr6S6lmLhKjyPr06cOsWbO45ZZb6NmzJ7feemul9QMHDmTw4MGcfvrp9OjRw9dVFh8fz7x58/jtb39LaWkpSUlJvpPsAL169WLOnDlcfvnlvP/++5xyyimVtjt8+HDcbmt2ptGjR3PfffdVS2i1adeuHVlZWfTr14/x48fzxBNPsGnTJkaOHAlYJ9Nfe+01OnToUOM20tPTufnmm+nfvz/dunXzdeXBzyfHk5KSfEm3Lk8//TTXXXcd//jHP7jgggtIS0urVmbo0KFMnDiRgQMH0rVrVzIzM33lHnroIYYPH07Xrl3p379/paTWq1cvzjzzTA4cOMCMGTNITEwMKqba6DQtOk2LirBQDynetGlTtWk1VPQqKSkhKSkJEeGNN95g7ty5LFiwoFq54uJiWrVqRUlJCWPGjGHmzJkMGTIkwBYtkydP5sILL+Syyy6rdf+BPk+1TdOiLRWllGrCcnJyuP322zHGkJ6ezksvvRSw3JQpU9i4cSNlZWXceOONtSaUcNKkopRSTdjo0aP5/vvv6yznnREgWK+88koDI6qdnqhXSikVMppUlIqg/fv3s3LlSgoKCiIdilIhoUlFqQjasGEDeXl5rFy5kry8vEiHo9Rx06SiVATt27eP5ORkHA4H27dvj3Q4Sh03PVGvVIQYY9i7dy8ZGRm4XC4KCwvDsp+ZM2eGdHtTpkypdX1+fv5xTR//9NNPM2XKFJKTkxtUvz5atWpFcXExe/fu5Xe/+12lCxXriss7XX56enrY44wm2lJRKkIKCgooLS0lLS2NlJQUSkpK8Hg8kQ7ruOXn5wecQDJYTz/9dMBZjYNVn+lNvDp16lRrQoHqcS1atKjREkrVYwr2GBvyXhwvbakoFSF79+4FIC0tjdjYWIwxlJSU1Di1ebTwnz7+3HPP5YknnuCJJ57gzTffxOl0cvHFF/OXv/yFY8eOccUVV5Cbm4vb7eZ///d/OXDgAHv37uWXv/wlGRkZfP7555W23a1bN6688krf8tdff51TTz2VyZMn07ZtW7777juGDBnCtGnTuO222zh06BDJycm88MIL9O7dmx9//JFrrrkGl8vFeeed59uu/43F3G4399xzDx9//DEiws033+xrVfrH1a1bN1avXk1GRgZPPfWU7/qRm266iTvvvJOdO3cyfvx4Ro0axYoVKzjppJNYsGBBtdmJDx06xNSpU/npp58AK3llZWXx4IMPsnfvXnbu3ElGRgannXZapdd///vf+c1vfsOhQ4do3749L7/8Ml26dKn2XvzjH/8I56+7Gk0qSkXIvn37iImJITU11bfs2LFjUZ9UHn30UdavX8+aNWsAKk0fb4xh4sSJLF26lEOHDtGpUyffPFUFBQWkpaXx1FNP8fnnn1eb2t4rNTWVb775htmzZ3PnnXf67gPyww8/sGTJEhwOB+eccw4zZsygZ8+erFq1imnTpvHZZ59xxx13cOutt3LDDTfwr3/9K+D2Z86cyY8//sh3331HbGwsR44coW3btjXGlZOTw8svv8yqVaswxjB8+HDOPPNM2rRpw9atW5k7dy4vvPACV1xxBe+88w7XXXddpfp33HEHd911F6NGjeKnn35i3LhxbNq0ybft5cuXk5SUxIMPPljp9YQJE7jhhhu48cYbeemll/jd737He++9V+29aGyaVJSKkH379tGhQwccDgcpKSmANdXGCSecEOHIQst/+niwjnHr1q2MHj2aP/zhD9xzzz1ceOGFQU/8ePXVV/t+3nXXXb7ll19+OQ6Hg+LiYlasWOGbLh7A6XQC8NVXX/kmZ7z++uu55557qm1/yZIlTJ061Tflftu2bWuNZ/ny5Vx88cW+3+Ell1zCsmXLmDhxIt27d2fQoEFA5ensq+5v48aNvteFhYW++bkmTpxYqWXj/3rlypXMnz/fdyz+Nx3zvheRoElFqQjZv38/PXv2BKyJFOPj4zl27FiEowo97/Txt9xyS7V1OTk5LFq0iPvuu4+xY8fywAMP1Lk9/+nZ/Z97v9Q9Hg/p6em+llJt9WuKtz5TwNc2f2LVqfZLS0urlfF4PKxcubJatxj8fEw1vfYX6L2IBD1Rr1QEuFwujh07VulEb0pKim+K9WgW7PTxe/fuJTk5meuuu44//OEPfPvttwHrVzVv3jzfT+/swf5SU1Pp3r07b731FmB96XunOcnKyvJNH191GnivsWPHMmPGDN9J7iNHjtQa15gxY3jvvfcoKSnh2LFjvPvuu/Wabn/s2LE888wzvtc1JcOqfvGLX1Q6lvrMxhxO2lJRKgK8w4fT0tJ8U7W3atWqznt1NERdQ4BDLdjp47dt28bdd99NTEwMcXFxPPfcc754x48fT8eOHaudqAerK2v48OF4PB7mzp0bMIY5c+Zw66238re//Y2KigquuuoqBg4cyPTp07nmmmuYPn16pTs1+rvpppv44YcfGDBgAHFxcdx8883cfvvtNcY1ZMgQJk+ezLBhw3z1Bw8eHLCrK5B//vOf3HbbbQwYMACXy8WYMWOYMWNGUPV+85vf8MQTT/hO1DcFOvW9Tn2vImDnzp3MmjWL66+/nqNHjwKwbds2Nm/ezLhx44iLi9Op7wPwH3GlGkd9p77X7i+lIsDbUvEf+eXtBz+eazSUirSwJhUROU9EtojINhG5N8B6EZF/2uvXisiQuuqKyBMistku/66IpNvLu4lIqYissR91tx+VipBAScV71z3vSCVVnfcaDdV0hS2piIgD+BcwHugLXC0ifasUGw/0tB9TgOeCqPsJ0M8YMwD4AbjPb3vbjTGD7MfU8ByZUsevsLCQxMRE4uPjfcu8I4W89zg/Hi25W1uFTkM+R+FsqQwDthljdhhjyoE3gElVykwCZhvL10C6iHSsra4xZrExxjv3wNfAyWE8BqXCorCwsFIrBX5OKsfbUklMTCQvL08Tizouxhjy8vLqfd/6cI7+OgnY7fc6FxgeRJmTgqwL8Btgnt/r7iLyHVAI/MkYs6xqBRGZgtUqokuXLkEdiFKhFiipOBwO4uLijjupnHzyyeTm5nLo0KHj2o5SiYmJnHxy/f5vD2dSCXT1UNV/nWoqU2ddEfkj4AK8g833AV2MMXkicgbwnoicboypNPWrMWYmMBOs0V91HoVSYVBQUEDHjh2rLU9ISDjupBIXF0f37t2PaxtKNVQ4k0ou0Nnv9cnA3iDLxNdWV0RuBC4EzjF2G98Y4wSc9vMcEdkOnAbomGHVpLhcLkpKSqq1VCA0SUWpSArnOZVsoKeIdBeReOAqYGGVMguBG+xRYCOAAmPMvtrqish5wD3ARGOMb+yliLS3T/AjIj2wTv7vCOPxKdUg3quy09LSqq1LSEgIyYl6pSIlbC0VY4xLRG4HPgYcwEvGmA0iMtVePwNYBJwPbANKgF/XVtfe9DNAAvCJPdfN1/ZIrzHAX0XEBbiBqcaYI+E6PqUayns/em2pqOYorNO0GGMWYSUO/2Uz/J4b4LZg69rLT62h/DvAO8cTr1KNIdA1Kl4JCQm43e6I3FxJqVDQK+qVamS1JRW9AFJFO00qSjWy4uJi31T3VYXqWhWlIkWTilKNrLi4uMa7O4byqnqlIkGTilKNLJikoi0VFa00qSjVyIqKimjdunXAdfHx8YiIJhUVtTSpKNXIamupiIgOK1ZRTe/8qFQjycnJweVyUV5eTkFBATk5OQHL6QWQKpppS0WpRuRtgXjPnQQSHx9PeXl5Y4WkVEhpUlGqEXlbILVNJ56QkKBJRUUtTSpKNaJgWypOp1Pvh6KikiYVpRqRN6nU1VLxeDxUVFQ0VlhKhYwmFaUaUVlZGSJCXFxcjWW8V9ofO3asscJSKmQ0qSjViJxOJwkJCdgzbAfk7RrTpKKikSYVpRqRN6nURlsqKpppUlGqEZWVldV6PgW0paKimyYVpRqRtlRUc6dJRalG4vF4KC8vrzOpOBwOYmNjNamoqKRJRalGEsxwYq/4+HhKSkrCHZJSIadJRalGEsyFj14JCQnaUlFRSZOKUo2kPkklPj5ek4qKSppUlGokwcz75aVJRUUrTSpKNZL6dn+VlJTo/F8q6mhSUaqROJ1O4uLiiImp+88uPj4ej8ej91VRUUeTilKNJJgLH730AkgVrTSpKNVIgrnw0UuTiopWmlSUaiROpzPolopeVa+ilSYVpRqBMaZeLRVNKipahTWpiMh5IrJFRLaJyL0B1ouI/NNev1ZEhtRVV0SeEJHNdvl3RSTdb919dvktIjIunMemVH2Ulpbi8Xg0qahmL2xJRUQcwL+A8UBf4GoR6Vul2Higp/2YAjwXRN1PgH7GmAHAD8B9dp2+wFXA6cB5wLP2dpSKuOLiYiC4a1QAYmJiSEpK0qSiok44WyrDgG3GmB3GmHLgDWBSlTKTgNnG8jWQLiIda6trjFlsjHHZ9b8GTvbb1hvGGKcx5kdgm70dpSLOm1SCbakApKSk6PxfKuqEM6mcBOz2e51rLwumTDB1AX4DfFSP/SEiU0RktYisPnToUBCHodTxKyoqAuqfVLSloqJNOJNKoPulVr08uKYyddYVkT8CLmBOPfaHMWamMSbTGJPZvn37AFWUCr36dn+BJhUVnWLDuO1coLPf65OBvUGWia+trojcCFwInGN+nscimP0pFRFFRUW++6QEKzk5WZOKijrhbKlkAz1FpLuIxGOdRF9YpcxC4AZ7FNgIoMAYs6+2uiJyHnAPMNEYU1JlW1eJSIKIdMc6+f9NGI9PqaAVFhaSlJRUrzopKSm+UWNKRYuwtVSMMS4RuR34GHAALxljNojIVHv9DGARcD7WSfUS4Ne11bU3/QyQAHwiIgBfG2Om2tt+E9iI1S12mzHGHa7jU6o+CgsL69X1BVZSASgpKaFVq1bhCEupkAtn9xfGmEVYicN/2Qy/5wa4Ldi69vJTa9nfw8DDDY1XqXApKioiNTW1XnW8SeXYsWOaVFTU0CvqlQozj8dDUVFRg7q/QC+AVNFFk4pSYVZcXIwxpsHdX5pUVDTRpKJUmBUWFgL1G04MmlRUdNKkolSYeZNKfbu/EhMTERFNKiqqaFJRKswa2lIREZ2qRUUdTSpKhVlhYSGxsbHExcXVu65eVa+ijSYVpcKsqKiI1q1bY19XVS+aVFS00aSiVJgVFhbW+xoVr1atWvkmo1QqGmhSUSrMjjepeIckKxUNNKkoFUbGmAZdTe/VunVr3G43ZWVlIY5MqfDQpKJUGB07dgy3231cLRVAu8BU1NCkolQY5efnA5Cent6g+q1btwZ+vh+LUk2dJhWlwqigoACAtLS0BtXXloqKNkElFRF5R0QuEBFNQkrVQ6haKppUVLQINkk8B1wDbBWRR0WkdxhjUqrZyM/PJzExsV73pvcXHx9PfHy8dn+pqBFUUjHGLDHGXAsMAXZi3SBrhYj8WkTqf5mwUi1EQUFBg7u+cnJyyMnJIS4ujp9++omcnJwQR6dU6AXdnSUi7YDJwE3Ad8B0rCTzSVgiU6oZKCgoaHDXl1dCQgJOpzM0ASkVZsGeU5kPLAOSgQnGmInGmHnGmN8Ceks6pQIwxpCfn9/glopXYmKiXqeiokawtxN+0b69r4+IJBhjnMaYzDDEpVTUKysro7y8/LiTirZUVDQJNqn8jer3i1+J1f2llArAO5w42O6v+fPnB1weExNDbGwsLpcrVKEpFTa1JhURORE4CUgSkcGAd5rVVKyuMKVUADk5Oezfvx+Affv2UVpaetzb1C4wFQ3qaqmMwzo5fzLwlN/yIuD+MMWkVLPgTST1ueOjy+WirKyMpKQkHA4HgG8ySU0qKhrUmlSMMbOAWSJyqTHmnUaKSalmobS0lJiYGOLj42stZ4xh+fLlvP766/z00094PB6SkpIYPHgwo0eP9iUXTSoqGtTV/XWdMeY1oJuI/L7qemPMUwGqKaWwJpNMSUmp9eZc+/fv569//SvffPMNqampDBs2jNTUVH766SdWrFjB7t27ueqqq4iPj9ekoqJCXd1fKfZPHTasVD2VlJSQnFzzqcfFixfzyCOP4PF4uOeeewB8rZLMzEzWr1/PwoULWbRoEZdddllIzssoFW51dX89b//8S+OEo1TzYIyhpKSEjIyMgOtfe+01nn76aU466SQmTpzoSyb++vXrR35+PkuXLqVfv36ccMIJ4Q5bqeMW7MWPj4tIqojEicinInJYRK4Ld3BKRavy8nLcbnfAlsrzzz/P008/Te/evbnuuuto06ZNjdsZOXIkHTp04KOPPtJJJVVUCHaalrHGmELgQiAXOA24u65KInKeiGwRkW0icm+A9SIi/7TXrxWRIXXVFZHLRWSDiHhEJNNveTcRKRWRNfZjRpDHplTIlZSUAFRLKu+88w4vvPACEyZM4KKLLgrYQvHncDg455xzKCws5IsvvmDmzJm+h1JNUbBJxTtp5PnAXGPMkboqiIgD+BcwHugLXC0ifasUGw/0tB9TsGZDrqvueuASYGmA3W43xgyyH1ODPDalQi5QUlm/fj2PP/44WVlZ/OlPfyImJrg/v27dutG5c2eWL19ORUVFWOJVKlSCTSrvi8hmIBP4VETaA3UNRRkGbDPG7DDGlANvAJOqlJkEzDaWr4F0EelYW11jzCZjzJYg41YqIqomldLSUu68805atWrF8OHDWbBgQdDbEhGysrIoKipi9erVYYlXqVAJdur7e4GRQKYxpgI4RvUEUdVJwG6/17n2smDKBFM3kO4i8p2IfCkiowMVEJEpIrJaRFYfOnQoiE0qVX/Hjh0jISHB17318ssvk5+fz4QJE0hMTKz39nr06EG7du34/PPPQx2qUiFVnzs59gGuFJEbgMuAsXWUDzQ43wRZJpi6Ve0DuhhjBgO/B14XkdRqGzFmpjEm0xiT2b59+zo2qVTDlJSUkJJijcj/6aefePXVV+nXrx9dunRp8DaHDh3Krl272LNnT6jCVCrkgh399SrwJDAKGGo/6pqdOBfo7Pf6ZGBvkGWCqVuJPWNynv08B9iONaBAqUbnvUbFGMOTTz5JfHw8Z5999nFts1+/fsTExLBy5coQRalU6AU7S3Em0Nd4JyEKTjbQU0S6A3uAq7BuSexvIXC7iLwBDAcKjDH7RORQEHUrsc/zHDHGuEWkB9bJ/x31iFepkPDO35WcnMzq1atZsWIFd955Z60XQgYjJSWF3r17s2rVKi6++OIQRatUaAXb/bUeOLE+GzbGuIDbgY+BTcCbxpgNIjJVRLwjsxZhffFvA14AptVWF0BELhaRXKxzPB+KyMf2tsYAa0Xke+BtYGowo9SUCrWjR48C1kn6F198kYyMDC6//PLj3q4xhkGDBlFYWMimTZuOe3tKhUOwLZUMYKOIfAP47hZkjJlYWyX7xl6Lqiyb4ffcALcFW9de/i7wboDl7wA66aWKuLy8PAB27NhBTk4Ov/rVr/jwww+Pe7vGGHr27ElKSgorVqw47u0pFQ7BJpUHwxmEUs2JN6nMnTuX5ORkBg8eHLJtiwiZmZl89dVXFBQUHPddJZUKtWCHFH8J7ATi7OfZwLdhjEupqJWXl8fBgwfJzs5mxIgRxMXF1V0pCN5TmmeccQYul4sPPvggJNtVKpSCHf11M9Z5iuftRScB74UpJqWi2pEjR8jOzg55K8WbVLp06UJ6ejpvvfVWyLatVKgEe6L+NiALKAQwxmwFOoQrKKWi2Y8//si3337LhAkTSEhICNl2vUnF4/EwZMgQ/vOf/1BYWBiy7SsVCsEmFac9XQoAIhJL3RcjKtXizJgxg6VLl+J2u0lPTw/LPtxuN2eccQZOp1O7wFSTE2xS+VJE7geSRORc4C3g/fCFpVR0cjqdrF69mlNOOYV27dqFfPuxsbG4XC569OhBp06dtAtMNTnBJpV7gUPAOuAWrKG+fwpXUEpFq2+//Zbi4mIyM+uacKJhHA4HbrebmJgYLr30Ur3Pimpygh395cE6MT/NGHOZMeaFel5dr1SL8PXXX5OWlkaPHj3Csn1vUjHGcMkll+B0Ovnkk0/Csi+lGqLWpGLfROtBETkMbAa2iMghEXmgccJTKnrs3LmTH374gSFDhiASaE7U4xcbG4sxBo/HQ1ZWFunp6SxcuDAs+1KqIepqqdyJNeprqDGmnTGmLdYcXVkicle4g1Mqmrz88suICEOHDg3bPrxT6bvdbuLi4hg/fjwffvghbrc7bPtUqj7qSio3AFcbY370LjDG7ACus9cppbC+5F9++WV69OhBOG+pEBtrTYLhcrkAmDhxIocPH2bVqlVh26dS9VFXUokzxhyuutAYc4ifbzGsVIv36aefsnv3bgYPHuz74g8Hb0vFm1TOO+88YmNjtQtMNRl1JZXyBq5TqkWZM2cOrVq1olevXmFNKiLiG1YMkJ6ezpgxY3j/fR3hr5qGuj79A0Uk0CW7AtT/nqhKNUOlpaW8++67jB49mri4uLAmFbC6wCoqKpg5cyYA7dq147PPPmP79u2ccsopYd23UnWptaVijHEYY1IDPFobY7T7Sylg0aJFFBUVkZmZicPhICamPnfprr/Y2FjfsGKAgQMHAmhrRTUJ4f2XSqlmbubMmTz//PO0bt2a2NjYsLdSoPLJ+ri4ONq3b0/Hjh2ZMWNGpbtLTpkyJeyxKFVVeP+lUqqZKy0tZe3atZxxxhkYYxo9qXgNGDCArVu3UlJSEvb9K1UbTSpKHYc1a9bgcrk444wzACKaVDweD5s3bw77/pWqjSYVpY5DdnY27dq14+STTwYI2Q25aiMiOByOSkmle/fuJCYmsmHDhrDvX6naaFJRqoEOHTrEpk2byMzM9H3BN0ZLxbsf/6TicDjo3bs3mzZtQqflU5GkSUWpBnrrrbfweDwMGzaMiooKYmNjwzbnV1XepOKfQPr27UteXh4HDhxolBiUCkSTilINNHfuXDp27EinTp18I7Eai3df/q2V008/HYCNGzc2WhxKVaVDipWqB+8Fh0eOHGH58uVMnDjRN2twYyYVbzdbRUWFb78ZGRl06NCBjRs3cvbZZzdaLEr505aKUg2QnZ0NwNChQ6moqAAa5yS9V6ARYGB1gW3ZssUXk1KNTVsqSjVAdnY23bp1o7S0FKfTSWxsLPv27Qv7fnft2uV7HhsbS1FREUePHqVr166AlVS++OILduzYEfZYlApEWypK1dP+/fvZvXu3774pMTExERlxZYypNjCgV69exMTE6HkVFTGaVJSqp+zsbETEdx96EcHj8TR6HIGSSmJiIqeeeqper6IiJqxJRUTOE5EtIrJNRO4NsF5E5J/2+rUiMqSuuiJyuYhsEBGPiGRW2d59dvktIjIunMemWiZjDNnZ2Zx22mmkp6cDVlKJVEvFu39/ffr0Yffu3Rw8eLDRY1IqbElFRBzAv4DxQF/gahHpW6XYeKCn/ZgCPBdE3fXAJcDSKvvrC1wFnA6cBzxrb0epkNm9ezcHDhzwdX15v9AjlVSMMdVmRfYOLf7kk08aPSalwtlSGQZsM8bsMMaUA28Ak6qUmQTMNpavgXQR6VhbXWPMJmPMlgD7mwS8YYxx2rc/3mZvR6mQWbVqFTExMQwZYjWqvedTInUVe6AusM6dO5OSksLixYsjEpNq2cKZVE4Cdvu9zrWXBVMmmLoN2R8iMkVEVovI6kOHDtWxSaV+5nK5+Oabb+jfvz8pKSlA5Lq+vLxJxT+GmJgY+vbty+LFi3XKFtXowplUAs1XUfUTXlOZYOo2ZH8YY2YaYzKNMZnt27evY5NK/ezTTz+lsLCQESNGAIG/0Bubx+NBRKpdl9KnTx/279/P+vXrIxSZaqnCmVRygc5+r08G9gZZJpi6DdmfUg326quvkpycTP/+/QGr5RLppOLdd9Wk0revdQpSu8BUYwtnUskGeopIdxGJxzqJvrBKmYXADfYosBFAgTFmX5B1q1oIXCUiCSLSHevk/zehPCDVchUVFTF//nwyMzN9V86Xl5cDRGQ4sT9jjC8WrzZt2vi6wJRqTGFLKsYYF3A78DGwCXjTGLNBRKaKyFS72CJgB9ZJ9ReAabXVBRCRi0UkFxgJfCgiH9t1NgBvAhuB/wC3GWPc4To+1bLMnz+f0tJSX9cXWEmlKZyz8Hg8AWMZO3YsS5cupbS0NEKRqZZImsIfRaRkZmaa1atXRzoMFQXGjBnDvn37+MMf/uAbbXXgwAEqKipwuyP7v0tMTAyxsbG0b9++0v1cOnfuzPnnn8/ixYs599xzIxiham5EJMcYkxlonV5Rr1QdNmzYwLJly7jlllt8CcXlcuHxeJpMSwXA6XRWWj5mzBji4+O1C0w1Kk0qStVhxowZxMfHM3nyZN8y7zmMppBUwLrzY9XzKikpKYwaNUqTimpUmlSUqsWxY8eYPXs2l19+ORkZGb7lFRUVER/55S8+Ph6n0xnwvMratWsbZQZlpUCTilK1euONNygsLGTq1KmVljudTuLj4yMUVXUJCQkYY6oNLR47diygQ4tV49GkolQNjDFMnz6d/v37k5WV5Vvudrtxu91NKql4Y/E/rzJz5kxWrVpFeno6H3zwQaRCUy2M3qRLqRp89NFHrFu3jlmzZlWaX8v7xZ2QkBCp0KpxOBzExcXhdDpp3bq1b3lMTAwDBgzgP//5D06ns0nFrJonbakoVYPHHnuMzp07c/XVV1daXl5e7hvG25QkJiYGHOI8YMAAiouL+eKLLyITmGpRNKkoFcDXX3/N0qVLueuuuyrde94Y4zufUnV24EhLTEwEoKysrNLy3r17k5yczMKFdU1KodTx06SiVACPPfYYbdq04eabb6603O124/F4mlw30q5du9izZw/GGPLz89m1a5fvfvZxcXGMGzeOhQsXNpnRaqr50qSilJ+ZM2fy17/+lQULFjBy5EhatWpVab23FdCUTtL7885aXNWECRPIzc1lzZo1jR+UalE0qShVxeLFi4mNjeWXv/xltXVOp5PY2Ngmdz7Fy5tUqt4N8oILLiAmJoZ33nknQpGplkKTilJ+Dh8+zMqVK8nKyiI1NbXSurKyMsrLy5tc15e/mm4x3KFDB84++2zmzp2rXWAqrJrmv1tKRchHH31ETEwM5513HmB1h4F1zkJEiIuLIz8/n6NHj0YyzFp5PJ5qSQXg2muv5de//jWrVq2qNNuyUqGkLRWlbD/++CMrVqxg9OjRtGnTptr6SN+PPljeLrCq51YuueQSEhISmDNnToQiUy2BJhWlbI888ggxMTGMGzcu4PqYmJiI35ArGDV1gaWmpjJhwgTmzZuHy+WKUHSqudOkohRWK+WVV16psZXi/c8/GpIK/NwFVrVVde2113Lo0CGWLFkSochUc6fnVJQCHn74YRwOh+9cSlXR0vXl5fF4cDgclJWVkZSU5Ds3VFFRQUpKCv/+979rPFaljoe2VFSLt2PHDmbNmsUtt9xCenp6tfXerqRoaaXAz11gVa+uj4uL4xe/+AXvvvsue/bsiVB0qjnTpKJavEceeQSHw8E999wTcL3T6Yyqri8vj8dDWVlZtbjPPPNMPB6Pr/WiVChpUlEtmn8rpVOnTgHLlJSURFXXl5c3mZSWllZa3r59e8aPH8/MmTOr3S1SqeOl51RUi1L1v/PZs2cDcNJJJwX8z93tduN0OqOulQJWF1hsbCylpaWkpKRUWnf77bdz/vnnM3/+fK666qoIRaiaI22pqBbrwIEDrFy5kjFjxgQ8lwI//5dfdTr5aJGcnExFRUW1O0KOGzeOU089laeffjrqWmCqadOkolqs999/n9jYWMaPHx9wvTGGkpKSSlPfR5ukpCTA6sLz9+KLLzJ06FBWrVpV47kkpRpCk4pqkXbv3k12djbnnHNOtTm+vMrLy3G73SQnJzdydKETExNDUlISpaWl1brwfvGLX5CSkqL3r1chpUlFtUgLFiwgOTmZsWPH1ljm2LFjvi/laJaSkuJrdfmLj4/nrLPOYu3atWzZsiVC0anmRpOKanG2bdvGunXrGDduXI2tEJfLhdPpJCkpqcnd4bG+4uLiiI+P941i83fWWWfhcDj4v//7vwhFp5obTSqqRTHG8O6775KamsrZZ59dYznvf/VVR01FG+8dII8dO4bb7SY3N7fS+tTUVEaMGMGsWbM4ePBghKJUzUlYk4qInCciW0Rkm4jcG2C9iMg/7fVrRWRIXXVFpK2IfCIiW+2fbezl3USkVETW2I8Z4Tw2FZ02bNjAtm3buOCCC2q8e+OuXbsoLi7G4/GQm5vruy1vNDPG+KZuqdpaOffccykrK+PZZ5+NUHSqOQlbUhERB/AvYDzQF7haRPpWKTYe6Gk/pgDPBVH3XuBTY0xP4FP7tdd2Y8wg+zE1PEemopXH4+G9994jIyODUaNG1VguJiYGEYnaYcQ18U6JX3XqlhNPPJGJEyfyzDPPVDvvolR9hbOlMgzYZozZYYwpB94AJlUpMwmYbSxfA+ki0rGOupOAWfbzWcBFYTwG1Yy8/vrr7N69mwkTJtR4O2D/eb6a2/Ub3mMqLi6udmynnXYaeXl53HzzzTp9izou4UwqJwG7/V7n2suCKVNb3ROMMfsA7J8d/Mp1F5HvRORLERkdKCgRmSIiq0Vk9aFDh+p7TCpKlZaWcv/999OlSxeGDRtWY7ny8vKomzyyPtxuNy6Xq1pr5dRTT6Vbt24sWbKk2R67ahzhTCqBhsxU/devpjLB1K1qH9DFGDMY+D3wuohUuwDBGDPTGJNpjMls3759HZtUzcXTTz/N7t27ueyyywLeatfL+198c/1i9Z5XqdpaERHGjh3LwYMH+f777yMYoYp24UwquUBnv9cnA3uDLFNb3QN2Fxn2z4MAxhinMSbPfp4DbAdOC8mRqKh28OBB/v73vzNp0iR69epVY7ny8nLfBY/NWatWrQK2VgYNGkRGRgaffPJJhCJTzUE4k0o20FNEuotIPHAVsLBKmYXADfYosBFAgd2lVVvdhcCN9vMbgQUAItLePsGPiPTAOvm/I3yHp6LFn//8Z0pLS3n88cdrLVdcXByVU9zXV1JSErGxsRQVFVVqrTgcDs455xy2b9/OihUrIhihimZhSyrGGBdwO/AxsAl40xizQUSmioh3ZNYirC/+bcALwLTa6tp1HgXOFZGtwLn2a4AxwFoR+R54G5hqjDkSruNT0WHdunW88MIL3HrrrZx2Ws0N14qKCpxOZ9RflxIMEaF169a43e5qo72ysrJITk7mySefjFB0KtpJcxvhUh+ZmZlm9erVkQ5DhYnH42HMmDFs2bKFzZs3065duxpHNuXn51NaWkqHDh2qXSDY3HTt2hVjDEeOHMHlctG+fftK55nmz5/PkiVL2LVrFyedVHVsjVIgIjnGmMxA6/SKetVszZ49m6+++orHH3+cdu3a1VjO5XJRWlpKcnIyDoejESOMHG9rxePxcOzYsUrrRo8ejdvt5tZbb2XmzJk6xFjVi96kSzVLeXl53H333WRlZeF0Omv9YiwuLgasE9gtSXx8PImJiRQXF1dKqO3bt6dv374sX76c8ePHt5hEq0JDWyqqWbrjjjvIz8/n2WefrXUIcUVFhe/OiC3ly9M7H9iuXbsoLCzEGMO+ffsqlRkzZgxHjx5l/fr1EYpSRStNKqrZWbBgAXPmzOFPf/oTAwYMqLHcrl27OHjwIMYYjh496vuibWncbjcOhwOn0+lbNmDAANLS0li6dGkEI1PRSJOKalby8vK45ZZbGDhwIPfdd1+tZWNiYoiJiWn216XUxTt9S0FBgW+IscPhYNSoUWzYsIHDhw9HOEIVTTSpqGbDGMPNN9/M4cOHufDCC3nllVdqPJficrlwOBx4PJ5mf11KMFwuF26323d+CfBNurl8+fJIhaWikCYV1Ww8//zzvPvuu1x00UV06dKlxnIej4ejR48C1pepshKy96S99z1p27Yt/fr146uvvqKioiLCEapooUlFNQsbNmzgrrvuYuzYsfzqV7+qsZwxhvz8fFwulyaUKlJTUxGRSt1gZ555JoWFhSxYsCDC0alooUlFRb2ysjKuvvpqUlNTmTVrVo2jvdxuN3l5eTidTtLS0prd1PbHy+Fw0Lp1a8rLy33zgp1++um0bduWGTP0nncqOHqdiop6d999N+vWrWPRokWceOKJAcuUlZWRn5+PMYb09HSSkpIaOcqmzzvyLTY2lqNHj/puLzxq1CgWLlzI1q1b6dmzZyRDVFFAWyoqqr3//vs888wz3HnnnYwfP77aeu8tgY8ePYrb7aa8vJyDBw+2yKHDwfKOhvNetzNq1CgcDodeWa+ConN/6dxfUWvPnj0MHDiQ5ORk7rnnHuLi4iqtr6io4OjRo7hcLjweT4sfOlwfMTExxMbG4nK56Ny5M4sXL+aLL74gNzeXxMTESIenIkzn/lLNysyZM5kxYwbnnHMORUVF3HTTTZUSiveWuYcPH8YY4xsuq4LnHWrtcDioqKjglltuIS8vj/nz50c6NNXEaVJRUek///kPW7Zs4corr6x0HmXXrl3s3buXoqIi3G43ZWVlekK+gbyj444ePcrQoUM55ZRTeO655yIclWrqNKmoqLNx40YWLlzIsGHDyMrK8i0vLy8nLi4OEdHWSYh4uw5nz57N5MmTWb58uV4MqWqlSUVFlZ07d/Liiy/SsWNHrrvuOkQEgJKSEvLy8oCfvwjV8TPG0LZtW0pLS3G5XLRr1477779fW3+qRppUVNQoKyvjsssuw+12M3XqVBISEjDGUFhYSEFBAQkJCVRUVOgXXojFx8czdepUTjnlFEaMGMGyZcv4y1/+Uu0+LEqBXqeiooTH4+Gmm24iJyeHadOmccIJJ+DxeMjPz8fpdJKcnExqaipFRUWRDrXZ2bVrF9u2baNPnz5MmzaNtWvX8tRTT1FeXs7IkSPJysqibdu2kQ5TNRGaVFRU+J//+R/mzJnD3/72N9q3b8+uXbt8I77cbjf5+fnk5+dHNshmTkQ44YQTePjhh/n1r3/Nxx9/TGJiImvWrKF79+6cdtppDBs2LNJhqgjT7i/V5P3973/nH//4B7fddhv3338/paWlvoSi508aX9++ffntb3/Lt99+y9atW+nUqRPbt29n2bJl1W72pVoeTSqqyTLGcO+993L//fdz9dVX8/jjj/Phhx/6plvR8yeRc+2113LxxRfz2muv8fnnn5OZmUlFRQUvvvgiy5cv10TfgukV9XpFfZNUXFzMtGnTePXVVxkzZgyTJk3i2LFjeDweUlJSfFPXq8jxeDx88skn5OTk0KdPH+6//34SEhLYuHEjXbp0YdKkSXqupZmq7Yp6TSqaVJqc+++/n5deeomDBw8yevRozjrrLGJiYnxTrbTkz2xTtGnTJr788ksKCgq4/vrrmTRpEmvXrsUYQ3JyMjfccAMnnHBCpMNUIaRJpQaaVJqW7du38+CDDzJnzhxat27NpZdeSteuXTWZRIGzzz6bF198kQULFlBWVsagQYPIysoiIyMDgI4dOzJo0CBOP/10UlJSIhytOl6aVGqgSSWyjDFs376d999/n7lz55KdnU1cXBzDhw8nKyuL9PR0iouLNZlEgUsuuQSwpnSZO3cuc+bMwel00rlzZ6655hrat29PcXExIkJGRgadOnUiIyPDd8tiFV00qdRAk0rjmT59Onv37iU3Nxewukx27NhBQUEBAG3atGHw4MFceOGFACQlJRETE6NT1Ecpp9PJ999/zzfffENhYSHdunXj0ksv5fTTT+fw4cOUlJQA1u+9a9eudOjQgXbt2tGuXTvS09N90+6rpkmTSg00qYRWUVERu3btYteuXWzfvp3169ezfv16tm7dyuHDh33lYmNj6dChA506daJPnz6kpKToCd1myu1207p1a1599VW2bNlC27ZtueKKK3znyfLy8jh69Cjl5eW+OjExMbRp08aXYNLS0io9WrVq5ZueR0VGxJKKiJwHTAccwIvGmEerrBd7/flACTDZGPNtbXVFpC0wD+gG7ASuMMYctdfdB/wX4AZ+Z4z5uLb4NKlU5nQ6OXDgAPv37+fo0aO+W8omJyfTqlUrWrVqhTGGQ4cO8frrr3Pw4EEOHDjAwYMHKSgoqHbxocPhICMjg44dO9K2bVsyMjLIyMggLS1NvxRakEsuuQRjDKtXr+bVV19lxYoVAPTq1YszzjiD3r1707lzZ1JTUxERUlNTOXLkCIcPHyY/P79SwgHrIsykpCTfLApDhgyhU6dOpKWlVbuVtMfjwel0ApCQkFDjraZV/UQkqYiIA/gBOBfIBbKBq40xG/3KnA/8FiupDAemG2OG11ZXRB4HjhhjHhWRe4E2xph7RKQvMBcYBnQClgCnGWNqnKo2VEnFGIMxxndC2XsvioqKClwul+/hvfGR/8PhcPge3i9aY4xvO263u9LDu93y8nIqKioqPXe5XJSXl+N0OikvLw/43PtYs2aNr573j9Y7ZNd7PCKC2+2mpKSEo0ePUlBQ4Ou28EpLS6Ndu3a0bduW9PR033+Y7du311mCVUB5eXn88MMPbNu2jX379vmm2Aer29ObMLwP743YkpOTSUpKqjSTgtPp9P0txcTEEB8f71vn/Tv0l5CQ4EtG3kfr1q1JSEjA4XBU+psUEd8jJiam0uuaHsGUC3ZbTVltSSWc07QMA7YZY3bYQbwBTAI2+pWZBMw2Vmb7WkTSRaQjViukprqTgLPs+rOAL4B77OVvGGOcwI8iss2OYWWoD2zv3r288sorvg9tqBJzTEyM7wvda+7cuWzfvt2XtML4TwBxcXG+Pyi32+378CckJJCWlsaJJ55Ienq6L4m0adPG9wdeNW5NKKom7dq1Y+TIkYwcORKPx8Phw4d9/7Tk5+dTXFxMSUkJR44cYc+ePaxbt67Oz1OgL+tbb72VESNG+M7PVFRU0KZNG0pKSigsLGT37t0UFhY22Qs1w51o+vbty0UXXRTSbUJ4k8pJwG6/17lYrZG6ypxUR90TjDH7AIwx+0Skg9+2vg6wrUpEZAowxX5ZLCJbgj2gBsoADtdZKsKMMVW7GXxxFxcX+6aVjwJR8X4HoHEfB2/r3t/06dOZPn16TVWaRNz11JRi7lrTinAmlUBpteq/2TWVCaZuQ/aHMWYmMLOObYWMiKyuqZnYlGncjUvjblzRGHe0xBzOs1a5QGe/1ycDe4MsU1vdA3YXGfbPg/XYn1JKqTAKZ1LJBnqKSHcRiQeuAhZWKbMQuEEsI4ACu2urtroLgRvt5zcCC/yWXyUiCSLSHegJfBOug1NKKVVd2Lq/jDEuEbkd+BhrWPBLxpgNIjLVXj8DWIQ18msb1pDiX9dW1970o8CbIvJfwE/A5XadDSLyJtbJfBdwW20jvxpRo3W1hZjG3bg07sYVjXFHRcwt+uJHpZRSoaVXAimllAoZTSpKKaVCRpPKcRKRy0Vkg4h4RCTTb3k3ESkVkTX2Y4bfujNEZJ2IbBORf9rT1WAPMphnL18lIt0aM2Z73X32/reIyLimEnOAY3hQRPb4vb/nN/QYIklEzrPj3CbWDBFNiojstN+zNSKy2l7WVkQ+EZGt9s82fuUDvveNEOdLInJQRNb7Lat3nI39Gakh7uj+bHuvhNZHwx5AH6AX1pX9mX7LuwHra6jzDTAS69qaj4Dx9vJpwAz7+VXAvEaOuS/wPZAAdAe2A46mEHOAY3gQ+EOA5fU+hgh+dhx2fD2AeDvuvpH+TFeJcSeQUWXZ48C99vN7gcfqeu8bIc4xwBD/v7mGxNnYn5Ea4o7qz7a2VI6TMWaTMSboq/Lta2tSjTErjfVpmA1cZK+ehDX1DMDbwDnh+I+jlph9U90YY37EGpU3rCnEXA8NOYZI8U1lZIwpB7zTETV1/r/zWVT+LFR77xsjIGPMUuDI8cQZic9IDXHXpMnEXRtNKuHVXUS+E5EvRWS0vewkrAs1vfynk/FNT2OMcQEFQLvGCpbap81pijHfLiJr7S4Eb9dGQ44hUmqKtSkxwGIRyRFriiOoMlUS4D9VUlM6nvrG2ZQ+I1H72Q7nNC3NhogsAU4MsOqPxpgFAZYD7AO6GGPyROQM4D0ROZ3ap5NpyPQ0ATUw5oZMmxOymKsFU8sxAM8BD9n7egj4B/CbWuIJW5zHoSnGVFWWMWavWHPsfSIim2spGw3HA03/MxLVn21NKkEwxvyqAXWcgNN+niMi24HTsP6LONmvqP90Mt6pZnJFJBZII/im8XHHTO3T5oQ95qqCPQYReQH4oEo8VWOt7RgipclPLWSM2Wv/PCgi72J1Zx0QkY7GmtC1KU+VVN84m8RnxBhzwPs8Gj/b2v0VJiLSXqz7wiAiPbCmjdlhN8OLRGSEfe7hBipPNeOdguYy4DO7j7SxBJzqpinGbH9JeF0MeEfPNOQYIiWYqYwiRkRSRKS19zkwFut9jpapkuoVZ1P5jET9ZztSIwSaywPrl56L1So5AHxsL78U2IA1WuNbYIJfnUysD8p24Bl+ntkgEXgL6wTcN0CPxozZXvdHO64t+I0giXTMAY7hVWAdsBbrj61jQ48hwp+f87FuSLcdq2sy4p9pv9h62J/f7+3P8h/t5e2AT4Gt9s+2db33jRDrXKwu5wr7s/1fDYmzsT8jNcQd1Z9tnaZFKaVUyGj3l1JKqZDRpKKUUipkNKkopZQKGU0qSimlQkaTilJKqZDRpKJaFBFx2zO/rheR90UkvY7yg6rMEjtRQjSbsIi8IiI/isj3IvKDiMwWkZP81i/yxicivxORTSIyx75OYYl9HFeGIhalQkWTimppSo0xg4wx/bCu/L+tjvKDsK4lAcAYs9AY82gI47nbGDMQa9bo74DP7QshMcacb4zJt8tNA843xlwLDAbi7OOYF8xO7NkOlAo7TSqqJVuJPfGeiAwTkRX2BKArRKSX/eX+V+BKb6tARCaLyDN2nVfse1esEJEdInKZvTxGRJ4V6541H9gtjstqC8RY/g/YD4y3t7NTRDLEuhdPD2ChiNwDvAYMsmM6Rax7aXxpT/r4sfeKbBH5QkQeEZEvgTvqKPeYiHxjt5hG28sdIvKkWPfpWCsiv7WXB9yOUqBJRbVQ9hQ65/DztCibgTHGmMHAA8AjxpqO/gGse8TU1CroCIwCLgS8LZhLsO6n0x+4Ces+F8H6Fujtv8AYMxVrLqdfGmMes7e5zBgzCPgJ+H/AZcaYM4CXgIf9qqcbY84E/llHuVhjzDDgTuDP9rIpWPftGGyMGQDMEZG4OrajWjhtEquWJklE1mB96ecAn9jL04BZItITa4bXuCC3954xxgNsFJET7GWjgLfs5ftF5PN6xFffe9H0AvphzSAM1o2/9vmtnxdkufn2zxys9wbgV1g3YHMBGGOOiEi/OrajWjhNKqqlKTXGDBKRNKzZX2/D+i/+IeBzY8zFYt0S+Ysgt+f0ey5VfjbEYKx5qoIlwAZjTE2toWNBlvMeh5ufvxeE6lOo17Ud1cJp95dqkYwxBcDvgD/YXTppwB579WS/okVA63pufjlwqX1u5QTgrLoqiOV3WN1p/6nHvrYA7UVkpL2dOLHu29PQcv4WA1O9J/lFpG0Dt6NaEE0qqsUyxnyHNQPvVVj3M/+7iHyF1aXj9TnQt57Dd9/BmnF2PfA8sArrjpiBPCEi32PNVDwU67xJeT2OoRzrlgOP2dtZA/yioeWqeBHrnM1au841DdyOakF0lmKlwkBEWhljikWkHdYtAbKMMfsjHZdS4abnVJQKjw/sCxfjgYc0oaiWQlsqSimlQkbPqSillAoZTSpKKaVCRpOKUkqpkNGkopRSKmQ0qSillAqZ/w+a58Q9/ZY5PwAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df_train.selectExpr('black_rating - white_rating').toPandas(), label='black white rating gap', color='gray')\n",
    "sns.distplot(testPredAndLabels.selectExpr(f'prediction - {target}').toPandas(), label='test prediction error', color='k')\n",
    "plt.title('Prediction Error vs Rating Gap')\n",
    "plt.xlabel('Rating Difference')\n",
    "plt.legend()\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Similarly to DecisionTreeRegressor, the RMSE score is very close to the standard deviation of the difference between `black_rating` and `white_rating`. It performs slightly better. If the model is given the `white_rating` feature, the RMSE would see a decrease of about 40, further improving the model prediction."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}