ml-finance-python

04_cnn_with_time_series.ipynb

(34717B)

 {
  "cells": [
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# How to use CNN with time series data"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "The regular measurements of time series result in a similar grid-like data structure as for the image data we have focused on so far. As a result, we can use CNN architectures for univariate and multivariate time series. In the latter case, we consider different time series as channels, similar to the different color signals.\n",
     "\n",
     "We will illustrate the time series use case with the univariate asset price forecast example we introduced in the last chapter. Recall that we create rolling monthly stock returns and use the 24 lagged returns alongside one-hot-encoded month information to predict whether the subsequent monthly return is positive or negative."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Imports & Settings"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
    "metadata": {},
    "outputs": [
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
       "Using TensorFlow backend.\n"
      ]
     }
    ],
    "source": [
     "%matplotlib inline\n",
     "import numpy as np\n",
     "import pandas as pd\n",
     "import matplotlib.pyplot as plt\n",
     "from sklearn.metrics import roc_auc_score\n",
     "import tensorflow as tf\n",
     "import keras\n",
     "from keras.utils import np_utils\n",
     "from keras.datasets import cifar10\n",
     "from keras.models import Sequential\n",
     "from keras.preprocessing.image import ImageDataGenerator\n",
     "from keras.layers import Dense, Dropout, Activation, Flatten, Conv1D, Conv2D, MaxPooling1D, MaxPooling2D\n",
     "from keras.callbacks import ModelCheckpoint, TensorBoard\n",
     "from keras.layers.normalization import BatchNormalization\n",
     "from keras import backend as K"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Prepare Data"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
     "data = pd.read_hdf('data.h5', 'returns')\n",
     "data = data.drop([c for c in data.columns if str(c).startswith('year')], axis=1)"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
     "X_train = data[:'2016'].drop('label', axis=1)\n",
     "y_train = data[:'2016'].label\n",
     "X_test = data['2017'].drop('label', axis=1)\n",
     "y_test = data['2017'].label"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
     "X_train = X_train.values.reshape(-1, X_train.shape[1], 1)\n",
     "X_test = X_test.values.reshape(-1, X_train.shape[1], 1)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Define Custom AUC Metric"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 5,
    "metadata": {},
    "outputs": [],
    "source": [
     "def auc_roc(y_true, y_pred):\n",
     "    # any tensorflow metric\n",
     "    value, update_op = tf.metrics.auc(y_true, y_pred)\n",
     "\n",
     "    # find all variables created for this metric\n",
     "    metric_vars = [i for i in tf.local_variables() if 'auc_roc' in i.name.split('/')[1]]\n",
     "\n",
     "    # Add metric variables to GLOBAL_VARIABLES collection.\n",
     "    # They will be initialized for new session.\n",
     "    for v in metric_vars:\n",
     "        tf.add_to_collection(tf.GraphKeys.GLOBAL_VARIABLES, v)\n",
     "\n",
     "    # force to update metric values\n",
     "    with tf.control_dependencies([update_op]):\n",
     "        value = tf.identity(value)\n",
     "        return value"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "## Build ConvNet"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "### Model Architecture"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "We design a simple one-layer CNN that uses one-dimensional convolutions combined with max pooling to learn time series patterns:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 6,
    "metadata": {},
    "outputs": [],
    "source": [
     "model = Sequential([\n",
     "    Conv1D(filters=32, kernel_size=3, activation='relu', input_shape=(X_train.shape[1], 1)),\n",
     "    MaxPooling1D(pool_size=2),       \n",
     "    Flatten(),\n",
     "    Dense(1, activation='relu'),\n",
     "    Activation('sigmoid')])"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "The model has 673 trainable parameters:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "_________________________________________________________________\n",
       "Layer (type)                 Output Shape              Param #   \n",
       "=================================================================\n",
       "conv1d_1 (Conv1D)            (None, 34, 32)            128       \n",
       "_________________________________________________________________\n",
       "max_pooling1d_1 (MaxPooling1 (None, 17, 32)            0         \n",
       "_________________________________________________________________\n",
       "flatten_1 (Flatten)          (None, 544)               0         \n",
       "_________________________________________________________________\n",
       "dense_1 (Dense)              (None, 1)                 545       \n",
       "_________________________________________________________________\n",
       "activation_1 (Activation)    (None, 1)                 0         \n",
       "=================================================================\n",
       "Total params: 673\n",
       "Trainable params: 673\n",
       "Non-trainable params: 0\n",
       "_________________________________________________________________\n"
      ]
     }
    ],
    "source": [
     "model.summary()"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "We compile using our custom auc_roc metric developed in the last chapter:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 8,
    "metadata": {},
    "outputs": [],
    "source": [
     "model.compile(loss='binary_crossentropy',\n",
     "              optimizer='Adam',\n",
     "              metrics=['binary_accuracy', auc_roc])"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "We train on returns for the years 2010-16 for 20 epochs using the default batch size of 32. Each epoch takes around 13s on a single NVIDIA GTX 1080 GPU."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 9,
    "metadata": {
     "scrolled": false
    },
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "Train on 206587 samples, validate on 29868 samples\n",
       "Epoch 1/20\n",
       "206587/206587 [==============================] - 14s 66us/step - loss: 0.6727 - binary_accuracy: 0.5845 - auc_roc: 0.5991 - val_loss: 0.7233 - val_binary_accuracy: 0.5206 - val_auc_roc: 0.6033\n",
       "Epoch 2/20\n",
       "206587/206587 [==============================] - 14s 68us/step - loss: 0.6687 - binary_accuracy: 0.5935 - auc_roc: 0.6037 - val_loss: 0.7170 - val_binary_accuracy: 0.5205 - val_auc_roc: 0.6056\n",
       "Epoch 3/20\n",
       "206587/206587 [==============================] - 13s 65us/step - loss: 0.6670 - binary_accuracy: 0.5955 - auc_roc: 0.6067 - val_loss: 0.7242 - val_binary_accuracy: 0.5268 - val_auc_roc: 0.6076\n",
       "Epoch 4/20\n",
       "206587/206587 [==============================] - 13s 62us/step - loss: 0.6659 - binary_accuracy: 0.5960 - auc_roc: 0.6084 - val_loss: 0.7191 - val_binary_accuracy: 0.5438 - val_auc_roc: 0.6092\n",
       "Epoch 5/20\n",
       "206587/206587 [==============================] - 13s 63us/step - loss: 0.6652 - binary_accuracy: 0.5969 - auc_roc: 0.6097 - val_loss: 0.7147 - val_binary_accuracy: 0.5453 - val_auc_roc: 0.6106\n",
       "Epoch 6/20\n",
       "206587/206587 [==============================] - 13s 64us/step - loss: 0.6647 - binary_accuracy: 0.5970 - auc_roc: 0.6110 - val_loss: 0.7155 - val_binary_accuracy: 0.5618 - val_auc_roc: 0.6114\n",
       "Epoch 7/20\n",
       "206587/206587 [==============================] - 14s 67us/step - loss: 0.6643 - binary_accuracy: 0.5978 - auc_roc: 0.6120 - val_loss: 0.7109 - val_binary_accuracy: 0.5656 - val_auc_roc: 0.6124\n",
       "Epoch 8/20\n",
       "206587/206587 [==============================] - 14s 68us/step - loss: 0.6637 - binary_accuracy: 0.5988 - auc_roc: 0.6128 - val_loss: 0.7173 - val_binary_accuracy: 0.5605 - val_auc_roc: 0.6133\n",
       "Epoch 9/20\n",
       "206587/206587 [==============================] - 13s 63us/step - loss: 0.6634 - binary_accuracy: 0.5997 - auc_roc: 0.6136 - val_loss: 0.7121 - val_binary_accuracy: 0.5627 - val_auc_roc: 0.6140\n",
       "Epoch 10/20\n",
       "206587/206587 [==============================] - 14s 66us/step - loss: 0.6633 - binary_accuracy: 0.5995 - auc_roc: 0.6143 - val_loss: 0.7158 - val_binary_accuracy: 0.5652 - val_auc_roc: 0.6145\n",
       "Epoch 11/20\n",
       "206587/206587 [==============================] - 14s 70us/step - loss: 0.6631 - binary_accuracy: 0.5999 - auc_roc: 0.6148 - val_loss: 0.7131 - val_binary_accuracy: 0.5627 - val_auc_roc: 0.6151\n",
       "Epoch 12/20\n",
       "206587/206587 [==============================] - 15s 72us/step - loss: 0.6628 - binary_accuracy: 0.6005 - auc_roc: 0.6153 - val_loss: 0.7068 - val_binary_accuracy: 0.5784 - val_auc_roc: 0.6156\n",
       "Epoch 13/20\n",
       "206587/206587 [==============================] - 14s 65us/step - loss: 0.6627 - binary_accuracy: 0.6010 - auc_roc: 0.6159 - val_loss: 0.7234 - val_binary_accuracy: 0.5697 - val_auc_roc: 0.6160\n",
       "Epoch 14/20\n",
       "206587/206587 [==============================] - 14s 69us/step - loss: 0.6625 - binary_accuracy: 0.6003 - auc_roc: 0.6162 - val_loss: 0.7050 - val_binary_accuracy: 0.5830 - val_auc_roc: 0.6164\n",
       "Epoch 15/20\n",
       "206587/206587 [==============================] - 13s 65us/step - loss: 0.6624 - binary_accuracy: 0.6009 - auc_roc: 0.6166 - val_loss: 0.7068 - val_binary_accuracy: 0.5781 - val_auc_roc: 0.6169\n",
       "Epoch 16/20\n",
       "206587/206587 [==============================] - 14s 68us/step - loss: 0.6624 - binary_accuracy: 0.6003 - auc_roc: 0.6170 - val_loss: 0.7071 - val_binary_accuracy: 0.5728 - val_auc_roc: 0.6172\n",
       "Epoch 17/20\n",
       "206587/206587 [==============================] - 14s 66us/step - loss: 0.6622 - binary_accuracy: 0.6012 - auc_roc: 0.6173 - val_loss: 0.7074 - val_binary_accuracy: 0.5598 - val_auc_roc: 0.6174\n",
       "Epoch 18/20\n",
       "206587/206587 [==============================] - 13s 61us/step - loss: 0.6622 - binary_accuracy: 0.6006 - auc_roc: 0.6175 - val_loss: 0.7220 - val_binary_accuracy: 0.5649 - val_auc_roc: 0.6176\n",
       "Epoch 19/20\n",
       "206587/206587 [==============================] - 13s 62us/step - loss: 0.6620 - binary_accuracy: 0.6005 - auc_roc: 0.6176 - val_loss: 0.7049 - val_binary_accuracy: 0.5791 - val_auc_roc: 0.6177\n",
       "Epoch 20/20\n",
       "206587/206587 [==============================] - 13s 62us/step - loss: 0.6619 - binary_accuracy: 0.6014 - auc_roc: 0.6179 - val_loss: 0.7032 - val_binary_accuracy: 0.5828 - val_auc_roc: 0.6180\n"
      ]
     }
    ],
    "source": [
     "training = model.fit(X_train,\n",
     "                     y_train, \n",
     "                     epochs=20,\n",
     "                     batch_size=32, \n",
     "                     validation_data=(X_test, y_test),\n",
     "                     shuffle=True,\n",
     "                     verbose=1)"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 10,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "Accuracy: 58.28%\n"
      ]
     }
    ],
    "source": [
     "accuracy = model.evaluate(X_test, y_test, verbose=0)[1]\n",
     "print('Accuracy: {:.2%}'.format(accuracy))"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "For 2017 returns, we find a test accuracy of 58.28% and test AUC of 0.5701. The network is still underfitting at this point because both training and validation AUC are still improving after 20 epochs, suggesting that longer training and potentially a higher-capacity network would improve results. You should try!"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 11,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
        "0.5709754730834962"
       ]
      },
      "execution_count": 11,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
     "y_score = model.predict(X_test)\n",
     "roc_auc_score(y_score=y_score, y_true=y_test)"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 13,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD8CAYAAABpcuN4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VNX5+PHPQ1bIRkgCAcJuWAUEI4uooCiCoIBW3KpSF7TWurS1RVuXn7WtdtHqt9QW64LWulFBiggCLiAKkiAECEti2BKyJwSQLcvz++Pe4BCyDJBksjzv12temTn33HvPvZmZZ865554jqooxxhhTnVa+LoAxxpjGzQKFMcaYGlmgMMYYUyMLFMYYY2pkgcIYY0yNLFAYY4ypkQUKY4wxNbJAYYwxpkYWKIwxxtTI39cFqAvR0dHavXt3XxfDGGOalKSkpHxVjaktX7MIFN27dycxMdHXxTDGmCZFRHZ5k8+anowxxtTIAoUxxpgaWaAwxhhTo2ZxjaIqJSUlZGRkcOTIEV8XpUUJDg4mLi6OgIAAXxfFGFNHmm2gyMjIICwsjO7duyMivi5Oi6CqFBQUkJGRQY8ePXxdHGNMHWm2TU9HjhwhKirKgkQDEhGioqKsFmdMM9NsAwVgQcIH7Jwb0/w026YnY4xptg7mQfYGyEqGTkOg18X1ujsLFMYY01ipQnEGZCdDlhsYsjbAgb3Hsxw876eEWqAwNVFVVJVWrZp1K6IxzV95ORSmuzUFj6BwuBAAlVbsD+nBrsCzWR8+kc/3d2TtkTh+6DeYX9Zz0bwKFCIyHnge8AP+papPV5FnGvAEoMAGVb3RTV8MjAC+UNVJHvlXAmHuy/bA16o6RUTGAB8AO9xl76vqk6d+aI3DlClT2LNnD0eOHOH+++9nxowZhIaGcvDgQQDmzp3LwoULee2118jJyeHuu+8mPT0dgBdffJHzzz//pG3u3LmTyy+/nOHDh5OUlMSiRYv48ssv+f3vf4+qMnHiRJ555hkAFi9ezCOPPEJZWRnR0dEsX7684Q7eGHOykiOwPxP27XYeuSlOQMjeCMec74XyVoEUhvQiLeh8EunC5wc6srG0C0cOB9E6wI++HcPo3yOcX3UKZ3iPdvVe5FoDhYj4AbOAy4AMYK2ILFDVFI888cDDwChVLRKR9h6b+BPQBrjLc7uqeqHH+v/FCQ4VVnoGlTP1//63mZS9++tqcwD07xTO41cOqDXfK6+8Qrt27Th8+DDnnXce11xzTbV577vvPkaPHs28efMoKys7Hkyqkpqaypw5cxgxYgR79+7lV7/6FUlJSURGRjJu3Djmz5/PqFGjuPPOO1mxYgU9evSgsLDwtI7VGOMlVThSDMV7YN8e5+8JzzPgYM4Jq5T6tSGnTTzbgi/lq/I4Vn3XiVSNo+SQPzFhQfTvGE7C4HBu6RhO/07hdI8Kwa9Vw3Ya8aZGMQxIU9V0ABF5G5gMpHjkuROYpapFAKqaW7FAVZe7tYQqiUg4cAnwo1MufRPwwgsvMG/ePAD27NlDampqtXk/+eQTXn/9dQD8/PyIiIioNm+3bt0YMWIEAGvXrmXMmDHExDiDQN50002sWLECPz8/LrroouP3NLRrV/+/PIxpdlTh2HdwuKiKRyHs34vu20P5vt3Ivj20KjnxB16JBFIU0IFciSaTwez0a8e3R9uxqyyKTKLYq9HooVb0iA6hf89wruwUQf9O4fTrGEb7sGAfHfSJvAkUnYE9Hq8zgOGV8vQGEJFVOM1TT6jqYi/LMAVYrqqeP/lHisgGYC/wC1Xd7OW2quTNL//68Nlnn7Fs2TK++uor2rRpw5gxYzhy5MgJXUhP956DkJCQuiqmMS1PeRkU7YSczfBdLhyqKgg4Dz1chJSXVLup/YSQqdFklEeToaPYq1FkarT7iOFQYCTtgoOICg0iKiSQqJBA2oUGEh8SRLuQQHrEhNA3Now2gY33knFdlcwfiAfGAHHAChEZqKr7vFj3BuBfHq/XAd1U9aCIXAHMd7d9AhGZAcwA6Nq165mVvp4UFxcTGRlJmzZt2Lp1K6tXrwagQ4cObNmyhT59+jBv3jzCwpxLNWPHjuXFF1/kgQceON70VFOtosKwYcO47777yM/PJzIykrfeeouf/vSnjBgxgnvuuYcdO3Ycb3qyWoVpcQ4VOgEhNwVyNrnPt0DJoROylfq15rB/BAdbhVKsoRSUR5NT0oWcktbs01D2Eer81VBKgiIICouiTXgUbSPaEhXqBoCQQM4PdQKAkxZE60A/Hx143fEmUGQCXTxex7lpnjKANapaAuwQke04X+5ra9qwiETjNG1NrUjzrFmo6iIR+buIRKtqvue6qjobmA2QkJCgXhxHgxs/fjz/+Mc/6NevH3369DneVPT0008zadIkYmJiSEhIOH4t4vnnn2fGjBm8/PLL+Pn58eKLLzJy5Mha99OxY0eefvppLr744uMXsydPngzA7NmzufrqqykvL6d9+/YsXbq0/g7YGF8qK4H8VCcQVASEnM0ndCUtCYokt008aWETSTzciZX7Y8gsa0cxIRzDGZ8sPNifjhGtiY0IplPbYGLDW9MrIpiObYPpGBFMbERrQoMa76//+iCqNX/Hiog/sB0YixMg1gI3ejYHub2iblDVW90v/2+Ac1S1wF0+BqcJaVKlbd8NjFTVWz3SYoEcVVURGQbMxalhVFvQhIQErTxx0ZYtW+jXr19tx2/qgZ17U++OHoSs9bD3G8h2g0L+Nig7BoC2CuBgWE8yAnuyuawLqw7E8sWBDuTRFhA6hDsXift3CqdbVAid3MAQGxHcooKAiCSpakJt+Wo9I6paKiL3Aktwrj+8oqqbReRJIFFVF7jLxolIClAGPOQRJFYCfYFQEckAblfVJe7mrwcqd7X9AfBjESkFDgPX1xQkjDHNXOlRp4aQuc4JDJnrnKCg5QCUhMRSEBJPesx1fHOsE58VxbD+UHtKDvkjgnORuEc4t3WKYEAnJzhEhwb5+KCallprFE1Bc61RFBQUMHbs2JPSly9fTlRUlA9K5J3mcO6Nj5SXQf52JxhkJsHedU5toaKm0Caa/e0Gst0vnhWHujIvpz0Zx0IBCPRvRd/YMPp3DHcDQgR9Y8MIaUE1hFNVZzUK4ztRUVGsX7/e18Uwpv7s2w0Za7+vLWRtOH7TGYFhaKdzyB9wGxvKe7JkX2c+2uPPwcIyAOLbh3LJuVGc06UtAzpF0DMmhAA/G6GgPligMMY0nGOHYNcqSFvmPArSnHS/IIgdiA6+gazQ/nx1pDtLskNZvaOI/UdKAegZHcJV50QxsmcUI3pGERNmzUcNxQKFMab+qDpNSRWBYecqKDsK/q2h+wVowu3sCTuHFcUxrNqxn9VJBRQdKgEO0y1KmHB2R0b2imJkryg6hDeOm89aIgsUxpi6dWQ/7PjcDQ7LnaErAKL7wHl3UNLjEtaU9WHRtn0s+ySH3ANFQBGd27ZmbL8OTo2hVxSd27b26WGY71mgMMacmfJyyNn4fWDYswbKSyEwDHqOhgt/zuFuY/g8J5glm3NY9lYOB44k0ybQj4v7tGd07xhG9oqiS7s2vj4SUw0LFMaYU7dvD+xcCemfw7efOMNgAMQOgvPvg7MupThqCMtTC1m8KZsVH2zlSEk5bdsEMH5ALJcPiOWC+GiCA5r+XcstgQWKRsJz6HFjGp0DOU5g2PE57FgJRe4sAK3bObOrnXUp9BpLrkbwcUoOS5Zn89W3n1FarsSGB3NdQhcuPzuWYd3b4W89k5ocCxTmuNLSUvz97S1hcMZH2rkSdqxwAkP+Nic9KAK6j4JhM6DHRdC+P7uKDrNkczZL/v0t63YXoerc5HbHhT0Zf3YsgzpH0KqBh8U2datlfCt8NNOZFKQuxQ6ECSfN33TczJkz6dKlCz/5yU8AeOKJJ/D39+fTTz+lqKiIkpISnnrqqeNjMtXk4MGDTJ48+aT1du7cyaRJk9i0aRMAf/7znzl48CBPPPEEaWlp3H333eTl5eHn58d7771Hr169Ttr2Z599xqOPPkpkZCRbt25l+/btPPvss7zyyisA3HHHHTzwwAMAvP766/z5z39GRBg0aBBvvPHGKZ8200gdKYZdX34fGHLcz0tACHQbCUNugu4XcrDdALbkfMfmzGI2r9zPhoyVbM9xasIDOoXzs0t7c/nZscS3Dz1hlGTTtLWMQOED1113HQ888MDxQPHuu++yZMkS7rvvPsLDw8nPz2fEiBFcddVVtX6ggoODmTdv3knr1eSmm25i5syZTJ06lSNHjlBeXl5t3nXr1rFp0yZ69OhBUlISr776KmvWrEFVGT58OKNHjyYwMJCnnnqKL7/8kujoaJsEqakrOQJ7VsO3nzrBIWu9MySGXxB0HQ4X/4b9HUeSrD3ZmHWYzbuKSflqPzsKllExmENUSCADOkcwLaELlw+ItYvRzVjLCBQ1/PKvL0OGDCE3N5e9e/eSl5dHZGQksbGxPPjgg6xYsYJWrVqRmZlJTk4OsbGxNW5LVXnkkUdOWq86Bw4cIDMzk6lTnUF5g4Nr7n8+bNiw45MbffHFF0ydOvX4fBdXX301K1euRES49tpriY6OBmwSpCZH1RkKI/1T5+Lzri+h9Ai08kfjzuPgsPvZGjyEr471JDn7KJu/LCaruBhnfE/o3LY1AzqFM2VIZwZ0CmdApwg6hAdZraGFaBmBwkeuvfZa5s6dS3Z2Ntdddx1vvvkmeXl5JCUlERAQQPfu3b2auKi69fz9/U+oKdgkSOYEB7Ih/TMnMKR/9v0UnNG9OTb4Fr5uNYh/Z3dhTcYxirY7E/OI7KZXTCjDerRjQKdwznZnW2vbJtBnh2F8zwJFPbruuuu48847yc/P5/PPP+fdd9+lffv2BAQE8Omnn7Jr1y6vtlNcXFzleh06dCA3N5eCggJCQ0NZuHAh48ePJywsjLi4OObPn8+UKVM4evQoZWVltGlTe9PAhRdeyPTp05k5cyaqyrx583jjjTcIDAxk6tSp/OxnPyMqKsomQWqMjh2C3V86zUnffgq57kwAbs+k0h5jWM0g3t5WztI1ORwtLadblDKufyxnd3YG0evXsXHPtGZ8w94R9WjAgAEcOHCAzp0707FjR2666SauvPJKBg4cSEJCAn379vVqO9WtFxAQwGOPPcawYcPo3LnzCdt74403uOuuu3jssccICAjgvffeo2fPnrXua+jQoUyfPp1hw4YBzsXsIUOGAPDrX/+a0aNH4+fnx5AhQ3jttddO8YyYOqXqdNL4drkTGHavdobH8AuEriPg0ifQnmP4pqQr89dn8b8P91J0KJN2IYFcf14XpgzpzDld2lrzkamVDTNu6pyd+3pUca1h8/uweR4Upjvp7ftDr0ug58XQ7Xx27Ffmf5PJ/PWZ7Co4RJB/K8YNiGXqkE5cGB9jo6wawIYZN6Z5yd36fXDI3w7SyrmPYdT9EH85hHek4OBRFiZnMe/jb1i/Zx8icH6vKO69+CzGnx1LWHCAr4/CNFEWKBqRjRs3cvPNN5+QFhQUxJo1axr1tk09KfgWNr3vBIjcFECg+wUw/G7odxWExnD4WBkfp2Tzwfq1fL49j7JypV/HcB65oi9XDe5MbISNuGrOnFeBwp0T+3mcqVD/paon9TcVkWnAE4ACG1T1Rjd9MTAC+MJzzmwReQ0YDRS7SdNVdb04DabPA1cAh9z0dadzcKrapNpfBw4cWG8TFdXntj01h6ZMnyra6dQaNr0P2clOWpcRMOGP0H8yhMWSd+Aon27JZfnWRFam5nPoWBkdI4K588KeTBnSib6x4T49BNP81BooRMQPmAVcBmQAa0VkgaqmeOSJBx4GRqlqkYi099jEn4A2wF1VbP4hVZ1bKW0CEO8+hgMvun9PSXBwMAUFBURFRTWpYNGUqSoFBQW13rdhKinO+D447HV/E3VOgMt/D/2noOGdSMnazydf57Js67ds2LMPgI4RwUwd0plJgzoxvEc7GybD1BtvahTDgDRVTQcQkbeByUCKR547gVmqWgSgqrkVC1R1uYiMOYUyTQZeV+en6WoRaSsiHVU16xS2QVxcHBkZGeTl5Z3KauYMBQcHExcX5+tiNH6HCp0mpeT3nDukATqeA5f+PxgwlSOhcXz1bQHLPsnhk61bySp27pEZ3KUtP7+sN5f0a0//juH2I8g0CG8CRWdgj8frDE7+hd8bQERW4TRPPaGqi73Y9u9E5DFgOTBTVY9Ws7/OwCkFioCAgON3GxvTKJQchm0fQfK7kLbUmbOhfX+45FEYMJXcgM58sjWXZQtyWZW2hcMlZbQJ9OPC+GgevLQ3Y/rG0D7Mamum4dXVxWx/nKaiMUAcsEJEBqrqvhrWeRjIBgKB2cCvgCe93aGIzABmAHTt2vX0Sm1MfSsvc0ZhTX4PUj6AYwcgrBOMuAcdNI3NZV1ZviWX5W9lkZyxFXCGy7g2IY5L+rZnRM8om7PB+Jw3gSIT6OLxOs5N85QBrFHVEmCHiGzHCRxrq9uoR1PSURF5FfjFKewPVZ2NE2BISEiwK6im8VCFnE2Q/A5snAsHsiAoHAZMRgdey5bAwXy4OYeFb2Sxq2A3IjCkS1seurwPY/u1p0+HMGtSMo2KN4FiLRAvIj1wvrCvB26slGc+cAPwqohE4zRFpde00YrrDm4vpynAJnfRAuBe91rIcKD4VK9PGOMTxRmw8T2naSk3BVr5Q/w4GPQH0tqOYkHKPhbO20t63pf4tRLO7xXFj0f34tL+HYgODfJ16Y2pVq2BQlVLReReYAnO9YdXVHWziDwJJKrqAnfZOBFJAcpwejMVAIjISqAvECoiGcDtqroEeFNEYgAB1gN3u7tchNM1Ng2ne+yP6u5wjaljh/c5TUrJ78KuL5y0LsNh4l/Y2WEcC1KP8uHHWWzLWYsIjOgRxe0X9GD8gFiiLDiYJqLZDuFhTL0pPeZcjE5+B7YtdsZXijoLBl1PZpdJfLA7gIUbskjJ2g/Aed0jmTSoExMGxtrFaNOo2BAextQlVchY6wSHTe/D4UJoEw0JPyK35xTmZ8ewcGM2yR85La5Durbl0Un9mTiwo90dbZo8CxTG1KQw3WlWSn7Hee4fDH0nUnb2ND463I/XVmeS+HkhUMiguAgentCXiYM6Ehdps72Z5sMChTGVVdwMt+EdyPgaEOhxIVz4Cw6dNYF3k4t5ecEO9hRuokd0CA9d3odJgzrSLcomgDLNkwUKYwBKj8L2xU5wSP0Yykucm+Eu/X8w8Fry/aKZ8+VO3vhfIvsOlXBut0h+M7E/l/XrYENnmGbPAoVpuVRhzxpY/x9ImQ9HiiE0FobfBYOug9iB7Cg4xEufpDM3KZmSsnIu69eBu0b35NxuNrufaTksUJiWp7zcqT2s/AtkJkJACPS7EgZNg55joJUfSbuKmP3vJD5OySHArxXXDI3jjgt70Csm1NelN6bBWaAwLUdZqTNK6xfPOjfERXaHSc/BwGkQFEp5ubJsSw6zV6STuKuIiNYB/GTMWdx6fndiwuyeB9NyWaAwzV/pUad5adVfnfkeYvrB1S/BgKvBz58jJWXM+3o3L61MJz3vO+IiW/P4lf2ZltCFkCD7iBhjnwLTfB37DpJegy//zxlvqdNQZ46H3hOgVSsOHCnh9RVpvLpqJ/kHj3J253BeuGEIV5wdi7/NKW3McRYoTPNzuAi+fglWv+jcGNf9QpjyonP9QYRDx0qZ8+UO/rniW/YdKmF07xjuuqgnI3vZJFfGVMUChWk+DubCV3+DtS/DsYPQezxc+HPoMgyAIyVlvLlmNy9+lkb+wWNc3CeGn13Wh4FxET4uuDGNmwUK0/Tt2w2rXoBv3oCyYzBgKlzwIMQOBOBYaTnvJO5h1idpZO8/wqizovjnZX04t1ukjwtuTNNggcI0XUU74bNnYOO7gMDg650AEdULgNKyct5fl8nzy1PJ3HeYhG6RPHvdYM7vFe3TYhvT1FigME3PoUJY8WdY+xJIKzjvDjj/pxDhzNVdVq78b8Nenl+eyo787xgUF8Hvrx7IRfHRdg3CmNNggcI0HSWHnQvUX/zVmVL0nBthzCMQ0RmA8nJl8eZsnlu6ndTcg/SNDeOlWxK4tF97CxDGnAELFKbxKy+DDW/BJ7+DA3sh/nK49Ano0B8AVWX5llz+snQ7W7L20ysmhFk3DmXC2bE2DpMxdcAChWm8VCF1KSx73LmTutNQuOYl6H6Bu1hZmZrPX5ZuZ8OefXSLasNz1w3mqsGd8bMAYUyd8SpQiMh44HmcqVD/papPV5FnGvAEoMAGVb3RTV8MjAC+UNVJHvnfBBKAEuBr4C5VLRGRMcAHwA436/uq+uRpHZ1pujKTYOnjsHMlRPaAa1+D/lPAbULaln2Apz5MYWVqPp3btuaZawZy9dA4AuxGOWPqXK2BQkT8gFnAZUAGsFZEFqhqikeeeOBhYJSqFolIe49N/AloA9xVadNvAj90n/8HuAN40X290jOomBakMB2W/9aZD6JNFEz4E5w7HfwDAcg/eJRnl27n7a93Exrkz28m9uPmkd0I8vfzbbmNaca8qVEMA9JUNR1ARN4GJgMpHnnuBGapahGAquZWLFDV5W4t4QSquqjiuYh8DcSdzgGYZuK7fFjxJ+dmOb8AuOghOP8+CA4H4GhpGa+u2smsT9I4VFLGLSO7c//YeCJDAn1ccGOaP28CRWdgj8frDGB4pTy9AURkFU7z1BOqutibAohIAHAzcL9H8kgR2QDsBX6hqpu92ZZpgo4dgtV/d3oylXwHQ2+BMQ9DWCzgXIf4aFM2f/hoC3sKD3NJ3/Y8ckU/zmpvw30b01Dq6mK2PxAPjMGpGawQkYGqus+Ldf8OrFDVle7rdUA3VT0oIlcA891tn0BEZgAzALp27XrmR2AaXsoC+OiXzoB9fSbCpY9DTJ/ji5Mz9vHUwi18vbOQPh3CeOP2YVwYH+PDAhvTMnkTKDKBLh6v49w0TxnAGlUtAXaIyHacL/e1NW1YRB4HYvC4fqGq+z2eLxKRv4tItKrme66rqrOB2QAJCQnqxXGYxuJgLix6yJlVLnYg/OBV6Dby+OLs4iP8cclW3l+XSVRIIL+bejbXJXSxEV2N8RFvAsVaIF5EeuAEiOuBGyvlmQ/cALwqItE4TVHpNW1URO4ALgfGqmq5R3oskKOqKiLDgFZAgZfHYxozVUh+Fxb/yhkC/JJHYdT9zjUJ4NCxUmavSOefn6dTVq7cPboX91zci/DgAB8X3JiWrdZAoaqlInIvsATn+sMrqrpZRJ4EElV1gbtsnIikAGXAQ6paACAiK4G+QKiIZAC3q+oS4B/ALuAr967Zim6wPwB+LCKlwGHgelW1GkNTV5wJCx+E1CUQdx5MnnW8mam8XJn3TSZ/WrKN7P1HmDiwIzMn9KVLuzY+LrQxBkCaw3dwQkKCJiYm+roYpiqqsG4OfPwolJXA2Mdg+F3QyunOunZnIb9dmEJyRjGD4iJ4dFJ/zuvezseFNqZlEJEkVU2oLZ/dmW3qT+EO+N99sGOFM3nQVS9Au54AFB8q4XeLUng3MYPY8GCenTaYKed0tiE3jGmELFCYuldeBl/PhuVPgvjBpL/C0FuhlXMxevGmLB79YDOF3x3j7tG9uG/sWbQJtLeiMY2VfTpN3crbDh/8BDK+hvhxMOm548N/5x44wuMfbOajTdn07xjOq9PP4+zONrucMY2dBQpTN8pK4MsXnImEAtvA1H/CoOtABFVlblIGT324hcMlZTx0eR9mXNTTxmUypomwQGHOXFayU4vITob+k+GKP0OoM9zXnsJDPDJvIytT8zmveyRPXzOIXjF2V7UxTYkFCnP6Sg7Dyr/AF89B63Yw7XUnUODMMjfny538ack2Wgn8dvIAbhrezS5WG9MEWaAwp668HDb9F5Y9AfszYPANcPnvoY3TrTU15wC//G8y3+zex5g+Mfxu6kA6t23t2zIbY06bBQpzanZ9BUsegb3rIHYQTP0H9LgQgGOl5bz42bfM+jSNkCA/nrvO6fJq05Aa07RZoDDeKUx3ahApH0BYR5jyIgy6/niX1w179vHLuclsyznAlYM78fiV/YkODfJtmY0xdcIChanZ4X3OPBFr/umMyTTmETj/XggMcRYfK+PZpdt4+YsdxIQF8dItCVzWv4OPC22MqUsWKEzVykog8VX47A9wuAiG3AQX/wbCOx7PsnlvMfe8uY5dBYe4cXhXZk7oawP4GdMMWaAwJ1KF7YudsZkKUqHHRTDud9Bx0AnZFm/K5sF31tO2TQBv3TmCkb2ifFRgY0x9s0BhvpeVDB//2hmbKSoebngbeo8Hj4vRqsrfP/uWPy3Zxjld2jL7lnNpHxbsw0IbY+qbBQoD+7Pgk6dg/ZvQOhIm/AkSfnR8nogKR0rKmPnfZOav38vkczrxzDWDCA7w81GhjTENxQJFS1Z6zLlZbtVfnWsS598LF/4CWrc9KWvegaPMeCORb3bv4xfjevOTi8+ybq/GtBAWKFqqI/vh3Zsh/TPnbupLnzg+BHhlKXv3c8ectRQdKuHFm4YyYWDHKvMZY5onCxQt0f4sePNayE2ByX93ejRV4+PN2TzwznrCgwN47+6RNtqrMS2QBYqWJm8b/PsaOFQIN70LZ11aZTZV5R+fp/PHJVsZ1DmCl25JoH24XbQ2piXyapxnERkvIttEJE1EZlaTZ5qIpIjIZhH5j0f6YhHZJyILK+XvISJr3G2+IyKBbnqQ+zrNXd799A/PnGDXV/DyOCg9Cj9aVG2QOFpaxs/f28Azi7cyaVAn3rlrpAUJY1qwWgOFiPgBs4AJQH/gBhHpXylPPPAwMEpVBwAPeCz+E3BzFZt+BnhOVc8CioDb3fTbgSI3/Tk3nzlTKR/A65MhJBruWAqdzqkyW/7Bo9z40hreX5fJzy7rzQvXn2M9m4xp4bypUQwD0lQ1XVWPAW8DkyvluROYpapFAKqaW7FAVZcDBzwzi9Nd5hJgrps0B5jiPp/svsZdPlase82ZWf0PePdW6DgYbvsYIrtXmW1L1n4m/20Vm/cWM+vGodw3Nt56NhljvAoUnYE9Hq8z3DRPvYHeIrJKRFaLyPhathlZ+vFFAAAbDElEQVQF7FPV0iq2eXx/7vJiN/8JRGSGiCSKSGJeXp4Xh9EClZfDx7+Bxb+CvhPh1gUQUvUd1MtScvjBi19SWl7Oe3edz8RB1rPJGOOoq4vZ/kA8MAaIA1aIyEBV3VdH2z+Jqs4GZgMkJCRofe2nySo9CvN/7Mwbcd4dMOGP0OrkJiRVZfaKdJ5evJWB7kXrDnY9whjjwZtAkQl08Xgd56Z5ygDWqGoJsENEtuMEjrXVbLMAaCsi/m6twXObFfvLEBF/IMLNb7x1eB+880PYudK5P2LUAycMw1HhaGkZv563iblJGUwc1JE//2AwrQPteoQx5kTeND2tBeLdXkqBwPXAgkp55uPUJhCRaJymqPTqNqiqCnwK/MBNuhX4wH2+wH2Nu/wTN7/xRnEmvDoBdn8FU2fDBQ9WGSSKD5Vw6ytfMzcpgwcujedvNwyxIGGMqVKtNQpVLRWRe4ElgB/wiqpuFpEngURVXeAuGyciKUAZ8JCqFgCIyEqgLxAqIhnA7aq6BPgV8LaIPAV8A7zs7vJl4A0RSQMKcQKT8UZOCrz5A+eu65vmQq+Lq8y2p/AQP3ptLbsLDvHX685hypDKl5yMMeZ70hx+rCckJGhiYqKvi+FbO1bC2zdBQGu46b2ThgWvsGHPPm6fs5aSMuWfN5/LiJ42PLgxLZWIJKlqQm357M7s5mDjXOfCdWQP+OFcaNu1ymwfb87mvre/ISYsiLenD+Os9qENXFBjTFNkgaKp+/JvzhwSXc+H69+ENu2qzPbqqh08uTCFQXFtefnWBJvP2hjjNQsUTdnnf4RPf+eM/jp1NgSc3K21rFx56sMUXl21k3H9O/D89XbR2hhzaixQNFWrXnCCxOAbYPKsKu+ROHysjPvf/oaPU3K4bVQPfj2xH36t7E5rY8ypsUDRFK2ZDUsfhQFT4aq/VRkk8g4c5Y45a0nOLObxK/vzo1E9fFBQY0xzYIGiqVn3Onz0EPSZCFe/BH4n/wvTcg8w/dW15B88yj9/eC7jBsT6oKDGmObCAkVTkvwuLLgPeo2Fa189aU5rgNXpBcx4PZFAfz/emTGSwV1OntbUGGNOhQWKpiLlA5h3N3S/AK77N/if3Gtp/jeZPDR3A92iQnh1+nl0adfGBwU1xjQ3Fiiagu1LYO7tEJcAN7wNgScGAFXlb5+k8Zel2xnRsx3//GECEW1Orm0YY8zpsEDR2H37KbxzM3QY4NxxHXTiTXIlZeX8et5G3k3M4OohnXn6mkEE+ns1caExxnjFAkVjtutLeOsGiDoLbp4HwREnLP7uaCl3vZHEF2n53Dc2ngcvtYmGjDF1zwJFY5WRCG9eC227wC0fnHTH9bHScu7+dxJfpRfwxx8MYlpCl2o2ZIwxZ8baKBqjrA3w76shJAZuWQChMScsLi9Xfjl3AytT8/nD1QMtSBhj6pUFisYmdwu8PgWCwp2pS8NPnpL06cVbmb9+Lw9d3seChDGm3lmgaEzy02DOVeAX6DQ3VTEK7L9WpjN7RTq3jOzGPWN6+aCQxpiWxq5RNBZFO+H1q0DLYfpCiDo5CHywPpOnPtzCFQNjefzKAXbh2hjTICxQNAbFmU5N4th3TpCI6XNSlpWpefzivQ0M79GOZ6edY4P7GWMajFdNTyIyXkS2iUiaiMysJs80EUkRkc0i8h+P9FtFJNV93OqmhYnIeo9Hvoj81V02XUTyPJbdURcH2mgdyHFqEoeLnC6wsQNPyrIxo5i730iiV0wos29JIDjAhgk3xjScWmsUIuIHzAIuAzKAtSKyQFVTPPLEAw8Do1S1SETau+ntgMeBBECBJHfdIuAcj/WTgPc9dvuOqt57xkfX2B0qhDemwP4suPl96Dz0pCy7Cr7jR699Tds2gcy5bRgRre2Oa2NMw/KmRjEMSFPVdFU9BrwNTK6U505glhsAUNVcN/1yYKmqFrrLlgLjPVcUkd5Ae2Dl6R9GE1ReDvPugvxUuOEt6DripCx5B45yyytfU1auzLltGB3CT56YyBhj6ps3gaIzsMfjdYab5qk30FtEVonIahEZfwrrXo9Tg1CPtGtEJFlE5opIlf0/RWSGiCSKSGJeXp4Xh9HIrHoOUj+G8X+AnqNPWnzwaCm3vbaWnP1HeHn6eTa/tTHGZ+qqe6w/EA+MAW4AXhIRb8e3vh54y+P1/4DuqjoIpwYyp6qVVHW2qiaoakJMTExVWRqvHSvhk6fg7GvgvJMvwRwrLefH/04iJWs/s24cytCukT4opDHGOLwJFJmA56/6ODfNUwawQFVLVHUHsB0ncNS4rogMBvxVNakiTVULVPWo+/JfwLleHkvTcCAb5t4G7XrBlc9DpS6ule+6Htuvg48KaowxDm8CxVogXkR6iEggTg1gQaU883FqE4hINE5TVDqwBBgnIpEiEgmMc9Mq3MCJtQlExPNW5KuALV4fTWNXVuoMF370AEx7HYLCTspid10bYxqbWns9qWqpiNyL8wXvB7yiqptF5EkgUVUX8H1ASAHKgIdUtQBARH6LE2wAnlTVQo/NTwOuqLTL+0TkKqAUKASmn/bRNTaf/g52fQFT/wkd+p+02O66NsY0RnLiNeSmKSEhQRMTE31djJptXwL/mQZDb4WrXjhp8QfrM7n/7fVcMTCW/7thqN1QZ4ypdyKSpKoJteWzsZ4aQtEueH+GczPdhD+etNjuujbGNGYWKOpb6VF4b7ozhtO01yHgxHsh7K5rY0xjZ2M91bePfwN718F1/4Z2PU9YlFV82O66NsY0ehYo6tOm/8LXs2HkvdDvypMWP78slf2HS1l0/wi769oY02hZ01N9yU+FBfdBl+Fw6RMnLd5V8B3vJWVw4/CunNX+5G6yxhjTWFigqA/HDsG7t4B/EPzgVfA7uUnp/z5Jw7+VWDdYY0yjZ4GirqnChz9zpjS95l8QUXloK9iR/x3vr8vg5hHdaG9NTsaYRs4CRV1b9zpseAtG/wp6XVJllueXbSfI34+7RlttwhjT+FmgqEtZybDoIeg5Bkb/ssosabkH+GDDXm45vxsxYUENWjxjjDkdFijqypFi57pEmyi45mVoVfX9EH9dlkqbAD/uushqE8aYpsECRV1Qhfn3wL7dcO2rEBJdZbat2fv5cGMW00d1p11IYAMX0hhjTo/dR1EXVv8dti6EcU9VOVNdheeXpRIa6M+dF/asNo8xxjQ2VqM4U7vXwNLHoO8k58a6amzeW8xHm7K57YIetG1jtQljTNNhgeJMfFcAc38EEXEwedZJkxB5em5pKuHB/tx2QY8GLKAxxpw5CxRnYtHP4WAuXDsHWlc/82tyxj6Wbcnhzgt72nhOxpgmxwLF6do8z3mMmQmdzqkx63NLt9O2TQDTR3VvmLIZY0wdskBxOg7mwYc/h05DYNQDNWZdt7uIT7flcddFvQgLttqEMabp8SpQiMh4EdkmImkiMrOaPNNEJEVENovIfzzSbxWRVPdxq0f6Z+4217uP9m56kIi84+5rjYh0P7NDrAeLfu7Mez3lRfCruePYc0u3ExUSyC0juzVQ4Ywxpm7V2j1WRPyAWcBlQAawVkQWqGqKR5544GFglKoWeXzptwMeBxIABZLcdYvcVW9S1cpzmN4OFKnqWSJyPfAMcN0ZHWVd2vQ+pHwAYx+D9v1qzLp2ZyErU/P59RX9CAmynsjGmKbJmxrFMCBNVdNV9RjwNjC5Up47gVkVAUBVc930y4GlqlroLlsKjK9lf5OBOe7zucBYkRq6EzWkg3mw6BfQaSicf3+t2Z/9eDvRoUH8cITVJowxTZc3gaIzsMfjdYab5qk30FtEVonIahEZ7+W6r7rNTo96BIPj66hqKVAMRHl1NPWpYlRYL5ucvvw2n6/SC7hnTC9aB9r0psaYpquuLmb7A/HAGOAG4CURqb6/qOMmVR0IXOg+bj6VHYrIDBFJFJHEvLy80yjyKdr0X9iyAC5+BNr3rTGrqvLXpal0CA/ixuFd679sxhhTj7wJFJlAF4/XcW6apwxggaqWqOoOYDtO4Kh2XVWt+HsA+A9OE9cJ+xMRfyACKKhcKFWdraoJqpoQExPjxWGcgYO5zqiwnc+FkT+tNfuqtAK+3lnIvRefRXCA1SaMMU2bN4FiLRAvIj1EJBC4HlhQKc98nNoEIhKN0xSVDiwBxolIpIhEAuOAJSLi7+ZDRAKAScAmd1sLgIreUT8APlFVPc3jO3OqsPBBOPadV01Oqspflm6jU0Qw087rUmNeY4xpCmrtiqOqpSJyL86Xvh/wiqpuFpEngURVXcD3ASEFKAMeUtUCABH5LU6wAXhSVQtFJAQnYAS421wGvOTmeRl4Q0TSgEKcwOQ7m/7rDPh32ZMQ06fW7J9tz+Ob3fv4/dSBBPlbbcIY0/SJL3+s15WEhARNTKzcy7YOHMiBvw+HqLPgtiXVzjFRQVWZPGsVhd8d45OfjyHQ3+5nNMY0XiKSpKoJteWzb7LqHG9yOgST/15rkABYviWX5Ixi7hsbb0HCGNNs2LdZdTa+B9s+hEt+AzG9a82uqjy7dDvdotpw9ZDKvYeNMabpskBRlQPZTi+nuGEw8iderbJkczYpWfu5f2w8/n52Wo0xzYd9o1VW0eRUegSmeNfkVF6uPLc0lZ4xIVw1uFMDFNIYYxqOBYrKkt+FbYvgkkchOt6rVRZtymJbzgGrTRhjmiX7VvO0Pws+egi6jIARP/ZqlbJy5a/LUundIZRJg6w2YYxpfixQVFCFhQ9A6VFnWlMvmpwAFibvJS33IA9c2hu/Vo1j7EJjjKlLFigqbHgbti92hg+PPsurVUrLynl+WSp9Y8MYPyC2ngtojDG+YYECnCanxb9ympyG3+31aktTckjP/44HLu1NK6tNGGOaKQsUqvC/+6H0mNe9nCos2LCXmLAgLuvfoR4LaIwxvmWBYv1/IHUJXPo4RPXyerWDR0v5ZGsuV5wda9cmjDHNWssOFPv3wuKHoev5MOyuU1p1+ZYcjpaWM8numzDGNHMtO1BkJkGrVjD5b87fU7AwOYvY8GDO7RpZT4UzxpjGodZhxpu1fldCz4shKPSUVjtwpITPt+XxwxHd7CK2MabZa9k1CjjlIAFOb6djZeVMHNSxHgpkjDGNiwWK07AwOYvObVsztGtt04IbY0zTZ4HiFBUfKmFlah4TB3VExJqdjDHNn1eBQkTGi8g2EUkTkZnV5JkmIikisllE/uORfquIpLqPW920NiLyoYhsdfM/7ZF/uojkich693HHmR5kXVqSkk1JmTJxoDU7GWNahlovZouIHzALuAzIANaKyAJVTfHIEw88DIxS1SIRae+mtwMeBxIABZJEZAFwFPizqn4qIoHAchGZoKofuZt8R1XvrbvDrDsfJmfRtV0bBsVF+LooxhjTILypUQwD0lQ1XVWPAW8DkyvluROYpapFAKqa66ZfDixV1UJ32VJgvKoeUtVP3bzHgHVA3JkfTv0q+u4Yq9LyrdnJGNOieBMoOgN7PF5nuGmeegO9RWSViKwWkfHerisibYErgeUeydeISLKIzBWRLl6UsUEs3pxNabk1OxljWpa6upjtD8QDY4AbgJfcAFAjEfEH3gJeUNV0N/l/QHdVHYRTA5lTzbozRCRRRBLz8vLq4BBq92FyFj2iQxjQKbxB9meMMY2BN4EiE/D8VR/npnnKABaoaomq7gC24wSO2tadDaSq6l8rElS1QFWPui//BZxbVaFUdbaqJqhqQkxMjBeHcWbyDx7ly2/zmTjQmp2MMS2LN4FiLRAvIj3cC8/XAwsq5ZmPU5tARKJxmqLSgSXAOBGJFJFIYJybhog8BUQAD3huSEQ823WuArac4jHVi8WbsilXmDTYmp2MMS1Lrb2eVLVURO7F+YL3A15R1c0i8iSQqKoL+D4gpABlwEOqWgAgIr/FCTYAT6pqoYjEAb8GtgLr3F/of1PVfwH3ichVQClQCEyvu8M9fQuT93JW+1D6dAjzdVGMMaZBiar6ugxnLCEhQRMTE+tt+7n7jzD8D8u575J4Hrysd73txxhjGpKIJKlqQm357M5sL3y0KRtVmGRjOxljWiALFF5YmLyXPh3CiLdmJ2NMC2SBohZZxYdZu7PIahPGmBbLAkUtFm3MBrAhxY0xLZYFilosTN5L/47h9Iw59XkrjDGmObBAUYOMokN8s3uf3TthjGnRLFDUYNHGLAAmDezk45IYY4zvWKCowcLkLAbFRdA1qo2vi2KMMT5jgaIauwsOkZxRbL2djDEtngWKaizcuBeAK2xIcWNMC2eBohofJmcxpGtb4iKt2ckY07JZoKjCjvzv2Lx3P5MG2UVsY4yxQFGFhRsqmp1ifVwSY4zxPQsUVfhwYxbndY+kY0RrXxfFGGN8zgJFJWm5B9iafcDmxTbGGJcFikoWJmchYr2djDGmggUKD6rKwuQshvdoR/vwYF8XxxhjGgWvAoWIjBeRbSKSJiIzq8kzTURSRGSziPzHI/1WEUl1H7d6pJ8rIhvdbb4g7nyoItJORJa6+Ze6c203iG05B0jLPchE6+1kjDHH1RooRMQPmAVMAPoDN4hI/0p54oGHgVGqOgB4wE1vBzwODAeGAY97fPG/CNwJxLuP8W76TGC5qsYDy93XDeLD5CxaCUw423o7GWNMBW9qFMOANFVNV9VjwNvA5Ep57gRmqWoRgKrmuumXA0tVtdBdthQYLyIdgXBVXa3OpN2vA1PcdSYDc9znczzS61VFs9P5vaKJDg1qiF0aY0yT4E2g6Azs8Xid4aZ56g30FpFVIrJaRMbXsm5n93lV2+ygqlnu82yggxdlPGOb9+5nR/53NkGRMcZU4l+H24kHxgBxwAoRGXimG1VVFRGtapmIzABmAHTt2vVMd8WHG7PwayWMH2DNTsYY48mbGkUm0MXjdZyb5ikDWKCqJaq6A9iOEziqWzfTfV7VNnPcpincv7lUQVVnq2qCqibExMR4cRjVc5qd9jLqrGgiQwLPaFvGGNPceBMo1gLxItJDRAKB64EFlfLMx6lNICLROE1R6cASYJyIRLoXsccBS9ympf0iMsLt7XQL8IG7rQVARe+oWz3S683GzGL2FB62IcWNMaYKtTY9qWqpiNyL86XvB7yiqptF5EkgUVUX8H1ASAHKgIdUtQBARH6LE2wAnlTVQvf5PcBrQGvgI/cB8DTwrojcDuwCpp35YdZsYXIWAX7C5f2t2ckYYyoTp9NR05aQkKCJiYmnta6qcsEzn9InNoxXpp9XxyUzxpjGS0SSVDWhtnwt/s7sb/bsI3OfNTsZY0x1WnygWLghi0D/VlzWv0F64RpjTJPTogNFebmyaGMWo3vHEBYc4OviGGNMo9SiA0XS7iKy9x+xZidjjKlBiw4UAozuHcPYftbsZIwx1amrO7ObpITu7Zhz2zBfF8MYYxq1Fl2jMMYYUzsLFMYYY2pkgcIYY0yNLFAYY4ypkQUKY4wxNbJAYYwxpkYWKIwxxtTIAoUxxpgaNYthxkUkD2fuitMRDeTXYXHqWmMvHzT+Mlr5zoyV78w05vJ1U9VapwhtFoHiTIhIojfjsftKYy8fNP4yWvnOjJXvzDT28nnDmp6MMcbUyAKFMcaYGlmggNm+LkAtGnv5oPGX0cp3Zqx8Z6axl69WLf4ahTHGmJpZjcIYY0yNWkygEJHxIrJNRNJEZGYVy4NE5B13+RoR6d6AZesiIp+KSIqIbBaR+6vIM0ZEikVkvft4rKHK5+5/p4hsdPedWMVyEZEX3POXLCJDG7BsfTzOy3oR2S8iD1TK0+DnT0ReEZFcEdnkkdZORJaKSKr7N7KadW9186SKyK0NWL4/ichW9384T0TaVrNuje+HeizfEyKS6fF/vKKadWv8vNdj+d7xKNtOEVlfzbr1fv7qlKo2+wfgB3wL9AQCgQ1A/0p57gH+4T6/HninAcvXERjqPg8DtldRvjHAQh+ew51AdA3LrwA+wpk4cASwxof/62yc/uE+PX/ARcBQYJNH2h+Bme7zmcAzVazXDkh3/0a6zyMbqHzjAH/3+TNVlc+b90M9lu8J4BdevAdq/LzXV/kqLf8L8Jivzl9dPlpKjWIYkKaq6ap6DHgbmFwpz2Rgjvt8LjBWRKQhCqeqWaq6zn1+ANgCdG6IfdehycDr6lgNtBURX0xGPhb4VlVP9wbMOqOqK4DCSsme77M5wJQqVr0cWKqqhapaBCwFxjdE+VT1Y1UtdV+uBuLqer/equb8ecObz/sZq6l87nfHNOCtut6vL7SUQNEZ2OPxOoOTv4iP53E/KMVAVIOUzoPb5DUEWFPF4pEiskFEPhKRAQ1aMFDgYxFJEpEZVSz35hw3hOup/sPpy/NXoYOqZrnPs4GqJmxvLOfyNpxaYlVqez/Up3vdprFXqmm6awzn70IgR1VTq1nuy/N3ylpKoGgSRCQU+C/wgKrur7R4HU5zymDg/4D5DVy8C1R1KDAB+ImIXNTA+6+ViAQCVwHvVbHY1+fvJOq0QTTKboci8mugFHizmiy+ej+8CPQCzgGycJp3GqMbqLk20eg/T55aSqDIBLp4vI5z06rMIyL+QARQ0CClc/YZgBMk3lTV9ysvV9X9qnrQfb4ICBCR6IYqn6pmun9zgXk41XtP3pzj+jYBWKeqOZUX+Pr8ecipaJJz/+ZWkcen51JEpgOTgJvcYHYSL94P9UJVc1S1TFXLgZeq2a+vz58/cDXwTnV5fHX+TldLCRRrgXgR6eH+6rweWFApzwKgonfJD4BPqvuQ1DW3PfNlYIuqPltNntiKayYiMgznf9cggUxEQkQkrOI5zgXPTZWyLQBucXs/jQCKPZpYGkq1v+J8ef4q8Xyf3Qp8UEWeJcA4EYl0m1bGuWn1TkTGA78ErlLVQ9Xk8eb9UF/l87zuNbWa/Xrzea9PlwJbVTWjqoW+PH+nzddX0xvqgdMrZztOb4hfu2lP4nwgAIJxmizSgK+Bng1YtgtwmiCSgfXu4wrgbuBuN8+9wGacHhyrgfMbsHw93f1ucMtQcf48yyfALPf8bgQSGvj/G4LzxR/hkebT84cTtLKAEpx28ttxrnstB1KBZUA7N28C8C+PdW9z34tpwI8asHxpOO37Fe/Dip6AnYBFNb0fGqh8b7jvr2ScL/+Olcvnvj7p894Q5XPTX6t433nkbfDzV5cPuzPbGGNMjVpK05MxxpjTZIHCGGNMjSxQGGOMqZEFCmOMMTWyQGGMMaZGFiiMMcbUyAKFMcaYGlmgMMYYU6P/D8cXMM9GWzMBAAAAAElFTkSuQmCC\n",
       "text/plain": [
        "<Figure size 432x288 with 1 Axes>"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
    ],
    "source": [
     "pd.DataFrame(training.history)[['auc_roc', 'val_auc_roc']].plot();"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
    "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
     "name": "ipython",
     "version": 3
    },
    "file_extension": ".py",
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.6.8"
   },
   "toc": {
    "base_numbering": 1,
    "nav_menu": {},
    "number_sections": true,
    "sideBar": true,
    "skip_h1_title": true,
    "title_cell": "Table of Contents",
    "title_sidebar": "Contents",
    "toc_cell": false,
    "toc_position": {},
    "toc_section_display": true,
    "toc_window_display": true
   }
  },
  "nbformat": 4,
  "nbformat_minor": 2
 }