ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
06_sentiment_analysis_yelp.ipynb
(105754B)
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "# Text classification and sentiment analysis: Yelp Reviews"
8 ]
9 },
10 {
11 "cell_type": "markdown",
12 "metadata": {},
13 "source": [
14 "Once text data has been converted into numerical features using the natural language processing techniques discussed in the previous sections, text classification works just like any other classification task.\n",
15 "\n",
16 "In this notebook, we will apply these preprocessing technique to Yelp business reviews to classify them by review scores and sentiment polarity."
17 ]
18 },
19 {
20 "cell_type": "markdown",
21 "metadata": {},
22 "source": [
23 "## Imports"
24 ]
25 },
26 {
27 "cell_type": "code",
28 "execution_count": 1,
29 "metadata": {
30 "ExecuteTime": {
31 "end_time": "2018-11-26T06:37:13.006374Z",
32 "start_time": "2018-11-26T06:37:12.515786Z"
33 }
34 },
35 "outputs": [],
36 "source": [
37 "%matplotlib inline\n",
38 "import warnings\n",
39 "from collections import Counter, OrderedDict\n",
40 "from pathlib import Path\n",
41 "\n",
42 "import numpy as np\n",
43 "import pandas as pd\n",
44 "from pandas.io.json import json_normalize\n",
45 "import pyarrow as pa \n",
46 "import pyarrow.parquet as pq\n",
47 "from fastparquet import ParquetFile \n",
48 "from scipy import sparse\n",
49 "from scipy.spatial.distance import pdist, squareform\n",
50 "\n",
51 "# Visualization\n",
52 "import matplotlib.pyplot as plt\n",
53 "from matplotlib.ticker import FuncFormatter, ScalarFormatter\n",
54 "import seaborn as sns\n",
55 "\n",
56 "# spacy, textblob and nltk for language processing\n",
57 "from textblob import TextBlob, Word\n",
58 "\n",
59 "# sklearn for feature extraction & modeling\n",
60 "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
61 "from sklearn.model_selection import train_test_split\n",
62 "from sklearn.naive_bayes import MultinomialNB\n",
63 "from sklearn.linear_model import LogisticRegression\n",
64 "from sklearn.metrics import roc_auc_score, roc_curve, accuracy_score, confusion_matrix\n",
65 "from sklearn.externals import joblib\n",
66 "\n",
67 "import lightgbm as lgb\n",
68 "\n",
69 "import json\n",
70 "from time import clock, time"
71 ]
72 },
73 {
74 "cell_type": "code",
75 "execution_count": 2,
76 "metadata": {
77 "ExecuteTime": {
78 "end_time": "2018-11-26T06:37:13.010613Z",
79 "start_time": "2018-11-26T06:37:13.007802Z"
80 }
81 },
82 "outputs": [],
83 "source": [
84 "plt.style.use('fivethirtyeight')\n",
85 "warnings.filterwarnings('ignore')"
86 ]
87 },
88 {
89 "cell_type": "markdown",
90 "metadata": {},
91 "source": [
92 "## Yelp Challenge: business reviews dataset"
93 ]
94 },
95 {
96 "cell_type": "markdown",
97 "metadata": {},
98 "source": [
99 "Finally, we apply sentiment analysis to the significantly larger Yelp business review dataset with five outcome classes. \n",
100 "\n",
101 "The data consists of several files with information on the business, the user, the review and other aspects that Yelp provides to encourage data science innovation. \n",
102 "\n",
103 "We will use around six million reviews produced over the 2010-2018 period. In addition to the text features resulting from the review texts, we will also use other information submitted with the review about the user. "
104 ]
105 },
106 {
107 "cell_type": "markdown",
108 "metadata": {},
109 "source": [
110 "The [Yelp](https://www.yelp.com/dataset) dataset covers a subset of Yelp's businesses, reviews, and user data. \n",
111 "\n",
112 "You can download the data come in json format after accepting the license. It contains 3.6GB (compressed) and around 9GB (uncompressed) of text data. \n",
113 "\n",
114 "After download, extract the user.json and reviews.json files into to `data/yelp/json`"
115 ]
116 },
117 {
118 "cell_type": "markdown",
119 "metadata": {},
120 "source": [
121 "### Set up data directories"
122 ]
123 },
124 {
125 "cell_type": "markdown",
126 "metadata": {},
127 "source": [
128 "We parse the json files and store the result in parquet format in our cental data directory so we can reuse the cleaned data. You can remove the large json files after parsing."
129 ]
130 },
131 {
132 "cell_type": "code",
133 "execution_count": null,
134 "metadata": {},
135 "outputs": [],
136 "source": [
137 "data_dir = Path('../data')"
138 ]
139 },
140 {
141 "cell_type": "code",
142 "execution_count": 3,
143 "metadata": {
144 "ExecuteTime": {
145 "end_time": "2018-11-26T06:37:18.194551Z",
146 "start_time": "2018-11-26T06:37:18.192238Z"
147 }
148 },
149 "outputs": [],
150 "source": [
151 "yelp_dir = Path('data', 'yelp')\n",
152 "parquet_dir = data_dir / 'yelp'\n",
153 "if not parquet_dir.exists():\n",
154 " parquet_dir.mkdir(exist_ok=True)\n",
155 "text_features_dir = yelp_dir / 'text_features'\n",
156 "if not text_features_dir.exists():\n",
157 " text_features_dir.mkdir(exist_ok=True) "
158 ]
159 },
160 {
161 "cell_type": "markdown",
162 "metadata": {},
163 "source": [
164 "### Parse json and store as parquet files"
165 ]
166 },
167 {
168 "cell_type": "code",
169 "execution_count": 4,
170 "metadata": {
171 "ExecuteTime": {
172 "end_time": "2018-11-21T16:31:03.603964Z",
173 "start_time": "2018-11-21T16:25:46.210795Z"
174 },
175 "scrolled": false
176 },
177 "outputs": [
178 {
179 "name": "stdout",
180 "output_type": "stream",
181 "text": [
182 "review\n",
183 "user\n"
184 ]
185 }
186 ],
187 "source": [
188 "for file in ['review', 'user']:\n",
189 " print(file)\n",
190 " json_file = yelp_dir / 'json' / f'{file}.json'\n",
191 " parquet_file = parquet_dir / f'{file}.parquet'\n",
192 "\n",
193 " data = json_file.read_text(encoding='utf-8')\n",
194 " json_data = '[' + ','.join([l.strip()\n",
195 " for l in data.split('\\n') if l.strip()]) + ']\\n'\n",
196 " data = json.loads(json_data)\n",
197 " df = json_normalize(data)\n",
198 " if file == 'review':\n",
199 " df.date = pd.to_datetime(df.date)\n",
200 " latest = df.date.max()\n",
201 " df['year'] = df.date.dt.year\n",
202 " df['month'] = df.date.dt.month\n",
203 " df = df.drop(['date', 'business_id', 'review_id'], axis=1)\n",
204 " if file == 'user':\n",
205 " df.yelping_since = pd.to_datetime(df.yelping_since)\n",
206 " df = (df.assign(member_yrs=lambda x: (latest - x.yelping_since)\n",
207 " .dt.days.div(365).astype(int))\n",
208 " .drop(['elite', 'friends', 'name', 'yelping_since'], axis=1))\n",
209 " df.dropna(how='all', axis=1).to_parquet(parquet_file)\n"
210 ]
211 },
212 {
213 "cell_type": "code",
214 "execution_count": 5,
215 "metadata": {
216 "ExecuteTime": {
217 "end_time": "2018-11-21T16:35:47.922755Z",
218 "start_time": "2018-11-21T16:35:47.140603Z"
219 },
220 "scrolled": false
221 },
222 "outputs": [
223 {
224 "name": "stdout",
225 "output_type": "stream",
226 "text": [
227 "<class 'pandas.core.frame.DataFrame'>\n",
228 "RangeIndex: 1637138 entries, 0 to 1637137\n",
229 "Data columns (total 19 columns):\n",
230 "average_stars 1637138 non-null float64\n",
231 "compliment_cool 1637138 non-null int64\n",
232 "compliment_cute 1637138 non-null int64\n",
233 "compliment_funny 1637138 non-null int64\n",
234 "compliment_hot 1637138 non-null int64\n",
235 "compliment_list 1637138 non-null int64\n",
236 "compliment_more 1637138 non-null int64\n",
237 "compliment_note 1637138 non-null int64\n",
238 "compliment_photos 1637138 non-null int64\n",
239 "compliment_plain 1637138 non-null int64\n",
240 "compliment_profile 1637138 non-null int64\n",
241 "compliment_writer 1637138 non-null int64\n",
242 "cool 1637138 non-null int64\n",
243 "fans 1637138 non-null int64\n",
244 "funny 1637138 non-null int64\n",
245 "review_count 1637138 non-null int64\n",
246 "useful 1637138 non-null int64\n",
247 "user_id 1637138 non-null object\n",
248 "member_yrs 1637138 non-null int64\n",
249 "dtypes: float64(1), int64(17), object(1)\n",
250 "memory usage: 237.3+ MB\n"
251 ]
252 }
253 ],
254 "source": [
255 "user = pd.read_parquet(parquet_dir / 'user.parquet')\n",
256 "user.info(null_counts=True)"
257 ]
258 },
259 {
260 "cell_type": "code",
261 "execution_count": 6,
262 "metadata": {
263 "ExecuteTime": {
264 "end_time": "2018-11-21T16:35:51.985181Z",
265 "start_time": "2018-11-21T16:35:51.970430Z"
266 }
267 },
268 "outputs": [
269 {
270 "data": {
271 "text/html": [
272 "<div>\n",
273 "<style scoped>\n",
274 " .dataframe tbody tr th:only-of-type {\n",
275 " vertical-align: middle;\n",
276 " }\n",
277 "\n",
278 " .dataframe tbody tr th {\n",
279 " vertical-align: top;\n",
280 " }\n",
281 "\n",
282 " .dataframe thead th {\n",
283 " text-align: right;\n",
284 " }\n",
285 "</style>\n",
286 "<table border=\"1\" class=\"dataframe\">\n",
287 " <thead>\n",
288 " <tr style=\"text-align: right;\">\n",
289 " <th></th>\n",
290 " <th>average_stars</th>\n",
291 " <th>compliment_cool</th>\n",
292 " <th>compliment_cute</th>\n",
293 " <th>compliment_funny</th>\n",
294 " <th>compliment_hot</th>\n",
295 " <th>compliment_list</th>\n",
296 " <th>compliment_more</th>\n",
297 " <th>compliment_note</th>\n",
298 " <th>compliment_photos</th>\n",
299 " <th>compliment_plain</th>\n",
300 " <th>compliment_profile</th>\n",
301 " <th>compliment_writer</th>\n",
302 " <th>cool</th>\n",
303 " <th>fans</th>\n",
304 " <th>funny</th>\n",
305 " <th>review_count</th>\n",
306 " <th>useful</th>\n",
307 " <th>user_id</th>\n",
308 " <th>member_yrs</th>\n",
309 " </tr>\n",
310 " </thead>\n",
311 " <tbody>\n",
312 " <tr>\n",
313 " <th>0</th>\n",
314 " <td>4.03</td>\n",
315 " <td>1</td>\n",
316 " <td>0</td>\n",
317 " <td>1</td>\n",
318 " <td>2</td>\n",
319 " <td>0</td>\n",
320 " <td>0</td>\n",
321 " <td>1</td>\n",
322 " <td>0</td>\n",
323 " <td>1</td>\n",
324 " <td>0</td>\n",
325 " <td>2</td>\n",
326 " <td>25</td>\n",
327 " <td>5</td>\n",
328 " <td>17</td>\n",
329 " <td>95</td>\n",
330 " <td>84</td>\n",
331 " <td>l6BmjZMeQD3rDxWUbiAiow</td>\n",
332 " <td>5</td>\n",
333 " </tr>\n",
334 " <tr>\n",
335 " <th>1</th>\n",
336 " <td>3.63</td>\n",
337 " <td>1</td>\n",
338 " <td>0</td>\n",
339 " <td>1</td>\n",
340 " <td>1</td>\n",
341 " <td>0</td>\n",
342 " <td>0</td>\n",
343 " <td>0</td>\n",
344 " <td>0</td>\n",
345 " <td>0</td>\n",
346 " <td>0</td>\n",
347 " <td>0</td>\n",
348 " <td>16</td>\n",
349 " <td>4</td>\n",
350 " <td>22</td>\n",
351 " <td>33</td>\n",
352 " <td>48</td>\n",
353 " <td>4XChL029mKr5hydo79Ljxg</td>\n",
354 " <td>5</td>\n",
355 " </tr>\n",
356 " <tr>\n",
357 " <th>2</th>\n",
358 " <td>3.71</td>\n",
359 " <td>0</td>\n",
360 " <td>0</td>\n",
361 " <td>0</td>\n",
362 " <td>0</td>\n",
363 " <td>0</td>\n",
364 " <td>0</td>\n",
365 " <td>1</td>\n",
366 " <td>0</td>\n",
367 " <td>0</td>\n",
368 " <td>0</td>\n",
369 " <td>0</td>\n",
370 " <td>10</td>\n",
371 " <td>0</td>\n",
372 " <td>8</td>\n",
373 " <td>16</td>\n",
374 " <td>28</td>\n",
375 " <td>bc8C_eETBWL0olvFSJJd0w</td>\n",
376 " <td>5</td>\n",
377 " </tr>\n",
378 " <tr>\n",
379 " <th>3</th>\n",
380 " <td>4.85</td>\n",
381 " <td>0</td>\n",
382 " <td>0</td>\n",
383 " <td>0</td>\n",
384 " <td>1</td>\n",
385 " <td>0</td>\n",
386 " <td>0</td>\n",
387 " <td>0</td>\n",
388 " <td>0</td>\n",
389 " <td>2</td>\n",
390 " <td>0</td>\n",
391 " <td>1</td>\n",
392 " <td>14</td>\n",
393 " <td>5</td>\n",
394 " <td>4</td>\n",
395 " <td>17</td>\n",
396 " <td>30</td>\n",
397 " <td>dD0gZpBctWGdWo9WlGuhlA</td>\n",
398 " <td>4</td>\n",
399 " </tr>\n",
400 " <tr>\n",
401 " <th>4</th>\n",
402 " <td>4.08</td>\n",
403 " <td>80</td>\n",
404 " <td>0</td>\n",
405 " <td>80</td>\n",
406 " <td>28</td>\n",
407 " <td>1</td>\n",
408 " <td>1</td>\n",
409 " <td>16</td>\n",
410 " <td>5</td>\n",
411 " <td>57</td>\n",
412 " <td>0</td>\n",
413 " <td>25</td>\n",
414 " <td>665</td>\n",
415 " <td>39</td>\n",
416 " <td>279</td>\n",
417 " <td>361</td>\n",
418 " <td>1114</td>\n",
419 " <td>MM4RJAeH6yuaN8oZDSt0RA</td>\n",
420 " <td>5</td>\n",
421 " </tr>\n",
422 " </tbody>\n",
423 "</table>\n",
424 "</div>"
425 ],
426 "text/plain": [
427 " average_stars compliment_cool compliment_cute compliment_funny \\\n",
428 "0 4.03 1 0 1 \n",
429 "1 3.63 1 0 1 \n",
430 "2 3.71 0 0 0 \n",
431 "3 4.85 0 0 0 \n",
432 "4 4.08 80 0 80 \n",
433 "\n",
434 " compliment_hot compliment_list compliment_more compliment_note \\\n",
435 "0 2 0 0 1 \n",
436 "1 1 0 0 0 \n",
437 "2 0 0 0 1 \n",
438 "3 1 0 0 0 \n",
439 "4 28 1 1 16 \n",
440 "\n",
441 " compliment_photos compliment_plain compliment_profile compliment_writer \\\n",
442 "0 0 1 0 2 \n",
443 "1 0 0 0 0 \n",
444 "2 0 0 0 0 \n",
445 "3 0 2 0 1 \n",
446 "4 5 57 0 25 \n",
447 "\n",
448 " cool fans funny review_count useful user_id member_yrs \n",
449 "0 25 5 17 95 84 l6BmjZMeQD3rDxWUbiAiow 5 \n",
450 "1 16 4 22 33 48 4XChL029mKr5hydo79Ljxg 5 \n",
451 "2 10 0 8 16 28 bc8C_eETBWL0olvFSJJd0w 5 \n",
452 "3 14 5 4 17 30 dD0gZpBctWGdWo9WlGuhlA 4 \n",
453 "4 665 39 279 361 1114 MM4RJAeH6yuaN8oZDSt0RA 5 "
454 ]
455 },
456 "execution_count": 6,
457 "metadata": {},
458 "output_type": "execute_result"
459 }
460 ],
461 "source": [
462 "user.head()"
463 ]
464 },
465 {
466 "cell_type": "code",
467 "execution_count": 8,
468 "metadata": {
469 "ExecuteTime": {
470 "end_time": "2018-11-21T16:36:18.752582Z",
471 "start_time": "2018-11-21T16:35:55.329305Z"
472 }
473 },
474 "outputs": [
475 {
476 "name": "stdout",
477 "output_type": "stream",
478 "text": [
479 "<class 'pandas.core.frame.DataFrame'>\n",
480 "RangeIndex: 6685900 entries, 0 to 6685899\n",
481 "Data columns (total 8 columns):\n",
482 "cool 6685900 non-null int64\n",
483 "funny 6685900 non-null int64\n",
484 "stars 6685900 non-null float64\n",
485 "text 6685900 non-null object\n",
486 "useful 6685900 non-null int64\n",
487 "user_id 6685900 non-null object\n",
488 "year 6685900 non-null int64\n",
489 "month 6685900 non-null int64\n",
490 "dtypes: float64(1), int64(5), object(2)\n",
491 "memory usage: 408.1+ MB\n"
492 ]
493 }
494 ],
495 "source": [
496 "review = pd.read_parquet(parquet_dir / 'review.parquet')\n",
497 "review.info(null_counts=True)"
498 ]
499 },
500 {
501 "cell_type": "markdown",
502 "metadata": {},
503 "source": [
504 "### Merge review and user files"
505 ]
506 },
507 {
508 "cell_type": "code",
509 "execution_count": 9,
510 "metadata": {
511 "ExecuteTime": {
512 "end_time": "2018-11-21T16:41:33.291590Z",
513 "start_time": "2018-11-21T16:37:54.568008Z"
514 }
515 },
516 "outputs": [
517 {
518 "name": "stdout",
519 "output_type": "stream",
520 "text": [
521 "<class 'pandas.core.frame.DataFrame'>\n",
522 "Int64Index: 6685900 entries, 0 to 6685899\n",
523 "Data columns (total 25 columns):\n",
524 "cool 6685900 non-null int64\n",
525 "funny 6685900 non-null int64\n",
526 "stars 6685900 non-null float64\n",
527 "text 6685900 non-null object\n",
528 "useful 6685900 non-null int64\n",
529 "year 6685900 non-null int64\n",
530 "month 6685900 non-null int64\n",
531 "average_stars 6685900 non-null float64\n",
532 "compliment_cool 6685900 non-null int64\n",
533 "compliment_cute 6685900 non-null int64\n",
534 "compliment_funny 6685900 non-null int64\n",
535 "compliment_hot 6685900 non-null int64\n",
536 "compliment_list 6685900 non-null int64\n",
537 "compliment_more 6685900 non-null int64\n",
538 "compliment_note 6685900 non-null int64\n",
539 "compliment_photos 6685900 non-null int64\n",
540 "compliment_plain 6685900 non-null int64\n",
541 "compliment_profile 6685900 non-null int64\n",
542 "compliment_writer 6685900 non-null int64\n",
543 "cool_user 6685900 non-null int64\n",
544 "fans 6685900 non-null int64\n",
545 "funny_user 6685900 non-null int64\n",
546 "review_count 6685900 non-null int64\n",
547 "useful_user 6685900 non-null int64\n",
548 "member_yrs 6685900 non-null int64\n",
549 "dtypes: float64(2), int64(22), object(1)\n",
550 "memory usage: 1.3+ GB\n"
551 ]
552 }
553 ],
554 "source": [
555 "combined = (review\n",
556 " .merge(user, on='user_id', how='left', suffixes=['', '_user'])\n",
557 " .drop('user_id', axis=1))\n",
558 "combined = combined[combined.stars > 0]\n",
559 "combined.info(null_counts=True)"
560 ]
561 },
562 {
563 "cell_type": "code",
564 "execution_count": 12,
565 "metadata": {
566 "ExecuteTime": {
567 "end_time": "2018-11-21T17:31:08.163423Z",
568 "start_time": "2018-11-21T17:27:07.206081Z"
569 }
570 },
571 "outputs": [],
572 "source": [
573 "combined.to_parquet(parquet_dir / 'combined.parquet')"
574 ]
575 },
576 {
577 "cell_type": "code",
578 "execution_count": 13,
579 "metadata": {
580 "ExecuteTime": {
581 "end_time": "2018-11-26T06:39:02.944907Z",
582 "start_time": "2018-11-26T06:38:38.732675Z"
583 },
584 "scrolled": false
585 },
586 "outputs": [],
587 "source": [
588 "combined = pd.read_parquet(parquet_dir / 'combined.parquet')"
589 ]
590 },
591 {
592 "cell_type": "markdown",
593 "metadata": {},
594 "source": [
595 "### Explore data"
596 ]
597 },
598 {
599 "cell_type": "markdown",
600 "metadata": {},
601 "source": [
602 "The following figure shows the number of reviews and the average number of stars per year."
603 ]
604 },
605 {
606 "cell_type": "code",
607 "execution_count": 14,
608 "metadata": {
609 "ExecuteTime": {
610 "end_time": "2018-11-26T06:40:38.345897Z",
611 "start_time": "2018-11-26T06:39:02.946238Z"
612 }
613 },
614 "outputs": [
615 {
616 "data": {
617 "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7wAAAE0CAYAAAAPPz+rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XlcVPX+x/HXAIJo6iACSoorFxUtd0RTcsncitxyqeuuuXXVm5ZeW81SW1xywS26Wt4szVIKl0zEXdQsTM0wzF3MBURkZ35/8GNyHEQwBBnez8fjPq5zzvec7+c7PvLM53w3Q2xsrAkRERERERERG2NX2AGIiIiIiIiI3A9KeEVERERERMQmKeEVERERERERm6SEV0RERERERGySEl4RERERERGxSUp4RURERERExCYp4RUpRF26dMFoNBZ2GCIiIiIiNkkJrxQLRqPR4n8uLi54eXnRoUMHFi9eTGpqamGHKPdJcnIyLVq0wGg0smnTpjuW27RpE0ajkZYtW5KcnFyAEYqIyO0yMjJYsWIFXbt2pXr16lSoUIGaNWvi7+/PyJEj+eqrryzKr1y5EqPRyPTp0wspYhF5UDkUdgAiBemVV14BID09ndOnTxMSEkJERATbtm3j888/L/B4Fi1aRGJiYoHXW5w4OTmxbNky2rZty5gxY9i9ezdubm4WZS5fvsyLL75IyZIlWbZsGU5OToUUrYiIZGRk0LdvXzZt2kTZsmXp2LEjnp6exMXFcfLkSdavX8+BAwfo0aNHYYcqIkWAEl4pViZPnmzx+fjx47Rp04YNGzawc+dOHnvssQKNp0qVKgVaX3FVt25d3nzzTSZNmsSYMWP44osvLM6PGTOGS5cuMXPmTOrUqVNIUYqICMCaNWvYtGkT9erV47vvvqNcuXIW55OSktizZ08hRSciRY2GNEux5uPjQ8uWLQE4dOhQtmV++uknBg8eTO3atXFzc8PHx4fhw4cTHR1tUa5Hjx4YjUZ++umnbO+zZcsWjEYjw4cPNx/LaQ7v9u3b6dOnDzVr1sTNzY169erx0ksvERMTY1FuyJAhGI1Gjh8/bnH8pZdeMg/RvZXJZKJGjRrUq1fP4thnn31Ghw4dqFmzJh4eHtStW5ennnqK5cuXZxvf7W4dThYREUFgYCBVqlShSpUq9OzZ847fS9awtSeffBIvLy88PDzw9/dn1qxZpKSkWJU3Go3Ur1+f2NhYJk6ciK+vL66urixcuDDH+F544QXat2/Ppk2bWLp0qfn4J598wsaNG3niiSd44YUXrK6Liopi1KhR1KtXDzc3N2rVqkX//v05fPiwVdmzZ8/y7rvv8sQTT+Dt7Y2bmxt16tRh2LBh/Pbbb1blf/vtN4xGIz169OD06dMMGzYMb29vXFxc2LJlS47tERGxVfv27QOgX79+VskuQMmSJWnTpo3588iRIxk9ejQAM2fOtJjCtGPHDgDi4uKYO3cuXbt2pU6dOri5uVGzZk369Oljru92d3vexMTEMGXKFJo0aYKnpydVqlShUaNGDB06NNtnRHayfgf88ccfzJ8/n6ZNm+Lh4YGvry9TpkwhPj4+2+tiYmKYNGkSjRo1wsPDg6pVq9KtWzfCw8Otyt76fN63bx/du3enatWqGI1GYmNj7xjbgAEDMBqN7Ny5M9vz27Ztw2g0MmTIEIvjcXFxvPPOO/j7+1OpUiUqV65Mx44d+eabb6zukZKSwpIlS+jZsyf16tXD3d2dqlWr8vTTT99xGlL9+vUxGo0kJSUxbdo0GjZsiJubG5MmTbpjW6R4Uw+vyP9zcLD+z+HLL79k1KhRODo60qlTJx5++GGio6P56quv2LhxI99++y2PPPIIkPlg/uGHH1i5ciUNGjSwulfWkOnnnnvurrHMmTOHN998ExcXFzp06ICHhwdHjhzh448/ZsOGDXz//fc8/PDDAAQEBPDVV1+xbds2fHx8zPfIeugdPXqUP//80zyMNzIykqtXr9KpUydz2TfffJO5c+fi5eXFM888Q7ly5YiJieGXX35h1apVDBgwILdfIwcPHmT27Nm0adOGYcOG8fvvvxMSEsKuXbv45ptv8PPzM5dNS0vj+eefZ+PGjdSqVYsePXrg5OTErl27mDp1KuHh4Xz11VdWfzcpKSk8/fTTXL9+nQ4dOuDo6Iinp2eOcRkMBhYuXEiLFi14/fXXad26NQ4ODrz66qtUqFCBBQsWWF2zZcsW+vfvT1paGk8++STVq1fn3LlzfPvtt2zevJkvv/yS1q1bW3zn8+fPp1WrVjRo0ABnZ2eioqJYu3YtGzduZPPmzdn2IP/555906NCBChUq0L17d5KTk7P9kSciUhyUL18egN9//z1X5bt06UJcXByhoaG0bNnSYrSWl5cXkPmCcdq0abRo0YInn3wSo9HImTNnCA0N5fvvv+fzzz+nQ4cOVve+0/Pm5s2bdOjQgVOnThEQEEDHjh0BOHfuHNu2baN169bUr18/122ePHkye/bsoVu3bpQtW5bvv/+eBQsWsHfvXkJDQy2m2hw5coRu3brx559/0rZtWzp37szVq1f57rvveOaZZ/joo4/45z//aVVHREQEs2bNokWLFvTv358LFy5gb29/x5iGDh3KunXr+OSTT7IdARccHAzAoEGDzMfOnz/PU089xe+//46/vz8DBw7k5s2bbN68mYEDB/LKK69YjLa7du0akyZNws/PjzZt2lChQgUuXrxIaGgovXv3Zs6cOQwcODDb+Pr3709kZCTt2rXDxcWFatWq3e1rlmLKEBsbayrsIETut6xe1NvfZJ44cYKAgAASEhLYtm2bRaIaHR1tfjsZGhpqkVDt2LGDZ555Bl9fX7Zv3w5kDrHy8fHB3t6eX3/9FUdHR3P5uLg4fHx8qFChApGRkdjZZQ6u6NKlC7t27bKIa9euXXTt2pUmTZqwevVqix7gVatWMWLECLp27cpnn30GwKlTp3j00Ufp3Lkz//vf/4DMB66vry9t2rQhLCyMjz/+2DzXad68ebz22mssWbKEZ599FoBq1arh7OzMgQMHKF26tMV3dOXKFVxdXe/6Ha9cudL8hv39999n2LBh5nPr1q1jwIABeHt7ExERgcFgMJd75513GDZsGDNmzDA/eDMyMhg/fjzLly9nxowZjBgxwnyvrO8jICCAzz//nFKlSt01tltt2LCBvn37Ur9+fUqUKMGPP/7IqlWrzD9Wbm13w4YNcXJyYsOGDdSqVct87pdffuGJJ57A1dWVQ4cOUaJECQAuXbpE6dKlrb7DAwcO0LVrV9q2bWv+O4LMH2DNmjUDMh/cs2fPzvHHh4hIcZCVxKSlpdGzZ086depEgwYNqF69uvn5cbusZ9DtCVWWuLg40tLSrJ5np0+fpn379pQrV479+/dbnMvpeRMaGkq/fv144YUXmDlzpsV16enpxMfH52oXhqzfAeXLlyc8PNw81Sk9PZ1//vOfhIaG8vrrr/Pvf//bfNzPz48zZ87w1VdfWSSiFy9epF27dly5coXIyEjc3d0tvhsgxwQyO/7+/vz+++8cPXqUChUqmI/HxMRQr149atSoYdFDHhgYyPbt21m6dCk9e/Y0H79+/Tpdu3bl8OHDhIeHmzsLkpOTuXz5svklfpbY2FiefPJJYmJiOHbsGM7OzuZz9evX58yZM9StW5eQkJBc/UaR4k1DmqVYmT59OtOnT2fatGkMHz6c1q1bk5CQwL/+9S+rXtmPP/6Y5ORk3n33Xavew1atWtGpUyciIyM5duwYkDnEqkePHly9epWNGzdalP/6669JSkqiT58+5mT3ThYtWoTJZGL27NlWD8s+ffrwyCOPsGHDBq5fvw5A1apVqVatGjt37iQ9PR34q3d3woQJlC5dmm3btpnvkXXu1p5JOzs7SpQokW0vd14fJDVq1LAa3hQYGIifnx9RUVHmB2NGRgaLFi3Czc2N6dOnWyR6dnZ2TJ06FYPBYDXfNsvbb7+d52QXoFOnTgwePJjDhw/z448/MmTIEKtkFzJ/IFy/fp0pU6ZYJLsA9erVo1+/fpw9e5bdu3ebj7u7u1sluwBNmjShefPmhIeHk5GRYXXe2dmZt956S8muiAjwyCOPsGTJEtzd3Vm9ejWDBw+mUaNGVKtWjd69e/PNN99gMuWtv6ZcuXLZPs+8vLwIDAwkKiqKM2fOZHttds+brGd5ds8he3v7PG85OGLECIt1Pezt7XnrrbcwGAzmF9wAmzdv5sSJEwwZMsSq17VixYq8+OKLJCUlsW7dOqs66tWrl6dkFzKnTaWkpFjEAPDpp5+Smppq0bt75MgRwsPD6dKli0WyC1C2bFkmTZqEyWRi9erV5uNOTk5WyS5kvmx4/vnniY2N5ccff8w2tv/85z9KdiVXNKRZipXb38ICvPbaa7z00ktWx7MSs927d/Pzzz9bnf/zzz+BzF66rGGq/fr1Izg4mP/97388/fTT5rJZw5n79et31xj37duHg4MDISEhhISEWJ1PSUkhPT2d6Ohoc5L++OOP89///pcff/yRpk2bEh4eTtmyZWnevDn+/v7mJDc1NZU9e/ZQp04dKlasaL7ns88+y6JFi2jWrBnPPPMM/v7++Pn54eLictd4b+fv759tUt+iRQv27dtHZGQkzZs358SJE1y5coXq1avz/vvvZ3uvrCHBt3NycsrTULHbvfPOO+ahWNOmTcu2zN69ewH4+eefs93m4uTJk0Dm339AQID5+Lfffsvy5cv5+eefuXr1KmlpaRbXXb9+3eqHUI0aNe7puxYRsVXdunWja9eu7Nixgz179nDkyBH27t3Lpk2b2LRpEx06dOCzzz6zGE11N3v37mXRokXs37+fP//802qdiAsXLlgtJnmn503Lli2pXLkyc+bM4dChQ3To0AE/Pz8effTRbF8e383t620AeHt74+7uTnR0NPHx8ZQpU8b82+Ts2bPZPpuy1hfJbt2IJk2a5Dmu3r1789Zbb/Hf//6XsWPHYjAYzGtvlCpVij59+pjLZsUWHx+fbWxXrlzJNrZjx47x0UcfsXv3bi5evGi1NeCFCxeyje1e2iPFkxJeKVayhg4nJiZy8OBBxo8fzzvvvEP16tXp3r27RdmrV68CMH/+/BzvmZCQYP5zkyZNqF27Nlu2bDHPm42Ojmbfvn34+/tTo0aNu8aYlSRll5zf6saNG+Y/BwQE8N///pfw8HCaNm3K9u3badmyJfb29gQEBLBlyxZOnjzJhQsXSEhIsOjdhcwEsEaNGnz22Wd89NFHzJ07Fzs7OwICApg6dWqeksusIVS3y5pDnNUznfX9njx58q5tze5edxrWlhu3Do269c+3unbtGpC5qFVObv37z5p7Xb58eR5//HEqV65MyZIlMRgMrF+/nmPHjmW7x6+Hh8e9NENExKaVKFGCtm3b0rZtWyBzZND69esZPXo0mzdvJjg42GLKS05CQkIYMGCAecGratWqUapUKezs7Ni5cye7du3K9t/nOz1vypQpw/fff8/MmTMJDQ01j6QqV64czz//PFOmTMnTKKScnp0xMTHmhDfr2bl+/XrWr19/x/vd+my6Wx05KVOmDH369GHp0qVs3bqVdu3asWXLFk6fPs3zzz9v8QI3K7bw8PBsF8/KLrb9+/fz9NNPk5aWRkBAAJ06daJMmTLY2dlx+PBhQkNDs/17AT07JfeU8Eqx5OzszGOPPcaaNWvw9/dn7NixtGzZ0uIfz7JlywKZCVleet/69u3LG2+8wRdffMGYMWPMczb79u2bq+vLli1LamrqHYdWZad169YYDAa2bdvGU089xYULFxg3bhyAufdx27Zt5rekt/ZIQubQqeHDhzN8+HCuXr3Knj17CAkJ4YsvvqBbt25ERESYFxG5m0uXLmV7PKtHPOt7zfr/jh07smrVqly3FfhbyW5uZcW3d+9eateufdfyycnJvP/++1SuXJmwsDCrvX6zVgrNTkG0R0SkqLOzs+OZZ57hl19+4YMPPmDbtm25TnjfffddHB0dCQsLs1jgEWDcuHHs2rUr2+ty+ve5UqVKzJkzh9mzZ/Pbb7+xa9cugoODWbBgAXFxcXd9YX6rS5cu4e3tbXU869lZpkwZ4K9n04oVKyxGkuXGvT5rhgwZwtKlSwkODqZdu3bmF8GDBw+2KJcV27Rp0xgzZkyu7v3BBx+QmJhISEgIrVq1sjg3a9YsQkND73itnp2SW5rDK8Va1apVGTt2LPHx8bzzzjsW55o2bQpgMUczN3r37o29vT2ff/45JpOJVatWUapUKbp165ar65s2bUp8fHyutzSAzHm2vr6+7N+/3zx/+PHHHwcyF3eoUKEC4eHhbN++HXt7+2yHTmUpX748Xbp0YdGiRfTo0YPLly+bh/fmxt69e7Odp5r1PWYtVPGPf/yDcuXKcfDgwWy3HypsWX//ud3r8eLFiyQkJODv72+V7MbFxfHLL7/ke4wiIsVRVvJ36zzerDUQstayuF10dDQ+Pj5WyW5GRkaennHZMRgM+Pj4MHjwYDZs2ICTkxPffvttnu6RXcIdFRXFpUuXqFGjhrnNeX025YfatWvTqlUrNm7cyIEDB9i8eTMNGjSgUaNGFuWyFmHMS2zR0dG4uLhYJbuQ/Xcici+U8EqxN2rUKFxdXVm5ciUnTpwwHx8+fDiOjo68+uqr2c6FSU9Pz7bXrmLFirRr144jR44wf/58zp49S9euXc0Pq7vJWklx3LhxnDt3zup8UlJStg+Txx9/nOTkZObPn4+np6f5oW4wGGjVqhXbtm3j4MGDNGrUyGLLm+TkZLZt22aVpJpMJvOb5ZIlS+YqdsjcRuLjjz+2OLZu3Tr27duHt7e3eVsiBwcHRowYwZ9//smECRO4efOm1b2yVposDAMGDKBs2bK8++672e7RbDKZLBYK8/T0pESJEhw8eNCiLSkpKUycONE8lFtERHK2Zs0awsLCsn15GhMTw4oVKwDLea9ZixedPXs223t6eXkRHR3N+fPnzcdMJhMzZszg119/zXOMR48e5Y8//rA6fvXqVVJTU/P03ITMBStvHdmVnp7OG2+8gclkstjOsHPnztSoUYNPPvnkjr2fWWtI5KehQ4eSnp7O888/T3p6ulXvLkCDBg1o2bIloaGhLF++PNuFxU6cOGHRTi8vL65du2b1UnjFihX88MMP+doGKb40pFmKvTJlyjBu3Dhee+013nnnHfNQHW9vbxYuXMjo0aPx9/enffv21KxZk/T0dM6dO8e+fftITk7m9OnTVvfs168fmzdvZurUqUDu9t7N0rp1a95++23eeOMNGjduzBNPPEG1atVISkrizJkz7N69Gy8vL6uN4AMCApg/fz5//vmnxSISkJkMf/311+Zyt0pMTOSZZ56hcuXKNG3alCpVqpCamsrOnTs5fPgwTZo0sZrzm5P27dvz6quvsmXLFnx9fc378Do7OzNv3jyLIUgTJ07k6NGjrFixgs2bN9O6dWsefvhhLl++zMmTJ9m7dy9Dhw419woXJDc3Nz755BMGDBhA27ZtCQgIoHbt2tjZ2XHu3DkOHDjAuXPnuHjxIvb29pQoUYKhQ4cSFBREy5Yt6dixI8nJyYSHh5t7fgvyjbyISFF14MABFi1ahIeHB82bN6dq1apA5jZ8mzdvJjExkWbNmllsf9esWTMeeugh1q5di6OjI5UrV8ZgMNC7d2+8vLwYNWoU48ePJyAggKeffhoHBwf27dvH8ePH6dixo9XuCnezbds2pkyZQtOmTfnHP/6Bu7s7MTExhIaGkpGRYZ5WlFvNmzenVatWFvvwHj16lEaNGlkMDy5RogSfffYZ3bt3p1+/fjRp0oRHH32U0qVLc+7cOSIjI4mKimL79u25noqUG126dMHT05Pz589TtmxZ81aHt1u2bBmBgYGMHTuWxYsX07RpU1xcXDh//jy//vorkZGRfPbZZ+bFwUaOHMkPP/xAp06deOaZZyhbtiyHDh1i7969BAYGZrvatEheKeEVIfPN5cKFC/nmm28YN24cjz76KAA9e/akXr16LFiwgPDwcMLCwihZsiQVK1akffv2BAYGZnu/Tp064eLiwrVr16hcuXK2Q3Vy8uKLL9K8eXMWLVrEnj172LhxIw899BCVKlWiV69eVgtsQeYqyCVKlCA1NdU8nDnLrUnu7clr6dKlmTp1Kjt27GD//v1s2LABZ2dnqlatyrRp0xg0aFCeVpxs3LgxL7/8MtOmTWPJkiUAtGnThtdee81q6ycHBwdWrFjBV199xcqVK/n++++5ceMG5cuXp0qVKowfP94qeS9I7dq1Y+fOnSxYsICtW7eyd+9eSpQogYeHB/7+/nTu3BknJydz+alTp+Lh4cHKlSsJDg7GaDTStm1bXnvtNV599dVCa4eISFHy4osv4u3tTVhYGEePHiUsLIybN2/i4uJi3k3g+eefN++BDpmLRa1cuZLp06ezdu1a88KOzZs3x8vLi0GDBuHo6EhQUBCff/45JUuWxN/fnwULFrB+/fo8J7zt2rXj7Nmz5mf09evXcXd3p1mzZowYMYI2bdrk6X7vvvsuISEhLF++nNOnT1OhQgVGjRrF5MmTLZ4zAHXr1mXXrl0EBQURGhpqnkLl4eFB7dq1zd9ffnJwcKB3797Mnj2b3r17Z7sFH2TOaw4LC2Pp0qWsW7eOr776itTUVNzd3alVqxYzZsyw2E6pffv2rFq1ig8++ICvv/4aOzs7GjduTEhICH/88YcSXskXhtjY2LxtZCYiko2sje1feeUVJk+eXNjhiIiIPPC6dOnCrl27+Pnnn8092Q+qbt26ERYWZt7eUKSo0BxeERERERG5o59++omwsDBatWqlZFeKHA1pFhERERERK0uWLOHChQusWrUKg8Gg6TlSJCnhFRERERERK/PmzePcuXNUr16dRYsWmXdaEClKNIdXREREREREbJLm8IqIiIiIiIhNUsIrIiIiIiIiNkkJr4iIiIiIiNgkJbz3QVRUlOoqAvUUZF222KaCrEttKhp12WKbCrouyV+29HdnS20BtedBp/Y8uGypLVAw7VHCKyIiIiIiIjZJCa+IiIiIiIjYJCW8IiIiIiIiYpOU8IqIiIiIiIhNUsIrIiIiIiIiNkkJr4iIiIiIiNgkJbwiIiIiIiJik3KV8O7atYs+ffpQp04djEYjK1euNJ9LTU3ljTfeoEWLFnh6euLj48PQoUM5c+aMxT2Sk5OZOHEiNWrUwNPTkz59+nDu3DmLMmfOnKF37954enpSo0YNXn75ZVJSUizK7Ny5k4CAADw8PHj00UcJDg62infZsmU88sgjeHh4EBAQwO7du/Mci4iIiBRtJlNhRyAiIoXNITeFEhISqFu3Ln379mXEiBEW527evMnPP//MhAkTqF+/PtevX+fVV1+lZ8+e7Nq1CweHzComT55MaGgoH3/8MS4uLkyZMoXevXsTHh6Ovb096enp9O7dGxcXF0JDQ7l27RojR47EZDLx/vvvA/DHH3/w7LPP8txzz7FkyRL27t3LSy+9hKurK4GBgQCsXbuWSZMm8eGHH9K8eXOWLVtGr1692Lt3L1WqVMlVLCIiUviMn+T0IrIU7Mz+fOygh+9PQFKkpGWYiEmBfxR2ICIiUqhylfB26NCBDh06ADBq1CiLc+XKleObb76xODZ79myaN2/O8ePH8fX1JS4ujk8//ZQFCxbQpk0bABYvXkz9+vXZtm0b7dq1Y+vWrRw7dozDhw9TuXJlAN566y3+9a9/8dprr1G2bFk++eQTKlasaE6AfXx8OHDgAPPnzzcnvAsWLKBfv34MGDAAgPfff58ffviB4OBg3njjjVzFIiIiIkWbCYhLNRR2GCIiUsjuyxze+Ph4AIxGIwA//fQTqamptG3b1lymcuXK+Pj4sG/fPgAiIiLw8fExJ7sA7dq1Izk5mZ9++slc5tZ7ZJU5dOgQqamppKSk8NNPP1mVadu2rbme3MQiIiIiRd/1tMKOQERECluuenjzIiUlhVdffZWOHTvy8MOZw8ouXbqEvb09rq6uFmXd3Ny4dOmSuYybm5vFeVdXV+zt7S3KPP7441b3SEtL48qVK5hMJtLT063uc3s9d4slO1FRUbn8Bu6t/N9hi3WpTaqrsOopyLrUprsp9QDEkPd7ent753v9D6oPP/yQt99+m2HDhplHX2Xnhx9+YMaMGRw7dgxHR0f8/Px4++23qVWr1n2NL9VkIC3DhIOdenpFRIqrfE1409LSGD58OHFxcXz++ed3LW8ymTAY/noI3frnW+VUxvT/K1IYDAaLP+dUT25iuV1efsBERUUV2A8eW6xLbVJdhVVPQdalNuXCHebo3k1+t7Ug/66Kkv3797N8+XJ8fX1zLPfHH3/Qr18/XnjhBRYvXsyNGzd444036NWrF4cOHbqvMdoBN1IzMDppfQ4RkeIq34Y0p6WlMWTIEI4cOcK6desoX768+Zy7uzvp6elcuXLF4prLly+be2Pd3d2telivXLli0WObXZnLly/j4OBA+fLlrXqE71TP3WIRERGRO4uLi2PYsGHMmzfPPH3pTn7++Wfzjg41atTgkUceYfz48Zw8edLqWZzfShhMXEvOuK91iIjIgy1fEt7U1FQGDRrEkSNHCAkJwcPDw+J8gwYNKFGiBGFhYeZj586d4/jx4/j5+QHQrFkzjh8/brE9UFhYGE5OTjRo0MBcZtu2bRb3DgsLo2HDhpQoUQJHR0caNGhgUU9Wmax6chOLiIiI3Nm4ceMIDAwkICDgrmWznrsrVqwgPT2d+Ph4Pv/8cxo1amQ1vSi/2QE307Q3kYhIcZarIc03btwgOjoagIyMDM6ePUtkZCQuLi5UqlSJAQMGcOjQIT7//HMMBgMxMTEAlC1bFmdnZ8qVK8c///lPXn/9ddzc3MxbAfn6+prn5LZt25Y6deowYsQIpk2bxrVr13j99dfp378/ZcuWBWDQoEEsXbqUSZMmMWjQIPbt28f//vc/li1bZo519OjRvPDCCzRu3Bg/Pz+Cg4O5ePEigwYNAshVLCIiUrzcyxZIxXX7o+XLlxMdHc3ixYtzVb5q1ap8/fXXDBw4kAkTJpCRkcEjjzzCmjVr7nNIZIaxAAAgAElEQVSkmRLTlfCKiBRnuUp4Dx06xFNPPWX+PH36dKZPn07fvn2ZNGkSoaGhAFYJ44IFC3juuecAePfdd7G3t2fQoEEkJSXRunVrFi1aZN731t7eni+++IIJEybQsWNHSpYsSc+ePZk2bZr5ftWqVePLL7/kP//5D8HBwVSsWJGZM2eatyQC6N69O1evXuX9998nJiaGOnXq8OWXX+Ll5WUuc7dYRERExFpUVBRTp05lw4YNODo65uqamJgYXnzxRfr06UOPHj24ceMG7777LgMHDiQkJAQ7uzsPNvs7C5ClZQAYiD51hoeumbC3gXWrCnKhu4Kg9jzY1J4Hly21Bf5+e+62zkauEt5WrVoRGxt7x/M5nctSsmRJ3n///RxXcaxSpQpffPFFjvd57LHH2L59e45lhg4dytChQ/9WLCIiImIpIiKCK1eu4O/vbz6Wnp7O7t27CQ4O5vz58zg5OVlcs3TpUkqVKsXUqVPNx5YsWYKvry/79u2zuNft/s5iYakZJg4f/J3KlSvjWb4EZR2L9kttW1s8Te15sKk9Dy5bagsUTHvyfVsiERERsU1dunShYcOGFsdGjx5NzZo1+fe//51tr29iYqLVCKqszxkZ939BKUc7A9eSM4p8wisiIvdGCa+IiIjkitFotFqVuVSpUri4uFC3bl0A3nrrLQ4ePMj69esB6NChAwsXLmTGjBn06tWL+Ph43n77bSpXrmxelPJ+crAzkJCqebwiIsVVvm1LJCIiInLx4kVOnjxp/hwQEMCyZcsIDQ2ldevW9OjRAwcHB9asWUPp0qULJCYtXCUiUnyph1dERETu2XfffWfxOSgoyKpMjx496NGjR0GFZCU53USGyYSdwQZWrhIRkTxRD6+IiIjYNBPaj1dEpLhSwisiIiI2zcnOwLWk+79AloiIPHiU8IqIiIhNc7AzcCNVCa+ISHGkhFdERERsnhauEhEpnpTwioiIiM3LWrhKRESKFyW8IiIiYvMygCT18oqIFDtKeEVERMTmOdoZiE3WPF4RkeJGCa+IiIjYvBJ2BuJT1MMrIlLcOBR2ACIiUrQYPzl3hzOlYGf252IHPXz/AhLJJS1cJSJS/KiHV0RERIqFpDQTJi1cJSJSrCjhFRERkWIhc+Gqwo5CREQKkhJeERERKRYcDBCnhatERIoVJbwiIiJSLDjaG4hLUcIrIlKcKOEVERGRYiMpQ3N4RUSKEyW8IiIiUmxo4SoRkeJFCa+IiIgUG+kmSNbCVSIixYYSXhERESk27A1wPVXzeEVEigslvCIiIlJsONkbiNVKzSIixYYSXhERESlWEtM1h1dEpLhQwisiIiLFSlKaEl4RkeJCCa+IiIgUK5kLVynpFREpDpTwioiISLFib4D4FM3jFREpDnKV8O7atYs+ffpQp04djEYjK1eutDhvMpmYPn06tWvXpmLFinTp0oVjx45ZlImNjWX48OF4eXnh5eXF8OHDiY2NtShz5MgROnfuTMWKFalTpw4zZ8602itv3bp1+Pn54e7ujp+fHyEhIfclFhEREbFNjnZwTQmviEixkKuENyEhgbp16zJjxgycnZ2tzs+dO5cFCxYwc+ZMtm7dipubG926dSM+Pt5cZujQoURGRrJ69WrWrFlDZGQkL7zwgvn89evX6datG+7u7mzdupUZM2Ywb9485s+fby4TERHB4MGD6dWrFzt27KBXr14MHDiQAwcO5GssIiIiYrsMBgOJmscrIlIs5Crh7dChA6+//jqBgYHY2VleYjKZCAoKYty4cQQGBlK3bl2CgoK4ceMGa9asAeD48eNs2bKFOXPm4OfnR7NmzZg9ezabNm0iKioKgNWrV5OYmEhQUBB169YlMDCQsWPHsnDhQnMvb1BQEK1atWLChAn4+PgwYcIEHnvsMYKCgvI1FhEREbm7Dz/8EKPRyMSJE3MsZzKZWLhwIU2bNsXd3R0fHx/efPPNggnyDpI0h1dEpFj423N4T506RUxMDG3btjUfc3Z2pkWLFuzbtw/I7Jl96KGH8PPzM5dp3rw5pUuXtijj7+9v0YPcrl07Lly4wKlTpwDYv3+/RT1ZZbLukV+xiIiISM7279/P8uXL8fX1vWvZKVOm8PHHH/Pmm28SERHBl19+SYsWLQogyjtLy4AUJb0iIjbP4e/eICYmBgA3NzeL425ubly4cAGAS5cu4erqisFgMJ83GAxUqFCBS5cumct4enpa3SPrXLVq1YiJicm2nqx75Fcs2clr729B9hbbYl1qk+oqrHoKsq6i26ZSBVR/3uspyLruVI+3t/c91F+0xMXFMWzYMObNm8d7772XY9moqCiWLFnCrl278PHxKaAI784OuJGaQXl7+8IORURE7qO/nfBmuTWBhMzhS7cnlbe7W5msocx3K3P7sfyI5XZ5+QETFRVVYD94bLEutUl1FVY9BVlXkW7TznN5vuSe6r+HegqyruKQ2N5J1tShgICAuya8oaGhVKtWjS1btvDss8+SkZFBy5Ytefvtt61eUBckJ3u4mpxB+ZJKeEVEbNnfHtLs4eEBYNU7evnyZfODzN3dncuXL1usuGwymbhy5YpFmezuAX/12Hp4eORYT37FIiIiItlbvnw50dHRTJkyJVfl//jjD86cOcPatWtZuHAhixcvJioqij59+pCRUXgrJRsMBhI1pFlExOb97R7eqlWr4uHhQVhYGI0aNQIgKSmJPXv2MHXqVACaNWvGjRs3iIiIMM+djYiIICEhwfy5WbNmvPnmmyQlJVGyZEkAwsLCqFSpElWrVgWgadOmhIWF8a9//ctcf1hYmPke+RWLiIiIWIuKimLq1Kls2LABR0fHXF2TkZFBcnIyixcvplatWgAsXryYJk2a8OOPP9KkSZMc67tXaRkABk6fOZ1jmZJXik7Sa2uLa6o9Dza158FlS22Bv9+eu424ylXCe+PGDaKjo4HMB9fZs2eJjIzExcWFKlWqMHLkSD788EO8vb2pVasWH3zwAaVLl6Znz54A+Pj40L59e8aPH8/cuXMxmUyMHz+eJ5980hxgz549mTlzJqNGjWLChAmcOHGCOXPm8PLLL5uHGo8YMYLOnTsza9YsunbtyrfffsuOHTvYuHEjkPm2Nj9iEREREWsRERFcuXIFf39/87H09HR2795NcHAw58+fx8nJyeIaDw8PHBwczMkuQM2aNXFwcODs2bM5Jrx/57mcmmHi8MHf8aridccyN9NMVHN3pITdnac0PSgKchpEQVB7Hmxqz4PLltoCBdOeXCW8hw4d4qmnnjJ/nj59OtOnT6dv374EBQUxduxYEhMTmThxIrGxsTRu3Ji1a9dSpkwZ8zVLly7llVdeoXv37gB06tTJYt5PuXLl+Prrr5kwYQJt2rTBaDQyevRoxowZYy7j5+dHcHAw06ZNY/r06VSvXp3g4GCLh2V+xCIiIiLWunTpQsOGDS2OjR49mpo1a/Lvf/87217f5s2bk5aWxsmTJ6levTqQOcw5LS2NKlWqFEjcd2IAElIzMDppHq+IiK3KVcLbqlUrYmNj73jeYDAwefJkJk+efMcyLi4uLFmyJMd6fH192bBhQ45lAgMDCQwMvO+xiIiIiCWj0YjRaLQ4VqpUKVxcXKhbty4Ab731FgcPHmT9+vUAPP744zz66KOMHj2a6dOnAzB58mSaNGlilTwXtKyFq5TwiojYrr+9aJWIiIhIlosXL3Ly5EnzZzs7O7744gvc3Nzo0qULPXr04OGHH+Z///sfdnaF+zPEzmDgZlrRmcMrIiJ5l2/bEomIiEjx891331l8DgoKsipTsWJFli9fXlAh5UmiEl4REZumhFdExAYYP8lpH9lS2e4zGzvo4fsXkEgRkWqCtAwTDkVg4SoREck7DWkWERGR4suUuXCViIjYJiW8IiIiUmw52cO1ZCW8IiK2SgmviIiIFFv2BgMJmscrImKzlPCKiIhIsZaYroRXRMRWKeEVERGRYi0lHdIzlPSKiNgiJbwiIiJSrJlMJhLSNI9XRMQWKeEVERGRYs3J3qCFq0REbJQSXhERESnWHOwMJKRqSLOIiC1SwisiIiLFnhauEhGxTUp4RUREpNhLTjeRYVLSKyJia5TwioiISLFnAhK1H6+IiM1RwisiIiLFnpOdFq4SEbFFSnhFRESk2HOwM3BDC1eJiNgcJbwiIiIiaOEqERFbpIRXREREBEhOy8CkhatERGyKEl4RERERIB318oqI2BolvCIiImJzMkwmohLsCDuflOtrHO0MxGrhKhERm+JQ2AGIiIiI5JfY5Axm/nSdb/5I5MJNJ4znbtCqohMOdoa7XlvCzkB8iglKF0CgIiJSINTDKyIiIjbD2cHAyhM3uXAzs6c2NsXEz1dSc329hjSLiNgWJbwiIiJiM5zsDXTxcrY4FnY+OdfXJ6WZtHCViIgNUcIrIiIiNqV7dcuEd/uFZNIycpfEZgBJ6fchKBERKRRKeEVERMSmBFRywuj415zd66kmDl7O3bBmBwPEaeEqERGbkS8Jb3p6OtOmTeORRx7Bw8ODRx55hGnTppGWlmYuYzKZmD59OrVr16ZixYp06dKFY8eOWdwnNjaW4cOH4+XlhZeXF8OHDyc2NtaizJEjR+jcuTMVK1akTp06zJw502ro0bp16/Dz88Pd3R0/Pz9CQkIszucmFhERESmaHO0NdKl6+7Dm3K3W7GhvIC5FCa+IiK3Il4R3zpw5LFu2jJkzZxIREcGMGTNYunQps2bNMpeZO3cuCxYsYObMmWzduhU3Nze6detGfHy8uczQoUOJjIxk9erVrFmzhsjISF544QXz+evXr9OtWzfc3d3ZunUrM2bMYN68ecyfP99cJiIigsGDB9OrVy927NhBr169GDhwIAcOHMhTLCIiIlJ0PVOtpMXnHRdSSMnlglRJuRz+LCIiD758SXgjIiLo2LEjnTp1omrVqnTu3JlOnTpx8OBBILNHNSgoiHHjxhEYGEjdunUJCgrixo0brFmzBoDjx4+zZcsW5syZg5+fH82aNWP27Nls2rSJqKgoAFavXk1iYiJBQUHUrVuXwMBAxo4dy8KFC829vEFBQbRq1YoJEybg4+PDhAkTeOyxxwgKCsp1LCIiInJ3H374IUajkYkTJ+aq/O+//07lypV5+OGH73Nk8FhFJ8rY/5W4JqSZOHA5JVfXauEqERHbkS8Jb/Pmzdm5cye//fYbAL/++is7duzgiSeeAODUqVPExMTQtm1b8zXOzs60aNGCffv2AZlJ80MPPYSfn5/FfUuXLm1Rxt/fH2fnv4YptWvXjgsXLnDq1CkA9u/fb1FPVpmse+QmFhEREcnZ/v37Wb58Ob6+vrkqn5KSwuDBg2nRosV9jiyTg52BRuUsV58KO5e71ZrTTZCshatERGyCQ37cZNy4cdy4cQM/Pz/s7e1JS0tjwoQJDB06FICYmBgA3NzcLK5zc3PjwoULAFy6dAlXV1cMhr8WmTAYDFSoUIFLly6Zy3h6elrdI+tctWrViImJybaerHvkJpbsZPUy51Zey/8dtliX2qS6Cquegqwrf+spVYD1F1Rdea+nIOu6Uz3e3t73UH/REhcXx7Bhw5g3bx7vvfderq5544038PX1pWXLluzates+R5ipSdl0wq/+9VNnV0wKyekmnOwNOVwF9ga4nppBSQf7+x2iiIjcZ/mS8K5du5ZVq1axbNkyateuzeHDh5k0aRJeXl7079/fXO7WZBYyhxffnuDe7m5lsoYc3a3M7cdyU+ZWefkBExUVVWA/eGyxLrVJdRVWPQVZV77Xs/Ncni+55/oLqq57qKcg6yoOie2dZE0LCggIyFXCu2nTJjZt2kR4eDjr168vgAgzeZfOoLyTHVf/f9Xlm2kmIi6l0KqSU47XOdpBbHIG7s5KeEVEirp8SXhff/11xowZQ48ePQDw9fXlzJkzzJ49m/79++Ph4QFk9sJWrlzZfN3ly5fNPa3u7u5cvnzZIvE0mUxcuXLFokxWT+2t94C/emw9PDyyLXPr+bvFIiIiItlbvnw50dHRLF68OFflL168yNixY/n0008pU6ZMnur6O6Mg0jLAzmCgUZkUtiT/9XPn2xNXqZp29y2K7ADDQw/WPN6CHOlSENSeB5va8+CypbbA32/P3V5A50vCe/PmTeztLd+C2tvbk5GR+Ua1atWqeHh4EBYWRqNGjQBISkpiz549TJ06FYBmzZpx48YNIiIizPN4IyIiSEhIMH9u1qwZb775JklJSZQsmbn6YlhYGJUqVaJq1aoANG3alLCwMP71r3+ZYwkLCzPfIzexiIiIiLWoqCimTp3Khg0bcHR0zNU1w4cPZ/DgwTRt2jTP9f2dXvTUDBOHD/7O0/+owJbLf21x+HO8A26VKuLskPOw5uR0E94eOfcEF6SCHOlSENSeB5va8+CypbZAwbQnXxat6tixI3PmzGHTpk2cOnWKkJAQFixYQNeuXYHM4cMjR45kzpw5rF+/nqNHjzJq1ChKly5Nz549AfDx8aF9+/aMHz+e/fv3ExERwfjx43nyySfNX0LPnj1xdnZm1KhRHD16lPXr1zNnzhxGjRpl7hUeMWIE27dvZ9asWfz222/MmjWLHTt2MHLkyFzHIiIiItYiIiK4cuUK/v7+uLq64urqyq5du1i2bBmurq4kJ1svCrV9+3ZmzpxpLv/iiy+SkJCAq6sr//3vf+97zPXKO1Ch5F8/d5LSYd+lu6/WnLlw1YPVwysiInmXLz287733Hu+88w4vvfQSly9fxsPDgwEDBvDyyy+by4wdO5bExEQmTpxIbGwsjRs3Zu3atRbDm5YuXcorr7xC9+7dAejUqZPF3KBy5crx9ddfM2HCBNq0aYPRaGT06NGMGTPGXMbPz4/g4GCmTZvG9OnTqV69OsHBwTRp0iRPsYiIiIilLl260LBhQ4tjo0ePpmbNmvz73//Ottd39+7dFp9DQ0P58MMP+eGHH6wWorwf7AwG2ng6sTo60Xxs6/kkHvfMuffW3gDxKRk4aR6viEiRli8Jb5kyZZgxYwYzZsy4YxmDwcDkyZOZPHnyHcu4uLiwZMmSHOvy9fVlw4YNOZYJDAwkMDDwb8UiIiIiloxGI0aj0eJYqVKlcHFxoW7dugC89dZbHDx40Lw4VdbxLIcOHcLOzs7q+P10e8K7NyaFm2kZlHK480A3Rzu4lpJBBSW8IiJFWr4MaRYRERGBzEWqTp48WdhhWKhjdMDD+a+fPCkZsPtizsOaDQYDiWka0iwiUtTlSw+viIiIFE/fffedxeegoKAcyz/33HM899xz9zMkK4b/H9a86ve/ennDzifTvnLJHK9L0hxeEZEiTz28IiIiYvPa3DZnN+LPFOJTM3K8Ji0DUpT0iogUaUp4RURExOb9o5wDnqX++tmTmothzXbAjbskxSIi8mBTwisiIiI2z2Aw0NbTcghz2HnrbZRu5WQPV5OV8IqIFGVKeEVERGxcYmIily9ftjh2+fJlZs2axeuvv87BgwcLKbKC1eZhy2HN+/9M4XrKnRNag8FAooY0i4gUaVq0SkRExMaNHz+eY8eOER4eDkBCQgLt2rXj9OnTQOZCUyEhITRv3rwww7zvapSxx+she07fSAcg3QQ7LibTxcv5jtdopWYRkaJNPbwiIiI2bu/evXTq1Mn8ec2aNZw+fZo1a9Zw/PhxfHx8+OCDDwoxwoKRtVrzrcLO5TysOS0DUjOU9IqIFFVKeEVERGxcTEwMDz/8sPnzhg0baNasGe3atcPd3Z3nnnuOyMjIQoyw4Nye8P54JZXYHObpGoAELVwlIlJkKeEVERGxcaVLlyY2NhaAtLQ0du/ezeOPP24+7+zsTHx8fCFFV7CqlXGgehl78+cME2y/cOdeXi1cJSJStCnhFRERsXENGzbk008/5eeff+aDDz7gxo0bdOzY0Xz+5MmTuLu7F2KEBctqWHMOqzXbGQzc1DxeEZEiS4tWiYjcR8ZPzt3hTCnYmf252EEPZ3tc5F5NmTKFbt260aZNG0wmE08//TQNGzY0n//222/x8/MrxAgLVltPJ4KP3zR//vlKKleSMnAtmX0/gBauEhEpupTwioiI2LgGDRpw4MAB9u3bR5kyZWjVqpX5XGxsLEOHDqVly5aFGGHBqvyQA95lHYi6ngZABhB+IZnu1bNfrTnVBGkZJhzsDAUYpYiI5AclvCIiIjYsKSmJuXPn0rRpUzp37mx13mg0MnLkyEKIrHA97ulkTngBws4n3THhxZS5cFU5J/vsz4uIyANLc3hFRERsWMmSJZk9ezZnz54t7FAeKLfP4z18NY1LienZlnWyh2tauEpEpEhSwisiImLj6tevT3R0dGGH8UDxLG1PbaPlQLfwO6zWbG8wkKB5vCIiRZISXhERERv3+uuvs2LFCjZt2lTYoTxQ8rJac2K6El4RkaJIc3hFRERs3EcffYTRaKRv3754enpSrVo1nJ0t56saDAa+/PLLQoqwcLTxdCLoaIL589FraVy8mU7FUtZzdVPSIT3DhL0WrhIRKVKU8IqIiNi4X3/9FYPBQOXKlQE4ffq0VRmDofglcu7O9tRzceCXa38tXrXtfDJ9apWyKmsymbiZlkEZRy1cJSJSlCjhFRERsXGHDx8u7BAeWG08nSwS3q13SHid7A3EJpso41iQ0YmIyN+lObwiIiJSbAV4OnFr3/ZvcWmcTbBerdnBzsCNVK3ULCJS1KiHV0REpBiJj4/n+vXrZGRYJ29VqlQphIgKV4WS9jxSvgQ/X001Hws/n8xz3ta9vDe1cJWISJGjhFdERKQYWLFiBR999FGO2xNdvXq1ACN6cLR52Mki4d16PinbhDc53USGyYRdMZzvLCJSVGlIs4iIiI379NNPGTt2LFWqVOHVV1/FZDIxcuRIxo8fj7u7O/Xr12fevHmFHWahaV3JyeIH0e/X0zl9I82qnAlI1H68IiJFihJeERERGxcUFESrVq34+uuvGThwIAAdOnTgtddeY+/evcTGxnL9+vXCDbIQlXeyo0GFEhbHstuT18nOwLVkzeMVESlKlPCKiIjYuOjoaLp27QqAnV3moz81NXMIr9FopH///ixbtqzQ4nsQtPV0svgcds464c1cuEo9vCIiRUm+JbwXL15kxIgR1KxZEw8PD/z8/Ni5c6f5vMlkYvr06dSuXZuKFSvSpUsXjh07ZnGP2NhYhg8fjpeXF15eXgwfPpzY2FiLMkeOHKFz585UrFiROnXqMHPmTEwmy4fPunXr8PPzw93dHT8/P0JCQizO5yYWERERW1G6dGnzs/Khhx7C3t6eixcvms+XL1+e8+fP5/m+H374IUajkYkTJ96xzI4dO+jbty8+Pj5UqlSJFi1a8Omnn+a9EffZYxWdsLtlau4fN9I5GW89rDlRC1eJiBQp+ZLwxsbG8uSTT2Iymfjyyy/Zt28f7733Hm5ubuYyc+fOZcGCBcycOZOtW7fi5uZGt27diI+PN5cZOnQokZGRrF69mjVr1hAZGckLL7xgPn/9+nW6deuGu7s7W7duZcaMGcybN4/58+eby0RERDB48GB69erFjh076NWrFwMHDuTAgQN5ikVERMRWeHt7c/ToUQAcHByoX78+q1atIjU1laSkJL744guqVq2ap3vu37+f5cuX4+vrm2O5iIgIfH19Wb58OXv27GHIkCGMGzeO1atX33N77gejkx2Nbx/WnE0vb3JahtWLdhEReXDlyyrNH330ERUrVmTx4sXmY9WqVTP/2WQyERQUxLhx4wgMDAQy5xN5e3uzZs0aBg0axPHjx9myZQsbN27Ez88PgNmzZ9OpUyeioqLw9vZm9erVJCYmEhQUhLOzM3Xr1uW3335j4cKFjBkzBoPBYJ6nNGHCBAB8fHzYsWMHQUFBfPzxx7mKRURExJZ06dKFoKAgkpKSKFmyJBMmTOCf//wn1apVw2AwkJCQwKJFi3J9v7i4OIYNG8a8efN47733ciz70ksvWXweMmQIO3bsYP369fTq1eue2nO/tPUsyf4//1qtOex8MoN8SmG4ZVXmdDJ7eUs5aKVmEZGiIF96eL/77jsaN27MoEGDqFWrFo899hhLliwxvwE9deoUMTExtG3b1nyNs7MzLVq0YN++fUDmG+CHHnrInOwCNG/enNKlS1uU8ff3x9nZ2VymXbt2XLhwgVOnTgGZb5xvrSerTNY9chOLiIiILXnxxRc5evQoJUuWBDIT4NDQUPr378/AgQMJCQmhd+/eub5f1kvjgICAe4onPj4eo9F4T9feT49VdOTWPPZMQjonrqdblHG0MxCrhatERIqMfOnh/eOPP/j4448ZNWoU48aN4/Dhw7zyyisADB8+nJiYGACLIc5Zny9cuADApUuXcHV1tXiLajAYqFChApcuXTKX8fT0tLpH1rlq1aoRExOTbT1Z98hNLNmJiorKxTdx7+X/DlusS21SXYVVT/7XZb2X5/2pv6DqKci68l5PQdZ1p3q8vb3vof6C17x5c5o3b57n65YvX050dLTFqK682LhxI+Hh4WzatOmuZf/Of4tpGQAGTp85nafr6pVx5Kfr9ubP6369xLOelnN5r9pBQqmCH9ZckP8OFgS158Gm9jy4bKkt8Pfbc7fnbr4kvBkZGTRs2JA33ngDgEcffZTo6GiWLVvG8OHDzeUMt23UbjKZrBLc292tTFYv8t3K3H4sN2VulZcfMFlDsAuCLdalNqmuwqrnvtS181yeL7mn+guqnoKs6x7qKci6ikpiC5mLUi1evPiOQ4jXrl3L0KFDuXr1ao73iYqKYurUqWzYsAFHR8c8x7F3716GDRvGzJkzady48V3L/53vODXDxOGDv+NVxStP13U2JPHTob/W9Dh4w4mXKley+o3gXSHv7f87CvLfwYKg9jzY1J4Hly21BQqmPfkypNnDwwMfHx+LY//4xz84e/as+TKDiZQAACAASURBVDxg7mXNcvnyZXNPq7u7O5cvX7ZYCMJkMnHlyhWLMtndA/7qsfXw8MixntzEIiIiYkvutshSRkZGji99s0RERHDlyhX8/f1xdXXF1dWVXbt2sWzZMlxdXUlOtl7kKcuePXvo1asXkydPZsiQIXluQ0Fp6eFIiVt+HV24mcHxOMse3qQ0kxauEhEpIvIl4W3evDknTpywOHbixAmqVKkCQNWqVfHw8CAsLMx8PikpiT179pjn7DZr1owbN24QERFhLhMREUFCQoJFmT179pCUlGQuExYWRqVKlcyrSzZt2tSinqwyWffITSwiIiK2JqeE9sCBA7maU9ulSxd2797Njh07zP9r2LAhPXr0YMeOHXfs9d21axe9evXi5ZdfZtSoUffchoJQuoQdfu6W7Qg7b5nIZwBJllN7RUTkAZUvQ5pHjRpFhw4d+OCDD+jevTuRkZEsWbKE1157Dch8yI4cOZIPP/wQb29vatWqxQcffEDp0qXp2bMnkLmacvv27Rk/fjxz587FZDIxfvx4nnzySXM3d8+ePZk5cyajRo1iwoQJnDhxgjlz5vDyyy+bH+QjRoygc+fOzJo1i65du/Ltt9+yY8cONm7cmOtYREREirqgoCCLlZcnT57M22+/bVUuLi6O69ev52rRKqPRaJUYlypVChcXF+rWrQvAW2+9xcGDB1m/fj2QuQ9v7969GTJkCM8++6x5LQ17e3sqVKhwz+27n9p4OrHzYor587bzyYyoU9r8W8PBAHHJGTg72N/pFiIi8oDIl4S3UaNGrFy5kqlTp/L+++9TuXJl/vOf/zB06FBzmbFjx5KYmMjEiROJjY2lcePGrF27ljJlypjLLF26lFdeeYXu3bsD0KlTJ4vtDsqVK8fXX3/NhAkTaNOmDUajkdGjRzNmzBhzGT8/P4KDg5k27f/Yu/O4qMr9geOfMwv7KrJoCi4guKWmgppmZGnuWpql93fVMpe0xbLUulaa5XLTNEtyyfZNDa9ombeUXEpBc03N0OtaCqLsss3M+f0xMjICijIzLH7frxfBnHM43+cLxjnfeZ7zPDOZNWsWDRs2ZMWKFbRr1+6m2iKEEEJUZ35+fpY3jE+fPk1gYCBBQUFWxyiKgpubG23atLGac6Mizp8/z4kTJyyvv/zySy5fvsyiRYtYtGiRZXv9+vU5ePCgTWLaWsdAZ5w1WRRNxpyca+JwmoHmtczr9DppFTIKTAS5S8ErhBBVnU0KXoAePXrQo0ePMvcrisLUqVOZOnVqmcf4+vqydOnS68Zp3rw5GzZsuO4x/fv3t6yxe6ttEUIIIaqzRx55hEceeQSAPn368OKLL97yMkLX891331m9jomJKfH62m1VnZtOoWOgMz+fuzqUefPf+ZaCFyDPJM/wCiFEdWCTZ3iFEEIIUXVNmTKFI0eOWG1btWoV7dq1IzQ0lMmTJ2MyydqyxUXXdbZ6veVcPqZiE1XJxFVCCFE9SMErhBBC1HBz584lISHB8vrPP//kqaeeQqPR0KZNG5YtW2b1vG9NoFj+c2uiApxwKTZiOTXPxO+XCi2vjSrky8RVQghR5UnBK4QQQtRwf/zxh9W6tytXrsTV1ZWffvqJVatWMWTIED7//PNKbKHt6TQK3now3OLQYxedwt1B1r28m4vN1qxVILNQesWFEKKqk4JXCCGEqOEyMzOtZlfetGkT0dHReHl5AdCxY0dOnz5dWc2zmwauKqDc8tDj0oY1G6+cy0kD6flS8AohRFUnBa8QQghRwwUGBnL06FEAzp07x4EDB7jvvvss+zMzM9Fqa96MwxoFWvrpb3nocaS/E+66q+Oi0/JV9l80D2tWFIVcozzDK4QQVZ3NZmkWQgghRNXUt29fli1bRn5+Pnv27MHZ2ZmePXta9v/+++80aNCg8hpoR85ahXAfHX+kG3DV3dxDvU5ahc5BTmw8e3Uoc/xf+dxV2wkwT1wlhBCiapMeXiGEEKKGmzp1Kv369WPlypUkJyfz3nvvERAQAJh7d9etW0d0dHQlt9J+fF201HXXkH8LPbKlDWsuei7YPHGVFL1CCFGVSQ+vEEIIUcO5u7uXuc69h4cHhw8fxs3NzcGtcqwQTz1ZBQXkGVV0mvL39Lb1d8JTr5BVaC5sMwtV9qYW0j7ACa0CWQUmnF1r3nBwIYSoKaSHVwghhLiNaTQavL290ev1ld0Uu2vqq0eFm5rESq9R6HLNbM3xV2ZrdtJAWoFMXCWEEFWZFLxCCCGEuC1oNQrNffU3PdnUtcOat57Pp9Ckmieukud4hRCiSpOCVwghhBC3DTe9hjDvmyt629TW4+10dRh0dqHK7gsFAOTJM7xCCFGlScErhBBCiNuKv6uWAJfyT2Kl0yjcU6f0Yc0GExRI0SuEEFWWFLxCCCGEuO008tLholUwlvN53vuuGda8/XwB+UYVrQIX825xoV8hhBB2J7M0CyFuOz4f/XWdvW6wveT+9JF32K9BQgiHUxSF5rX0/HahAG05Jlm+00+Pr7NCWr65QL5sUNl1oYDOQc6czjYS6KZFo9zcOr9CCCHsT3p4hRBCCHFb0mkUmvnquFyOIclaRaFrGcOaNQqcyjLYpY1CCCEqRgpeIYQQQty2PJ20NPTQkWu48fJC1w5r/uV8PnkGFb1G4dxlozzLK4QQVZAUvEIIIYS4rdVx11LLRUuh6foFa4taemq7XL11yjPCzhTzbM0uWoWkjEK7tlMIIcTNk4JXCCGEELe9Jt46tIqC6TqTWGkUhXtLDGvOs+zLKFDJLJAJrIQQoiqRglcIIYQQtz1FUWhRS0/BDUY2R99hXfDuTC7gssFcJLvpFI5lSsErhBBViRS8QgghhBCAk1YhwkdnKWBL08xHR6Dr1dunfBPsSM63vC4wqpzPkaJXCCGqCil4hRBCCCGu8HHWUt9dS14ZE1ApisK9dUufrRnMz/KeyjFcd2i0EEIIx5GCVwghhBCimPqeOrydNBjKmMQq+pqCNyGlgJzCq2OhtcBJWaZICCGqBCl4hRBCCCGuEe6jQ0VBLaWnNtxbR123q7dQhSb46a+rvbw6jcL5yyZZpkgIIaoAKXiFEEIIIa6hURRa1tKRX0rRqihKiV7eZX/kkJp39dldFy2yTJEQQlQBUvAKIYQQ4pbNmzcPHx8fXnzxxesed+jQIXr16kVQUBBNmzZlzpw5pfaeViUuOg1NfPTkljKJVZ8QV5yK3UVlF6q8cyDbkpMsUySEEFWDXQre0i5+qqoya9YsIiIiCAoKonfv3hw5csTq+9LT0xk9ejTBwcEEBwczevRo0tPTrY4pzwVz7dq1REVFERAQQFRUFOvWrbPaX562CCGEEOL6du3axSeffELz5s2ve1xmZiYDBw4kICCAzZs3M3v2bBYtWsR7773noJbeulouWoLcNCV6euu4aRkV4W617ZfkAjYVG9rsplNIypCCVwghKpPNC96yLn4LFy7k/fffZ86cOWzevBl/f38GDhxIVlaW5ZhRo0Zx4MABVq1axerVqzlw4ABjxoyx7C/PBTMxMZHHH3+cwYMHs23bNgYPHsyIESPYvXv3TbVFCCGEEGXLyMjgySefZNGiRfj4+Fz32FWrVpGbm0tMTAzNmjWjf//+PPvssyxevLjK9/ICNPDU4aZTMF7T1ocbudLcV2e17d3fs7mUf3UCq0KTLFMkhBCVyaYFb1kXP1VViYmJ4bnnnqN///40a9aMmJgYsrOzWb16NQBHjx7lp59+YsGCBURFRREZGck777zDxo0bSUpKAsp3wYyJiaFLly5MmjSJ8PBwJk2aROfOnYmJiSl3W4QQQghxfUXX0a5du97w2MTERDp27Iirq6tlW7du3Th37hynTp2yZzNtQlEUmvrqufZxXq2i8FIrT/TF7qYyC1UWHMiy3JcULVNkLGPGZyGEEPalu/Eh5Vf84jd37lzL9lOnTpGcnMx9991n2ebq6kqnTp1ISEhg5MiRJCYm4uHhQVRUlOWYDh064O7uTkJCAmFhYWVeMN98801OnTpFgwYN2LVrF6NHj7ZqV7du3Vi6dGm52yKEEEKIsn3yySf873//Y8mSJeU6PiUlhbp161pt8/f3t+xr0KBBqd9X9IZ3RdjiHEVcDHA0R8FZe3WbAgwM1LHynN6ybev5AlYf/JsoX3PPrkGFnPMq9V2pEFvmUhVIPlWb5FN11aRcoOL5hIWFXXe/zQre6138kpOTgasXtyL+/v6cO3cOMF/w/Pz8UBTFsl9RFGrXrk1KSorlmBtdMJOTk0uNU3SO8rSlNDf7i3DkP8SaGEtyklj2jePmwPiOiiU5OTpWWXFudOGt7pKSkpgxYwYbNmzAycmp3N9X/PoOWHpAr91eXEV/lklJSTb/fQReNvK/LAOu2qvtHn2HyoFf0vkj/erau5+fc+aB8Fr4OJu7fy8bVIL9nXDWlp3v9dgjl8ok+VRtkk/VVZNyAcfkY5OCt7wXv9IudtcWuNe60TGlXTBvFKe8xxR3M78IR/5DrImxJCeJZfc42/+66W+55fiOiiU5OTxWTbrhuBmJiYlcvHiRjh07WrYZjUZ+/fVXVqxYwd9//42zs/WSPQEBAZY3noukpqYCJd+AruoC3bRkFppIyzPhdKV41WnMQ5tHb02jaELnjAKVhb9n81pbL+DqMkUtapX/TQIhhBAVZ5NneItf/Pz8/PDz8+OXX35h+fLl+Pn5UatWLYBSL3ZFF7qAgABSU1OtJq9QVZWLFy9aHXOjC2ZgYOB14wQGBt6wLUIIIYQoXe/evfn111/Ztm2b5aNNmzY8/PDDbNu2rdQ3viMjI9mxYwd5eXmWbfHx8dSpU4eQkBBHNt8mQr10OGkVTMXuWRp56fhnE+uRAvF/57PtnHnWZo2ikFmgkp4vE1gJIYQj2aTgvdHFLzQ0lMDAQOLj4y3fk5eXx44dOyzP7EZGRpKdnU1iYqLlmMTERHJycqyOudEFs3379lZxio4pOkdISMgN2yKEEEKI0vn4+NCsWTOrDzc3N3x9fWnWrBmKojB9+nT69etn+Z5Bgwbh6urKU089xeHDh4mLi2PBggU89dRT1x1dVVUpikLzWnoKTdbbh4a6EeZlPXhu/sEsMgrMB7rpFI5nGqvFzNRCCFFT2KTgLc/Fb9y4cSxYsIC4uDgOHz7MU089hbu7O4MGDQIgPDyc+++/n4kTJ7Jr1y4SExOZOHEiPXr0sAwbK88Fc+zYsWzdupX58+fz559/Mn/+fLZt28a4ceMAytUWIYQQQty68+fPc+LECctrb29v1qxZw7lz54iOjubFF19k/PjxTJgwoRJbWTF6jUJTXx25xaZu1mkUJrf2pPhjumn5Ku/9nm15XWhUOXdZenmFEMJRbDpL8/U8++yz5Obm8uKLL5Kenk7btm2JjY3F09PTcsyyZcuYPHkyDz30EAA9e/a0mu256II5adIkoqOj8fHxKXHBjIqKYsWKFcycOZNZs2bRsGFDVqxYQbt27W6qLUIIIYQon++++87qddFSgMU1b96cDRs2OKpJDuHlpCXEQ+V0tgEXrbkPIdRbxz/C3Pjkz8uW4378K5/ouvl0CnLGRadwJttIoKsWrab69W4LIUR1Y7eC99qLn6IoTJ06lalTp5b5Pb6+vpblg8pSngtm//796d+/f5n7y9MWIYRj+Xx0vUmD3MqcVCh95B32aZAQQpRDXXcdGQUqOYUmdFcK2H+EubH9fD7HM6/25M47kE3LWno8nTRoFTiRZSDUW1/WaYUQQtiITYY0CyGEEELcrsJ9dIBieTZXr1GY3MqT4h24F/NNvH/YPLRZp1FIyTWRb5RneYUQwt6k4BVCCCGEqACNotCils6qgG3io2doY+tZm384k09CsnnW5qJlioQQQtiXFLxCCCGEEBXkotMQ5qPnsuFq0fvPJm408NBaHff2gWyyC03mZYoKZZkiIYSwNyl4hRBCCCFswM9FS6CrxtLT66RVmNLG0+pm60KeiQ8O5wDgplU4JssUCSGEXUnBK4QQQghhI428dLhoFYxXitgIHz1DGrtaHbP+dB67LxQAYJBlioQQwq6k4BVCCCGEsBFFUWheS4/BdHXbiHB36rtbD23+9/4sLhtMlmWKjCbp5RVCCHuQglcIIYQQwoZ0GoVmvjpyrwxtdtYqTG7tSfFVd5NzTSy5MrS5aJkiIYQQticFrxBCCCGEjXk6aanvriXvStHbopaeQY2shzavPZXH3tQC8zJFeSZyC02lnUoIIUQFSMErhBBCCGEH9Tx0eDtpMFwZrvxEuDt3lDK0Odeg4qqBY9LLK4QQNicFrxBCCCGEnYT76FBRUFUVF53C5FYeVkOb/75sYvkfOSiKQlahSlqeTGAlhBC2JAWvEEIIIYSdaBSFFr46y9DmO/2cGNjQemhz7IlcDlwsxE2rcDxLlikSQghb0lV2A4QQVZvPR39dZ68bbC+5P33kHfZrkBBCVDOueg1hPnqSMgy4ahWejHBnR3I+5y6bn9lVgbn7s1h+jy+KAn/lGKnnIbdoQghhC9LDK4QQQghhZ7VdtAS4aMg3qrjqFF5q5Wm1/2yOkRVHc3DWKvyVI8sUCSGErUjBK4QQQgjhAI28dDhrFUyqSpvaTvQLcbHav/p/uRy6VIhWgeOZMoGVEELYghS8QgghhBAOoCgKzWvpKVp9aGwzdwJdr96KmYA5+7MwqpCaL8sUCSGELUjBK4QQQgjhIHqNQlNfHZcNKm46DS9eM7T5dLaRj//MkWWKhBDCRqTgFUIIIYRwIC8nLcEeWvKMKu38negdbD20+ZtjuRzNMJAtyxQJIUSFScErhBBCCOFg9Tx0eDtpMJhUxjVzx9/Femjz7H1ZaIFjmQZZpkgIISpACl4hhBBCiEoQ7qNDBdx1CpPu9LDadzLLyGdJlzGp5mWKhBBC3BopeIUQQgghKoFGUWjhqyfPqBIV6EyPes5W+784dpnT2Qb+yjFikE5eIYS4JVLwCiGEEEJUEle9hlBvPblGlQnNPfBzLja0WTUPbTapKmdzlUpspRBCVF+6ym6AEEIIIcTtzN9VS0aBiUt5Jl6404OXd2Va9h3PNPLN8Vw6OcMfaYVoFNAqoNOYZ3zWaRScNKDTKFf2KZZjNIoUyUIIIQWvEEIIIUQla+ylI7uwkA6BTtx/hzM//ZVv2fdZ0mVCmyjkGa+OazaqKqoKRtXcE6yiogIKRUWuigYFRQHNlY+iYlijgBZzQawooNeATlHQKeCkBV8XrRTLQogaQwpeIYQQQpTbsmXL+Oijjzhz5gwAERERTJo0iR49epT5PZs2bWL27NkcOXIEJycnoqKieOONNwgNDXVUs6s8RVFoXkvPngsFPN3Cg99SC0jLNxe4RhWWn3aiQ6iKTmMuRLWKAkrxGzkFg0klx6CSU6iSXWgix6CSXaiSbbjyutD8Osdwzf5Ck2W7q06hax1n5kZ5cYeHvlJ+FkIIYUtS8AohhBCi3OrWrcv06dNp3LgxJpOJr776imHDhvHzzz/TokWLEsefPHmSoUOHMmbMGJYsWUJ2djavvfYagwcPZu/evZWQQdWl1yhE+Og4nGZgYktPXt19dWjzyVwN03/LxNdZYylUc64UqkUFrS2W7M0uVPnudB47Uwp4s50Xj4a5V/ykQghRiWwyadX8+fOJjo6mfv36NG7cmCFDhnD48GGrY1RVZdasWURERBAUFETv3r05cuSI1THp6emMHj2a4OBggoODGT16NOnp6VbHHDp0iF69ehEUFETTpk2ZM2dOifXp1q5dS1RUFAEBAURFRbFu3bqbbosQQgghSurduzcPPPAAjRo1IjQ0lGnTpuHh4cGuXbtKPX7//v0UFhby2muv0ahRI+68804mTpzIiRMnuHjxooNbX/V5O2up564lMsCJ6LrWszZvO19A3Kk8Nv2VT0JKAb+nGTiZbSQ1zzbFbnEX80yM3Z7OPzdfJC1flkUSQlRfNil4t2/fzhNPPMHGjRuJi4tDp9MxYMAA0tLSLMcsXLiQ999/nzlz5rB582b8/f0ZOHAgWVlZlmNGjRrFgQMHWLVqFatXr+bAgQOMGTPGsj8zM5OBAwcSEBDA5s2bmT17NosWLeK9996zHJOYmMjjjz/O4MGD2bZtG4MHD2bEiBHs3r37ptoihBBCiOszGo18++235OTkEBkZWeoxrVu3Rq/X8+mnn2I0GsnKyuKrr77irrvuws/Pz8Etrh7qe+rwctIwvpk73k6V+yxt3Kk8Oq5J4btTuZXaDiGEuFU2GdIcGxtr9XrJkiUEBwezc+dOevbsiaqqxMTE8Nxzz9G/f38AYmJiCAsLY/Xq1YwcOZKjR4/y008/8cMPPxAVFQXAO++8Q8+ePUlKSiIsLIxVq1aRm5tLTEwMrq6uNGvWjD///JPFixczYcIEFEUhJiaGLl26MGnSJADCw8PZtm0bMTExfPjhh+VqixBCCCHKdujQIbp3705eXh7u7u58/vnnNG/evNRjQ0JCWLNmDSNGjGDSpEmYTCbuvPNOVq9efcM4SUlJFW6rLc5RGTQqZGZp+GcdhfdOOaFy48JXQcVVC24aFTctuGrNn920V1+7W7Zf3eeqAXetik6BdSk6vk/RWcU7n2viH5sv0ivAwEuNCnGz0QNx1fV3UxbJp2qrSfnUpFyg4vmEhYVdd79dnuHNzs7GZDLh4+MDwKlTp0hOTua+++6zHOPq6kqnTp1ISEhg5MiRJCYm4uHhYSl2ATp06IC7uzsJCQmEhYWRmJhIx44dcXV1tRzTrVs33nzzTU6dOkWDBg3YtWsXo0ePtmpPt27dWLp0abnbIoQQQoiyhYWFsW3bNjIyMoiLi2PcuHGsX7+eZs2alTg2OTmZp59+mkcffZSHH36Y7Oxs3nrrLUaMGMG6devQaMoebHajm5gbKXrDvLoKKTQRdLGA0DsMxB9Pxc/HG3e9Bg+9Yv7QKXjoNbjrzK9ddUq5Zlc2mFQKTOavNZhnZnbWKjhrFbo2U+j5dz6z92ZxNufqUGYVhe9S9By87MLizj50ruNSodyq++/mWpJP1VaT8qlJuYBj8rFLwTtlyhRatmxpGd6UnJwMgL+/v9Vx/v7+nDt3DoCUlBT8/PxQiv2hVhSF2rVrk5KSYjmmbt26Jc5RtK9BgwYkJyeXGqfoHOVpixBCCCHK5uTkRKNGjQBo06YNe/bsYfHixVaPGBVZtmwZbm5uzJgxw7Jt6dKlNG/enISEBDp27Oiwdlc3rnoNYd56FEXBt46B4Prln0BKVc1FrUE199XqNeaC1lmj4OaswdNJwU2nwUlbskAeFqolsrYTM/dmEncqz2rf6Wwj/TZe5MkIN2a088ZZZ5On44QQwm5sXvC+/PLL7Ny5kx9++AGtVmu1T7nmXUdVVUsUuNe60TFFE1bd6Jhrt5XnmOJutqvdkUMNamIsyakqxXJzUPybj+PIWLf+M626P7+amJMjY5UVpya9815eJpOJgoKCUvfl5uaWuB8oem0ymezeturO31VLeoGJc2X8qMy9tSoqRevoKjhrwFmrwctJg7tewUVbvp7fIoqi0MRXz7KutYhOymHWvixScq82wKTCkiOX2fxXAcu6+tK6tlNF0xRCCLuxacE7depUYmNjWbduHQ0aNLBsDwwMBMy9sPXq1bNsT01NtfS0BgQEkJqaalV4qqrKxYsXrY4p6qktfg642mMbGBhY6jHF99+oLaW5mRsYRw41qImxJKcqFmv7Xzf9LbcU/xbiODLWLf9Mq/DPrybm5MhYt2NhC/D666/TvXt37rjjDrKzs1m9ejXbt29n5cqVAEyfPp3ffvuNuLg4ALp3787ixYuZPXs2gwcPJisrizfeeIN69erRunXrykyl2gj10nFYgcsGFUVR0WsUnDTm3lp3Fw1eTgquOg16jW0nuHLWKoyM8KBzkDP/2pXBxrP5VvuTMg08sP4Cz7b0YGobL7Q2ji+EqB5UVSXXaF4DvPiyaZavDSqXDVfWAr+yVnhOoYkCE0wMsn/7bFbwTp48mdjYWNavX0+TJk2s9oWEhBAYGEh8fDx33XUXAHl5eezYscMyxCkyMpLs7GwSExMtz/EmJiaSk5NjeR0ZGcnrr79OXl4eLi7mZ0fi4+OpU6cOISEhALRv3574+HieeeYZS/z4+HjLOcrTFiGqOp+Prndz7lbmzXv6yDvs0yAhxG0jOTmZ0aNHk5KSgpeXF82bN2f16tV069YNgPPnz3PixAnL8V27dmX58uUsXLiQRYsW4eLiQrt27Vi9ejXu7rLGa3koikITD5UGtZ1w1nJTvbW2EOaj5/Nufnz0Rzaz9mWRln91OchCFd4+kM0PZ/NY0bUWTXz0Dm2bEMI2VFXlQp6JYxkGjmUaSMk1mdf6LipQDSYuFy9YrxSwRUWteuMQpXo20KZplMomBe+kSZP45ptv+Pzzz/Hx8bE8J+vu7o6HhweKojBu3DjmzZtHWFgYoaGhvP3227i7uzNo0CDAPJvy/fffz8SJE1m4cCGqqjJx4kR69OhheRd90KBBzJkzh6eeeopJkyZx7NgxFixYwEsvvWTpFR47diy9evVi/vz59OnTh/Xr17Nt2zZ++OEHgHK1RQghhBCli4mJuen9Dz/8MA8//LC9mnRb0Cngqqu8HlS9RmF0M0/urevCizvT2XLOegj775cMdIlL4ZU2nkxo4enwolwIUT5ZhSaOZxg4nmkubIsK3OMZBjILb7VsvXWHMhXC7RzDJgXv8uXLASzL/BSZPHkyU6dOBeDZZ58lNzeXF198kfT0dNq2bUtsbCyenp6W45ctW8bkyZN56KGHAOjZsydz58617Pf29mbNmjVMmjSJ6OhofHx8GD9+PBMmTLAcExUVxYoVK5g5cyazZs2iYcOGrFixgnbt2lmOKU9bhBBCCCGEtSY+er7tXpuYQ9nM3Z9FVrEb5HwjvLo7i+9O57HsHl+CX/4ULwAAIABJREFUPaW3V4jKUGBUOZl1tZA9lnn16/O5VWvuBL3W/kW2TQre9PT0Gx6jKApTp061FMCl8fX1tSwfVJbmzZuzYcOG6x7Tv3//EsX3zbZFCCGEEEKUpNMoPN3Skx71nHn613QSUgqt9iekFNLxPxd4s70Xw8PdrzspqBDi1phUlb9yjOae2gzr4vZUthGTgztrXbTgrjNPlOemMy+b5q43vzZ/rVj2F98XUnjZ7m2zy7JEQgghhBCiZmvi68T3PWuz4GA2b+/PJtd49Q47x6Dy3I4M1p7K44MuvgS6aa9zJiHE9RQYVfakFrD9fAE7Tjlx7lAy/8s0kGe88ffeDFetQqi3llAvPcEeWjydzOt8X1ukXlvAuusUdLc4aZ0jFjGRglcIIYQQQtwSrUbDC6286FXfhXHb09l30bq3N/7vfCLXJDO/ow8PN7q1pcaEuN0UL3C3n88nIbmg2BtKOsBwy+fWKRDiqSXUW0+ol45QLx2Nvc2f67hpauSIDCl4hRBCCCFEhTSt5cSmPrWZsy+LhQezyS/2mGBGgcoTW9JYezKXhXf74uusqbyGClEFXb/AvTV13TSEeusI9dJbCtpQLx3BnlqbL2FW1UnBK4QQQgghKkyr0fDyXd70DXFlzLY0DqdZ90LFncrj1+Rk3u/sQ6NKaqMQVUGBUeW31AK2n8tn+/kCElNurcD1dlII9dIR5q2z9Ng29tbRyFOLu17eWCoiBa8QQgghhLCZln5ObOnrz5t7s3jv92wMxe7jU/NMDPnpEn0CnHildiFNvHVob7PeJnH7sUWBW8dNQ5cgZxpr0rm3yR2Eeuuo5VwzhyDbmhS8QgghhBDCpvRaDa+386ZfA1dGb7nEsUzr2XXWp+hY/58UXLTQwldPW38n2vo70aa2nsZeOlnHV1Rr+UVDlG1Q4Hau40znIGcaempRFIWkpIuEBTrbqeU1kxS8QgghhBDCLu6q7cSvAwJ5dVcGS4/kcO0KoHlG2J1ayO7UQjiSA4C7TuHOWnra+utpU9uJNrWdLDf7QlRFlw0m9l8stEuBKypOCl4hhBBCCGE3TlqF2R18GNDQldFb0zidff21VHIMKjtSCtiRUgCYi2AvvULr2nruulIAt/YzL5siBYFwFFVVOXfZRFJGIUkZBv7MMK9/+2eGgbM5N78+UB03DZ2DnOkiBa7dScErhBBCCCHsrkOgMwkDA1n0exZxx9I5nacjs7B8vWCZhSpbzxWw9VyBZZuvs0IbP/Mw6Na1nWjjp+cOd8cWDQaTSq5RpfDarmtRbeUaVI5nGiyFbdHHsQwD2YZbnzlZCtzKIwWvEEIIIYRwCFedwkutvRjglswdDe/gcJqB3y4UcPBSIUfSzb1l2eUsgtPyVTb/nc/mv/Mt2/xdNFYFcANPHflGc1GaZ7jms1El16CSZ4Q8g8plo4k8AyWOKfo6t9j3F31vUf2j4Mod+8/TyEtHQ08tDT11NPTS0eDK115OMmNuVaKqKsm5pmK9tIWW3toz2UYqtiCQWZCrhi51pMCtCqTgFUIIIYQQDqUo4K7X0j5AS/sA8wQ8qmouJg+lF5KQbC6C/7hSBF8uZ8/ahTwT/z2bz3/P5t/4YBtSUTibY+RsjpGt50ru93PW0NDraiHc0PNqYRzgKjPt2ku+0dxbuzVVy9rLWZbCNinDQFY531gpDwUI9tASGeAkBW4VJAWvEEIIIYSodIqi4KpXaOfvTDv/q7PQ5hlMHEorZMeVIvhwWiHHMoy3tG5pZbmYb+LiBRO7LxSW2OeuUwgp6hX21FkK40ZeOuq5a9HJsk03lGtQScoo5Gi6gaPpBv5IN79ZciLLgPmfiTOQWeE4HjqFMG8dYT46mnjrzV9762jkqcNFJ7+nqkoKXiFsyOejv8rY4wbbS9+XPvIO+zVICCGEqOZcdBra+jvTtlgRnG80cfBiITtTCth/8UoRnGkg/+bnDqoQDeCkNc82fatyDCqH0wwcTjOU2KdToL7H1Z7hUC8dzXx1NPXV4+9y+/UM5xSaSMow8Ee6gaPp5mHwR9MLOZllm2HIYO6treeuJcxbRxMf3ZWi1lzcBklvfLUkBa8QQgghhKhWnLUa2gU40y6gWBFsMHHgUgEJyYXsu1jA4XQDmQUmnDUKTloFF62Cs9ZcQDtrwEWn4Hplu4tWwU1n/vDQmz+76zXmzzoF1yuf3fQaq+P1GnPP9ME/ktAHNeBEpoH/ZRo4duXzySwjf+UYudW5jgwqnMgyciLLCH9bD9P2c9ZYit9mvnqa+pi/rgnPC2cVmiw9tUevFLV/pBtuOMP3zXDTKYR6mYvaJt5XC9vGXjpcpbe2RpGCVwghhBBCVHvOOg3tA1xoH+Bi2aaqqkN65Fy0EOajJ8JHX2Kf0aRyNsfIySwDJ7KMHM80cDzTwIlMA6eyjeV+PvlaF/NNbDtfwLbzBVbb67lrzYWwj/5KMWweflsVh9ym55s4ml7I0Yzixe2tLfNTljpuGurpC2lT18vca+utI9RbT1036a29XUjBK4QQQgghaqSqUNBoNQohnjpCPHV0vWafqqqk5pk4caUYPnGlGP5fpoGT2UYu5t38ekdFk2cVn7hLo0BjLx1NfXTm3uArhXBDT53NnhHOM6ikFZhIyzdxKd/8OS3fRPo1r9PyTaQVqFzINZKca7v1nOq5a4m4kl+4j44IH/MwZC8nDUlJSYSF+dgslqhepOAVQgghhBCiEiiKgr+rFn9XLZEBJfdnF5o4mWXkRJa5CD6SVsjhNANHMwpv6nllk4plPdm4U3mW7c5aCPfW09T3SiHsY/46JV+h4FKhpYC99sO6oFW5lG9yyCRiRbMhF7Uz3EdPxJXnbN311X8ot7APKXiFEEIIIYSogjz0GlrU0tCilvVQaaNJ5WSWkcPphRxJK+TQpUIOpRVyIsvIzdSd+UY4cKmQA5cKgdxie1yBFFukcEu0CoR4aonw0dPMx9xjG+5jfsZWnq8VN0sKXiGEEEIIIaoRrUahsbeOxt46+oa4WrbnG1WSMgwcTrtSCF/pEbblM7G2pFOgoaeOCF/zcOsIHz0RvuaJo5y1UtgK25CCVwghhBBCiBrAWavQopa+RI9wZoF51uMj6eYlnA5dMi/pk3oLzwiXRaeAt5MGH2eFWs4afK98+F35XMtFg6/T1e0+zhrucNeil3WGhZ1JwStqvLLXxoWy1seVtXGFEEIIUVN4OWloH+BE+wAnq+0Xco0cSbfuET6WYQCTCT83nbk4ddLg56KhlrOGWi5afIsVtD5OV4tZD51SJSYJE+JaUvAKIYQQQghxGyqaMOueOs5W282zGodVUquEsC2ZzkwIIYQQ5bZs2TI6depE/fr1qV+/Pg888AAbN2687veoqsrixYtp3749AQEBhIeH8/rrrzumwUIIIW5r0sMrhBBCiHKrW7cu06dPp3HjxphMJr766iuGDRvGzz//TIsWLUr9nldeeYWNGzcyY8YMmjdvTkZGBsnJyQ5uuRBCiNuRFLxCCCGEKLfevXtbvZ42bRoffvghu3btKrXgTUpKYunSpfzyyy+Eh4c7qplCCCEEIEOahRBCCHGLjEYj3377LTk5OURGRpZ6zPfff0+DBg346aefaNWqFS1btmTs2LFcuHDBwa0VQghxO7qte3iXL1/Ou+++S3JyMhEREcyaNYtOnTpVdrNuG2XPnlz6zMkgsycLIURVcOjQIbp3705eXh7u7u58/vnnNG/evNRjT548yZkzZ4iNjWXx4sUoisK0adN49NFH+fHHH9Foyn7vPSkpqcJttcU5qoqalAtIPlWd5FN11aRcoOL53GiCtdu24I2NjWXKlCnMmzePDh06sHz5cgYPHszOnTupX79+ZTdPCCGEqLLCwsLYtm0bGRkZxMXFMW7cONavX0+zZs1KHGsymcjPz2fJkiWEhoYCsGTJEtq1a8eePXto167ddeNURE2aabYm5QKST1Un+VRdNSkXcEw+t+2Q5vfff5+hQ4cyfPhwwsPD+fe//01gYCArVqyo7KYJIYQQVZqTkxONGjWiTZs2vPbaa7Rs2ZLFixeXemxgYCA6nc5S7AI0btwYnU7H2bNnHdVkIYQQtyklPT1drexGOFpBQQF16tThww8/ZMCAAZbtkyZN4vDhw3z//feV2DohhBCieunbty916tRh6dKlJfZt3ryZhx56iL1799KwYUMATpw4QZs2bdi0aRNt27Z1dHOFEELcRm7LHt6LFy9iNBrx9/e32u7v709KSkoltUoIIYSo+l5//XV+/fVXTp06xaFDh5g+fTrbt29n8ODBAEyfPp1+/fpZjr/33ntp1aoV48ePZ//+/ezfv5/x48fTrl072rRpU1lpCCGEuE3cts/wAiiKYvVaVdUS24QQQghxVXJyMqNHjyYlJQUvLy+aN2/O6tWr6datGwDnz5/nxIkTluM1Gg3ffPMNkydPpnfv3ri4uBAdHc2bb7553QmrhBBCCFu4LQtePz8/tFptid7c1NTUEr2+QgghhLgqJibmpvcHBQXxySef2KtJQgghRJluy7dWnZycaN26NfHx8Vbb4+PjiYqKqqRWCSGEEEIIIYSwpduyhxdg/PjxjBkzhrZt2xIVFcWKFSs4f/48I0eOrOymCSGEEEIIIYSwgduyhxfgoYceYtasWfz73/+mS5cu7Ny5k5UrVxIcHFzZTRNCCCFue/Pnzyc6Opr69evTuHFjhgwZwuHDh62OUVWVWbNmERERQVBQEL179+bIkSNWx6SnpzN69GiCg4MJDg5m9OjRpKenlxrz+PHj1KtXjzvuuKNa56OqKosXL6Z9+/YEBAQQHh7O66+/Xm3z2bRpEw888AD16tWjUaNGPPbYYxw7dqzK5fL222/To0cP6tati4+PT6mxzpw5w5AhQ6hbty6NGjXipZdeoqCgwGa5ODKfgwcP8sQTT9C8eXOCgoJo164d7777LiaTqVrmU9zFixdp2rQpPj4+XLx4sVrn880339C5c2cCAwNp1KgRY8aMqZa57Nmzh/79+xMSEkJwcDD9+vXjt99+K1c7b9uCF2DUqFEcPHiQlJQUtmzZwt13322T83766adkZGTY5FxVxZ49e1i7di179+612Tlt/QekPNLS0qxeJyQksGPHDnJzc20e68yZM+zevZu9e/faNdfs7Gy2b99ObGwsa9asYfv27WRnZ9stXmkMBgNnzpxxaEx7+fvvvzl58qTd48ycOZPU1FS7xwHz/ASFhYV2O39eXh7/+c9/WLRoEWvXriUvL89m5963b5/NzlUeOTk57Nu3j/z8fAByc3NZtWoV33zzDefOnXNoW25327dv54knnmDjxo3ExcWh0+kYMGCA1d/xhQsX8v777zNnzhw2b96Mv78/AwcOJCsry3LMqFGjOHDgAKtWrWL16tUcOHCg1Bu+goICHn/8cTp16lTt83nllVf48MMPef3110lMTGTlypU2z8tR+Zw8eZKhQ4fSsWNHtm7dyn/+8x/y8vIss4JXpVzy8/Pp06cP48aNKzWO0WhkyJAhZGdn8/333/Phhx8SFxfHK6+8YrNcHJnPvn378PPz44MPPmDnzp1MnTqVuXPn8s4771TLfIp76qmnaNmypU3zqIx8PvjgA1599VWefvppduzYwbp16+jVq1e1yyU7O5uHH36YoKAg/vvf//Ljjz8SFBTEQw89ZHWestyW6/Dam7+/P9u3byc8PNxm59y8eTP33HMPOp15FPqqVatYuHAh//vf/wgMDGTMmDGMHTvWJrFmzJhBp06duP/++0lNTeWxxx5j9+7d6HQ6DAYDkZGRfPHFF9SuXbtCcWrVqkWXLl0YPnw4ffv2Ra/X26T9pTl+/DiPPPIIJ06coG3btnz99dcMHz6cX375BYB69erx7bff0qRJkwrHWr58OQsWLODvv/+22h4ZGcns2bNp3bp1hWOAuch85ZVX+PTTT8nLy0Or1QLmC6qLiwvDhw/njTfesOvPtcjBgwfp2rUrly5dqtB5VFVlwYIFxMXF4ePjwxNPPEGfPn0s+1NSUoiIiKhwHIDMzEyee+45duzYQefOnXnvvfeYMmUKH3/8MYqiEBkZyTfffIO3t3eF4lz7JguY8wwPD+e7774jLCwMAF9f3wrFAfj444957LHHcHZ2RlVV5s+fz7vvvktWVhYuLi6MGDGCmTNnVnhm3HHjxtGrVy/69u3LyZMn6du3L6mpqQQFBZGcnIy/vz9r166lQYMGFc7J19eXBg0aMHz4cIYOHUpAQECFz1mWPXv28NBDD5GRkUFwcDBr1qzhscce4+zZsyiKglar5dtvv6Vdu3Z2a4MoW3Z2NsHBwXzxxRf07NkTVVWJiIjgySefZNKkSYD5DYqwsDDeeOMNRo4cydGjR4mKiuKHH36gQ4cOAOzYsYOePXuya9cuy/9/AFOnTiUjI4O7776bl156ib/++qta5pOUlETHjh355ZdfbHofUln5rF27lpEjR3LhwgXLdW7r1q3069eP48eP4+fnVyVyKW7t2rUMHz68RE/1jz/+yCOPPMLBgwepV68eYO59e+aZZ0hKSsLLy8vmudgzn9K8+uqrbNmyhS1bttglF7B/PjExMWzYsIEXXniB/v372+3fmb3zSU9Pp1mzZnzxxRdER0fbrf2OyGXv3r1ER0ezb98+y73FyZMnLXMy3WiJu9u6h7ei6tevX+qHwWDgvvvus7y2hUGDBllunNeuXcvYsWPp0KED8+bN48EHH+S1115j9erVNon11VdfERgYCMC0adNQVZXdu3dz4cIF9uzZg06ns8m7kaqqYjQaefLJJ4mIiOBf//oXR48erfB5SzNt2jRCQkJYu3YtYWFhDBo0CEVROHToEEeOHKFJkya89tprFY6zaNEi5s2bxzPPPMOCBQsICwtjypQprFy5kpCQEHr16mWzXvJXXnmFuLg4Fi5cyLFjx0hNTSU1NZVjx47x7rvvEhcXx7Rp02wSy1Hee+895s+fzz333EPDhg0ZNWoUb7zxhtUxqmqb9+jeeOMNfv/9d5577jnOnTvHiBEjSEhIYMOGDaxbt4709HQWLlxY4TiNGzcu8REaGorBYODBBx+kUaNGNG7c2AYZwfPPP09mZiZgLn7nz5/PCy+8wLp165g2bRqff/45y5cvr3Cc//73v4SGhgLwr3/9i2bNmnH06FH27t3Ln3/+SevWrZk6dWqF4xSJjIzknXfeoUWLFvzf//0fmzZtstm5i5s+fTrdu3dn3759PPzwwwwaNIimTZty8uRJTp48SY8ePZgxY4ZdYosby87OxmQyWYa7nTp1iuTkZO677z7LMa6urnTq1ImEhAQAEhMT8fDwsJqQskOHDri7u1uOAdi4cSMbN25kzpw5DsrGfvl8//33NGjQgJ9++olWrVrRsmVLxo4dy4ULF6plPq1bt0av1/Ppp59iNBrJysriq6++4q677rJbEXIruZRHYmIi4eHhlmIXoFu3buTn59t1NIu98ilNVlbWDYcLV5Q989m/fz8LFy7kgw8+cNiyafbKJz4+HqPRSEpKClFRUTRt2pRhw4bZdRSbvXIJDQ2ldu3afP755+Tn55Ofn8+nn35KvXr1iIiIuOH337aTVtmCqqrcfffd9O/f32rbM888w/PPP0+dOnVsGqtITEwML7zwAi+//DIAjz32GHXr1mXx4sUMGjSowrEuXbpkeZfxl19+YenSpZYb8oYNG/LWW2/ZbCjRRx99RH5+Pp999hlffPEFixcvJioqiuHDhzNgwABcXFxsEichIYG1a9fSokULWrduTUhICN999x1169YFzAWxLXJatmwZ7777Lg888AAAnTp1onv37vz555/cf//9+Pj4MGPGDNasWVPhWKtXr2bFihV07drVarufnx+DBw/G39+fJ554gtmzZ1c4VqtWra6731bDZT/77DPeffddBg4cCMA///lPHn30UfLz85k5cyZQcv3sW7VhwwYWL17MPffcQ79+/WjWrBlffvmlpZdh+vTp/Otf/+LVV1+tUJzAwEDuvPNOxo8fb7l4qqrKgAEDePfddwkJCalwLkWK/5347LPPePnllxk/fjwAd999N+7u7ixZsoTRo0dXKE5OTg6urq6AuVf0iy++sPzN8PDwYOrUqfTs2bNCMYqbOXMmCxYsIDY2lk8//ZRBgwZRr149/u///o9hw4bZ7HnLffv28dNPPxESEsLkyZNZuHAhy5Yts4ySmDhxok2HgombM2XKFFq2bElkZCRgXg8YKLGcoL+/v2X4eUpKCn5+flZ/NxRFoXbt2palCc+fP8+zzz7LZ599hqenpyNSAeyXz8mTJzlz5gyxsbEsXrwYRVGYNm0ajz76KD/++KPdbuLtlU9ISAhr1qxhxIgRTJo0CZPJxJ133mmzN/ltlUt5pKSklDhHWUtl2pK98rnWvn37+PLLL1m6dOmtN7Yc7JVPTk4Oo0aNYs6cOdStW5fjx4/brtHXYa98Tp48iclk4u2332bWrFn4+voyd+5c+vTpQ2JiIm5ubrZL4gp75eLp6cn69esZOnQo8+fPByA4OJj//Oc/lvuR65Ee3grYsmULycnJ/Prrr/Tv35+hQ4cybNgwFEWhd+/eDB06lKFDh9o87vHjx0vcdPXs2dNmEziEhIRYHiYv7cKoKIpNn9GrV68eU6dO5eDBg3z11Vf4+voyYcIEwsPDefHFF20So6CgAA8PDwDc3d3RaDSW1wBeXl42ySk1NdVqWHTjxo3JzMy0PKv5j3/8g127dlU4Dpifm6xVq1aZ+2vVqmWz31NycjLR0dGMGjWq1I8BAwbYJM6ZM2e46667LK9bt27NunXrWL16tU17DAEuXLhAo0aNAKhTpw6urq5WwxubNm1qkyGNv/zyC4qiMH/+fMLCwujcuTNdunRBURTatm1L586d6dy5c4XjFCm6cTx16lSJN0PuueceTp06VeEYYWFh7N69GzD/v3Pt0KOMjAybvTFRxNXVlWHDhrFx40Z+/fVXevXqRUxMDK1atWLIkCE2iXHtTTdgGUJZ9LWtRhiIm/Pyyy+zc+dOPvvsM6vfCZR8E0xV1VJ/l2UdM3r0aB5//HHat29vh5aXzp75mEwm8vPzWbJkCXfffTedOnViyZIl/Pbbb+zZs8cO2dg3n+TkZJ5++mkeffRRNm/ezPr16/Hw8GDEiBE2nxypormUR1nH2/pvZhF751MkKSmJIUOGMG7cOKuOIFuzZz6TJ08mKirKru2/lj3zMZlMFBYWMmfOHO6//37atm3L0qVLSU1N5YcffrBJ+4uzZy65ublMmDCB9u3b89NPP7Fx40buvPNOhg4dSk5Ozg2/XwreCmjcuDE//vgjnp6edO3a1W4XkiKHDh1i3759uLi4YDQarfaZTCab/eEfMWIE06ZN49ixY4wePZpp06Zx4sQJwPxu0csvv0y3bt0qHOfaf+iKotCjRw++/PJLfv/9dyZMmMB///vfCscBc/HyySefoKoqn332GX5+fsTGxlr2r1q1yibDShs3bszmzZstr3/++WecnJwsQ8SdnZ1tdlHr3LkzL7/8cqnvkJ07d45p06bRpUsXm8Rq2rQpzZs35+mnny7147HHHrNJHD8/P86ePWu1LSwsjLi4OGJjYy2jGmyhVq1aVpOJ9erVy+p53ZycHJycnGwS5+uvv6ZHjx5ER0ezfv36Cp/zen744Qfi4uJwcXEpcRHIzc21Se/OhAkTmDZtGlu2bOH5559nypQpbNmyhXPnzrF161YmTpxI3759KxwHSr8JbNq0KXPmzOGPP/7g/ffft9kkba1bt+add97h9OnT/Pvf/6ZBgwZWPRVLliyhadOmNoklym/q1Kl8++23xMXFWT0XXvR39dqesdTUVEtvQkBAAKmpqVZvVKiqysWLFy3HbN26lTlz5uDn54efnx9PP/00OTk5+Pn58fHHH1e7fAIDA9HpdJbHDsB8bdLpdCX+vlaHfJYtW4abmxszZsygVatW3H333SxdupRffvmlwsNvbZlLeQQEBJQ4x8WLFzEajTd1nvKydz5F/vzzT/r06cNDDz1k89nAi7N3Plu2bOHLL7+0/C0oKnybNGlS4vEqW7B3PkXnKf4sv7e3N0FBQTb/W2DvXFatWsWJEydYvHgxd911F+3bt2f58uWcPXu2XPdVMqS5gvR6PbNmzeLee+9l2LBhPPHEE3Z7l+6hhx6yXBR27txJ27ZtLfsOHDhg9UxIRTz11FOcPXuWjh070rBhQ06fPk3btm0tk1a1atWKFStWVDjO9XpKgoKCePHFF23Wwzt58mSGDRvGokWL0Ov1xMbG8vTTT7N582a0Wi379++3ybONzz//PE8++SSbN2/GxcWF7777jjFjxlj+TWzfvt1mN8zz5s3jkUceoUWLFoSHh+Pv74+iKKSkpHD06FEiIiJYuXKlTWJFRUVddwSBh4eHTWYA7dChA+vWrSsxY3qTJk1Yu3atzYoogGbNmrF3717LcO1rf//79u2zySRmRcaNG0enTp148sknbfZGTmmefvppy9fbtm2zejZu165dNplIasiQIaSlpTF06FBMJhNGo9EyDB3MI07eeuutCseB6/+dcHZ2ZsiQITbr4X311VcZNGgQX3/9NbVr12bdunVMmDCBsLAwFEUhKyuLr7/+2iaxRPlMnjyZ2NhY1q9fX+L/x5CQEAIDA4mPj7eMDMnLy2PHjh2WZ60jIyPJzs4mMTHR8v9CYmIiOTk5lte//vqr1Xm///575s2bx6ZNmyyPvVSnfDp06IDBYODEiRM0bNgQML9ZbTAYbDaviCPzyc3NLdFbVPTalj28Fc2lPCIjI3n77bf566+/LI9ixMfH4+zsbLMJLYs4Ih+AP/74g379+jFgwABmzZpls/ZfyxH5rFmzxmqJqD179jBhwgTWr19vs7k2ijgin6JHtI4dO2b595adnU1ycrKydfYuAAANEElEQVRN/xY4Ipfc3FwURbF6016j0aAoSrn+DkjBayM9evQgPj6e0aNHYzAYbH7+/fv3W70uPhwXzM9QPvvsszaL99ZbbzFy5Eg2bNhgeQYgMDCQDh06cO+999qkqJ88eTLu7u42aO2NdevWjYSEBPbt20ebNm0IDg7m+++/Z9myZeTm5vL6669zzz33VDjOwIED8fDwYOXKleTn5zNr1iyGDx9u2T9gwACbDf+tV68e27dvZ9OmTezatcvy7llkZCQzZszgvvvus9mzWjd6Drhhw4Y26bl87rnnypy4IyIigri4ONauXVvhOABLly697s+nVq1aNl8qolWrVvz888+89NJL1K1b1yY9yMWVNiN0cQEBATaZnA1g7NixPPbYY8THx5f4G2HLG4P333/fbjOXXuuuu+7i4MGDJCUlERoaioeHB+vWrWPlypXk5eURHR1tNexd2NekSZP45ptv+Pzzz/Hx8bE8C+bu7o6HhweKojBu3DjmzZtHWFgYoaGhvP3227i7u1vmswgPD+f+++9n4sSJLFy4EFVVmThxIj169LD8Lps1a2YVd+/evWg0mhLbq0s+9957L61atWL8+PGWAmTq1Km0a9fuhjOZVsV8unfvzuLFi5k9ezaDBw8mKyuLN954g3r16tmsSLRFLmB+LCctLY3Tp08D5s4IgEaNGuHh4cF9991H06ZNGTt2LDNnziQtLY1XX32Vf/7znzb9O+eofI4cOUK/fv3o0qULL7zwgiUOXO3Zq075FB8VAVeXz2zSpIlNJ0hzZD69evViypQpvPPOO/j4+DBr1ixq165Njx49qlUu0dHRvPrqq7zwwguMGTMGk8nEO++8g1arLdf9uyxLJIQQQogqp6yZXidPnmx5pl9VVWbPns3HH39Meno6bdu25e2337YqVtPS0pg8eTIbNmwAzCMQ5s6dW+b5v/jiC7ssS+TIfM6fP8/kyZPZtGkTLi4uREdH8+abb9p0WS9H5vPtt9+ycOFCjh8/jouLC+3atWP69Onlmp3VkbmMGzeOr776qsR51q1bZ3nE6MyZM0yaNImtW7fi4uLCoEGDmDlzJs7OzjbJxZH5zJo1q8yZzcuzhFF5OfL3U9y2bdvo27evzZclcmQ+WVlZvPzyy6xbtw5VVenQoQOzZ8+2jP6oTrnEx8czZ84cDh8+jKIotGzZkmnTplmNZCuLFLw2cPz4cRISEkhJSUFRFAICAoiMjLT58IfSYvn7+xMVFVWtY1V2Th06dLBMXmTPOPbKqSw5OTns27evxPDg6hxLcqoesWpiTo6OJYQQQgjbkIK3AjIyMhg7diw//PAD7u7u1K5d2zLZwuXLl3nwwQf54IMPbDJMpSbGkpzs6+DBg3Tt2pVLly7VmFiSU/WIVRNzcnQsIYQQQtiGzNJcAS+99BInT55kw4YNnD17ln379rF//37Onj1refb1pZdekliVHMeRsRyZkxBCCCGEEOL6pIe3AoKDg4mNjaVdu3al7k9MTGTQoEGWB7AlVuXEcWQsR+Z0vTV4i7NFb5SjYklOFSM5VZ9YQgghhHAMmaXZjmw1Q+7tGktyuj5XV1fGjRtHy5YtS91/+vRpm83I66hYklP1iFUTc3J0LCGEEEI4hhS8FfDggw/yzDPPsHDhQtq3b2+1b9euXTz33HP07NlTYlVyHEfGcmROLVu2xNfX17Iw+rUOHjxokziOjCU5VY9YNTEnR8cSQgghhGNIwVsBc+fOZdSoUXTv3h1PT0/8/PxQFIXU1FSys7Pp1q1bmdO1SyzJqaIeeOABMjMzy9zv6+vLo48+Wq1iSU7VI1ZNzMnRsYQQQgjhGPIMrw0cPXqUxMRELly4AGBZlqhJkyYSqwrFcWQsR+YkhBBCCCGEKJ0UvEIIIYQQQgghaiQZ0lxBOTk5rF69moSEBFJSUlAUBX9/fzp06MDDDz+Mu7u7xKoCcRwZqybm5MhYklP1iFUTc3J0LCGEEELYn/TwVsAff/zBwIEDyc7OplOnTvj7+6OqKqmpqezYsQMPDw9iY2OJiIiQWJJTtc3JkbEkJ8npdoklhBCVKTc3F1dX18puhhAOIQVvBfTp0wd/f39iYmJwcXGx2peXl8dTTz1FSkoK69evl1iVGMeRsWpiTo6MJTlVjORUfWIJIW4v8fHxDBw4kM8++4y+ffta7du4cSNDhgzh66+/5sEHH+T8+fPMmjWL/2/v7kKa3AM4jv9magW9LAhdodPMclorvAgjCsoW2KsRkUnlC4FtN7NgxCZExxWVWDAmqIykDEGSomIr2yCiF6joTRnMoIbWhaKJWy2Lss1zcTg7Saduds5/27PfB7rZ9vh9fjfS49yj0+nE2NgYMjIyUFlZCb1eD5lMFj6uqakJDocDr1+/xvj4OBYvXgytVouKioopX1+tVmPp0qWora2F2WyG2+3G4cOHYTKZhGwnijZe8EZgwYIFuHv37i9/2u/xeLBx40YMDQ2xFcWOyJYUN4lscVNkuCl+WkSUWEKhENRqNQoLC9HR0THluYMHD+LevXt49eoVfD4fNmzYgO/fv6OyshIKhQKPHj1CV1cXtFotzpw5Ez4uPz8fGo0GBQUFkMlkcDgcePjwISwWC6qqqsKvU6vVSE5Ohs/nQ0VFBXJycpCRkQGNRiNqPlFUJUX7BOKZXC7Hmzdvfvm81+uFXC5nK8odkS0pbhLZ4qb4aElxk+gWESWWpKQklJWVweVywe/3hx8PBALo7u7Grl27kJycjJMnT+Lr16+4f/8+TCYTqqurYbPZoNfrYbPZ8Pbt2/Cxz58/R1NTE3Q6HbRaLRwOB9avXw+r1fpTv7+/H83NzTCbzaiqquLFLiWUaUaj8Y9on0S8CgQCqK+vBwDMnDkTwWAQnz59gtfrRWdnJ44dO4aamhqsXbuWLW6K200iW9zETYnSIqLEs3DhQrS2tiIrKwuFhYUAgCtXruDGjRtobGyEQqGATqfDli1bUFxcjC9fvoT/zZo1C52dnVixYgXUajUAICUlBQAwMTGBjx8/4vPnz/D7/bh+/Tp0Ol34oxktLS2YPXs2LBZLdIYTRRl/pTlCFosFra2tGB4eDn+uYnJyEunp6dDpdKitrWUrBjoiW1LcJLLFTfHRkuIm0S0iSjwajQYpKSno7u4GAJSWlmJwcBBPnz7F+/fvsWTJkt8ebzabodfrAQA3b95EY2Mj3G43gsHglNe53W5kZmYC+OtXmjMzM3Hr1q3/YRFR7OMF739kYGAAIyMjAIC0tDRkZ2ezFYMdkS0pbhLZ4qb4aElxk+gWESWOtrY2GAwG9PT0IDU1FcuWLUNdXR0MBgOGh4eRl5eH3bt3Y//+/f96fE5ODpRKJR4/fozNmzdj9erVKC8vh0KhQGpqKlwuF5qbm9Hb24usrCwA/9y06urVqyKnEsUMXvASEREREQng9/uRl5cHg8GA6dOn4/jx4+jt7YVSqUQwGMSiRYtQXFyMixcv/vbrGI1GtLe3o7+/f8pd5U+cOIFz587xgpfoB7xpVYT8fj+cTieePHmCycmpPzsYHx9HQ0MDWzHQEdmS4iaRLW6Kj5YUN4luEVHikcvlKCkpQVdXFy5fvow1a9ZAqVQCAKZNm4YdO3bA4XCgp6fnp2M/fPiAiYmJ8GtlMhlCoVD4eb/f/9MdoImI7/BGpK+vDzt37sTo6ChCoRBWrlyJS5cuhb9xjYyMQKVSYWxsjC1uittNIlvcxE2J0iKixHX79m3s3bsXAGC1Wqf83dzR0VFs2rQJQ0NDOHDgAAoKChAIBODxeGC32/HixQukp6fjwYMH2L59O4qKilBWVgafz4f29nbMnTsXbreb7/AS/YDv8Eagvr4eq1atwrt379DX14fs7GyUlJTA6/WyFUMdkS0pbhLZ4qb4aElxk+gWESUujUaDtLQ0zJgxA6WlpVOemz9/Pu7cuYPq6mo4nU4cPXoUVqsVAwMDMBqNmDdvHgBg3bp1aGlpgc/ng8lkQkdHB2pqanDo0KFoTCKKaXyHNwK5ubmw2+3Iz88PP1ZXV4dr167Bbrdjzpw5/9m7AVJscVNkpNjipshwU/y0iChxhUIhLF++HEVFRbhw4UK0T4dI8pKjfQLx7Nu3b+E/W/G3U6dOYXJyElu3bsX58+fZioGOyJYUN4lscVN8tKS4SXSLiBKXy+XC4OAgysvLo30qRAmBF7wRyM3NxcuXL6FSqaY8fvr0aYRCIezbt4+tGOiIbElxk8gWN8VHS4qbRLeIKPE8e/YMHo8HZ8+ehUqlgkajifYpESUEfoY3Atu2bfvlDQAaGhqwZ8+en+7yyZb4jsiWFDeJbHFTfLSkuEl0i4gST1tbG44cOQK5XA6bzYakJP43nEgEfoaXiIiIiIiIJIk/WiIiIiIiIiJJ4gUvERERERERSRIveImIiIiIiEiSeMFLREREREREksQLXiIiIiIiIpIkXvASERERERGRJP0Jn33mo8i+tUwAAAAASUVORK5CYII=\n",
618 "text/plain": [
619 "<Figure size 1008x288 with 2 Axes>"
620 ]
621 },
622 "metadata": {},
623 "output_type": "display_data"
624 }
625 ],
626 "source": [
627 "fig, axes = plt.subplots(ncols=2, figsize=(14, 4))\n",
628 "combined.year.value_counts().sort_index().plot.bar(title='Reviews per Year', ax=axes[0]);\n",
629 "sns.lineplot(x='year', y='stars', data=combined, ax=axes[1])\n",
630 "axes[1].set_title('Stars per year');"
631 ]
632 },
633 {
634 "cell_type": "code",
635 "execution_count": 15,
636 "metadata": {
637 "ExecuteTime": {
638 "end_time": "2018-11-26T06:40:38.395097Z",
639 "start_time": "2018-11-26T06:40:38.347029Z"
640 }
641 },
642 "outputs": [
643 {
644 "data": {
645 "text/plain": [
646 "0 165419\n",
647 "1 320468\n",
648 "2 575835\n",
649 "3 783098\n",
650 "4 848641\n",
651 "5 848339\n",
652 "6 843802\n",
653 "7 869333\n",
654 "8 613929\n",
655 "9 424050\n",
656 "10 226052\n",
657 "11 119301\n",
658 "12 39268\n",
659 "13 8140\n",
660 "14 225\n",
661 "Name: member_yrs, dtype: int64"
662 ]
663 },
664 "execution_count": 15,
665 "metadata": {},
666 "output_type": "execute_result"
667 }
668 ],
669 "source": [
670 "combined.member_yrs.value_counts().sort_index()"
671 ]
672 },
673 {
674 "cell_type": "code",
675 "execution_count": 16,
676 "metadata": {
677 "ExecuteTime": {
678 "end_time": "2018-11-26T06:40:38.453826Z",
679 "start_time": "2018-11-26T06:40:38.396927Z"
680 }
681 },
682 "outputs": [
683 {
684 "data": {
685 "text/plain": [
686 "1.0 14.989141\n",
687 "2.0 8.112505\n",
688 "3.0 11.057300\n",
689 "4.0 21.971388\n",
690 "5.0 43.869666\n",
691 "Name: stars, dtype: float64"
692 ]
693 },
694 "execution_count": 16,
695 "metadata": {},
696 "output_type": "execute_result"
697 }
698 ],
699 "source": [
700 "combined.stars.value_counts(normalize=True).sort_index().mul(100)"
701 ]
702 },
703 {
704 "cell_type": "markdown",
705 "metadata": {},
706 "source": [
707 "### Create train-test split"
708 ]
709 },
710 {
711 "cell_type": "code",
712 "execution_count": 17,
713 "metadata": {
714 "ExecuteTime": {
715 "end_time": "2018-11-21T16:54:02.322183Z",
716 "start_time": "2018-11-21T16:54:01.675175Z"
717 }
718 },
719 "outputs": [],
720 "source": [
721 "train = combined[combined.year < 2018]\n",
722 "test = combined[combined.year == 2018]"
723 ]
724 },
725 {
726 "cell_type": "code",
727 "execution_count": 19,
728 "metadata": {
729 "ExecuteTime": {
730 "end_time": "2018-11-21T16:57:34.506035Z",
731 "start_time": "2018-11-21T16:54:04.469343Z"
732 }
733 },
734 "outputs": [],
735 "source": [
736 "train.to_parquet(parquet_dir / 'train.parquet')\n",
737 "test.to_parquet(parquet_dir / 'test.parquet')"
738 ]
739 },
740 {
741 "cell_type": "code",
742 "execution_count": 20,
743 "metadata": {
744 "ExecuteTime": {
745 "end_time": "2018-11-26T06:43:57.832860Z",
746 "start_time": "2018-11-26T06:43:35.086606Z"
747 }
748 },
749 "outputs": [],
750 "source": [
751 "train = pd.read_parquet(parquet_dir / 'train.parquet')\n",
752 "test = pd.read_parquet(parquet_dir / 'test.parquet')"
753 ]
754 },
755 {
756 "cell_type": "markdown",
757 "metadata": {},
758 "source": [
759 "### Benchmark Accuracy"
760 ]
761 },
762 {
763 "cell_type": "markdown",
764 "metadata": {},
765 "source": [
766 "Using the most frequent number of stars (=5) to predict the test set achieve an accuracy close to 51%:"
767 ]
768 },
769 {
770 "cell_type": "code",
771 "execution_count": 21,
772 "metadata": {
773 "ExecuteTime": {
774 "end_time": "2018-11-26T06:45:53.948801Z",
775 "start_time": "2018-11-26T06:45:53.909151Z"
776 }
777 },
778 "outputs": [],
779 "source": [
780 "test['predicted'] = train.stars.mode().iloc[0]"
781 ]
782 },
783 {
784 "cell_type": "code",
785 "execution_count": 22,
786 "metadata": {
787 "ExecuteTime": {
788 "end_time": "2018-11-26T06:45:56.989648Z",
789 "start_time": "2018-11-26T06:45:56.969783Z"
790 }
791 },
792 "outputs": [
793 {
794 "data": {
795 "text/plain": [
796 "0.5096963305260762"
797 ]
798 },
799 "execution_count": 22,
800 "metadata": {},
801 "output_type": "execute_result"
802 }
803 ],
804 "source": [
805 "accuracy_score(test.stars, test.predicted)"
806 ]
807 },
808 {
809 "cell_type": "markdown",
810 "metadata": {},
811 "source": [
812 "### Create Yelp review document-term matrix"
813 ]
814 },
815 {
816 "cell_type": "code",
817 "execution_count": 23,
818 "metadata": {
819 "ExecuteTime": {
820 "end_time": "2018-11-21T17:09:18.133401Z",
821 "start_time": "2018-11-21T16:57:34.508544Z"
822 }
823 },
824 "outputs": [
825 {
826 "data": {
827 "text/plain": [
828 "<5508238x10000 sparse matrix of type '<class 'numpy.int64'>'\n",
829 "\twith 250524808 stored elements in Compressed Sparse Row format>"
830 ]
831 },
832 "execution_count": 23,
833 "metadata": {},
834 "output_type": "execute_result"
835 }
836 ],
837 "source": [
838 "vectorizer = CountVectorizer(stop_words='english', ngram_range=(1, 2), max_features=10000)\n",
839 "train_dtm = vectorizer.fit_transform(train.text)\n",
840 "train_dtm"
841 ]
842 },
843 {
844 "cell_type": "code",
845 "execution_count": 24,
846 "metadata": {
847 "ExecuteTime": {
848 "end_time": "2018-11-21T17:10:45.497868Z",
849 "start_time": "2018-11-21T17:09:18.134872Z"
850 }
851 },
852 "outputs": [],
853 "source": [
854 "sparse.save_npz(text_features_dir / 'train_dtm', train_dtm)"
855 ]
856 },
857 {
858 "cell_type": "code",
859 "execution_count": 25,
860 "metadata": {
861 "ExecuteTime": {
862 "end_time": "2018-11-22T20:41:49.703532Z",
863 "start_time": "2018-11-22T20:41:43.251565Z"
864 }
865 },
866 "outputs": [],
867 "source": [
868 "train_dtm = sparse.load_npz(text_features_dir / 'train_dtm.npz')"
869 ]
870 },
871 {
872 "cell_type": "code",
873 "execution_count": 26,
874 "metadata": {
875 "ExecuteTime": {
876 "end_time": "2018-11-21T17:11:38.827700Z",
877 "start_time": "2018-11-21T17:10:45.499840Z"
878 }
879 },
880 "outputs": [],
881 "source": [
882 "test_dtm = vectorizer.transform(test.text)\n",
883 "sparse.save_npz(text_features_dir / 'test_dtm', test_dtm)"
884 ]
885 },
886 {
887 "cell_type": "code",
888 "execution_count": 27,
889 "metadata": {
890 "ExecuteTime": {
891 "end_time": "2018-11-26T06:54:34.163446Z",
892 "start_time": "2018-11-26T06:54:33.419690Z"
893 }
894 },
895 "outputs": [],
896 "source": [
897 "test_dtm = sparse.load_npz(text_features_dir / 'test_dtm.npz')"
898 ]
899 },
900 {
901 "cell_type": "markdown",
902 "metadata": {},
903 "source": [
904 "### Train Multiclass Naive Bayes"
905 ]
906 },
907 {
908 "cell_type": "markdown",
909 "metadata": {},
910 "source": [
911 "Next, we train a Naive Bayes classifier using a document-term matrix produced by the CountVectorizer with default settings."
912 ]
913 },
914 {
915 "cell_type": "code",
916 "execution_count": 28,
917 "metadata": {
918 "ExecuteTime": {
919 "end_time": "2018-11-22T20:42:27.094801Z",
920 "start_time": "2018-11-22T20:42:24.352441Z"
921 }
922 },
923 "outputs": [],
924 "source": [
925 "nb = MultinomialNB()\n",
926 "nb.fit(train_dtm,train.stars)\n",
927 "predicted_stars = nb.predict(test_dtm)"
928 ]
929 },
930 {
931 "cell_type": "markdown",
932 "metadata": {},
933 "source": [
934 "### Evaluate results"
935 ]
936 },
937 {
938 "cell_type": "markdown",
939 "metadata": {},
940 "source": [
941 "The prediction produces 64.7% accuracy on the test set, a 24.4% improvement over the benchmark:"
942 ]
943 },
944 {
945 "cell_type": "code",
946 "execution_count": 29,
947 "metadata": {
948 "ExecuteTime": {
949 "end_time": "2018-11-22T20:42:34.405562Z",
950 "start_time": "2018-11-22T20:42:34.367343Z"
951 }
952 },
953 "outputs": [
954 {
955 "data": {
956 "text/plain": [
957 "0.6440158551434961"
958 ]
959 },
960 "execution_count": 29,
961 "metadata": {},
962 "output_type": "execute_result"
963 }
964 ],
965 "source": [
966 "accuracy_score(test.stars, predicted_stars)"
967 ]
968 },
969 {
970 "cell_type": "code",
971 "execution_count": 30,
972 "metadata": {
973 "ExecuteTime": {
974 "end_time": "2018-11-22T20:43:39.611929Z",
975 "start_time": "2018-11-22T20:43:39.309889Z"
976 }
977 },
978 "outputs": [
979 {
980 "data": {
981 "text/html": [
982 "<div>\n",
983 "<style scoped>\n",
984 " .dataframe tbody tr th:only-of-type {\n",
985 " vertical-align: middle;\n",
986 " }\n",
987 "\n",
988 " .dataframe tbody tr th {\n",
989 " vertical-align: top;\n",
990 " }\n",
991 "\n",
992 " .dataframe thead th {\n",
993 " text-align: right;\n",
994 " }\n",
995 "</style>\n",
996 "<table border=\"1\" class=\"dataframe\">\n",
997 " <thead>\n",
998 " <tr style=\"text-align: right;\">\n",
999 " <th></th>\n",
1000 " <th>1</th>\n",
1001 " <th>2</th>\n",
1002 " <th>3</th>\n",
1003 " <th>4</th>\n",
1004 " <th>5</th>\n",
1005 " </tr>\n",
1006 " </thead>\n",
1007 " <tbody>\n",
1008 " <tr>\n",
1009 " <th>1</th>\n",
1010 " <td>153289</td>\n",
1011 " <td>37619</td>\n",
1012 " <td>7355</td>\n",
1013 " <td>3023</td>\n",
1014 " <td>3896</td>\n",
1015 " </tr>\n",
1016 " <tr>\n",
1017 " <th>2</th>\n",
1018 " <td>25124</td>\n",
1019 " <td>30016</td>\n",
1020 " <td>18722</td>\n",
1021 " <td>4593</td>\n",
1022 " <td>2880</td>\n",
1023 " </tr>\n",
1024 " <tr>\n",
1025 " <th>3</th>\n",
1026 " <td>11841</td>\n",
1027 " <td>17546</td>\n",
1028 " <td>39995</td>\n",
1029 " <td>23913</td>\n",
1030 " <td>6615</td>\n",
1031 " </tr>\n",
1032 " <tr>\n",
1033 " <th>4</th>\n",
1034 " <td>7871</td>\n",
1035 " <td>5717</td>\n",
1036 " <td>22960</td>\n",
1037 " <td>110209</td>\n",
1038 " <td>44228</td>\n",
1039 " </tr>\n",
1040 " <tr>\n",
1041 " <th>5</th>\n",
1042 " <td>29517</td>\n",
1043 " <td>3545</td>\n",
1044 " <td>6686</td>\n",
1045 " <td>135578</td>\n",
1046 " <td>424924</td>\n",
1047 " </tr>\n",
1048 " </tbody>\n",
1049 "</table>\n",
1050 "</div>"
1051 ],
1052 "text/plain": [
1053 " 1 2 3 4 5\n",
1054 "1 153289 37619 7355 3023 3896\n",
1055 "2 25124 30016 18722 4593 2880\n",
1056 "3 11841 17546 39995 23913 6615\n",
1057 "4 7871 5717 22960 110209 44228\n",
1058 "5 29517 3545 6686 135578 424924"
1059 ]
1060 },
1061 "execution_count": 30,
1062 "metadata": {},
1063 "output_type": "execute_result"
1064 }
1065 ],
1066 "source": [
1067 "stars = index=list(range(1,6))\n",
1068 "pd.DataFrame(confusion_matrix(test.stars, predicted_stars), \n",
1069 " columns=stars,\n",
1070 " index=stars)"
1071 ]
1072 },
1073 {
1074 "cell_type": "markdown",
1075 "metadata": {
1076 "slideshow": {
1077 "slide_type": "slide"
1078 }
1079 },
1080 "source": [
1081 "\n",
1082 "### Combine non-text features with the document-term matrix"
1083 ]
1084 },
1085 {
1086 "cell_type": "markdown",
1087 "metadata": {},
1088 "source": [
1089 "The dataset contains various numerical features. The vectorizers produce [scipy.sparse matrices](https://docs.scipy.org/doc/scipy/reference/sparse.html). To combine the vectorized text data with other features, we need to first convert these to sparse matrices as well; many sklearn objects and other libraries like lightgbm can handle these very memory-efficient data structures. Converting the sparse matrix to a dense numpy array risks memory overflow.\n",
1090 "\n",
1091 "Most variables are categorical so we use one-hot encoding since we have a fairly large dataset to accommodate the increase in features.\n",
1092 "\n",
1093 "We convert the encoded numerical features and combine them with the document-term matrix:"
1094 ]
1095 },
1096 {
1097 "cell_type": "markdown",
1098 "metadata": {},
1099 "source": [
1100 "#### One-hot-encoding "
1101 ]
1102 },
1103 {
1104 "cell_type": "code",
1105 "execution_count": 31,
1106 "metadata": {
1107 "ExecuteTime": {
1108 "end_time": "2018-11-22T21:03:01.316248Z",
1109 "start_time": "2018-11-22T21:02:59.935577Z"
1110 }
1111 },
1112 "outputs": [],
1113 "source": [
1114 "df = pd.concat([train.drop(['text', 'stars'], axis=1).assign(source='train'),\n",
1115 " test.drop(['text', 'stars'], axis=1).assign(source='test')])"
1116 ]
1117 },
1118 {
1119 "cell_type": "code",
1120 "execution_count": 32,
1121 "metadata": {
1122 "ExecuteTime": {
1123 "end_time": "2018-11-22T21:03:19.138369Z",
1124 "start_time": "2018-11-22T21:03:05.824537Z"
1125 }
1126 },
1127 "outputs": [
1128 {
1129 "name": "stdout",
1130 "output_type": "stream",
1131 "text": [
1132 "<class 'pandas.core.frame.DataFrame'>\n",
1133 "Int64Index: 6685900 entries, 0 to 6685899\n",
1134 "Data columns (total 25 columns):\n",
1135 "average_stars 6685900 non-null int64\n",
1136 "compliment_cool 6685900 non-null int64\n",
1137 "compliment_cute 6685900 non-null int64\n",
1138 "compliment_funny 6685900 non-null int64\n",
1139 "compliment_hot 6685900 non-null int64\n",
1140 "compliment_list 6685900 non-null int64\n",
1141 "compliment_more 6685900 non-null int64\n",
1142 "compliment_note 6685900 non-null int64\n",
1143 "compliment_photos 6685900 non-null int64\n",
1144 "compliment_plain 6685900 non-null int64\n",
1145 "compliment_profile 6685900 non-null int64\n",
1146 "compliment_writer 6685900 non-null int64\n",
1147 "cool 6685900 non-null int64\n",
1148 "cool_user 6685900 non-null int64\n",
1149 "fans 6685900 non-null int64\n",
1150 "funny 6685900 non-null int64\n",
1151 "funny_user 6685900 non-null int64\n",
1152 "review_count 6685900 non-null int64\n",
1153 "useful 6685900 non-null int64\n",
1154 "useful_user 6685900 non-null int64\n",
1155 "member_yrs 6685900 non-null int64\n",
1156 "month 6685900 non-null int64\n",
1157 "predicted 1177662 non-null float64\n",
1158 "source 6685900 non-null object\n",
1159 "year 6685900 non-null int64\n",
1160 "dtypes: float64(1), int64(23), object(1)\n",
1161 "memory usage: 1.3+ GB\n"
1162 ]
1163 }
1164 ],
1165 "source": [
1166 "uniques = df.nunique()\n",
1167 "binned = pd.concat([(df.loc[:, uniques[uniques > 20].index]\n",
1168 " .apply(pd.qcut, q=10, labels=False, duplicates='drop')),\n",
1169 " df.loc[:, uniques[uniques <= 20].index]], axis=1)\n",
1170 "binned.info(null_counts=True)"
1171 ]
1172 },
1173 {
1174 "cell_type": "code",
1175 "execution_count": 36,
1176 "metadata": {
1177 "ExecuteTime": {
1178 "end_time": "2018-11-22T21:05:25.101495Z",
1179 "start_time": "2018-11-22T21:05:21.505198Z"
1180 }
1181 },
1182 "outputs": [
1183 {
1184 "name": "stdout",
1185 "output_type": "stream",
1186 "text": [
1187 "<class 'pandas.core.frame.DataFrame'>\n",
1188 "Int64Index: 6685900 entries, 0 to 6685899\n",
1189 "Columns: 118 entries, source to year_2018\n",
1190 "dtypes: object(1), uint8(117)\n",
1191 "memory usage: 848.0+ MB\n"
1192 ]
1193 }
1194 ],
1195 "source": [
1196 "dummies = pd.get_dummies(binned, columns=binned.columns.drop('source'), drop_first=True)\n",
1197 "dummies.info()"
1198 ]
1199 },
1200 {
1201 "cell_type": "code",
1202 "execution_count": 37,
1203 "metadata": {
1204 "ExecuteTime": {
1205 "end_time": "2018-11-22T21:06:17.495638Z",
1206 "start_time": "2018-11-22T21:06:15.660761Z"
1207 }
1208 },
1209 "outputs": [
1210 {
1211 "name": "stdout",
1212 "output_type": "stream",
1213 "text": [
1214 "<class 'pandas.core.frame.DataFrame'>\n",
1215 "Int64Index: 5508238 entries, 0 to 6685896\n",
1216 "Columns: 117 entries, average_stars_1 to year_2018\n",
1217 "dtypes: uint8(117)\n",
1218 "memory usage: 656.6 MB\n"
1219 ]
1220 }
1221 ],
1222 "source": [
1223 "train_dummies = dummies[dummies.source=='train'].drop('source', axis=1)\n",
1224 "train_dummies.info()"
1225 ]
1226 },
1227 {
1228 "cell_type": "markdown",
1229 "metadata": {},
1230 "source": [
1231 "#### Train set"
1232 ]
1233 },
1234 {
1235 "cell_type": "code",
1236 "execution_count": 38,
1237 "metadata": {
1238 "ExecuteTime": {
1239 "end_time": "2018-11-22T21:06:24.503323Z",
1240 "start_time": "2018-11-22T21:06:20.675097Z"
1241 },
1242 "slideshow": {
1243 "slide_type": "slide"
1244 }
1245 },
1246 "outputs": [
1247 {
1248 "data": {
1249 "text/plain": [
1250 "(5508238, 117)"
1251 ]
1252 },
1253 "execution_count": 38,
1254 "metadata": {},
1255 "output_type": "execute_result"
1256 }
1257 ],
1258 "source": [
1259 "# Cast other feature columns to float and convert to a sparse matrix.\n",
1260 "train_numeric = sparse.csr_matrix(train_dummies.astype(np.int8))\n",
1261 "train_numeric.shape"
1262 ]
1263 },
1264 {
1265 "cell_type": "code",
1266 "execution_count": 39,
1267 "metadata": {
1268 "ExecuteTime": {
1269 "end_time": "2018-11-22T21:06:43.353223Z",
1270 "start_time": "2018-11-22T21:06:37.697296Z"
1271 },
1272 "slideshow": {
1273 "slide_type": "slide"
1274 }
1275 },
1276 "outputs": [
1277 {
1278 "data": {
1279 "text/plain": [
1280 "(5508238, 10117)"
1281 ]
1282 },
1283 "execution_count": 39,
1284 "metadata": {},
1285 "output_type": "execute_result"
1286 }
1287 ],
1288 "source": [
1289 "# Combine sparse matrices.\n",
1290 "train_dtm_numeric = sparse.hstack((train_dtm, train_numeric))\n",
1291 "train_dtm_numeric.shape"
1292 ]
1293 },
1294 {
1295 "cell_type": "code",
1296 "execution_count": 40,
1297 "metadata": {
1298 "ExecuteTime": {
1299 "end_time": "2018-11-22T21:08:24.531255Z",
1300 "start_time": "2018-11-22T21:06:43.354518Z"
1301 }
1302 },
1303 "outputs": [],
1304 "source": [
1305 "sparse.save_npz(text_features_dir / 'train_dtm_numeric', train_dtm_numeric)"
1306 ]
1307 },
1308 {
1309 "cell_type": "markdown",
1310 "metadata": {},
1311 "source": [
1312 "#### Repeat for test set"
1313 ]
1314 },
1315 {
1316 "cell_type": "code",
1317 "execution_count": 41,
1318 "metadata": {
1319 "ExecuteTime": {
1320 "end_time": "2018-11-22T21:08:26.072988Z",
1321 "start_time": "2018-11-22T21:08:24.532631Z"
1322 },
1323 "slideshow": {
1324 "slide_type": "slide"
1325 }
1326 },
1327 "outputs": [
1328 {
1329 "data": {
1330 "text/plain": [
1331 "(1177662, 10117)"
1332 ]
1333 },
1334 "execution_count": 41,
1335 "metadata": {},
1336 "output_type": "execute_result"
1337 }
1338 ],
1339 "source": [
1340 "test_dummies = dummies[dummies.source=='test'].drop('source', axis=1)\n",
1341 "test_numeric = sparse.csr_matrix(test_dummies.astype(np.int8))\n",
1342 "test_dtm_numeric = sparse.hstack((test_dtm, test_numeric))\n",
1343 "test_dtm_numeric.shape"
1344 ]
1345 },
1346 {
1347 "cell_type": "code",
1348 "execution_count": 42,
1349 "metadata": {
1350 "ExecuteTime": {
1351 "end_time": "2018-11-22T21:08:35.882484Z",
1352 "start_time": "2018-11-22T21:08:26.074497Z"
1353 }
1354 },
1355 "outputs": [],
1356 "source": [
1357 "sparse.save_npz(text_features_dir / 'test_dtm_numeric', test_dtm_numeric)"
1358 ]
1359 },
1360 {
1361 "cell_type": "code",
1362 "execution_count": 43,
1363 "metadata": {
1364 "ExecuteTime": {
1365 "end_time": "2018-11-23T18:12:06.454426Z",
1366 "start_time": "2018-11-23T18:11:55.682001Z"
1367 }
1368 },
1369 "outputs": [],
1370 "source": [
1371 "train_dtm_numeric = sparse.load_npz(text_features_dir / 'train_dtm_numeric.npz')\n",
1372 "test_dtm_numeric = sparse.load_npz(text_features_dir / 'test_dtm_numeric.npz')"
1373 ]
1374 },
1375 {
1376 "cell_type": "markdown",
1377 "metadata": {},
1378 "source": [
1379 "### Logistic Regression"
1380 ]
1381 },
1382 {
1383 "cell_type": "markdown",
1384 "metadata": {},
1385 "source": [
1386 "We proceed to train a one-vs-all Logistic Regression that trains one model per class while treating the remaining classes as the negative class and predicts probabilities for each class using the different models.\n",
1387 "\n",
1388 "Using only the text features, we train and evaluate the model as follows:"
1389 ]
1390 },
1391 {
1392 "cell_type": "code",
1393 "execution_count": 44,
1394 "metadata": {
1395 "ExecuteTime": {
1396 "end_time": "2018-11-22T21:08:55.069249Z",
1397 "start_time": "2018-11-22T21:08:55.062720Z"
1398 }
1399 },
1400 "outputs": [],
1401 "source": [
1402 "logreg = LogisticRegression(C=1e9)"
1403 ]
1404 },
1405 {
1406 "cell_type": "markdown",
1407 "metadata": {},
1408 "source": [
1409 "#### Text features only"
1410 ]
1411 },
1412 {
1413 "cell_type": "code",
1414 "execution_count": null,
1415 "metadata": {
1416 "ExecuteTime": {
1417 "end_time": "2018-11-21T18:31:17.317673Z",
1418 "start_time": "2018-11-21T17:50:15.118828Z"
1419 },
1420 "slideshow": {
1421 "slide_type": "slide"
1422 }
1423 },
1424 "outputs": [],
1425 "source": [
1426 "logreg.fit(X=train_dtm, y=train.stars)\n",
1427 "y_pred_class = logreg.predict(test_dtm)"
1428 ]
1429 },
1430 {
1431 "cell_type": "code",
1432 "execution_count": null,
1433 "metadata": {
1434 "ExecuteTime": {
1435 "end_time": "2018-11-26T06:51:48.392513Z",
1436 "start_time": "2018-11-26T06:51:48.383821Z"
1437 }
1438 },
1439 "outputs": [],
1440 "source": [
1441 "joblib.dump(logreg, 'train_dtm.joblib') "
1442 ]
1443 },
1444 {
1445 "cell_type": "code",
1446 "execution_count": null,
1447 "metadata": {
1448 "ExecuteTime": {
1449 "end_time": "2018-11-26T06:54:07.052479Z",
1450 "start_time": "2018-11-26T06:54:07.047514Z"
1451 }
1452 },
1453 "outputs": [],
1454 "source": [
1455 "logreg = joblib.load('log_reg_multi/train_dtm.joblib')"
1456 ]
1457 },
1458 {
1459 "cell_type": "code",
1460 "execution_count": null,
1461 "metadata": {
1462 "ExecuteTime": {
1463 "end_time": "2018-11-26T06:54:42.935813Z",
1464 "start_time": "2018-11-26T06:54:42.717740Z"
1465 }
1466 },
1467 "outputs": [],
1468 "source": [
1469 "y_pred_class = logreg.predict(test_dtm)"
1470 ]
1471 },
1472 {
1473 "cell_type": "markdown",
1474 "metadata": {},
1475 "source": [
1476 "##### Evaluate Results"
1477 ]
1478 },
1479 {
1480 "cell_type": "markdown",
1481 "metadata": {},
1482 "source": [
1483 "The model achieves significantly higher accuracy at 73.6%:"
1484 ]
1485 },
1486 {
1487 "cell_type": "code",
1488 "execution_count": null,
1489 "metadata": {
1490 "ExecuteTime": {
1491 "end_time": "2018-11-26T06:54:49.824441Z",
1492 "start_time": "2018-11-26T06:54:49.801577Z"
1493 }
1494 },
1495 "outputs": [],
1496 "source": [
1497 "print(accuracy_score(test.stars, y_pred_class))"
1498 ]
1499 },
1500 {
1501 "cell_type": "markdown",
1502 "metadata": {},
1503 "source": [
1504 "#### Combined Features"
1505 ]
1506 },
1507 {
1508 "cell_type": "markdown",
1509 "metadata": {},
1510 "source": [
1511 "##### One-vs-all Logistic Regression"
1512 ]
1513 },
1514 {
1515 "cell_type": "markdown",
1516 "metadata": {},
1517 "source": []
1518 },
1519 {
1520 "cell_type": "code",
1521 "execution_count": null,
1522 "metadata": {
1523 "ExecuteTime": {
1524 "end_time": "2018-11-22T21:59:50.309518Z",
1525 "start_time": "2018-11-22T21:10:55.158840Z"
1526 },
1527 "slideshow": {
1528 "slide_type": "slide"
1529 }
1530 },
1531 "outputs": [],
1532 "source": [
1533 "# Use logistic regression with all features.\n",
1534 "logreg.fit(train_dtm_numeric.astype(float), train.stars)\n",
1535 "y_pred_class = logreg.predict(test_dtm_numeric.astype(float))"
1536 ]
1537 },
1538 {
1539 "cell_type": "code",
1540 "execution_count": null,
1541 "metadata": {
1542 "ExecuteTime": {
1543 "end_time": "2018-11-22T21:59:50.315309Z",
1544 "start_time": "2018-11-22T21:59:50.311075Z"
1545 }
1546 },
1547 "outputs": [],
1548 "source": [
1549 "joblib.dump(logreg, 'train_dtm_numeric.joblib') "
1550 ]
1551 },
1552 {
1553 "cell_type": "code",
1554 "execution_count": null,
1555 "metadata": {
1556 "ExecuteTime": {
1557 "end_time": "2018-11-22T21:59:50.348097Z",
1558 "start_time": "2018-11-22T21:59:50.316228Z"
1559 }
1560 },
1561 "outputs": [],
1562 "source": [
1563 "accuracy_score(test.stars, y_pred_class)"
1564 ]
1565 },
1566 {
1567 "cell_type": "markdown",
1568 "metadata": {},
1569 "source": [
1570 "##### Multinomial Logistic Regression"
1571 ]
1572 },
1573 {
1574 "cell_type": "markdown",
1575 "metadata": {},
1576 "source": [
1577 "Logistic regression also provides a multinomial training option that is faster and more accurate than the one-vs-all implementation. We use the lbfgs solver (see sklearn [documentation](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) for details)."
1578 ]
1579 },
1580 {
1581 "cell_type": "code",
1582 "execution_count": null,
1583 "metadata": {
1584 "ExecuteTime": {
1585 "end_time": "2018-11-22T23:47:30.063002Z",
1586 "start_time": "2018-11-22T23:40:39.092482Z"
1587 }
1588 },
1589 "outputs": [],
1590 "source": [
1591 "multi_logreg = LogisticRegression(C=1e9, multi_class='multinomial', solver='lbfgs')\n",
1592 "multi_logreg.fit(train_dtm_numeric.astype(float), train.stars)"
1593 ]
1594 },
1595 {
1596 "cell_type": "code",
1597 "execution_count": null,
1598 "metadata": {
1599 "ExecuteTime": {
1600 "end_time": "2018-11-22T23:47:30.068053Z",
1601 "start_time": "2018-11-22T23:47:30.064593Z"
1602 }
1603 },
1604 "outputs": [],
1605 "source": [
1606 "joblib.dump(multi_logreg, 'train_dtm_numeric_multi.joblib')"
1607 ]
1608 },
1609 {
1610 "cell_type": "code",
1611 "execution_count": null,
1612 "metadata": {
1613 "ExecuteTime": {
1614 "end_time": "2018-11-22T23:47:31.312857Z",
1615 "start_time": "2018-11-22T23:47:30.068969Z"
1616 }
1617 },
1618 "outputs": [],
1619 "source": [
1620 "y_pred_class = multi_logreg.predict(test_dtm_numeric.astype(float))"
1621 ]
1622 },
1623 {
1624 "cell_type": "markdown",
1625 "metadata": {},
1626 "source": [
1627 "In this case, tuning of the regularization parameter C did not lead to very significant improvements."
1628 ]
1629 },
1630 {
1631 "cell_type": "code",
1632 "execution_count": null,
1633 "metadata": {
1634 "ExecuteTime": {
1635 "end_time": "2018-11-22T23:47:31.337396Z",
1636 "start_time": "2018-11-22T23:47:31.314381Z"
1637 }
1638 },
1639 "outputs": [],
1640 "source": [
1641 "accuracy_score(test.stars, y_pred_class)"
1642 ]
1643 },
1644 {
1645 "cell_type": "code",
1646 "execution_count": null,
1647 "metadata": {
1648 "ExecuteTime": {
1649 "end_time": "2018-11-23T10:15:32.149142Z",
1650 "start_time": "2018-11-22T23:57:31.684556Z"
1651 },
1652 "scrolled": true
1653 },
1654 "outputs": [],
1655 "source": [
1656 "for solver in ['newton-cg', 'sag', 'saga']:\n",
1657 " start = time()\n",
1658 " multi_logreg = LogisticRegression(C=1e9, multi_class='multinomial', solver=solver)\n",
1659 " multi_logreg.fit(train_dtm_numeric.astype(float), train.stars)\n",
1660 " joblib.dump(multi_logreg, f'train_dtm_numeric_multi_{solver}.joblib')\n",
1661 " y_pred_class = multi_logreg.predict(test_dtm_numeric.astype(float))\n",
1662 " print(f'{solver}: {time()-start:.2f}s | {accuracy_score(test.stars, y_pred_class):.2%}', flush=True)"
1663 ]
1664 },
1665 {
1666 "cell_type": "code",
1667 "execution_count": null,
1668 "metadata": {
1669 "ExecuteTime": {
1670 "end_time": "2018-11-23T16:49:15.010878Z",
1671 "start_time": "2018-11-23T15:46:38.300352Z"
1672 }
1673 },
1674 "outputs": [],
1675 "source": [
1676 "for C in [1e7, 1e6, 1e5, 1e4, 1e3, 1e2, 1e1, 1, 0.1,0]:\n",
1677 " start = time()\n",
1678 " multi_logreg = LogisticRegression(C=C, multi_class='multinomial', solver='lbfgs')\n",
1679 " multi_logreg.fit(train_dtm_numeric.astype(float), train.stars)\n",
1680 " joblib.dump(multi_logreg, f'train_dtm_numeric_multi_{int(C*10):d}.joblib')\n",
1681 " y_pred_class = multi_logreg.predict(test_dtm_numeric.astype(float))\n",
1682 " print(f'{C}: {time()-start:.2f}s | {accuracy_score(test.stars, y_pred_class):.2%}', flush=True)"
1683 ]
1684 },
1685 {
1686 "cell_type": "markdown",
1687 "metadata": {},
1688 "source": [
1689 "### Gradient Boosting"
1690 ]
1691 },
1692 {
1693 "cell_type": "markdown",
1694 "metadata": {},
1695 "source": [
1696 "For illustration, we also train a lightgbm Gradient Boosting tree ensemble with default settings and multiclass objective."
1697 ]
1698 },
1699 {
1700 "cell_type": "code",
1701 "execution_count": null,
1702 "metadata": {
1703 "ExecuteTime": {
1704 "end_time": "2018-11-23T18:29:09.279963Z",
1705 "start_time": "2018-11-23T18:29:01.645297Z"
1706 }
1707 },
1708 "outputs": [],
1709 "source": [
1710 "lgb_train = lgb.Dataset(data=train_dtm_numeric.tocsr().astype(np.float32), \n",
1711 " label=train.stars.sub(1), \n",
1712 " categorical_feature=list(range(train_dtm_numeric.shape[1])))"
1713 ]
1714 },
1715 {
1716 "cell_type": "code",
1717 "execution_count": null,
1718 "metadata": {
1719 "ExecuteTime": {
1720 "end_time": "2018-11-23T18:29:09.578466Z",
1721 "start_time": "2018-11-23T18:29:09.281388Z"
1722 }
1723 },
1724 "outputs": [],
1725 "source": [
1726 "lgb_test = lgb.Dataset(data=test_dtm_numeric.tocsr().astype(np.float32), \n",
1727 " label=test.stars.sub(1), \n",
1728 " reference=lgb_train)"
1729 ]
1730 },
1731 {
1732 "cell_type": "code",
1733 "execution_count": null,
1734 "metadata": {
1735 "ExecuteTime": {
1736 "end_time": "2018-11-23T18:29:09.582339Z",
1737 "start_time": "2018-11-23T18:29:09.580028Z"
1738 }
1739 },
1740 "outputs": [],
1741 "source": [
1742 "param = {'objective':'multiclass', 'num_class': 5}"
1743 ]
1744 },
1745 {
1746 "cell_type": "code",
1747 "execution_count": null,
1748 "metadata": {
1749 "ExecuteTime": {
1750 "end_time": "2018-11-23T21:52:17.824036Z",
1751 "start_time": "2018-11-23T18:29:09.583871Z"
1752 },
1753 "scrolled": false
1754 },
1755 "outputs": [],
1756 "source": [
1757 "booster = lgb.train(params=param, \n",
1758 " train_set=lgb_train, \n",
1759 " num_boost_round=500, \n",
1760 " early_stopping_rounds=20,\n",
1761 " valid_sets=[lgb_train, lgb_test])"
1762 ]
1763 },
1764 {
1765 "cell_type": "code",
1766 "execution_count": null,
1767 "metadata": {
1768 "ExecuteTime": {
1769 "end_time": "2018-11-23T21:52:17.985172Z",
1770 "start_time": "2018-11-23T21:52:17.933310Z"
1771 }
1772 },
1773 "outputs": [],
1774 "source": [
1775 "booster.save_model(str(text_features_dir / 'lgb_model.txt'))"
1776 ]
1777 },
1778 {
1779 "cell_type": "code",
1780 "execution_count": null,
1781 "metadata": {
1782 "ExecuteTime": {
1783 "end_time": "2018-11-23T22:00:54.906351Z",
1784 "start_time": "2018-11-23T21:59:21.482150Z"
1785 }
1786 },
1787 "outputs": [],
1788 "source": [
1789 "y_pred_class = booster.predict(test_dtm_numeric.astype(float))"
1790 ]
1791 },
1792 {
1793 "cell_type": "markdown",
1794 "metadata": {},
1795 "source": [
1796 "The basic settings did not improve over the multinomial logistic regression, but further parameter tuning remains an unused option."
1797 ]
1798 },
1799 {
1800 "cell_type": "code",
1801 "execution_count": null,
1802 "metadata": {
1803 "ExecuteTime": {
1804 "end_time": "2018-11-23T22:03:09.846172Z",
1805 "start_time": "2018-11-23T22:03:09.816105Z"
1806 }
1807 },
1808 "outputs": [],
1809 "source": [
1810 "accuracy_score(test.stars, y_pred_class.argmax(1) + 1)"
1811 ]
1812 },
1813 {
1814 "cell_type": "code",
1815 "execution_count": null,
1816 "metadata": {},
1817 "outputs": [],
1818 "source": [
1819 "y_pred_class_class_classd_classed_classred"
1820 ]
1821 },
1822 {
1823 "cell_type": "code",
1824 "execution_count": null,
1825 "metadata": {
1826 "ExecuteTime": {
1827 "end_time": "2018-11-23T22:18:00.865341Z",
1828 "start_time": "2018-11-23T22:18:00.857910Z"
1829 }
1830 },
1831 "outputs": [],
1832 "source": [
1833 "fi = booster.feature_importance(importance_type='gain')\n",
1834 "pd.Series(fi).div(fi.sum()).sort_values(ascending=False).head()"
1835 ]
1836 },
1837 {
1838 "cell_type": "markdown",
1839 "metadata": {},
1840 "source": [
1841 "### Naive Bayes"
1842 ]
1843 },
1844 {
1845 "cell_type": "code",
1846 "execution_count": null,
1847 "metadata": {
1848 "ExecuteTime": {
1849 "end_time": "2018-11-23T22:04:57.061729Z",
1850 "start_time": "2018-11-23T22:04:46.049681Z"
1851 }
1852 },
1853 "outputs": [],
1854 "source": [
1855 "nb = MultinomialNB()\n",
1856 "nb.fit(train_dtm_numeric,train.stars)\n",
1857 "predicted_stars = nb.predict(test_dtm_numeric)\n",
1858 "accuracy_score(test.stars, predicted_stars)"
1859 ]
1860 },
1861 {
1862 "cell_type": "markdown",
1863 "metadata": {
1864 "slideshow": {
1865 "slide_type": "slide"
1866 }
1867 },
1868 "source": [
1869 "## Textblob for Sentiment Analysis"
1870 ]
1871 },
1872 {
1873 "cell_type": "code",
1874 "execution_count": null,
1875 "metadata": {
1876 "ExecuteTime": {
1877 "end_time": "2018-11-21T17:32:40.765746Z",
1878 "start_time": "2018-11-21T17:32:40.571104Z"
1879 },
1880 "slideshow": {
1881 "slide_type": "slide"
1882 }
1883 },
1884 "outputs": [],
1885 "source": [
1886 "sample_review = combined.text.sample(1).iloc[0]\n",
1887 "print(sample_review)"
1888 ]
1889 },
1890 {
1891 "cell_type": "code",
1892 "execution_count": null,
1893 "metadata": {
1894 "ExecuteTime": {
1895 "end_time": "2018-11-21T17:32:40.940304Z",
1896 "start_time": "2018-11-21T17:32:40.934791Z"
1897 },
1898 "slideshow": {
1899 "slide_type": "slide"
1900 }
1901 },
1902 "outputs": [],
1903 "source": [
1904 "# Polarity ranges from -1 (most negative) to 1 (most positive).\n",
1905 "TextBlob(sample_review).sentiment.polarity"
1906 ]
1907 },
1908 {
1909 "cell_type": "code",
1910 "execution_count": null,
1911 "metadata": {
1912 "ExecuteTime": {
1913 "end_time": "2018-11-23T23:55:42.292091Z",
1914 "start_time": "2018-11-23T23:55:42.290276Z"
1915 },
1916 "slideshow": {
1917 "slide_type": "slide"
1918 }
1919 },
1920 "outputs": [],
1921 "source": [
1922 "# Define a function that accepts text and returns the polarity.\n",
1923 "def detect_sentiment(text):\n",
1924 " return TextBlob(text).sentiment.polarity"
1925 ]
1926 },
1927 {
1928 "cell_type": "code",
1929 "execution_count": null,
1930 "metadata": {
1931 "ExecuteTime": {
1932 "end_time": "2018-11-24T01:05:21.178283Z",
1933 "start_time": "2018-11-23T23:56:01.144736Z"
1934 }
1935 },
1936 "outputs": [],
1937 "source": [
1938 "combined['sentiment'] = combined.text.apply(detect_sentiment)"
1939 ]
1940 },
1941 {
1942 "cell_type": "code",
1943 "execution_count": null,
1944 "metadata": {
1945 "ExecuteTime": {
1946 "end_time": "2018-11-24T05:10:18.867787Z",
1947 "start_time": "2018-11-24T05:06:08.373534Z"
1948 }
1949 },
1950 "outputs": [],
1951 "source": [
1952 "combined.to_parquet(parquet_dir / 'combined_tb.parquet', compression='gzip')"
1953 ]
1954 },
1955 {
1956 "cell_type": "code",
1957 "execution_count": null,
1958 "metadata": {
1959 "ExecuteTime": {
1960 "end_time": "2018-11-21T17:46:59.746135Z",
1961 "start_time": "2018-11-21T17:46:59.298644Z"
1962 }
1963 },
1964 "outputs": [],
1965 "source": [
1966 "sample_reviews = combined[['stars', 'text']].sample(100000)"
1967 ]
1968 },
1969 {
1970 "cell_type": "code",
1971 "execution_count": null,
1972 "metadata": {
1973 "ExecuteTime": {
1974 "end_time": "2018-11-21T17:48:10.849876Z",
1975 "start_time": "2018-11-21T17:47:01.859678Z"
1976 },
1977 "slideshow": {
1978 "slide_type": "slide"
1979 }
1980 },
1981 "outputs": [],
1982 "source": [
1983 "# Create a new DataFrame column for sentiment (Warning: SLOW!).\n",
1984 "sample_reviews['sentiment'] = sample_reviews.text.apply(detect_sentiment)"
1985 ]
1986 },
1987 {
1988 "cell_type": "code",
1989 "execution_count": null,
1990 "metadata": {
1991 "ExecuteTime": {
1992 "end_time": "2018-11-24T05:03:22.544002Z",
1993 "start_time": "2018-11-24T05:03:21.568279Z"
1994 },
1995 "slideshow": {
1996 "slide_type": "slide"
1997 }
1998 },
1999 "outputs": [],
2000 "source": [
2001 "# Box plot of sentiment grouped by stars\n",
2002 "sns.boxenplot(x='stars', y='sentiment', data=combined);"
2003 ]
2004 },
2005 {
2006 "cell_type": "code",
2007 "execution_count": null,
2008 "metadata": {
2009 "ExecuteTime": {
2010 "end_time": "2018-11-24T05:04:13.803888Z",
2011 "start_time": "2018-11-24T05:04:13.796660Z"
2012 },
2013 "slideshow": {
2014 "slide_type": "slide"
2015 }
2016 },
2017 "outputs": [],
2018 "source": [
2019 "# Widen the column display.\n",
2020 "pd.set_option('max_colwidth', 500)"
2021 ]
2022 },
2023 {
2024 "cell_type": "code",
2025 "execution_count": null,
2026 "metadata": {
2027 "ExecuteTime": {
2028 "end_time": "2018-11-24T05:04:14.717109Z",
2029 "start_time": "2018-11-24T05:04:14.705564Z"
2030 },
2031 "slideshow": {
2032 "slide_type": "slide"
2033 }
2034 },
2035 "outputs": [],
2036 "source": [
2037 "# Reviews with most negative sentiment\n",
2038 "combined[combined.sentiment == -1].text.head()"
2039 ]
2040 },
2041 {
2042 "cell_type": "code",
2043 "execution_count": null,
2044 "metadata": {
2045 "ExecuteTime": {
2046 "end_time": "2018-11-24T05:04:31.689717Z",
2047 "start_time": "2018-11-24T05:04:31.663071Z"
2048 },
2049 "slideshow": {
2050 "slide_type": "slide"
2051 }
2052 },
2053 "outputs": [],
2054 "source": [
2055 "# Negative sentiment in a 5-star review\n",
2056 "combined[(combined.stars == 5) & (combined.sentiment < -0.3)].head(1)"
2057 ]
2058 },
2059 {
2060 "cell_type": "code",
2061 "execution_count": null,
2062 "metadata": {
2063 "ExecuteTime": {
2064 "end_time": "2018-11-24T05:05:10.547475Z",
2065 "start_time": "2018-11-24T05:05:10.532685Z"
2066 },
2067 "slideshow": {
2068 "slide_type": "slide"
2069 }
2070 },
2071 "outputs": [],
2072 "source": [
2073 "# Positive sentiment in a 1-star review\n",
2074 "combined.loc[(combined.stars == 1) & (combined.sentiment > 0.5), 'text'].head(1)"
2075 ]
2076 },
2077 {
2078 "cell_type": "code",
2079 "execution_count": null,
2080 "metadata": {
2081 "slideshow": {
2082 "slide_type": "slide"
2083 }
2084 },
2085 "outputs": [],
2086 "source": [
2087 "# Reset the column display width.\n",
2088 "pd.reset_option('max_colwidth')"
2089 ]
2090 }
2091 ],
2092 "metadata": {
2093 "celltoolbar": "Slideshow",
2094 "kernelspec": {
2095 "display_name": "Python 3",
2096 "language": "python",
2097 "name": "python3"
2098 },
2099 "language_info": {
2100 "codemirror_mode": {
2101 "name": "ipython",
2102 "version": 3
2103 },
2104 "file_extension": ".py",
2105 "mimetype": "text/x-python",
2106 "name": "python",
2107 "nbconvert_exporter": "python",
2108 "pygments_lexer": "ipython3",
2109 "version": "3.6.8"
2110 },
2111 "toc": {
2112 "base_numbering": 1,
2113 "nav_menu": {},
2114 "number_sections": true,
2115 "sideBar": true,
2116 "skip_h1_title": true,
2117 "title_cell": "Table of Contents",
2118 "title_sidebar": "Contents",
2119 "toc_cell": false,
2120 "toc_position": {
2121 "height": "calc(100% - 180px)",
2122 "left": "10px",
2123 "top": "150px",
2124 "width": "316px"
2125 },
2126 "toc_section_display": true,
2127 "toc_window_display": true
2128 }
2129 },
2130 "nbformat": 4,
2131 "nbformat_minor": 2
2132 }