ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
lab_107.ipynb
(29778B)
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "# The Efficient Frontier - Part I\n",
8 "\n",
9 "This week, we are going to learn how to compute the efficient frontier when we have a set of expected returns, volatilities (or variances) and correlations (or covariances). It's a fair question as to how we can get these numbers for the future, but for now, we'll assume that historic returns are a reasonable estimate. In future sections, we'll learn how to improve on it.\n",
10 "\n",
11 "Let's start by importing a new dataset. This is the Ken French dataset of the returns of 30 different industry portfolios. \n",
12 "\n",
13 "This datafile has a number of minor problems that we'll sort through as we go:\n"
14 ]
15 },
16 {
17 "cell_type": "code",
18 "execution_count": 4,
19 "metadata": {},
20 "outputs": [],
21 "source": [
22 "import pandas as pd\n",
23 "ind = pd.read_csv(\"../data/ind30_m_vw_rets.csv\", header=0, index_col=0)/100\n",
24 "ind.index = pd.to_datetime(ind.index, format=\"%Y%m\").to_period('M')"
25 ]
26 },
27 {
28 "cell_type": "code",
29 "execution_count": 5,
30 "metadata": {},
31 "outputs": [
32 {
33 "data": {
34 "text/html": [
35 "<div>\n",
36 "<style scoped>\n",
37 " .dataframe tbody tr th:only-of-type {\n",
38 " vertical-align: middle;\n",
39 " }\n",
40 "\n",
41 " .dataframe tbody tr th {\n",
42 " vertical-align: top;\n",
43 " }\n",
44 "\n",
45 " .dataframe thead th {\n",
46 " text-align: right;\n",
47 " }\n",
48 "</style>\n",
49 "<table border=\"1\" class=\"dataframe\">\n",
50 " <thead>\n",
51 " <tr style=\"text-align: right;\">\n",
52 " <th></th>\n",
53 " <th>Food</th>\n",
54 " <th>Beer</th>\n",
55 " <th>Smoke</th>\n",
56 " <th>Games</th>\n",
57 " <th>Books</th>\n",
58 " <th>Hshld</th>\n",
59 " <th>Clths</th>\n",
60 " <th>Hlth</th>\n",
61 " <th>Chems</th>\n",
62 " <th>Txtls</th>\n",
63 " <th>...</th>\n",
64 " <th>Telcm</th>\n",
65 " <th>Servs</th>\n",
66 " <th>BusEq</th>\n",
67 " <th>Paper</th>\n",
68 " <th>Trans</th>\n",
69 " <th>Whlsl</th>\n",
70 " <th>Rtail</th>\n",
71 " <th>Meals</th>\n",
72 " <th>Fin</th>\n",
73 " <th>Other</th>\n",
74 " </tr>\n",
75 " </thead>\n",
76 " <tbody>\n",
77 " <tr>\n",
78 " <th>1926-07</th>\n",
79 " <td>0.0056</td>\n",
80 " <td>-0.0519</td>\n",
81 " <td>0.0129</td>\n",
82 " <td>0.0293</td>\n",
83 " <td>0.1097</td>\n",
84 " <td>-0.0048</td>\n",
85 " <td>0.0808</td>\n",
86 " <td>0.0177</td>\n",
87 " <td>0.0814</td>\n",
88 " <td>0.0039</td>\n",
89 " <td>...</td>\n",
90 " <td>0.0083</td>\n",
91 " <td>0.0922</td>\n",
92 " <td>0.0206</td>\n",
93 " <td>0.0770</td>\n",
94 " <td>0.0193</td>\n",
95 " <td>-0.2379</td>\n",
96 " <td>0.0007</td>\n",
97 " <td>0.0187</td>\n",
98 " <td>0.0037</td>\n",
99 " <td>0.0520</td>\n",
100 " </tr>\n",
101 " <tr>\n",
102 " <th>1926-08</th>\n",
103 " <td>0.0259</td>\n",
104 " <td>0.2703</td>\n",
105 " <td>0.0650</td>\n",
106 " <td>0.0055</td>\n",
107 " <td>0.1001</td>\n",
108 " <td>-0.0358</td>\n",
109 " <td>-0.0251</td>\n",
110 " <td>0.0425</td>\n",
111 " <td>0.0550</td>\n",
112 " <td>0.0814</td>\n",
113 " <td>...</td>\n",
114 " <td>0.0217</td>\n",
115 " <td>0.0202</td>\n",
116 " <td>0.0439</td>\n",
117 " <td>-0.0238</td>\n",
118 " <td>0.0488</td>\n",
119 " <td>0.0539</td>\n",
120 " <td>-0.0075</td>\n",
121 " <td>-0.0013</td>\n",
122 " <td>0.0446</td>\n",
123 " <td>0.0676</td>\n",
124 " </tr>\n",
125 " <tr>\n",
126 " <th>1926-09</th>\n",
127 " <td>0.0116</td>\n",
128 " <td>0.0402</td>\n",
129 " <td>0.0126</td>\n",
130 " <td>0.0658</td>\n",
131 " <td>-0.0099</td>\n",
132 " <td>0.0073</td>\n",
133 " <td>-0.0051</td>\n",
134 " <td>0.0069</td>\n",
135 " <td>0.0533</td>\n",
136 " <td>0.0231</td>\n",
137 " <td>...</td>\n",
138 " <td>0.0241</td>\n",
139 " <td>0.0225</td>\n",
140 " <td>0.0019</td>\n",
141 " <td>-0.0554</td>\n",
142 " <td>0.0005</td>\n",
143 " <td>-0.0787</td>\n",
144 " <td>0.0025</td>\n",
145 " <td>-0.0056</td>\n",
146 " <td>-0.0123</td>\n",
147 " <td>-0.0386</td>\n",
148 " </tr>\n",
149 " <tr>\n",
150 " <th>1926-10</th>\n",
151 " <td>-0.0306</td>\n",
152 " <td>-0.0331</td>\n",
153 " <td>0.0106</td>\n",
154 " <td>-0.0476</td>\n",
155 " <td>0.0947</td>\n",
156 " <td>-0.0468</td>\n",
157 " <td>0.0012</td>\n",
158 " <td>-0.0057</td>\n",
159 " <td>-0.0476</td>\n",
160 " <td>0.0100</td>\n",
161 " <td>...</td>\n",
162 " <td>-0.0011</td>\n",
163 " <td>-0.0200</td>\n",
164 " <td>-0.0109</td>\n",
165 " <td>-0.0508</td>\n",
166 " <td>-0.0264</td>\n",
167 " <td>-0.1538</td>\n",
168 " <td>-0.0220</td>\n",
169 " <td>-0.0411</td>\n",
170 " <td>-0.0516</td>\n",
171 " <td>-0.0849</td>\n",
172 " </tr>\n",
173 " <tr>\n",
174 " <th>1926-11</th>\n",
175 " <td>0.0635</td>\n",
176 " <td>0.0729</td>\n",
177 " <td>0.0455</td>\n",
178 " <td>0.0166</td>\n",
179 " <td>-0.0580</td>\n",
180 " <td>-0.0054</td>\n",
181 " <td>0.0187</td>\n",
182 " <td>0.0542</td>\n",
183 " <td>0.0520</td>\n",
184 " <td>0.0311</td>\n",
185 " <td>...</td>\n",
186 " <td>0.0163</td>\n",
187 " <td>0.0377</td>\n",
188 " <td>0.0364</td>\n",
189 " <td>0.0384</td>\n",
190 " <td>0.0160</td>\n",
191 " <td>0.0467</td>\n",
192 " <td>0.0652</td>\n",
193 " <td>0.0433</td>\n",
194 " <td>0.0224</td>\n",
195 " <td>0.0400</td>\n",
196 " </tr>\n",
197 " </tbody>\n",
198 "</table>\n",
199 "<p>5 rows × 30 columns</p>\n",
200 "</div>"
201 ],
202 "text/plain": [
203 " Food Beer Smoke Games Books Hshld Clths Hlth \\\n",
204 "1926-07 0.0056 -0.0519 0.0129 0.0293 0.1097 -0.0048 0.0808 0.0177 \n",
205 "1926-08 0.0259 0.2703 0.0650 0.0055 0.1001 -0.0358 -0.0251 0.0425 \n",
206 "1926-09 0.0116 0.0402 0.0126 0.0658 -0.0099 0.0073 -0.0051 0.0069 \n",
207 "1926-10 -0.0306 -0.0331 0.0106 -0.0476 0.0947 -0.0468 0.0012 -0.0057 \n",
208 "1926-11 0.0635 0.0729 0.0455 0.0166 -0.0580 -0.0054 0.0187 0.0542 \n",
209 "\n",
210 " Chems Txtls ... Telcm Servs BusEq Paper Trans Whlsl \\\n",
211 "1926-07 0.0814 0.0039 ... 0.0083 0.0922 0.0206 0.0770 0.0193 -0.2379 \n",
212 "1926-08 0.0550 0.0814 ... 0.0217 0.0202 0.0439 -0.0238 0.0488 0.0539 \n",
213 "1926-09 0.0533 0.0231 ... 0.0241 0.0225 0.0019 -0.0554 0.0005 -0.0787 \n",
214 "1926-10 -0.0476 0.0100 ... -0.0011 -0.0200 -0.0109 -0.0508 -0.0264 -0.1538 \n",
215 "1926-11 0.0520 0.0311 ... 0.0163 0.0377 0.0364 0.0384 0.0160 0.0467 \n",
216 "\n",
217 " Rtail Meals Fin Other \n",
218 "1926-07 0.0007 0.0187 0.0037 0.0520 \n",
219 "1926-08 -0.0075 -0.0013 0.0446 0.0676 \n",
220 "1926-09 0.0025 -0.0056 -0.0123 -0.0386 \n",
221 "1926-10 -0.0220 -0.0411 -0.0516 -0.0849 \n",
222 "1926-11 0.0652 0.0433 0.0224 0.0400 \n",
223 "\n",
224 "[5 rows x 30 columns]"
225 ]
226 },
227 "execution_count": 5,
228 "metadata": {},
229 "output_type": "execute_result"
230 }
231 ],
232 "source": [
233 "ind.head()"
234 ]
235 },
236 {
237 "cell_type": "code",
238 "execution_count": 6,
239 "metadata": {},
240 "outputs": [
241 {
242 "data": {
243 "text/plain": [
244 "Index(['Food ', 'Beer ', 'Smoke', 'Games', 'Books', 'Hshld', 'Clths', 'Hlth ',\n",
245 " 'Chems', 'Txtls', 'Cnstr', 'Steel', 'FabPr', 'ElcEq', 'Autos', 'Carry',\n",
246 " 'Mines', 'Coal ', 'Oil ', 'Util ', 'Telcm', 'Servs', 'BusEq', 'Paper',\n",
247 " 'Trans', 'Whlsl', 'Rtail', 'Meals', 'Fin ', 'Other'],\n",
248 " dtype='object')"
249 ]
250 },
251 "execution_count": 6,
252 "metadata": {},
253 "output_type": "execute_result"
254 }
255 ],
256 "source": [
257 "ind.columns"
258 ]
259 },
260 {
261 "cell_type": "markdown",
262 "metadata": {},
263 "source": [
264 "Note that the column names have embedded spaces. We can strip out the leading and trailing spaces in the Series by using the `.str.strip` method."
265 ]
266 },
267 {
268 "cell_type": "code",
269 "execution_count": 7,
270 "metadata": {},
271 "outputs": [],
272 "source": [
273 "ind.columns = ind.columns.str.strip()"
274 ]
275 },
276 {
277 "cell_type": "code",
278 "execution_count": 8,
279 "metadata": {},
280 "outputs": [
281 {
282 "data": {
283 "text/plain": [
284 "(1110, 30)"
285 ]
286 },
287 "execution_count": 8,
288 "metadata": {},
289 "output_type": "execute_result"
290 }
291 ],
292 "source": [
293 "ind.shape"
294 ]
295 },
296 {
297 "cell_type": "markdown",
298 "metadata": {},
299 "source": [
300 "This looks good, so let's add the following code to our module for future use:\n",
301 "\n",
302 "```python\n",
303 "def get_ind_returns():\n",
304 " \"\"\"\n",
305 " Load and format the Ken French 30 Industry Portfolios Value Weighted Monthly Returns\n",
306 " \"\"\"\n",
307 " ind = pd.read_csv(\"data/ind30_m_vw_rets.csv\", header=0, index_col=0)/100\n",
308 " ind.index = pd.to_datetime(ind.index, format=\"%Y%m\").to_period('M')\n",
309 " ind.columns = ind.columns.str.strip()\n",
310 " return ind\n",
311 "```\n",
312 "\n",
313 "and then test it by loading the module as usual."
314 ]
315 },
316 {
317 "cell_type": "code",
318 "execution_count": 9,
319 "metadata": {},
320 "outputs": [
321 {
322 "ename": "FileNotFoundError",
323 "evalue": "[Errno 2] No such file or directory: 'data/ind30_m_vw_rets.csv'",
324 "output_type": "error",
325 "traceback": [
326 "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
327 "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
328 "\u001b[0;32m<ipython-input-9-4ba90382cfad>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0medhec_risk_kit_107\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merk\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mind\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0merk\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_ind_returns\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0mind\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
329 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/nb/edhec_risk_kit_107.py\u001b[0m in \u001b[0;36mget_ind_returns\u001b[0;34m()\u001b[0m\n\u001b[1;32m 29\u001b[0m \u001b[0mLoad\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mformat\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mKen\u001b[0m \u001b[0mFrench\u001b[0m \u001b[0;36m30\u001b[0m \u001b[0mIndustry\u001b[0m \u001b[0mPortfolios\u001b[0m \u001b[0mValue\u001b[0m \u001b[0mWeighted\u001b[0m \u001b[0mMonthly\u001b[0m \u001b[0mReturns\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 30\u001b[0m \"\"\"\n\u001b[0;32m---> 31\u001b[0;31m \u001b[0mind\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"data/ind30_m_vw_rets.csv\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheader\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex_col\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 32\u001b[0m \u001b[0mind\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_datetime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mind\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mformat\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"%Y%m\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_period\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'M'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 33\u001b[0m \u001b[0mind\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mind\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstrip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
330 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)\u001b[0m\n\u001b[1;32m 603\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwds_defaults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 604\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 605\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 607\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
331 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 455\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 456\u001b[0m \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 457\u001b[0;31m \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 458\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 459\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
332 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m 812\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 813\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 814\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 815\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 816\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
333 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, engine)\u001b[0m\n\u001b[1;32m 1043\u001b[0m )\n\u001b[1;32m 1044\u001b[0m \u001b[0;31m# error: Too many arguments for \"ParserBase\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1045\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mmapping\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1046\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1047\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_failover_to_python\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
334 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, src, **kwds)\u001b[0m\n\u001b[1;32m 1860\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1861\u001b[0m \u001b[0;31m# open handles\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1862\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_open_handles\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1863\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandles\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1864\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\"storage_options\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"encoding\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"memory_map\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"compression\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
335 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_open_handles\u001b[0;34m(self, src, kwds)\u001b[0m\n\u001b[1;32m 1355\u001b[0m \u001b[0mLet\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mreaders\u001b[0m \u001b[0mopen\u001b[0m \u001b[0mIOHanldes\u001b[0m \u001b[0mafter\u001b[0m \u001b[0mthey\u001b[0m \u001b[0mare\u001b[0m \u001b[0mdone\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtheir\u001b[0m \u001b[0mpotential\u001b[0m \u001b[0mraises\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1356\u001b[0m \"\"\"\n\u001b[0;32m-> 1357\u001b[0;31m self.handles = get_handle(\n\u001b[0m\u001b[1;32m 1358\u001b[0m \u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1359\u001b[0m \u001b[0;34m\"r\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
336 "\u001b[0;32m~/trading/coursera/portfolio-construction-01/venv/lib/python3.8/site-packages/pandas/io/common.py\u001b[0m in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m 640\u001b[0m \u001b[0merrors\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"replace\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 641\u001b[0m \u001b[0;31m# Encoding\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 642\u001b[0;31m handle = open(\n\u001b[0m\u001b[1;32m 643\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 644\u001b[0m \u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
337 "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'data/ind30_m_vw_rets.csv'"
338 ]
339 }
340 ],
341 "source": [
342 "%load_ext autoreload\n",
343 "%autoreload 2\n",
344 "%matplotlib inline\n",
345 "\n",
346 "import edhec_risk_kit_107 as erk\n",
347 "ind = erk.get_ind_returns()\n",
348 "ind.shape"
349 ]
350 },
351 {
352 "cell_type": "code",
353 "execution_count": null,
354 "metadata": {},
355 "outputs": [],
356 "source": [
357 "erk.drawdown(ind[\"Food\"])[\"Drawdown\"].plot.line()"
358 ]
359 },
360 {
361 "cell_type": "code",
362 "execution_count": null,
363 "metadata": {},
364 "outputs": [],
365 "source": [
366 "erk.var_gaussian(ind[[\"Food\", \"Beer\", \"Smoke\"]], modified=True)"
367 ]
368 },
369 {
370 "cell_type": "code",
371 "execution_count": null,
372 "metadata": {},
373 "outputs": [],
374 "source": [
375 "erk.var_gaussian(ind).sort_values().plot.bar()"
376 ]
377 },
378 {
379 "cell_type": "markdown",
380 "metadata": {},
381 "source": [
382 "Let's use this as an opportunity to write functions for annualized returns, volatility and sharpe ratios. Add the following to the `edhec_risk_kit.py` file:\n",
383 "\n",
384 "```python\n",
385 "def annualize_rets(r, periods_per_year):\n",
386 " \"\"\"\n",
387 " Annualizes a set of returns\n",
388 " We should infer the periods per year\n",
389 " but that is currently left as an exercise\n",
390 " to the reader :-)\n",
391 " \"\"\"\n",
392 " compounded_growth = (1+r).prod()\n",
393 " n_periods = r.shape[0]\n",
394 " return compounded_growth**(periods_per_year/n_periods)-1\n",
395 "\n",
396 "def annualize_vol(r, periods_per_year):\n",
397 " \"\"\"\n",
398 " Annualizes the vol of a set of returns\n",
399 " We should infer the periods per year\n",
400 " but that is currently left as an exercise\n",
401 " to the reader :-)\n",
402 " \"\"\"\n",
403 " return r.std()*(periods_per_year**0.5)\n",
404 "\n",
405 "def sharpe_ratio(r, riskfree_rate, periods_per_year):\n",
406 " \"\"\"\n",
407 " Computes the annualized sharpe ratio of a set of returns\n",
408 " \"\"\"\n",
409 " # convert the annual riskfree rate to per period\n",
410 " rf_per_period = (1+riskfree_rate)**(1/periods_per_year)-1\n",
411 " excess_ret = r - rf_per_period\n",
412 " ann_ex_ret = annualize_rets(excess_ret, periods_per_year)\n",
413 " ann_vol = annualize_vol(r, periods_per_year)\n",
414 " return ann_ex_ret/ann_vol\n",
415 "\n",
416 "```"
417 ]
418 },
419 {
420 "cell_type": "code",
421 "execution_count": null,
422 "metadata": {},
423 "outputs": [],
424 "source": [
425 "erk.sharpe_ratio(ind, 0.03, 12).sort_values()"
426 ]
427 },
428 {
429 "cell_type": "code",
430 "execution_count": null,
431 "metadata": {},
432 "outputs": [],
433 "source": [
434 "erk.sharpe_ratio(ind, 0.03, 12).sort_values().plot.bar(title=\"Industry Sharpe Ratios 1926-2018\")"
435 ]
436 },
437 {
438 "cell_type": "code",
439 "execution_count": null,
440 "metadata": {},
441 "outputs": [],
442 "source": [
443 "erk.sharpe_ratio(ind[\"2000\":], 0.03, 12).sort_values().plot.bar(title='Industry Sharpe Ratios since 2000')"
444 ]
445 },
446 {
447 "cell_type": "markdown",
448 "metadata": {},
449 "source": [
450 "## Expected Returns and the Covariance Matrix\n",
451 "\n",
452 "Generating the efficient frontier requires a set of expected returns and a covariance matrix. For now, let's assume that we can estiamte these simply by looking back in time and naively assuming they will hold in the future. Clearly, they will not, but we will have plenty of time to dig into that in future lectures. For the moment, assume that our naive method of estimating these parameters will suffice.\n",
453 "\n",
454 "We can generate an estimate of expected returns using the `annualize_rets()` function, that returns a vector of expected returns. For instance, let's generate the set of expected returns based on historic returns from the 5 year period from 1996 through 2000:"
455 ]
456 },
457 {
458 "cell_type": "code",
459 "execution_count": null,
460 "metadata": {},
461 "outputs": [],
462 "source": [
463 "er = erk.annualize_rets(ind[\"1995\":\"2000\"], 12)"
464 ]
465 },
466 {
467 "cell_type": "code",
468 "execution_count": null,
469 "metadata": {},
470 "outputs": [],
471 "source": [
472 "er.sort_values().plot.bar()"
473 ]
474 },
475 {
476 "cell_type": "markdown",
477 "metadata": {},
478 "source": [
479 "Finally, let's generate the covariance matrix. Fortunately, this is easy enough to do using the `.cov` method:"
480 ]
481 },
482 {
483 "cell_type": "code",
484 "execution_count": null,
485 "metadata": {},
486 "outputs": [],
487 "source": [
488 "cov = ind[\"1995\":\"2000\"].cov()\n",
489 "cov.shape"
490 ]
491 },
492 {
493 "cell_type": "markdown",
494 "metadata": {},
495 "source": [
496 "In the next lab session, we'll take the expected returns vector and the covariance matrix we've constructed and start to plot the efficient frontier!"
497 ]
498 },
499 {
500 "cell_type": "code",
501 "execution_count": 10,
502 "metadata": {},
503 "outputs": [
504 {
505 "ename": "NameError",
506 "evalue": "name 'cov' is not defined",
507 "output_type": "error",
508 "traceback": [
509 "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
510 "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
511 "\u001b[0;32m<ipython-input-10-b49a49fe1426>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mcov\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
512 "\u001b[0;31mNameError\u001b[0m: name 'cov' is not defined"
513 ]
514 }
515 ],
516 "source": [
517 "cov"
518 ]
519 },
520 {
521 "cell_type": "code",
522 "execution_count": null,
523 "metadata": {},
524 "outputs": [],
525 "source": []
526 },
527 {
528 "cell_type": "code",
529 "execution_count": null,
530 "metadata": {},
531 "outputs": [],
532 "source": []
533 },
534 {
535 "cell_type": "code",
536 "execution_count": null,
537 "metadata": {},
538 "outputs": [],
539 "source": []
540 },
541 {
542 "cell_type": "code",
543 "execution_count": null,
544 "metadata": {},
545 "outputs": [],
546 "source": []
547 },
548 {
549 "cell_type": "code",
550 "execution_count": null,
551 "metadata": {},
552 "outputs": [],
553 "source": []
554 },
555 {
556 "cell_type": "code",
557 "execution_count": null,
558 "metadata": {},
559 "outputs": [],
560 "source": []
561 },
562 {
563 "cell_type": "code",
564 "execution_count": null,
565 "metadata": {},
566 "outputs": [],
567 "source": []
568 },
569 {
570 "cell_type": "code",
571 "execution_count": null,
572 "metadata": {},
573 "outputs": [],
574 "source": []
575 },
576 {
577 "cell_type": "code",
578 "execution_count": null,
579 "metadata": {},
580 "outputs": [],
581 "source": []
582 },
583 {
584 "cell_type": "code",
585 "execution_count": null,
586 "metadata": {},
587 "outputs": [],
588 "source": []
589 },
590 {
591 "cell_type": "code",
592 "execution_count": null,
593 "metadata": {},
594 "outputs": [],
595 "source": []
596 },
597 {
598 "cell_type": "code",
599 "execution_count": null,
600 "metadata": {},
601 "outputs": [],
602 "source": []
603 },
604 {
605 "cell_type": "code",
606 "execution_count": null,
607 "metadata": {},
608 "outputs": [],
609 "source": []
610 },
611 {
612 "cell_type": "code",
613 "execution_count": null,
614 "metadata": {},
615 "outputs": [],
616 "source": []
617 },
618 {
619 "cell_type": "code",
620 "execution_count": null,
621 "metadata": {},
622 "outputs": [],
623 "source": []
624 }
625 ],
626 "metadata": {
627 "kernelspec": {
628 "display_name": "Python 3",
629 "language": "python",
630 "name": "python3"
631 },
632 "language_info": {
633 "codemirror_mode": {
634 "name": "ipython",
635 "version": 3
636 },
637 "file_extension": ".py",
638 "mimetype": "text/x-python",
639 "name": "python",
640 "nbconvert_exporter": "python",
641 "pygments_lexer": "ipython3",
642 "version": "3.8.8"
643 }
644 },
645 "nbformat": 4,
646 "nbformat_minor": 2
647 }