ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
notebook.ipynb
(11833B)
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "# Exercises: Introduction to Pairs Trading\n",
8 "By Chris Fenaroli, Delaney Mackenzie, and Maxwell Margenot\n",
9 "\n",
10 "## Lecture Link :\n",
11 "https://www.quantopian.com/lectures/introduction-to-pairs-trading\n",
12 "\n",
13 "### IMPORTANT NOTE: \n",
14 "These exercises correspond to the Introduction to Pairs Trading lecture, which is part of the Quantopian Lecture Series. It is meant to cement a few concepts presented in the lecture, specifically cointegration and construction of cointegrated pair spreads.\n",
15 "\n",
16 "We expect you to rely heavily on the code presented in the corresponding lecture. Please copy and paste regularly from that lecture when starting to work on the problems, as trying to do them from scratch will likely be too difficult. There is an answer key available for this problem set. Please email us at feedback@quantopian.com for results.\n",
17 "\n",
18 "Part of the Quantopian Lecture Series:\n",
19 "\n",
20 "* [www.quantopian.com/lectures](https://www.quantopian.com/lectures)\n",
21 "* [github.com/quantopian/research_public](https://github.com/quantopian/research_public)\n",
22 "\n",
23 "----"
24 ]
25 },
26 {
27 "cell_type": "markdown",
28 "metadata": {},
29 "source": [
30 "## Key Concepts"
31 ]
32 },
33 {
34 "cell_type": "code",
35 "execution_count": null,
36 "metadata": {
37 "collapsed": true
38 },
39 "outputs": [],
40 "source": [
41 "# Useful Functions\n",
42 "def find_cointegrated_pairs(data):\n",
43 " n = data.shape[1]\n",
44 " score_matrix = np.zeros((n, n))\n",
45 " pvalue_matrix = np.ones((n, n))\n",
46 " keys = data.keys()\n",
47 " pairs = []\n",
48 " for i in range(n):\n",
49 " for j in range(i+1, n):\n",
50 " S1 = data[keys[i]]\n",
51 " S2 = data[keys[j]]\n",
52 " result = coint(S1, S2)\n",
53 " score = result[0]\n",
54 " pvalue = result[1]\n",
55 " score_matrix[i, j] = score\n",
56 " pvalue_matrix[i, j] = pvalue\n",
57 " if pvalue < 0.05:\n",
58 " pairs.append((keys[i], keys[j]))\n",
59 " return score_matrix, pvalue_matrix, pairs"
60 ]
61 },
62 {
63 "cell_type": "code",
64 "execution_count": 1,
65 "metadata": {
66 "collapsed": true
67 },
68 "outputs": [],
69 "source": [
70 "# Useful Libraries\n",
71 "import numpy as np\n",
72 "import pandas as pd\n",
73 "\n",
74 "import statsmodels\n",
75 "import statsmodels.api as sm\n",
76 "from statsmodels.tsa.stattools import coint, adfuller\n",
77 "# just set the seed for the random number generator\n",
78 "np.random.seed(107)\n",
79 "\n",
80 "import matplotlib.pyplot as plt"
81 ]
82 },
83 {
84 "cell_type": "markdown",
85 "metadata": {},
86 "source": [
87 "-----"
88 ]
89 },
90 {
91 "cell_type": "markdown",
92 "metadata": {},
93 "source": [
94 "#Exercise 1: Testing Artificial Examples\n",
95 "\n",
96 "We'll use some artificially generated series first as they are much cleaner and easier to work with. In general when learning or developing a new technique, use simulated data to provide a clean environment. Simulated data also allows you to control the level of noise and difficulty level for your model.\n",
97 "\n",
98 "##a. Cointegration Test I\n",
99 "\n",
100 "Determine whether the following two artificial series $A$ and $B$ are cointegrated using the `coint()` function and a reasonable confidence level."
101 ]
102 },
103 {
104 "cell_type": "code",
105 "execution_count": null,
106 "metadata": {
107 "collapsed": false
108 },
109 "outputs": [],
110 "source": [
111 "A_returns = np.random.normal(0, 1, 100)\n",
112 "A = pd.Series(np.cumsum(A_returns), name='X') + 50\n",
113 "\n",
114 "some_noise = np.random.exponential(1, 100)\n",
115 "\n",
116 "B = A - 7 + some_noise\n",
117 "\n",
118 "#Your code goes here"
119 ]
120 },
121 {
122 "cell_type": "markdown",
123 "metadata": {},
124 "source": [
125 "##b. Cointegration Test II\n",
126 "\n",
127 "Determine whether the following two artificial series $C$ and $D$ are cointegrated using the `coint()` function and a reasonable confidence level."
128 ]
129 },
130 {
131 "cell_type": "code",
132 "execution_count": null,
133 "metadata": {
134 "collapsed": false
135 },
136 "outputs": [],
137 "source": [
138 "C_returns = np.random.normal(1, 1, 100) \n",
139 "C = pd.Series(np.cumsum(C_returns), name='X') + 100\n",
140 "\n",
141 "D_returns = np.random.normal(2, 1, 100)\n",
142 "D = pd.Series(np.cumsum(D_returns), name='X') + 100\n",
143 "\n",
144 "#Your code goes here"
145 ]
146 },
147 {
148 "cell_type": "markdown",
149 "metadata": {},
150 "source": [
151 "----"
152 ]
153 },
154 {
155 "cell_type": "markdown",
156 "metadata": {},
157 "source": [
158 "#Exercise 2: Testing Real Examples\n",
159 "\n",
160 "##a. Real Cointegration Test I\n",
161 "\n",
162 "Determine whether the following two assets `UAL` and `AAL` were cointegrated during 2015 using the `coint()` function and a reasonable confidence level."
163 ]
164 },
165 {
166 "cell_type": "code",
167 "execution_count": null,
168 "metadata": {
169 "collapsed": false
170 },
171 "outputs": [],
172 "source": [
173 "ual = get_pricing('UAL', fields=['price'], \n",
174 " start_date='2015-01-01', end_date='2016-01-01')['price']\n",
175 "aal = get_pricing('AAL', fields=['price'], \n",
176 " start_date='2015-01-01', end_date='2016-01-01')['price']\n",
177 "\n",
178 "#Your code goes here"
179 ]
180 },
181 {
182 "cell_type": "markdown",
183 "metadata": {},
184 "source": [
185 "##b. Real Cointegration Test II\n",
186 "\n",
187 "Determine whether the following two assets `FCAU` and `HMC` were cointegrated during 2015 using the `coint()` function and a reasonable confidence level."
188 ]
189 },
190 {
191 "cell_type": "code",
192 "execution_count": null,
193 "metadata": {
194 "collapsed": true
195 },
196 "outputs": [],
197 "source": [
198 "fcau = get_pricing('FCAU', fields=['price'], \n",
199 " start_date='2015-01-01', end_date='2016-01-01')['price']\n",
200 "hmc = get_pricing('HMC', fields=['price'], \n",
201 " start_date='2015-01-01', end_date='2016-01-01')['price']\n",
202 "\n",
203 "#Your code goes here"
204 ]
205 },
206 {
207 "cell_type": "markdown",
208 "metadata": {},
209 "source": [
210 "----"
211 ]
212 },
213 {
214 "cell_type": "markdown",
215 "metadata": {},
216 "source": [
217 "# Exercise 3: Searching for Cointegrated Pairs\n",
218 "\n",
219 "Use the `find_cointegrated_pairs` function, defined in the \"Helper Functions\" section above, to find any cointegrated pairs among a set of metal and mining securities.\n",
220 "\n",
221 "Note that not all of these securities in this exercise are within the [QTradableStocksUS](https://www.quantopian.com/posts/working-on-our-best-universe-yet-qtradablestocksus). As you continue your development, focus on securities within the QTU to be eligible for an [allocation](https://quantopian.com/allocation)."
222 ]
223 },
224 {
225 "cell_type": "code",
226 "execution_count": null,
227 "metadata": {
228 "collapsed": true
229 },
230 "outputs": [],
231 "source": [
232 "symbol_list = ['MTRN', 'CMP', 'TRQ', 'SCCO', 'HCLP','SPY']\n",
233 "prices_df = get_pricing(symbol_list, fields=['price']\n",
234 " , start_date='2015-01-01', end_date='2016-01-01')['price']\n",
235 "prices_df.columns = map(lambda x: x.symbol, prices_df.columns)\n",
236 "\n",
237 "#Your code goes here"
238 ]
239 },
240 {
241 "cell_type": "markdown",
242 "metadata": {},
243 "source": [
244 "----"
245 ]
246 },
247 {
248 "cell_type": "markdown",
249 "metadata": {},
250 "source": [
251 "#Exercise 4: Out of Sample Validation\n",
252 "\n",
253 "##a. Calculating the Spread\n",
254 "\n",
255 "Using pricing data from 2015, construct a linear regression to find a coefficient for the linear combination of `MTRN` and `SCCO` that makes their spread stationary."
256 ]
257 },
258 {
259 "cell_type": "code",
260 "execution_count": null,
261 "metadata": {
262 "collapsed": true
263 },
264 "outputs": [],
265 "source": [
266 "S1 = prices_df['MTRN']\n",
267 "S2 = prices_df['SCCO']\n",
268 "\n",
269 "#Your code goes here"
270 ]
271 },
272 {
273 "cell_type": "markdown",
274 "metadata": {},
275 "source": [
276 "##b. Testing the Coefficient\n",
277 "\n",
278 "Use your coefficient from part a to plot the weighted spread using prices from the first half of 2016, and check whether the result is still stationary."
279 ]
280 },
281 {
282 "cell_type": "code",
283 "execution_count": null,
284 "metadata": {
285 "collapsed": true
286 },
287 "outputs": [],
288 "source": [
289 "S1_out = get_pricing('MTRN', fields=['price'], \n",
290 " start_date='2016-01-01', end_date='2016-07-01')['price']\n",
291 "S2_out = get_pricing('SCCO', fields=['price'], \n",
292 " start_date='2016-01-01', end_date='2016-07-01')['price']\n",
293 "\n",
294 "#Your code goes here"
295 ]
296 },
297 {
298 "cell_type": "markdown",
299 "metadata": {},
300 "source": [
301 "----"
302 ]
303 },
304 {
305 "cell_type": "markdown",
306 "metadata": {},
307 "source": [
308 "#Extra Credit Exercise: Hurst Exponent\n",
309 "\n",
310 "This exercise is more difficult and we will not provide initial structure.\n",
311 "\n",
312 "The Hurst exponent is a statistic between 0 and 1 that provides information about how much a time series is trending or mean reverting. We want our spread time series to be mean reverting, so we can use the Hurst exponent to monitor whether our pair is going out of cointegration. Effectively as a means of process control to know when our pair is no longer good to trade.\n",
313 "\n",
314 "Please find either an existing Python library that computes, or compute yourself, the Hurst exponent. Then plot it over time for the spread on the above pair of stocks.\n",
315 "\n",
316 "These links may be helpful:\n",
317 "\n",
318 "* https://en.wikipedia.org/wiki/Hurst_exponent\n",
319 "* https://www.quantopian.com/posts/pair-trade-with-cointegration-and-mean-reversion-tests"
320 ]
321 },
322 {
323 "cell_type": "code",
324 "execution_count": null,
325 "metadata": {
326 "collapsed": true
327 },
328 "outputs": [],
329 "source": [
330 "# Your code goes here"
331 ]
332 },
333 {
334 "cell_type": "markdown",
335 "metadata": {},
336 "source": [
337 "*This presentation is for informational purposes only and does not constitute an offer to sell, a solic\n",
338 "itation to buy, or a recommendation for any security; nor does it constitute an offer to provide investment advisory or other services by Quantopian, Inc. (\"Quantopian\"). Nothing contained herein constitutes investment advice or offers any opinion with respect to the suitability of any security, and any views expressed herein should not be taken as advice to buy, sell, or hold any security or as an endorsement of any security or company. In preparing the information contained herein, Quantopian, Inc. has not taken into account the investment needs, objectives, and financial circumstances of any particular investor. Any views expressed and data illustrated herein were prepared based upon information, believed to be reliable, available to Quantopian, Inc. at the time of publication. Quantopian makes no guarantees as to their accuracy or completeness. All information is subject to change and may quickly become unreliable for various reasons, including changes in market conditions or economic circumstances.*"
339 ]
340 }
341 ],
342 "metadata": {
343 "kernelspec": {
344 "display_name": "Python 2",
345 "language": "python",
346 "name": "python2"
347 },
348 "language_info": {
349 "codemirror_mode": {
350 "name": "ipython",
351 "version": 2
352 },
353 "file_extension": ".py",
354 "mimetype": "text/x-python",
355 "name": "python",
356 "nbconvert_exporter": "python",
357 "pygments_lexer": "ipython2",
358 "version": "2.7.12"
359 }
360 },
361 "nbformat": 4,
362 "nbformat_minor": 0
363 }