ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
notebook.ipynb
(38265B)
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "##Factors\n",
8 "A factor is a function from an asset and a moment in time to a number.\n",
9 "```\n",
10 "F(asset, timestamp) -> float\n",
11 "```\n",
12 "In Pipeline, [Factors](https://www.quantopian.com/help#quantopian_pipeline_factors_Factor) are the most commonly-used term, representing the result of any computation producing a numerical result. Factors require a column of data and a window length as input.\n",
13 "\n",
14 "The simplest factors in Pipeline are [built-in Factors](https://www.quantopian.com/help#built-in-factors). Built-in Factors are pre-built to perform common computations. As a first example, let's make a factor to compute the average close price over the last 10 days. We can use the `SimpleMovingAverage` built-in factor which computes the average value of the input data (close price) over the specified window length (10 days). To do this, we need to import our built-in `SimpleMovingAverage` factor and the [USEquityPricing dataset](https://www.quantopian.com/help#importing-datasets)."
15 ]
16 },
17 {
18 "cell_type": "code",
19 "execution_count": 1,
20 "metadata": {
21 "collapsed": true
22 },
23 "outputs": [],
24 "source": [
25 "from quantopian.pipeline import Pipeline\n",
26 "from quantopian.research import run_pipeline\n",
27 "\n",
28 "# New from the last lesson, import the USEquityPricing dataset.\n",
29 "from quantopian.pipeline.data.builtin import USEquityPricing\n",
30 "\n",
31 "# New from the last lesson, import the built-in SimpleMovingAverage factor.\n",
32 "from quantopian.pipeline.factors import SimpleMovingAverage"
33 ]
34 },
35 {
36 "cell_type": "markdown",
37 "metadata": {
38 "collapsed": true
39 },
40 "source": [
41 "###Creating a Factor\n",
42 "Let's go back to our `make_pipeline` function from the previous lesson and instantiate a `SimpleMovingAverage` factor. To create a `SimpleMovingAverage` factor, we can call the `SimpleMovingAverage` constructor with two arguments: inputs, which must be a list of `BoundColumn` objects, and window_length, which must be an integer indicating how many days worth of data our moving average calculation should receive. (We'll discuss `BoundColumn` in more depth later; for now we just need to know that a `BoundColumn` is an object indicating what kind of data should be passed to our Factor.).\n",
43 "\n",
44 "The following line creates a `Factor` for computing the 10-day mean close price of securities."
45 ]
46 },
47 {
48 "cell_type": "code",
49 "execution_count": 2,
50 "metadata": {
51 "collapsed": true
52 },
53 "outputs": [],
54 "source": [
55 "mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)"
56 ]
57 },
58 {
59 "cell_type": "markdown",
60 "metadata": {},
61 "source": [
62 "It's important to note that creating the factor does not actually perform a computation. Creating a factor is like defining the function. To perform a computation, we need to add the factor to our pipeline and run it."
63 ]
64 },
65 {
66 "cell_type": "markdown",
67 "metadata": {},
68 "source": [
69 "###Adding a Factor to a Pipeline\n",
70 "Let's update our original empty pipeline to make it compute our new moving average factor. To start, let's move our factor instantatiation into `make_pipeline`. Next, we can tell our pipeline to compute our factor by passing it a `columns` argument, which should be a dictionary mapping column names to factors, filters, or classifiers. Our updated `make_pipeline` function should look something like this:"
71 ]
72 },
73 {
74 "cell_type": "code",
75 "execution_count": 3,
76 "metadata": {
77 "collapsed": true
78 },
79 "outputs": [],
80 "source": [
81 "def make_pipeline():\n",
82 " \n",
83 " mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)\n",
84 " \n",
85 " return Pipeline(\n",
86 " columns={\n",
87 " '10_day_mean_close': mean_close_10\n",
88 " }\n",
89 " )"
90 ]
91 },
92 {
93 "cell_type": "markdown",
94 "metadata": {},
95 "source": [
96 "To see what this looks like, let's make our pipeline, run it, and display the result."
97 ]
98 },
99 {
100 "cell_type": "code",
101 "execution_count": 4,
102 "metadata": {},
103 "outputs": [
104 {
105 "data": {
106 "text/html": [
107 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
108 "<table border=\"1\" class=\"dataframe\">\n",
109 " <thead>\n",
110 " <tr style=\"text-align: right;\">\n",
111 " <th></th>\n",
112 " <th></th>\n",
113 " <th>10_day_mean_close</th>\n",
114 " </tr>\n",
115 " </thead>\n",
116 " <tbody>\n",
117 " <tr>\n",
118 " <th rowspan=\"61\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
119 " <th>Equity(2 [AA])</th>\n",
120 " <td>13.559500</td>\n",
121 " </tr>\n",
122 " <tr>\n",
123 " <th>Equity(21 [AAME])</th>\n",
124 " <td>3.962500</td>\n",
125 " </tr>\n",
126 " <tr>\n",
127 " <th>Equity(24 [AAPL])</th>\n",
128 " <td>129.025700</td>\n",
129 " </tr>\n",
130 " <tr>\n",
131 " <th>Equity(25 [AA_PR])</th>\n",
132 " <td>88.362500</td>\n",
133 " </tr>\n",
134 " <tr>\n",
135 " <th>Equity(31 [ABAX])</th>\n",
136 " <td>61.920900</td>\n",
137 " </tr>\n",
138 " <tr>\n",
139 " <th>Equity(39 [DDC])</th>\n",
140 " <td>19.287072</td>\n",
141 " </tr>\n",
142 " <tr>\n",
143 " <th>Equity(41 [ARCB])</th>\n",
144 " <td>37.880000</td>\n",
145 " </tr>\n",
146 " <tr>\n",
147 " <th>Equity(52 [ABM])</th>\n",
148 " <td>32.083400</td>\n",
149 " </tr>\n",
150 " <tr>\n",
151 " <th>Equity(53 [ABMD])</th>\n",
152 " <td>66.795000</td>\n",
153 " </tr>\n",
154 " <tr>\n",
155 " <th>Equity(62 [ABT])</th>\n",
156 " <td>47.466000</td>\n",
157 " </tr>\n",
158 " <tr>\n",
159 " <th>Equity(64 [ABX])</th>\n",
160 " <td>12.919000</td>\n",
161 " </tr>\n",
162 " <tr>\n",
163 " <th>Equity(66 [AB])</th>\n",
164 " <td>31.547000</td>\n",
165 " </tr>\n",
166 " <tr>\n",
167 " <th>Equity(67 [ADSK])</th>\n",
168 " <td>60.212000</td>\n",
169 " </tr>\n",
170 " <tr>\n",
171 " <th>Equity(69 [ACAT])</th>\n",
172 " <td>36.331000</td>\n",
173 " </tr>\n",
174 " <tr>\n",
175 " <th>Equity(70 [VBF])</th>\n",
176 " <td>18.767000</td>\n",
177 " </tr>\n",
178 " <tr>\n",
179 " <th>Equity(76 [TAP])</th>\n",
180 " <td>74.632000</td>\n",
181 " </tr>\n",
182 " <tr>\n",
183 " <th>Equity(84 [ACET])</th>\n",
184 " <td>19.873000</td>\n",
185 " </tr>\n",
186 " <tr>\n",
187 " <th>Equity(86 [ACG])</th>\n",
188 " <td>7.810000</td>\n",
189 " </tr>\n",
190 " <tr>\n",
191 " <th>Equity(88 [ACI])</th>\n",
192 " <td>0.996100</td>\n",
193 " </tr>\n",
194 " <tr>\n",
195 " <th>Equity(100 [IEP])</th>\n",
196 " <td>91.821200</td>\n",
197 " </tr>\n",
198 " <tr>\n",
199 " <th>Equity(106 [ACU])</th>\n",
200 " <td>18.641000</td>\n",
201 " </tr>\n",
202 " <tr>\n",
203 " <th>Equity(110 [ACXM])</th>\n",
204 " <td>18.045500</td>\n",
205 " </tr>\n",
206 " <tr>\n",
207 " <th>Equity(112 [ACY])</th>\n",
208 " <td>11.571000</td>\n",
209 " </tr>\n",
210 " <tr>\n",
211 " <th>Equity(114 [ADBE])</th>\n",
212 " <td>76.072000</td>\n",
213 " </tr>\n",
214 " <tr>\n",
215 " <th>Equity(117 [AEY])</th>\n",
216 " <td>2.423400</td>\n",
217 " </tr>\n",
218 " <tr>\n",
219 " <th>Equity(122 [ADI])</th>\n",
220 " <td>63.205900</td>\n",
221 " </tr>\n",
222 " <tr>\n",
223 " <th>Equity(128 [ADM])</th>\n",
224 " <td>48.788500</td>\n",
225 " </tr>\n",
226 " <tr>\n",
227 " <th>Equity(134 [SXCL])</th>\n",
228 " <td>NaN</td>\n",
229 " </tr>\n",
230 " <tr>\n",
231 " <th>Equity(149 [ADX])</th>\n",
232 " <td>14.150500</td>\n",
233 " </tr>\n",
234 " <tr>\n",
235 " <th>Equity(153 [AE])</th>\n",
236 " <td>54.099000</td>\n",
237 " </tr>\n",
238 " <tr>\n",
239 " <th>...</th>\n",
240 " <td>...</td>\n",
241 " </tr>\n",
242 " <tr>\n",
243 " <th>Equity(48961 [NYMT_O])</th>\n",
244 " <td>NaN</td>\n",
245 " </tr>\n",
246 " <tr>\n",
247 " <th>Equity(48962 [CSAL])</th>\n",
248 " <td>29.992000</td>\n",
249 " </tr>\n",
250 " <tr>\n",
251 " <th>Equity(48963 [PAK])</th>\n",
252 " <td>15.531875</td>\n",
253 " </tr>\n",
254 " <tr>\n",
255 " <th>Equity(48969 [NSA])</th>\n",
256 " <td>13.045000</td>\n",
257 " </tr>\n",
258 " <tr>\n",
259 " <th>Equity(48971 [BSM])</th>\n",
260 " <td>17.995000</td>\n",
261 " </tr>\n",
262 " <tr>\n",
263 " <th>Equity(48972 [EVA])</th>\n",
264 " <td>21.413250</td>\n",
265 " </tr>\n",
266 " <tr>\n",
267 " <th>Equity(48981 [APIC])</th>\n",
268 " <td>14.814000</td>\n",
269 " </tr>\n",
270 " <tr>\n",
271 " <th>Equity(48989 [UK])</th>\n",
272 " <td>24.946667</td>\n",
273 " </tr>\n",
274 " <tr>\n",
275 " <th>Equity(48990 [ACWF])</th>\n",
276 " <td>25.250000</td>\n",
277 " </tr>\n",
278 " <tr>\n",
279 " <th>Equity(48991 [ISCF])</th>\n",
280 " <td>24.985000</td>\n",
281 " </tr>\n",
282 " <tr>\n",
283 " <th>Equity(48992 [INTF])</th>\n",
284 " <td>25.030000</td>\n",
285 " </tr>\n",
286 " <tr>\n",
287 " <th>Equity(48993 [JETS])</th>\n",
288 " <td>24.579333</td>\n",
289 " </tr>\n",
290 " <tr>\n",
291 " <th>Equity(48994 [ACTX])</th>\n",
292 " <td>15.097333</td>\n",
293 " </tr>\n",
294 " <tr>\n",
295 " <th>Equity(48995 [LRGF])</th>\n",
296 " <td>24.890000</td>\n",
297 " </tr>\n",
298 " <tr>\n",
299 " <th>Equity(48996 [SMLF])</th>\n",
300 " <td>29.456667</td>\n",
301 " </tr>\n",
302 " <tr>\n",
303 " <th>Equity(48997 [VKTX])</th>\n",
304 " <td>9.115000</td>\n",
305 " </tr>\n",
306 " <tr>\n",
307 " <th>Equity(48998 [OPGN])</th>\n",
308 " <td>NaN</td>\n",
309 " </tr>\n",
310 " <tr>\n",
311 " <th>Equity(48999 [AAPC])</th>\n",
312 " <td>10.144000</td>\n",
313 " </tr>\n",
314 " <tr>\n",
315 " <th>Equity(49000 [BPMC])</th>\n",
316 " <td>20.810000</td>\n",
317 " </tr>\n",
318 " <tr>\n",
319 " <th>Equity(49001 [CLCD])</th>\n",
320 " <td>NaN</td>\n",
321 " </tr>\n",
322 " <tr>\n",
323 " <th>Equity(49004 [TNP_PRD])</th>\n",
324 " <td>24.750000</td>\n",
325 " </tr>\n",
326 " <tr>\n",
327 " <th>Equity(49005 [ARWA_U])</th>\n",
328 " <td>NaN</td>\n",
329 " </tr>\n",
330 " <tr>\n",
331 " <th>Equity(49006 [BVXV])</th>\n",
332 " <td>NaN</td>\n",
333 " </tr>\n",
334 " <tr>\n",
335 " <th>Equity(49007 [BVXV_W])</th>\n",
336 " <td>NaN</td>\n",
337 " </tr>\n",
338 " <tr>\n",
339 " <th>Equity(49008 [OPGN_W])</th>\n",
340 " <td>NaN</td>\n",
341 " </tr>\n",
342 " <tr>\n",
343 " <th>Equity(49009 [PRKU])</th>\n",
344 " <td>NaN</td>\n",
345 " </tr>\n",
346 " <tr>\n",
347 " <th>Equity(49010 [TBRA])</th>\n",
348 " <td>NaN</td>\n",
349 " </tr>\n",
350 " <tr>\n",
351 " <th>Equity(49131 [OESX])</th>\n",
352 " <td>NaN</td>\n",
353 " </tr>\n",
354 " <tr>\n",
355 " <th>Equity(49259 [ITUS])</th>\n",
356 " <td>NaN</td>\n",
357 " </tr>\n",
358 " <tr>\n",
359 " <th>Equity(49523 [TLGT])</th>\n",
360 " <td>NaN</td>\n",
361 " </tr>\n",
362 " </tbody>\n",
363 "</table>\n",
364 "<p>8236 rows × 1 columns</p>\n",
365 "</div>"
366 ],
367 "text/plain": [
368 " 10_day_mean_close\n",
369 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 13.559500\n",
370 " Equity(21 [AAME]) 3.962500\n",
371 " Equity(24 [AAPL]) 129.025700\n",
372 " Equity(25 [AA_PR]) 88.362500\n",
373 " Equity(31 [ABAX]) 61.920900\n",
374 " Equity(39 [DDC]) 19.287072\n",
375 " Equity(41 [ARCB]) 37.880000\n",
376 " Equity(52 [ABM]) 32.083400\n",
377 " Equity(53 [ABMD]) 66.795000\n",
378 " Equity(62 [ABT]) 47.466000\n",
379 " Equity(64 [ABX]) 12.919000\n",
380 " Equity(66 [AB]) 31.547000\n",
381 " Equity(67 [ADSK]) 60.212000\n",
382 " Equity(69 [ACAT]) 36.331000\n",
383 " Equity(70 [VBF]) 18.767000\n",
384 " Equity(76 [TAP]) 74.632000\n",
385 " Equity(84 [ACET]) 19.873000\n",
386 " Equity(86 [ACG]) 7.810000\n",
387 " Equity(88 [ACI]) 0.996100\n",
388 " Equity(100 [IEP]) 91.821200\n",
389 " Equity(106 [ACU]) 18.641000\n",
390 " Equity(110 [ACXM]) 18.045500\n",
391 " Equity(112 [ACY]) 11.571000\n",
392 " Equity(114 [ADBE]) 76.072000\n",
393 " Equity(117 [AEY]) 2.423400\n",
394 " Equity(122 [ADI]) 63.205900\n",
395 " Equity(128 [ADM]) 48.788500\n",
396 " Equity(134 [SXCL]) NaN\n",
397 " Equity(149 [ADX]) 14.150500\n",
398 " Equity(153 [AE]) 54.099000\n",
399 "... ...\n",
400 " Equity(48961 [NYMT_O]) NaN\n",
401 " Equity(48962 [CSAL]) 29.992000\n",
402 " Equity(48963 [PAK]) 15.531875\n",
403 " Equity(48969 [NSA]) 13.045000\n",
404 " Equity(48971 [BSM]) 17.995000\n",
405 " Equity(48972 [EVA]) 21.413250\n",
406 " Equity(48981 [APIC]) 14.814000\n",
407 " Equity(48989 [UK]) 24.946667\n",
408 " Equity(48990 [ACWF]) 25.250000\n",
409 " Equity(48991 [ISCF]) 24.985000\n",
410 " Equity(48992 [INTF]) 25.030000\n",
411 " Equity(48993 [JETS]) 24.579333\n",
412 " Equity(48994 [ACTX]) 15.097333\n",
413 " Equity(48995 [LRGF]) 24.890000\n",
414 " Equity(48996 [SMLF]) 29.456667\n",
415 " Equity(48997 [VKTX]) 9.115000\n",
416 " Equity(48998 [OPGN]) NaN\n",
417 " Equity(48999 [AAPC]) 10.144000\n",
418 " Equity(49000 [BPMC]) 20.810000\n",
419 " Equity(49001 [CLCD]) NaN\n",
420 " Equity(49004 [TNP_PRD]) 24.750000\n",
421 " Equity(49005 [ARWA_U]) NaN\n",
422 " Equity(49006 [BVXV]) NaN\n",
423 " Equity(49007 [BVXV_W]) NaN\n",
424 " Equity(49008 [OPGN_W]) NaN\n",
425 " Equity(49009 [PRKU]) NaN\n",
426 " Equity(49010 [TBRA]) NaN\n",
427 " Equity(49131 [OESX]) NaN\n",
428 " Equity(49259 [ITUS]) NaN\n",
429 " Equity(49523 [TLGT]) NaN\n",
430 "\n",
431 "[8236 rows x 1 columns]"
432 ]
433 },
434 "execution_count": 4,
435 "metadata": {},
436 "output_type": "execute_result"
437 }
438 ],
439 "source": [
440 "result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')\n",
441 "result"
442 ]
443 },
444 {
445 "cell_type": "markdown",
446 "metadata": {},
447 "source": [
448 "Now we have a column in our pipeline output with the 10-day average close price for all 8000+ securities (display truncated). Note that each row corresponds to the result of our computation for a given security on a given date stored. The `DataFrame` has a [MultiIndex](http://pandas.pydata.org/pandas-docs/version/0.16.2/advanced.html) where the first level is a datetime representing the date of the computation and the second level is an [Equity](http://localhost:3000/help#api-sidinfo) object corresponding to the security. For example, the first row (`2015-05-05 00:00:00+00:00`, `Equity(2 [AA])`) will contain the result of our `mean_close_10` factor for AA on May 5th, 2015.\n",
449 "\n",
450 "If we run our pipeline over more than one day, the output looks like this."
451 ]
452 },
453 {
454 "cell_type": "code",
455 "execution_count": 5,
456 "metadata": {},
457 "outputs": [
458 {
459 "data": {
460 "text/html": [
461 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
462 "<table border=\"1\" class=\"dataframe\">\n",
463 " <thead>\n",
464 " <tr style=\"text-align: right;\">\n",
465 " <th></th>\n",
466 " <th></th>\n",
467 " <th>10_day_mean_close</th>\n",
468 " </tr>\n",
469 " </thead>\n",
470 " <tbody>\n",
471 " <tr>\n",
472 " <th rowspan=\"30\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
473 " <th>Equity(2 [AA])</th>\n",
474 " <td>13.559500</td>\n",
475 " </tr>\n",
476 " <tr>\n",
477 " <th>Equity(21 [AAME])</th>\n",
478 " <td>3.962500</td>\n",
479 " </tr>\n",
480 " <tr>\n",
481 " <th>Equity(24 [AAPL])</th>\n",
482 " <td>129.025700</td>\n",
483 " </tr>\n",
484 " <tr>\n",
485 " <th>Equity(25 [AA_PR])</th>\n",
486 " <td>88.362500</td>\n",
487 " </tr>\n",
488 " <tr>\n",
489 " <th>Equity(31 [ABAX])</th>\n",
490 " <td>61.920900</td>\n",
491 " </tr>\n",
492 " <tr>\n",
493 " <th>Equity(39 [DDC])</th>\n",
494 " <td>19.287072</td>\n",
495 " </tr>\n",
496 " <tr>\n",
497 " <th>Equity(41 [ARCB])</th>\n",
498 " <td>37.880000</td>\n",
499 " </tr>\n",
500 " <tr>\n",
501 " <th>Equity(52 [ABM])</th>\n",
502 " <td>32.083400</td>\n",
503 " </tr>\n",
504 " <tr>\n",
505 " <th>Equity(53 [ABMD])</th>\n",
506 " <td>66.795000</td>\n",
507 " </tr>\n",
508 " <tr>\n",
509 " <th>Equity(62 [ABT])</th>\n",
510 " <td>47.466000</td>\n",
511 " </tr>\n",
512 " <tr>\n",
513 " <th>Equity(64 [ABX])</th>\n",
514 " <td>12.919000</td>\n",
515 " </tr>\n",
516 " <tr>\n",
517 " <th>Equity(66 [AB])</th>\n",
518 " <td>31.547000</td>\n",
519 " </tr>\n",
520 " <tr>\n",
521 " <th>Equity(67 [ADSK])</th>\n",
522 " <td>60.212000</td>\n",
523 " </tr>\n",
524 " <tr>\n",
525 " <th>Equity(69 [ACAT])</th>\n",
526 " <td>36.331000</td>\n",
527 " </tr>\n",
528 " <tr>\n",
529 " <th>Equity(70 [VBF])</th>\n",
530 " <td>18.767000</td>\n",
531 " </tr>\n",
532 " <tr>\n",
533 " <th>Equity(76 [TAP])</th>\n",
534 " <td>74.632000</td>\n",
535 " </tr>\n",
536 " <tr>\n",
537 " <th>Equity(84 [ACET])</th>\n",
538 " <td>19.873000</td>\n",
539 " </tr>\n",
540 " <tr>\n",
541 " <th>Equity(86 [ACG])</th>\n",
542 " <td>7.810000</td>\n",
543 " </tr>\n",
544 " <tr>\n",
545 " <th>Equity(88 [ACI])</th>\n",
546 " <td>0.996100</td>\n",
547 " </tr>\n",
548 " <tr>\n",
549 " <th>Equity(100 [IEP])</th>\n",
550 " <td>91.821200</td>\n",
551 " </tr>\n",
552 " <tr>\n",
553 " <th>Equity(106 [ACU])</th>\n",
554 " <td>18.641000</td>\n",
555 " </tr>\n",
556 " <tr>\n",
557 " <th>Equity(110 [ACXM])</th>\n",
558 " <td>18.045500</td>\n",
559 " </tr>\n",
560 " <tr>\n",
561 " <th>Equity(112 [ACY])</th>\n",
562 " <td>11.571000</td>\n",
563 " </tr>\n",
564 " <tr>\n",
565 " <th>Equity(114 [ADBE])</th>\n",
566 " <td>76.072000</td>\n",
567 " </tr>\n",
568 " <tr>\n",
569 " <th>Equity(117 [AEY])</th>\n",
570 " <td>2.423400</td>\n",
571 " </tr>\n",
572 " <tr>\n",
573 " <th>Equity(122 [ADI])</th>\n",
574 " <td>63.205900</td>\n",
575 " </tr>\n",
576 " <tr>\n",
577 " <th>Equity(128 [ADM])</th>\n",
578 " <td>48.788500</td>\n",
579 " </tr>\n",
580 " <tr>\n",
581 " <th>Equity(134 [SXCL])</th>\n",
582 " <td>NaN</td>\n",
583 " </tr>\n",
584 " <tr>\n",
585 " <th>Equity(149 [ADX])</th>\n",
586 " <td>14.150500</td>\n",
587 " </tr>\n",
588 " <tr>\n",
589 " <th>Equity(153 [AE])</th>\n",
590 " <td>54.099000</td>\n",
591 " </tr>\n",
592 " <tr>\n",
593 " <th>...</th>\n",
594 " <th>...</th>\n",
595 " <td>...</td>\n",
596 " </tr>\n",
597 " <tr>\n",
598 " <th rowspan=\"30\" valign=\"top\">2015-05-07 00:00:00+00:00</th>\n",
599 " <th>Equity(48981 [APIC])</th>\n",
600 " <td>14.646000</td>\n",
601 " </tr>\n",
602 " <tr>\n",
603 " <th>Equity(48989 [UK])</th>\n",
604 " <td>24.878000</td>\n",
605 " </tr>\n",
606 " <tr>\n",
607 " <th>Equity(48990 [ACWF])</th>\n",
608 " <td>25.036667</td>\n",
609 " </tr>\n",
610 " <tr>\n",
611 " <th>Equity(48991 [ISCF])</th>\n",
612 " <td>24.875000</td>\n",
613 " </tr>\n",
614 " <tr>\n",
615 " <th>Equity(48992 [INTF])</th>\n",
616 " <td>24.813000</td>\n",
617 " </tr>\n",
618 " <tr>\n",
619 " <th>Equity(48993 [JETS])</th>\n",
620 " <td>24.343600</td>\n",
621 " </tr>\n",
622 " <tr>\n",
623 " <th>Equity(48994 [ACTX])</th>\n",
624 " <td>15.020400</td>\n",
625 " </tr>\n",
626 " <tr>\n",
627 " <th>Equity(48995 [LRGF])</th>\n",
628 " <td>24.788000</td>\n",
629 " </tr>\n",
630 " <tr>\n",
631 " <th>Equity(48996 [SMLF])</th>\n",
632 " <td>29.370000</td>\n",
633 " </tr>\n",
634 " <tr>\n",
635 " <th>Equity(48997 [VKTX])</th>\n",
636 " <td>9.232500</td>\n",
637 " </tr>\n",
638 " <tr>\n",
639 " <th>Equity(48998 [OPGN])</th>\n",
640 " <td>4.950000</td>\n",
641 " </tr>\n",
642 " <tr>\n",
643 " <th>Equity(48999 [AAPC])</th>\n",
644 " <td>10.167000</td>\n",
645 " </tr>\n",
646 " <tr>\n",
647 " <th>Equity(49000 [BPMC])</th>\n",
648 " <td>20.906667</td>\n",
649 " </tr>\n",
650 " <tr>\n",
651 " <th>Equity(49001 [CLCD])</th>\n",
652 " <td>8.010000</td>\n",
653 " </tr>\n",
654 " <tr>\n",
655 " <th>Equity(49004 [TNP_PRD])</th>\n",
656 " <td>24.633333</td>\n",
657 " </tr>\n",
658 " <tr>\n",
659 " <th>Equity(49005 [ARWA_U])</th>\n",
660 " <td>10.010000</td>\n",
661 " </tr>\n",
662 " <tr>\n",
663 " <th>Equity(49006 [BVXV])</th>\n",
664 " <td>NaN</td>\n",
665 " </tr>\n",
666 " <tr>\n",
667 " <th>Equity(49007 [BVXV_W])</th>\n",
668 " <td>NaN</td>\n",
669 " </tr>\n",
670 " <tr>\n",
671 " <th>Equity(49008 [OPGN_W])</th>\n",
672 " <td>0.817500</td>\n",
673 " </tr>\n",
674 " <tr>\n",
675 " <th>Equity(49009 [PRKU])</th>\n",
676 " <td>NaN</td>\n",
677 " </tr>\n",
678 " <tr>\n",
679 " <th>Equity(49010 [TBRA])</th>\n",
680 " <td>NaN</td>\n",
681 " </tr>\n",
682 " <tr>\n",
683 " <th>Equity(49015 [ADAP])</th>\n",
684 " <td>NaN</td>\n",
685 " </tr>\n",
686 " <tr>\n",
687 " <th>Equity(49016 [COLL])</th>\n",
688 " <td>NaN</td>\n",
689 " </tr>\n",
690 " <tr>\n",
691 " <th>Equity(49017 [GLSS])</th>\n",
692 " <td>NaN</td>\n",
693 " </tr>\n",
694 " <tr>\n",
695 " <th>Equity(49018 [HTGM])</th>\n",
696 " <td>NaN</td>\n",
697 " </tr>\n",
698 " <tr>\n",
699 " <th>Equity(49019 [LRET])</th>\n",
700 " <td>NaN</td>\n",
701 " </tr>\n",
702 " <tr>\n",
703 " <th>Equity(49020 [MVIR])</th>\n",
704 " <td>NaN</td>\n",
705 " </tr>\n",
706 " <tr>\n",
707 " <th>Equity(49131 [OESX])</th>\n",
708 " <td>NaN</td>\n",
709 " </tr>\n",
710 " <tr>\n",
711 " <th>Equity(49259 [ITUS])</th>\n",
712 " <td>NaN</td>\n",
713 " </tr>\n",
714 " <tr>\n",
715 " <th>Equity(49523 [TLGT])</th>\n",
716 " <td>NaN</td>\n",
717 " </tr>\n",
718 " </tbody>\n",
719 "</table>\n",
720 "<p>24705 rows × 1 columns</p>\n",
721 "</div>"
722 ],
723 "text/plain": [
724 " 10_day_mean_close\n",
725 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 13.559500\n",
726 " Equity(21 [AAME]) 3.962500\n",
727 " Equity(24 [AAPL]) 129.025700\n",
728 " Equity(25 [AA_PR]) 88.362500\n",
729 " Equity(31 [ABAX]) 61.920900\n",
730 " Equity(39 [DDC]) 19.287072\n",
731 " Equity(41 [ARCB]) 37.880000\n",
732 " Equity(52 [ABM]) 32.083400\n",
733 " Equity(53 [ABMD]) 66.795000\n",
734 " Equity(62 [ABT]) 47.466000\n",
735 " Equity(64 [ABX]) 12.919000\n",
736 " Equity(66 [AB]) 31.547000\n",
737 " Equity(67 [ADSK]) 60.212000\n",
738 " Equity(69 [ACAT]) 36.331000\n",
739 " Equity(70 [VBF]) 18.767000\n",
740 " Equity(76 [TAP]) 74.632000\n",
741 " Equity(84 [ACET]) 19.873000\n",
742 " Equity(86 [ACG]) 7.810000\n",
743 " Equity(88 [ACI]) 0.996100\n",
744 " Equity(100 [IEP]) 91.821200\n",
745 " Equity(106 [ACU]) 18.641000\n",
746 " Equity(110 [ACXM]) 18.045500\n",
747 " Equity(112 [ACY]) 11.571000\n",
748 " Equity(114 [ADBE]) 76.072000\n",
749 " Equity(117 [AEY]) 2.423400\n",
750 " Equity(122 [ADI]) 63.205900\n",
751 " Equity(128 [ADM]) 48.788500\n",
752 " Equity(134 [SXCL]) NaN\n",
753 " Equity(149 [ADX]) 14.150500\n",
754 " Equity(153 [AE]) 54.099000\n",
755 "... ...\n",
756 "2015-05-07 00:00:00+00:00 Equity(48981 [APIC]) 14.646000\n",
757 " Equity(48989 [UK]) 24.878000\n",
758 " Equity(48990 [ACWF]) 25.036667\n",
759 " Equity(48991 [ISCF]) 24.875000\n",
760 " Equity(48992 [INTF]) 24.813000\n",
761 " Equity(48993 [JETS]) 24.343600\n",
762 " Equity(48994 [ACTX]) 15.020400\n",
763 " Equity(48995 [LRGF]) 24.788000\n",
764 " Equity(48996 [SMLF]) 29.370000\n",
765 " Equity(48997 [VKTX]) 9.232500\n",
766 " Equity(48998 [OPGN]) 4.950000\n",
767 " Equity(48999 [AAPC]) 10.167000\n",
768 " Equity(49000 [BPMC]) 20.906667\n",
769 " Equity(49001 [CLCD]) 8.010000\n",
770 " Equity(49004 [TNP_PRD]) 24.633333\n",
771 " Equity(49005 [ARWA_U]) 10.010000\n",
772 " Equity(49006 [BVXV]) NaN\n",
773 " Equity(49007 [BVXV_W]) NaN\n",
774 " Equity(49008 [OPGN_W]) 0.817500\n",
775 " Equity(49009 [PRKU]) NaN\n",
776 " Equity(49010 [TBRA]) NaN\n",
777 " Equity(49015 [ADAP]) NaN\n",
778 " Equity(49016 [COLL]) NaN\n",
779 " Equity(49017 [GLSS]) NaN\n",
780 " Equity(49018 [HTGM]) NaN\n",
781 " Equity(49019 [LRET]) NaN\n",
782 " Equity(49020 [MVIR]) NaN\n",
783 " Equity(49131 [OESX]) NaN\n",
784 " Equity(49259 [ITUS]) NaN\n",
785 " Equity(49523 [TLGT]) NaN\n",
786 "\n",
787 "[24705 rows x 1 columns]"
788 ]
789 },
790 "execution_count": 5,
791 "metadata": {},
792 "output_type": "execute_result"
793 }
794 ],
795 "source": [
796 "result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-07')\n",
797 "result"
798 ]
799 },
800 {
801 "cell_type": "markdown",
802 "metadata": {},
803 "source": [
804 "Note: factors can also be added to an existing `Pipeline` instance using the `Pipeline.add` method. Using `add` looks something like this:\n",
805 " >>> my_pipe = Pipeline()\n",
806 " >>> f1 = SomeFactor(...)\n",
807 " >>> my_pipe.add(f1, 'f1')"
808 ]
809 },
810 {
811 "cell_type": "markdown",
812 "metadata": {},
813 "source": [
814 "###Latest\n",
815 "The most commonly used built-in `Factor` is `Latest`. The `Latest` factor gets the most recent value of a given data column. This factor is common enough that it is instantiated differently from other factors. The best way to get the latest value of a data column is by getting its `.latest` attribute. As an example, let's update `make_pipeline` to create a latest close price factor and add it to our pipeline:"
816 ]
817 },
818 {
819 "cell_type": "code",
820 "execution_count": 6,
821 "metadata": {},
822 "outputs": [],
823 "source": [
824 "def make_pipeline():\n",
825 "\n",
826 " mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)\n",
827 " latest_close = USEquityPricing.close.latest\n",
828 "\n",
829 " return Pipeline(\n",
830 " columns={\n",
831 " '10_day_mean_close': mean_close_10,\n",
832 " 'latest_close_price': latest_close\n",
833 " }\n",
834 " )"
835 ]
836 },
837 {
838 "cell_type": "markdown",
839 "metadata": {},
840 "source": [
841 "And now, when we make and run our pipeline again, there are two columns in our output dataframe. One column has the 10-day mean close price of each security, and the other has the latest close price."
842 ]
843 },
844 {
845 "cell_type": "code",
846 "execution_count": 7,
847 "metadata": {},
848 "outputs": [
849 {
850 "data": {
851 "text/html": [
852 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
853 "<table border=\"1\" class=\"dataframe\">\n",
854 " <thead>\n",
855 " <tr style=\"text-align: right;\">\n",
856 " <th></th>\n",
857 " <th></th>\n",
858 " <th>10_day_mean_close</th>\n",
859 " <th>latest_close_price</th>\n",
860 " </tr>\n",
861 " </thead>\n",
862 " <tbody>\n",
863 " <tr>\n",
864 " <th rowspan=\"5\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
865 " <th>Equity(2 [AA])</th>\n",
866 " <td>13.5595</td>\n",
867 " <td>14.015</td>\n",
868 " </tr>\n",
869 " <tr>\n",
870 " <th>Equity(21 [AAME])</th>\n",
871 " <td>3.9625</td>\n",
872 " <td>NaN</td>\n",
873 " </tr>\n",
874 " <tr>\n",
875 " <th>Equity(24 [AAPL])</th>\n",
876 " <td>129.0257</td>\n",
877 " <td>128.699</td>\n",
878 " </tr>\n",
879 " <tr>\n",
880 " <th>Equity(25 [AA_PR])</th>\n",
881 " <td>88.3625</td>\n",
882 " <td>NaN</td>\n",
883 " </tr>\n",
884 " <tr>\n",
885 " <th>Equity(31 [ABAX])</th>\n",
886 " <td>61.9209</td>\n",
887 " <td>55.030</td>\n",
888 " </tr>\n",
889 " </tbody>\n",
890 "</table>\n",
891 "</div>"
892 ],
893 "text/plain": [
894 " 10_day_mean_close \\\n",
895 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 13.5595 \n",
896 " Equity(21 [AAME]) 3.9625 \n",
897 " Equity(24 [AAPL]) 129.0257 \n",
898 " Equity(25 [AA_PR]) 88.3625 \n",
899 " Equity(31 [ABAX]) 61.9209 \n",
900 "\n",
901 " latest_close_price \n",
902 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 14.015 \n",
903 " Equity(21 [AAME]) NaN \n",
904 " Equity(24 [AAPL]) 128.699 \n",
905 " Equity(25 [AA_PR]) NaN \n",
906 " Equity(31 [ABAX]) 55.030 "
907 ]
908 },
909 "execution_count": 7,
910 "metadata": {},
911 "output_type": "execute_result"
912 }
913 ],
914 "source": [
915 "result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')\n",
916 "result.head(5)"
917 ]
918 },
919 {
920 "cell_type": "markdown",
921 "metadata": {},
922 "source": [
923 "`.latest` can sometimes return things other than `Factors`. We'll see examples of other possible return types in later lessons."
924 ]
925 },
926 {
927 "cell_type": "markdown",
928 "metadata": {},
929 "source": [
930 "## Default Inputs\n",
931 "Some factors have default inputs that should never be changed. For example the [VWAP built-in factor](https://www.quantopian.com/help#built-in-factors) is always calculated from `USEquityPricing.close` and `USEquityPricing.volume`. When a factor is always calculated from the same `BoundColumns`, we can call the constructor without specifying `inputs`."
932 ]
933 },
934 {
935 "cell_type": "code",
936 "execution_count": 8,
937 "metadata": {
938 "collapsed": true
939 },
940 "outputs": [],
941 "source": [
942 "from quantopian.pipeline.factors import VWAP\n",
943 "vwap = VWAP(window_length=10)"
944 ]
945 },
946 {
947 "cell_type": "markdown",
948 "metadata": {
949 "collapsed": true
950 },
951 "source": [
952 "In the next lesson, we will look at combining factors."
953 ]
954 }
955 ],
956 "metadata": {
957 "kernelspec": {
958 "display_name": "Python 2 (virtualenv)",
959 "language": "python",
960 "name": "python2"
961 },
962 "language_info": {
963 "codemirror_mode": {
964 "name": "ipython",
965 "version": 2
966 },
967 "file_extension": ".py",
968 "mimetype": "text/x-python",
969 "name": "python",
970 "nbconvert_exporter": "python",
971 "pygments_lexer": "ipython2",
972 "version": "2.7.12"
973 }
974 },
975 "nbformat": 4,
976 "nbformat_minor": 1
977 }