ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
notebook.ipynb
(44764B)
1 {
2 "cells": [
3 {
4 "cell_type": "code",
5 "execution_count": 1,
6 "metadata": {
7 "collapsed": true
8 },
9 "outputs": [],
10 "source": [
11 "from quantopian.pipeline import Pipeline\n",
12 "from quantopian.research import run_pipeline\n",
13 "from quantopian.pipeline.data.builtin import USEquityPricing\n",
14 "from quantopian.pipeline.factors import SimpleMovingAverage"
15 ]
16 },
17 {
18 "cell_type": "markdown",
19 "metadata": {},
20 "source": [
21 "##Filters\n",
22 "A Filter is a function from an asset and a moment in time to a boolean:\n",
23 "```\n",
24 "F(asset, timestamp) -> boolean\n",
25 "```\n",
26 "In Pipeline, [Filters](https://www.quantopian.com/help#quantopian_pipeline_filters_Filter) are used for narrowing down the set of securities included in a computation or in the final output of a pipeline. There are two common ways to create a `Filter`: comparison operators and `Factor`/`Classifier` methods.\n",
27 "\n",
28 "###Comparison Operators\n",
29 "Comparison operators on `Factors` and `Classifiers` produce Filters. Since we haven't looked at `Classifiers` yet, let's stick to examples using `Factors`. The following example produces a filter that returns `True` whenever the latest close price is above $20."
30 ]
31 },
32 {
33 "cell_type": "code",
34 "execution_count": 2,
35 "metadata": {
36 "collapsed": true
37 },
38 "outputs": [],
39 "source": [
40 "last_close_price = USEquityPricing.close.latest\n",
41 "close_price_filter = last_close_price > 20"
42 ]
43 },
44 {
45 "cell_type": "markdown",
46 "metadata": {},
47 "source": [
48 "And this example produces a filter that returns True whenever the 10-day mean is below the 30-day mean."
49 ]
50 },
51 {
52 "cell_type": "code",
53 "execution_count": 3,
54 "metadata": {
55 "collapsed": true
56 },
57 "outputs": [],
58 "source": [
59 "mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)\n",
60 "mean_close_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30)\n",
61 "mean_crossover_filter = mean_close_10 < mean_close_30"
62 ]
63 },
64 {
65 "cell_type": "markdown",
66 "metadata": {},
67 "source": [
68 "Remember, each security will get its own `True` or `False` value each day."
69 ]
70 },
71 {
72 "cell_type": "markdown",
73 "metadata": {},
74 "source": [
75 "###Factor/Classifier Methods\n",
76 "Various methods of the `Factor` and `Classifier` classes return `Filters`. Again, since we haven't yet looked at `Classifiers`, let's stick to `Factor` methods for now (we'll look at `Classifier` methods later). The `Factor.top(n)` method produces a `Filter` that returns `True` for the top `n` securities of a given `Factor`. The following example produces a filter that returns `True` for exactly 200 securities every day, indicating that those securities were in the top 200 by last close price across all known securities."
77 ]
78 },
79 {
80 "cell_type": "code",
81 "execution_count": 4,
82 "metadata": {
83 "collapsed": true
84 },
85 "outputs": [],
86 "source": [
87 "last_close_price = USEquityPricing.close.latest\n",
88 "top_close_price_filter = last_close_price.top(200)"
89 ]
90 },
91 {
92 "cell_type": "markdown",
93 "metadata": {},
94 "source": [
95 "For a full list of `Factor` methods that return `Filters`, see [this link](https://www.quantopian.com/help#quantopian_pipeline_factors_Factor).\n",
96 "\n",
97 "For a full list of `Classifier` methods that return `Filters`, see [this link](https://www.quantopian.com/help#quantopian_pipeline_classifiers_Classifier)."
98 ]
99 },
100 {
101 "cell_type": "markdown",
102 "metadata": {},
103 "source": [
104 "##Dollar Volume Filter\n",
105 "As a starting example, let's create a filter that returns `True` if a security's 30-day average dollar volume is above $10,000,000. To do this, we'll first need to create an `AverageDollarVolume` factor to compute the 30-day average dollar volume. Let's include the built-in `AverageDollarVolume` factor in our imports:"
106 ]
107 },
108 {
109 "cell_type": "code",
110 "execution_count": 5,
111 "metadata": {
112 "collapsed": true
113 },
114 "outputs": [],
115 "source": [
116 "from quantopian.pipeline.factors import AverageDollarVolume"
117 ]
118 },
119 {
120 "cell_type": "markdown",
121 "metadata": {},
122 "source": [
123 "And then, let's instantiate our average dollar volume factor."
124 ]
125 },
126 {
127 "cell_type": "code",
128 "execution_count": 6,
129 "metadata": {
130 "collapsed": true
131 },
132 "outputs": [],
133 "source": [
134 "dollar_volume = AverageDollarVolume(window_length=30)"
135 ]
136 },
137 {
138 "cell_type": "markdown",
139 "metadata": {},
140 "source": [
141 "By default, `AverageDollarVolume` uses `USEquityPricing.close` and `USEquityPricing.volume` as its `inputs`, so we don't specify them."
142 ]
143 },
144 {
145 "cell_type": "markdown",
146 "metadata": {},
147 "source": [
148 "Now that we have a dollar volume factor, we can create a filter with a boolean expression. The following line creates a filter returning `True` for securities with a `dollar_volume` greater than 10,000,000:"
149 ]
150 },
151 {
152 "cell_type": "code",
153 "execution_count": 7,
154 "metadata": {
155 "collapsed": false
156 },
157 "outputs": [],
158 "source": [
159 "high_dollar_volume = (dollar_volume > 10000000)"
160 ]
161 },
162 {
163 "cell_type": "markdown",
164 "metadata": {},
165 "source": [
166 "To see what this filter looks like, let's can add it as a column to the pipeline we defined in the previous lesson."
167 ]
168 },
169 {
170 "cell_type": "code",
171 "execution_count": 8,
172 "metadata": {
173 "collapsed": true
174 },
175 "outputs": [],
176 "source": [
177 "def make_pipeline():\n",
178 "\n",
179 " mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)\n",
180 " mean_close_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30)\n",
181 "\n",
182 " percent_difference = (mean_close_10 - mean_close_30) / mean_close_30\n",
183 " \n",
184 " dollar_volume = AverageDollarVolume(window_length=30)\n",
185 " high_dollar_volume = (dollar_volume > 10000000)\n",
186 "\n",
187 " return Pipeline(\n",
188 " columns={\n",
189 " 'percent_difference': percent_difference,\n",
190 " 'high_dollar_volume': high_dollar_volume\n",
191 " }\n",
192 " )"
193 ]
194 },
195 {
196 "cell_type": "markdown",
197 "metadata": {},
198 "source": [
199 "If we make and run our pipeline, we now have a column `high_dollar_volume` with a boolean value corresponding to the result of the expression for each security."
200 ]
201 },
202 {
203 "cell_type": "code",
204 "execution_count": 9,
205 "metadata": {
206 "collapsed": false
207 },
208 "outputs": [
209 {
210 "data": {
211 "text/html": [
212 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
213 "<table border=\"1\" class=\"dataframe\">\n",
214 " <thead>\n",
215 " <tr style=\"text-align: right;\">\n",
216 " <th></th>\n",
217 " <th></th>\n",
218 " <th>high_dollar_volume</th>\n",
219 " <th>percent_difference</th>\n",
220 " </tr>\n",
221 " </thead>\n",
222 " <tbody>\n",
223 " <tr>\n",
224 " <th rowspan=\"61\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
225 " <th>Equity(2 [AA])</th>\n",
226 " <td>True</td>\n",
227 " <td>0.017975</td>\n",
228 " </tr>\n",
229 " <tr>\n",
230 " <th>Equity(21 [AAME])</th>\n",
231 " <td>False</td>\n",
232 " <td>-0.002325</td>\n",
233 " </tr>\n",
234 " <tr>\n",
235 " <th>Equity(24 [AAPL])</th>\n",
236 " <td>True</td>\n",
237 " <td>0.016905</td>\n",
238 " </tr>\n",
239 " <tr>\n",
240 " <th>Equity(25 [AA_PR])</th>\n",
241 " <td>False</td>\n",
242 " <td>0.021544</td>\n",
243 " </tr>\n",
244 " <tr>\n",
245 " <th>Equity(31 [ABAX])</th>\n",
246 " <td>False</td>\n",
247 " <td>-0.019639</td>\n",
248 " </tr>\n",
249 " <tr>\n",
250 " <th>Equity(39 [DDC])</th>\n",
251 " <td>False</td>\n",
252 " <td>0.074730</td>\n",
253 " </tr>\n",
254 " <tr>\n",
255 " <th>Equity(41 [ARCB])</th>\n",
256 " <td>False</td>\n",
257 " <td>0.007067</td>\n",
258 " </tr>\n",
259 " <tr>\n",
260 " <th>Equity(52 [ABM])</th>\n",
261 " <td>False</td>\n",
262 " <td>0.003340</td>\n",
263 " </tr>\n",
264 " <tr>\n",
265 " <th>Equity(53 [ABMD])</th>\n",
266 " <td>True</td>\n",
267 " <td>-0.024682</td>\n",
268 " </tr>\n",
269 " <tr>\n",
270 " <th>Equity(62 [ABT])</th>\n",
271 " <td>True</td>\n",
272 " <td>0.014385</td>\n",
273 " </tr>\n",
274 " <tr>\n",
275 " <th>Equity(64 [ABX])</th>\n",
276 " <td>True</td>\n",
277 " <td>0.046963</td>\n",
278 " </tr>\n",
279 " <tr>\n",
280 " <th>Equity(66 [AB])</th>\n",
281 " <td>False</td>\n",
282 " <td>0.013488</td>\n",
283 " </tr>\n",
284 " <tr>\n",
285 " <th>Equity(67 [ADSK])</th>\n",
286 " <td>True</td>\n",
287 " <td>-0.003921</td>\n",
288 " </tr>\n",
289 " <tr>\n",
290 " <th>Equity(69 [ACAT])</th>\n",
291 " <td>False</td>\n",
292 " <td>-0.007079</td>\n",
293 " </tr>\n",
294 " <tr>\n",
295 " <th>Equity(70 [VBF])</th>\n",
296 " <td>False</td>\n",
297 " <td>0.005507</td>\n",
298 " </tr>\n",
299 " <tr>\n",
300 " <th>Equity(76 [TAP])</th>\n",
301 " <td>True</td>\n",
302 " <td>-0.008759</td>\n",
303 " </tr>\n",
304 " <tr>\n",
305 " <th>Equity(84 [ACET])</th>\n",
306 " <td>False</td>\n",
307 " <td>-0.056139</td>\n",
308 " </tr>\n",
309 " <tr>\n",
310 " <th>Equity(86 [ACG])</th>\n",
311 " <td>False</td>\n",
312 " <td>0.010096</td>\n",
313 " </tr>\n",
314 " <tr>\n",
315 " <th>Equity(88 [ACI])</th>\n",
316 " <td>False</td>\n",
317 " <td>-0.022089</td>\n",
318 " </tr>\n",
319 " <tr>\n",
320 " <th>Equity(100 [IEP])</th>\n",
321 " <td>False</td>\n",
322 " <td>0.011293</td>\n",
323 " </tr>\n",
324 " <tr>\n",
325 " <th>Equity(106 [ACU])</th>\n",
326 " <td>False</td>\n",
327 " <td>0.003306</td>\n",
328 " </tr>\n",
329 " <tr>\n",
330 " <th>Equity(110 [ACXM])</th>\n",
331 " <td>False</td>\n",
332 " <td>-0.029551</td>\n",
333 " </tr>\n",
334 " <tr>\n",
335 " <th>Equity(112 [ACY])</th>\n",
336 " <td>False</td>\n",
337 " <td>-0.057763</td>\n",
338 " </tr>\n",
339 " <tr>\n",
340 " <th>Equity(114 [ADBE])</th>\n",
341 " <td>True</td>\n",
342 " <td>0.009499</td>\n",
343 " </tr>\n",
344 " <tr>\n",
345 " <th>Equity(117 [AEY])</th>\n",
346 " <td>False</td>\n",
347 " <td>0.012543</td>\n",
348 " </tr>\n",
349 " <tr>\n",
350 " <th>Equity(122 [ADI])</th>\n",
351 " <td>True</td>\n",
352 " <td>0.009271</td>\n",
353 " </tr>\n",
354 " <tr>\n",
355 " <th>Equity(128 [ADM])</th>\n",
356 " <td>True</td>\n",
357 " <td>0.015760</td>\n",
358 " </tr>\n",
359 " <tr>\n",
360 " <th>Equity(134 [SXCL])</th>\n",
361 " <td>False</td>\n",
362 " <td>NaN</td>\n",
363 " </tr>\n",
364 " <tr>\n",
365 " <th>Equity(149 [ADX])</th>\n",
366 " <td>False</td>\n",
367 " <td>0.007232</td>\n",
368 " </tr>\n",
369 " <tr>\n",
370 " <th>Equity(153 [AE])</th>\n",
371 " <td>False</td>\n",
372 " <td>-0.112999</td>\n",
373 " </tr>\n",
374 " <tr>\n",
375 " <th>...</th>\n",
376 " <td>...</td>\n",
377 " <td>...</td>\n",
378 " </tr>\n",
379 " <tr>\n",
380 " <th>Equity(48961 [NYMT_O])</th>\n",
381 " <td>False</td>\n",
382 " <td>NaN</td>\n",
383 " </tr>\n",
384 " <tr>\n",
385 " <th>Equity(48962 [CSAL])</th>\n",
386 " <td>True</td>\n",
387 " <td>0.000000</td>\n",
388 " </tr>\n",
389 " <tr>\n",
390 " <th>Equity(48963 [PAK])</th>\n",
391 " <td>False</td>\n",
392 " <td>0.000000</td>\n",
393 " </tr>\n",
394 " <tr>\n",
395 " <th>Equity(48969 [NSA])</th>\n",
396 " <td>True</td>\n",
397 " <td>0.000000</td>\n",
398 " </tr>\n",
399 " <tr>\n",
400 " <th>Equity(48971 [BSM])</th>\n",
401 " <td>True</td>\n",
402 " <td>0.000000</td>\n",
403 " </tr>\n",
404 " <tr>\n",
405 " <th>Equity(48972 [EVA])</th>\n",
406 " <td>True</td>\n",
407 " <td>0.000000</td>\n",
408 " </tr>\n",
409 " <tr>\n",
410 " <th>Equity(48981 [APIC])</th>\n",
411 " <td>False</td>\n",
412 " <td>0.000000</td>\n",
413 " </tr>\n",
414 " <tr>\n",
415 " <th>Equity(48989 [UK])</th>\n",
416 " <td>False</td>\n",
417 " <td>0.000000</td>\n",
418 " </tr>\n",
419 " <tr>\n",
420 " <th>Equity(48990 [ACWF])</th>\n",
421 " <td>False</td>\n",
422 " <td>0.000000</td>\n",
423 " </tr>\n",
424 " <tr>\n",
425 " <th>Equity(48991 [ISCF])</th>\n",
426 " <td>False</td>\n",
427 " <td>0.000000</td>\n",
428 " </tr>\n",
429 " <tr>\n",
430 " <th>Equity(48992 [INTF])</th>\n",
431 " <td>False</td>\n",
432 " <td>0.000000</td>\n",
433 " </tr>\n",
434 " <tr>\n",
435 " <th>Equity(48993 [JETS])</th>\n",
436 " <td>False</td>\n",
437 " <td>0.000000</td>\n",
438 " </tr>\n",
439 " <tr>\n",
440 " <th>Equity(48994 [ACTX])</th>\n",
441 " <td>False</td>\n",
442 " <td>0.000000</td>\n",
443 " </tr>\n",
444 " <tr>\n",
445 " <th>Equity(48995 [LRGF])</th>\n",
446 " <td>False</td>\n",
447 " <td>0.000000</td>\n",
448 " </tr>\n",
449 " <tr>\n",
450 " <th>Equity(48996 [SMLF])</th>\n",
451 " <td>False</td>\n",
452 " <td>0.000000</td>\n",
453 " </tr>\n",
454 " <tr>\n",
455 " <th>Equity(48997 [VKTX])</th>\n",
456 " <td>False</td>\n",
457 " <td>0.000000</td>\n",
458 " </tr>\n",
459 " <tr>\n",
460 " <th>Equity(48998 [OPGN])</th>\n",
461 " <td>False</td>\n",
462 " <td>NaN</td>\n",
463 " </tr>\n",
464 " <tr>\n",
465 " <th>Equity(48999 [AAPC])</th>\n",
466 " <td>False</td>\n",
467 " <td>0.000000</td>\n",
468 " </tr>\n",
469 " <tr>\n",
470 " <th>Equity(49000 [BPMC])</th>\n",
471 " <td>False</td>\n",
472 " <td>0.000000</td>\n",
473 " </tr>\n",
474 " <tr>\n",
475 " <th>Equity(49001 [CLCD])</th>\n",
476 " <td>False</td>\n",
477 " <td>NaN</td>\n",
478 " </tr>\n",
479 " <tr>\n",
480 " <th>Equity(49004 [TNP_PRD])</th>\n",
481 " <td>False</td>\n",
482 " <td>0.000000</td>\n",
483 " </tr>\n",
484 " <tr>\n",
485 " <th>Equity(49005 [ARWA_U])</th>\n",
486 " <td>False</td>\n",
487 " <td>NaN</td>\n",
488 " </tr>\n",
489 " <tr>\n",
490 " <th>Equity(49006 [BVXV])</th>\n",
491 " <td>False</td>\n",
492 " <td>NaN</td>\n",
493 " </tr>\n",
494 " <tr>\n",
495 " <th>Equity(49007 [BVXV_W])</th>\n",
496 " <td>False</td>\n",
497 " <td>NaN</td>\n",
498 " </tr>\n",
499 " <tr>\n",
500 " <th>Equity(49008 [OPGN_W])</th>\n",
501 " <td>False</td>\n",
502 " <td>NaN</td>\n",
503 " </tr>\n",
504 " <tr>\n",
505 " <th>Equity(49009 [PRKU])</th>\n",
506 " <td>False</td>\n",
507 " <td>NaN</td>\n",
508 " </tr>\n",
509 " <tr>\n",
510 " <th>Equity(49010 [TBRA])</th>\n",
511 " <td>False</td>\n",
512 " <td>NaN</td>\n",
513 " </tr>\n",
514 " <tr>\n",
515 " <th>Equity(49131 [OESX])</th>\n",
516 " <td>False</td>\n",
517 " <td>NaN</td>\n",
518 " </tr>\n",
519 " <tr>\n",
520 " <th>Equity(49259 [ITUS])</th>\n",
521 " <td>False</td>\n",
522 " <td>NaN</td>\n",
523 " </tr>\n",
524 " <tr>\n",
525 " <th>Equity(49523 [TLGT])</th>\n",
526 " <td>False</td>\n",
527 " <td>NaN</td>\n",
528 " </tr>\n",
529 " </tbody>\n",
530 "</table>\n",
531 "<p>8236 rows × 2 columns</p>\n",
532 "</div>"
533 ],
534 "text/plain": [
535 " high_dollar_volume \\\n",
536 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) True \n",
537 " Equity(21 [AAME]) False \n",
538 " Equity(24 [AAPL]) True \n",
539 " Equity(25 [AA_PR]) False \n",
540 " Equity(31 [ABAX]) False \n",
541 " Equity(39 [DDC]) False \n",
542 " Equity(41 [ARCB]) False \n",
543 " Equity(52 [ABM]) False \n",
544 " Equity(53 [ABMD]) True \n",
545 " Equity(62 [ABT]) True \n",
546 " Equity(64 [ABX]) True \n",
547 " Equity(66 [AB]) False \n",
548 " Equity(67 [ADSK]) True \n",
549 " Equity(69 [ACAT]) False \n",
550 " Equity(70 [VBF]) False \n",
551 " Equity(76 [TAP]) True \n",
552 " Equity(84 [ACET]) False \n",
553 " Equity(86 [ACG]) False \n",
554 " Equity(88 [ACI]) False \n",
555 " Equity(100 [IEP]) False \n",
556 " Equity(106 [ACU]) False \n",
557 " Equity(110 [ACXM]) False \n",
558 " Equity(112 [ACY]) False \n",
559 " Equity(114 [ADBE]) True \n",
560 " Equity(117 [AEY]) False \n",
561 " Equity(122 [ADI]) True \n",
562 " Equity(128 [ADM]) True \n",
563 " Equity(134 [SXCL]) False \n",
564 " Equity(149 [ADX]) False \n",
565 " Equity(153 [AE]) False \n",
566 "... ... \n",
567 " Equity(48961 [NYMT_O]) False \n",
568 " Equity(48962 [CSAL]) True \n",
569 " Equity(48963 [PAK]) False \n",
570 " Equity(48969 [NSA]) True \n",
571 " Equity(48971 [BSM]) True \n",
572 " Equity(48972 [EVA]) True \n",
573 " Equity(48981 [APIC]) False \n",
574 " Equity(48989 [UK]) False \n",
575 " Equity(48990 [ACWF]) False \n",
576 " Equity(48991 [ISCF]) False \n",
577 " Equity(48992 [INTF]) False \n",
578 " Equity(48993 [JETS]) False \n",
579 " Equity(48994 [ACTX]) False \n",
580 " Equity(48995 [LRGF]) False \n",
581 " Equity(48996 [SMLF]) False \n",
582 " Equity(48997 [VKTX]) False \n",
583 " Equity(48998 [OPGN]) False \n",
584 " Equity(48999 [AAPC]) False \n",
585 " Equity(49000 [BPMC]) False \n",
586 " Equity(49001 [CLCD]) False \n",
587 " Equity(49004 [TNP_PRD]) False \n",
588 " Equity(49005 [ARWA_U]) False \n",
589 " Equity(49006 [BVXV]) False \n",
590 " Equity(49007 [BVXV_W]) False \n",
591 " Equity(49008 [OPGN_W]) False \n",
592 " Equity(49009 [PRKU]) False \n",
593 " Equity(49010 [TBRA]) False \n",
594 " Equity(49131 [OESX]) False \n",
595 " Equity(49259 [ITUS]) False \n",
596 " Equity(49523 [TLGT]) False \n",
597 "\n",
598 " percent_difference \n",
599 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 0.017975 \n",
600 " Equity(21 [AAME]) -0.002325 \n",
601 " Equity(24 [AAPL]) 0.016905 \n",
602 " Equity(25 [AA_PR]) 0.021544 \n",
603 " Equity(31 [ABAX]) -0.019639 \n",
604 " Equity(39 [DDC]) 0.074730 \n",
605 " Equity(41 [ARCB]) 0.007067 \n",
606 " Equity(52 [ABM]) 0.003340 \n",
607 " Equity(53 [ABMD]) -0.024682 \n",
608 " Equity(62 [ABT]) 0.014385 \n",
609 " Equity(64 [ABX]) 0.046963 \n",
610 " Equity(66 [AB]) 0.013488 \n",
611 " Equity(67 [ADSK]) -0.003921 \n",
612 " Equity(69 [ACAT]) -0.007079 \n",
613 " Equity(70 [VBF]) 0.005507 \n",
614 " Equity(76 [TAP]) -0.008759 \n",
615 " Equity(84 [ACET]) -0.056139 \n",
616 " Equity(86 [ACG]) 0.010096 \n",
617 " Equity(88 [ACI]) -0.022089 \n",
618 " Equity(100 [IEP]) 0.011293 \n",
619 " Equity(106 [ACU]) 0.003306 \n",
620 " Equity(110 [ACXM]) -0.029551 \n",
621 " Equity(112 [ACY]) -0.057763 \n",
622 " Equity(114 [ADBE]) 0.009499 \n",
623 " Equity(117 [AEY]) 0.012543 \n",
624 " Equity(122 [ADI]) 0.009271 \n",
625 " Equity(128 [ADM]) 0.015760 \n",
626 " Equity(134 [SXCL]) NaN \n",
627 " Equity(149 [ADX]) 0.007232 \n",
628 " Equity(153 [AE]) -0.112999 \n",
629 "... ... \n",
630 " Equity(48961 [NYMT_O]) NaN \n",
631 " Equity(48962 [CSAL]) 0.000000 \n",
632 " Equity(48963 [PAK]) 0.000000 \n",
633 " Equity(48969 [NSA]) 0.000000 \n",
634 " Equity(48971 [BSM]) 0.000000 \n",
635 " Equity(48972 [EVA]) 0.000000 \n",
636 " Equity(48981 [APIC]) 0.000000 \n",
637 " Equity(48989 [UK]) 0.000000 \n",
638 " Equity(48990 [ACWF]) 0.000000 \n",
639 " Equity(48991 [ISCF]) 0.000000 \n",
640 " Equity(48992 [INTF]) 0.000000 \n",
641 " Equity(48993 [JETS]) 0.000000 \n",
642 " Equity(48994 [ACTX]) 0.000000 \n",
643 " Equity(48995 [LRGF]) 0.000000 \n",
644 " Equity(48996 [SMLF]) 0.000000 \n",
645 " Equity(48997 [VKTX]) 0.000000 \n",
646 " Equity(48998 [OPGN]) NaN \n",
647 " Equity(48999 [AAPC]) 0.000000 \n",
648 " Equity(49000 [BPMC]) 0.000000 \n",
649 " Equity(49001 [CLCD]) NaN \n",
650 " Equity(49004 [TNP_PRD]) 0.000000 \n",
651 " Equity(49005 [ARWA_U]) NaN \n",
652 " Equity(49006 [BVXV]) NaN \n",
653 " Equity(49007 [BVXV_W]) NaN \n",
654 " Equity(49008 [OPGN_W]) NaN \n",
655 " Equity(49009 [PRKU]) NaN \n",
656 " Equity(49010 [TBRA]) NaN \n",
657 " Equity(49131 [OESX]) NaN \n",
658 " Equity(49259 [ITUS]) NaN \n",
659 " Equity(49523 [TLGT]) NaN \n",
660 "\n",
661 "[8236 rows x 2 columns]"
662 ]
663 },
664 "execution_count": 9,
665 "metadata": {},
666 "output_type": "execute_result"
667 }
668 ],
669 "source": [
670 "result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')\n",
671 "result"
672 ]
673 },
674 {
675 "cell_type": "markdown",
676 "metadata": {},
677 "source": [
678 "##Applying a Screen\n",
679 "By default, a pipeline produces computed values each day for every asset in the Quantopian database. Very often however, we only care about a subset of securities that meet specific criteria (for example, we might only care about securities that have enough daily trading volume to fill our orders quickly). We can tell our Pipeline to ignore securities for which a filter produces `False` by passing that filter to our Pipeline via the `screen` keyword.\n",
680 "\n",
681 "To screen our pipeline output for securities with a 30-day average dollar volume greater than $10,000,000, we can simply pass our `high_dollar_volume` filter as the `screen` argument. This is what our `make_pipeline` function now looks like:"
682 ]
683 },
684 {
685 "cell_type": "code",
686 "execution_count": 10,
687 "metadata": {
688 "collapsed": true
689 },
690 "outputs": [],
691 "source": [
692 "def make_pipeline():\n",
693 "\n",
694 " mean_close_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)\n",
695 " mean_close_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30)\n",
696 "\n",
697 " percent_difference = (mean_close_10 - mean_close_30) / mean_close_30\n",
698 "\n",
699 " dollar_volume = AverageDollarVolume(window_length=30)\n",
700 " high_dollar_volume = dollar_volume > 10000000\n",
701 "\n",
702 " return Pipeline(\n",
703 " columns={\n",
704 " 'percent_difference': percent_difference\n",
705 " },\n",
706 " screen=high_dollar_volume\n",
707 " )"
708 ]
709 },
710 {
711 "cell_type": "markdown",
712 "metadata": {},
713 "source": [
714 "When we run this, the pipeline output only includes securities that pass the `high_dollar_volume` filter on a given day. For example, running this pipeline on May 5th, 2015 results in an output for ~2,100 securities"
715 ]
716 },
717 {
718 "cell_type": "code",
719 "execution_count": 11,
720 "metadata": {
721 "collapsed": false
722 },
723 "outputs": [
724 {
725 "name": "stdout",
726 "output_type": "stream",
727 "text": [
728 "Number of securities that passed the filter: 2110\n"
729 ]
730 },
731 {
732 "data": {
733 "text/html": [
734 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
735 "<table border=\"1\" class=\"dataframe\">\n",
736 " <thead>\n",
737 " <tr style=\"text-align: right;\">\n",
738 " <th></th>\n",
739 " <th></th>\n",
740 " <th>percent_difference</th>\n",
741 " </tr>\n",
742 " </thead>\n",
743 " <tbody>\n",
744 " <tr>\n",
745 " <th rowspan=\"61\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
746 " <th>Equity(2 [AA])</th>\n",
747 " <td>0.017975</td>\n",
748 " </tr>\n",
749 " <tr>\n",
750 " <th>Equity(24 [AAPL])</th>\n",
751 " <td>0.016905</td>\n",
752 " </tr>\n",
753 " <tr>\n",
754 " <th>Equity(53 [ABMD])</th>\n",
755 " <td>-0.024682</td>\n",
756 " </tr>\n",
757 " <tr>\n",
758 " <th>Equity(62 [ABT])</th>\n",
759 " <td>0.014385</td>\n",
760 " </tr>\n",
761 " <tr>\n",
762 " <th>Equity(64 [ABX])</th>\n",
763 " <td>0.046963</td>\n",
764 " </tr>\n",
765 " <tr>\n",
766 " <th>Equity(67 [ADSK])</th>\n",
767 " <td>-0.003921</td>\n",
768 " </tr>\n",
769 " <tr>\n",
770 " <th>Equity(76 [TAP])</th>\n",
771 " <td>-0.008759</td>\n",
772 " </tr>\n",
773 " <tr>\n",
774 " <th>Equity(114 [ADBE])</th>\n",
775 " <td>0.009499</td>\n",
776 " </tr>\n",
777 " <tr>\n",
778 " <th>Equity(122 [ADI])</th>\n",
779 " <td>0.009271</td>\n",
780 " </tr>\n",
781 " <tr>\n",
782 " <th>Equity(128 [ADM])</th>\n",
783 " <td>0.015760</td>\n",
784 " </tr>\n",
785 " <tr>\n",
786 " <th>Equity(154 [AEM])</th>\n",
787 " <td>0.026035</td>\n",
788 " </tr>\n",
789 " <tr>\n",
790 " <th>Equity(161 [AEP])</th>\n",
791 " <td>0.010405</td>\n",
792 " </tr>\n",
793 " <tr>\n",
794 " <th>Equity(166 [AES])</th>\n",
795 " <td>0.022158</td>\n",
796 " </tr>\n",
797 " <tr>\n",
798 " <th>Equity(168 [AET])</th>\n",
799 " <td>0.005853</td>\n",
800 " </tr>\n",
801 " <tr>\n",
802 " <th>Equity(185 [AFL])</th>\n",
803 " <td>-0.002239</td>\n",
804 " </tr>\n",
805 " <tr>\n",
806 " <th>Equity(197 [AGCO])</th>\n",
807 " <td>0.032124</td>\n",
808 " </tr>\n",
809 " <tr>\n",
810 " <th>Equity(216 [HES])</th>\n",
811 " <td>0.036528</td>\n",
812 " </tr>\n",
813 " <tr>\n",
814 " <th>Equity(239 [AIG])</th>\n",
815 " <td>0.012322</td>\n",
816 " </tr>\n",
817 " <tr>\n",
818 " <th>Equity(253 [AIR])</th>\n",
819 " <td>-0.012412</td>\n",
820 " </tr>\n",
821 " <tr>\n",
822 " <th>Equity(266 [AJG])</th>\n",
823 " <td>0.012267</td>\n",
824 " </tr>\n",
825 " <tr>\n",
826 " <th>Equity(270 [AKRX])</th>\n",
827 " <td>-0.024963</td>\n",
828 " </tr>\n",
829 " <tr>\n",
830 " <th>Equity(273 [ALU])</th>\n",
831 " <td>-0.021750</td>\n",
832 " </tr>\n",
833 " <tr>\n",
834 " <th>Equity(300 [ALK])</th>\n",
835 " <td>0.015147</td>\n",
836 " </tr>\n",
837 " <tr>\n",
838 " <th>Equity(301 [ALKS])</th>\n",
839 " <td>-0.033228</td>\n",
840 " </tr>\n",
841 " <tr>\n",
842 " <th>Equity(328 [ALTR])</th>\n",
843 " <td>0.012284</td>\n",
844 " </tr>\n",
845 " <tr>\n",
846 " <th>Equity(337 [AMAT])</th>\n",
847 " <td>-0.050162</td>\n",
848 " </tr>\n",
849 " <tr>\n",
850 " <th>Equity(351 [AMD])</th>\n",
851 " <td>-0.101477</td>\n",
852 " </tr>\n",
853 " <tr>\n",
854 " <th>Equity(353 [AME])</th>\n",
855 " <td>-0.003008</td>\n",
856 " </tr>\n",
857 " <tr>\n",
858 " <th>Equity(357 [TWX])</th>\n",
859 " <td>0.000365</td>\n",
860 " </tr>\n",
861 " <tr>\n",
862 " <th>Equity(368 [AMGN])</th>\n",
863 " <td>0.008860</td>\n",
864 " </tr>\n",
865 " <tr>\n",
866 " <th>...</th>\n",
867 " <td>...</td>\n",
868 " </tr>\n",
869 " <tr>\n",
870 " <th>Equity(48126 [HABT])</th>\n",
871 " <td>0.063080</td>\n",
872 " </tr>\n",
873 " <tr>\n",
874 " <th>Equity(48129 [UBS])</th>\n",
875 " <td>0.025888</td>\n",
876 " </tr>\n",
877 " <tr>\n",
878 " <th>Equity(48169 [KLXI])</th>\n",
879 " <td>0.021062</td>\n",
880 " </tr>\n",
881 " <tr>\n",
882 " <th>Equity(48215 [QSR])</th>\n",
883 " <td>0.037460</td>\n",
884 " </tr>\n",
885 " <tr>\n",
886 " <th>Equity(48220 [LC])</th>\n",
887 " <td>-0.035048</td>\n",
888 " </tr>\n",
889 " <tr>\n",
890 " <th>Equity(48317 [JUNO])</th>\n",
891 " <td>-0.103370</td>\n",
892 " </tr>\n",
893 " <tr>\n",
894 " <th>Equity(48384 [QRVO])</th>\n",
895 " <td>-0.050578</td>\n",
896 " </tr>\n",
897 " <tr>\n",
898 " <th>Equity(48465 [SWNC])</th>\n",
899 " <td>0.061669</td>\n",
900 " </tr>\n",
901 " <tr>\n",
902 " <th>Equity(48486 [BOX])</th>\n",
903 " <td>-0.003837</td>\n",
904 " </tr>\n",
905 " <tr>\n",
906 " <th>Equity(48531 [VSTO])</th>\n",
907 " <td>0.017196</td>\n",
908 " </tr>\n",
909 " <tr>\n",
910 " <th>Equity(48543 [SHAK])</th>\n",
911 " <td>0.175877</td>\n",
912 " </tr>\n",
913 " <tr>\n",
914 " <th>Equity(48544 [HIFR])</th>\n",
915 " <td>0.027339</td>\n",
916 " </tr>\n",
917 " <tr>\n",
918 " <th>Equity(48547 [ONCE])</th>\n",
919 " <td>-0.112191</td>\n",
920 " </tr>\n",
921 " <tr>\n",
922 " <th>Equity(48575 [XHR])</th>\n",
923 " <td>-0.008521</td>\n",
924 " </tr>\n",
925 " <tr>\n",
926 " <th>Equity(48629 [INOV])</th>\n",
927 " <td>-0.068366</td>\n",
928 " </tr>\n",
929 " <tr>\n",
930 " <th>Equity(48662 [JPM_PRF])</th>\n",
931 " <td>0.002978</td>\n",
932 " </tr>\n",
933 " <tr>\n",
934 " <th>Equity(48672 [TOTL])</th>\n",
935 " <td>0.000991</td>\n",
936 " </tr>\n",
937 " <tr>\n",
938 " <th>Equity(48730 [AGN_PRA])</th>\n",
939 " <td>-0.008843</td>\n",
940 " </tr>\n",
941 " <tr>\n",
942 " <th>Equity(48821 [CJES])</th>\n",
943 " <td>0.099492</td>\n",
944 " </tr>\n",
945 " <tr>\n",
946 " <th>Equity(48823 [SEDG])</th>\n",
947 " <td>0.056643</td>\n",
948 " </tr>\n",
949 " <tr>\n",
950 " <th>Equity(48863 [GDDY])</th>\n",
951 " <td>-0.003563</td>\n",
952 " </tr>\n",
953 " <tr>\n",
954 " <th>Equity(48892 [IGT])</th>\n",
955 " <td>0.005591</td>\n",
956 " </tr>\n",
957 " <tr>\n",
958 " <th>Equity(48925 [ADRO])</th>\n",
959 " <td>-0.076840</td>\n",
960 " </tr>\n",
961 " <tr>\n",
962 " <th>Equity(48933 [PRTY])</th>\n",
963 " <td>-0.001741</td>\n",
964 " </tr>\n",
965 " <tr>\n",
966 " <th>Equity(48934 [ETSY])</th>\n",
967 " <td>-0.030142</td>\n",
968 " </tr>\n",
969 " <tr>\n",
970 " <th>Equity(48943 [VIRT])</th>\n",
971 " <td>-0.009077</td>\n",
972 " </tr>\n",
973 " <tr>\n",
974 " <th>Equity(48962 [CSAL])</th>\n",
975 " <td>0.000000</td>\n",
976 " </tr>\n",
977 " <tr>\n",
978 " <th>Equity(48969 [NSA])</th>\n",
979 " <td>0.000000</td>\n",
980 " </tr>\n",
981 " <tr>\n",
982 " <th>Equity(48971 [BSM])</th>\n",
983 " <td>0.000000</td>\n",
984 " </tr>\n",
985 " <tr>\n",
986 " <th>Equity(48972 [EVA])</th>\n",
987 " <td>0.000000</td>\n",
988 " </tr>\n",
989 " </tbody>\n",
990 "</table>\n",
991 "<p>2110 rows × 1 columns</p>\n",
992 "</div>"
993 ],
994 "text/plain": [
995 " percent_difference\n",
996 "2015-05-05 00:00:00+00:00 Equity(2 [AA]) 0.017975\n",
997 " Equity(24 [AAPL]) 0.016905\n",
998 " Equity(53 [ABMD]) -0.024682\n",
999 " Equity(62 [ABT]) 0.014385\n",
1000 " Equity(64 [ABX]) 0.046963\n",
1001 " Equity(67 [ADSK]) -0.003921\n",
1002 " Equity(76 [TAP]) -0.008759\n",
1003 " Equity(114 [ADBE]) 0.009499\n",
1004 " Equity(122 [ADI]) 0.009271\n",
1005 " Equity(128 [ADM]) 0.015760\n",
1006 " Equity(154 [AEM]) 0.026035\n",
1007 " Equity(161 [AEP]) 0.010405\n",
1008 " Equity(166 [AES]) 0.022158\n",
1009 " Equity(168 [AET]) 0.005853\n",
1010 " Equity(185 [AFL]) -0.002239\n",
1011 " Equity(197 [AGCO]) 0.032124\n",
1012 " Equity(216 [HES]) 0.036528\n",
1013 " Equity(239 [AIG]) 0.012322\n",
1014 " Equity(253 [AIR]) -0.012412\n",
1015 " Equity(266 [AJG]) 0.012267\n",
1016 " Equity(270 [AKRX]) -0.024963\n",
1017 " Equity(273 [ALU]) -0.021750\n",
1018 " Equity(300 [ALK]) 0.015147\n",
1019 " Equity(301 [ALKS]) -0.033228\n",
1020 " Equity(328 [ALTR]) 0.012284\n",
1021 " Equity(337 [AMAT]) -0.050162\n",
1022 " Equity(351 [AMD]) -0.101477\n",
1023 " Equity(353 [AME]) -0.003008\n",
1024 " Equity(357 [TWX]) 0.000365\n",
1025 " Equity(368 [AMGN]) 0.008860\n",
1026 "... ...\n",
1027 " Equity(48126 [HABT]) 0.063080\n",
1028 " Equity(48129 [UBS]) 0.025888\n",
1029 " Equity(48169 [KLXI]) 0.021062\n",
1030 " Equity(48215 [QSR]) 0.037460\n",
1031 " Equity(48220 [LC]) -0.035048\n",
1032 " Equity(48317 [JUNO]) -0.103370\n",
1033 " Equity(48384 [QRVO]) -0.050578\n",
1034 " Equity(48465 [SWNC]) 0.061669\n",
1035 " Equity(48486 [BOX]) -0.003837\n",
1036 " Equity(48531 [VSTO]) 0.017196\n",
1037 " Equity(48543 [SHAK]) 0.175877\n",
1038 " Equity(48544 [HIFR]) 0.027339\n",
1039 " Equity(48547 [ONCE]) -0.112191\n",
1040 " Equity(48575 [XHR]) -0.008521\n",
1041 " Equity(48629 [INOV]) -0.068366\n",
1042 " Equity(48662 [JPM_PRF]) 0.002978\n",
1043 " Equity(48672 [TOTL]) 0.000991\n",
1044 " Equity(48730 [AGN_PRA]) -0.008843\n",
1045 " Equity(48821 [CJES]) 0.099492\n",
1046 " Equity(48823 [SEDG]) 0.056643\n",
1047 " Equity(48863 [GDDY]) -0.003563\n",
1048 " Equity(48892 [IGT]) 0.005591\n",
1049 " Equity(48925 [ADRO]) -0.076840\n",
1050 " Equity(48933 [PRTY]) -0.001741\n",
1051 " Equity(48934 [ETSY]) -0.030142\n",
1052 " Equity(48943 [VIRT]) -0.009077\n",
1053 " Equity(48962 [CSAL]) 0.000000\n",
1054 " Equity(48969 [NSA]) 0.000000\n",
1055 " Equity(48971 [BSM]) 0.000000\n",
1056 " Equity(48972 [EVA]) 0.000000\n",
1057 "\n",
1058 "[2110 rows x 1 columns]"
1059 ]
1060 },
1061 "execution_count": 11,
1062 "metadata": {},
1063 "output_type": "execute_result"
1064 }
1065 ],
1066 "source": [
1067 "result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')\n",
1068 "print 'Number of securities that passed the filter: %d' % len(result)\n",
1069 "result"
1070 ]
1071 },
1072 {
1073 "cell_type": "markdown",
1074 "metadata": {},
1075 "source": [
1076 "##Inverting a Filter\n",
1077 "The `~` operator is used to invert a filter, swapping all `True` values with `Falses` and vice-versa. For example, we can write the following to filter for low dollar volume securities:"
1078 ]
1079 },
1080 {
1081 "cell_type": "code",
1082 "execution_count": 12,
1083 "metadata": {
1084 "collapsed": true
1085 },
1086 "outputs": [],
1087 "source": [
1088 "low_dollar_volume = ~high_dollar_volume"
1089 ]
1090 },
1091 {
1092 "cell_type": "markdown",
1093 "metadata": {},
1094 "source": [
1095 "This will return `True` for all securities with an average dollar volume below or equal to $10,000,000 over the last 30 days."
1096 ]
1097 },
1098 {
1099 "cell_type": "markdown",
1100 "metadata": {},
1101 "source": [
1102 "In the next lesson, we will look at combining filters."
1103 ]
1104 }
1105 ],
1106 "metadata": {
1107 "kernelspec": {
1108 "display_name": "Python 2",
1109 "language": "python",
1110 "name": "python2"
1111 },
1112 "language_info": {
1113 "codemirror_mode": {
1114 "name": "ipython",
1115 "version": 2
1116 },
1117 "file_extension": ".py",
1118 "mimetype": "text/x-python",
1119 "name": "python",
1120 "nbconvert_exporter": "python",
1121 "pygments_lexer": "ipython2",
1122 "version": "2.7.11"
1123 }
1124 },
1125 "nbformat": 4,
1126 "nbformat_minor": 0
1127 }