ml-finance-python
python scripts for finance machine learning
git clone https://9o.is/git/ml-finance-python.git
notebook.ipynb
(14722B)
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "##Creating a Pipeline\n",
8 "In this lesson, we will take a look at creating an empty pipeline. First, let's import the Pipeline class:"
9 ]
10 },
11 {
12 "cell_type": "code",
13 "execution_count": 1,
14 "metadata": {
15 "collapsed": true
16 },
17 "outputs": [],
18 "source": [
19 "from quantopian.pipeline import Pipeline"
20 ]
21 },
22 {
23 "cell_type": "markdown",
24 "metadata": {},
25 "source": [
26 "In a new cell, let's define a function to create our pipeline. Wrapping our pipeline creation in a function sets up a structure for more complex pipelines that we will see later on. For now, this function simply returns an empty pipeline:"
27 ]
28 },
29 {
30 "cell_type": "code",
31 "execution_count": 2,
32 "metadata": {
33 "collapsed": true
34 },
35 "outputs": [],
36 "source": [
37 "def make_pipeline():\n",
38 " return Pipeline()"
39 ]
40 },
41 {
42 "cell_type": "markdown",
43 "metadata": {},
44 "source": [
45 "In a new cell, let's instantiate our pipeline by running `make_pipeline()`:"
46 ]
47 },
48 {
49 "cell_type": "code",
50 "execution_count": 3,
51 "metadata": {
52 "collapsed": false
53 },
54 "outputs": [],
55 "source": [
56 "my_pipe = make_pipeline()"
57 ]
58 },
59 {
60 "cell_type": "markdown",
61 "metadata": {},
62 "source": [
63 "###Running a Pipeline\n",
64 "\n",
65 "Now that we have a reference to an empty Pipeline, `my_pipe` let's run it to see what it looks like. Before running our pipeline, we first need to import `run_pipeline`, a research-only function that allows us to run a pipeline over a specified time period."
66 ]
67 },
68 {
69 "cell_type": "code",
70 "execution_count": 4,
71 "metadata": {
72 "collapsed": true
73 },
74 "outputs": [],
75 "source": [
76 "from quantopian.research import run_pipeline"
77 ]
78 },
79 {
80 "cell_type": "markdown",
81 "metadata": {},
82 "source": [
83 "Let's run our pipeline for one day (2015-05-05) with `run_pipeline` and display it. Note that the 2nd and 3rd arguments are the start and end dates of the simulation, respectively."
84 ]
85 },
86 {
87 "cell_type": "code",
88 "execution_count": 5,
89 "metadata": {
90 "collapsed": false
91 },
92 "outputs": [],
93 "source": [
94 "result = run_pipeline(my_pipe, '2015-05-05', '2015-05-05')"
95 ]
96 },
97 {
98 "cell_type": "markdown",
99 "metadata": {},
100 "source": [
101 "A call to `run_pipeline` returns a [pandas DataFrame](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) indexed by date and securities. Let's see what the empty pipeline looks like:"
102 ]
103 },
104 {
105 "cell_type": "code",
106 "execution_count": 6,
107 "metadata": {
108 "collapsed": false
109 },
110 "outputs": [
111 {
112 "data": {
113 "text/html": [
114 "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
115 "<table border=\"1\" class=\"dataframe\">\n",
116 " <thead>\n",
117 " <tr style=\"text-align: right;\">\n",
118 " <th></th>\n",
119 " <th></th>\n",
120 " </tr>\n",
121 " </thead>\n",
122 " <tbody>\n",
123 " <tr>\n",
124 " <th rowspan=\"61\" valign=\"top\">2015-05-05 00:00:00+00:00</th>\n",
125 " <th>Equity(2 [AA])</th>\n",
126 " </tr>\n",
127 " <tr>\n",
128 " <th>Equity(21 [AAME])</th>\n",
129 " </tr>\n",
130 " <tr>\n",
131 " <th>Equity(24 [AAPL])</th>\n",
132 " </tr>\n",
133 " <tr>\n",
134 " <th>Equity(25 [AA_PR])</th>\n",
135 " </tr>\n",
136 " <tr>\n",
137 " <th>Equity(31 [ABAX])</th>\n",
138 " </tr>\n",
139 " <tr>\n",
140 " <th>Equity(39 [DDC])</th>\n",
141 " </tr>\n",
142 " <tr>\n",
143 " <th>Equity(41 [ARCB])</th>\n",
144 " </tr>\n",
145 " <tr>\n",
146 " <th>Equity(52 [ABM])</th>\n",
147 " </tr>\n",
148 " <tr>\n",
149 " <th>Equity(53 [ABMD])</th>\n",
150 " </tr>\n",
151 " <tr>\n",
152 " <th>Equity(62 [ABT])</th>\n",
153 " </tr>\n",
154 " <tr>\n",
155 " <th>Equity(64 [ABX])</th>\n",
156 " </tr>\n",
157 " <tr>\n",
158 " <th>Equity(66 [AB])</th>\n",
159 " </tr>\n",
160 " <tr>\n",
161 " <th>Equity(67 [ADSK])</th>\n",
162 " </tr>\n",
163 " <tr>\n",
164 " <th>Equity(69 [ACAT])</th>\n",
165 " </tr>\n",
166 " <tr>\n",
167 " <th>Equity(70 [VBF])</th>\n",
168 " </tr>\n",
169 " <tr>\n",
170 " <th>Equity(76 [TAP])</th>\n",
171 " </tr>\n",
172 " <tr>\n",
173 " <th>Equity(84 [ACET])</th>\n",
174 " </tr>\n",
175 " <tr>\n",
176 " <th>Equity(86 [ACG])</th>\n",
177 " </tr>\n",
178 " <tr>\n",
179 " <th>Equity(88 [ACI])</th>\n",
180 " </tr>\n",
181 " <tr>\n",
182 " <th>Equity(100 [IEP])</th>\n",
183 " </tr>\n",
184 " <tr>\n",
185 " <th>Equity(106 [ACU])</th>\n",
186 " </tr>\n",
187 " <tr>\n",
188 " <th>Equity(110 [ACXM])</th>\n",
189 " </tr>\n",
190 " <tr>\n",
191 " <th>Equity(112 [ACY])</th>\n",
192 " </tr>\n",
193 " <tr>\n",
194 " <th>Equity(114 [ADBE])</th>\n",
195 " </tr>\n",
196 " <tr>\n",
197 " <th>Equity(117 [AEY])</th>\n",
198 " </tr>\n",
199 " <tr>\n",
200 " <th>Equity(122 [ADI])</th>\n",
201 " </tr>\n",
202 " <tr>\n",
203 " <th>Equity(128 [ADM])</th>\n",
204 " </tr>\n",
205 " <tr>\n",
206 " <th>Equity(134 [SXCL])</th>\n",
207 " </tr>\n",
208 " <tr>\n",
209 " <th>Equity(149 [ADX])</th>\n",
210 " </tr>\n",
211 " <tr>\n",
212 " <th>Equity(153 [AE])</th>\n",
213 " </tr>\n",
214 " <tr>\n",
215 " <th>...</th>\n",
216 " </tr>\n",
217 " <tr>\n",
218 " <th>Equity(48961 [NYMT_O])</th>\n",
219 " </tr>\n",
220 " <tr>\n",
221 " <th>Equity(48962 [CSAL])</th>\n",
222 " </tr>\n",
223 " <tr>\n",
224 " <th>Equity(48963 [PAK])</th>\n",
225 " </tr>\n",
226 " <tr>\n",
227 " <th>Equity(48969 [NSA])</th>\n",
228 " </tr>\n",
229 " <tr>\n",
230 " <th>Equity(48971 [BSM])</th>\n",
231 " </tr>\n",
232 " <tr>\n",
233 " <th>Equity(48972 [EVA])</th>\n",
234 " </tr>\n",
235 " <tr>\n",
236 " <th>Equity(48981 [APIC])</th>\n",
237 " </tr>\n",
238 " <tr>\n",
239 " <th>Equity(48989 [UK])</th>\n",
240 " </tr>\n",
241 " <tr>\n",
242 " <th>Equity(48990 [ACWF])</th>\n",
243 " </tr>\n",
244 " <tr>\n",
245 " <th>Equity(48991 [ISCF])</th>\n",
246 " </tr>\n",
247 " <tr>\n",
248 " <th>Equity(48992 [INTF])</th>\n",
249 " </tr>\n",
250 " <tr>\n",
251 " <th>Equity(48993 [JETS])</th>\n",
252 " </tr>\n",
253 " <tr>\n",
254 " <th>Equity(48994 [ACTX])</th>\n",
255 " </tr>\n",
256 " <tr>\n",
257 " <th>Equity(48995 [LRGF])</th>\n",
258 " </tr>\n",
259 " <tr>\n",
260 " <th>Equity(48996 [SMLF])</th>\n",
261 " </tr>\n",
262 " <tr>\n",
263 " <th>Equity(48997 [VKTX])</th>\n",
264 " </tr>\n",
265 " <tr>\n",
266 " <th>Equity(48998 [OPGN])</th>\n",
267 " </tr>\n",
268 " <tr>\n",
269 " <th>Equity(48999 [AAPC])</th>\n",
270 " </tr>\n",
271 " <tr>\n",
272 " <th>Equity(49000 [BPMC])</th>\n",
273 " </tr>\n",
274 " <tr>\n",
275 " <th>Equity(49001 [CLCD])</th>\n",
276 " </tr>\n",
277 " <tr>\n",
278 " <th>Equity(49004 [TNP_PRD])</th>\n",
279 " </tr>\n",
280 " <tr>\n",
281 " <th>Equity(49005 [ARWA_U])</th>\n",
282 " </tr>\n",
283 " <tr>\n",
284 " <th>Equity(49006 [BVXV])</th>\n",
285 " </tr>\n",
286 " <tr>\n",
287 " <th>Equity(49007 [BVXV_W])</th>\n",
288 " </tr>\n",
289 " <tr>\n",
290 " <th>Equity(49008 [OPGN_W])</th>\n",
291 " </tr>\n",
292 " <tr>\n",
293 " <th>Equity(49009 [PRKU])</th>\n",
294 " </tr>\n",
295 " <tr>\n",
296 " <th>Equity(49010 [TBRA])</th>\n",
297 " </tr>\n",
298 " <tr>\n",
299 " <th>Equity(49131 [OESX])</th>\n",
300 " </tr>\n",
301 " <tr>\n",
302 " <th>Equity(49259 [ITUS])</th>\n",
303 " </tr>\n",
304 " <tr>\n",
305 " <th>Equity(49523 [TLGT])</th>\n",
306 " </tr>\n",
307 " </tbody>\n",
308 "</table>\n",
309 "<p>8236 rows × 0 columns</p>\n",
310 "</div>"
311 ],
312 "text/plain": [
313 "Empty DataFrame\n",
314 "Columns: []\n",
315 "Index: [(2015-05-05 00:00:00+00:00, Equity(2 [AA])), (2015-05-05 00:00:00+00:00, Equity(21 [AAME])), (2015-05-05 00:00:00+00:00, Equity(24 [AAPL])), (2015-05-05 00:00:00+00:00, Equity(25 [AA_PR])), (2015-05-05 00:00:00+00:00, Equity(31 [ABAX])), (2015-05-05 00:00:00+00:00, Equity(39 [DDC])), (2015-05-05 00:00:00+00:00, Equity(41 [ARCB])), (2015-05-05 00:00:00+00:00, Equity(52 [ABM])), (2015-05-05 00:00:00+00:00, Equity(53 [ABMD])), (2015-05-05 00:00:00+00:00, Equity(62 [ABT])), (2015-05-05 00:00:00+00:00, Equity(64 [ABX])), (2015-05-05 00:00:00+00:00, Equity(66 [AB])), (2015-05-05 00:00:00+00:00, Equity(67 [ADSK])), (2015-05-05 00:00:00+00:00, Equity(69 [ACAT])), (2015-05-05 00:00:00+00:00, Equity(70 [VBF])), (2015-05-05 00:00:00+00:00, Equity(76 [TAP])), (2015-05-05 00:00:00+00:00, Equity(84 [ACET])), (2015-05-05 00:00:00+00:00, Equity(86 [ACG])), (2015-05-05 00:00:00+00:00, Equity(88 [ACI])), (2015-05-05 00:00:00+00:00, Equity(100 [IEP])), (2015-05-05 00:00:00+00:00, Equity(106 [ACU])), (2015-05-05 00:00:00+00:00, Equity(110 [ACXM])), (2015-05-05 00:00:00+00:00, Equity(112 [ACY])), (2015-05-05 00:00:00+00:00, Equity(114 [ADBE])), (2015-05-05 00:00:00+00:00, Equity(117 [AEY])), (2015-05-05 00:00:00+00:00, Equity(122 [ADI])), (2015-05-05 00:00:00+00:00, Equity(128 [ADM])), (2015-05-05 00:00:00+00:00, Equity(134 [SXCL])), (2015-05-05 00:00:00+00:00, Equity(149 [ADX])), (2015-05-05 00:00:00+00:00, Equity(153 [AE])), (2015-05-05 00:00:00+00:00, Equity(154 [AEM])), (2015-05-05 00:00:00+00:00, Equity(157 [AEG])), (2015-05-05 00:00:00+00:00, Equity(161 [AEP])), (2015-05-05 00:00:00+00:00, Equity(162 [AEPI])), (2015-05-05 00:00:00+00:00, Equity(166 [AES])), (2015-05-05 00:00:00+00:00, Equity(168 [AET])), (2015-05-05 00:00:00+00:00, Equity(185 [AFL])), (2015-05-05 00:00:00+00:00, Equity(192 [ATAX])), (2015-05-05 00:00:00+00:00, Equity(197 [AGCO])), (2015-05-05 00:00:00+00:00, Equity(216 [HES])), (2015-05-05 00:00:00+00:00, Equity(225 [AHPI])), (2015-05-05 00:00:00+00:00, Equity(239 [AIG])), (2015-05-05 00:00:00+00:00, Equity(247 [AIN])), (2015-05-05 00:00:00+00:00, Equity(253 [AIR])), (2015-05-05 00:00:00+00:00, Equity(266 [AJG])), (2015-05-05 00:00:00+00:00, Equity(270 [AKRX])), (2015-05-05 00:00:00+00:00, Equity(273 [ALU])), (2015-05-05 00:00:00+00:00, Equity(283 [ALCO])), (2015-05-05 00:00:00+00:00, Equity(289 [MATX])), (2015-05-05 00:00:00+00:00, Equity(300 [ALK])), (2015-05-05 00:00:00+00:00, Equity(301 [ALKS])), (2015-05-05 00:00:00+00:00, Equity(311 [ALOG])), (2015-05-05 00:00:00+00:00, Equity(312 [ALOT])), (2015-05-05 00:00:00+00:00, Equity(328 [ALTR])), (2015-05-05 00:00:00+00:00, Equity(332 [ALX])), (2015-05-05 00:00:00+00:00, Equity(337 [AMAT])), (2015-05-05 00:00:00+00:00, Equity(351 [AMD])), (2015-05-05 00:00:00+00:00, Equity(353 [AME])), (2015-05-05 00:00:00+00:00, Equity(357 [TWX])), (2015-05-05 00:00:00+00:00, Equity(366 [AVD])), (2015-05-05 00:00:00+00:00, Equity(368 [AMGN])), (2015-05-05 00:00:00+00:00, Equity(371 [HWAY])), (2015-05-05 00:00:00+00:00, Equity(392 [AMS])), (2015-05-05 00:00:00+00:00, Equity(393 [AMSC])), (2015-05-05 00:00:00+00:00, Equity(397 [AMSW_A])), (2015-05-05 00:00:00+00:00, Equity(405 [AMWD])), (2015-05-05 00:00:00+00:00, Equity(410 [AN])), (2015-05-05 00:00:00+00:00, Equity(412 [ANAT])), (2015-05-05 00:00:00+00:00, Equity(430 [ANN])), (2015-05-05 00:00:00+00:00, Equity(438 [AON])), (2015-05-05 00:00:00+00:00, Equity(447 [AP])), (2015-05-05 00:00:00+00:00, Equity(448 [APA])), (2015-05-05 00:00:00+00:00, Equity(450 [CLFD])), (2015-05-05 00:00:00+00:00, Equity(451 [APB])), (2015-05-05 00:00:00+00:00, Equity(455 [APC])), (2015-05-05 00:00:00+00:00, Equity(460 [APD])), (2015-05-05 00:00:00+00:00, Equity(465 [APH])), (2015-05-05 00:00:00+00:00, Equity(468 [API])), (2015-05-05 00:00:00+00:00, Equity(474 [APOG])), (2015-05-05 00:00:00+00:00, Equity(484 [ATU])), (2015-05-05 00:00:00+00:00, Equity(508 [AIRM])), (2015-05-05 00:00:00+00:00, Equity(510 [ARG])), (2015-05-05 00:00:00+00:00, Equity(523 [AAN])), (2015-05-05 00:00:00+00:00, Equity(526 [AROW])), (2015-05-05 00:00:00+00:00, Equity(535 [ARTW])), (2015-05-05 00:00:00+00:00, Equity(538 [ARW])), (2015-05-05 00:00:00+00:00, Equity(542 [ASA])), (2015-05-05 00:00:00+00:00, Equity(547 [ASB])), (2015-05-05 00:00:00+00:00, Equity(548 [ASBI])), (2015-05-05 00:00:00+00:00, Equity(553 [ASEI])), (2015-05-05 00:00:00+00:00, Equity(557 [ASGN])), (2015-05-05 00:00:00+00:00, Equity(559 [ASH])), (2015-05-05 00:00:00+00:00, Equity(567 [ASMI])), (2015-05-05 00:00:00+00:00, Equity(576 [ASR])), (2015-05-05 00:00:00+00:00, Equity(579 [ASTE])), (2015-05-05 00:00:00+00:00, Equity(595 [GAS])), (2015-05-05 00:00:00+00:00, Equity(600 [OA])), (2015-05-05 00:00:00+00:00, Equity(607 [ATML])), (2015-05-05 00:00:00+00:00, Equity(610 [ATNI])), (2015-05-05 00:00:00+00:00, Equity(612 [ATO])), ...]\n",
316 "\n",
317 "[8236 rows x 0 columns]"
318 ]
319 },
320 "execution_count": 6,
321 "metadata": {},
322 "output_type": "execute_result"
323 }
324 ],
325 "source": [
326 "result"
327 ]
328 },
329 {
330 "cell_type": "markdown",
331 "metadata": {},
332 "source": [
333 "The output of an empty pipeline is a DataFrame with no columns. In this example, our pipeline has an index made up of all 8000+ securities (truncated in the display) for May 5th, 2015, but doesn't have any columns.\n",
334 "\n",
335 "In the following lessons, we'll take a look at how to add columns to our pipeline output, and how to filter down to a subset of securities."
336 ]
337 }
338 ],
339 "metadata": {
340 "kernelspec": {
341 "display_name": "Python 2",
342 "language": "python",
343 "name": "python2"
344 },
345 "language_info": {
346 "codemirror_mode": {
347 "name": "ipython",
348 "version": 2
349 },
350 "file_extension": ".py",
351 "mimetype": "text/x-python",
352 "name": "python",
353 "nbconvert_exporter": "python",
354 "pygments_lexer": "ipython2",
355 "version": "2.7.11"
356 }
357 },
358 "nbformat": 4,
359 "nbformat_minor": 0
360 }