Gathering my own data

I have recently started refreshing my knowledge in Perl and have, in the process, re-realized the amazing scripting power of linux. There's a whole lot that can be done with file structure and file contents than what is remotely possible in Windows. Keeping that as a separate topic I'll first dive into using Perl to connect with data sources to download stock quotes and store them in a mysql database that can be used for further analysis.

Creating a MySql Database:
I found a very useful resource on the web to help understand MySql especially for those who have a faint memory from using it in the past: http://www.yolinux.com/TUTORIALS/LinuxTutorialMySQL.html
This tutorial gives a very elaborate overview on how to obtain MySQL and then how to create and modify user accounts and databases.
After getting acquainted with the basics this tutorial will help to create the actual database and provides the syntax on how to insert data using Perl into the just created database: http://scriptingmysql.wordpress.com/2011/08/05/inserting-data-into-mysql-with-perl/

Obtaining Stock Quotes:
I looked for many different alternatives to obtain stock quotes using Perl and I found the Finance::QuoteHist package to be quite relevant to this purpose - http://search.cpan.org/~msisk/Finance-QuoteHist-1.19/lib/Finance/QuoteHist.pm

Final Code snippet:
With all the above information I wrote the following code to obtain the ticker symbols from a local file and then fetch the stock quotes and store them into a local mysql database:


#!/usr/bin/perl

use Finance::QuoteHist;
use DBI;

# Obtain the ticker symbols from file
my $tic_file = '<path of file>/tics2.txt';
 open (fh, "< $tic_file"); #Note the syntax
 my @lines = <fh>;
 close(fh);
#my @tics = @lines[0..99];
#print $#lines, $#tics;

#print "@lines[1..100]";
$dbh = DBI->connect('dbi:mysql:PerlTest','root','<my password>') or die "Connection Error: $DBI::errstr\n";
$q = Finance::QuoteHist->new
     (
      symbols    => \@lines,
      start_date => '03/14/2013', # or '1 year ago', see Date::Manip
      end_date   => 'today',
     );

  # Quotes
  foreach $row ($q->quotes())
  {
    print "@$row \n";   #double quotes displays properly
    ($symbol, $date, $open, $high, $low, $close, $volume) = @$row;


    $sql = "insert into SampleQuotes (ticker, date, closing) values ('$symbol','$date','$open') ";
    $sth = $dbh->prepare($sql);
    $sth->execute() or die "SQL Error: $DBI::errstr\n";
  }

There is a whole lot that needs to be done in this script around Error Handling, vis-a-vis connection issues, duplication issues, missing data, etc and that is going to be the next topic.

Statistical Arbitrage!

With my experience in statistical analysis and after further gaining information on equity markets I was curious to explore the possibility of generating alpha by combining the two skills. I got an opportunity where I worked on a similar idea and then I extended it on the data I had downloaded from yahoo.
In this strategy I try to find a statistical equivalent of pairs trading and try to extract inefficiencies in price movements.

Abstract

The objective of this analysis is to obtain an alpha that is based on statistically exploring ineffi ciencies in stock prices. The strategy involves decomposing stock prices in each industry into principal components that explain the most variance and then regress the stock prices on those components to obtain the stock's  dependence on them. Next the components are forecasted using GARCH model and hence the forecasted evolution of the stocks is also obtained based on the regression results. Based on these forecasts I will create a long-short neutral arbitrage strategy with the aim of achieving high risk adjusted returns.

Statistical Arbitrage on US Equity
https://github.com/kunalrajani/statistical-arbitrage

Disclaimer: Please read
Please note that this is my independent work where I have used data from yahoo finance to explore statistical concepts from a course I took in statistics. It may inadvertently have an overlap with a work that somebody else has already done and I have no intentions of replicating it. I would be glad to know of any such clash and post a clarification on this post.
I am open to having my work being redistributed or used but only after due credit and a reference has been made. Feel free to contact me to avoid any misunderstandings or if you need more details from this paper. 

*The analysis is still underway and the conclusions are under review

Understanding Factor models and Cone programming

This was a very interesting project we undertook. Not because its result was any enormous value addition but because it helped in grasping concepts and procedures critical to function in practice. It helped in going through the rigorous process of data cleaning and implementing factor models and checking their significance to predict stock returns. Further on we explored optimization using cone programming, which is a special case of interior point methods, to maximize the sharpe ratio and obtain a market beating portfolio.

Here is the Abstract:

The goal of our project is to utilize factor models to explain returns and optimize the Sharpe ratio to create a portfolio that outperforms the S&P 500. After re fining our data we have a universe of 335 stock in which we can invest. We re-balance our portfolio quarterly and incorporate factor models and Sharpe ratio optimization through cone programming to form the portfolio. The rest of the paper is organized as follows: Section 1 is a short introduction of our paper, Section 2 gives a brief idea of the data available and what kind of choices we made to reach the final universe of stocks, Section 3 gives an idea of the general methodology used in the paper, Section 4 describes the results that we have reached, Section 5 presents the significance test we performed, Section 6 presents the results of different sensitivity analysis and Section 7 summarizes the project and gives suggestions for further research.

Investment Allocation using Factor models and Cone programming optimization

Let's get on the street - Part II

Furthering my experimentation with the real world data, I delved into the minds and psychology of investors. I had taken a course on Behavioral Finance that explains certain irrational phenomena that governs the nature of humans when then invest. This nature has certain shortcomings that are overcome by what has emerged to be known as systematic investing where a computer algorithm makes the decisions rather than a human who can be misguided by emotions.

The following Abstract gives a brief overview of the analysis:

There are many instances when a stock price experiences a sudden jump or a decline and, besides the financial crisis, it has to do with a fundamental change in the business of the stock. Eg. the consumer reaction to netflix's fee structure or a revelation of the revenue reporting mechanism of Groupon or the billions of dollars of trading losses by a bank or the passing of a law that gives huge tax incentives to renewable energy. These events cause almost an immediate reaction by the market reflected by the stock price changes and then as these events are studied in depth and their true impact is understood, the stocks migrate towards their true valuation. The relationship between a surge or a drop and the later migration is something that interacts with human nature and, if understood well, can be leveraged to generate investment idea. In this paper I shall explore such drastic movements and try to understand what their impact is on the stock prices. This is more complicated than it seems because of a change in investors' perceptions after the financial crisis and because of different nature of information dissemination for firms in different segments.

Investing on Behavioral bias

Disclaimer: Please read
Please note that this is my independent work where I have used data from yahoo finance to explore time series concepts from a course I took in statistics. It may inadvertently have an overlap with a work that somebody else has already done and I have no intentions of replicating it. I would be glad to know of any such clash and post a clarification on this post.
I am open to having my work being redistributed or used but only after due credit and a reference has been made. Feel free to contact me to avoid any misunderstandings or if you need more details from this paper.

*The analysis is still underway and the conclusions are under review

Let's get on the street - Part I

Now that I'm in my third semester of my program I've started to apply a few concepts that I learnt in my earlier courses to see if they actually make sense in the real world. It's hard to digest the fact that even doing an "a+b" in the real world is not that easy - acquiring data, ensuring that it is exactly what you want, cleaning it, checking if it makes sense and clipping out those that don't and still be left with enough meaningful numbers  involve a series of massive challenges that only a practitioner would comprehend. And that is exactly why I ventured on this endeavor, to map classroom coaching to real world implementation.

I would have actually wanted to branch off from this blog since it has mostly been about life at Cornell but for now I think I would include all my analysis here until I figure out the best way to split the two different topics. In this post I attempt to implement ARMA time series model on large cap stock returns in an attempt to predict their future returns. It's a first attempt but nevertheless gives an excellent idea about the issues I described above and if the time series concepts work by a brute force methodology.

ARMA model for large cap stocks returns

Disclaimer: Please read
Please note that this is my independent work where I have used data from yahoo finance to explore time series concepts from a course I took in statistics. It may inadvertently have an overlap with a work that somebody else has already done and I have no intentions of replicating it. I would be glad to know of any such clash and post a clarification on this post.
I am open to having my work being redistributed or used but only after due credit and a reference has been made. Feel free to contact me to avoid any misunderstandings or if you need more details from this paper.