Thursday, May 15, 2014

My first hands-on experience with Big Data

Big Data- Is it just another buzzword in the corporate world or truly the next big thing in the industry worldwide? The question did bug me for a long time that led me to browse through the internet reading various newsletters, journals and business articles on the topic. So, when we were supposed to chose a topic for our final year  project in MBA course, namely the Applied Management Research Project my immediate choice was this. However, implementing big data analytics without the knowledge of advanced statistical methods like support vector machine and  tools like Hadoop was a big challenge. So after studying various research papers I decided to go for text analytics, a  subset of big data and selected my favorite platform R for the same. The topic of the project is: Text analytics- An application on Indian stock markets. Objective was to check if top news of the day has any significant effect on the returns of Indian Stock market. The most widely followed composite index Nifty is the proxy for Indian Stock Market in this project and text analytics algorithm is used on the top news article(s) of the day to predict the sentiment scores.

Special thanks to my friend Chayan Roy, an expert in R programming for guiding me in coding the text analytics algorithm in R and my guide professor A.K.Misra for his valuable inputs on training the algorithm and interpreting results.

Following is the link to the project report and presentation.
 Project Report