Data Aggregation and Management for Asset Manager

Solution

SG Analytics’ team started the data collection and consolidation process utilizing the client’s internal data, the Sysomos API for Twitter data, a custom web scraping engine that SG Analytics developed in-house for data acquisition from finance specific websites, and various publications to start storing, analyzing, and processing of different data types such as text, images, log files, JSON, XML, CSV, RDBMS, clickstream, and market data.

In the next step, SG Analytics used social media along with predictive models to understand the addition of a social sentiment as an attribute to the forecasting model. A joint team of SG Analytics’ data scientists and quantitative analysts tested how well sentiment information extracted from Twitter could be used to predict future shifts in stock prices.

The team applied predictive analytics models, such as simple regression, sentiment analytics, and selforganizing fuzzy neural networks (SOFNN) to estimate the movement of stock market prices vs. the volume as well as the sentiment of news articles and tweets. These were used to incorporate real-time industry data along with historical data to use Twitter and Stock twits data to predict public sentiment and use the predicted sentiment and previous day’s stock price values to predict the stock market movements.

The team created industry-specific lexicons to consider context including features such as good, bad, sarcasm, various emotions, etc.

Tools Used

• Python
• Sysomos
• Excel
• SQL

Value Delivered

• SG Analytics’ models considered multiple aspects of security mispricing in the forecasting process and enabled the client to leverage signals from social media to pre-empt movement in stock prices across sectors and geographies.
• The solution dynamically updated factors to model real time-market conditions and returned expectation, thus enabling the client to take fast investment/divestment decisions.