Is Big Data Really that Big a Deal?

Graphic Designed by: Rose Lee

The BRB Bottomline: Over the past twenty years, there has been a tremendous increase in the utilizable and interpretable data available to companies. This has led to an arms race amongst investors to gain an informational edge. Business managers have started using collection software and data analysis techniques to look beyond the purely-financial data and gain deeper insights about their customers and business. While many investors are aware of using social media and web scraping to gauge customers’ views on businesses and their products, hedge funds and asset managers have also started using alternative data such as credit card receipts and geolocation because of  its potential to achieving higher alpha. So what is alternative data and does it really provide investors with an advantage when making investing decisions?

Alternative data refers to data that is collected from non-traditional sources, which, when analysed, can provide insights that would not be available through conventional financial data sources. However, since the conventional non-financial data used to make investment decisions varies from industry to industry, the definition of what exactly constitutes alternative data also varies from one industry to another. Wall Street banks seek to capitalize on such data sets. For example, Bank of America Merrill Lynch is using Hadoop, an open source platform that allows it to take advantage of distributed processing (computer systems that use more than one computer to run an application) to manage petabytes of data for regulatory compliance and advanced analytics. In order to gain a better understanding of what constitutes alternative data and how it is used to make investment decisions, we’ll take a look at some of the data that investment managers are purchasing.

Unconventional Datasets already used by Fundhouses

One of the characteristic alternative datasets is data collected from omnipresent Bluetooth and Wi-Fi enabled devices such as mobile phones and watches, which are then analysed to understand not only industry but also economic trends. These devices present a treasure trove of multi-layered data that can be used to not only better understand customer preferences but also gauge variations over time and in different places. Using the signals emitted by these devices, companies can figure out the number of FitBit and Netflix users in a metropolitan statistical area (MSA), how long customers spend on these devices and apps, when they use them, and where the highest concentration of users are located. Customer location can also be used to understand, in real-time, how many or few customers are present at a particular location and which retail stores, supermarkets and restaurants these customers visit. Furthermore, consumers use mobile phones and similar devices to provide customer feedback and to express their views on the various products they use. This data can then be used to conduct sentiment analysis or predict future trends about consumption of the products or brands being tracked.

Source: Deloitte

Beyond tracking signals emitted from consumers’ devices, hedge funds also purchase credit card receipts, satellite imaging, and employment data. Credit card receipts are analysed to understand the proportion of disposable income spent on a specific product. In addition to investment funds, consumer credit rating agencies, such as Equifax and Experian, use this data to provide a more holistic credit score. Satellites and drones have also been used to track parking lot fill rates as an input to predict quarterly sales before quarterly reports are released—proven to be effective and advantageous for people involved in short-selling. For example, according to CNN news, one hedge bought a fleet of drones to check the lumber inventory of one the lumber companies it had invested in to confirm rumors about a buildup of lumber. The firm sold shares at a price where the lumber stockpile had yet to be released into the market. When the excess inventory was reported, lumber prices fell along with the prospects of the lumber company’s sales, and the stock price plummeted. Beyond this, even employment data, such as posted job positions are tracked to predict strategic changes. For example, if a silicon chip company posts a position looking for someone with a pharmaceutical background, it could indicate that the company is expanding into medicines. 

So, what’s the hype all about? Is alternative data actually helpful?

Heavyweights like J.P. Morgan Chase and Goldman Sachs have started incorporating alternative datasets into their businesses. In February 2019, Bloomberg announced that it would sell alternative datasets through its proprietary terminal, thus providing alternative data with an institutional backing and legitimacy. 

Nevertheless, does access to this data really provide investors with an edge and a higher chance at generating alpha? There are conflicting views on this. For the efficient market hypothesis to pan out, information asymmetry must not exist. In other words, buyers and sellers in the market need to have equal access to information before making investment decisions. However, unequal access to alternative data, especially because such data is expensive and only available to corporate investors, has led to the creation of an information asymmetry wherein some participants have the ability to make more informed decisions than others.

Source – Arabesque Insights:a quantitative investment based company

According to EY’s 2017 Global Hedge Fund and Investor Survey, there has been a major increase in the percentage of hedge funds incorporating alternative data into their investment processes, indicating that the managers believe that this data is worth the cost. According to a 2018 Morningstar report, “Far from creating a level playing field, where more readily available information simply leads to greater market efficiency, the impact of the information revolution is the opposite: it is creating hard-to access ‘realms’ for long-term alpha generation for those players with the scale and resources to take advantage of it.” Importantly, the majority of hedge funds that use alternative data have quantitative investors for fund managers(EY). The increasing traction of alternative data amongst quantitative investors is also reflected by the growing belief that “Big Data is the New Quant Investing,” as mentioned in this Forbes article. According to a Greenwich Associates report, nearly 30% of quantitative funds attribute at least 20% of their alpha to alternative data. Furthermore, 42% of all asset managers believe that the alpha they achieve by using alternative data lasts for at least four years. Thus, information asymmetry seems to be the basis of the temporal edge that investors gain through alternative data. 

Conversely, the majority of fundamental investors continue to believe that while alternative data might help investors make more informed decisions and validate their ideas, the cost isn’t worth the benefit. There is always a chance of an exogenous shock, beyond the company’s control. The data is not able to predict such events, which could negate all the gains of spending on this data and result in misled decisions if fundamental data wasn’t used in making the investment decision. Moreover, the value of this data depreciates rapidly, so strategies built on it can get expensive. According to a Bloomberg article discussing alternative data, Warren Buffet believes that the S&P 500 index will do better than quant-based hedge funds, including those which have access to alternative data. Furthermore, the information asymmetry that provides the investor with an edge, seems to be deteriorating because widely used e-brokers such as TD Ameritrade have begun expanding their own alternative data offerings, allowing some people access to data, such as from the social-sentiment app LikeFolio, in their trading accounts.

Who could benefit the most from alternative data? Should every investor purchase this data?

While alternative data has the potential to help investors make more informed decisions, it is important to realize that the effectiveness of this data depends on who uses it and for what purpose. To quote John Chisholm, Co-CEO and Co-CIO at Acadian Asset Management

“It’s important to match the data set and resulting signal to a manager’s style—­specifically, their breadth and time­ horizon/turnover. An event-driven strategy using retail sales receipts data would typically cover a narrow universe of retailers and consumer products companies. The resulting signal would be low-breadth and likely work over time frames of days or weeks. That wouldn’t be suitable for a diversified, lower-turnover investment process. Brand sentiment data could produce a slower-moving set of signals useful for lower-turnover managers but, again, would typically be available for a limited universe of companies.” (Bloomberg)

Therefore, usage of such data helps consulting firms and investment bankers, whose primary focus is to gain an edge and beat others to current information to provide timely advice or complete deals. Hedge funds have already spent millions to ensure that the time between placing a trade and its execution is reduced by as much as milliseconds. Thus, it would be safe to say that they would benefit from the temporal edge that alternative data creates by providing another layer of current data. 

However, long-term investors invest in companies after considering multi-year trends and the overall strength and solvency of the business. Beyond financial data, they look at factors such as the management team and its incentive structure to gauge whether the company can survive rough quarters and generate an alpha in the long run. Even data such as market sentiments and thus social media would fluctuate in the short run and can only be employed to track sentiment trends over multiple quarters. Thus, for long-term and fundamental investors, alternative data should only be used if it provides strategic insights or could help the investor decide the sell time. 

Take Home Points

Alternative data has been gaining a lot of traction lately. While it might help short-term investors and traders achieve an alpha by creating an information asymmetry, it should only be used to enhance analysis or validate conclusions reached through traditional financial and business analysis. To quote Ashby Monk, a Stanford University academic who studied how funds use data, “ the best way for most investors to use alternative data is as a risk-management tool rather than as an idea generator”. If the data is chosen in the right way—ensuring exclusivity in terms of access, minimal noise, and a connection to financial data in some manner—it provides a temporal edge. Long-term investors can also benefit from alternative data if they use data such as job listings, to gain strategic insights. Thus, while not enough back testing has been done, in a world that is increasingly characterized by its fast pace and constant disruption, alternative data has the potential to become a major input in the decision-making toolkit, and one should stay on the lookout for how it could potentially help them generate an alpha.


  1. Hello! I simply wish to give you a huge thumbs up for your great info you’ve got right here on this post. I’ll be returning to your site for more soon.

Leave a Reply

Your email address will not be published. Required fields are marked *