Merge Volume 3 Article 5 2019 Reliability of Clustering in Forecasting Stock Prices of Companies Traded on the Stock Exchanges Shristi Dhakal Follow this and additional works at: https://athenacommons.muw.edu/merge Part of the E-Commerce Commons, and the Portfolio and Security Analysis Commons Recommended Citation Dhakal, Shristi. "Reliability of Clustering in Forecasting Stock Prices of Companies Traded on the Stock Exchanges." Merge, vol. 3, 2019, pp. 1-20. This Article is brought to you for free and open access by ATHENA COMMONS. It has been accepted for inclusion in Merge by an authorized editor of ATHENA COMMONS. For more information, please contact [email protected]. Running head: CAN CLUSTERING BE USED TO BEAT THE MARKET? 1 Reliability of Clustering in Forecasting Stock Prices of Companies Traded on the Stock Exchanges Shristi Dhakal Mississippi University for Women CAN CLUSTERING BE USED TO BEAT THE MARKET? 2 Abstract Whether stock prices can be accurately forecasted or not is usually associated with whether markets are efficient or not. The idea of market efficiency suggested by the Efficient Market Hypothesis has been debated among financial professionals for a long time, especially due to the occurrence of financial bubbles in the past. Some argue that stock prices prediction is no different than the results of “a series of tosses of a coin, rolls of a die, or spins of a roulette wheel,” while others argue that stock prices are affected by past patterns, which can be used to forecast future prices [11]. Although a concrete answer has yet to be found on the behavior of the stock market, researchers have continued exploring the topic and have established various quantitative models for forecasting, one of which is clustering. This paper evaluates the application of the clustering method of stock forecasting by analyzing the financial statements of technology companies over a period of four years. Keywords: stock prediction, clustering CAN CLUSTERING BE USED TO BEAT THE MARKET? 3 1. Introduction Stock market prediction is a complex topic. It requires analyzing a lot of past and present data. With markets fluctuating every now and then, the amount of information is only increasing day by day. So, there are more chances that predictions will fail if our understanding of the huge amount of available data is not aided by analyses using accurate mathematical methods. Among many prediction methods that are mathematical, clustering is one of them, and it is used to identify patterns and behaviors using correlations as measure of similarity. The application of clustering is interpreted differently by various studies depending on the data and outcomes of the study, but it is believed to have yielded consistent results in most cases when applied with stocks. 1.1 Background 1.1.1 Efficient Market Hypothesis The Efficient Market Hypothesis (EMH) is an investment theory which states that stock markets are efficient and reflect all information about individual stocks and the stock market in general [2, 5]. The theory is related to the idea of a “random walk,” which suggests that prices move randomly [5]. The reasoning behind the random walk idea is that if information about a company flows in the market without hindrances and it quickly gets reflected in the stock prices, then tomorrow’s prices will reflect only tomorrow’s news and will not depend on the prices changes today [5]. Since tomorrow’s news is unpredictable, stock price movements must also be unpredictable and random [5]. Hence, prices fully reflect all known information, and therefore, investors, neither through technical analysis, which is the study of past price movements to predict future price movements, nor fundamental analysis, which is the analysis of financial CAN CLUSTERING BE USED TO BEAT THE MARKET? 4 information such as asset values and profit margins, can obtain future information that helps to pick ‘winning’ stocks and achieve returns that are higher than average without taking comparable risk [5]. The rational expectations set by the EMH was believed by many around the 1970s until behavioral economists started questioning it due to the occurrence of anomalies that did not quite fit with the idea of the efficient markets theory [10]. Economists who analyzed the psychological and behavioral aspects of finance argued that the EMH did not explain market irrationalities such as the crash of 1987 and the internet bubble of the 1990s enough. Some of these economists, by studying the psychological elements of stock prices, even claimed that future prices are somewhat predictable based on the past patterns and some fundamental valuation metrics, which may allow investors to select stocks that can provide risk-adjusted returns [5]. These two opposite arguments have sparked debates on market efficiency and its predictability for many years. Today, the two components of the EMH- 1) stock prices quickly reflect all relevant information, and 2) if part 1 is true, investors cannot select ‘winning’ stocks and beat the market, are argued separately. Some experts agree with the first part, whereas some disagree, but even those who disagree with the first part would agree with the second part because investing in stocks that perform better than the market average appears to be difficult. 1.1.2 Forecasting in Stock Market Regardless of the controversies in the stock market theories, forecasting is considered important by many in future decision making. Past patterns have been useful in detecting future behaviors. Investors such as Warren Buffet have forecasted prices that have eventually turned out to be true and have been outperforming the markets for many years. Although the success of CAN CLUSTERING BE USED TO BEAT THE MARKET? 5 these investors is debated as being chance events by proponents of the EMH, it certainly does not appeal in the same way to the opponents, which is why complex mathematical models like clustering have been developed. 1.2 Cluster Analysis Cluster analysis is the method of grouping objects into similar and dissimilar groups so that objects in one group are similar to one another and are different from the objects in other groups [6]. There are typically two methods of clustering: hierarchical and partitional clustering. This paper uses hierarchical clustering, which is based on the relationship between the two nearest clusters and, unlike other clustering, the data sets “are not partitioned into a particular cluster in a single step.” Rather, they go through a series of partitions, from “a single cluster containing all objects to N clusters each containing a single object” [6]. 2. Related Work Patterns drawn from clustering have been used in the past to effectively forecast stock prices. Babu et al. (2012) have suggested that clustering can be helpful for investors in finding the interrelationship between the long term underlying trends of stock prices that is normally hidden by the short-term fluctuations. Lee et al. (2010) have analyzed the role of clustering in investigating the relationship between financial reports and short-term stock prices and have recommended better techniques for better predictions. Similarly, several other studies have suggested that clustering is valuable in time series forecasting and has produced good results. CAN CLUSTERING BE USED TO BEAT THE MARKET? 6 3. Hypothesis The objective of this paper is to see if clustering can be used to beat the market. So, based on the objective, below are the null and the alternative hypotheses: Null Hypothesis: There is no difference in the amount of profit that can be earned using clustering. Alternative Hypothesis: There is a difference in the amount of profit that can be earned using clustering. 4. Methodology The primary method used for this project is data collection and analysis. Information from the financial statements are used to predict the profit margin. 4.1. Financial Statements Financial statements contain information such as the assets, liabilities, equity, cash inflows and outflows, revenues, income/loss, and profits of a company. It comprises three parts: Balance Sheet, Income Statement, and Cash Flow Statement. The Balance Sheet provides an overview of a company’s assets, liabilities, and shareholder’s equity; the Income Statement gives an overview of revenues, expenses, net income, and earnings per share; and the Cash Flow Statement combines both and gives an overall picture of the operating, investing, and financing activities of a company. Such information is considered useful when determining the worth of a company for investment. Studies suggest that financial statements can identify fundamentals that are not visible in prices, and there exists a relation between stock prices and earnings of a company, which could be used for making informed investments decisions. [6]. CAN CLUSTERING BE USED TO BEAT THE MARKET? 7 The information from financial statements can be interpreted differently as it can infer multiple things. In this paper, the information is used as a similarity measure to group companies into clusters. Grouping of companies based on financial performance gives investors a general idea of which companies perform better financially, and this could be a valuable information while selecting companies to invest in. 5. Discussion of data In a perfect world, 60 companies are required to get sufficient results so that two groups can be formed with 30 companies in each group. Also, it is preferred that the companies be in the same industry to make relevant comparisons. Hence, 60 top performing technology companies traded on the New York Stock Exchange (NYSE) and the National Association of Securities Dealers Automated Quotations (NASDAQ) were randomly selected and following information from their financial statements from 2014 to 2017 was recorded on a spreadsheet: i. From balance sheet: cash, current assets, fixed assets, current liabilities, long term debt, and equity. ii. From income statement: Earnings Before Interest and Taxes (EBIT) and profit margin.