The logic of price discovery

An article considering the problem of the volatility of prices in financial markets from the perspective of the development of scientific knowledge, where expectations of an uncertain future can be tested and falsified through a trial and error process.

David Harrison, DAC Beachcroft LLP Speedread

Why, in the light of recent turbulence, should financial prices be more erratic and volatile than real economy prices?

The testing of expectations of the future about normal goods and services in the economy can take place through the trial and error process of the price mechanism. Competition policy can address distortions in this mechanism. But for capital assets the position is more complicated: the important factor is long-term yield, not fluctuating prices arising from the revaluations of previous investments in liquid investment markets. A more scientific and falsifiable approach in finance and investment, which competition policy might encourage, would be a shift away from what Keynes described as forecasting the psychology of the market, towards forecasting the prospective yields of assets over their whole life.

This article presents a theoretical discussion of the issues, and concludes with consideration of possible practical implications.

Introduction

Competition authorities around the developed world are still grappling with the after-effects of the financial crisis. New and unexpected forms of price manipulation have come to light in finance, on a scale "almost beyond comprehension" (in the words of John Plender of the Financial Times, in "Capitalism: Money, Morals and Markets" (2015, at page 303)). In the UK, the Bank of England's 2015 Fair and Effective Markets Review produced a range of recommendations that it is hoped will restore trust in the wholesale fixed income, currency and commodity markets, in the wake of various high profile abuses.

At the same time, persistently low levels of company investment have been linked to - termism in financial markets (by economist Andrew Smithers, in the book "The Road to Recovery" (2013)), and the way in which investors employ fund managers (see The Economist, "A new contract for growth", August 2015). In the US, Hillary Clinton, speaking in 2015 of the dangers of "quarterly capitalism", has echoed concerns, heard increasingly on both sides of the Atlantic, about short-termism.

As recent turbulence illustrates, prices in financial markets often appear less reliable and more erratic and volatile than those in real economy markets. Why should this be? One possibility is that they are controlled by fewer participants, with greater market power. Another is that they are intrinsically more subjective, and reflect expectations of the future more than current economic value.

This article considers the problem from the perspective of the development of scientific knowledge, where expectations of an uncertain future can be tested and falsified through a trial and error process. Might a similar approach help finance and investment to be put on a more stable and secure footing? The logic of scientific discovery

In the book "The Logic of Scientific Discovery" (1959), published originally in Vienna in 1934 as "Logik der Forschung", and in his subsequent works, the philosopher of science Karl Popper held that induction by repetition is not the basis of scientific knowledge. Rather than proceeding from observations of singular events to universal statements, or theories, Popper reversed matters: science proceeds by putting forward theories that can be tested by observation, and that can be falsified.

In this way, as Popper went on to claim in "Objective Knowledge: an Evolutionary Approach" (1972), the problem of induction, or what he called 'Hume's problem', could be solved. In his "Treatise of Human Nature" (1739), David Hume had pointed out that there is no logical justification in reasoning from repeated instances of which we have experience to other instances of which we have no experience. It is only by custom or habit, or for psychological reasons, that we are conditioned by repetitions to expect that instances of which we have no experience will conform to those of which we have experience.

Popper argued that while Hume was right to say that there was no such thing as induction by repetition in logic, this is an erroneous way of viewing the growth of scientific knowledge. Science proceeds by putting forward theories that may never actually be proved by observation, or testing, but which can, on the other hand, be disproved, or falsified. The process of testing, disproving and falsification of theories is a highly rational process, involving critical discussion, and returns reason to its rightful place in human understanding.

Popper, therefore, made an important demarcation in "The Logic of Scientific Discovery" between science and pseudo-science (or metaphysics). It is of the essence of scientific knowledge that it can be tested, falsified or disproved in some way. If an idea, or a hypothesis, cannot be tested or falsified at all it cannot qualify as a scientific theory. It is then a conjecture, or a myth, which might have some social value or explanatory power, but is not knowledge in the fullest scientific sense. Conjectures and hypotheses need to be put into a form which allows them to be tested against verifiable and repeatable observation if they are to qualify as scientific theories.

From this starting point, Popper went on to create an evolutionary and biological explanation of the growth of knowledge. All living organisms have in-built expectations of basic regularities, or patterns (such as the change from day to night) and it is the testing of these expectations against events which determines how successful organisms are. Scientific knowledge is the most communicable and developed form of knowledge there is, but ‘all life is problem solving’ (in the title of one of Popper’s books). The difference between Einstein and the amoeba is just one step: both try to solve problems, but, unlike the amoeba, Einstein can create theories that live and die instead of him. The testing of expectations of regularities by a process of trial and error happens all the time, and is a function of all organic life.

Uncertainty, expectations and prices

If we rise up the evolutionary scale from the amoeba to economic life, the existence of uncertainty of the future was described by economist Frank Knight in "Risk, Uncertainty and Profit" (1921) as itself basic to the market economy. Genuine uncertainty of an unknown future (to be distinguished from calculable risk) creates economic profits which are exploited by entrepreneurs. As Knight put it (at Section III.XI.8):

"It will be observed that the main uncertainty which affects the entrepreneur is that connected with the sale price of his product. His position in the price system is typically that of a purchaser of productive services at present prices to convert into finished goods for sale at the prices prevailing when the operation is finished. There is no uncertainty as to the prices of the things he buys. He bears the technological uncertainty as to the amount of physical product he will secure, but the probable error in calculations of this sort is generally not large; the gamble is in the price factor in relation to the product."

The price mechanism attenuates this uncertainty, so there is, for example, as Knight pointed out, no uncertainty as to the prices of the things an entrepreneur buys. The link between the price mechanism in a market economy and the diffusion of knowledge in society was then developed further by Friedrich Hayek, in his 1945 essay "The Use of Knowledge in Society". Hayek described how the price system can act as a kind of machinery for registering change, or

"a system of telecommunications which enables individual producers to watch merely the movement of a few pointers, as an engineer might watch the hands of a few dials, in order to adjust their activities to changes of which they may never know more than is reflected in the price movement."

The knowledge in society which Hayek was referring to is not quite scientific knowledge, in the sense of scientific theory. It is, nonetheless, useful, practical, problem-solving knowledge, which enables economically productive processes to take place. In Popperian terms, the price mechanism allows expectations of an uncertain future to be checked and corrected, through a trial and error process.

It follows that distortions of the price mechanism adversely affect this knowledge-distribution process. Where monopoly power exists, and firms can set their own prices without regard to others in the market, or where there is no price mechanism at all, economically useful knowledge is not distributed properly through society. Competition law today prohibits firms from reducing uncertainty by illegitimate means, such as by co-ordinating their sales prices with one another, in cartels.

There is a close conceptual link between an open market economy, where free competition allows the price mechanism to promote the efficient allocation of resources, and the open society, as espoused by Popper. In both economic and political terms, the future remains 'open': it is always subject to uncertainty, but, as in science, knowledge can grow, improvements can be made and problems solved in the light of trial and error.

Capital assets

The position becomes more complicated when we turn from productive processes in the economy to capital assets. Investment in new capital assets may allow more production to take place at some point in the future, but that future is more remote, and subject to even greater uncertainty. Capital assets are more difficult to price than products (goods and services), because they are bought and sold infrequently (if at all). There is no ready Hayekian 'system of telecommunications' to transmit price signals around the economy.

In "The General Theory of Employment, Interest and Money" (1936), John Maynard Keynes described how capital assets provide a stream of income to those who own them, but views about the prospective yield of capital assets fluctuate as views about the long-term future fluctuate. Keynes made a distinction between short-term expectations (or what price a manufacturer can expect to get for his output when he starts the process of production) and long- term expectations (or what a manufacturer can hope to earn in the shape of future returns if he adds to his stock of capital equipment). The former is the same as the uncertainty of the entrepreneur, as described by Knight, above, which can be attenuated through the price mechanism. But the latter, which relates to the level of investment by a manufacturer, is more volatile and erratic, because it concerns a distant future which is more uncertain. Keynes made clear that when he was referring to uncertainty of knowledge he did mean genuinely uncertain, and was not distinguishing what is known for certain from what can be assigned a probability: "The sense in which I am using the term [of uncertain knowledge] is that in which the prospect of a European war is uncertain, or the price of copper and the rate of interest twenty years hence, or the obsolescence of a new invention, or the position of private wealth-owners in the social system in 1970. About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know."

So the price mechanism can act as an effective test or check of short-term economic expectations, but is absent, or less reliable, when it comes to long-term expectations, governing capital assets, and investment in capital assets, concerning a future about which we genuinely know little or nothing.

What, in practice, happens? This is the subject of the famous chapter 12 in "The General Theory of Employment, Interest and Money" ("The State of Long-Term Expectation"). In the face of uncertainty of a long-term future, revaluations on the stock market of previous investments (in the form of securities) act as a proxy price for future investment in new capital assets. There are several things wrong with this, as Keynes set out in detail, but a principal defect is that those expert professionals whose business it is to buy and sell securities are not usually competing to establish the best long-term forecasts of investments, but rather their short-term resale value based on a "conventional basis of " where the aim is "anticipating what average opinion expects the average opinion to be".

Keynes distinguished between two forms of investor behaviour: (or the activity of forecasting the psychology of the market) and enterprise (or the activity of forecasting the prospective yield of assets over their whole life). As the organisation of investment markets improves, the risk of the predominance of speculation over enterprise increases. Indeed, in the view of Keynes, it is the scarcely avoidable outcome of having successfully organised liquid investment markets. (In modern terminology the two types of investor behaviour equate to 'momentum', or trend-following, trading and 'fundamental value' investing, based on estimating the future cash flows from assets.)

If we now return to Popper, it is not easy to see how forecasting the psychology of the market might qualify as science. The market view (or "conventional basis of valuation") is not testable or falsifiable, but permanently shifting, representing the sum of current expectations of an uncertain future, transmitted between the closed group of investors revaluing existing securities, with feedback effects caused by each trying to anticipate the views of others.

Nor do the prices thus formed provide through trial and error useful practical knowledge (in the Hayekian sense) of the long-term value of capital assets; they are instead merely current expectations of the resale value of existing securities.

This is a defect of the 'efficient market hypothesis', as described in 1970 by Eugene Fama in "Efficient Capital Markets: A Review of Theory and Empirical Work". A capital market as described by Fama allocates ownership of the capital stock of the economy, and is said to be efficient where firms can make production-investment decisions, and investors can choose among the securities representing ownership of firms' activities, under the assumption that securities prices at any time fully reflect all available information. A price-setting process based on forecasting the psychology of the market may well produce unpredictable securities prices (making it difficult for individual investors to beat the market); but such prices are not useful signals for the production or investment decisions which firms themselves take.

Empirical evidence of the circularity of the price-setting process can be found at the time of crises. In his report "Investor Behavior in the October 1987 Stock Market Crash: Survey Evidence", Robert Shiller discovered that no news story or other recognisable event from outside the market was thought to be immediately responsible for the US stock market crash of 1987 (an unprecedented price drop of 22.6% in one day). Instead, feedback channels within the market, with investors reacting to one another, appeared to cause price declines to feed into further price declines. According to Shiller's survey of almost 1,000 US investors, most interpreted the crash as due to the psychology of other investors:

"Investors had expectations before the 1987 crash that something like a 1929 crash was a possibility, and comparisons with 1929 were an integral part of the phenomenon. It would be wrong to think that the crash could be understood without reference to the expectations engendered by this historical comparison. In a sense many people were playing out an event again that they knew well."

'Price discovery' in this kind of market is not revealing the future value of new capital assets, enabling firms to make new investment decisions. Nor is it revealing the future value of securities representing titles to existing capital assets. What is being 'discovered' is, instead, the current psychological state of the expectations of participants in the market - which may, of course, not be the same the following day.

By contrast, to go back to Popper once more, forecasts of the prospective yield of assets over their whole life are, at least in principle, testable and falsifiable. For any given asset, whether it is a physical capital asset, or a security producing a flow of cash, it is possible to test expectations of long-term value by means of waiting to see what the stream of income actually is. In a similar way, expectations of yields of particular classes of capital asset (such as buildings, infrastructure and public utilities) can be verified and checked.

Similar comments could be made about companies themselves: as John Bogle, the founder of the US mutual fund Vanguard, has pointed out, over the past century the real economic value of US companies (measured in terms of dividends and earnings growth) has tended to progress at a regular and steady pace, broadly in line with the growth in US GDP. These are measurable and reasonably predictable trends (see "America's Financial System - Powerful But Flawed", Lecture Phi Beta Kappa National Lecture Series, Temple University, 2010).

With all due allowance for difficulties of measurement, and comparability of data, it is, at least in principle, possible to test, or falsify, forecasts, conjectures and hypotheses about long-term yield. In this way expectations can be checked against one another, by a process of trial and error, and a body of useful knowledge created, perhaps even approaching the status of scientific knowledge.

Why might such a body of knowledge be useful?

The accumulation of savings

Compared to Keynes' day there has been a pronounced shift from investment by individuals to investment by institutions, and the pooling of savings into very large funds. The value of assets under professional management by institutional investors worldwide has increased from an estimated $11 trillion in 1990 to over $60 trillion in 2006; by 2014 it was estimated to have reached $87 trillion (equivalent to about one year's global GDP), and, by 2020, could top $100 trillion. These are vast sums, which need investing properly if the world economy is to thrive.

In "The General Theory", Keynes introduced the concept of 'liquidity-preference' as a response to uncertainty of the future. By this he meant a propensity to hold savings in a liquid form (cash or near cash) when faced with uncertainty of the future rate of interest, since if the future rate of interest is known it is more advantageous to purchase interest-bearing debt than hold non- interest-bearing cash as a store of wealth. Another reason for liquidity-preference is the existence of an organised market for dealing in debts, which allows scope for speculation (or "the object of securing profit from knowing better than the market what the future will bring forth"). According to a report of 2011 by the World Economic Forum, of the $65 trillion in privately managed assets reckoned to exist in 2009, only about $6.5 trillion was used for long-term investing (defined as investing with the expectation of holding an asset for an indefinite period of time). These figures suggest that a high proportion of today’s privately managed assets (possibly as much as 90%) are held only for the short term.

Holding a high proportion of savings in liquid or near liquid form creates practical difficulties, since, as research at the London School of Economics indicates, movements in flows of very large funds themselves have an impact on securities prices. Funds appear to become locked into behaviour patterns where they follow one another, buying and selling the stock of existing securities between themselves in short-term momentum trading. This may (or may not) be profitable in the short term for those managing funds, but is not in the long-term interests of the savers on whose behalf funds are managed. For funds with long-term objects and time horizons short-term behaviour erodes their value (see "The Future of Finance, the LSE Report" (2010): Chapter Three: 'Why are financial markets so inefficient and exploitative - and a suggested remedy' (Paul Woolley)).

The impact on securities prices may also be significant. As has been observed by the chief economist of the Bank of England (Andrew Haldane, Speech "On Microscopes and Telescopes", Bank of England, 2015): "One striking feature of the past few years has been the extremely high correlation among asset prices globally, in particular among advanced economies. This is true of both 'safe' rates of return on government assets and 'risky' rates of return on private assets. In either case, correlations are extremely high, hovering around 0.9. This begs a number of questions, both research and policy. What is the root cause of these correlations? One possibility is portfolio shifts by global asset managers, allocating their portfolio on an asset-by-asset basis."

Keynes considered liquidity to be a mixed blessing. On the one hand, the existence of organised liquid investment markets, whether for the purchase of securities in capital assets (the stock market), or for the purchase of debts (the bond market), helps provide an alternative to the unproductive hoarding of savings, and so helps stimulate investment. On the other hand, as such markets develop the scope for speculation increases, in itself making long-term views more difficult. He suggested the only radical cure for the crises of confidence afflicting modern economic life would be to allow an individual no choice between consuming his income and ordering the production of the specific capital asset which, although the evidence is precarious, impresses him as the most promising investment available (at page 161):

"It might be that, at times when he was more than usually assailed by doubts concerning the future, he would turn in his perplexity towards more consumption and less new investment. But that would avoid the disastrous, cumulative and far-reaching repercussions of its being open to him, when thus assailed by doubts, to spend his income neither on the one nor on the other."

The shift since Keynes' day from investing by individuals to investing by institutions has significantly changed matters. Since large pools of savings now exist, in the form of very large funds, there is little need to offer liquidity in investment markets to individuals as an alternative to hoarding cash. Individual savers generally have no choice: their savings are already allocated to professionally-managed pension funds, insurance funds or other types of pooled saving. Liquidity in investment markets affects these intermediaries, not the end savers, who have delegated control over savings to them. However, as we have seen, the greater the liquidity in a market the more scope there is for 'speculation' (forecasting the psychology of the market) to predominate over 'enterprise' (forecasting the prospective yield of assets over their whole life), with adverse implications for prices formed in such a market.

A body of useful knowledge about long-term yield might, therefore, serve to counteract short- termism and liquidity-preference in institutional investment, by providing a reasonable basis for allocating savings over the long term. In this way, the proportion of pooled savings held in short- term form might be reduced, and a higher proportion used for the production of new capital assets, or investment.

Recapitulation

It might help to summarise the argument so far.

Reasoning from Popper we might say that in economic life, as in other forms of existence, the future is always uncertain. It is 'open', and not determined by the past. Nonetheless, expectations about the future are created, based largely on previously observed regularities.

In the short term in a market economy, expectations about the prices of normal goods and services can be tested through the price mechanism. By a process of trial and error, production can be adjusted in the light of prices obtained. Competition policy can be used to address distortions in this pricing mechanism, such as through the exercise of monopoly power or price co-ordination by competing sellers.

But in the long term, which is relevant to investment in new capital assets allowing the production of more goods and services in future, it is not possible to rely on a direct pricing mechanism. Instead, a proxy price mechanism has developed, whereby revaluations of existing capital assets act as a guide to the value of new capital assets. But this proxy price mechanism is flawed, making it a poor basis for new investment decisions.

Faced with uncertainty of the future, an alternative approach would be to build up a testable and falsifiable theory of various types of capital asset valuation, as might be developed in other branches of human knowledge, where predictions are made on the basis of hypotheses that can be checked against observation. There is a plentiful supply of data over the past century relating to returns from previous capital assets, returns from assets in the form of securities and returns from companies themselves to allow such a theoretical approach to be developed.

There is, moreover, an increasing need to develop such an approach, given the rising volumes of savings accumulating around the world. If savings are not used for consumption, and are not used for new investment, they then subtract from effective demand in the economy (which is, according to "The General Theory", simply the sum of consumption plus investment) with repercussions that, in terms used by Keynes, may be "disastrous, cumulative and far-reaching".

Practical implications

In "The General Theory", Keynes thought that ultimately the State, or public authorities, would need to have a greater role in the allocation of investment. After surveying in chapter 12 the problems associated with private markets for investment, he concluded (at page 164): "I expect to see the State, which is in a position to calculate the marginal efficiency of capital- goods on long views and on the basis of the general social advantage, taking an ever greater responsibility for directly organising investment: since it seems likely that the fluctuations in the market estimates of the marginal efficiency of different types of capital, calculated on the principles I have described above, will be too great to be offset by any practicable changes in the rate of interest."

This conclusion, no doubt surprising to modern readers, follows from Keynes' analysis of the 'marginal efficiency of capital' as deriving from two factors: the current supply price of capital assets, plus expectations of their future yield. Because fluctuations in expectations of yield, through the operation of organised investment markets, are so great, and because there is a fundamental macroeconomic need to maintain sufficient investment in the economy to support growth and employment, Keynes was driven to rely on the State as the regulator of the level of investment.

If the underlying issue is excessive fluctuations in expectations of yield are there alternative ways of arriving at the same end? A number of possibilities suggest themselves.

A body of knowledge of long-term yield, as suggested above, appears a useful step forward. Who might create such a body? The State (or a regulator) is one possibility. An academic institution is another. A further possibility is a private entity, perhaps like a credit ratings agency, which might operate on a commercial basis. A capital market that does encourage competition for the best long-term forecasts of investment might even be designed, using such a body of knowledge.

In relation to the behaviour of large institutional investors, Woolley and Vayanos put forward several useful practical suggestions, such as limiting the volume of fund turnover, and setting benchmarks for fund performance which are connected to long-term economic value, and not based on the flawed prices created in capital markets themselves (see "Taming the Finance Monster", Central Banking Journal, 2012).

Competition policy might have a role too. Firms in product markets for goods and services are prohibited from reducing uncertainty by co-ordinating their expectations of the future with other firms. Co-ordinating expectations in investment markets appears just as undesirable, if it leads to group, or herding, behaviour that distorts asset prices. As noted in the IMF's Global Financial Stability Report of 2014, "If international investors buy or sell assets simply because they observe other investors doing so, this can amplify boom-bust cycles in financial markets".

The narrow line between parallel conduct and concerted practice can be crossed if competitors exchange internal information allowing them to predict one another's behaviour (see Practical Law Competition, "Following the leader: could herding in the financial markets be a concerted practice or just parallel conduct?"). The normal competition law requirement of decision-taking independence of firms in real economy markets appears just as important in financial markets.

Finally, for debt finance from banks for capital assets, what economist Hyman Minsky has termed 'hedge financing' is desirable on safety grounds: this is where the cash flow from operating capital assets is more than sufficient to pay the bank debt incurred. Again, better knowledge of cash flow (yield) from various types of capital assets could help here too; and perhaps offset the recurring propensity for banks to lend to similar capital assets (usually property) all at the same time.

In short, as an alternative to the State having to take responsibility for allocating investment, competitive market mechanisms might yet have a part to play.

Conclusion: the open society and its economics

The title of this article may call for a little comment. As economist John Kay has noted in his new book, "Other People's Money (2015)", "The term 'price discovery' has no obvious meaning at all" (at page 226).

In relation to capital assets there is not a price to 'discover', but a yield - a stream of future income. No matter how many times a capital asset is bought and sold, the yield will not change. But, given an uncertain future, current resale prices, as represented by securities, although erratic and volatile, may easily displace, or occlude, expectations of future income.

The assumption that current prices can foretell the future is, interestingly, akin to the custom and habit, or psychology, which according to Hume conditions us to expect that the future will resemble the past. Induction by repetition has no basis in logic, however, and extrapolating too easily from the present to the future has dangers when a large part of the market does this collectively, in a "conventional basis of valuation" - and then corrects itself abruptly, as in the 1987 crash, and other financial crises.

Popper's reversal of matters in the twentieth century to explain the growth of scientific knowledge, and the logic of scientific discovery, might be applied with benefit to create a more logical and ‘scientific’ approach to forecasting yield. This then does become 'discovery', in the sense that a new body of knowledge is created which can be improved by a process of trial and error.

Does this imply a revolution in finance? Not necessarily. Keynes thought that long-term investing, although difficult, was possible, and that "it makes a vast difference to an investment market whether or not [long-term investors] predominate in their influence over the game-players".

If, on the other hand, 'game-players' predominate in their influence over long-term investors, and expert professionals compete only to anticipate one another's opinions, this might be viewed as effectively distorting competition, to the detriment of the end savers on whose behalf funds are managed. If so, structural market remedies might be sought, to redress the balance in favour of the long term.

Popper himself was noted for advocating 'piece-meal social engineering', or, in other words, incremental, testable, changes in social organisation to see if they work, on a trial basis. In an open society and an open market economy improvements are always possible, and an evolution in finance may be the most promising avenue for the savings of the past to be transformed into the investment of the future.

• David Harrison is Legal Director at DAC Beachcroft LLP and author of Competition Law and Financial Services (Routledge, London and New York, 2014).