Under Review
Asset Pricing under Computational Complexity
Peter Bossaerts, Elizabeth Bowman, Shijie Huang, Carsten Murawski, Shireen Tang and Nitin Yadav
August 28, 2018 Under Review
1 ASSET PRICING UNDER COMPUTATIONAL COMPLEXITY 1
2 2
3 Peter Bossaerts, Elizabeth Bowman, Shijie Huang, Carsten 3
4 Murawski, Shireen Tang and Nitin Yadav 4
5 We study asset pricing in a setting where correct valuation of securities re- 5 quires market participants to solve instances of the 0-1 knapsack problem,a 6 6 computationally “hard” problem. We identify a tension between the need for 7 7 absence of arbitrage at the market level and individuals’ tendency to use me- 8 thodical approaches to solve the knapsack problem that are effective but easily 8 9 violate the sure-thing principle. We formulate hypotheses that resolve this ten- 9
10 sion. We demonstrate experimentally that: (i) Market prices only reveal inferior 10 solutions; (ii) Valuations of the average trader are better than those reflected in 11 11 market prices; (iii) Those with better valuations earn more through trading; (iv) 12 12 Price quality is unaffected by the size of the search space, but decreases with 13 instance complexity as measured in the theory of computation; (v) Participants 13 14 learn from market prices, in interaction with trading volume. 14
15 Keywords: Efficient Markets Hypothesis, Computational Complexity, Fi- 15
16 nancial Markets, Noisy Rational Expectations Equilibrium, Absence of Arbi- 16 trage. 17 17
18 18
19 19
20 20
21 21
22 22
23 23
24 24
25 25
26 26
27 All authors are of the Brain, Mind & Markets Laboratory, Department of 27
28 Finance, Faculty of Business and Economics, The University of Melbourne, 28 Victoria 3010, Australia. Corresponding author: Peter Bossaerts; email: 29 29 [email protected]; phone: +61.3.3095.3257.
1 date: August 28, 2018 2
1 “An intellect which at a certain moment 1 would know all forces that set nature in motion, 2 and all positions of all items of which nature is composed, 2 3 if this intellect were also vast enough to submit these data to analysis [...]; 3 for such an intellect nothing would be uncertain.” 4 4 Pierre-Simon de Laplace, A Philosophical Essay on Probabilities, 1814 5 5
6 1. INTRODUCTION 6
7 7 The Efficient Markets Hypothesis (EMH) is one of the foundations of 8 8 modern asset pricing theory. Markets are called “efficient” when “security 9 9 prices fully reflect all available information” (Fama, 1991, p. 1575). Theo- 10 10 retical analyses of this statement usually envisage a setting where the value 11 11 of securities is a function of information that is distributed in the economy, 12 12 and correct valuation requires aggregation of the different bits of informa- 13 13 tion (Grossman and Stiglitz, 1976).1 As such, uncertainty can be resolved 14 14 efficiently through sampling, because a law of large numbers holds: one could 15 15 start with a randomly chosen agent, collect her information, then visit ad- 16 16 ditional agents, and add their information; as more agents are sampled, one 17 17 obtains a gradually improving estimate of the true value of the security 18 18 at hand, eventually reaching perfect precision (see, e.g., Grossman (1976, 19 19 1978); Hellwig (1980); Diamond and Verrecchia (1981)). 20 20 Here, we consider a fundamentally different setting. Uncertainty emerges 21 21 because of computational complexity, instead of a lack of information. Specif- 22 22 ically, we envisage a situation where correct valuation of securities require 23 23 market participants to combine publicly available information in the right 24 24 way. As such, valuation is a combinatorial problem that is computationally 25 25 complex.2 26 26 1The market microstructure literature sometimes takes a different approach, where 27 better-informed compete on the basis of the same information (see, e.g., Holden and 27 Subrahmanyam (1992)). There, the issue is whether prices eventually reveal their infor- 28 28 mation, and how (see Bossaerts et al. (2014) for an experimental investigation.) 2 29 We follow computer science convention and define computational complexity as the 29 amount of computational resources required to solve a given problem (Arora et al., 2010).
date: August 28, 2018 3
1 To put this differently: uncertainty about the value of a security is the 1 2 consequence of not knowing what information is pertinent, rather than not 2 3 having the information in the first place. We would claim that this char- 3 4 acterization of uncertainty is germane to the “fundamentalist” approach 4 5 to finance: the value of equity, for instance, requires one to think about 5 6 what projects belong to a firm, how much and what types of debt it should 6 7 issue, where it should operate, etc. But even in standard quantitative anal- 7 8 ysis, our characterization of uncertainty has become more relevant recently: 8 9 many market participants have access to large data sets, so the issue is not 9 10 unavailability of data; instead, the problem is that of not knowing which 10 11 data are relevant. 11
12 We formalize this as follows. We envisage a situation where the values of 12 13 securities depend on the solution of instances of the 0-1 knapsack problem 13 14 (KP), a combinatorial optimization problem (Kellerer et al., 2004). In the 14 15 KP, the decision-maker is asked to find the sub-set of items of different 15 16 values and weights that maximizes the total value of the knapsack (KS), 16 17 subject to a weight constraint. The KP is computationally hard. There is 17 18 no known algorithm that both finds the solution and is efficient, that is, can 18 19 compute the solution in polynomial time.3 Intuitively, it is “easy” to verify 19 20 that a particular set of items achieves a given total value, but it is “hard” 20 21 21
22 22 Note that recent analyses of “complexity” in finance and economics either take a less 23 formal approach (e.g., Skreta and Veldkamp, 2009), or model complexity in probabilistic 23 terms (e.g., Spiegler, 2016). Problems that require resources (usually in terms of time) 24 beyond those available are referred to as “computationally intractable” (Cook, 1983). 24 3In computational complexity theory, an algorithm is called “efficient” if the rate at 25 25 which the amount of time to compute the solution grows as the size of the computa- 26 tional problem increases, is upper-bounded by a polynomial. The KP can be solved with 26 dynamic programming. This algorithm is polynomial in the number of items; however, 27 the memory required to implement dynamic programming grows exponentially. There- 27 fore, dynamic programming substitutes exponentially growing time to address memory 28 28 for exponentially growing time to compute. Experiments demonstrate that humans do 29 not appear to use dynamic programming when solving the KP, which is not surprising: 29 human working memory is very small (Murawski and Bossaerts, 2016).
date: August 28, 2018 4
1 to find the set of items with the highest total value.4 1 2 One could argue that there is really no uncertainty here, since all the 2 3 information (knapsack capacity, items, item values and weights) is publicly 3 4 available. There is a long tradition in decision theory, however, to insist that 4 5 situations like this represent uncertainty. De Finetti, for instance, defines an 5 6 event as a decision problem, i.e., a question that can be answered in only two 6 7 ways: yes (true) or no (false). Uncertainty then emerges when the answer 7 8 cannot be determined purely using logical inference (De Finetti, 1995). The 8 9 question is then reduced to: is it appropriate to use the tools of probability 9 10 theory in order to find the right answer? Can we pretend that a law of large 10 11 numbers holds, sample, update one’s prior belief (about the answer) using 11 12 Bayes’ law, and thereby reduce uncertainty? Some, like Kolmogorov, would 12 13 claim that the answer is no (Kolmogorov, 1983). 13 14 For Savage (Savage, 1972), the answer is yes. As long as the decision- 14 15 maker exhibits choices that satisfy a minimal number of rationality restric- 15 16 tions (such as the sure-thing principle), her actions can be modeled as if she 16 17 has a utility profile over outcome quantities, and beliefs that she updates 17 18 using Bayes’ law. Whether she learns effectively is not an issue; she chooses 18 19 coherently, and remains coherent as she samples. See Bossaerts et al. (2018). 19 20 To be sure, in the context of the KP, randomly trying out knapsacks and 20 21 learning using Bayes’ law is extremely ineffective.5 We shall illustrate this 21 22 with an example later on, where an agent who adheres to Savage’s principles 22 23 would find the optimum with 95% chance only after 400 trials, in a problem 23 24 24 4Technically, the version of the KP used here is the optimization version of the prob- 25 25 lem. This problem is NP-hard. The corresponding decision version of the problem is 26 NP-complete. The optimisation version is at least as hard as the corresponding decision 26 version. 27 5Sampling does not have to be random, but the probabilistic law generating outcomes 27 has to be known, as in e.g., importance sampling (Hastings, 1970), for otherwise Bayes’ 28 28 law cannot be applied. But one cannot know the probabilistic law that generates knapsack 29 values given the optima without having solved the KP in the first place, but then learning 29 is pointless.
date: August 28, 2018 5
1 where there are merely 82 possible capacity-filled knapsacks... It would be 1 2 better to just list all the possible knapsacks and pick the best one. 2 3 Recent research has demonstrated that humans follow effective, method- 3 4 ical approaches when solving the KP. Murawski and Bossaerts (2016) dis- 4 5 covered that humans do not sample knapsacks, and instead follow strate- 5 6 gies that are reminiscent of the algorithms that computer scientists use to 6 7 solve such problems (Kellerer et al., 2004). While not always guaranteeing 7 8 a solution, these algorithms are far more efficient than random sampling. 8 9 One such algorithm is the greedy algorithm, whereby items are put into the 9 10 knapsack in decreasing order of the value-over-weight ratio, until reaching 10 11 capacity. As a result, humans perform in the KP as do computers: effort 11 12 required increases, and resulting performance decreases, as the objective 12 13 computational complexity of the instance increases.6 13 14 But these strategies easily violate Savage’s axioms. The greedy algorithm, 14 15 for instance, is inconsistent with the sure-thing principle, and it is simple 15 16 to show why. Consider two KPs with the same capacity, namely, 2. One KP 16 17 has three items with value and weight pairs (2,1), (3.5,2) and (1,1), while 17 18 the second one has only the last two items. Which would the decision-maker 18 19 choose if she is to maximize value? The greedy algorithm would have her 19 20 choose the second one, since it reaches a value of 3.5, while the first one 20 21 only generates a value of 3. Yet everything she can obtain with the second 21 22 one she can also get with the first one since the items in the first problem 22 23 are in a superset of the second one. 23 24 When convenience is called for, finance scholars have no issue with viola- 24 25 tions of the Savage axioms, and hence, expected utility. Mean-variance op- 25 26 timization, for instance, violates Savage’s axioms; like the greedy agorithm, 26 27 27 6Evidence is emerging that in simpler computational problems, such as that of deter- 28 28 mining the least costly fund investment under a variety of fee structures, humans do not 29 follow methodical approaches, but instead learn through trial-and-error. See Anufriev 29 et al. (2018).
date: August 28, 2018 6
1 it too violates the sure-thing principle. Yet mean-variance optimization is 1 2 effective when deriving optimal portfolios of many securities. It would be 2 3 cumbersome to do the same with preference profiles that satisfy Savage’s 3 4 axioms, such as expected log utility. So, finance scholars use mean-variance 4 5 optimization, opting for efficacy over rationality. 5 6 At the market level, however, violations of the Savage axioms would be 6 7 objectionable. Indeed, to violate the sure-thing principle is tantamount to 7 8 allowing for arbitrage opportunities. The evolution of market prices should 8 9 be free of arbitrage opportunities; price updates should be as if using Bayes’ 9 10 law. This is the Fundamental Theorem of Asset Pricing, of course (Dybvig 10 11 and Ross, 2003). As such, we must not expect market prices to reflect the 11 12 effective solution algorithms that individuals appear to follow. 12 13 This fundamental tension between how markets should behave and how 13 14 individuals in those markets do behave was already pointed out in Dybvig 14 15 and Ross (2003) in the context of violations of expected utility related to loss 15 16 and ambiguity aversion. Experiments have illuminated how individuals can 16 17 violate rationality assumptions yet market prices do not allow for arbitrage 17 18 opportunities; see, e.g., Bossaerts et al. (2010); Asparouhova et al. (2015). 18 19 The conclusion is simple: markets must not be modeled after the behavior 19 20 of their participants. 20 21 21 What could be the consequences of a tension between a market that does 22 22 not allow for arbitrage opportunities at the cost of learning effectiveness, and 23 23 individuals whose actions may violate the sure-thing principle, but makes 24 24 them learn faster? Here, we hypothesized that, in a market where valuation 25 25 depends on the solution of a 0-1 KS problem: (i) Market prices can only 26 26 reveal inferior solutions, (ii) Individual traders will, on average, do better 27 27 than the market, (iii) Individuals who find the solution earn more through 28 28 trading. 29 29 We test these predictions with a laboratory experiment, organized as
date: August 28, 2018 7
1 follows. We endowed participants with shares of several securities (10 or 1 2 12), each of which corresponded to an item in a given instance of the KP. 2 3 All securities lived for a single period, after which they paid a liquidating 3 4 dividend. The dividend equalled one dollar if the corresponding item was 4 5 in the solution of the instance of the KP; and zero otherwise. This makes 5 6 the securities akin to Arrow securities, were it not that multiple securities 6 7 could pay off at once, since in general more than one item could be in the 7 8 optimal knapsack. After markets opened, participants traded the securities 8 9 in a computerised continuous open-book system (a version of the continuous 9 10 double auction where infra-marginal orders are kept in the system until 10 11 cancelled).7 All participants were provided with the same information about 11 12 the instance to be solved, that is, each item’s value and weight, and the total 12 13 capacity of the knapsack. Participants had access to a computer program 13 14 where they could try out candidate solutions. This also meant that we could 14 15 track their solution attempts.8 15 16 There exists an asset pricing paradigm that is consistent with Savage’s 16 17 axioms yet does make the the predictions that prices will not fully reveal the 17 18 right information, and where the average trader performs better than the 18 19 market. This is the Noisy Rational Expectations Equilibrium (NREE) pio- 19 20 neered in Grossman (1977). There, noise is usually modeled as uncertainty 20 21 over aggregate supplies (Grossman and Stiglitz, 1976), or uncertainty about 21 22 the demand from “irrational” traders (Long et al., 1990), or randomness in 22 23 non-traded risk (Biais et al., 2003). In our setting, one can think of noise as 23 24 uncertainty generated by computational complexity. As such, we hypoth- 24 25 esized that price quality would decrease in the level of noise, measured in 25 26 terms of instance complexity. 26 27 We measure instance complexity using the Sahni-k metric. This metric 27
28 28 7We used the software developed by Adhoc Markets (http://www.adhocmarkets.com). 8 29 The program is part of the ULEEF GAMES suite and and can be accessed at 29 http://uleef.business.utah.edu/games.
date: August 28, 2018 8
1 increases in the number of items that have to be put in the knapsack be- 1 2 fore the greedy algorithm can be used to complete the knapsack and obtain 2 3 the optimal solution. Murawski and Bossaerts (2016) showed that Sahni-k 3 4 explains individual performance. In choosing knapsack instances, we made 4 5 sure that Sahni-k was unrelated to the number of possible filled knapsacks, 5 6 so that reduction of price quality would not merely the result of an increase 6 7 in the size of the search space. As a matter of fact, it would be most inter- 7 8 esting to discover that price quality does not decrease with the number of 8 9 possible full knapsacks.9 9 10 In NREE, individuals manage to read prices, despite the inferior qual- 10 11 ity of the information revealed in those prices (compared to the average 11 12 trader’s). We wondered whether in our setting too, traders would improve 12 13 their solutions because of the availability of prices, and if so, how. Because 13 14 we tracked individuals’ attempts at solving the KS instances at hand, we 14 15 were in the position to shed light on the channels through which prices 15 16 make individuals revise their solutions. In earlier work, we had shown that 16 17 more participants find the optimal solution in the presence of markets than 17 18 if they were asked to solve the same instances individually (Meloso et al., 18 19 2009a). 19 20 We find broad support for all our hypotheses: (i) Market prices reveal 20 21 little information about the optimal solution; in fact, they behave as if 21 22 a representative agent is merely randomly sampling knapsacks; (ii) The 22 23 average trader performs better than the market; (iii) Individuals who are 23 24 close to the solution earn more through trading; (iv) Market (and individual) 24 25 performance decreases in instance complexity, or Sahni-k, and not in the size 25 26 of the search space; (v) Individuals manage to improve their solutions by 26 27 reacting to high volume in low-priced securities. 27
28 28 9In the classical paradigm of valuation through aggregation of disparate information, 29 asset pricing theory predicts that increase in the number of possible values does not 29 impede full revelation, as long as this number is finite (Radner, 1979).
date: August 28, 2018 9
1 Our findings are in sharp contrast with those from experiments that build 1 2 on the classical uncertainty paradigm. There, information bits are dispersed 2 3 among participants, and correct valuation merely requires one to add the 3 4 bits together. When security payoffs are common-knowledge (i.e., everyone 4 5 knows how everyone else values the payoffs given aggregate information), 5 6 controlled experiments with centralized, double auctions, have confirmed 6 7 that markets manage to aggregate available information (Plott and Sunder, 7 8 1982, 1988; Plott, 2000).10 The aggregation is so good that participants will 8 9 refuse to pay for information (Sunder, 1992; Copeland and Friedman, 1992). 9 10 That is, the Grossman-Stiglitz paradox emerges (Grossman and Stiglitz, 10 11 1976). 11 12 The remainder of the paper is organized as follows. In the next section, 12 13 using a simple example inspired by our experimental setup, we illustrate 13 14 how random sampling is far less effective in the 0-1 knapsack problem than 14 15 when information merely needs to be aggregated. Section 3 presents the 15 16 experimental paradigm. Section 4 reports the results. We then discuss the 16 17 further implications in a Conclusion. 17
18 18 2. SAMPLING IN THE PRESENCE OF COMPUTATIONAL COMPLEXITY: AN 19 19 ILLUSTRATION 20 20 Here, we illustrate how resolution of uncertainty is fundamentally dif- 21 21 ferent when uncertainty emerges because of lack of information vs. when 22 22 it emerges because of computational complexity. We assume an agent who 23 23 adheres to Savage’s principles (Savage, 1972), and hence, assigns beliefs to 24 24 the possible values, and gradually updates these based on random sampling 25 25 and Bayes’ law. We refer to this agent as the “Savage agent.” 26 26 We first consider a situation of lack of information. There, one needs to 27 27 10The ability of decentralized markets to aggregate dispersed information is hotly dis- 28 28 puted, with Wolinsky (1990) and Duffie and Manso (2007) taking opposite views. See 29 Asparouhova and Bossaerts (2017) and Asparouhova et al. (2017) for experimental evi- 29 dence.
date: August 28, 2018 10
1 gather information, and we assume, as in the classical paradigm, that a law 1 2 of large numbers holds, so that information collection and subsequent aver- 2 3 aging gradually lead one close to the truth. Imagine there are ten securities 3 4 that pay a liquidating dividend of 1 or 0 dollars, with equal chance. Each 4 5 period (“trial”), our agent receives a signal, drawn from a Bernoulli distri- 5 6 bution with p = 0.70 if the final dividend is 1, and with p = 0.30 otherwise. 6 7 The agent uses Bayes’ law to update beliefs about the final payoff. Assuming 7 8 a quadratic loss function, the agent’s valuation is the posterior mean, start- 8 9 ing from an unconditional estimate of 0.5. The top panel of Fig. 1 shows 9 10 how valuations of the securities quickly separate between those that will 10 11 end up paying 1 and those that expire worthless. 11
12 As such, repeated sampling gets one closer to the fundamental value: the 12 13 chance that one deviates too far from knowing the truth decreases with 13 14 sample size. As time progresses, the chance of one valuation to veer off (say, 14 15 from close to 1 down to 0) is drastically reduced. That is, even for finite 15 16 samples (finite number of signals), the valuation estimate is “probably ap- 16 17 proximately correct.” Computer scientists would refer to this situation as 17 18 one of finite sample complexity (Valiant, 1984), that is, a close approxima- 18 19 tion of the true value can be obtained with a finite number of samples. 19 20 20 Now consider a different problem. Here, all the information is available 21 21 from the beginning, but uncertainty emerges because of computational com- 22 22 plexity. Consider securities whose terminal values depend on the solution 23 23 of an instance of the 0-1 KP (instance is #8 in Murawski and Bossaerts 24 24 (2016)). There is a security for each available item. If the item is in the 25 25 optimal solution, the security’s terminal value equals 1; otherwise it expires 26 26 worthless. As before, we assume that the agent assigns a prior value to each 27 27 security, say 0.5. She then gradually adjusts beliefs by sampling, as follows. 28 28 Each trial, the agent randomly tries a subset of the items that fills the KS 29 29 to capacity. The agent then computes the value of the KS and compares to
date: August 28, 2018 11
1 1
Values are 0 0 0 0 0 0 0 1 1 1 2 1 2
3 0.9 3
0.8 4 1 4 2 0.7 3 5 4 5 5 0.6 6 7 6 8 6 0.5 9 10 7 7 Value estimate 0.4
8 0.3 8
9 0.2 9
10 0.1 10
0 11 5 10 15 20 25 30 11 Trials
12 Optimal: 2,5,8 12 1
13 0.9 13
14 0.8 14
0.7 15 1 15 2 3 0.6 4 16 5 16 0.5 6 7 17 8 17 9 Value estimate 0.4 10 18 18 0.3 19 19 0.2
20 0.1 20
21 0 21 5 10 15 20 25 30 Trials 22 22
23 Figure 1.— Simulations When Valuations Require Averageing (Top) vs. 23 Combinatorics (Bottom). Top: Evolution of valuations of ten securities where correct 24 valuations require information aggregation; as sampling increases over time, the proba- 24 bility of being correct increases gradually. This is the situation in traditional theoretical 25 analyses of EMH. Terminal values of the 10 securities are listed on top, in ascending 25 26 order of security number. Bottom: Evolution of prices of ten securities in a situation 26 where correct valuations require combining information in the right way; as sampling 27 increases over time, the probability of being correct – even approximately – does not 27 increase gradually. Only the securities with numbers listed on top (“Optimal”) pay a 28 liquidating dividend of 1; all others expire worthless. See text for details. 28 29 29
date: August 28, 2018 12
1 the maximum value obtained in previous trials. If the new KS value is less 1 2 than the previous maximal value, then she constructs signals as follows. The 2 3 signal for a security equals 1 when the corresponding item was in the earlier 3 4 (better) solution, and 0 otherwise. If the new KS value is higher than the 4 5 trailing maximum, then the signal for a security equals 1 when it is in the 5 6 current solution; otherwise the signal is 0. The agent then updates security 6 7 valuations in trial t by weighing the valuation in the previous trial t − 1 7 8 by (t − 1)/t and the signal in the present trial by 1/t. This may not sound 8 9 like Bayesian because it does not use the true likelihood (of observing full 9 10 knapsack values given the optimal solution). However, this is without con- 10 11 sequence, because a Savage agent does not need to use the true likelihood: 11 12 subjective beliefs are allowed. We merely posit that there exists a belief that 12 13 is justified by the reinforcement learning rule. 13
14 The bottom panel of Fig. 1 illustrates the evolution of securities values 14 15 under the above updating rule. Even after 30 trials, it is still unclear which 15 16 item is in the optimal solution (which security will pay one dollar). Worse, 16 17 valuations can erratically move from high to low and vice versa even in later 17 18 trials. Since there are only a finite, albeit large, number of possible capacity- 18 19 filled knapsacks, the algorithm will eventually find the optimal solution. This 19 20 will happen when, just by chance, the best knapsack is drawn. But this takes 20 21 time. In the present situation, there are (a mere) 82 possible capacity-filled 21 22 knapsacks, so the chance of drawing the best knapsack in any trial equals 22 23 1/82. This implies, among others, that the chance that the algorithm finds 23 24 the optimal knapsack within 30 trials is about 5%; it takes approximately 24 25 four hundred trials to bring this chance up to 95%. 25 26 26 As such, assigning beliefs to the outcomes and learning by sampling, while 27 27 very effective when the cost of processing information is low, is highly inef- 28 28 fective when the cost of processing information is high. In our example, it 29 29 would have been better to just list all 82 possible capacity-filled knapsacks
date: August 28, 2018 13
1 and pick the optimum. This simple, deterministic procedure, finds the op- 1 2 timum for sure in 82 steps, while the sampling approach may not find it 2 3 in 5% of the cases even after a much as four hundred trials. Of course, the 3 4 deterministic approach quickly becomes ineffective too. In some of the KP 4 5 instances we will consider next, there are up to 400 capacity-filled knap- 5 6 sacks. 6
7 7
8 8 3. EXPERIMENTAL DESIGN 9 9
10 Participants 10 11 11
12 Participants were recruited from the University of Melbourne community, 12
13 in four experimental sessions with 18 (one session) or 20 (three sessions) 13
14 participants. To be eligible, participants had to be current students of the 14
15 University of Melbourne aged between 18 to 30 years old with normal or 15
16 corrected-to-normal vision. The final sample included a total of 78 partici- 16
17 pants (age range: 18 to 26 years, mean age = 22, standard deviation = 4, 17
18 gender: 44 male, 34 female). The study was approved by the University of 18
19 Melbourne Human Research Ethics Committee (Ethics ID: 1647059.1) and 19
20 was conducted in accordance with the World Medical Association Declara- 20
21 tion of Helsinki. All participants provided written informed consent. 21
22 22
23 23 Task 24 24
25 25 Participants attempted five instances of the 0-1 knapsack problem, while 26 26 simultaneously trading in an online marketplace. In each instance, partic- 27 27 ipants selected items with given values and weights from a set, in order 28 28 to maximize the total combined value within the weight constraint for the 29 29 selection. Formally, participants were asked to solve the following maximiza-
date: August 28, 2018 14
1 tion problem: 1
2 2 I I 3 X X 3 max xivi s.t. xiwi ≤ C and xi ∈ {0, 1} ∀i, x 4 i=1 i=1 4
5 where i, w, v and C denote the item number, item weight, item value and 5 6 knapsack capacity, respectively. The number of items in an instance varied 6 7 between 10 and 12. Instances were taken from two prior studies (Meloso 7 8 et al., 2009b; Murawski and Bossaerts, 2016). The sequence of instances 8 9 in an experimental session was counterbalanced across sessions. Parameters 9 10 for the five instances are provided in Table I. 10 11 The KP instances were made available electronically on a computer in- 11 12 terface where participants could try out different solutions.11 The software 12 13 recorded every move of an item into and out of the knapsack. Importantly, 13 14 the software did not indicate if a candidate solution was optimal. 14 15 Each item in an instance mapped into a security in the online market- 15 16 place. Therefore, between 10 to 12 markets were available in each instance. 16 17 Exchange was organized using the continuous double-sided open book sys- 17 18 tem, like most purely electronic stock markets globally. Trading was done 18 19 on the online experimental markets platform Flex-E-Markets.12 Participants 19 20 traded for 15 minutes, or in later rounds, less. The user interfaces of the 20 21 two systems, knapsack solver and online marketplace, are shown in Fig. 2. 21 22 Sessions started with a brief motivation and a reading of the instruc- 22 23 tions, after which participants were given ample time to familiarise them- 23 24 selves with knapsack solver, online marketplace, and how to exploit, through 24 25 trading, knowledge acquired when attempting to solve a practice instance. 25 26 Motivation, instructions and practice took a total of one hour. After a break, 26 27 we ran the (five) rounds that counted for final earnings. In total, a session 27 28 28 11The application is part of a game suite called ULEEF GAMES; it can be accessed 29 at http://uleef.business.utah.edu/games. 29 12See http://www.flexemarkets.com.
date: August 28, 2018 15
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
11 11
12 12
13 13
14 14
15 15
16 16
17 17 Figure 2.— Knapsack Problem Interface and Online Trading Platform 18 Left: Knapsack solver, where participants were shown the instance, and could attempt 18 19 solutions by moving items from the OUT panel (right) to the IN KNAPSACK panel 19 (left). Capacity, capacity used, and KS value were displayed on top (left). Right: Online 20 20 marketplace. Each of the 12 colour-coded markets corresponded to an item in the KS, 21 and promised to pay one dollar if the item was in the optimal solution. The book of 21
22 limit orders was shown in the middle, listed by price level, and colour-coded (blue: bids; 22 red: asks). To the right is the order form, where participants could submit buy and sell 23 limit orders, or cancel previous orders. 23 24 24
25 25
26 26
27 27
28 28
29 29
date: August 28, 2018 16
1 took between two and two-and-a-half hours. 1 2 Participant instructions including the timeline of a typical experimental 2 3 session can be found in the Supplementary Online Material (SOM). 3 4 We recorded every order and trade in the marketplace, as well as every 4 5 move into or out of the knapsack from each participant. Timestamps were 5 6 synchronized between the knapsack solver and the online marketplace.13 6
7 7
8 Participant incentives 8 9 9 Participants took positions in the items they believed to be in the optimal 10 10 knapsack by buying shares of the corresponding security. They could also 11 11 sell shares corresponding to items they believed not to be in the solution; 12 12 short sales were not permitted, however. In every instance, participants 13 13 were endowed with $25 in cash holdings (Australian dollars, approximately 14 14 eighteen U.S. dollars), and 12 shares randomly allocated to securities. The 15 15 price range of a share was bounded between $0 and $1. 16 16 Final earnings consisted of: (i) liquidating dividends for the shares held 17 17 at market close; (ii) any change in cash holdings between the beginning and 18 18 end of trading. Earnings were cumulative across instances. Additionally, 19 19 participants received a fixed reward ($2) for submitting a proposed solution 20 20 through the knapsack solver, as well as a show-up fee of $5. 21 21
22 22
23 Initial Allocations 23
24 We designed initial allocations of securities to induce trade, by concentrat- 24 25 ing individual endowments in particular markets. While initial allocations 25 26 were randomized, they were “fair” in the sense that all participants received 26 27 the same number of shares in correct items across the five instances. Ini- 27 28 28 13The marketplace server became unreachable during the second round in the final 29 session, so we have no trade data for that round. This denial of service originated with 29 the server service provider, and hence was beyond our control.
date: August 28, 2018 17
1 tial allocations were such that $31.20 in liquidating dividends were paid 1 2 per participant on average. Participants were not told that they had “fair” 2 3 initial allocations. We imposed fairness in the belief that earnings would 3 4 suffer from the Hirshleifer Effect if prices were to fully reveal all available 4 5 information (see Discussion for more details). 5 6 Although the concepts of “risk” and “risk aversion” have yet to be defined 6 7 precisely in the context of computational complexity, we intuited that un- 7 8 certainty about the solution to a KP instance would induce participants to 8 9 diversify holdings across multiple securities. As a result, participants would 9 10 trade not only because of perception of superior information. We eliminated 10 11 aggregate risk by ensuring that there were an equal numbers of shares across 11 12 securities. This was meant to avoid price distortions that could arise from 12 13 differences in relative supplies. 13 14 Additionally, payment for submission of a solution through the KP in- 14 15 terface was fixed and independent of whether the submission was correct. 15 16 This made it impossible for participants to hedge between trading in the 16 17 marketplace and submission of solutions through the KP interface. 17
18 18
19 Expert Traders 19
20 20 All participants were given the same information about the KP instance 21 21 in a session at the beginning of each session. Thus, there was no information 22 22 heterogeneity and no information asymmetry. However, we did not provide 23 23 participants with the solutions, and since our instances were hard while per- 24 24 formance was variable (the proportion of participants who submitted the 25 25 correct solution varied between 6.4% and 60.3%; see SOM for details), het- 26 26 erogeneity (and asymmetry) arose spontaneously as participants started to 27 27 search for the correct solution. Because all traders were given the same infor- 28 28 mation, we cannot really talk about “informed” and “uninformed” traders 29 29 when referring to those who found the optimal solution and those who failed
date: August 28, 2018 18
1 to find it. So, for the purposes of our study, we define the former as an “Ex- 1 2 pert Trader.” However, we re-emphasize that even expert traders may not 2 3 have been aware that they knew the solution. Such is the nature of the KP... 3
4 4
5 4. RESULTS 5
6 Descriptive Statistics 6 7 7 Each of the 78 participants solved five instances of the KP (390 attempts 8 8 in total). We first looked at computational performance of participants, that 9 9 is, participants’ ability to find optimal solutions. To this end, we examined 10 10 the proportion of participants that were able to solve an instance. Overall, 11 11 37.2% of attempts were correct. Performance varied both by participant 12 12 (min = 0, max = 1, SD = 0.26) and experimental session (min = 0.33, 13 13 max = 0.47, SD = 0.06). 14 14 Next, we investigated whether computational performance in an instance 15 15 was related to the instance’s complexity. We measured instance complexity 16 16 with Sahni-k. This metric increases with both the number of computational 17 17 steps and the amount of memory required to solve an instance. Intuitively, 18 18 Sahni-k is equal to the number of items that have to be selected into the 19 19 knapsack before the knapsack can be filled up using the greedy algorithm 20 20 to find the solution. The greedy algorithm fills the knapsack by selecting 21 21 items in decreasing order of the ratio of value over weight until none of 22 22 the remaining items fits into the knapsack. If Sahni-k equals 0, the greedy 23 23 algorithm generates the solution of the instance. If k is greater than 0, 24 24 the Sahni algorithm generates all feasible k-element subsets, fills up the 25 25 knapsack using the greedy algorithm and finds the set of items with the 26 26 highest total value. The Sahni-k of the instances in this study varied from 27 27 0 to 4; they are listed in Table I.14 28 28 14 29 Note that the number of sets the algorithm considers is given by the binomial co- 29 efficient with n equal to the number of items in the instance and k equal to Sahni-k.
date: August 28, 2018 19
1 The proportion of participants who solved the instance correctly de- 1 2 creased from 60.3% when Sahni-k was equal to 0, to 6.4% when Sahni-k 2 3 was equal to 4. To test the negative relation between computational per- 3 4 formance and Sahni-k, we estimated a mixed-effects model with a binary 4 5 variable set to 1 if an attempt was correct as dependent variable (0 oth- 5 6 erwise), a fixed effect for Sahni-k and random effects (varying intercepts) 6 7 for participant and experimental session. We found a significant main effect 7 8 of Sahni-k (β = −0.578, p < 0.001). The pattern confirms the validity of 8 9 Sahni-k as a measure of instance difficulty for humans, first documented 9 10 in Meloso et al. (2009b) and Murawski and Bossaerts (2016).15 This means 10 11 that the negative relation between Sahni-k and computational performance 11 12 previously documented at individual level Murawski and Bossaerts (2016) 12 13 also exists at the level of markets (see SOM for further details). 13 14 We used the number of items participants moved into and out of the 14 15 knapsack as a proxy for effort. The mean total number of moves in an 15 16 instance was 24.7 (min = 3, max = 123, SD = 19.5; descriptive statistics 16 17 can be found in SOM). To test whether the number of moves depended on 17 18 instance complexity, we related the number of moves a participant made, to 18 19 Sahni-k of the instance (mixed-effects model with a fixed effect for Sahni-k 19 20 and random effects for participant and session). We found a positive effect 20 21 of Sahni-k on the number of item moves (β = 1.649, p < 0.05). This means 21 22 that participants expended more effort on harder instances. This finding is 22 23 consistent with Murawski and Bossaerts (2016), who also found a positive 23 24 correlation between proxies of instance difficulty and effort, using a larger 24 25 number of instances than in the present study. 25
26 26 The number of subsets being considered increases with k and quickly grows beyond the 27 capacity of human working memory. For an instance with 12 items, the number of sets 27 considered if k is equal to 1, is 1 whereas if k is equal to 4, the number of sets considered 28 28 equals 495. 15 29 For a comparison of Sahni-k as a measure of instance complexity with other mea- 29 sures, see Murawski and Bossaerts (2016).
date: August 28, 2018 20
1 The mean number of trades per session was 135.0 (min = 91, max = 202, 1 2 SD = 28.0). The mean number of trades varied by both session (min = 2 3 117.4, max = 161.6, SD = 18.9) and instance (min = 127.2, max = 165.7, 3 4 SD = 17.0). The number of trades did not vary with instance difficulty as 4 5 measured with Sahni-k (one-way ANOVA, F (4, 14) = 1.27, p > 0.1). The 5 6 number of trades in items in the optimal solution did not differ from the 6 7 number of trades in items that were not in the optimal solution (two-sample 7 8 t-test, t(52) = −0.811, p > 0.1). 8 9 Fig. 3 plots the evolution of trade prices in the third round of the first 9 10 session. Each security is indicated by the weight and value of the correspond- 10 11 ing item (weight value) and whether the item is in the optimal knapsack. 11 12 Notice how prices do not monotonously decrease towards zero (indicating 12 13 that the corresponding item is “OUT”) or one (the corresponding item is 13 14 “IN”). Shares in item 129 15, for instance, bounce back and forth between 14 15 a minimum of 25 cents and a maximum of 95 cents. By the end, they were 15 16 trading at 60 cents. The shares expired worthless because the correspond- 16 17 ing item was not in the optimal knapsack (more descriptive statistics on 17 18 prices, including range of prices per item, stratified by instance difficulty, 18 19 are available in the SOM). 19 20 Finally, participants earned $6.32 on average per instance (min = −1.66, 20 21 max = 11.41, SD = 1.92) and 31.58 on average in total (min = −8.30, 21 22 max = 49.25, SD = 9.59) from trading in the market.16 To test whether the 22 23 earnings from trading depended on instance complexity, we related earnings 23 24 of a participant in an instance, to Sahni-k of the instance (mixed-effects 24 25 model with a fixed effect for Sahni-k and random effects for participant and 25 26 26 16Instance earnings could be negative because participants were allowed to buy, and 27 hence spend cash, on shares that eventually did not pay a dividend; the change in cash 27 was subtracted from total dividends earned on final share holdings. Earnings cumulated 28 28 across instances. At the end of the experimental session, participants were paid the 29 minimum of $25 and cumulative earnings plus sign-up reward of $5 plus per-instance 29 submission reward of $2.
date: August 28, 2018 21
1 First Session, Instance 3 1
1 184_28 (1)IN
2 0.9 2
66_10 IN 0.8 184_28 (2) 3 IN 3 219_24 OUT 0.7 229_31 IN 4 60_9 OUT 129_15 4 0.6 OUT
5 0.5 5 72_1 OUT Traded Price 0.4 77_3 (2) OUT 6 184_25 IN 6 0.3
7 0.2 7
0.1 144_14 OUT 8 8 0 06:47:31 06:48:58 06:50:24 06:51:50 06:53:17 06:54:43 06:56:10 Time (hr:min:sec) 9 9
10 Figure 3.— Evolution of Traded Prices in One Instance. 10 11 Shown are time series of transaction prices for the 3rd instance in the first session. 11
12 Series are identified by the weight and value of the corresponding item (weight value) 12 and whether the item was in the optimal knapsack (“IN,” in which case the shares paid 13 13 one dollar; when “OUT,” they expired worthless). 14 14
15 session on the intercept). We did not find a systematic relation between 15
16 Sahni-k and earnings from trading (β = −0.177, p > 0.1).17 16
17 17
18 Testing our predictions 18 19 19 In the following, we test the five predictions. 20 20
21 Prediction 1: Securities Prices Reveal Only Inferior Solutions 21 22 22 The first prediction is that markets do not reveal optimal solutions. In 23 23 our sessions, there was always at least one participant who solved to in- 24 24 stance at hand, and often multiple participants (see SOM). Consequently, 25 25
17 26 Mean earnings were highest in the instance with Sahni-k equal to 0 and lowest in 26 the instance with Sahni-k = 2. Mean earnings amounted to $8.41, $5.13, $5.03, $5.93 and 27 $7.08 for Sahni-k 0 to 4, respectively. Mean maximum payoff was $21.05 in the instance 27 with Sahni-k equal to 0, compared to $12.50, $13.70, $16.25, and $11.95 for instances with 28 28 Sahni-k equal to 1 to 4, respectively. Notice the non-monotonicity of earnings. This is 29 related to the way we allocated shares, which, as mentioned before, attempted to balance 29 incentives to trade and fairness.
date: August 28, 2018 22
1 if prices were to perfectly reflect available knowledge, they should allow us 1 2 to formulate an algorithm that always finds the optimum. We hypothesized 2 3 this not to be the case. 3 4 To evaluate the prediction, we constructed a “market performance” met- 4 5 ric, defined as the distance from the optimal solution, in item space, of 5 6 “market solutions” implied by trade prices. Distance in item space is mea- 6 7 sured as a score which is incremented by one point if a correct item was in 7 8 the submitted knapsack or an incorrect item was left out of the knapsack. 8 9 The score is subsequently scaled by dividing by the total number of items 9 10 in the knapsack. 10 11 To construct “market solutions,” we interpreted the last traded price of an 11 12 item as the market’s “belief” that the item belonged in the optimal solution. 12 13 We then bootstrapped a “market knapsack” by drawing without replace- 13 14 ment items based on these beliefs and filling the knapsack until capacity was 14 15 reached. We computed the performance score of this “market knapsack.” 15 16 We repeated this procedure 10,000 times. We then averaged the resulting 16 17 market performance scores across the bootstraps. If prices correctly valued 17 18 securities, and hence, correctly revealed an instance’s solution, we would 18 19 draw only from “IN” items, and hence, obtain a perfect performance met- 19 20 ric. 20 21 Information revealed in market prices allowed us to reach a performance 21 22 score of 78% in case of the instance with Sahni-k equal to 0. This score de- 22 23 creased monotonically to 68%, 64%, 57% and 52% as Sahni-k increased from 23 24 1 to 4. We conclude that market prices always revealed inferior solutions, 24 25 and that the solutions became worse as instance complexity increased. 25 26 26
27 Prediction 2: The Average Trader Outperforms The Market 27
28 28 In the next step, we examined whether the market did better in solving 29 29 the instances than the average participant. Specifically, we compared the
date: August 28, 2018 23
1 performance of the “market knapsack” to the performance of the knapsacks 1 2 submitted by individual participants. To do so, we constructed performance 2 3 scores for individuals the way we did for the market. We found that the mar- 3 4 ket knapsack performed worse than the knapsack submitted by the average 4 5 participant for every level of computational complexity (Fig. 5). In three 5 6 out of five instances, the market’s score was significantly worse (two-sample 6 7 t test with unequal variances; Bonferroni-Holm family-wise error correction 7 8 at the p = 0.05 level). Performance of the market decreased with instance 8 9 difficulty (Sahni-k; slope = −0.06, p < 0.001), at the same rate as for indi- 9 10 vidual participants. 10
11 11
12 Prediction 3: Expert Traders Make Money 12
13 13 To determine whether Expert Traders earned more money from trading, 14 14 we correlated individual computational performance in an instance with 15 15 earnings (in Australian dollars) from trading in the marketplace. We com- 16 16 puted the former as distance in item space of submitted knapsack from the 17 17 optimal solution, using the score computed as described above. 18 18 The mean payoff in an instance for Expert Traders (those who found the 19 19 solution of the instance) was $8.21 (min = −0.85, max = 21.05, SD = 20 20 3.51), compared to $5.11 (min = −10.55, max = 13.7, SD = 3.93) of 21 21 the remaining participants. Moreover, in every instance, the highest payoff 22 22 among all participants was earned by an Expert Trader. 23 23 To test whether the ability to earn money increased in expertise, we re- 24 24 lated computational performance, as measured by the score described above, 25 25 to earnings from trading. A higher score implies that the participant had 26 26 more correct items in the submitted knapsack and more incorrect items out 27 27 of the submitted knapsack, and as such, the participant knew the correct 28 28 value of more securities than someone with a lower score. We estimated 29 29 a generalized linear mixed-effects model with earnings in a session as de-
date: August 28, 2018 24
1 pendent variable, a fixed effect for score (computational performance) and 1 2 random intercepts for participant and instance. We found a significant ef- 2 3 fect of score (β = 7.011, p < 0.001). A ten percentage point increase in 3 4 computational performance score was associated with additional earnings 4 5 of 70 cents (see Fig. 4, top panel). 5 6 Considering overall session earnings per participant and computational 6 7 performance, we found that they were highly correlated as well: an increase 7 8 in performance score of 10 percentage points increased earnings in the mar- 8 9 ketplace by almost $4 on average (Fig. 4, bottom panel). Note that only 9 10 two participants received a perfect score, implying that only two partici- 10 11 pants solved all five KP instances correctly. 11
12 12
13 Prediction 4: Market Performance Decreases with Instance Complexity, 13
14 Not with Size of Search Space 14
15 15 We proposed that a measure of computational complexity unrelated to 16 16 the number of possible filled knapsacks determines price quality. One way 17 17 to measure computational complexity of the instance at hand is to use the 18 18 Sahni-k metric introduced before. As Sahni-k increases, both the number 19 19 of computational steps and the amount of memory required to solve the 20 20 instance increases. Errors in computations or memory retrieval accumulate, 21 21 and hence, noise in the proposed solution increases, or the computer runs 22 22 out of computational resources. As mentioned before, we propose Sahni-k 23 23 since this metric was found to predict individual performance in solving 24 24 KP instances (Murawski and Bossaerts, 2016). Sahni-k is inspired by algo- 25 25 rithms that approximate KP solutions. The higher Sahni-k, the more likely 26 26 a computer using those algorithms will not find the optimum. 27 27 We found that market performance (performance of knapsack solutions 28 28 one can construct from trade prices) gradually decreased as Sahni-k in- 29 29 creases (Fig. 5). The decrease in performance is highly significant (univari-
date: August 28, 2018 25
1 1 25 2 Sahni-k 0 2 r: 0.3026 Sahni-k 1 20 Sahni-k 2 3 Sahni-k 3 3 Sahni-k 4
4 15 4
5 5 10 6 6
5
7 7 Payoff ($) Payoff
8 0 8
9 9 -5 coeff: 7.011 (p < 0.001) 10 10 -10 11 11
-15 12 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 12 Performance score (scaled to 1) 13 13
14 14
15 60 15 r: 0.422 16 16 50 17 17
40 18 18
19 30 19
20 20 20
21 Total session payoff ($) 21
10 22 coeff: 37.1 22 (p < 0.001)
23 0 23
24 24 -10 25 0.4 0.5 0.6 0.7 0.8 0.9 1 25 Performance score (scaled to 1) 26 26
27 Figure 4.— Individual Trading Payoff Against Performance Score Of 27 Submitted Knapsack. Individual payoff from trading, per instance (top; colour-coded 28 by difficulty) and per session (bottom), against average performance score for the submit- 28 ted solutions; performance score is measured as distance in item space of the submitted 29 knapsack; see text for details; per-instance observations are stratified by Sahni-k. Slope 29 coefficient per instance (mixed-effects regression; see text) equals 7.01 (p < 0.001); Pear- son correlation r = 30.3%. Slope coefficient per participants equals 37.1 (p < 0.001); Pearson correlation r = 42.2%. date: August 28, 2018 26
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
11 11
12 12
13 13
14 14
15 15
16 16
17 17
18 18
19 19
20 20 Figure 5.— Performance Of Instance Solutions Generated From Last 21 Trade Prices Compared To Average Individual Submissions. Market perfor- 21 mance score, measured as distance in item space of “market knapsack” obtained from 22 simulations based on last trade prices (blue), against corresponding average score of in- 22 dividual participants (red), across all sessions, stratified by instance difficulty (Sahni-k). 23 Error bars cover +/- 1 standard error (Market: across sessions; Individuals: across indi- 23 24 viduals in all sessions). Dashed lines are corresponding regression lines (slope of market 24 line: −0.006; p value < 0.001)). Asterisks in boxes indicate that the average individual 25 score is significantly higher than market score at p = 0.05 significance level, corrected for 25 multiple comparisons. INSERT: Market performance score against number of possible 26 knapsacks filled at capacity. Dashed lines: regression line. 26 27 27
28 28
29 29
date: August 28, 2018 27
1 ate LS of mean performance on Sahni-k: slope = −0.06, p < 0.001). 1 2 In contrast, market performance did not change with the number of pos- 2 3 sible knapsacks filled to capacity; see Insert of Fig. 5 (|slope| < 0.001, 3 4 p > 0.10). That is, merely increasing the size of the search space does 4 5 not lower price quality. 5
6 6
7 Prediction 5: Traders Improve Their Solutions by Reacting to High Volume 7
8 in Low-Priced Securities 8
9 9 In traditional “rational expectations” asset pricing models (Radner, 1979), 10 10 lesser informed traders are assumed to know the mapping from states to 11 11 prices and use this mapping to infer states from observed prices. In this 12 12 setting, the term “state” refers to an expectation based on all the informa- 13 13 tion available in the economy. Here, we can interpret “state” as the correct 14 14 solution to the KP instance. 15 15 We investigated whether traders indeed “read” prices. Specifically, we 16 16 studied to what extent trade induced traders to re-visit and improve their 17 17 knapsack, moving it closer to optimum in item space. Overall, descriptive 18 18 statistics of moves of correct and incorrect items in and out of knapsacks 19 19 can be found in SOM. 20 20 Prior research has shown that an important reason why humans may not 21 21 find the optimum is because they tend not to re-consider incorrect items that 22 22 they put into the knapsack early on (Murawski and Bossaerts, 2016). Poor 23 23 episodic memory may explain this reluctance to re-visit early moves. Here, 24 24 we ask whether trade in such items made it more likely that uninformed 25 25 traders took them out. 26 26 To test whether participants improved their knapsack based on informa- 27 27 tion available in the market, we regressed the probability of removing an 28 28 incorrect item that was included early on, onto the price of, and the trading 29 29 volume in, the corresponding security, as well as their interaction using a
date: August 28, 2018 28
1 generalized linear model with a logit link function.18 We consider an item to 1 2 be included “early on” if the participant moved it into the knapsack within 2 3 the first two minutes of trading. We found that there was a significant main 3 4 effect of price (β = −3.491, p < 0.001). This means a 10% decrease in 4 5 price of a security was associated with a 11% increase in the probability of 5 6 removing the corresponding item from the knapsack. 6 7 The effect was stronger when the security corresponding to the item was 7 8 traded more heavily (interaction term of price and trading volume, β = 8 9 13.725, p < 0.05). However, there was no main effect of trading volume 9 10 (β = −7.125, p = 0.068). The negative relation between security price and 10 11 probability of removal is displayed in Fig. 6. 11
12 12
13 13
14 14
15 15
16 16
17 17
18 18
19 19
20 20
21 21
22 22 Figure 6.— Impact Of Trade Prices On Removal Of Incorrect Items 23 Incorrect items chosen into knapsack within two minutes of trading: Fraction of those 23 24 items that are eventually removed, as a function of average trade price. Red line: least 24 squares fit of β/(1 + x)2 where x = price level, β = 0.85 (p < 0.01). 25 25
26 These findings suggest that trade, combined with low prices, induced par- 26 27 ticipants to re-consider incorrect inclusions of items, thus improving overall 27 28 28 18Model selection analysis based on the Bayesian Information Criterion suggested that 29 a model with random effects for participant, instance difficulty (Sahni-k) and session 29 fitted worse than a fixed-parameter regression.
date: August 28, 2018 29
1 computational performance, and hence, securities valuation. 1 2 We did not find an effect on item selection from trade in high-priced secu- 2 3 rities. We conjecture that short-sale restrictions contributed to the asymme- 3 4 try between high-priced and low-priced securities: if an item was deemed to 4 5 be overpriced, participants could only sell shares that they already owned, 5 6 and hence could not put more pressure on prices. 6
7 7
8 5. CONCLUSION 8
9 9 In this study, we investigated to what extent computational complexity 10 10 affected market outcomes. Like with loss or ambiguity aversion (Dybvig and 11 11 Ross, 2003), there is a fundamental tension between individual behaviour 12 12 and the need for market prices to be free of arbitrage opportunities. Indi- 13 13 viduals solve computationally “hard” problems in a methodical way, which, 14 14 although highly effective, easily leads to violations of the sure-thing princi- 15 15 ple. These violations cannot be exposed at the market level, because they 16 16 would amount to arbitrage opportunities. We formulated hypotheses aimed 17 17 at resolving the tension. These hypotheses are in line with predictions of the 18 18 Noisy Rational Expectations Equilibrium, provided one interprets “noise” 19 19 as (instance) complexity. 20 20 We reported results from a markets experiment aimed at testing our pre- 21 21 dictions. We found that prices generally revealed incorrect solutions, and 22 22 that price quality deteriorated as instance complexity increased, though 23 23 price quality was no worse when the search space increased (number of pos- 24 24 sible capacity-filled knapsacks). We also documented that Expert Traders 25 25 (those who submitted the optimal solution) earned significantly more from 26 26 trading on their information; in general, the closer one was to the correct 27 27 solution, the more was earned through trading. Market prices converted 28 28 into candidate solutions that were worse than those submitted by the aver- 29 29 age participant. Information revealed in prices did feed back into individual
date: August 28, 2018 30
1 problem solving, but only in conjunction with volume. 1 2 A number of remarks are in order, which should put our findings in per- 2 3 spective. 3 4 We found that traders learned from prices, which is one of the core tenets 4 5 of “rational expectations” asset pricing theory. To our knowledge, we are 5 6 the first to show this directly, by recording participants’ thinking (their 6 7 attempts to solve the valuation problems) and how it changed as a function 7 8 of prices. Here, we discovered that markets actually mitigated a strong bias 8 9 that keeps individuals from discovering the correct solution of KP instances, 9 10 namely, hesitation to re-consider items that were added to the knapsack 10 11 early on (Murawski and Bossaerts, 2016). Prices, in conjunction with trade, 11 12 made participants re-visit parts of the solution that they had constructed 12 13 within the first two minutes of trading. 13 14 Our finding suggests that markets may provide a powerful mechanism to 14 15 help individuals solve complex problems. This raises the issue whether a 15 16 market mechanism is preferable to mechanisms that provide clearer incen- 16 17 tives for problem solving, such as a prize system, where only the first to 17 18 submit the correct solution wins a prize. Meloso et al. (2009b) reports that 18 19 more participants manage to find the correct solution of KP instances with 19 20 markets than with a prize mechanism. 20 21 This last finding is relevant for the debate on the desirability of patents 21 22 as a way to promote intellectual discovery, for two reasons. First, the prize 22 23 system is analogous to the current patent system. Second, intellectual dis- 23 24 covery can be thought of as the solution of a combinatorial optimization 24 25 problem such as the KP (Boldrin and Levine, 2002).19 Evidently, markets 25 26 may facilitate intellectual discovery better than patents.20 26
27 19A recent empirical study corroborates this claim, by showing that patents filed with 27 the U.S. Patent Office between 1790 and 2010 were mostly for inventions that combined 28 28 existing technologies in novel ways rather than opening up fundamentally new avenues 29 of exploration (Youn et al., 2015). 29 20To illustrate how, consider markets in Li, Na, Mg and other chemicals (we are
date: August 28, 2018 31
1 The conclusion that markets may trump patents is important for eco- 1 2 nomic history as well. It is generally accepted that the patent system pro- 2 3 vided the main impetus for technological advances over the last century and 3 4 a half (Khan and Sokoloff, 2001). However, at the same time markets pen- 4 5 etrated all parts of life, and it may very well have been that markets were 5 6 the major facilitator of innovation rather than patents. This conjecture is 6 7 consistent with historical evidence that technological advances can be far 7 8 bigger during epochs with share trading but without patents than in epochs 8 9 with only patents (Nuvolari, 2004). 9 10 Our findings allow one to put into perspective the evidence on EMH (the 10 11 Efficient Markets Hypothesis) from historical analyses of field data. Fama 11 12 (1991, 1998) surveys a vast body of studies that appears to suggest that 12 13 EMH holds, because anomalies could be attributed to chance (sometimes 13 14 the null will be rejected) or to methodological errors. Behavioral finance 14 15 scholars reject this conclusion, starting with Bondt and Thaler (1985), who 15 16 claimed that long-run price reversals are caused by overreaction. Among 16 17 others, Hirshleifer (2001) summarizes the evidence against EMH, and argues 17 18 against chance or methodological mistakes. 18 19 Our experiment could shed new light on the EMH controversy. When het- 19 20 erogeneous information emerges because agents hold dispersed information 20 21 from which security prices can easily be computed, markets may be expected 21 22 to satisfy EMH. But it is obvious that not all real-world situations can be 22 23 described as such. Instead, correct valuation may be computationally hard, 23 24 at which point the conclusions from our experiment become important, and 24 25 25
26 using standard chemical abbreviations). These are potential components of future battery 26 technology. To determine which chemicals markets to invest in requires one to assess 27 which component, or maybe combination of components, will be required for the best 27 battery technology. This casts intellectual discovery squarely in terms of the KP, and 28 28 to profit from it, one needs markets as we organized them. Inventors are induced to 29 participate in the marketplace, and earn money by buying those components that they 29 believe are in the best battery technology, while selling others.
date: August 28, 2018 32
1 violations of EMH will ensue. Examples of high computational complexity 1 2 includes corporate valuation when valuation depends on how the corpora- 2 3 tion is structured (Which departments should be spun off? Which companies 3 4 should be acquired in order to create “synergistic” valuation effects? Etc.). 4 5 To measure computational complexity, we used standard notions from 5 6 computer science. These notions are based on a theoretical computational 6 7 model (Turing machine), and it was not a priori obvious that they would 7 8 extend to human computing. Recent evidence suggests that computational 8 9 complexity theory does extend to decision-making by humans (Murawski 9 10 and Bossaerts, 2016), and the findings reported here corroborate this notion. 10 11 Therefore, evidence is mounting that the theory of computation is universal, 11 12 in support of the Church-Turing thesis (Church, 1934; Turing, 1936). 12 13 In the present study, we showed that computational complexity theory 13 14 extends even to markets: we found that markets performed worse when valu- 14 15 ation had higher computational complexity, where complexity was measured 15 16 in terms of Sahni-k. This metric, one possible measure of computational re- 16 17 sources required to solve an instance, tracks human performance in solving 17 18 KP instances (Murawski and Bossaerts, 2016). 18 19 19 Finally, we chose to have participant trade Arrow securities whose payoffs 20 20 depended on whether the corresponding item was in the optimal solution. 21 21 One could envisage alternative security designs. For instance, we could have 22 22 let participant trade a single security that paid the value of the optimal 23 23 solution to the KP at hand. According to traditional asset pricing logic, 24 24 the latter should generate worse outcomes: a given knapsack value may 25 25 translate into multiple possible knapsack compositions, many of which could 26 26 be far off the optimal knapsack. That is, one security may not be enough 27 27 to “invert” from knapsack values to choice of items. Intuitively, directly 28 28 trading one Arrow security per item seemed to be the most effective way 29 29 for participants to share information about potential solutions to the KP
date: August 28, 2018 33
1 at hand. The market with Arrow securities seems to be more “complete.” 1 2 Whether this is true remains to be proven empirically. We leave this for 2 3 future work. If pricing is more accurate, and more participants find the 3 4 optimal solution when only one security is traded (whose payoff depends 4 5 on the value of the optimal knapsack), then we would definitely know that 5 6 asset pricing under computational complexity is fundamentally different 6 7 from classical asset pricing theory. Our intuition that “more securities is 7 8 better” would fail. 8
9 9
10 10 Contributions: Bossaerts designed the experiment, Tang run experiment and 11 11 conducted the statistical analysis; Bossaerts, Murawski and Tang wrote the 12 12 manuscript; Huang, Bowman and Yadav contributed to software design and 13 13 data acquisition. 14 14
15 Acknowledgment: Financial support from a R@map chair at The University 15
16 of Melbourne (Bossaerts) is gratefully acknowledged. The experiment re- 16
17 ported here was approved by the University of Melbourne Human Research 17
18 Ethics Committee (Ethics ID: 1647059.1) and was conducted in accordance 18
19 with the World Medical Association Declaration of Helsinki. All partici- 19
20 pants provided written informed consent. We are grateful for commentary 20
21 received from many colleagues, especially from S´ebastienPouget, whose dis- 21
22 cussion at the Financial Management Association 2017 meetings led us to 22
23 re-evaluate the results in the context of the Noisy Rational Expectations 23
24 Equilibrium. A previous version of this paper circulated under the title “In- 24
25 formation Aggregation under Varying Levels of Computational Complexity: 25
26 Experiments.” 26
27 27 REFERENCES 28 28 Anufriev, M., T. Bao, A. Sutan, and J. Tuinstra (2018): “Fee Structure and 29 29 Mutual Fund Choice: An Experiment,” Working Paper Series 45, Economics Discipline
date: August 28, 2018 34
1 Group, UTS Business School, University of Technology, Sydney. 1
2 Arora, S., B. Barak, M. Brunnermeier, and R. Ge (2010): “Computational com- 2 plexity and information asymmetry in financial products,” in ICS, 49–65. 3 3 Asparouhova, E. and P. Bossaerts (2017): “Experiments on Percolation of Infor- 4 4 mation in Dark Markets,” The Economic Journal, 127, F518–F544. 5 Asparouhova, E., P. Bossaerts, J. Eguia, and W. Zame (2015): “Asset pricing 5 6 and asymmetric reasoning,” Journal of Political Economy, 123, 66–122. 6
7 Asparouhova, E., P. Bossaerts, and W. Yang (2017): “Costly Information Acqui- 7 sition in Decentralized Markets: An Experiment,” SSRN Working Paper. 8 8 Biais, B., P. Bossaerts, and C. Spatt (2003): “Equilibrium Asset Pricing under 9 9 Heterogeneous Information,” Available from author’s webpage. 10 Boldrin, M. and D. Levine (2002): “The Case against Intellectual Property,” The 10 11 American Economic Review, 92, 209–212. 11
12 Bondt, W. F. M. D. and R. Thaler (1985): “Does the Stock Market Overreact?” 12 The Journal of Finance, 40, pp. 793–805. 13 13 Bossaerts, P., C. Frydman, and J. Ledyard (2014): “The Speed of Information 14 14 Revelation and Eventual Price Quality in Markets with Insiders: Comparing Two 15 Theories*,” Review of Finance, 18, 1–22. 15 16 Bossaerts, P., P. Ghirardato, S. Guarnaschelli, and W. R. Zame (2010): “Am- 16
17 biguity in asset markets: Theory and experiment,” The Review of Financial Studies, 17 23, 1325–1359. 18 18 Bossaerts, P., N. Yadav, and C. Murawski (2018): “Uncertainty and computational 19 19 complexity,” Philosophical Transactions of the Royal Society B, under review. 20 Church, A. (1934): “The Richard Paradox,” The American Mathematical Monthly, 41, 20
21 356–361. 21 (1983): “An overview of computational complexity,” Communications of 22 Cook, S. A. 22 the ACM, 26, 400–408. 23 23 Copeland, T. E. and D. Friedman (1992): “The market value of information: Some 24 experimental results,” Journal of Business, 241–266. 24 25 De Finetti, B. (1995): “The logic of probability,” Philosophical Studies, 77, 181–190. 25
26 Diamond, D. W. and R. E. Verrecchia (1981): “Information aggregation in a noisy 26 rational expectations economy,” Journal of Financial Economics, 9, 221–235. 27 27 Duffie, D. and G. Manso (2007): “Information percolation in large markets,” The 28 28 American economic review, 97, 203–209. 29 Dybvig, P. H. and S. A. Ross (2003): “Arbitrage, state prices and portfolio theory,” 29
date: August 28, 2018 35
1 Handbook of the Economics of Finance, 1, 605–637. 1
2 Fama, E. (1998): “Market Efficiency, long-term returns, and behavioral finance,” Journal 2 of Financial Economics, 49, 283–306. 3 3 Fama, E. F. (1991): “Efficient Capital Markets: II,” The Journal of Finance, 46, pp. 4 4 1575–1617. 5 Grossman, S. (1976): “On the efficiency of competitive stock markets where trades have 5 6 diverse information,” The Journal of finance, 31, 573–585. 6
7 ——— (1978): “Further results on the informational efficiency of competitive stock mar- 7 kets,” Journal of Economic Theory, 18, 81–101. 8 8 Grossman, S. J. (1977): “The existence of futures markets, noisy rational expectations 9 9 and informational externalities,” The Review of Economic Studies, 44, 431–449. 10 Grossman, S. J. and J. E. Stiglitz (1976): “Information and Competitive Price 10 11 Systems,” The American Economic Review, 66, 246–253. 11
12 Hastings, W. K. (1970): “Monte Carlo sampling methods using Markov chains and 12 their applications,” Biometrika, 57, 97–109. 13 13 Hellwig, M. F. (1980): “On the aggregation of information in competitive markets,” 14 14 Journal of economic theory, 22, 477–498. 15 Hirshleifer, D. (2001): “Investor Psychology and Asset Pricing,” The Journal of Fi- 15 16 nance, 56, 1533–1597. 16
17 Holden, C. W. and A. Subrahmanyam (1992): “Long-lived private information and 17 imperfect competition,” The Journal of Finance, 47, 247–270. 18 18 Kellerer, H., U. Pferschy, and D. Pisinger (2004): Knapsack Problems, Springer 19 19 Science & Business Media. 20 Khan, B. Z. and K. L. Sokoloff (2001): “History lessons: the early development 20
21 of intellectual property institutions in the United States,” The Journal of Economic 21 Perspectives, 15, 233–246. 22 22 Kolmogorov, A. N. (1983): “Combinatorial foundations of information theory and the 23 23 calculus of probabilities,” Russian mathematical surveys, 38, 29. 24 24 Long, J. B. D., A. Shleifer, L. H. Summers, and R. J. Waldmann (1990): “Noise 25 Trader Risk in Financial Markets,” The Journal of Political Economy, 98, 703–738. 25
26 Meloso, D., J. Copic, and P. Bossaerts (2009a): “Promoting intellectual discovery: 26 patents versus markets,” Science, 323, 1335–1339. 27 27 ——— (2009b): “Promoting Intellectual Discovery: Patents vs. Markets,” Science, 323, 28 28 1335. 29 Murawski, C. and P. Bossaerts (2016): “How Humans Solve Complex Problems: 29
date: August 28, 2018 36
1 The Case of the Knapsack Problem,” Scientific Reports, 6. 1
2 Nuvolari, A. (2004): “Collective Invention During the British Industrial Revolu- 2 tion:The Case of the Cornish Pumping Engine,” Cambridge Journal of Economics, 3 3 28, 347–363. 4 4 Plott, C. and S. Sunder (1988): “Rational Expectations and the Aggregation of 5 Diverse Information in Laboratory Security Markets,” Econometrica, 56, 1085–1118. 5 6 Plott, C. R. (2000): “Markets as Information Gathering Tools,” Southern Economic 6
7 Journal, 67, 2–15. 7 Plott, C. R. and S. Sunder (1982): “Efficiency of experimental security markets with 8 8 insider information: An application of rational-expectations models,” The Journal of 9 9 Political Economy, 663–698. 10 Radner, R. (1979): “Rational Expectations Equilibrium: Generic Existence and the 10 11 Information Revealed by Prices,” Econometrica, 47, 655–678. 11
12 Savage, L. J. (1972): The foundations of statistics, Courier Corporation. 12 Skreta, V. and L. Veldkamp (2009): “Ratings shopping and asset complexity: A 13 13 theory of ratings inflation,” Journal of Monetary Economics, 56, 678–695. 14 14 Spiegler, R. (2016): “Competition and Obfuscation,” Annual Review of Economics, 8. 15 Sunder, S. (1992): “Market for information: Experimental evidence,” Econometrica: 15 16 Journal of the Econometric Society, 667–695. 16
17 Turing, A. M. (1936): “On computable numbers, with an application to the Entschei- 17 dungsproblem,” J of Math, 230–265. 18 18 Valiant, L. G. (1984): “A theory of the learnable,” Communications of the ACM, 27, 19 19 1134–1142. 20 Wolinsky, A. (1990): “Information revelation in a market with pairwise meetings,” 20
21 Econometrica: Journal of the Econometric Society, 1–23. 21 (2015): “Invention 22 Youn, H., D. Strumsky, L. M. A. Bettencourt, and J. Lobo 22 as a combinatorial process: evidence from US patents,” Journal of the Royal Society 23 23 Interface, 12, 20150272. 24 24
25 25
26 26
27 27
28 28
29 29
date: August 28, 2018 37
1 TABLE I 1 Five KS Instances Used In The Experiment. Available items, capacity C, 2 and Sahni-k k, for the instances in the experiment. Density is defined as 2 v w 3 the ratio of value to weight . IN means the item is in the solution; 3 OUT means it is not. NFKS denotes the number of possible knapsacks 4 filled to capacity. Problem 5 has a second solution (not indicated), with 4 a higher weight and higher Sahni-k however. 5 1 (k = 1) C = 1, 900, NFKS = 80 5 6 Items 1 2 3 4 5 6 7 8 9 10 6 v 505 435 350 505 640 465 170 220 500 50 7 w 564 489 406 595 803 641 252 330 750 177 7 Density 0.90 0.89 0.86 0.85 0.80 0.73 0.67 0.67 0.67 0.28 8 Solution IN IN OUT IN OUT OUT IN OUT OUT OUT 8
9 9 2 (k = 3) C = 1, 044, NFKS = 22 10 Items 1 2 3 4 5 6 7 8 9 10 10 v 300 350 400 450 47 20 8 70 5 5 11 w 205 252 352 447 114 50 28 251 19 20 11 Density 1.46 1.39 1.14 1.01 0.41 0.40 0.29 0.28 0.26 0.25 12 12 Solution IN OUT IN IN OUT OUT OUT OUT IN IN 13 13 3 (k = 2) C = 850, NFKS = 399 14 Items 1 2 3 4 5 6 7 8 9 10 14 v 28 28 10 9 25 31 15 24 14 3 15 w 184 184 66 60 184 229 129 219 144 77 15 16 Density 0.15 0.15 0.15 0.15 0.14 0.14 0.12 0.11 0.10 0.04 16 Solution IN IN IN OUT IN IN OUT OUT OUT OUT 17 Items 11 12 17 v 3 1 18 w 77 72 18 19 Density 0.04 0.01 19 Solution OUT OUT 20 20 4 (k = 0) C = 1500, NFKS = 36 21 Items 1 2 3 4 5 6 7 8 9 10 21 v 37 32 44 23 45 85 62 106 71 72 22 22 w 50 46 180 107 220 435 360 700 530 820 23 Density 0.74 0.70 0.24 0.21 0.20 0.20 0.17 0.15 0.13 0.09 23 Solution IN IN IN IN IN IN IN OUT OUT OUT 24 24 5 (k = 4) C = 1300, NFKS = 301 25 25 Items 1 2 3 4 5 6 7 8 9 10 26 v 227 129 113 127 303 144 84 147 86 201 26 w 212 121 106 120 288 137 80 140 82 192 27 Density 1.07 1.07 1.07 1.06 1.05 1.05 1.05 1.05 27 Solution IN IN OUT OUT IN IN IN IN IN OUT 28 Items 11 12 28 29 v 251 167 29 w 240 160 Density 1.05 1.04 Solution IN OUT date: August 28, 2018