Modeling Alternatives to Exponential Discounting
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Arxiv:1705.10602V2 [Math.OC] 22 Apr 2020 Time-Consistent Investment and Consumption Strategies Under a General Discount Functi
Time-consistent investment and consumption strategies under a general discount function I. Alia ∗ F. Chighoub † N. Khelfallah ‡ J. Vives § April 23, 2020 Abstract In the present paper we investigate the Merton portfolio management problem in the context of non-exponential discounting, a context that give rise to time-inconsistency of the decision maker. We consider equilibrium policies within the class of open-loop controls, that are characterized, in our context, by means of a variational method which leads to a stochastic system that consists of a flow of forward-backward stochas- tic differential equations and an equilibrium condition. An explicit representation of the equilibrium policies is provided for the special cases of power, logarithmic and exponential utility functions. Keys words: Stochastic Optimization, Investment-Consumption Problem, Merton Port- folio Problem, Non-Exponential Discounting, Time Inconsistency, Equilibrium Strategies, Stochastic Maximum Principle. MSC 2010 subject classifications, 93E20, 60H30, 93E99, 60H10. 1 Introduction Background The common assumption in classical investment-consumption problems under discounted utility is that the discount rate is assumed to be constant over time which leads to the dis- arXiv:1705.10602v2 [math.OC] 22 Apr 2020 count function be exponential. This assumption provides the possibility to compare outcomes occurring at different times by discounting future utility by some constant factor. But on the other hand, results from experimental studies contradict this assumption indicating that dis- count rates for the near future are much lower than discount rates for the time further away in future. Ainslie, in [1], established experimental studies on human and animal behaviour ∗Laboratory of Applied Mathematics, University Mohamed Khider, Po. -
Hyperbolic Discounting and Learning Over Multiple Horizons
HYPERBOLIC DISCOUNTING AND LEARNING OVER MULTIPLE HORIZONS William Fedus Carles Gelada Yoshua Bengio Google Brain Google Brain University of Montreal (Mila) University of Montreal (Mila) [email protected] CIFAR Senior Fellow [email protected] [email protected] Marc G. Bellemare Hugo Larochelle Google Brain Google Brain [email protected] [email protected] ABSTRACT Reinforcement learning (RL) typically defines a discount factor (γ) as part of the Markov Decision Process. The discount factor values future rewards by an exponential scheme that leads to theoretical convergence guarantees of the Bellman equation. However, evidence from psychology, economics and neuroscience sug- 1 gests that humans and animals instead have hyperbolic time-preferences ( 1+kt for k > 0). In this work we revisit the fundamentals of discounting in RL and bridge this disconnect by implementing an RL agent that acts via hyperbolic discount- ing. We demonstrate that a simple approach approximates hyperbolic discount functions while still using familiar temporal-difference learning techniques in RL. Additionally, and independent of hyperbolic discounting, we make a surprising discovery that simultaneously learning value functions over multiple time-horizons is an effective auxiliary task which often improves over a strong value-based RL agent, Rainbow. 1 INTRODUCTION The standard treatment of the reinforcement learning (RL) problem is the Markov Decision Process (MDP) which includes a discount factor 0 ≤ γ < 1 that exponentially reduces the present value of future rewards (Bellman, 1957; Sutton & Barto, 1998). A reward rt received in t-time steps is t devalued to γ rt, a discounted utility model introduced by Samuelson (1937). This establishes a time- preference for rewards realized sooner rather than later. -
Intertemporal Consumption and Savings Behavior: Neoclassical, Behavioral, and Neuroeconomic Approaches
Intertemporal Consumption and Savings Behavior: Neoclassical, Behavioral, and Neuroeconomic Approaches Joshua J. Kim June 2014 Abstract This paper summarizes neoclassical, behavioral, and neuroeconomic models of intertemporal consumption and savings behavior. I summarize the construction and implications of Modigliani & Brumberg's Life-Cycle Hypothesis [4] and Laibson's quasi-hyperbolic consumption function [8] as background and motivation for Bisin & Benhabib's neuroeconomic model of dynamic consumption behavior [3]. In particular, I focus on the mathe- matical construction of each model, their different behavioral assumptions of agent rationality, and the resulting implications for economic theory. 1 The Neoclassical Life-Cycle Hypothesis The development of modern economic theories analyzing intertemporal consumption began in the early 1950s with the publication of Modigliani & Brumberg's Life-cycle Hypothesis (LCH) [4]. By making several arguably innocuous assumptions, Modigliani & Brumberg produce several important and non-trival predictions about macroeconomic processes such as the relationship in aggregate economics between savings and growth. Furthermore, since its construction, the LCH has served as the standard economic approach to the study of consumption and savings behavior and has served as a foundational basis for subsequent models of intertemporal consumption. The structure of the rest of this chapter is as follows: subsection 1.1 presents the theoretical foundations and assumptions of the LCH. Subsection 1.2 discusses some of the implications and results. Subsection 1.3 concludes the discussion on neoclassical models of intertemporal consumption and discusses several objections and limitations of the LCH. 1 1.1 Theoretical Foundations Before we begin the construction of the life-cycle hypothesis, let us first define a few terms that readers unfamiliar with economic theory may not know. -
Variable-Time-Preference-He-Et-Al.Pdf
Cognitive Psychology 111 (2019) 53–79 Contents lists available at ScienceDirect Cognitive Psychology journal homepage: www.elsevier.com/locate/cogpsych Variable time preference ⁎ T Lisheng Hea, , Russell Golmanb, Sudeep Bhatiaa,1 a University of Pennsylvania, United States b Carnegie Mellon University, United States ARTICLE INFO ABSTRACT Keywords: We re-examine behavioral patterns of intertemporal choice with recognition that time pre- Decision making ferences may be inherently variable, focusing in particular on the explanatory power of an ex- Intertemporal choice ponential discounting model with variable discount factors – the variable exponential model. We Stochastic choice provide analytical results showing that this model can generate systematically different choice Preference variability patterns from an exponential discounting model with a fixed discount factor. The variable ex- Computational modeling ponential model accounts for the common behavioral pattern of decreasing impatience, which is typically attributed to hyperbolic discounting. The variable exponential model also generates violations of strong stochastic transitivity in choices involving intertemporal dominance. We present the results of two experiments designed to evaluate the variable exponential model in terms of quantitative fit to individual-level choice data. Data from these experiments reveal that allowing for a variable discount factor significantly improves the fit of the exponential model, and that a variable exponential model provides a better account of individual-level choice probabilities than hyperbolic discounting models. In a third experiment we find evidence of strong stochastic transitivity violations when intertemporal dominance is involved, in accordance with the variable exponential model. Overall, our analytical and experimental results indicate that exponential discounting can explain intertemporal choice behavior that was supposed to be beyond its descriptive scope if the discount factor is permitted to vary at random. -
Non-Exponential Discounting: a Direct Test and Perhaps a New Puzzle
1 Nonexponential Discounting: A Direct Test And Perhaps A New Puzzle Richard Startz University of California, Santa Barbara Kwok Ping Tsang Virginia Tech July 2012 Abstract Standard models of intertemporal utility maximization assume that agents discount future utility flows at a constant rate—exponential discounting. Euler equations estimated over different time horizons should have equal discount rates but they do not. Rising term yield premia imply discount rates that rise with longer horizons since uncertainty is much too small to account for the difference in interest rates. Such deviations from exponential discounting are large enough to make a significant difference in consumption choices over long horizons. Our results can be viewed as providing estimates of horizon-specific discounts, or as a further puzzle concerning intertemporal substitution and uncertainty. Keywords: intertemporal consumer choice, discounting, hyperbolic discounting, consumption, portfolio puzzles, CAPM. JEL Classifications: D11, D91, E21 Introduction We are grateful for comments from Shelly Lundberg and John Campbell, and for support from the Cecil and Jane Castor Endowment at the University of Washington. Thanks to the New York Federal Reserve for having the first author as a guest, though the views here do not represent the views of either the NY Fed or the Federal Reserve system. Further thanks go to seminar participants at the NY Fed, UC Santa Barbara, and Academia Sinica. Please address all correspondence to the first author at [email protected]. Authors’ addresses: Dept. of Economics, 217 North Hall, University of California, Santa Barbara, CA 93106-9210 and Department of Economics, Virginia Tech, 3032 Pamplin Hall (0316), Blacksburg, VA 24061, [email protected]. -
Homo Economicus Is Stochastically Patient
Homo Economicus is Stochastically Patient Eric R. Ulm, Daniel Bauer, Justin Sydnor∗ July 2020 Abstract DeJarnette et al. (2020) introduce the concept of generalized expected discounted utility (GEDU) and show that it can explain the empirical finding that people tend to be risk-averse over time lotteries but violates an intuitively appealing property they call stochastic impatience. This note extends their analysis to the classical case of an agent that maximizes expected utility of future consumption subject to a budget constraint. We show that these classical agents ad- here to GEDU preferences with risk aversion over time lotteries and will often be stochastically patient. We argue that while stochastic impatience may be an intuitively appealing criterion when lottery payouts map directly to concurrent utility experiences, when agents can save, bor- row, and optimize, stochastic patience is a natural result. We present results of a simple survey experiment consistent with this view, finding that individuals exhibit stochastic impatience for small-scale monetary risks but stochastic patience for large-scale risks. JEL classification: D81; D90; C91. Keywords: Risk and Time Preferences, Stochastic Impatience, Expected Consumption Utility. Introduction DeJarnette et al. (2020) advance our understanding of consumers’ preferences across risk and time by analyzing time lotteries. As one of their key results, they show that under Expected Discounted Utility (EDU), individuals must be risk-seeking over time lotteries (RSTL). Yet their experimental results show that the majority of people are risk-averse over time lotteries (RATL). They show that a modification of EDU they call Generalized EDU (GEDU) can account for this risk aversion over time lotteries. -
Encyclopedia of Cognitive Science Intertemporal
1 Title Page: Encyclopedia of Cognitive Science Intertemporal Decision Making (article reference code: 708) David Laibson Professor of Economics, Department of Economics Harvard University Littauer M-14, Department of Economics, Harvard University, Cambridge, Massa- chusetts 02138 Phone: 617-496-3402; Fax: 617-495-8570 [email protected] 2 Headword (title of article): Intertemporal Decision Making Keywords: discount factor, discount function, discount rate, exponential, hyperbolic, dynamic inconsistency A-Heads: Discounted utility • Dynamic inconsistency and self-control • Article Definition: Intertemporal decisions imply tradeoffs between current and future rewards. Intertemporal discounting models formalize these tradeoffs, by quantifying thevaluesofdelayedpayoffs. 3 1. Introduction: Decision makers confront a wide range of critical choices that involve tradeoffs between current and future rewards. For example, young workers save part of their paycheck to raise their quality of life in retirement. Habitual heroin users also make decisions with intertemporal consequences when they choose a short-term drug- induced pleasure that jeopardizes their long-term well-being. To evaluate such tradeoffs, decision makers compare the costs and benefits of ac- tivities that occur at different dates in time. The theory of discounted utility provides one framework for evaluating such delayed payoffs. This theory has normative and positive content. It has been proposed as both a description of what people should do to maximize their well-being, and to describe what people actually do when faced with intertemporal decisions. Both applications of the model are controversial. Discounted utility models typically assume that delayed rewards are not as de- sirable as current rewards or similarly that delayed costs are not as undesirable as current costs. -
Notes on the Economics of Household Energy Consumption and Technology Choice
Notes on the Economics of Household Energy Consumption and Technology Choice Alan H. Sanstad* Lawrence Berkeley National Laboratory Abstract The Office of Management and Budget (OMB) and the Department of Energy (DOE) have initiated a joint effort to examine the issue of consumer welfare impacts of appliance energy efficiency standards, and to extend and discuss enhancements to the methodology by which these impacts are defined and estimated in the regulatory process. DOE's economic analysis of efficiency standards generally takes a life-cycle cost investment perspective focused on the trade- off between initial and operating costs for efficient equipment. In this perspective, the time value-of-money is represented by the cost of capital. In a more general framework, additional trade-offs exist between investment and consumption, and consumer choice over the planning horizon also reflects preferences for future consumption. In this framework, these preferences combine with the cost-of-capital as drivers of consumer choice. This document presents a first version of a mathematical framework for analyzing the similarities and differences between these two decision modeling approaches, and thus starts to address several theoretical economic issues raised by OMB. It is anticipated that further elaboration of this framework may support empirical analysis to develop practical quantitative tools for improved assessment of the effects of appliance standards. * Staff Scientist, Lawrence Berkeley National Laboratory. #1 Cyclotron Rd., Berkeley, CA 94720. Voice: 510-486-6433; Fax: 510-486-6996; e-mail: [email protected]. 1. Introduction U. S. federal appliance energy efficiency standards are established using a set of criteria pertaining to their effects on industry, consumers, environmental quality, and other factors. -
Neuroeconomic Studies of Impulsivity: Now Or Just As Soon As Possible?
University of Pennsylvania ScholarlyCommons Departmental Papers (Psychology) Department of Psychology 5-2-2007 Neuroeconomic Studies of Impulsivity: Now or Just as Soon as Possible? Paul W. Glimcher New York University Joseph W. Kable University of Pennsylvania, [email protected] Kenway Louie New York University Follow this and additional works at: https://repository.upenn.edu/psychology_papers Part of the Psychology Commons Recommended Citation Glimcher, P. W., Kable, J. W., & Louie, K. (2007). Neuroeconomic Studies of Impulsivity: Now or Just as Soon as Possible?. American Economic Review, 97 (2), 142-147. http://dx.doi.org/10.1257/aer.97.2.142 This paper is posted at ScholarlyCommons. https://repository.upenn.edu/psychology_papers/30 For more information, please contact [email protected]. Neuroeconomic Studies of Impulsivity: Now or Just as Soon as Possible? Abstract Existing behavioral studies of intertemporal choice suggest that both human and animal choosers are impulsive. One possible explanation for this is that they discount future gains in a hyperbolic or quasi- hyperbolic fashion (David I. Laibson 1997; Shane Frederick, George Loewenstein, and Ted O’Donoghue 2002). This observation stands in contrast to standard normative theory, which predicts exponential discounting for any single maximizing agent (Robert H. Strotz 1956). This disparity between empirical and normative approaches is typically explained by proposing that human choosers suffer from inner conflict, balancing an impulse for an immediate gratification against other forces calling for delayed gratification (Richard H. Thaler and H. M. Shefrin 1981; Laibson 1997; Drew Fudenberg and David K. Levine 2006; Jess Benhabib and Alberto Bisin 2005; B. Douglas Bernheim and Antonio Rangel 2004; Faruk Gul and Wolfgang Pesendorfer 2001). -
Measuring Time Preferences
NBER WORKING PAPER SERIES MEASURING TIME PREFERENCES Jonathan D. Cohen Keith Marzilli Ericson David Laibson John Myles White Working Paper 22455 http://www.nber.org/papers/w22455 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA 02138 July 2016 We thank Steven Durlauf and five anonymous referees at the Journal of Economic Literature for thoughtful comments. We thank Layne Kirshon, Omeed Maghzian, Lea Nagel, and Kartik Vira for outstanding research assistance. We acknowledge financial support from the National Institutes of Health (NIA R01AG021650 and P01AG005842) and the Pershing Square Fund for Research in the Foundations of Human Behavior. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer- reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications. © 2016 by Jonathan D. Cohen, Keith Marzilli Ericson, David Laibson, and John Myles White. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. Measuring Time Preferences Jonathan D. Cohen, Keith Marzilli Ericson, David Laibson, and John Myles White NBER Working Paper No. 22455 July 2016, Revised February 2019 JEL No. D03,D9 ABSTRACT We review research that measures time preferences—i.e., preferences over intertemporal tradeoffs. We distinguish between studies using financial flows, which we call “money earlier or later” (MEL) decisions and studies that use time-dated consumption/effort. -
Rethinking Economic Modelling: the Case for Behavioural Economics
BEHAVIOURAL ECONOMICS Rethinking Economic Modelling: the Case for Behavioural Economics Pierre-Louis Boczmak, Senior Sophister The concept of Economic Man has been at the heart of economics since its early days. In this essay, Pierre Louis Boczmak assesses two models based on this precept; Ex- pected Utility and Discounted Utility using Stigler’s three criteria for a sound economic model: empirical accuracy, generality, and tractability. This essay also presents two alternative models offered by behavioural economist who deviate from the assumption of Homo Economicus. This essay then concludes that the be- havioural models, Prospect Theory and Hyperbolic dis- counting take into account the limitations of Economic Man and improve the empirical accuracy of economic models without significantly impacting the other two criteria. However, much work is still needed in mod- eling the insights of behavioural economics before the discipline can move away from the concept of Economic Man. INTRODUCTION large body of research has identified the relations between countries as well A as internal social and political order as being strongly determined by eco- nomic prosperity, itself overwhelmingly associated with market economy. Thus, bettering our understanding of today’s market economy could yield a wide range of benefits in the future. This essay shall therefore be concerned with the use- 85 STUDENT ECONOMIC REVIEW VOL. XXXIII fulness of the concept of Economic Man for theorising and modelling today’s economy, and, by extension, the assessment of those models which make use of the concept. For this purpose, this paper will not be investigating the concept’s usefulness from a philosophical and sociological standpoint, and recognises that real people’s actions are much more complex than what the Economic Man con- cept allows for. -
Overlapping Generations Under Hyperbolic Discounting
Die approbierte Originalversion dieser Diplom-/ Masterarbeit ist in der Hauptbibliothek der Tech- nischen Universität Wien aufgestellt und zugänglich. http://www.ub.tuwien.ac.at The approved original version of this diploma or master thesis is available at the main library of the Vienna University of Technology. http://www.ub.tuwien.ac.at/eng DIPLOMARBEIT Overlapping Generations under Hyperbolic Discounting zur Erlangung des akademischen Grades einer Diplom-Ingenieurin im Rahmen des Studiums Statistik-Wirtschaftsmathematik ausgef¨uhrtam Institut f¨urStochastik und Wirtschaftsmathematik der Fakult¨at f¨urMathematik und Geoinformation der Technischen Universit¨atWien von Katharina Ruhmer Matrikelnummer 01227422 unter der Leitung von Ass. Prof. Dipl.-Math. Dr. Timo Trimborn Wien, 19.12.2018 (Unterschrift Verfasserin) (Unterschrift Betreuer) Abstract Hyperbolic discounting has been argued to inevitably lead to time-inconsistent be- haviour. In this thesis a time-consistent method of hyperbolic discounting is intro- duced to the Blanchard-Yaari-Buiter model of overlapping generations in continuous time. Results are compared by using an argument that ensures equivalent present values of a constant utility stream for both discounting methods. Analysis at an aggregate level in a closed economy shows that for high life expectancies the econ- omy is balanced at a lower interest rate under hyperbolic discounting than under conventional exponential discounting and vice versa. This is due to the fact that hyperbolically discounting individuals exhibit higher discount rates in the short run but more savings in the long run. Contents 1 Introduction2 2 The Model4 2.1 An Economy of Perpetual Youth....................4 2.2 The Setup.................................5 3 Exponential Discounting7 3.1 Individual Households..........................7 3.2 Aggregation...............................