DOCSLIB.ORG

Optimization-Based Approximate Dynamic Programming

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

OPTIMIZATION-BASED APPROXIMATE DYNAMIC
PROGRAMMING

A Dissertation Presented by
MAREK PETRIK

Submitted to the Graduate School of the
University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of

DOCTOR OF PHILOSOPHY
September 2010
Department of Computer Science

  • c
  • ꢀ Copyright by Marek Petrik 2010

All Rights Reserved

OPTIMIZATION-BASED APPROXIMATE DYNAMIC
PROGRAMMING

A Dissertation Presented by
MAREK PETRIK

Approved as to style and content by: Shlomo Zilberstein, Chair Andrew Barto, Member Sridhar Mahadevan, Member Ana Muriel, Member Ronald Parr, Member
Andrew Barto, Department Chair Department of Computer Science
To my parents Fedor and Mariana

ACKNOWLEDGMENTS

I want to thank the people who made my stay at UMass not only productive, but also very enjoyable. I am grateful to my advisor, Shlomo Zilberstein, for guiding and supporting me throughout the completion of this work. Shlomo’s thoughtful advice and probing questions greatly influenced both my thinking and research. His advice was essential not only in forming and refining many of the ideas described in this work, but also in assuring that I become a productive member of the research community. I hope that, one day, I will be able to become an advisor who is just as helpful and dedicated as he is.

The members of my dissertation committee were indispensable in forming and steering the topic of this dissertation. The class I took with Andrew Barto motivated me to probe the foundations of reinforcement learning, which became one of the foundations of this thesis. Sridhar Mahadevan’s exciting work on representation discovery led me to deepen my understanding and appreciate better approximate dynamic programming. I really appreciate the detailed comments and encouragement that Ron Parr provided on my research and thesis drafts. Ana Muriel helped me to better understand the connections between my research and applications in operations research. Coauthoring papers with Jeff Johns, Bruno Scherrer, and Gavin Taylor was a very stimulating and learning experience. My research was also influenced by interactions with many other researches. The conversations with Raghav Aras, Warren Powell, Scott Sanner, and Csaba Szepesvari were especially illuminating. This work was also supported by generous funding from the Air Force Office of Scientific Research.

Conversations with my lab mate Hala Mostafa made the long hours in the lab much more enjoyable. While our conversations often did not involve research, those that did, motivated me to think deeper about the foundations of my work. I also found sharing ideas with my fellow grad students Martin Allen, Chris Amato, Alan Carlin, Phil Kirlin, Akshat Kumar, Sven Seuken, Siddharth Srivastava, and Feng Wu helpful in understanding the broader research topics. My free time at UMass kept me sane thanks to many great friends that I found here.

Finally and most importantly, I want thank my family. They were supportive and helpful throughout the long years of my education. My mom’s loving kindness and my dad’s intense fascination with the world were especially important in forming my interests and work habits. My wife Jana has been an incredible source of support and motivation in both research and private life; her companionship made it all worthwhile. It was a great journey.

v

ABSTRACT
OPTIMIZATION-BASED APPROXIMATE DYNAMIC
PROGRAMMING

SEPTEMBER 2010 MAREK PETRIK
Mgr., UNIVERZITA KOMENSKEHO, BRATISLAVA, SLOVAKIA
M.Sc., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST

Directed by: Professor Shlomo Zilberstein
Reinforcement learning algorithms hold promise in many complex domains, such as resource management and planning under uncertainty. Most reinforcement learning algorithms are iterative — they successively approximate the solution based on a set of samples and features. Although these iterative algorithms can achieve impressive results in some domains, they are not sufficiently reliable for wide applicability; they often require extensive parameter tweaking to work well and provide only weak guarantees of solution quality.

Some of the most interesting reinforcement learning algorithms are based on approximate dynamic programming (ADP). ADP, also known as value function approximation, approximates the value of being in each state. This thesis presents new reliable algorithms for ADP that use optimization instead of iterative improvement. Because these optimization–based algorithms explicitly seek solutions with favorable properties, they are easy to analyze, offer much stronger guarantees than iterative algorithms, and have few or no parameters to tweak. In particular, we improve on approximate linear programming — an existing method — and derive approximate bilinear programming — a new robust approximate method.

The strong guarantees of optimization–based algorithms not only increase confidence in the solution quality, but also make it easier to combine the algorithms with other ADP components. The other components of ADP are samples and features used to approximate the value function. Relying on the simplified analysis of optimization–based methods, we derive new bounds on the error due to missing samples. These bounds are simpler, tighter, and more practical than the existing bounds for iterative algorithms and can be used to evaluate solution quality in practical settings. Finally, we propose homotopy methods that use

vi the sampling bounds to automatically select good approximation features for optimization– based algorithms. Automatic feature selection significantly increases the flexibility and applicability of the proposed ADP methods.

The methods presented in this thesis can potentially be used in many practical applications in artificial intelligence, operations research, and engineering. Our experimental results show that optimization–based methods may perform well on resource-management problems and standard benchmark problems and therefore represent an attractive alternative to traditional iterative methods.

vii

CONTENTS

Page

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

CHAPTER

1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
1.1 Planning Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Challenges and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

PART I: FORMULATIONS

2. FRAMEWORK: APPROXIMATE DYNAMIC PROGRAMMING . . . . . . .9

2.1 Framework and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Model: Markov Decision Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Value Functions and Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.4 Approximately Solving Markov Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.5 Approximation Error: Online and Offline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3. ITERATIVE VALUE FUNCTION APPROXIMATION . . . . . . . . . . . . . . . . 25

3.1 Basic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Bounds on Approximation Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Monotonous Approximation: Achieving Convergence . . . . . . . . . . . . . . . . . . . . . . . 30

3.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 viii

4. APPROXIMATE LINEAR PROGRAMMING: TRACTABLE BUT

LOOSE APPROXIMATION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.2 Sample-based Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.3 Offline Error Bounds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.4 Practical Performance and Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.5 Expanding Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.6 Relaxing Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.7 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.9 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5. APPROXIMATE BILINEAR PROGRAMMING: TIGHT

APPROXIMATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.1 Bilinear Program Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.2 Sampling Guarantees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.3 Solving Bilinear Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.4 Discussion and Related ADP Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 5.5 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

PART II: ALGORITHMS

6. HOMOTOPY CONTINUATION METHOD FOR APPROXIMATE

LINEAR PROGRAMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.1 Homotopy Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.2 Penalty-based Homotopy Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6.3 Efficient Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.4 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.5 Discussion and Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 6.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

7. SOLVING APPROXIMATE BILINEAR PROGRAMS . . . . . . . . . . . . . . . . . 82

7.1 Solution Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

7.2 General Mixed Integer Linear Program Formulation . . . . . . . . . . . . . . . . . . . . . . . 83 7.3 ABP-Specific Mixed Integer Linear Program Formulation . . . . . . . . . . . . . . . . . . . 85

7.4 Homotopy Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 ix
7.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

8. SOLVING SMALL-DIMENSIONAL BILINEAR PROGRAMS . . . . . . . . . 90

8.1 Bilinear Program Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 8.2 Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 8.3 Successive Approximation Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 8.4 Online Error Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 8.5 Advanced Pivot Point Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.6 Offline Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 8.7 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

PART III: SAMPLING, FEATURE SELECTION, AND
SEARCH

9. SAMPLING BOUNDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
9.1 Sampling In Value Function Approximation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 9.2 State Selection Error Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 9.3 Uniform Sampling Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 9.4 Transition Estimation Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 9.5 Implementation of the State Selection Bounds. . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 9.6 Discussion and Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 9.7 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 9.8 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

10.FEATURE SELECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
10.1 Feature Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 10.2 Piecewise Linear Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 10.3 Selecting Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 10.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 10.5 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 10.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

11.HEURISTIC SEARCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 11.2 Search Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 11.3 Learning Heuristic Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 11.4 Feature Combination as a Linear Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

x
11.5 Approximation Bounds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 11.6 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 11.7 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

12.CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

PART: APPENDICES

APPENDICES

A. NOTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 B. PROBLEM DESCRIPTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 C. PROOFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

xi
Introduction (1)
Heuristic Search (11)

Framework (2)
ALP (4)
Iterative Methods (3)
ABP (5)
Sampling Bounds (9)
Homotopy ALP (6)
Solving ABP (7)

Features (10)
Small-Dim ABP (8)

Figure 1. Chapter Dependence

xii

CHAPTER 1
INTRODUCTION

Planning is the process of creating a sequence of actions that achieve some desired goals. Typical planning problems are characterized by a structured state space, a set of possible actions, a description of the effects of each action, and an objective function. This thesis treats planning as an optimization problem, seeking to find plans that minimize the cost of reaching the goals or some other performance measure.

Automatic planning in large domains is one of the hallmarks of intelligence and is, as a result, a core research area of artificial intelligence. Although the development of truly general artificial intelligence is decades — perhaps centuries — away, problems in many practical domains can benefit tremendously from automatic planning. Being able to adaptively plan in an uncertain environment can significantly reduce costs, improve efficiency, and relieve human operators from many mundane tasks. The two planning applications below illustrate the utility of automated planning and the challenges involved.

Blood inventory management A blood bank aggregates a supply of blood and keeps an inventory to satisfy hospital demands. The hospital demands are stochastic and hard to predict precisely. In addition, blood ages when it is stored and cannot be kept longer than a few weeks. The decision maker must decide on blood-type substitutions that minimize the chance of future shortage. Because there is no precise model of blood demand, the solution must be based on historical data. Even with the available historical data, calculating the optimal blood-type substitution is a large stochastic problem. Figure 1.1 illustrates the blood inventory management problem and Section B.2 describes the formulation in more detail.

Hospital 1
Blood Bank

B
0

AB A
A

Hospital 2

Figure 1.1. Blood Inventory Management
1

Irrigation

Electricity Price

Reservoir

Water Inflow

Figure 1.2. Reservoir Management
Water reservoir management An operator needs to decide how much and when to discharge water from a river dam in order to maximize energy production, while satisfying irrigation and flood control requirements. The challenges in this domain are in some sense complementary to blood inventory management with fewer decision options but greater uncertainty in weather and energy prices. Figure 1.2 illustrates the reservoir management problem and Section B.3 describes the formulation in more detail.

Many practical planning problems are solved using domain-specific methods. This entails building specialized models, analyzing their properties, and developing specialized algorithms. For example, blood inventory and reservoir management could be solved using the standard theory of inventory management (Axsater, 2006). The drawback of specialized methods is their limited applicability. Applying them requires significant human effort and specialized domain knowledge. In addition, the domain can often change during the lifetime of the planning system.

Domain-specific methods may also be inapplicable if the domain does not clearly fit into an existing category. For example, because of the compatibility among blood types, blood inventory management does not fit well the standard inventory control framework. In reservoir management, the domain-specific methods also do not treat uncertainty satisfactorily, neither do they work easily with historical data (Forsund, 2009).

This thesis focuses on general planning methods — which are easy to apply to a variety of settings — as an alternative to domain–specific ones. Having general methods that can reliably solve a large variety of problems in many domains would enable widespread application of planning techniques. The pinnacle of automatic planning, therefore, is to develop algorithms that work reliably in many settings.

1.1 Planning Models

The purpose of planning models is to capture prior domain knowledge, which is crucial in developing appropriate algorithms. The models need to capture the simplifying properties of given domains and enable more general application of the algorithms.

A planning model must balance domain specificity with generality. Very domain-specific models, such as the traveling salesman problem, may be often easy to study but have limited applicability. In comparison, very general models, such as mixed integer linear programs, are harder to study and their solution methods are often inefficient.

2
Aside from generality, the suitability of a planning model depends on the specific properties of the planning domain. For example, many planning problems in engineering can be modeled using partial differential equations (Langtangen, 2003). Such models can capture faithfully continuous physical processes, such as rotating a satellite in the orbit. These models also lead to general, yet well-performing methods, which work impressively well in complex domains. It is, however, hard to model and address stochasticity using partial differential equations. Partial differential equations also cannot be easily used to model discontinuous processes with discrete states.

Markov decision process (MDP) — the model used in this thesis — is another common general planning model. The MDP is a very simple model; it models only states, actions, transitions, and associated rewards. MDPs focus on capturing the uncertainty in domains with complex state transitions. This thesis is concerned with maximizing the discounted infinite-horizon sum of the rewards. That means that the importance of future outcomes is discounted, but not irrelevant. Infinite-horizon discounted objectives are most useful in long-term maintenance domains, such as inventory management.

Recommended publications

  • Daft Punk Collectible Sales Skyrocket After Breakup: 'I Could've Made

    BILLBOARD COUNTRY UPDATE APRIL 13, 2020 | PAGE 4 OF 19 ON THE CHARTS JIM ASKER [email protected] Bulletin SamHunt’s Southside Rules Top Country YOURAlbu DAILYms; BrettENTERTAINMENT Young ‘Catc NEWSh UPDATE’-es Fifth AirplayFEBRUARY 25, 2021 Page 1 of 37 Leader; Travis Denning Makes History INSIDE Daft Punk Collectible Sales Sam Hunt’s second studio full-length, and first in over five years, Southside sales (up 21%) in the tracking week. On Country Airplay, it hops 18-15 (11.9 mil- (MCA Nashville/Universal Music Group Nashville), debutsSkyrocket at No. 1 on Billboard’s lion audience After impressions, Breakup: up 16%). Top Country• Spotify Albums Takes onchart dated April 18. In its first week (ending April 9), it earned$1.3B 46,000 in equivalentDebt album units, including 16,000 in album sales, ac- TRY TO ‘CATCH’ UP WITH YOUNG Brett Youngachieves his fifth consecutive cording• Taylor to Nielsen Swift Music/MRCFiles Data. ‘I Could’veand total Made Country Airplay No.$100,000’ 1 as “Catch” (Big Machine Label Group) ascends SouthsideHer Own marks Lawsuit Hunt’s in second No. 1 on the 2-1, increasing 13% to 36.6 million impressions. chartEscalating and fourth Theme top 10. It follows freshman LP BY STEVE KNOPPER Young’s first of six chart entries, “Sleep With- MontevalloPark, which Battle arrived at the summit in No - out You,” reached No. 2 in December 2016. He vember 2014 and reigned for nine weeks. To date, followed with the multiweek No. 1s “In Case You In the 24 hours following Daft Punk’s breakup Thomas, who figured out how to build the helmets Montevallo• Mumford has andearned Sons’ 3.9 million units, with 1.4 Didn’t Know” (two weeks, June 2017), “Like I Loved millionBen in Lovettalbum sales.
  • 1. Summer Rain by Carl Thomas 2. Kiss Kiss by Chris Brown Feat T Pain 3

    1. Summer Rain by Carl Thomas 2. Kiss Kiss by Chris Brown Feat T Pain 3

    1. Summer Rain By Carl Thomas 2. Kiss Kiss By Chris Brown feat T Pain 3. You Know What's Up By Donell Jones 4. I Believe By Fantasia By Rhythm and Blues 5. Pyramids (Explicit) By Frank Ocean 6. Under The Sea By The Little Mermaid 7. Do What It Do By Jamie Foxx 8. Slow Jamz By Twista feat. Kanye West And Jamie Foxx 9. Calling All Hearts By DJ Cassidy Feat. Robin Thicke & Jessie J 10. I'd Really Love To See You Tonight By England Dan & John Ford Coley 11. I Wanna Be Loved By Eric Benet 12. Where Does The Love Go By Eric Benet with Yvonne Catterfeld 13. Freek'n You By Jodeci By Rhythm and Blues 14. If You Think You're Lonely Now By K-Ci Hailey Of Jodeci 15. All The Things (Your Man Don't Do) By Joe 16. All Or Nothing By JOE By Rhythm and Blues 17. Do It Like A Dude By Jessie J 18. Make You Sweat By Keith Sweat 19. Forever, For Always, For Love By Luther Vandros 20. The Glow Of Love By Luther Vandross 21. Nobody But You By Mary J. Blige 22. I'm Going Down By Mary J Blige 23. I Like By Montell Jordan Feat. Slick Rick 24. If You Don't Know Me By Now By Patti LaBelle 25. There's A Winner In You By Patti LaBelle 26. When A Woman's Fed Up By R. Kelly 27. I Like By Shanice 28. Hot Sugar - Tamar Braxton - Rhythm and Blues3005 (clean) by Childish Gambino 29.
  • A Phenomenological Case Study of College English Students

    A Phenomenological Case Study of College English Students

    Old Dominion University ODU Digital Commons Teaching & Learning Theses & Dissertations Teaching & Learning Fall 2019 I Had to Do the Reading: A Phenomenological Case Study of College English Students Jennifer Eleanor Frank Old Dominion University, [email protected] Follow this and additional works at: https://digitalcommons.odu.edu/teachinglearning_etds Part of the Curriculum and Instruction Commons, English Language and Literature Commons, and the Higher Education Commons Recommended Citation Frank, Jennifer E.. "I Had to Do the Reading: A Phenomenological Case Study of College English Students" (2019). Doctor of Philosophy (PhD), Dissertation, Teaching & Learning, Old Dominion University, DOI: 10.25777/6na1-8q73 https://digitalcommons.odu.edu/teachinglearning_etds/59 This Dissertation is brought to you for free and open access by the Teaching & Learning at ODU Digital Commons. It has been accepted for inclusion in Teaching & Learning Theses & Dissertations by an authorized administrator of ODU Digital Commons. For more information, please contact [email protected]. I HAD TO DO THE READING: A PHENOMENOLOGICAL CASE STUDY OF COLLEGE ENGLISH STUDENTS by Jennifer Eleanor Frank B.A. May 2005, George Mason University M.S.Ed. August 2012, Old Dominion University A Dissertation Submitted to the Faculty of Old Dominion University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY EDUCATION OLD DOMINION UNIVERSITY November 2019 Approved by: Judith Dunkerly-Bean (Director) Helen Crompton (Member) Thomas W. Bean (Member) ABSTRACT Jennifer E. Frank Old Dominion University, 2019 Director: Dr. Judith Dunkerly-Bean The purposes of this qualitative phenomenological case study were to investigate multiple student experiences in a general elective introduction to literature course when music was added as an autonomously structured assignment.
  • Karaoke Mietsystem Songlist

    Karaoke Mietsystem Songlist

    Karaoke Mietsystem Songlist Ein Karaokesystem der Firma Showtronic Solutions AG in Zusammenarbeit mit Karafun. Karaoke-Katalog Update vom: 13/10/2020 Singen Sie online auf www.karafun.de Gesamter Katalog TOP 50 Shallow - A Star is Born Take Me Home, Country Roads - John Denver Skandal im Sperrbezirk - Spider Murphy Gang Griechischer Wein - Udo Jürgens Verdammt, Ich Lieb' Dich - Matthias Reim Dancing Queen - ABBA Dance Monkey - Tones and I Breaking Free - High School Musical In The Ghetto - Elvis Presley Angels - Robbie Williams Hulapalu - Andreas Gabalier Someone Like You - Adele 99 Luftballons - Nena Tage wie diese - Die Toten Hosen Ring of Fire - Johnny Cash Lemon Tree - Fool's Garden Ohne Dich (schlaf' ich heut' nacht nicht ein) - You Are the Reason - Calum Scott Perfect - Ed Sheeran Münchener Freiheit Stand by Me - Ben E. King Im Wagen Vor Mir - Henry Valentino And Uschi Let It Go - Idina Menzel Can You Feel The Love Tonight - The Lion King Atemlos durch die Nacht - Helene Fischer Roller - Apache 207 Someone You Loved - Lewis Capaldi I Want It That Way - Backstreet Boys Über Sieben Brücken Musst Du Gehn - Peter Maffay Summer Of '69 - Bryan Adams Cordula grün - Die Draufgänger Tequila - The Champs ...Baby One More Time - Britney Spears All of Me - John Legend Barbie Girl - Aqua Chasing Cars - Snow Patrol My Way - Frank Sinatra Hallelujah - Alexandra Burke Aber Bitte Mit Sahne - Udo Jürgens Bohemian Rhapsody - Queen Wannabe - Spice Girls Schrei nach Liebe - Die Ärzte Can't Help Falling In Love - Elvis Presley Country Roads - Hermes House Band Westerland - Die Ärzte Warum hast du nicht nein gesagt - Roland Kaiser Ich war noch niemals in New York - Ich War Noch Marmor, Stein Und Eisen Bricht - Drafi Deutscher Zombie - The Cranberries Niemals In New York Ich wollte nie erwachsen sein (Nessajas Lied) - Don't Stop Believing - Journey EXPLICIT Kann Texte enthalten, die nicht für Kinder und Jugendliche geeignet sind.
  • The Carroll News

    The Carroll News

    John Carroll University Carroll Collected The aC rroll News Student 2-26-1998 The aC rroll News- Vol. 90, No. 18 (1998) John Carroll News Follow this and additional works at: https://collected.jcu.edu/carrollnews Recommended Citation John Carroll News, "The aC rroll News- Vol. 90, No. 18 (1998)" (1998). The Carroll News. 1216. https://collected.jcu.edu/carrollnews/1216 This Newspaper is brought to you for free and open access by the Student at Carroll Collected. It has been accepted for inclusion in The aC rroll News by an authorized administrator of Carroll Collected. For more information, please contact [email protected]. Volume 90 • Number 18 John Carroll University Cleveland, Ohio Vandals strike: Graffiti ·may be gang related Ed Klein signs o[ gang insignia. In the up­ Williams said. News Ed itor per left corner is the message According to Tom Reilley, di­ Spray-painted graffiti greeted "XTCY," which Williams believed rector of auxiliary plant services, members of the campus commu­ may represent the drug Ecstasy. the paint on the graffiti matched nity last Thursday morning. In the middle of the door is an that used on the Pacelli Lion, Using blue and white spray atypical marking for the Cleve­ which fraternities and soroties paint, an individualorgroup van­ land neighborhood of East !20th traditionally spray pain t as a dalized several exterior doors on street. pledging activity. "They weren't Murphy and Dolan Halls, trash According to Williams, this is too bright, they used the same receptacles near Kellar Commons, significant because graffiti artists paint," Reilley said. the front steps of St.
  • Summer Camp Song Book

    Summer Camp Song Book

    Summer Camp Song Book 05-209-03/2017 TABLE OF CONTENTS Numbers 3 Short Neck Buzzards ..................................................................... 1 18 Wheels .............................................................................................. 2 A A Ram Sam Sam .................................................................................. 2 Ah Ta Ka Ta Nu Va .............................................................................. 3 Alive, Alert, Awake .............................................................................. 3 All You Et-A ........................................................................................... 3 Alligator is My Friend ......................................................................... 4 Aloutte ................................................................................................... 5 Aouettesky ........................................................................................... 5 Animal Fair ........................................................................................... 6 Annabelle ............................................................................................. 6 Ants Go Marching .............................................................................. 6 Around the World ............................................................................... 7 Auntie Monica ..................................................................................... 8 Austrian Went Yodeling .................................................................
  • Climate Risk Perceptions and Demand for Flood Insurance

    Climate Risk Perceptions and Demand for Flood Insurance

    Climate Risk Perceptions and Demand for Flood Insurance Abstract We study how individuals' beliefs about climate change influence their adaptation behavior through the choice and level of flood insurance coverage. Using the heteroge- neous impact of widening partisan polarization on climate change beliefs and exogenous flood insurance premium increases, we show that when more people are worried about global warming, higher the demand for flood insurance in areas where flood insurance is not mandatory. In areas where flood insurance is mandatory, higher the fraction of population who is worried about global warming, higher the propensity to carry volun- tary contents coverage, and lower the likelihood of choosing the maximum deductible amount. JEL Classification: D14, D81, D83, G11, G41 Keywords: global warming, adaptation, flood insurance, perceptions 1. Introduction Flooding is the costliest natural disaster in the United States and the current estimates from the Federal Emergency Management Agency (FEMA) significantly underrepresent the 41 million households exposed to 1 in 100 year flood events (Wing et al., 2018; First Street Foundation, 2020).1 Global warming can potentially increase future flood risks and lead to substantial economic losses (Solomon et al., 2007; Mousavi et al., 2011; Smith, 2020). Flood insurance is a form of effective adaptation measure that helps property owners minimize their losses and recover quickly (Billings et al., 2019), and a simple expected utility analysis shows that purchasing flood insurance is a rational and net positive benefit decision for people at most wealth levels. Despite the potential benefits, subsidized flood insurance premiums, and increasing flood risks, flood insurance take-up rate is less than 5% in areas where flood insurance is not mandated by federal law.2 At a time when the frequency and severity of floods are increasing in-part due to global warming (EASAC, 2018; Smith, 2020), homeowners who are not worried about global warm- ing may assign a lower probability of flood damage to their homes.
  • Songs by Artist

    Songs by Artist

    Songs by Artist Title Title (Hed) Planet Earth 2 Live Crew Bartender We Want Some Pussy Blackout 2 Pistols Other Side She Got It +44 You Know Me When Your Heart Stops Beating 20 Fingers 10 Years Short Dick Man Beautiful 21 Demands Through The Iris Give Me A Minute Wasteland 3 Doors Down 10,000 Maniacs Away From The Sun Because The Night Be Like That Candy Everybody Wants Behind Those Eyes More Than This Better Life, The These Are The Days Citizen Soldier Trouble Me Duck & Run 100 Proof Aged In Soul Every Time You Go Somebody's Been Sleeping Here By Me 10CC Here Without You I'm Not In Love It's Not My Time Things We Do For Love, The Kryptonite 112 Landing In London Come See Me Let Me Be Myself Cupid Let Me Go Dance With Me Live For Today Hot & Wet Loser It's Over Now Road I'm On, The Na Na Na So I Need You Peaches & Cream Train Right Here For You When I'm Gone U Already Know When You're Young 12 Gauge 3 Of Hearts Dunkie Butt Arizona Rain 12 Stones Love Is Enough Far Away 30 Seconds To Mars Way I Fell, The Closer To The Edge We Are One Kill, The 1910 Fruitgum Co. Kings And Queens 1, 2, 3 Red Light This Is War Simon Says Up In The Air (Explicit) 2 Chainz Yesterday Birthday Song (Explicit) 311 I'm Different (Explicit) All Mixed Up Spend It Amber 2 Live Crew Beyond The Grey Sky Doo Wah Diddy Creatures (For A While) Me So Horny Don't Tread On Me Song List Generator® Printed 5/12/2021 Page 1 of 334 Licensed to Chris Avis Songs by Artist Title Title 311 4Him First Straw Sacred Hideaway Hey You Where There Is Faith I'll Be Here Awhile Who You Are Love Song 5 Stairsteps, The You Wouldn't Believe O-O-H Child 38 Special 50 Cent Back Where You Belong 21 Questions Caught Up In You Baby By Me Hold On Loosely Best Friend If I'd Been The One Candy Shop Rockin' Into The Night Disco Inferno Second Chance Hustler's Ambition Teacher, Teacher If I Can't Wild-Eyed Southern Boys In Da Club 3LW Just A Lil' Bit I Do (Wanna Get Close To You) Outlaw No More (Baby I'ma Do Right) Outta Control Playas Gon' Play Outta Control (Remix Version) 3OH!3 P.I.M.P.
  • The Near-Death Experience As a Shamanic Initiation: a Case Study

    The Near-Death Experience As a Shamanic Initiation: a Case Study

    P1: Vendor Journal of Near-Death Studies PH019-297002 February 28, 2001 10:13 Style file version March 18, 1999 The Near-Death Experience as a Shamanic Initiation: A Case Study J. Timothy Green, Ph.D. Capistrano Beach, CA ABSTRACT: The field of near-death studies shares a number of interesting, often compelling, similarities with the ancient spiritual tradition known as shamanism. Not least among these similarities is the fact that a near-death experience (NDE) is a time-honored form of shamanic initiation. I present a case example illustrating how a deep NDE can propel a person who had no prior knowledge or interest in shamanism into spontaneous, often classic, shamanic experiences, while living an apparently normal life in the midst of modern Western society. KEY WORDS: near-death experience; shamanism. Shamanism is an ancient spiritual tradition that had been practiced in many tribal and preliterate societies for at least 30,000 years, and which is based on the ability to enter into a state of ecstasy. Although there are many aspects to shamanism and many different types of shamanic practitioners, central to all shamanic activities is what is re- ferred to as the shamanic journey. During these journeys, the shaman leaves his or her body, enters into spiritual reality, and communicates with spirit helpers he or she encounters there. Often shamans have what is called a tutelary spirit helper, which usually appears in the form of a person or deity. Shamans also develop relationships with the spirits of different animals, referred to as power animals, who assist them in their work.
  • Inside the October 2018 Issue... Horrorscopes, Homecoming Superlatives, Spooky Singles, a Mac Miller Tribute

    Inside the October 2018 Issue... Horrorscopes, Homecoming Superlatives, Spooky Singles, a Mac Miller Tribute

    October 2018 Issue Inside the october 2018 issue... Horrorscopes, Homecoming Superlatives, Spooky Singles, a Mac Miller Tribute, The Arrowhead and so much more! page 1 The Arrowhead OPINION LETTER FROM THE EDITOR: ARROWHEAD STAFF EDITOR-IN-CHIEF THIS IS OUR YEAR! Gianna DiPaolo BY GIANNA DIPAOLO It’s finally the time where we can walk with our heads held high because we’re the COPY EDITOR oldest in the school. Although we’re about two months into school, the senior year hype Jess Kappeler is still alive. The student section couldn’t be more fun, we just finished our last home- coming, and early action for college is about to expire. We have a lot to figure out this PHOTO EDITOR year, but we should probably make this the best year to remember. (Que “We’re All in Bri Mikeska This Together”). Being as this is our last ever year of high school, I think that we should go out with a SPORTS EDITORS bang. No more petty drama or looking back wishing we did things differently- this is the Justin Hawkins year that we put that all to the side and come together. We have one year before we part Sean Nolan our ways and start our lives, so I think we should give North Hills something to reminisce on. WEBSITE EDITOR It is our responsibility now to leave this school in better shape than we found it. Rebekah Froelich Alumni are coming back saying, “this school is going downhill,” but we are going to be the class that saves it.
  • Phonetics and Phonology of Nyagrong Minyag: an Endangered Language of Western China

    Phonetics and Phonology of Nyagrong Minyag: an Endangered Language of Western China

    UNIVERSITY OF HAWAI‘I AT MĀNOA PHD DISSERTATION The Phonetics and Phonology of Nyagrong Minyag, an Endangered Language of Western China John R. Van Way 2018 A DISSERTATION SUBMITTED TO THE UHM GRADUATE DIVISION IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY IN LINGUISTICS DISSERTATION COMMITTEE: Lyle Campbell, Chairperson Victoria Anderson Bradley McDonnell Jonathan Evans Daisuke Takagi Dedicated to the people of Nyagrong khatChO Acknowledgments Funding for research and projects that have led to this dissertation has been awarded by the Endan- gered Languages Documentation Program, the Bilinski Foundation, the Firebird Foundation, and the National Science Foundation East Asia and Pacific Summer Institute. This work would not have been possible without the generous support of these funding agencies. My deepest appreciation goes to Bkrashis Bzangpo, who shared his language with me and em- barked on this journey of language documentation with me. Without his patience, kindness and generosity, this project would not have been possible. I thank the members of Bkrashis’s family who lent their time and support to his project. And I thank the many speakers of Nyagrong Minyag who gave their voices to this project. I would like to thank the many teachers who have inspired, encouraged and supported the re- search and writing of this dissertation. First, I would like to acknowledge my mentor and advisor, Lyle Campbell, who taught me so much about linguistics, fieldwork and language documentation. His support has helped me in myriad ways throughout the journey of graduate school—coursework, funding applications, research, fieldwork, writing, etc. Lyle has inspired me to be the best mentorI can to my own students.
  • INTERVAL MEASUREMENT of Sljbjective MAGNITUDES with Sljbliminal D:Tf'ferences

    INTERVAL MEASUREMENT of Sljbjective MAGNITUDES with Sljbliminal D:Tf'ferences

    INTERVAL MEASUREMENT OF SlJBJECTIVE MAGNITUDES WITH SlJBLIMINAL D:tF'FERENCES BY MURIEL WOOD GERLACH TECHNICAL REPORT NO. 7 APRIL 17, 1957 PREPARED UNDER CONTRACT Nonr 225(17) (NR 171-034) FOR OFFICE OF NAVAL RESEARCH REPRODUCTION IN WHOLE OR IN PART IS PERMITTED FOR ANY PURPOSE OF THE UNITED STATES GOVERNMENT BEHAVIORAL SCIENCES DIVISION APPLIED MATHEMATICS AND STATISTICS LABORATORY STANFORD UNIVERSITY STANFORD, CAL:tF'ORNIA I l f INTERVAL MEASUREMENT OF SUBJECTIVE MAGNITUDES WITH SUBLIMINAL DIFFERENCES by Muriel Wood Gerlach I. INTRODUCTION: THE PROBLEM OF PSYCHOLOGICAL MEASUREMENT There are many aspects of things and events which are best discrimi­ nated, if at all, by the direct responses of living beings, human or animal. For example, while the objective weight of an object may be determined by placing it on an equal arm balance, the determination of its felt weight to an individual requires his own response to hefting the object. The problem of subjective (psychological, sensory) measure­ ment arises when we are concerned with discriminating such aspects of things as their felt weight, seen color, perceived tone, or psychological value, (in contrast to their objective weight, luminosity, physical acoustics, or monetary worth). The discrimination in question is obvi­ ously a matter of the sensory responses of living observers, it is not based on the mechanical or electrical variations of physical instruments (scales, dials, meters). Is such discrimination capable of yielding quantitative orders? Can the subjectively discriminated aspects of things be systematically arranged to form consistent arrays? In brief, is psychological measurement possible? The purpose of our introduction is to discuss some of the answers which have been given to this question and to indicate the relevance of our theory to the iiproblemii of subjec- tive measurement as we see it.