Stochastic Resonance: from Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D
Total Page:16
File Type:pdf, Size:1020Kb
Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information STOCHASTIC RESONANCE From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Stochastic resonance occurs when random noise provides a signal processing bene- fit, and has been observed in many physical and biological systems. Many aspects have been hotly debated by scientists for nearly 30 years, with one of the main questions being whether biological neurons utilise stochastic resonance. This book addresses in detail various theoretical aspects of stochastic resonance with a focus on its extension to suprathreshold stochastic resonance, in the context of stochastic signal quantization theory. Models in which suprathreshold stochastic resonance occur support significantly enhanced “noise benefits”, and exploitation of the effect may prove extremely useful in the design of future engineering systems such as distributed sensor networks, nano-electronics, and biomedical prosthetics. For the first time, this book reviews and systemizes the topic in a way that brings clarity to a range of researchers from computational neuroscientists through to electronic engineers. To set the scene, the initial chapters review stochastic reso- nance and outline some of the controversies and debates that have surrounded it. The book then discusses the suprathreshold stochastic resonance effect as a form of stochastic quantization. Next, it considers various extensions, such as optimiza- tion and tradeoffs for stochastic quantizers. On the topic of biomedical prosthetics, the book culminates in a chapter on the application of suprathreshold stochastic resonance to the design of cochlear implants. Each chapter ends with a review summarizing the main points, and open questions to guide researchers into finding new research directions. © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information STOCHASTIC RESONANCE From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization MARK D. McDONNELL Research Fellow University of South Australia and The University of Adelaide NIGEL G. STOCKS Professor of Engineering University of Warwick CHARLES E.M. PEARCE Elder Professor of Mathematics The University of Adelaide DEREK ABBOTT Professor of Electrical and Electronic Engineering The University of Adelaide © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao˜ Paulo, Delhi Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridge.org/9780521882620 c M. D. McDonnell, N. G. Stocks, C. E. M. Pearce and D. Abbott 2008 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2008 Printed in the United Kingdom at the University Press, Cambridge A catalogue record for this publication is available from the British Library Library of Congress Cataloguing in Publication data Stochastic resonance : from suprathreshold stochastic resonance to stochastic signal quantization / Mark D. McDonnell ...[et al.]. p. cm. Includes bibliographical references and index. ISBN 978-0-521-88262-0 1. Stochastic processes. 2. Signal processing. 3. Mathematical physics. I. McDonnell, Mark D., 1975– II. Title. QC20.7.S8S78 2008 621.382 2–dc22 2008024827 ISBN 978-0-521-88262-0 hardback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information To Juliet, Suzanne, Frances, and Camille. © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information Contents List of figures page x Listoftables xiv Preface xv Foreword xvii Acknowledgments xix 1 Introduction and motivation 1 1.1 Background and motivation 1 1.2 From stochastic resonance to stochastic signal quantization 3 1.3 Outline of book 4 2 Stochastic resonance: its definition, history, and debates 6 2.1 Introducing stochastic resonance 6 2.2 Questions concerning stochastic resonance 9 2.3 Defining stochastic resonance 10 2.4 A brief history of stochastic resonance 14 2.5 Paradigms of stochastic resonance 20 2.6 How should I measure thee? Let me count the ways ... 30 2.7 Stochastic resonance and information theory 34 2.8 Is stochastic resonance restricted to subthreshold signals? 39 2.9 Does stochastic resonance occur in vivo in neural systems? 45 2.10 Chapter summary 46 3 Stochastic quantization 47 3.1 Information and quantization theory 47 3.2 Entropy, relative entropy, and mutual information 48 3.3 The basics of lossy source coding and quantization theory 49 3.4 Differences between stochastic quantization and dithering 52 3.5 Estimation theory 58 3.6 Chapter summary 58 vii © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information viii Contents 4 Suprathreshold stochastic resonance: encoding 59 4.1 Introduction 59 4.2 Literature review 61 4.3 Suprathreshold stochastic resonance 67 4.4 Channel capacity for SSR 101 4.5 SSR as stochastic quantization 112 4.6 Chapter summary 117 5 Suprathreshold stochastic resonance: large N encoding 120 5.1 Introduction 120 5.2 Mutual information when fx (x) = fη(θ − x) 125 5.3 Mutual information for uniform signal and noise 131 5.4 Mutual information for arbitrary signal and noise 135 5.5 A general expression for large N channel capacity 150 5.6 Channel capacity for ‘matched’ signal and noise 158 5.7 Chapter summary 163 6 Suprathreshold stochastic resonance: decoding 167 6.1 Introduction 167 6.2 Averaging without thresholding 172 6.3 Linear decoding theory 174 6.4 Linear decoding for SSR 179 6.5 Nonlinear decoding schemes 195 6.6 Decoding analysis 206 6.7 An estimation perspective 213 6.8 Output signal-to-noise ratio 225 6.9 Chapter summary 230 7 Suprathreshold stochastic resonance: large N decoding 233 7.1 Introduction 233 7.2 Mean square error distortion for large N 234 7.3 Large N estimation perspective 241 7.4 Discussion on stochastic resonance without tuning 244 7.5 Chapter summary 246 8 Optimal stochastic quantization 248 8.1 Introduction 248 8.2 Optimal quantization model 252 8.3 Optimization solution algorithms 258 8.4 Optimal quantization for mutual information 260 8.5 Optimal quantization for MSE distortion 268 8.6 Discussion of results 271 8.7 Locating the final bifurcation 286 8.8 Chapter summary 289 © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-88262-0 - Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization Mark D. McDonnell, Nigel G. Stocks, Charles E. M. Pearce and Derek Abbott Frontmatter More information Contents ix 9 SSR, neural coding, and performance tradeoffs 291 9.1 Introduction 291 9.2 Information theory and neural coding 296 9.3 Rate–distortion tradeoff 309 9.4 Chapter summary 321 10 Stochastic resonance in the auditory system 323 10.1 Introduction 323 10.2 The effect of signal distribution on stochastic resonance 324 10.3 Stochastic resonance in an auditory model 330 10.4 Stochastic resonance in cochlear implants 344 10.5 Chapter summary 356 11 The future of stochastic resonance and suprathreshold stochastic resonance 358 11.1 Putting it all together 358 11.2 Closing remarks 360 Appendix 1 Suprathreshold stochastic resonance 362 A1.1 Maximum values and modes of Py|x (n|x) 362 A1.2 A proof of Equation (4.38) 363 A1.3 Distributions 363 A1.4 Proofs that fQ(τ) is a PDF, for specific cases 370 A1.5 Calculating mutual information by numerical integration 371 Appendix 2 Large N suprathreshold stochastic resonance 373 A2.1 Proof of Eq. (5.14) 373 A2.2 Derivation of Eq. (5.18) 374 A2.3 Proof that fS(x) is a PDF 375 Appendix 3 Suprathreshold stochastic resonance decoding 377 A3.1 Conditional output moments 377 A3.2 Output moments 378 A3.3 Correlation and correlation coefficient expressions