Course'Team Alan%Heavens% ICIC Data Analysis Workshop Andrew%Jaffe% Principled statistical methods for researchers Jonathan%Pritchard% ! 8-11 September 2014 Daniel%Mortlock% Roberto%Tro;a%% Justin Alsing Hik Shariff Sponsored by STFC ! Rachel Groom and Winton Capital Sandie Bernor South Kensington Campus South Kensington Campus Hyde Park Logis>cs%and%events Kensington Gore Royal Albert • Fire%exits% Hall We • Prince’s Gate I/O:%Tea/coffee/lunch%(Blacke;%311),%toilets% 2 Gardens are e’s Gate Princ here Queen’s Gate • Breakfast%8.15P8.45%a.m.% 1 Beit Quad Prince Consort Road Ethos 3 Sports 45 8 Centre Blackett 7 Royal School 12 • 9 10 of Mines Events:% Prince’s Gardens (North Side) 6 e Bon 11 oderic Hill R Bessemer Business School 17 35 13 14 ACEX • Talk%by%Tom%Babbedge%(Winton)%today%~5%p.m.% al 36 Huxley 16 18 Prince’s 19 15 Gardens Electric Eastside Engineering 22 • Barbecue%tonight%6%p.m.%58%Princes%Gate% 20 Sherfeld 21 Faculty 23 28 25 Prince’s Gardens (Watts Way) Queen’s 26 27 City and Lawn Guilds • Southside Drinks%recep>on%5:30%p.m.%tomorrow% Library Skempton Building 29 24 Exhibition Road 58 Princes Gate Imperial College Road • Public%engagement%lunch,%Wednesday% 30 32 Chemistry 33 Chemistry 50 metres RCS1 31 Sir Alexander 34 Fleming South 4 Frankland Road Kensington Cromwell Road e Thurloe Plac Buildings where wheelchair access is not possible at this time Thurloe Street 1 Beit Quadrangle 12 Goldsmiths Building 21 Grantham Institute 30 Sir Ernst Chain Building – 2 Imperial College Union 13 Huxley Building for Climate Change Wolfson Laboratories 3 Ethos Sports Centre 14 ACE Extension 22 Faculty Building 31 Flowers Building 4 Prince’s Gdns, North Side 15 William Penney 23 58 Prince’s Gate 32 Chemistry Building Garden Hall Laboratory 24 170 Queen’s Gate 33 Sir Alexander Fleming 5 Weeks Hall 16 Electrical Engineering 25 Central Library Building 6 Blackett Laboratory 17 Business School 26 Queen’s Tower 34 Chemistry RCS1 7 Roderic Hill Building 18 53 Prince’s Gate 27 Skempton Building 35 52 Prince’s Gate 8 Bone Building 19 Eastside 28 City and Guilds 36 Alumni Visitor Centre 9 Royal School of Mines 20 Sherfeld Building Building 10 Aston Webb Student Hub 29 Southside 11 Bessemer Building Conference Ofce Outline%of%course ICIC%Data%Analysis%Workshop: the%Bayesics • Basic%principles% • Sampling%% • Numerical%methods%(Parameter%inference)% • MCMC% Alan%Heavens%% • Hybrid/Hamiltonian%Monte%Carlo% Imperial%College%London% • Bayesian%Hierarchical%Models% ICIC%Data%Analysis%Workshop% • Bayesian%Evidence%(Model%selec>on) 8%September%2014 5 Outline LCDM%fits%the%WMAP%% data%well. • Inverse%problems:%from%data%to%theory% ∆T ∆T • Probability%review,%and%Bayes’%theorem% Correla>on%Func>on (φ) (φ + ✓) h T T i Power%Spectrum%Cl • Parameter%inference% • Priors% • Marginalisa>on% • Posteriors Inverse%problems Examples • Most%cosmological%problems%are%inverse' • Hypothesis%tes>ng% problems,'where%you%have%a%set%of%data,%and%you% – Is%the%CMB%radia>on%consistent%with%(ini>ally)%gaussian% want%to%infer%something.%%% fluctua>ons?% • Parameter%inference% • P%generally%harder%than%predic>ng%the%outcomes% – In%the%Big%Bang%model,%what%is%the%value%of%the%ma;er% when%you%know%the%model%and%its%parameters% density%parameter?% • Examples% • Model%selec>on% – Do%cosmological%data%favour%the%Big%Bang%theory%or%the% – Hypothesis%tes>ng% Steady%State%theory?% – Parameter%inference% – Is%the%gravity%law%General%Rela>vity%or%a%different% theory? – Model%selec>on What%is%probability?% Bayes’%Theorem • Frequen>st%view:%p%describes%the%rela>ve%frequency'of' • Rules%of%probability:% outcomes'in%infinitely%long%trials% ! • p(x)%+%p(not%x)%=%1%% % sum%rule% • Bayesian%view:%p%expresses%our%degree'of'belief% • p(x,y)%=%p(x|y)%p(y)%% % product%rule% ! • p(x)%=%Σk%p(x,yk)%% % % marginalisa>on% • Bayesian%view%is%what%we%seem%to%want%from% experiments:%e.g.%given'the'Planck'data,'what'is'the' • Sum⟶%integral%%%%%%%%%%%%%%%%con>nuum%limit%(p=pdf)% probability'that'the'density'parameter'of'the'Universe' p(x)= dy p(x, y) is'between'0.9'and'1.1? ! • p(x,y)=p(y,x)%gives%Z Bayes’'theorem p(x y)p(y) p(y x)= | | p(x) p(x|y)%is%not%the%same%as%p(y|x) The%Monty%Hall%problem:%% An%exercise%in%using%Bayes’%theorem • x%=%female,%y=pregnant% You choose % ? • p(y|x)%=%0.03% % this one • p(x|y)%=%1 Do%you%change%your%choice?% ! This%is%the%Monty%Hall%problem Bayes’%Theorem%and%Inference Posteriors,%likelihoods,%priors%and%evidence • If%we%accept%p%as%a%degree%of%belief,%then%what% p(x ✓)p(✓) p(✓ x)= | we%ohen%want%to%determine%is*% | p(x) ! Evidence% Prior %%%%%%%%%:%model%parameter(s),%x:%the%data% Posterior Likelihood%L or%Model%Likelihood p(x ✓)p(✓) p(✓ x)= | Remember%that%we%interpret%these%in%the%context%of%a%model%M,%so%all%probabili>es%are% To%compute%it,%use%Bayes’%theorem% | p(x) condi>onal%on%M%(and%on%any%prior%info%I).%%E.g.% Note%that%these%probabili>es%are%all%condi>onal% The%evidence%looks%rather%odd%–%what%is%the%probability'of'the'data?%%For%parameter% on%a)%prior%informa>on%I,%b)%a%model%M% es>ma>on,%we%can%ignore%it%–%it%simply%normalises%the%posterior.%%If%you%need%it, p(x)= p(x ✓ )p(✓ ) or p(x)= d✓ p(x ✓)p(✓) p(✓ x)=p(✓ x, I, M) or p(✓ xIM) | k k | | | | Xk Z *This%is%RULE%1:%%start%by%wri>ng%down%what%it%is%you%want%to%know% No>ng%that%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%makes%its%role%clearer.%%% %%%RULE%2:%There%is%no%RULE%n,%n>1 In%model'selecFon'(from%M'and%M’),%% Forward%modelling Case%study:%the%mean • Given%a%set%of%N%independent%samples%{xi}%from%the% same%distribu>on,%with%gaussian%dispersion%σ,%what%is% the%mean%of%the%distribu>on%%%%%%%%%%%%%%%%%?%µ = x With%noise% h i proper>es%we%can% • Bayes:%compute%the%posterior'probability% p(µ x ) predict%the% |{ i} Sampling' • DistribuFon'(the% Frequen>st:%devise%an%esFmator%%%%%%for%µˆ μ.%Ideally%it% probability%of% should%be%unbiased,%%so%%%%%%%%%%%%%%%%%%and%have%as%small%an%µˆ = µ obtaining%a%general% h i set%of%data).%% error%as%possible%(minimum'variance).% The%Likelihood% refers%to%the% • These%lead%to%superficially%iden>cal%results%(although% specific%data%we% they%aren’t),%but%the%interpreta>on%is%very%different% have)%P%it%isn’t%a% probability,%strictly. Note:%this%is%just%the%expecta>on%value%of%x;%% • %%%%%%%%Bayesian:%no%es>mators%P%just%posteriors the%distribu>on%is%needed 18 Set%up%the%problem State%your%priors • What%is%the%model%for%the%data,%M?%%% • In%easy%cases,%the%effect%of%the%prior%is%simple% • As%experiment%gathers%more%data,%the%likelihood%tends%to%get% • M:'x'='μ'+'n%% narrower,%and%the%influence%of%the%prior%diminishes% • • Data:%a%set%of%values%{xi},%i=1…N% Rule%of%thumb:%if%changing%your%prior†%to%another%reasonable% one%changes%the%answers%a%lot,%you%could%do%with%more%data% 2 2 • Prior%info'I:%noise%<n>=0%<n >=σ %(known);% • Reasonable%priors?%Noninforma>ve*%–%constant%prior% gaussian%distributed% • scale%parameters%in%%%%%%%%%%%%%%%;%uniform%in%log%of%parameter% (Jeffreys’%prior*)% • θ:%the%mean,%μ% • Beware:%in%more%complicated,%mul>dimensional%cases,%your% • Rule%1:%what%do%we%want?%% prior%may%have%subtle%effects…% ! • p(μ%|%{xi}%)% †%I%mean%the%raw%theore>cal%one,%not%modified%by%an%experiment% *%Actually,%it’s%be;er%not%to%use%these%terms%–%other%people%use%them%to%mean%different% • See%Jonathan’s%lectures%for%the%solu>on things%–%just%say%what%your%prior%is! 19 From%Sivia%&%Skilling’s%Data'Analysis'book.%%%IS%THE%COIN%FAIR?% The%effect%of%priors Priors%=%“It’s%likely%to%be%nearly%fair”,%“It’s%likely%to%be%very%unfair” Model:%independent%throws%of%coin.%%Parameter%θ%=%probability%of%H p(θ|H) p(θ|HH) Uniform%Prior p(θ|HHT) p(θ|HHTT) 21 Sivia%&%Skilling 22 • VSA%CMB%% VSA%posterior experiment% (Slosar%et%al%2003) Priors:%%ΩΛ≥0%% 10%≤%age%≤%20%Gyr h%≈%0.7%±%0.1 There%are%no%data%in% these%plots%–%it%is%all% coming%from%the%prior! p(✓1)= d✓j=1 p(x ✓) p(✓) 6 | Z Inferring%the%parameter(s) Errors • What%to%report,%when%you%have%the%posterior?% • If%we%assume%uniform%priors,%then%the%posterior%is%propor>onal%to%the%likelihood.%%% • Commonly%the%mode%is%used%(the%peak%of%the% ! posterior)% If%further,%we%assume%that%the%likelihood%is%singlePmoded%(one%peak%at%%%%%%%)%,%we%can%✓0 make%a%Taylor%expansion%of%lnL:% • ! Mode%=%Maximum'Likelihood'EsFmator,'if'the'priors' 1 ⌅2 ln L ! ln L(x; θ)=lnL(x; θ0)+ (θα θ0α) (θ⇥ θ0⇥)+... are'uniform' ! 2 − ⌅⇤↵⌅⇤β − 1 ! L(x; θ)=L0 exp (θ↵ θ0↵)H↵(θβ θ0β)+... • The%posterior'mean'may%also%be%quoted,%but% ! − 2 − − beware% where%the%Hessian%matrix%is%defined%by%these%equa>ons.%%Comparing%this%with%a%⇥ ⇤ gaussian,%the%condiFonal'error'(keeping%all%other%parameters%fixed)%is% ! 1 • Ranges%containing%x%%of%the%posterior%probability%of% σα = ! pH↵↵ the%parameter%are%called%credibility'intervals%(or% Marginalising%over%all%other%parameters%gives%the%marginal'error 1 Bayesian'confidence'intervals) σ↵ = (H− )↵↵ p 26 Mul>modal%posteriors%etc NonPgaussian%likelihoods:%number% • Peak%may%not%be%gaussian% counts • Mul>modal?%Characterising%it% • A%radio%source%is%observed%with%a%telescope%which%can% by%a%mode%and%an%error%is% detect%sources%with%fluxes%above%S0.%The%radio%source% probably%inadequate.%%May% has%a%flux%S1%=%2S0%(assume%it%is%precisely%measured).% What%is%the%slope%of%the%number%counts?%% have%to%present%the%full% From%BPZ Pα posterior.% (Assume%N(S)dS%∝%S %dS)% • Mean%posterior%may%not%be% ! Bruzual & Charlot Can%you%tell%anything? useful%in%this%case%–%it%could% be%very%unlikely,%if%it%is%a%valley% between%2%peaks.