Cosmology in the Machine Learning Era Francisco (Paco) Villaescusa-Navarro

Waterloo Center for October/21/2020 Outline

• Motivation

• Small scales: benefits and problems

• A machine learning approach

• Our vision Motivation: the LCDM model

{Ω, Ω, ℎ, �, �, �, �, �, � … } Goal: smallest error on the parameters

Ω ± �Ω Ω ± �Ω Answer fundamental questions: ℎ ± �ℎ

� ± �� 1) What is the nature of ?

� ± �� 2) What are the neutrino masses? � ± �� 3) How fast is the Universe expanding? � ± �� 4) What are the properties of the Universe initial conditions?

� ± �� 5) … � ± �� Parameter inference

Observations Theory

�"(�) �!(�|�⃗)

What summary statistics shall we use to determine �⃗ with the smallest error? Parameter inference: summary statistics

Gaussian density field Fully described by the power spectrum

Non-Gaussian density field Mathematically “intractable” P(k), B(k), peaks, voids, … Summary • Goal: constrain cosmological parameters with the highest accuracy Parameter inference: information content

How well can we constraint some parameters �⃗ = {Ω, Ω, ℎ, �, �, �} given some observables �⃗ = {P k , P k , P k … P k , … }?

��⃗ ��⃗ F = � �� �� Fisher matrix

�� ≥ � The Quijote Simulations (https://github.com/franciscovillaescusa/Quijote-simulations)

• A set of 44,100 full N-body simulations; largest set to-date

• More than 7,000 in {Wm, Wb, h, ns, s8, Mn, w} hyperplane

• More than 50 trillion particles over a volume larger than entire

• Billions of halos and voids identified

• 35 Million CPU hours; 1 Petabyte of data

• Everything publicly available The Quijote Simulations: team

ChangHoon Hahn Elena Massara Arka Banerjee Ana M. Delgado Doogesh Ramanah Tom Charnock Elena Giusarma Yin Li Erwan Allys Antoine Brochard Cora Uhlemann (Berkeley) (Flatiron) (Stanford) (Flatiron) (IAP, Paris) (IAP, Paris) (Flatiron/MTech) (IPMU/Berkeley) (Ecole Normale) (Ecole Normale) (Cambridge)

Chi-Ting Chiang Siyu He Alice Pisani Andrej Obuljen Yu Feng Emanuele Castorina Gabriella Contardo Christina Kreisch Andrina Nicola Justin Alsing (BNL) (Flatiron) (Princeton) (Waterloo) (Berkeley) (Berkeley) (Flatiron) (Princeton) (Princeton) (Oskar Klein)

Roman Scoccimarro Licia Verde Matteo Viel Shirley Ho Stephane Mallat Ben Wandelt (NYU) (Barcelona) (SISSA) (Flatiron/Princeton) (College de France) (IAP, Paris) (Flatiron/Princeton) Information content: Pk FVN et al. 2019 Information content: Pk + HMF + VSF

FVN et al. (in prep) Information content

FVN et al. (in prep) Massara, FVN et al. 2020 Information content

Hahn, FVN, Castorina, Scoccimarro 2019 Uhlemann, Friedrick, FVN, et al. 2019 Summary • Goal: constrain cosmological parameters with the highest accuracy

• Lots of information on small scales and beyond P(k) Let’s go to small scales…

Warning: Baryonic effects

Conservative solution: Marginalize them Neural networks

Temperature, Rain humidity, probability pressure, wind speed…

Output = �(input) cat

{Ω!, Ω", ℎ, �#, �$, �%} Neural networks

ℎ = �(�� + �� + �)

ℎ

� � = �(ℎ� + ℎ� + ℎ��)

ℎ �

�

� if x > 0 �(�) = 0 otherwise

ℎ

ℎ = �(�� + �� + �) Toy Example

FVN et al. 2020b • Neural networks can extract ALL information

• Neural networks can marginalize over baryonic effects

• Neural networks can extract cosmological information embedded in the regime dominated by astrophysics Toy Example: Gaussian density fields

� � = �/ �

A is the cosmological parameter

Goal: infer value of A from the map itself Toy Example: Gaussian density fields

Neural � Network Toy Example: Gaussian density fields

Input 100,000 maps with A=1

Error on A can be calculated with Fisher matrix

2 � = � � Toy Example

• Neural networks can extract ALL information

• Neural networks can marginalize over baryonic effects

• Neural networks can extract cosmological information embedded in the regime dominated by astrophysics Toy Example: Gaussian density fields

Baryonic effects Toy Example: Gaussian density fields

Neural � Network Toy Example: Gaussian density fields

Input 100,000 maps with

A=1 & kpivot=0.5 h/Mpc

Error on A can be calculated with Fisher matrix

2 � = � � Toy Example

• Neural networks can extract ALL information

• Neural networks can marginalize over baryonic effects

• Neural networks can extract cosmological information embedded in the regime dominated by astrophysics Toy Example � = �� �

� = 1.1×10

� = 1.7×10

� = 1.5×10 Toy Example

• Neural networks can extract ALL information

• Neural networks can marginalize over baryonic effects

• Neural networks can extract cosmological information embedded in the regime dominated by astrophysics Machine Learning Approach

• Train neural networks to extract the maximum cosmological information from the 3D field; no need of summary statistics

• Marginalize over baryonic effects; show many examples with different baryonic effects and Astrophysics with MachinE Learning Simulations

• A suite of 4,233 simulations

• 2,049 N-body; Gadget-III

• 2,184 state-of-the-art (magneto-)hydrodynamic sims

• AREPO/IllustrisTNG + GIZMO/SIMBA

• More than 100 billion resolution elements over combined volume of ~(400 Mpc/h)3

• More than 4,000 cosmologies & astrophysics models

• More than 140,000 snapshots

• Designed for machine learning applications The CAMEL simulations From SFRH to parameters

{Ω#, �$, �%, �&, �', �(} = �(����(�)) From parameters to SFRH

�F�� = �(Ω#, �$, �%, �&, �', �() Symbolic regression

�F�� = �(Ω#, �$, �%, �&, �', �() Generative models: GANS Generative models: GANS Generative models: GANS Autoencoders Autoencoders: anomalies Vision/Dream

Quijote Super Resolution Thousands of cosmologies 10 h M⊙

CAMELS Likelihood-free inference Thousands of astrophysics Extract all information. Marginalize over baryonic effects models

input cnn 1 cnn 2 cnn 3 cnn 4 cnn 5 fcl 1 fcl 2 fcl 3 output

64 x 64 x 64 x 1

250 250 250 32 x 32 x 32 x 4 17 x 17 x 17 x 8 5 LeRelu LeRelu BN, BN, 9 x 9 x 9 x 32 5 x 5 x 5 x 64 3 x 3 x 3 x 128 LeRelu LeRelu BN, BN, BN, , Drop Out 0.5, Drop Out 0.5 LeRelu LeRelu LeRelu , Avg. Pool Conclusions

• Goal: get the tighter constraints on the parameters

• QUIJOTE: 44,100 N-body sims; perfect for large scales

• CAMELS: 4,233 N-body + hydro sims; perfect for small scales

• Dream: Merge Quijote with CAMELS. Extract ALL information down to k = 30 h/Mpc