Improving Numerical Integration and Event Generation with Normalizing Flows — HET Brown Bag Seminar, University of Michigan

Home , Numerical integration

Improving Numerical Integration and Event Generation with Normalizing Flows — HET Brown Bag Seminar, University of Michigan —

Claudius Krause

Fermi National Accelerator Laboratory

September 25, 2019

In collaboration with: Christina Gao, Stefan H¨oche,Joshua Isaacson arXiv: 191x.abcde

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 1 / 27 Monte Carlo Simulations are increasingly important.

https://twiki.cern.ch/twiki/bin/view/AtlasPublic/ComputingandSoftwarePublicResults

MC event generation is needed for signal and background predictions. ⇒ The required CPU time will increase in the next years. ⇒ Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 2 / 27 Monte Carlo Simulations are increasingly important.

106

3 10− parton level W+0j 105 particle level W+1j 10 4 W+2j particle level − W+3j 104 WTA (> 6j) W+4j 5 10− W+5j 3 W+6j 10 Sherpa MC @ NERSC

Mevt W+7j

/ 6 Sherpa / Pythia + DIY @ NERSC 10− W+8j 2

10 Frequency W+9j CPUh 7 10− 101

8 10− + 100 W +jets, LHC@14TeV

pT,j > 20GeV, ηj < 6 9 | | 10− 1 10− 0 50000 100000 150000 200000 250000 300000 0 1 2 3 4 5 6 7 8 9 Ntrials Njet Stefan H¨oche,Stefan Prestel, Holger Schulz [1905.05120;PRD]

The bottlenecks for evaluating large ﬁnal state multiplicities are

a slow evaluation of the matrix element a low unweighting eﬃciency

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 3 / 27 Monte Carlo Simulations are increasingly important.

106

3 10− parton level W+0j 105 particle level W+1j 10 4 W+2j particle level − W+3j 104 WTA (> 6j) W+4j 5 10− W+5j 3 W+6j 10 Sherpa MC @ NERSC

Mevt W+7j

/ 6 Sherpa / Pythia + DIY @ NERSC 10− W+8j 2

10 Frequency W+9j CPUh 7 10− 101

8 10− + 100 W +jets, LHC@14TeV

pT,j > 20GeV, ηj < 6 9 | | 10− 1 10− 0 50000 100000 150000 200000 250000 300000 0 1 2 3 4 5 6 7 8 9 Ntrials Njet Stefan H¨oche,Stefan Prestel, Holger Schulz [1905.05120;PRD]

The bottlenecks for evaluating large ﬁnal state multiplicities are

a slow evaluation of the matrix element a low unweighting eﬃciency

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 3 / 27 Improving Numerical Integration and Event Generation with Normalizing Flows

Part I: The “traditional” approach

Part II: The Machine Learning approach

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 4 / 27 Z F = f (~x) d D x ⇒ Z σ = dσ(pi , ϑi , ϕi ), D = 3nﬁnal 4 ⇒ −

Given a distribution f (~x), how can we sample according to it?

? = ⇒

I: There are two problems to be solved. . .

f (~x)

dσ(pi , ϑi , ϕi )

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 5 / 27 Given a distribution f (~x), how can we sample according to it?

I: There are two problems to be solved. . .

Z f (~x) F = f (~x) d D x ⇒ Z dσ(pi , ϑi , ϕi ) σ = dσ(pi , ϑi , ϕi ), D = 3nﬁnal 4 ⇒ −

? = ⇒

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 5 / 27 I: There are two problems to be solved. . .

Z f (~x) F = f (~x) d D x ⇒ Z dσ(pi , ϑi , ϕi ) σ = dσ(pi , ϑi , ϕi ), D = 3nﬁnal 4 ⇒ −

Given a distribution f (~x), how can we sample according to it?

? = ⇒

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 5 / 27 We need a fast and eﬀective numerical integration! ⇒

I: . . . but they are closely related.

1 Starting from a pdf, . . . 2 . . . we can integrate it and ﬁnd its cdf, . . . 3 . . . to ﬁnally use its inverse to transform a uniform distribution.

1 2 3

⇒ ⇒

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 6 / 27 I: . . . but they are closely related.

1 Starting from a pdf, . . . 2 . . . we can integrate it and ﬁnd its cdf, . . . 3 . . . to ﬁnally use its inverse to transform a uniform distribution.

1 2 3

⇒ ⇒

We need a fast and eﬀective numerical integration! ⇒

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 6 / 27 We therefore have to ﬁnd a q(x) that approximates the shape of f (x). is “easy” enough such that we can sample from its inverse cdf.

I: Importance Sampling is very eﬃcient for high-dimensional integration.

Z 1 MC 1 X f (x) dx f (xi ) xi ... uniform −−→ N 0 i

Z 1 f (x) MC 1 X f (xi ) = q(x)dx xi ... q(x) q(x) −−−−−−−−−−−−→importance sampling N q(xi ) 0 i

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 7 / 27 I: Importance Sampling is very eﬃcient for high-dimensional integration.

Z 1 MC 1 X f (x) dx f (xi ) xi ... uniform −−→ N 0 i

Z 1 f (x) MC 1 X f (xi ) = q(x)dx xi ... q(x) q(x) −−−−−−−−−−−−→importance sampling N q(xi ) 0 i

We therefore have to ﬁnd a q(x) that approximates the shape of f (x). is “easy” enough such that we can sample from its inverse cdf.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 7 / 27 If q(x) f (x), all events would have the same weight as the distribution∝ reproduces f (x) directly.

# accepted events mean w We deﬁne the Unweighting Eﬃciency = # all events = max w ,

p(xi ) f (xi ) with wi = = . q(xi ) Fq(xi )

I: The unweighting eﬃciency measures the quality of the approximation q(x).

If q(x) were constant, each event xi would require a weight of f (xi ) to reproduce the distribution of f (x). “Weighted Events” ⇒

To unweight, we need to accept/reject each event with probability f (xi ) max f (x) . The resulting set of kept events is unweighted and reproduces the shape of f (x).

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 8 / 27 # accepted events mean w We deﬁne the Unweighting Eﬃciency = # all events = max w ,

p(xi ) f (xi ) with wi = = . q(xi ) Fq(xi )

I: The unweighting eﬃciency measures the quality of the approximation q(x).

If q(x) were constant, each event xi would require a weight of f (xi ) to reproduce the distribution of f (x). “Weighted Events” ⇒

To unweight, we need to accept/reject each event with probability f (xi ) max f (x) . The resulting set of kept events is unweighted and reproduces the shape of f (x).

If q(x) f (x), all events would have the same weight as the distribution∝ reproduces f (x) directly.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 8 / 27 I: The unweighting eﬃciency measures the quality of the approximation q(x).

If q(x) were constant, each event xi would require a weight of f (xi ) to reproduce the distribution of f (x). “Weighted Events” ⇒

To unweight, we need to accept/reject each event with probability f (xi ) max f (x) . The resulting set of kept events is unweighted and reproduces the shape of f (x).

If q(x) f (x), all events would have the same weight as the distribution∝ reproduces f (x) directly.

# accepted events mean w We deﬁne the Unweighting Eﬃciency = # all events = max w ,

p(xi ) f (xi ) with wi = = . q(xi ) Fq(xi )

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 8 / 27 1.0

0.8 It does have problems if the features are not aligned with the coordinate axes. 0.6 0.4 The current python implementation also uses 0.2 stratiﬁed sampling. 0.0 0.0 0.2 0.4 0.6 0.8 1.0

I: The VEGAS algorithm is very eﬃcient.

The VEGAS algorithm Peter Lepage 1980 assumes the integrand factorizes and bins the 1-dim projection. then adapts the bin edges such that area of each bin is the same.

= ⇒

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 9 / 27 I: The VEGAS algorithm is very eﬃcient.

The VEGAS algorithm Peter Lepage 1980 assumes the integrand factorizes and bins the 1-dim projection. then adapts the bin edges such that area of each bin is the same.

= ⇒

1.0

0.8 It does have problems if the features are not aligned with the coordinate axes. 0.6 0.4 The current python implementation also uses 0.2 stratiﬁed sampling. 0.0 0.0 0.2 0.4 0.6 0.8 1.0

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 9 / 27 Improving Numerical Integration and Event Generation with Normalizing Flows

Part I: The “traditional” approach

Part II: The Machine Learning approach

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 10 / 27 Part II: The Machine Learning approach

Part II.1: Neural Network Basics

Part II.2: Numerical Integration with Neural Networks

Part II.3: Examples

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 11 / 27 II.1: Neural Networks are nonlinear functions, inspired by the human brain.

Each neuron transforms the input with a weight W and a bias ~b.

xi σ(W ~x + b) W ~x + ~b

The activation function σ makes it nonlinear.

“rectiﬁed linear unit (relu)” “leaky relu” “sigmoid”

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 12 / 27 II.1: The Loss function quantiﬁes our goal.

We have two choices: Kullback-Leibler (KL) divergence: R p(x) 1 P p(xi ) p(xi ) DKL = p(x) log dx log , xi ... q(x) q(x) ≈ N q(xi ) q(xi ) Pearson χ2 divergence: 2 2 R (p(x) q(x)) 1 P p(xi ) Dχ2 = − dx 2 1, xi ... q(x) q(x) ≈ N q(xi ) −

They give the gradient that is needed for the optimization:

(1 or 2) 1 X p(xi ) θD(KL or χ2) θ log q(xi ), xi ... q(x) ∇ ≈ −N q(xi ) ∇

We use the ADAM optimizer for stochastic gradient descent: The learning rate for each parameter is adapted separately, but based on previous iterations.

This is eﬀective for sparse and noisy functions. Kingma/Ba [arXiv:1412.6980]

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 13 / 27 Part II: The Machine Learning approach

Part II.1: Neural Network Basics

Part II.2: Numerical Integration with Neural Networks

Part II.3: Examples

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 14 / 27 The Jacobian is needed to evaluate the loss, the integral, and to sample. However,⇒ it scales as (n3) and is too costly for high-dimensional integrals! O

II.2: Using the NN as coordinate transform is too costly. We could use the NN as nonlinear coordinate transform:

We use a deep NN with ndim nodes in the ﬁrst and last layer to map a uniformly distributed x to a target q(x). The distribution induced by the map y(x) (=NN) is given by the Jacobian of the map: 1 q(y) = q(y(x)) = ∂y − ∂x Klimek/Perelstein [arXiv:1810.11509]

Jacobian −−−−→ y = x 2 1 ∂y − = 1 ∂x 2x

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 15 / 27 II.2: Using the NN as coordinate transform is too costly. We could use the NN as nonlinear coordinate transform:

Jacobian −−−−→ y = x 2 1 ∂y − = 1 ∂x 2x

The Jacobian is needed to evaluate the loss, the integral, and to sample. However,⇒ it scales as (n3) and is too costly for high-dimensional integrals! O

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 15 / 27 The idea was introduced as “Nonlinear Independent Component Estimation” (NICE) in Dinh et al. [arXiv:1410.8516]. In Rezende/Mohamed [arXiv:1505.05770], Normalizing Flows were first discussed with planar and radial flows. Our approach follows the ideas of Mülleret al. [arXiv:1808.03856], but with the modifications of Durkan et al. [arXiv:1906.04032]. Our code uses TensorFlow 2.0-beta, www.tensorflow.org.

II.2: Normalizing Flows are numerically cheaper.

A Normalizing Flow: is a deterministic, bijective, smooth mapping between two statistical distributions. is composed of a series of easy transformations, the “Coupling Layers”. is still ﬂexible enough to learn complicated distributions. The NN does not learn the transformation, but the parameters of a series⇒ of easy transformations.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 16 / 27 II.2: Normalizing Flows are numerically cheaper.

The idea was introduced as “Nonlinear Independent Component Estimation” (NICE) in Dinh et al. [arXiv:1410.8516]. In Rezende/Mohamed [arXiv:1505.05770], Normalizing Flows were first discussed with planar and radial flows. Our approach follows the ideas of Mülleret al. [arXiv:1808.03856], but with the modifications of Durkan et al. [arXiv:1906.04032]. Our code uses TensorFlow 2.0-beta, www.tensorflow.org.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 16 / 27 II.2: The Coupling Layer is the fundamental Building Block.

x y NN permutation

xB C(xB ; m(xA)) forward: The C are numerically cheap, invertible, and yA = xA separable in xB,i .

yB,i = C(xB,i ; m(xA)) Jacobian: inverse: ∂y 1 ∂C ∂C(x ; m(x )) ∂xA B,i A = ∂C = Πi xA = yA x 0 x ∂ ∂xB ∂ B,i 1 xB,i = C − (yB,i ; m(xA)) (n) ⇒ O

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 17 / 27 b 1 X− C = Qk + αQb k=1 x (b 1)w α = − w− C ∂ Qbi = Πi w ∂xB

rational quadratic spline coupling function: Durkan et al. [arXiv:1906.04032] Gregory/Delbourgo [IMA Journal of Numerical Analysis, ’82] cdf

2 still rather easy a2α + a1α + a0 C = 2 b2α + b1α + b0 more ﬂexible

The NN predicts the cdf bin widths, heights, and derivatives that go in ai &bi .

II.2: The Coupling Function is a piecewise approximation to the cdf. piecewise linear coupling function: M¨ulleret al. [arXiv:1808.03856]

pdf cdf

The NN predicts the pdf bin heights Qi .

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 18 / 27 rational quadratic spline coupling function: Durkan et al. [arXiv:1906.04032] Gregory/Delbourgo [IMA Journal of Numerical Analysis, ’82] cdf

2 still rather easy a2α + a1α + a0 C = 2 b2α + b1α + b0 more ﬂexible

The NN predicts the cdf bin widths, heights, and derivatives that go in ai &bi .

II.2: The Coupling Function is a piecewise approximation to the cdf. piecewise linear coupling function: M¨ulleret al. [arXiv:1808.03856] b 1 X− pdf cdf C = Qk + αQb k=1 x (b 1)w α = − w− C ∂ Qbi = Πi w ∂xB The NN predicts the pdf bin heights Qi .

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 18 / 27 II.2: The Coupling Function is a piecewise approximation to the cdf. piecewise linear coupling function: M¨ulleret al. [arXiv:1808.03856] b 1 X− pdf cdf C = Qk + αQb k=1 x (b 1)w α = − w− C ∂ Qbi = Πi w ∂xB The NN predicts the pdf bin heights Qi . rational quadratic spline coupling function: Durkan et al. [arXiv:1906.04032] Gregory/Delbourgo [IMA Journal of Numerical Analysis, ’82] cdf

2 still rather easy a2α + a1α + a0 C = 2 b2α + b1α + b0 more ﬂexible

The NN predicts the cdf bin widths, heights, and derivatives that go in ai &bi .

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 18 / 27 II.2: We need (log n) Coupling Layers. O

How many Coupling Layers do we need? Enough to learn all correlations between the variables. As few as possible to have a fast code.

This depends on the applied permutations and the xA xB -splitting: (pppttt) (tttppp) vs. (pppptt) (ppttpp) (ttpppp)− ↔ ↔ ↔

More pass-through dimensions (p) means more points required for accurate loss. Fewer pass-through dimensions means more CLs needed.

For #p #t, we can prove: 4 #CLs 2 log ndim ≈ ≤ ≤ 2

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 19 / 27 There are diﬀerent ways to encode the input dimensions xA. For example xA = (0.2, 0.7):

direct: xi = (0.2, 0.7)

one-hot (8 bins): xi = ((0, 1, 0, 0, 0, 0, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0))

one-blob (8 bins): xi = ((0.55, 0.99, 0.67, 0.16, 0.01, 0, 0, 0), (0, 0, 0.01, 0.11, 0.55, 0.99, 0.67, 0.16)) M¨ulleret al. [arXiv:1808.03856]

II.2: We utilize diﬀerent NN architectures. Available Architectures: M¨ulleret al. [arXiv:1808.03856] “Fully Connected” Neural Net (NN): “U-shaped” Neural Net (Unet):

Input Layer Input Layer

Dense Layer with 64 nodes Dense Layer with 128 nodes

Dense Layer with 64 nodes Dense Layer with 64 nodes

Dense Layer with 64 nodes DL w/ 32 nodes

Dense Layer with 64 nodes Dense Layer with 64 nodes

Dense Layer with 64 nodes Dense Layer with 128 nodes

Output Layer Output Layer

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 20 / 27 II.2: We utilize diﬀerent NN architectures. Available Architectures: M¨ulleret al. [arXiv:1808.03856] “Fully Connected” Neural Net (NN): “U-shaped” Neural Net (Unet):

Input Layer Input Layer

Dense Layer with 64 nodes Dense Layer with 128 nodes

Dense Layer with 64 nodes Dense Layer with 64 nodes

Dense Layer with 64 nodes DL w/ 32 nodes

Dense Layer with 64 nodes Dense Layer with 64 nodes

Dense Layer with 64 nodes Dense Layer with 128 nodes

Output Layer Output Layer

There are diﬀerent ways to encode the input dimensions xA. For example xA = (0.2, 0.7):

direct: xi = (0.2, 0.7)

one-hot (8 bins): xi = ((0, 1, 0, 0, 0, 0, 0, 0), (0, 0, 0, 0, 0, 1, 0, 0))

one-blob (8 bins): xi = ((0.55, 0.99, 0.67, 0.16, 0.01, 0, 0, 0), (0, 0, 0.01, 0.11, 0.55, 0.99, 0.67, 0.16)) M¨ulleret al. [arXiv:1808.03856]

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 20 / 27 Part II: The Machine Learning approach

Part II.1: Neural Network Basics

Part II.2: Numerical Integration with Neural Networks

Part II.3: Examples

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 21 / 27 II.3: The 4-d Camel function illustrates the learning of the NN. Our test function: 2 Gaussian peaks, randomly placed in a 4d space. Target Distribution: Before training:

Final Integral: 0.0063339(41) VEGAS plain: 0.0063349(92) VEGAS full: 0.0063326(21) Trained eﬃciency: 14.8 % Untrained eﬃciency: 0.6 %

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 22 / 27 II.3: The 4-d Camel function illustrates the learning of the NN. Our test function: 2 Gaussian peaks, randomly placed in a 4d space. Target Distribution: After 5 epochs:

Final Integral: 0.0063339(41) VEGAS plain: 0.0063349(92) VEGAS full: 0.0063326(21) Trained eﬃciency: 14.8 % Untrained eﬃciency: 0.6 %

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 22 / 27 II.3: Sherpa needs a high-dimensional integrator.

Sherpa is a Monte Carlo event generator for the Simulation of High-Energy Reactions of PArticles. We use Sherpa to map the unit-hypercube of our integration domain to momenta and angles. To improve eﬃciency, Sherpa uses a recursive multichannel algorithm.

ndim = 3nfinal 4 + nfinal 1 ⇒ | {z − } | {z− } kinematics multichannel compute the matrix element of the process. The COMIX++ ME-generator uses color-sampling, so we need to integrate over final state color configurations, too.

ndim = 4nﬁnal 3 + 2(ncolor ) ⇒ − https://sherpa.hepforge.org/

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 23 / 27 + II.3: Already in e e− 3j we are more eﬀective. →

spectator of q color ← Target distribution q color ← spectator of g color ← g color ← propagator of decaying fermion ← cos ϑ of decaying fermion with beam ← ϕ of decaying fermion with beam ← cos ϑ of decay ← ϕ of decay ←

multichannel ←

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 24 / 27 + II.3: Already in e e− 3j we are more eﬀective. →

spectator of q color ← Learned distribution q color ← spectator of g color ← g color ← propagator of decaying fermion ← cos ϑ of decaying fermion with beam ← ϕ of decaying fermion with beam ← cos ϑ of decay ← ϕ of decay ←

multichannel ←

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 25 / 27 + II.3: Already in e e− 3j we are more eﬀective. →

weight distribution Sherpa Our code

σour code = 4887.1 4.6pb σSherpa = 4877.0 17.7pb ± ± unweighting eﬃciency = 12.9% unweighting eﬃciency = 2.8%

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 26 / 27 I presented the idea of Normalizing Flows. I discussed their superiority for large integration dimensions.

I showed the results of two diﬀerent examples + In e e− 3j, we “beat” Sherpa by roughly a factor of 5. →

Improving Numerical Integration and Event Generation with Normalizing Flows

I summarized the concepts of numerical integration and the “traditional” VEGAS algorithm. I introduced Neural Networks as versatile nonlinear functions.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 27 / 27 I showed the results of two diﬀerent examples + In e e− 3j, we “beat” Sherpa by roughly a factor of 5. →

Improving Numerical Integration and Event Generation with Normalizing Flows

I summarized the concepts of numerical integration and the “traditional” VEGAS algorithm. I introduced Neural Networks as versatile nonlinear functions.

I presented the idea of Normalizing Flows. I discussed their superiority for large integration dimensions.

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 27 / 27 Improving Numerical Integration and Event Generation with Normalizing Flows

I summarized the concepts of numerical integration and the “traditional” VEGAS algorithm. I introduced Neural Networks as versatile nonlinear functions.

I presented the idea of Normalizing Flows. I discussed their superiority for large integration dimensions.

I showed the results of two diﬀerent examples + In e e− 3j, we “beat” Sherpa by roughly a factor of 5. →

Claudius Krause (Fermilab) Machine Learning Phase Space September 25, 2019 27 / 27