DECOMPOSITION OF THE MAIN THREAD
THROUGHPUT
1 HELLO! I am Nikolay Matvienko JS Developer at Grid Dynamics You can find me at twitter.com/ github.com/
2 WEB UI BACKEND ORCHESTRATION LAYER MICROSERVICES Node.js
3 ADDING NEW FEATURE
MAIN THREAD MAIN THREAD
4
WHATEVER IS FAST TODAY IS SLOW TOMORROW AS DEMANDS CAN ONLY GO UP
5 BLOCKED EVENT LOOP
INCOMING REQUESTS HTTP response
DB response USERS THAT ARE MODULE response WAITING FOR RESPONSE HTTP response 6 IN QUEUE
7 BIG LATENCY HROUGHPUT
8 SCALING
- CLUSTER module WEB SERVER / REVERSE PROXY - PM2
WEB SERVER / REVERSE PROXY - PHUSION PASSENGER APP APP APP APP - NGINX worker worker worker worker
APP APP FREE FREE worker worker CORE CORE
9 LOAD BALANCING 1. 2. PROCESS WEB SERVER / REVERSE PROXY MANAGER
APP APP worker worker
FREE FREE CORE CORE
10 RESPONSE TIME
11 DISPERSION
GARBAGE COLLECTION
COMPUTATIONS FRAMEWORK LOGGING
PROFILLER METRICS COLLECTION
Server-Side RENDERING
12
.... TOO MANY REQUESTS ARE HANDLED IN NODE.JS 13 GARBAGE
COLLECTION
PROFILLER
14 “THE WORLD IS MINE”
© GARBAGE COLLECTOR 1959
15 GARBAGE COLLECTION
MAIN THREAD MAIN THREAD
STOP THE SERVER STOP THE WORLD
16 INCREMENTAL COLLECTION
MAIN THREAD
LESS PERFORMANCE IMPACT
17 GC DECOMPOSITION
MAIN THREAD
1. PARALLEL MARK-SWEEP
2. PARALLEL execution SCAVENGER
3. PARALLEL mark overhead MARK-EVACUATE evacuate overhead 18 … 50 years later “THE WORLD IS MINE NOW”
© JS COMPUTING OPERATION IN NODE.JS 2009
19 DEMO APP ARCHITECTURE
CHOOSE APP CONFIGU RATION LOGIC
20
CURRENT : 114 REQ/S THROUGHPUT COMPUTATIONS
PROFILLER
21 CPU-BOUND TASKS
MAIN THREAD
INCOMING LOGIC IN CB REQUESTS REDIS response
REDIS response
HTTP response 22 HOW TO PERFORM?
1. Child Process fork 2. Threads/Workers 3. Microservices libraries 4. Native Modules
23 PROCESS POOL
of W workers
APP W workers
W CHILD CHILD PROCESS PROCESS tasks
24 CLUSTER Node.js Cluster
IN-PROCESS WEB WEB WEB WEB SERVER SERVER SERVER SERVER COMPUTING APP APP APP APP
PARALLEL OFF-PROCESS
COMPUTING CPU CPU CPU CPU TASKS TASKS TASKS TASKS WORKER WORKER WORKER WORKER
25 CPU TASKS PARALLELIZATION RESULT 3X 338 MORE REQ/S WITH OFF-PROCESS 114 (PARALLEL) JS COMPUTATION 26 FRAMEWORK
CURRENT : 338 REQ/S THROUGHPUT FRAMEWORK
PROFILLER
27 FRAMEWORK
~ 1.5 – 2X SLOWLY HAPI, EXPRESS, RESTIFY…
Node.js HTTP SERVER
28 OPTIMIZATIONS
FRAMEWORK
Node.js HTTP SERVER
29 LONG SERIALIZATION
30 FRAMEWORK CHANGE RESULT
440 +30% 338 MORE REQ/S WITH FASTIFY
31 LOGGING
CURRENT 440 REQ/S THROUGHPUT: LOG PROFILLER
32 LOGGING MAIN THREAD
LOGGERS: MESSAGE 1. Winston MESSAGE 2. Banyan LOGIC 3. Morgan and others 33 OFF-PROCESS LOGGER TRANSPORT MAIN THREAD APP
LOGGERS:
1. Pino MAIN THREAD LOGGER TRANSPORT 2. Roarr
34 PERFORMANCE OVERHEAD:
APP STORE Node.js
LOGGER
PERFORMANCE OVERHEAD:
LOGGER APP TRANSPORT STORE Node.js Node.js LOGGER 35 CLUSTER
WEB WEB WEB WEB SERVER SERVER SERVER SERVER
process.stdout
LOGGER LOGGER LOGGER LOGGER
Elastic search 36 OFF-PROCESS LOGGING RESULT
518 +17% 440 MORE REQ/S 385 WITH OFF-PROCESS LOGGER TRANSPORT
37 APPLICATION PERFORMANCE MONITORING
CURRENT 518 REQ/S THROUGHPUT:
PROFILLER APM
38 APPLICATION PERFORMANCE MONITORING
MAIN THREAD
APM vendors/agents: 1. NewRelic METRICS COLLECTION 2. Dynatrace AGGREGATION 3. OpenTracing TRANSPORT 4. node-measured
39 IN-PROCESS Cluster/Load balancer APM
AGENT WEB WEB WEB WEB SERVER SERVER SERVER SERVER WORKER WORKER WORKER WORKER THE APM AGENT PROBLEMS ARE APPLICATION PROBLEMS APM APM APM APM
Analytics & Time series Monitoring SaaS DB Dashboard
40 OFF-PROCESS APM AGENT
Message queue
.send() .on()
WEB WEB WEB WEB SERVER SERVER SERVER SERVER APP APP APP APP
Analytics & Time series Message Monitoring DB Queue Dashboard
41 OFF-PROCESS MONITORING RESULT
+25% 652 MORE REQ/S 518 WITH OFF-PROCESS METRIC AGENT
42
CURRENT 652 REQ/S THROUGHPUT:
PROFILLER SSR
43 SERVER-SIDE RENDERING
MAIN THREAD
INCOMING RENDERING REQUESTS REDIS response
REDIS response
HTTP response 44 STREAMING SERVER-SIDE RENDERING renderStream.pipe(res, { end: 'false' }); renderStream.on('end', () => Asynchronous execution in {response.end('