END-TO-END JAVASCRIPT WITH THE STACK By João Silva for the CERN Spring Campus 2014 University of Oviedo, Gijón, Spain WHO AM I? { "name": "João Silva", "country": "Portugal", "employer": "CERN", "title": "Software Engineer", "technologies": [ "Java", "JavaScript", "Groovy", "Grails", "Activiti Workflow" ], "email": "[email protected]" } HISTORY → THE DAWN OF JAVASCRIPT Netscape Navigator HISTORICAL BACKGROUND Created in 1995 by Brendan Eich of Netscape Mocha → LiveScript → JavaScript Influenced by Scheme and Self First-class functions Closures Prototypes Object literals and array literals Glue language Form validations Adding interactivity to web pages BROWSER WARS Internet Explorer vs Netscape Navigator ECMASCRIPT standardize the syntax and semantics of a general purpose, cross-platform, vendor-neutral scripting language SERVER-SIDE JAVASCRIPT Netscape LiveWire (1996) Rhino (1997) Microsoft IIS (1997) Several others since WHAT WENT WRONG? Slow engines JavaScript's perception as a plumbing language Lack of tooling Better alternatives WHAT HAS CHANGED? FASTER ENGINES (V8) V8 FEATURES Written in C++ Compiles JavaScript directly into machine code Runtime profiler which identifies hot functions Can run standalone or can be embedded as a library Blazingly fast BETTER TOOLING Module system (CommonJS, AMD) Test frameworks (QUnit, Jasmine) Code quality (JSLint, JSHint) Dependency management (bower, npm) Task runner (Grunt) etc. JAVASCRIPT IS COOL NOW Top Language on GitHub for 2013 (> 250k repositories) Used for Web servers, Desktop applications, Presentations... ...GAMES sos END-TO-END JAVASCRIPT → THE MEAN STACK Monoglot programming NODE.JS Server-side JavaScript WHAT IS NODE.JS? Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. THE COST OF I/O Source: Ryan Dahl's JSConf node.js presentation WAITING... var results = db.query('select * from bigtable'); // Blocking // Process results fs.writeFile('/var/log/', 'bla'); var response = JSON.stringify(results); Most web applications are I/O bound, not CPU bound Node.js was designed to exploit this imbalance SCALING WITH THREADS Context-switching overhead Execution stacks take up memory C10k problem SCALING WITH PROCESSES Process scheduling overhead High memory usage SCALING WITH AN EVENT LOOP Single-threaded Leverages asynchronous I/O APIs of the host OS Never blocks for I/O operations Handles large number of concurrent connections THE NODE.JS EVENT LOOP BLOCKING I/O var results = db.query('select * from bigtable' // Process results NON-BLOCKING I/O db.query('select * from bigtable', function(results) // Process results }); JavaScript fits perfectly the event loop model with first-class functions and closures NODE.JS ARCHITECTURE EXAMPLES READING A FILE var fs = require('fs'); fs.readFile('hello.txt', function(err, data) { if (err) throw err; console.log(data); }); TCP ECHO SERVER var net = require('net'); var server = net.createServer(function (socket) { socket.write('Welcome to the echo server\r\n'); socket.pipe(socket); }); server.listen(1337); HTTP SERVER var http = require('http'); var server = http.createServer(function (request, response) { response.writeHead(200, {'Content-Type': 'text/html'}); response.end('<h1>Hello World</h1>'); }); server.listen(1337); console.log('Server running at http://127.0.0.1:1337/'); SUMMARY Server-side JavaScript environment Uses Chrome's V8 engine Event-driven Non-blocking I/O Single-threaded event loop EXPRESS web application framework for node.js WHAT IS EXPRESS? Express is a minimal and flexible node.js web application framework, providing a robust set of features for building single and multi-page, and hybrid web applications. WEB APPLICATION WITH NODE.JS var http = require("http"); var fs = require("fs"); http.createServer(function(req, res) { // Homepage if (req.url == "/") { res.writeHead(200, { "Content-Type": "text/html" }); res.end("Welcome to the homepage!"); } // About page else if (req.url == "/about") { res.writeHead(200, { "Content-Type": "text/html" }); fs.readFile(__dirname + "/about.html", function (err, data) { res.end(data); }); } // 404'd! else { res.writeHead(404, { "Content-Type": "text/plain" }); res.end("404 error! File not found."); } }).listen(1337); WEB APPLICATION WITH EXPRESS var express = require("express"); var app = express(); app.get("/", function(request, response) { response.send("<h1>Welcome to the home page!</h1>"); }); app.get("/about", function(request, response) { response.render("about.html"); }); app.all("*", function(request, response) { response.send(404, "404 error! File not found."); }); app.listen(1337); EXPRESS FEATURES ROUTING Map different requests to specific handlers var express = require("express"); var app = express(); app.post("/action/:id?", function(request, response) { var id = request.params.id; response.send(id ? "hello " + id : "hello"); }); app.get("/json", function(request, response) { response.send({ foo: "bar" }); }); app.all("*", function(request, response) { response.send(404, "404 error! File not found."); }); app.listen(1337); HANDLING FORM DATA var express = require('express'); var bodyParser = require('body-parser'); var app = express(); app.use(bodyParser()); app.get('/', function(req, res) { res.send('<form method="post" action="/">' + 'Name: <input type="text" name="username" />' + '<input type="submit" />' + '</form>'); }); app.post('/', function(req, res) { // Always sanitize your data to prevent XSS, remember Derek's presentation! res.send('¡Bienvenido!, ' + sanitize(req.body.username)); }); app.listen(1337); VIEW RENDERING index.jade doctype html html(lang="en") head title =pageTitle body h1 Welcome to the #{pageTitle}! p Jade is a cool, high-performance templating engine for Node.js. app.js var express = require('express'); var app = express(); app.get('/', function(req, res) { res.render('index.jade', { pageTitle: 'CERN Spring Campus 2014' }); }); LAYOUTS AND PARTIAL VIEWS layout.jade html body h1 Profound Quotes block content quotes.jade extends layout block content p The cave that you fear to enter holds the treasure you seek. app.js var express = require('express'); var app = express(); app.set('view engine', 'jade'); app.get('/', function(req, res) { res.render('quotes'); }); LAYOUTS AND PARTIAL VIEWS <!DOCTYPE html> <html> <body> <h1>Profound Quotes</h1> <p>The cave that you fear to enter holds the treasure you seek. </body> </html> MIDDLEWARE Middle layer between Express and the network var express = require('express'); var app = express(); // A middleware is a function (request handler) with three arguments var ensureAuthenticated = function(req, res, next) { if (req.isAuthenticated()) { var username = req.username; req.user = User.findByUsername(username); return next(); } return res.redirect("/login"); }; app.use(ensureAuthenticated); app.get('/', function(req, res) { res.send('Welcome ' + req.user); }); Authentication Authorization Error handling Security Session Sanitization of input CONFIGURATION app.configure(function () { app.set('views', __dirname + '/views'); app.use(staticProvider(__dirname + '/public'); app.use(bodyParser()); app.use(csrf()); app.use(session()); }); app.configure('development', function () { app.use(errorHandler({ dumpExceptions: true, showStack: true }); app.disable('emails'); }); app.configure('production', function () { app.use(errorHandler()); app.enable('emails'); }); SUMMARY Web application framework for node.js Routing Templating engines (Jade, EJS, HAML, etc.) Layouts and partial views Middleware (authentication, security, and much more) Environment-based configuration MONGODB open-source document database WHAT IS MONGODB? MongoDB (from "humongous") is an open- source document database, and the leading NoSQL database. #NOSQL Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. NON-RELATIONAL Source: http://martinfowler.com/bliki/AggregateOrientedDatabase.html NOSQL FLAVOURS Graph-oriented databases (Neo4j) Aggregate-oriented databases Key-Value Store (redis) Column Store (Cassandra) Document Store (MongoDB) DOCUMENT-ORIENTED DATABASE Data stored as whole documents Documents are independent units Application logic is easier to write (reduced impedance mismatch) Unstructured data can be stored easily (schema-less) Source: http://docs.mongodb.org/manual/core/crud-introduction/ MONGODB FEATURES Documented-oriented database Auto-sharding for horizontal scalability Built-in replication for high availability Schema-less Map-Reduce aggregation Documents are stored as BSON, a binary variant of JSON MONGODB TERMINOLOGY DATA MODEL DESIGN DATA MODEL DESIGN 1. Embedded Data Models Use when there are "contains" relationships between entities Use when there are one-to-many relationships between entities, and the "many" always appears with the parent document DATA MODEL DESIGN 2. Normalized Data Models using References Use when embedding would result in duplication Use to represent complex many-to-many relationships READ OPERATIONS Source: http://docs.mongodb.org/manual/core/read-operations-introduction/ QUERY INTERFACE db.collection.find() SQL WRITE OPERATIONS Source: http://docs.mongodb.org/manual/core/write-operations-introduction/ CREATE db.collection.insert() SQL UPDATE db.collection.update() SQL DELETE