A Solid Back End for a Solid Qt Mobile Application © 2013 The Qt Company About Me © 2014 The Qt Company Lauri Nevala nevalau nevalla nevalla • Over ten years of experience in creating web and mobile based applications • Working in Qt Cloud Services team at The Qt Company © 2014 The Qt Company Awesome mobile applications… © 2014 The Qt Company …and they all need back-end systems! © 2014 The Qt Company How do you implement your back end solution? © 2014 The Qt Company You have an idea BUY AND SETUP HARDWARE INSTALL OS INSTALL SOFTWARE JUST START BUILDING DEFINE ACCESS CONTROL SETUP ANALYTICS XBUILD APP APP © 2014 The Qt Company PaaS Platform as a Service © 2014 The Qt Company What is PaaS? On#Premises# IaaS$ PaaS$ SaaS# * Applica'on* Applica'on* Applica'on* Applica'on* Manage Data* * Data* Data* Data* You* Run'me* Run'me* Run'me* Run'me* Manage * You* * Middleware* Middleware* Middleware* Middleware* Provides OS* OS* OS* * OS* Manage * * Vendor You* Virtualiza'on* Virtualiza'on* Virtualiza'on* Virtualiza'on* Provides Servers* Servers* * Servers* Servers* Provides * Storage* Storage* Vendor Storage* Storage* Vendor Networking* Networking* Networking* Networking* With PaaS, developer can focus just on building great apps. No need to worry about servers, operating system, loadbalancing, network or infrastructure. © 2014 The Qt Company PaaS Illustra(on by Rouge Group © 2014 The Qt Company “We believe PaaS will enable organizations become more agile and responsive” “Organizations who are late, don't adopt PaaS will fall behind” – Yefim Natis, Gartner © 2014 The Qt Company Modern Application Development 1. Mobile First Approach - Factoring applications into many apps. 2. Cloud-Backed - Adopting more abstract and machine-independent application models for back ends in the form of PaaS. 3. Agile Teams - Rapid iterations, frequent releases, integration of users 4. Continuously Integrated and Delivered - No downtimes 5. DevOps-Enabled - Developers own deployments 6. User Experience-Centric - Focusing on the design and appearance of the application 7. Service Oriented Architecture - Use of micro services to compose application 8. Lightweight - Less complex software that is less time-consuming to install, learn and use 9. Socially Oriented - UX patterns from social networks (timelines, event streams etc), Data is updated using event-based push-oriented patterns. Integrated search functionality. 10.API Factored and Surfaced - Componentize and granularize back-end software in order to achieve composable and easy-to-evolve back-end platforms. © 2014 The Qt Company © 2014 The Qt Company Managed Application Runtime (MAR) Scalable, Multi-language, Multi-database, Application Platform as a Service © 2014 The Qt Company Supported Frameworks Supported frameworks by 3rd party build packs Scala, Clojure, Play, Gradle, Grails, PHP, Go, Meteorite, Perl, Dart, Nginx, Apache, Jekyll © 2014 The Qt Company Built-In Services © 2014 The Qt Company Add Ons Enginio Data Storage (EDS) Managed WebSocket (MWS) © 2014 The Qt Company Enginio Data Storage (EDS) Flexible and powerful cloud data storage with built-in user and data access control © 2014 The Qt Company Managed WebSocket (MWS) Fully managed service implementing a bi- directional, real-time communication gateway for WebSockets. © 2014 The Qt Company Developer Friendly Deployment Deploy using Git – the most common VCS among developers > git push qtc master © 2014 The Qt Company Scalable Architecture Built-In Applica2on Your Application Services Instances End Users Configure Deploy Scale With Your Business MySQL MongoDB Git Push API Push Git Load Balancer myawesome.com Variables Environment Redis Scaling © 2014 The Qt Company Web Console © 2014 The Qt Company Command Line Tool © 2014 The Qt Company I Have an Idea $$$ © 2014 The Qt Company Mobile application that shows tweets in real time © 2014 The Qt Company Back End • DB where to store tweets • Worker that retrieves/receives tweets from Twitter • API that offers tweets to client app • WebSocket server that push new tweets to client app © 2014 The Qt Company Database • Enginio Data Storage + Schemaless + REST API © 2014 The Qt Company WebSocket Server • Managed WebSocket + Easy to start + REST API © 2014 The Qt Company Worker + API • Managed Application Runtime + PaaS + Just build the app © 2014 The Qt Company 2. new tweet 1. get initial tweets 3. save tweet 4. send WebSocket message © 2014 The Qt Company Worker - Ruby require 'tweetstream' require 'qtc-sdk' ! TweetStream.configure do |config| config.consumer_key = ENV['CONSUMER_KEY'] config.consumer_secret = ENV['CONSUMER_SECRET'] config.oauth_token = ENV['OAUTH_TOKEN'] config.oauth_token_secret = ENV['OAUTH_SECRET'] config.auth_method = :oauth end eds = Qtc::Eds::Client.new(ENV['EDS_BACKEND_ID']) mws = Qtc::Mws::Client.new(ENV['MWS_GATEWAY_ID'], {access_token: ENV['MWS_SECURITY_TOKEN']}) tweets = eds.collection('tweets') daemon = TweetStream::Daemon.new('tracker', :multiple => true, :no_pidfiles => true) daemon.track('#QtDD14') do |status| if !status.text.start_with?('RT @') tweet = { created: status.created_at, tweet_id: status.id, text: status.text, name: status.user.name, screen_name: status.user.screen_name, profile_img_url: status.user.profile_image_uri, hashtags: status.hashtags.map{|h| h.text.downcase } } tweets.insert(tweet) mws.send_message(tweet.to_json, {sockets: nil, tags: ['QtDD14']}) end end © 2014 The Qt Company API - Ruby / Sinatra require 'sinatra' require 'qtc-sdk' ! set :port, ENV['PORT'] get '/tweets/:hashtag' do eds = Qtc::Eds::Client.new(ENV['EDS_BACKEND_ID']) tweets = eds.collection('tweets') query = { :q => { 'hashtags' => {'$in' => [params[:hashtag]] } }, :limit => 50, :sort => [{"sortBy" => "createdAt","direction" => "desc"}] } content_type :json, 'charset' => 'utf-8' result = tweets.find(query) result.to_json end get '/websocket_uri' do mws = Qtc::Mws::Client.new(ENV['MWS_GATEWAY_ID'], {access_token: ENV['MWS_SECURITY_TOKEN']}) socket = mws.create_socket(["QtDD14"]) content_type :json,'charset' => 'utf-8' socket.to_json end © 2014 The Qt Company Procfile • When using most common worker: ruby ./worker.rb run frameworks, MAR can web: rackup -p $PORT -E production automatically detect the executable. • You can explicitly declare what can be executed in a Procfile. • Each line declares a process type - a command that can be executed from your built application © 2014 The Qt Company > git push qtc master © 2014 The Qt Company lanevala@it-l-m0015 ~/projects/qtdd-twitter-stream[master*]$ git push qtc master Counting objects: 9, done. Delta compression using up to 8 threads. Compressing objects: 100% (8/8), done. Writing objects: 100% (9/9), 1.46 KiB | 0 bytes/s, done. Total 9 (delta 6), reused 0 (delta 0) -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-2.1.2 -----> Installing dependencies using 1.6.3 Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment Fetching gem metadata from https://rubygems.org/........ Using eventmachine 1.0.3 Using json 1.8.1 Using thread_safe 0.3.4 Using multi_json 1.10.1 Using naught 1.0.0 Using rack 1.5.2 Using bundler 1.6.3 Using em-socksify 0.3.0 Using http 0.6.1 Using faraday 0.9.0 Using em-twitter 0.3.3 Installing httpclient 2.5.2 Installing inifile 3.0.0 Using memoizable 0.4.2 Using puma 2.9.1 Using rails_12factor 0.0.3 Using em-http-request 1.1.2 Using twitter 5.11.0 Using sinatra 1.4.5 Using tweetstream 2.6.1 Installing faye-websocket 0.7.5 Using qtc-sdk 0.0.5 from https://github.com/nevalla/qtc-sdk-ruby.git (at master) Your bundle is complete! Gems in the groups development and test were not installed. It was installed into ./vendor/bundle Bundle completed (9.04s) Cleaning up the bundler cache. Removing httpclient (2.4.0) Removing qtc-sdk-ruby (8e7a9e3858e8) -----> Discovering process types Procfile declares types -> worker, web Default process types for Ruby -> rake, console, web -----> Compiled slug size is 17M -----> Deploying app Uploading app container .. done. mar-eu-1-13viqrf8 deployed to http://mar-eu-1-13viqrf8.qtcloudapp.com To [email protected]:mar-eu-1-13viqrf8 36e9190..b237ec8 master -> master © 2014 The Qt Company lanevala@it-l-m0015 ~/projects/qtdd-twitter-stream[master*]$ Demo © 2014 The Qt Company Buildpacks • When you deploy your Application, MAR slug compiler uses buildpack to prepares your code for execution. • Each programming language and framework support is implemented as buildpack. • Buildpack script will build the runtime environment on top of the Stack Image, a mini Linux distribution, and install everything that is needed. © 2014 The Qt Company • Stack Image is a mini Linux distribution which defines the core operating system and runtime environment. MAR Stack Images are based on following Linux distributions: • Lucid based on Ubuntu 10.04 LTS Lucid Lynx • Trusty based on Ubuntu 14.04 LTS Trusty Tahr will be available soon! © 2014 The Qt Company How to scale the application? • MAR application processes can be scaled up or down dynamically. • A typical web application has at least web process type declared in Procfile. • You can set wanted concurrency level for process by issuing scale command using qtc-cli: © 2014 The Qt Company $ qtc-cli mar scale web=2 $ qtc-cli mar scale worker=4 $ qtc-cli mar scale worker=0 © 2014 The Qt Company Configuration Variables • When developing server-side applications, it is very often required to have a configuration for application that is varying between environments. For example staging,