The Open Analytics Platform

Sharing and Deploying Data Science with KNIME Server

Roland Burger Marten Pfannenschmidt KNIME

© 2019 KNIME AG. All rights reserved. KNIME Server • Collaboration – Share Expertise and Templates/Blueprints • Automation – Schedule, Monitor, Update • Deployment – To Humans: Analytical Applications (“Guided Analytics”) – To Machines: via REST and custom APIs • Management – Manage distributed setups – Integrate with existing enterprise setup (authentication)

• Analytics as a • Data Access service • Data Blending • Guided Analytics • Transformation Data Blending • Analytical Apps & • In-database Marketing and Sales Decisions Dashboards • “In” Big Data

Data Analytics Financial Decisions

Predictive Analytics Data Apps Product R&D Decisions Science

Machine Learning Manufacturing Decisions

Artificial Intelligence IT Decisions • Statistics • Mining • AI/ML/DL Models • R/Python Scripts • Legacy Languages

KNIME Community Integrations Extensions KNIME Data Blending Analytics Platform KNIME Partner Extensions Extensions Data Analytics

Data Data Engineers Data Scientists App Developers Predictive Analytics Science Data Analysts ML/AI Engineers

Machine Learning

Artificial Intelligence

Collaborate Automate Manage Deploy

KNIME Server Marketing and Sales

Financial

Data Science Teams IT Teams Business Teams Apps Product R&D

Guided Analytic Apps Manufacturing Mobile Applications

Web Cloud-based Applications Services IT

REST API

Load > Integrate > Transform > Analyze > Visualize

KNIME Community Extensions Integrations KNIME

KNIME Analytics Platform Partner Extensions Extensions

KNIME Server

Collaborate Automate Manage Deploy

Workflow Hub Workflows Metanodes, Nodes Data Collaboration

Shared Repository, Versioning

Provisioning Remote, Client Control Distributed Executors Automation

Web Front-ends Guided Analytic Apps RESTful Services Deployment

Management Configuration & Client & User Security Versioning Preferences

• Connect to KNIME Server from local KNIME Analytics Platform – Authentication possible via LDAP / Active Directory – Centralized administration and security

Workflows • Interact with KNIME Server: Workflow Groups – Upload workflows for sharing and collaboration with colleagues Data – Execute workflows, create schedules, set permissions, … Metanode Templates

– Organize your work in workflow Jobs groups (directories)

• Right-click -> Deploy to Server – Share workflows, data, etc. for collaboration – Continuous delivery in all steps of workflow life cycle

• A place to share knowledge about Workflows. – Share – Discover – Search – Rate – Comment – Version

• Create template from metanode and store on server • Templates are linked -> check for new version and apply updates automatically • Increase efficiency through reusable workflow components • Ensure compliance with company policies

• Execute workflows directly on KNIME Server • Scale beyond limitations of desktop machines

• Automate workflow execution in regular intervals • Orchestrate job pipeline by using Call Workflow nodes

• Monitor progress of execution on server (almost) live • Very helpful for longer-running jobs

What‘s my workflow doing right now?

• Change node configurations directly in the remote job – Resume execution if there was an error – Try different configuration options without re-executing the full job – Add nodes, change node connections, all live and in the secure server environment

• All workflows on KNIME Server available as web apps • Step-by-step execution of workflows from any browser • Simple, clean interface for end users – Guided Analytics • Customize layout to match corporate design

Interaction Points

• Enables external integration Deploy KNIME workflows • Build applications around KNIME Server as web services • e.g. for Microservices and real time scoring

Output in Input data server response

KNIME Workflow for e.g. sentiment prediction, churn score, chemical property

KNIME SERVER Take from pool

External KNIME REST call Executor Pooled jobs

Workflow Pre-loaded jobs Results immediately pooled and waiting executed for work No “load latency” for client

• Swagger UI: Explore API definition of workflow in browser • Provides all information needed to interact with API

• Programmatically control KNIME Server – Upload/download/delete resources • Staging from Dev to Prod – Upload licenses – Empty trash/restore items – Execute workflows – Schedule jobs – Set permissions All server functionality – Create users & groups, etc. available via REST API

• Create snapshots of workflow revisions • Allow roll-back to previous versions • Full workflow history available in client and on Workflow Hub

• Inspect differences between versions of the same workflow

Highlight differences: - Nodes included/excluded - Node configurations

• Manage permissions for workflows and data on KNIME Server • Control which groups can access, modify, or execute a workflow • Use LDAP / Active Directory groups for centralized administration

How do I How do I configure install Python? database drivers?

KNIME KNIME KNIME Analytics Analytics Analytics Platform Platform Platform

KNIME Website Internal knowledgebase

• Different departments/teams have different Marketing Finance R&D requirements

• Multiple OS deployments • Windows 7 • Windows 10 • Linux • macOS Windows 10 Windows 7 Linux and macOS Hive Oracle Python Spark MS Access R

• Client-profiles • Python-Linux KNIME Server • Python-macOS • R-Linux client-profiles • R-macOS Big Data- Databases- Python- • Databases-Win7 Win10 Win7 macOS (etc) • Big Data-Win10

Profiles can include: Preferences, drivers, and more Marketing Finance R&D

• Need more workflow horsepower?

• KNIME Server currently supports ‘Scale Up’

• KNIME Server Distributed Executors allows ‘Scale Out’

https://upload.wikimedia.org/wikipedia/commons/thumb/8/8b/Server.svg/2000px-Server.svg.png https://upload.wikimedia.org/wikipedia/commons/5/53/Server-multiple.svg

Executor 1

Executor 2

Requests to KNIME Server Executor 3 execute workflows RabbitMQ

KNIME Server + Distributed Executors

Executor 1

Request to KNIME Server execute workflow RabbitMQ Scripted launch on AWS CloudFormation KNIME Server + Distributed Executors

Executor 1 CPU load increases

Launching Executor 2 More KNIME Server requests to execute RabbitMQ workflows AWS Auto-scaling Group KNIME Server + Distributed Executors

Executor 1

Executor 2 Requests to KNIME Server execute workflows RabbitMQ AWS Auto-scaling Group KNIME Server + Distributed Executors

KNIME KNIME KNIME Analytics Analytics Analytics Platform Platform Platform

KNIME Server KNIME Server KNIME Server

Workflow Hub Workflows Metanodes, Nodes Data Collaboration

Shared Repository, Versioning Ready to get started

Provisioning Remote, Client Control with KNIME Server? Distributed Executors Contact us for a free Automation trial:

Web Front-ends Guided Analytic Apps RESTful Services [email protected] Deployment

Management Configuration & Client & User Security Versioning Preferences

© 2019 KNIME AG. All rights reserved. 37 The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH, and are registered in the United States. KNIME® is also registered in Germany.