http://aka.ms/bobwardms https://github.com/Microsoft/sqllinuxlabs

SQL Server on Linux and Containers A Brave New World

Speaker Name Principal Architect Microsoft [email protected] @bobwardms linkedin.com/in/bobwardms

#ITDEVCONNECTIONS | ITDEVCONNECTIONS.COM The response to open source demand

Businesses are Microsoft is The world is demanding embracing choice delivering on choice SQL Server on Linux

HDInsight on Linux Heterogeneous environments 6M+

Docker pulls of Machine Learning Server on Linux SQL Server on Linux image

Multiple data types

T-SQL PHP Linux in Azure Java Node.js C/C++ Python C#/VB.NET Ruby

Different development languages SQL Server drivers and connectivity 36%

On-premises, cloud, and SQL Server Ops Studio open source, hybrid environments enterprise DB market runs on Linux cross-platform administration

SQL Server on Linux - What we have learned

Thousands of instances and CPUs 6M+ Docker Pulls using SQL Server on Linux

Is it the same as Windows?

Is the performance the same as on Windows?

Is the licensing the same as on Windows?

Do I have to know Linux?

What Linux are most customers using?

Is SQL Server different in a container? What’s in SQL Server on Linux

Windows Linux

Editions Developer, Express, Web, Standard, Enterprise   Database Engine, Integration Services, SQL Server Agent   Services Analysis Services, Reporting Services, MDS, DQS  Maximum number of cores Unlimited Unlimited Maximum memory utilized per instance 24 TB 12 TB Maximum database size 524 PB 524 PB Mission critical performance Basic OLTP (Basic In-Memory OLTP, Basic operational analytics)   and HADR Advanced OLTP (Advanced In-Memory OLTP, Advanced operational analytics, adaptive query processing)   SQL Server Replication   NEW Basic high availability (2-node single database failover, non-readable secondary)  

Advanced HA (Always On - multi-node, multi-db failover, readable secondaries)  

Basic security (Basic auditing, Row-level security, Data masking, Always Encrypted, Active Directory Authentication)   Security Advanced security (Transparent Data Encryption)  

PolyBase  Planned

Basic data warehousing/data marts (Basic In-Memory ColumnStore, Partitioning, Compression)   Data warehousing Advanced data warehousing (Advanced In-Memory ColumnStore)  

Advanced data integration (Fuzzy grouping and look ups)  

Windows ecosystem: Full-fidelity Management & Dev Tool (SSMS & SSDT), command line tools   Tools Linux/OSX/Windows ecosystem: Dev tools (VS Code), DB Admin GUI tool, command line tools  

Programmability (T-SQL, CLR, Data Types, JSON, Graph)   Developer Distributed Transactions   NEW Machine Learning Services   NEW Linux loves packages • Major Linux distributions follow the pattern of using a package manager. • RHEL: RPM and yum. • Ubuntu: Debian and apt-get • SUSE: RPM and zypper. • Package: binaries + instructions to install. • Benefits: dependency management, automatic versioning, one-tool-to-rule- them-all, architecture control.

mssql- mssql- mssql- mssql- mssql- server tools server-fts server-ha server-is mssql-conf • mssql-conf is a configuration script that installs with SQL Server 2017 on RHEL, SUSE, and Ubuntu. This utility can be used to set: • Enable SQL Server Agent • TCP port • Default data directory SQL Server also supports config through • Default log directory environment variables • Default dump directory • Default backup directory • Core dump type • Enable Availability Groups • Set startup traceflags • Set collation SQL Server Linux Architecture Based on Microsoft Research System Resource & Drawbridge Project Latency Sensitive Code Paths Everything else

SQLAGENT SQLSERVR API LibOS (Win API and SQLOS (SQLPAL) Linux Kernel) SQL Platform Process Abstraction Layer (SQLPAL) (Ring 3) SQL PAL ABI Host Extension mapping to OS system calls (IO, Memory, CPU scheduling) Linux APIs (mmap, pthread_create, …)

fork Ring 0 Linux Kernel Parent “Watchdog” process SQL Demo Deploy and explore SQL Server on Linux Performance on SQL Server 2017 on Linux

#1 TPC-H Benchmark1 World’s first enterprise-class Columnstore Indexes and diskless database In-Memory OLTP capabilities 1,009,065 QphH with $0.47 per QphH

180 billion rows scanned in Adaptive Query Processing Linux performance best <20 seconds with 480 CPUs and Automatic Tuning practices guidance Layers of security for enterprise-grade protection

Encryption-at-rest Encryption in transit 200 Information Transparent data encryption (TDE) Transport Layer Security (TLS 1.2) Backup encryption 180 Protection Client-side encryption Cell-level encryption

Always Encrypted 160

2017) -

140

120 Database access Application access 100 SQL Server Logins Row-Level Security Access SQL Active Directory Authentication Dynamic Data Masking (2010 Vulnerabilities 80 Management Roles and permissions 60 Schemas 40

20

Tracking & investigation 0 Threat Fine-grained audit Protection Vulnerability Assessment Data Discovery & Classification HADR with SQL Server on Linux

Failover Cluster Instance High Availability Group Read-Scale Availability Group Pacemaker and Corosync CLUSTER_TYPE = EXTERNAL CLUSTER_TYPE=NONE Single SQL Server instance Auto failover No clustering required SQL resource agent (mssql-server-ha) Pacemaker Manual or forced failover SQL resource agent (mssql-server-ha) Sync or async replicas Open-source resource agent is Read scale routing available on GitHub Repository1 3 replicas required Cross-platform Configuration only replica for Shared Storage metadata ISCI, NFS, SMB Full AG capabilities

Industry-leading solutions available from HA/DR partners2 SQL Server 2017 and 2019 on Linux Tools and programmability

Windows SQL Server Management Studio (SSMS) Windows SQL Server Data Tools (SSDT) 3rd party tools continue to work Existing drivers/frameworks supported

Existing command line tools • sqlcmd, bcp, sqlpackage Visual Studio Code mssql extension Azure Data Studio mssql-cli mssql-scripter SQL Server on containers

Portable Container configuration Run anywhere Docker is supported Persisted storage Lightweight Reduced disk, CPU, and memory footprint Switch for Database simple Consistent upgrades Consistent image of SQL Server, scripts, and tools

Efficient Faster deployment, reduced patching, and less downtime Container Container Container Shared storage HA in Kubernetes Built-in HADR orchestration with no clustering required

Node Node Node

Pod Pod

SQL Server Load Balancer Service SQL Server

User

Pod

SQL Server Kubernetes

Persistent Volume Storage Mission critical availability Availability groups on Kubernetes Keep SQL Server running

Pod Load balancer Operator • Always On availability group enhancements • Resumable online index creation • Online Clustered Columnstore Pod Pod Pod index creation and rebuild SQL Server SQL Server SQL Server AG primarysecondary primarysecondary secondary • Availability groups on Kubernetes AG agent AG agent AG agent

Load balancer SQL Demo SQL Server and Docker Containers SQL Server 2019 on Linux CTP 2.0 New Capabilities

SQL Database Database

Replication OpenLDAP support for 3rd Persisted Memory Optimized I/O Availability Groups with party providers Memory Pressure Notifications Kubernetes

T-SQL PHP Java Node.js C/C++ Python C#/VB.NET Ruby

Machine Learning Services with R/Python Microsoft Container Registry Distributed Transactions RHEL Container Image Next steps

Evaluate SQL Server 2017 at http://aka.ms/sql2017

Leverage resources http://aka.ms/sqllinux http://aka.ms/sqlblog https://github.com/Microsoft/bobsql https://github.com/Microsoft/sqllinuxlabs http://aka.ms/sqlchannel http://aka.ms/sqldev

Join SQL Server Early Adoption Program https://aka.ms/eapsignup

#ITDEVCONNECTIONS | ITDEVCONNECTIONS.COM