Setting Up Your Open-Source Development Environment

Mark Irish [email protected] Staff Software Developer IBM

April 28, 2021 POW3R Digital Outline

 Open-Source Software on IBM i  Setting Up Your Development Environment  Access Client Solutions  Installing  Setting up SSH  Mounting the File System  Using a Sane Text Editor  Setting up ODBC  Containers (optional)

© 2021 IBM Corporation Open-Source Software on IBM i

© 2021 IBM Corporation Open-Source Software (OSS)

 Open-source software has publicly available source code  Why would anyone make open-source software?

– Greater transparency of what software does

– Others can find/fix bugs and add functionality

– Wider adoption and popularity  “Free”, but...  Use of open-source software is dictated by a license

© 2021 IBM Corporation Open-Source Software on IBM i

 Delivered on PASE (Portable Application Solutions Environment)  Delivered through yum ()

– Very “normal”, will feel familiar to Unix users (Linux, Mac)  600+ packages available  DO NOT USE 5733-OPS (out of support)

© 2021 IBM Corporation OSS as a Gateway

 Artificial Intelligence (AI) / Machine Learning (ML)

– IBM Watson Solutions ● Watson Studio ● Watson Machine Learning ● Watson Machine Learning Accelerator  Quantum  Internet of Things (IoT)  REST  and more!

© 2021 IBM Corporation Things That Don’t Go Together

© 2021 IBM Corporation Things That Don’t Go Together

© 2021 IBM Corporation Things That Don’t Go Together

© 2021 IBM Corporation Need a More “Normal” Environment

 The 5250 “green screen” terminal isn’t going to cut it for developing open- source applications on/with IBM i  Need to set up a “normal” environment, that involves:

– An SSH terminal

– A mounted file-system on the development machine

– A modern text-editor

– An ODBC connection to Db2 for i

© 2021 IBM Corporation Using Access Client Solutions

© 2021 IBM Corporation Access Client Solutions (ACS)

 I’ve never met an IBM i developer who didn’t already use ACS, but…  ...you need to install ACS  ACS will allow us to:

– Manage our open-source software

– Run a 5250 emulator when we need to do 5250 things

– Create and explore database schemas/tables/views with a GUI

– Easily run SQL commands (and see what syntax errors exist)

– Much more!

© 2021 IBM Corporation Access Client Solutions (ACS)

 ACS is a platform-independent dashboard for managing IBM i  Is based, and can be run on almost any OS that supports Java

– Linux

– Mac

– Windows

– IBM i  Not strictly required, but makes managing OSS much easier

 https://www.ibm.com/support/pages/ibm-i-access-client-solutions

© 2021 IBM Corporation Installing ACS

 Installation steps depend on where you are installing it  General steps:

– Download the zipped archive

– Unzip the archive

– Run the included program (install_acs_64)

– Launch ACS

© 2021 IBM Corporation Access Client Solutions (ACS)

© 2021 IBM Corporation The Open-Source Environment

© 2021 IBM Corporation Getting “The Open-Source Environment”

 “The Open-Source Environment” is really just yum  yum manages RPMs on IBM i  yum searches repositories hosted on the internet (or intranet) for packages  IBM-hosted public repository for IBM i RPMs:

– ftp://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo

 Community-hosted repositories also exist  yum will automatically manage dependencies for your installations

© 2021 IBM Corporation RPMs

 yum allows us to install RPMs  RPM stands for RPM Package Manager  Contain pre-compiled binaries (and other files), packaged by IBM  RPM file names look like: --...rpm

– e.g.: nodejs10-10.15.0-2.ibmi7.2.ppc64.rpm

© 2021 IBM Corporation Getting yum with ACS

 http://ibm.biz/ibmi-rpms

 Make sure SSHD is running on your system

– STRTCPSVR *SSHD  In ACS, go to Tools > Open Source Package Management

– You will be prompted to install the “Open source environment”

– IBM-hosted IBM i RPM repository will automatically be configured

© 2021 IBM Corporation Getting yum with ACS

© 2021 IBM Corporation Using SSH to use a Remote Terminal

© 2021 IBM Corporation SSH to IBM i

 Running QSH in 5250 doesn’t cut it  Need a more “normal” development environment like what open-source software expects

© 2021 IBM Corporation SSH to IBM i

 Need to have SSHD running

– STRTCPSVR *SSHD  If you don’t have the SSH daemon on your system, or haven’t generated host keys, consult:

– https://www.ibm.com/support/pages/starting-openssh-server-sshd

© 2021 IBM Corporation Getting bash on IBM i

 bash is an industry-standard shell  Most open-source software tutorials you see will assume you are using bash  Using ACS, go to Tools > Open Source Package Management  Select “Available Packages”, then install bash

© 2021 IBM Corporation Getting bash on IBM i

© 2021 IBM Corporation Getting bash on IBM i

 Just because we have bash, doesn’t mean that bash will be used by default  There is a helpful SQL procedure to set it correctly! Simply run:

CALL QSYS2.SET_PASE_SHELL_INFO('', '/QOpenSys/pkgs/bin/bash')

 Now when you login with SSH, you will automatically be using bash, like most other open-source users!

© 2021 IBM Corporation SSH to IBM i from Windows

 Download PuTTY from

– https://www.putty.org/

© 2021 IBM Corporation SSH to IBM i from Windows

© 2021 IBM Corporation SSH to IBM i from Windows

 Can save system and username so you only have to enter password

© 2021 IBM Corporation SSH to IBM i from Mac or Linux

 Need to download openssh

– MIGHT be already installed on your OS, can check by running something like:

which ssh

 How to install will depend on your OS / distribution, but generally will look something like

install openssh

© 2021 IBM Corporation SSH to IBM i from Mac or Linux

 Generate a public and private key using ssh-keygen:

ssh-keygen -t rsa -b 4096

 This will likely place the keys under your in the .ssh directory  Public: id_rsa.pub is often shared with other services servers  Private: id_rsa is for you to protect!

© 2021 IBM Corporation SSH to IBM i from Mac or Linux

 Can connect to IBM i by running

ssh @

...and then entering your password

 A better solution exists...

© 2021 IBM Corporation SSH to IBM i from Mac or Linux

 In your ~/.ssh directory, create or edit a file called config

Host HostName User ForwardAgent yes ServerAliveInterval 5

 Now you can just run ssh and it will known which username to use and forward your SSH key to use for other services  Still have to enter password, but...

© 2021 IBM Corporation SSH to IBM i from Mac or Linux

 Set up the public key on the server using ssh-copy-id:

ssh-copy-id -i ~/.ssh/id_rsa.pub

 You can now login without a password!  Simply run ssh and you will be logged in to a terminal session running on IBM i, using your default shell

© 2021 IBM Corporation Mounting the File System

© 2021 IBM Corporation Mounting the File System

 Mounting the file system means creating a link from a directory on your development system to a remote file system  In this case, we want to access the IFS as if it were on our local machine  Allows us to use development tools (IDEs) more easily  Changes we make are (nearly) instantly propagated to the IFS

© 2021 IBM Corporation Mounting the File System on Windows

 Windows makes this a little more difficult than it needs to be…  There is a Win-SSHFS that will allow you to connect to a POSIX file system (like PASE!)  Instructions and download can be found at:

– https://github.com/billziss-gh/sshfs-win

© 2021 IBM Corporation Mounting the File System (Linux/Mac)

 Going to use sshfs, which is transferring file system data over the same SSH protocols used to get a remote terminal session  Installation

– Mac: https://github.com/osxfuse/sshfs/releases

– Linux: See if it is preinstalled with your OS, or use your package manager ● Package name might be something like fuse-sshfs

© 2021 IBM Corporation Using sshfs

 Need to create a directory to use as our mount point

– mkdir /mnt/myibmisystem

 Then simply mount the part of the IFS you want to use!

– sudo sshfs -o allow_other [email protected]:/ /mnt/myibmisystem

 -o allow_other : Allows you to access the mounted files without using sudo

[email protected]:/ : Mount the file system from the root

 /mnt/myibmisystem : The directory where the file system is mounted

© 2021 IBM Corporation Using a Modern IDE / Text Editor

© 2021 IBM Corporation Visual Studio Code

 A modern, fast, light-weight text editor  Syntax highlighting for almost every language  Built-in debugging for most languages  Extension support (Several IBM i extensions)  NOT VISUAL STUDIO  Sometimes called “VS Code” or “Code”

© 2021 IBM Corporation Visual Studio Code

© 2021 IBM Corporation ODBC to Connect to the Database

© 2021 IBM Corporation ODBC

 Open Database Connectivity (ODBC) is a standardized database interface maintained by Microsoft

© 2021 IBM Corporation ODBC

 ODBC is the preferred/recommended method of connecting to open-source on IBM i

– Robust IBM i driver allows for MOST everything you need to do

– ODBC connectors already exist for most open-source languages  There is some setup required, but once you set it up, you don’t have to worry about it again  I have an entire hour-long presentation on ODBC (I’ve included it at the end of this presentation!), but the gist is...

© 2021 IBM Corporation ODBC Driver Manager

 On Windows, pre-installed (ODBC Data Source Administrator)  On Linux/Mac, need to use package manager to download unixODBC

– Mac: homebrew install unixodbc unixodbc-dev

– Linux: install unixODBC unixODBC-dev/devel

© 2021 IBM Corporation ODBC IBM i Driver

 Download the driver from IBM i Access – Client Solutions page:

– https://www-01.ibm.com/support/docview.wss?uid=isg3T1026805

– Select the OS you are connecting from (PASE/Windows/Linux/Mac)  Unzip the download  Use your package manager or click the .exe to install the driver on your system

© 2021 IBM Corporation Create a DSN

 A datasource name (DSN) is a set of common connection options that you can use in a connection string  Can create system-wide or user-specific DSNs

© 2021 IBM Corporation Example DSN

[MyDSN] Description = The DSN for my IBM i system Driver = IBM i Access ODBC Driver System = myibmisystem.location.com UserID = MyUser Password = SuPeRsEcReT

© 2021 IBM Corporation ODBC

 There is a lot more to ODBC  ODBC can be confusing at first  Resources:

– https://ibmi-oss-docs.readthedocs.io/en/latest/odbc/README.html

– https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzaik/con nectkeywords.htm – Appendix A at the end of this presentation!

© 2021 IBM Corporation Using Containers to Manage Open Source

© 2021 IBM Corporation Containers

 Containers isolate parts of a computer (file system, memory, etc.) to ensure applications can’t interfere with one another.  A chroot (change root) is a container where the root directory (‘/’) of a process is changed, limiting the process’s access to the file system  Again, I have a 60-minute presentation on all of this, which I have attached to the end of these slides

© 2021 IBM Corporation Containers

App App App App

Libraries Libraries Libraries Libraries App App

Guest OS Guest OS Container Engine Libraries Libraries

Host OS Host OS Host OS

Infrastructure Infrastructure Infrastructure

VM Ware Docker chroots Virtual Box

© 2021 IBM Corporation Containers on IBM i

 On IBM i, containers are implemented in PASE through chroots  chroots create a “jail” in the IFS where processes can be run without fear they affect the rest of the IFS  chroots are created through ibmichroot package, available on yum  chroots are not Docker containers

© 2021 IBM Corporation Containers on IBM i

 How the IFS looks OUTSIDE the chroot

dev bin bin bin bin home etc dev etc bin lib include lib home lib lib lib64 lib / QOpenSys QOpenSys container pkgs libexec pkgs libexec lib sbin sbin sbin sbin sbin share tmp tmp share tmp tmp usr usr usr usr var var var var

© 2021 IBM Corporation Containers on IBM i

 How the IFS looks INSIDE the chroot

bin bin dev etc bin home lib lib QOpenSys / pkgs libexec lib sbin sbin sbin tmp share tmp

usr usr var var

56 © 2021 IBM Corporation Open-Source Software in a chroot

 If you install a minimal chroot, there is no yum inside the chroot  You can define you own list of file to copy over to the chroot, but that is advanced chrooting  Most developers will use yum OUTSIDE the chroot to install yum INTO the chroot

© 2021 IBM Corporation yum and chroots

 When installing open-source software INTO a chroot, use --installroot:

– yum install --installroot=/QOpenSys/

© 2021 IBM Corporation Putting Someone in Jail

 To force someone into a chroot when they SSH onto the system, use CHGUSRPRF:

CHGUSRPRF USRPRF() HOMEDIR(‘/QOpenSys/ ./home/’)

 HOMEDIR does some magic, where everything before the . is the root, and everything after is the home directory (inside the root) to put them in

© 2021 IBM Corporation Putting Someone in Jail

 Why?:

– Give developers their own environments that they can customize for their liking

– Creating an environment where developers can use without interfering with one another

– Manage open-source software on a per-chroot or per-user basis

© 2021 IBM Corporation Conclusions

© 2021 IBM Corporation Conclusions

 Creating a more “normal” development environment will help to use open- source software as everyone else does

– IBM i open-source can be just like everyone else!  The most important part is using a “modern” shell, like bash, through a remote SSH session  It is going to feel weird if you are used to 5250, but non-IBM i developers you hire will feel right at home

© 2021 IBM Corporation Resources

 All-in-one resource for IBM i open-source:

– https://ibm.github.io/ibmi-oss-resources/

 Guide on getting started with open-source software on IBM i:

– http://ibm.biz/ibmi-rpms

 IBM i open-source software issues board:

– https://bitbucket.org/ibmi/opensource/issues?status=new&status=open

© 2021 IBM Corporation Appendix A: ODBC on IBM i

Mark Irish [email protected] Staff Software Developer IBM Outline

 Introduction to ODBC  Why ODBC?  Installing ODBC and the IBM i Driver

− On IBM i

− On Linux/Mac

− On Windows  Connection Options  Using ODBC with Open-Source

− Node.js

− Python

− PHP  Conclusions

© 2021 IBM Corporation Introduction to ODBC

© 2021 IBM Corporation Introduction to ODBC

 Open Database Connectivity (ODBC) is a standardized database interface maintained by Microsoft  Database vendors create a driver that implements that interface

© 2021 IBM Corporation ODBC Layers: Application

 In modern development, made of two layers

– Connector  Developed by someone else  Abstracts the ODBC API to make it easier to develop your program

– Your code  Simple APIs to connect, run queries, run procedures, etc.

© 2021 IBM Corporation ODBC Layers: Driver Manager

 The “implementation” of ODBC  Two Driver Managers work with IBM i driver:

– unixODBC  IBM i, Linux, MacOS

– ODBC Data Source Administrator  Windows  iodbc does NOT work with our driver

© 2021 IBM Corporation ODBC Layers: Driver

 Translates ODBC commands to a specific database management system  Driver available for connecting to IBM i (Db2 for i)  Driver available to connect from:

– IBM i

– Linux

– macOS

– Windows

© 2021 IBM Corporation ODBC Layers: Data Source

 The database that ODBC is interacting with  In our case, Db2 for i on IBM i

© 2021 IBM Corporation ODBC Drivers

 ODBC keeps track of installed drivers in a file called odbcinst.ini

– On Windows, stored in ODBC Data Source Administrator

 odbcinst.ini [IBM i Access ODBC Driver] Description=IBM i Access for Linux ODBC Driver Driver=/opt/ibm/iaccess/lib/libcwbodbc.so

© 2021 IBM Corporation ODBC Data Source Names

 Define common connection attributes in a Data Source Name (DSN)  Stored in odbc.ini or ~/.odbc.ini

– odbc.ini: For system-wide DSNs

– ~/.odbc.ini: For user-specific DSNs

 odbc.ini [DUMMY] Driver = IBM i Access ODBC Driver System = myibmi.dummy.location UserID = Mark Irish Password = PaSsWoRd

© 2021 IBM Corporation ODBC Connection Strings

 Connection strings are used to connect to the data source  Can point to attributes in the DSN, or declare all the options you want

DSN=DUMMY;

or

DRIVER=IBM i Access ODBC Driver;SYSTEM=myibmi.system.com; UID=MyUID;PWD=SuPeRSeCrEt

© 2021 IBM Corporation Putting it All Together

 odbcinst.ini [IBM i Access ODBC Driver] Description=IBM i Access for Linux ODBC Driver Driver=/opt/ibm/iaccess/lib/libcwbodbc.so

 odbc.ini [DUMMY] Driver = IBM i Access ODBC Driver System = myibmi.dummy.location UserID = Mark Irish Password = PaSsWoRd

Connection String: DSN=DUMMY;

© 2021 IBM Corporation Why ODBC?

© 2021 IBM Corporation Why ODBC?

 ODBC driver managers already available on most systems  IBM i driver already available on most systems  ODBC connectors already available for most open-source technologies  Code written with ODBC is portable  Larger developer base means more community support

© 2021 IBM Corporation ODBC already available on most systems

 IBM i

– unixODBC available through yum  Linux

– unixODBC available through your package manager  macOS

– unixODBC available through homebrew  Windows

– Driver manager comes preinstalled

© 2021 IBM Corporation IBM i driver available on most systems

 IBM i  Linux  macOS  Windows

 All available through IBM i Access Client Solutions

– https://www.ibm.com/support/pages/ibm-i-access-client-solutions

© 2021 IBM Corporation ODBC connectors available for open-source

 Node.js

– odbc package available on  Python

– pyodbc package available on pypi  PHP

– ODBC connector pre-built natively 

– odbc package available on R Studio  + many more!

© 2021 IBM Corporation Code written with ODBC is portable

 Write code on System A against Database A  Move code to System B, still connects to Database A  Very useful for developing on one system, deploying on another

© 2021 IBM Corporation Code written with ODBC is portable

© 2021 IBM Corporation Code written with ODBC is portable

© 2021 IBM Corporation Code written with ODBC is portable

© 2021 IBM Corporation Code written with ODBC is portable

© 2021 IBM Corporation Larger developer base

 More than just IBM i developers use ODBC

– MSSQL/SQL Server, MySQL, PostgreSQL, Sybase, 4D, etc.  More developers means more support

– Support for driver manager

– Support for connector  Still depend on IBM and IBM i users for driver support

© 2021 IBM Corporation Installing ODBC and the IBM i Driver

© 2021 IBM Corporation Installing ODBC and the IBM i Driver

 Instructions will differ depending on which you are using  Generally, the steps are:

– Install ODBC driver manager

– Install ODBC driver

– Configure your drivers

– Configure your datasources (DSNs)  Instructions available outside of this presentation

– https://ibmi-oss-docs.readthedocs.io/en/latest/odbc/installation.html

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Requires our yum package manager, used for downloading/installing all open- source software on IBM i

– http://ibm.biz/ibmi-rpms

 In a terminal, install unixODBC with yum:

yum install unixODBC unixODBC-devel

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Download the driver from the IBM i Access - Client Solutions page

– https://www-01.ibm.com/support/docview.wss?uid=isg3T1026805

– Select “Downloads for IBM i Access Client Solutions”

– Download “ACS PASE App Pkg”

– Unzip the download

– Move /ppc64/ibmi-access-….ppc64.rpm to the IFS

– Use yum to install the downloaded RPM:

yum install //to/rpm/ibm-iaccess-1.1.0.14-0.ibmi7.2.ppc64.rpm

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Now have the driver manager and the driver installed  The driver will add an entry to /QOpenSys/etc/odbcinst.ini:

[IBM i Access ODBC Driver] Description=IBM i Access ODBC Driver Driver=/QOpenSys/pkgs/lib/libcwbodbc.so Threading=0 DontDLClose=1 UsageCount=8

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Can add DSNs to /QOpenSys/etc/odbc.ini or ~/.odbc.ini:

[MyDSN] Description = The DSN for my IBM i system Driver = IBM i Access ODBC Driver System = myibmisystem.location.com UserID = MyUser Password = SuPeRsEcReT

© 2021 IBM Corporation System = *LOCAL

 Only on IBM i, you can define a System as *LOCAL, which will connect to the client system using *CURRENT credentials

– Whichever user is running the job, their authorization lists are used

[MyDSN] Description = The DSN for my IBM i system Driver = IBM i Access ODBC Driver System = *LOCAL

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Unix

 Instructions for Linux and MacOS are similar (and similar to IBM i)  Install unixODBC from a terminal:

– Linux:  install unixODBC unixODBC-dev/devel

– macOS:  homebrew install unixodbc unixodbc-dev  Instructions available outside of this presentation

– https://ibmi-oss-docs.readthedocs.io/en/latest/odbc/installation.html

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Unix

 Download the driver from the IBM i Access - Client Solutions page

– https://www-01.ibm.com/support/docview.wss?uid=isg3T1026805

– Select “Downloads for IBM i Access Client Solutions”

– Download “ACS Linux/Mac App Pkg”

– Unzip the download

– Install the resulting .pkg/.deb/.rpm with your package manager

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Now have the driver manager and the driver installed  The driver will add an entry to /etc/odbcinst.ini:

[IBM i Access ODBC Driver] Description=IBM i Access ODBC Driver Driver=/usr/local/lib/libcwbodbc.so Threading=0 DontDLClose=1 UsageCount=8

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on IBM i

 Can add DSNs to /etc/odbc.ini or ~/.odbc.ini:

[MyDSN] Description = The DSN for my IBM i system Driver = IBM i Access ODBC Driver System = myibmisytem.myurl.com UserID = MyUserID Password = MySecretPassword

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Windows

 ODBC Data Source Administrator comes preinstalled on Windows, we don’t need to download a driver manager!

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Windows

 Download the driver from the IBM i Access - Client Solutions page

– https://www-01.ibm.com/support/docview.wss?uid=isg3T1026805

– Select “Downloads for IBM i Access Client Solutions”

– Download “ACS Windows App Pkg English (64bit)”

– Unzip the download

– Run setup.exe

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Windows

 Now have the driver manager and the driver installed  Can open ODBC Data Source Administrator to view Drivers

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Windows

 DSNs are created on the UserDSN or SystemDSN tabs  Options are available through the menu

© 2021 IBM Corporation Installing ODBC and the IBM i Driver on Windows

 Can also modify DSN values in the Registry

© 2021 IBM Corporation Connection Keywords

© 2021 IBM Corporation Connection Keywords

 In order to connect to the database, you need to pass a connection string  Values passed as: KEY=VALUE;  Connection string can:

– Assign connection keywords/values in the connection string itself

– Point to a DSN, which defines keywords/values

– Both of the above

© 2021 IBM Corporation Connection Keywords

 Connecting with connection string values only:

DRIVER=IBM i Access ODBC Driver; SYSTEM=ibmisystem.mydomain.com;UID=MYUSER; PWD=SeCrEtPaSsWoRd;...=…;

 Values can be escaped with curly braces: {}

PWD={UgLy* /P4$$;W0r %&};

© 2021 IBM Corporation Connection Keywords

 Connecting with DSN values:

connection string:

DSN=SAMPLEDSN;

~/.odbc.ini:

[SAMPLEDSN] Driver = IBM i Access ODBC Driver System = ibmisystem.mydomain.com UserID = MyUser Password = SeCrEtPaSsWoRd

© 2021 IBM Corporation Connection Keywords

 Connecting with connection string and DSN values:

connection string:

DSN=SAMPLE_DSN;UID=MyUser;PWD=SeCrEtPaSsWoRd

~/.odbc.ini:

[SAMPLE_DSN] Driver = IBM i Access ODBC Driver System = ibmisystem.mydomain.com

© 2021 IBM Corporation Connection Keywords

 Connection keywords allow us to:

– Declare what system we want to use

– Pass our User and Password for authentication

– Tweak driver behavior  Keywords can be defined either in a connection string or in our DSN definitions

– The key may differ depending on where you define it  On duplication, connection string values override DSN values

© 2021 IBM Corporation Connection Keywords

 Connection keywords can be found at: https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzaik/connectkeywords.htm

 Over 80 keywords available that let you:

– Set the default transaction isolation level

– Set the default libraries

– Set the maximum decimal precision returned

– Set the date format used with SQL statements

– And more!

© 2021 IBM Corporation Using ODBC with Open Source

© 2021 IBM Corporation Using ODBC with Open Source

 One of the reasons we recommend ODBC is because nearly every open-source technology has an OBDC connector  Usually use the included package manager to download the ODBC package  Examples for:

– Node.js

– Python

– PHP

© 2021 IBM Corporation Node.js

 The preferred package for use with Node.js is odbc  In your application, download with npm (Node.js package manager)

npm install odbc

 Depending on what system you are downloading on, odbc may need to compile code to build to connector

 Documentation: https://www.npmjs.com/package/odbc

© 2021 IBM Corporation Node.js

 Query and procedure results are returned as:  An array of JavaScript objects, where column names = property names  Information about each column returned  The parameters passed to the query (or back from a procedure)  The count of columns affected by the query

© 2021 IBM Corporation Node.js: Simple Example

const odbc = require(‘odbc’); async function example() { const pool = await odbc.pool(‘DSN=MYDSN’); const result = await pool.query(‘SELECT * FROM QIWS.QCUSTCDT’); console.log(result); }

example();

© 2021 IBM Corporation Node.js: Simple Example

$ node app.js [ { CUSNUM: 938485, LSTNAM: ‘Johnson ‘, INIT: ‘J A’, STREET: ‘3 Alpine Way ‘, CITY: ‘Helen ‘, STATE: ‘GA’, ZIPCOD: 30545 CDTLMT: 9999 CHGCOD: 2 BALDUE: 3987.5 CDTDUE: 33.5 } statement: ‘SELECT * FROM QIWS.CUSTCDT’, parameters: [], return: undefined, count: -1, columns: [ … ]

© 2021 IBM Corporation Node.js

 Can use either callback functions or promises  Can retrieve data from tables/views, bind parameters to queries, and more  Can stored call procedures

– Can even call procedures that wrap around RPG PGMs and SRVPGMs!  New fixes and functionality coming out all the time!

© 2021 IBM Corporation Python

 The preferred package for use with Python is pyodbc  In your application, download with

Linux/Mac/Windows: pip3 install pyodbc IBM i: yum install python3-pyodbc

 Depending on what system you are downloading on, pyodbc may need to compile C code to build to connector

 Documentation: https://pypi.org/project/pyodbc/

© 2021 IBM Corporation Python: Simple Example

import pyodbc

connection = pyodbc.connect(‘DSN=MYDSN’) cursor = connection.cursor() cursor.execute(“SELECT * FROM QIWS.QCUSTCDT”) row = cursor.fetchall() for row in rows: print(row)

© 2021 IBM Corporation Python: Simple Example

$ python3 odbc-example.py (Decimal('938485'), 'Johnson ', 'J A', '3 Alpine Way ', 'Helen ', 'GA', Decimal('30545'), Decimal('9999'), Decimal('2'), Decimal('3987.50'), Decimal('33.50')) (Decimal('583990'), 'Abraham ', 'M T', '392 Mill St ', 'Isle ', 'MN', Decimal('56342'), Decimal('9999'), Decimal('3'), Decimal('500.00'), Decimal('0')) ...

© 2021 IBM Corporation Python

 pyodbc allows you to retrieve data with queries, bind parameters to queries and more!  No dedicated function for calling stored procedures

– Have to use cursor.execute() with the ODBC CALL escape sequence:

– params = (123, “string value”) cursor.execute(“{CALL MYLIB.MYSCHEMA (?, ?)}”, params)

© 2021 IBM Corporation PHP

 CommunityPlus+ PHP (available on IBM i) comes with odbc and PDO_ODBC

– No need to download anything extra!  I’m not a fan of ibm_db2, which doesn’t use ODBC but instead CLI (Call-Level Interface)

 Documentation: https://www.php.net/manual/en/book.uodbc.php

© 2021 IBM Corporation PHP: Simple Example

$value) { echo $key . ": " . $value . "\n"; } echo "\n"; } odbc_close($conn); ?>

© 2021 IBM Corporation PHP: Simple Example

$ php -f test.php Row: CUSNUM: 938472 LSTNAM: Henning INIT: G K STREET: 4859 Elm Ave CITY: Dallas STATE: TX ZIPCOD: 75217 CDTLMT: 5000 CHGCOD: 3 BALDUE: 37.00 CDTDUE: 0

© 2021 IBM Corporation PHP

 LOTS of different functions to call, all prefixed with odbc_

 Functions to fetch data in rows or objects, and can fetch one at a time or more

 No dedicated function to calling procedures

 Lots of special functions for getting data from catalog tables

– columns

– tables

– primary keys

– foreign keys

– etc...

© 2021 IBM Corporation Conclusions

© 2021 IBM Corporation Conclusions

 ODBC is the preferred method of connecting to IBM i from open-source software  ODBC is available on almost every system and with almost every technology  Initial ODBC set up can be confusing

– Once a system is set up, you don’t have to think about it again  Connecting to IBM i through open-source software with ODBC is easy

© 2021 IBM Corporation Resources

 ibmi-oss-docs

– https://ibmi-oss-docs.readthedocs.io/en/latest/odbc/README.html

 Connection String Keywords Webpage

– https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzaik/connectkeywords.htm

 My email

[email protected]

© 2021 IBM Corporation Appendix B: Managing Open Source with Containers

Mark Irish Staff Software Developer IBM [email protected] @markdirish Outline • Introduction to chroots  chroots v. other “containers”  How a chroot works • Installing and creating a chroot  using yum  creating a chroot • Managing software in a chroot

129 © 2021 IBM Corporation Containers • Containers are strategies for isolating parts of a computer (file system, memory, etc.) usually to ensure that applications can’t interfere with one another

• There are several different types of containers, including:  Virtual Machines  Docker Containers  chroots

• A chroot (change root) is a type of container where the root directory (‘/’) of a process is changed, thereby limiting the process’s access to only that part of the file system.

130 © 2021 IBM Corporation Different types of containers

App App App App

Libraries Libraries Libraries Libraries App App

Guest OS Guest OS Container Engine Libraries Libraries

Host OS Host OS Host OS

Infrastructure Infrastructure Infrastructure

VM Ware Docker chroots Virtual Box

131 © 2021 IBM Corporation Containers on IBM i • On IBM i, containers are implemented in PASE through chroots

• chroots create a “jail”, a confined area of the IFS where processes can be run without fear that they affect the rest of the IFS

• chroots are created through the `ibmichroot` package, available from the IBM open- source software repository

• chroots are not Docker containers

132 © 2021 IBM Corporation How a chroot works • A chroot works be redefining the root directory, making it appear that a subset of the IFS is the entire thing:

dev bin bin bin bin home etc dev etc bin lib include lib home lib lib lib64 lib / QOpenSys QOpenSys container pkgs libexec pkgs libexec lib sbin sbin sbin sbin sbin share tmp tmp share tmp tmp usr usr usr usr var var var var 133 © 2021 IBM Corporation How a chroot works • The contents of the container’s new root are partially copied from the real root

• It is important that certain libraries and utilities are copied over in the creation of a chroot, otherwise doing productive work would be impossible  Exactly what is copied is up to you, but there is a “minimal list”

• None of the open-source software installed in the main root is copied over; You must reinstall it into the chroot using yum

134 © 2021 IBM Corporation What IBM i chroots can and can’t do • chroots can:  create isolated environments for open-source software (which doesn’t know of IBM i resources)  allow open-source development to occur in a sterile and confined environment

• chroots cannot:  isolate underlying IBM i resources  isolate the authorizations of the user/process in a chroot

135 © 2021 IBM Corporation Creating chroots • Need to download ibmichroot from the open-soure software repository using yum

• Can be done through Access Client Solutions or in a shell

136 © 2021 IBM Corporation Creating chroots

137 © 2021 IBM Corporation Creating chroots

138 © 2021 IBM Corporation Creating a chroot • Now that ibmichroot is installed, we can use it to create a chroot with it

• Requirements for making a chroot:  Need *IOSYSCFG authority to create device nodes  Need *ALLOBJ to run yum  chroot must be made under the /QOpenSys directory

• Run: chroot_setup /QOpenSys/

139 © 2021 IBM Corporation Creating a chroot

140 © 2021 IBM Corporation Getting into your chroot voluntarily • If you are outside of the chroot, you can get “into it” (change the root) using the chroot command from your shell:

• chroot /QOpenSys/ /QOpenSys/usr/bin/bash

• Command take two arguments: 1. The location that you want to be the new root (what you want to change the root to) 2. The path (inside the chroot) to the first program executed (usually a shell)  If not defined, runs user’s default shell

 Can use exit to get out of a chroot you’ve put yourself in 141 © 2021 IBM Corporation Getting into your chroot voluntarily • Use cases:  Run software in a chroot, but otherwise let your user enter and leave chroot at will  Building applications in a sterile environment  Running applications isolated from one another  Creating a sandbox to play around with new packages

142 © 2021 IBM Corporation Forcing someone into a chroot • To force someone into a chroot when they SSH onto the system, use CHGUSRPRF

• CHGUSRPRF USRPRF() HOMEDIR(‘/QOpenSys/ ./home/’)

• HOMEDIR does some magic, where everything before the . is the root, and everything after is the home directory (inside the root) to put them in.

143 © 2021 IBM Corporation Forcing someone into a chroot • Use cases:  Give developers their own environments that they can customize to their liking  Put new developers in an environment where they can get adjusted  Reduce the risk of errant commands affecting the entire IFS (e.g. a rogue rm -rf)

144 © 2021 IBM Corporation A chroot’s perspective • How the IFS looks OUTSIDE the chroot

dev bin bin bin bin home etc dev etc bin lib include lib home lib lib lib64 lib / QOpenSys QOpenSys container pkgs libexec pkgs libexec lib sbin sbin sbin sbin sbin share tmp tmp share tmp tmp usr usr usr usr var var var var 145 © 2021 IBM Corporation A chroot’s perspective • How the IFS looks INSIDE the chroot

bin bin dev etc bin home lib lib QOpenSys / pkgs libexec lib sbin sbin sbin tmp share tmp

usr usr var var

146 © 2021 IBM Corporation Open-source software in a chroot • If you install a minimal chroot, there is no yum (“open-source environment”) inside the chroot

• You can define your own list of files to copy over to the chroot, but that is advanced chrooting

• Most developers will use yum OUTSIDE of the chroot to install yum INTO the chroot

147 © 2021 IBM Corporation yum to install yum into a chroot • When installing open-source software INTO a chroot, use --installroot  yum install yum yum-utils --installroot=/QOpenSys/

• You now have yum inside your chroot!

• Need to create a connection to the IBM repo:  yum-config-manager --add-repo http://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo

• Can now jail someone, and they will have access to install their own open-source software (if they have *ALLOBJ authority)

148 © 2021 IBM Corporation yum to install other software into a chroot • Maybe your developers don’t have *ALLOBJ, or you want to limit the software available to them in the chroot

• Instead of giving them yum, give them the software they need

• When installing open-source software INTO a chroot, use --installroot

• yum install --installroot=/QOpenSys/

• They now have the software they need, but don’t have the ability to download aditional software. Everything managed by a user outside of the chroot

149 © 2021 IBM Corporation Common package management tasks • The way you interact with open-source software inside a chroot is the same way you interact with it outside of a chroot

• When using ACS, you can select a container where to install software (this is akin to --installroot)

150 © 2021 IBM Corporation Installing packages (ACS)

151 © 2021 IBM Corporation Installing packages (shell) • Inside of chroot  yum install nodejs12

• Outside of chroot  yum install nodejs12 --installroot=/QOpenSys/

152 © 2021 IBM Corporation Update packages (ACS)

153 © 2021 IBM Corporation Update packages (shell) • Inside of chroot  yum update nodejs12

• Outside of chroot  yum update nodejs12 --installroot=/QOpenSys/

154 © 2021 IBM Corporation Remove packages (ACS)

155 © 2021 IBM Corporation Removing packages (shell) • Inside of chroot  yum remove nodejs12

• Outside of chroot  yum remove nodejs12 --installroot=/QOpenSys/

156 © 2021 IBM Corporation Key Takeaways 1. chroots (PASE containers) work by changing the root of the file structure, thereby creating a “jail” from which

2. chroots do not affect the underlying system resources available to a user, and in no way affect the “IBM i” resources that the user has access to

3. chroots allow you to manage open-source software by creating an environment where software can run without affecting other parts of the IFS

157 © 2021 IBM Corporation