Open Source on IBM i Webinar Series Day 1 ERWIN EARLEY ([email protected]), SR. SOLUTIONS CONSULTANT, PERFORCE, NOVEMBER 2019 2 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Overview
• Open Source Solutions are a driving factor/influencer in much of IT today ▪ The IBM i platform can now benefit from Open Source in significantly expanded ways then ever seen before on the platform
• This workshop provides an overview of Open Source on the IBM i Platform • The workshop is designed to leverage both lecture (presentation) as well as provide hands-on (after-hours) opportunities.
2 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 2 3 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Workshop Agenda • Day 1: Monday 18-November 2019 ▪ Introduction to Open Source on IBM I ▪ Why is Open Source on IBM i Important ▪ Understanding the PASE environment as the enabler of Open Source on IBM I ▪ Getting Familiar with the PASE environment ▪ After-Hours Lab: Getting Familiar with PASE • Day 2: Wednesday 20-November-2019 ▪ Exploring containers on IBM i ▪ Managing Open Source on IBM i ▪ Exploring Open Source Programming Languages ▪ After-Hours Lab: Containers & Setting up Development Environment • Day 3: Friday 22-November-2019 ▪ Open Source Programming Languages Integration with Db2 and ILE ▪ Looking at the IBM i variant of the LAMP stack ▪ Deploying LAMP-based solutions ▪ After-Hours Lab: Open Source Programming Languages ▪ After-Hours Lab: Deploying LAMP applications
3 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com IBM Systems Technical University © 3 4 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Logistics
• Workshop scheduled to run from 11:00AM – 2:00PM (Central) ▪ Monday / Wednesday / Friday • Will break about 75 minutes into the session • Materials available on the workshop system: ▪ http://idevphp.idevcloud.com • Hands-on exercises to be completed outside of the webinar presentation time
4 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 5 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Day 1 Agenda • Open Source Overview
• Introduction to Open Source on IBM i
• Why is Open Source on IBM i Important
• Understanding the PASE Environment as the enabler of Open Source on IBM i
• Getting Familiar with the PASE environment
• After-Hours Lab: Getting Familiar with PASE
5 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com IBM Systems Technical University © 5 6 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Free OSS ? • “Software that gives users rights to run, copy, distribute, change and improve it as they see it, without them asking permission from or make payments to any external group or person”. Mitre FOSS report 2002
6 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 7 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Free as in “Freedom” • Freedom to study the code • Freedom to improve the program • Freedom to run the program anytime, for any purpose on any machine. • Freedom to redistribute. Free Speech not Free Beer
7 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 8 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Free OS Software
• Apache • Linux • BIND • MySQL • Emacs • Open Office • FreeBSD • Perl • Ghostscript • Samba • Jakarta • Sendmail • Jboss • Snort • LaTex • Squid
8 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 9 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Why OSS ? • Customizable • Improvable • Redistributable. • Runs Everywhere, for everyone. • Transparency. • Free !!!
9 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 10 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Surprising !! • How can the technical work of dozens, or even hundreds, of developers be coordinated without the usual project management apparatus? • How do useful, well-designed products emerge from the individual actions of widely-distributed developers, with minimal central planning and relatively impoverished communication?
10 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 11 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com The OSS Developer WAY ! • Discuss the “itch” with a few • Code Development starts • Create website and “announce”. • Community evolves • Functionalities added • Feedback and Communication flows
11 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 12 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Basic Open Source Software Development Process
Manage Configuration
Download and Develop OSS Install Code
Communicate Experience Assert End-Use Requirements- Design
Read, Analyze and Redesign
OSS Community Development Process
12 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 13 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Software ?
• Open source software is FreeOSS that uses any license approved by the Open Source Initiative (OSI) from their list of approved open source licenses • www.opensource.org/licenses/
13 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 14 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Understanding the Open Source Process and Distributions
• Contributors: Submit code changes to maintainers for consideration • Maintainers: Ownership (from a maintenance viewpoint) of a particular component (application) ▪ The maintainer is responsible for review of changes submitted by contributors, incorporation of accepted changes into the component, and releases of an updated component (application)
14 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 14 15 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com End-User Influence on Open Source
• End-Users can report defects and request enhancements through the open source application’s defect tracking system ▪ Typically ‘bugzilla’ • End-users have the “freedom” to review the open source, code their own changes, and submit them to the open source project maintainer.
15 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 15 16 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Communication is Informal!! • Community communications ▪ Threaded discussion forums ▪ Email (list servers) ▪ Newsgroups ▪ IRChat/Instant messages ▪ Community digests (“Kernel Cousins”)
16 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 17 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com People involved
17 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 18 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com OSSD into commercial world • IBM considers it important ▪ Customers ask for it ▪ Developing in open community leads to innovation ▪ Develops open standards
18 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 19 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Why is OSSD successful ? • Collaboration is decentralized. Integration is controlled • Too many resources to fail • So many eyeballs looking at the code • Self-motivated , self-assigned programmers. • Large scale Peer Review • User Driven requirements
19 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 20 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com OSSD vs PSD: It differs a lot ! • Different Paradigms • Cathedral vs Bazaar • To name a few ▪ Assignment of work ▪ Change Management ▪ Testing & Bug Fixing
20 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 21 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Assignment of work
• You assign (Baby Sitter) ▪ Laggard ▪ Sit inside the boundary • I pick (Parent) ▪ More responsibility ▪ More care ▪ Avenues for exploration ▪ Quality product
21 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 22 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Testing and Bug Fixing • PS Industry spends 40% of time on testing and bug fixing. ▪ White box testing ▪ Black box testing
• In OSSD it is a community process ▪ Bugzilla ▪ IssueTracker
22 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 23 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Assessing Open Source Solutions
• Evaluation/assessment of any solution is imperative to the successful implementation of a quality solution • Steps in the evaluation process are really no different from those you would use with closed source (or commercial) solutions, just the time spent differs ▪ Step 1: Identify candidate applications that may meet the business requirements ▪ Step 2: Review existing reviews/evaluations of the application(s) ▪ Step 3: Compare basic attributes of the application(s) with against specific requirements ▪ Step 4: Perform an in depth analysis of the top candidate applications
• While the evaluation steps for proprietary and open source applications are the same the source of information for the identification and review steps are quite different ▪ Proprietary solutions tend to be identified by the commercial vendor ▪ Typically, the vendor provides documentation and literature on their product(s) ▪ In the Open Source space, target applications are identified through a variety of mechanisms including search engines and well known open source repositories such as sourceforge.net and freshmeat.net
23 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 23 24 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Evaluation
• Sources of information to assist with Identification and Review steps: ▪ The Enterprise Open Source Directory (http://www.eosdirectory.com) provides an on- line catalog of open source projects ▪ The opensourceCMS site (php.opensource.com) has a ratings page that provides customer ratings of popular CMS packages ▪ Wikipedia has a list of free and open source applications by category (http://en.wikipedia.org/wiki/List_of_free_and_open_source_software_packages) ▪ The Free Software Foundation maintains a Free Software Directory (http://directory.fsf.org/) ▪ An Open Source Software Directory, compete with user ratings for the applications, is also available (http://www.opensourcesofsoftwaredirectory.com) ▪ The InsideCRM web site (http://www.insidecrm.com) provides a list of the top 10 CRM packages. Rankings of CRM packages can also be found at the CRMSoftware360 website (http://www.crmsoftware360.com) ▪ The wiki rankings available on the web are somewhat dated; however, you might want to take a look at the Top 5 open source Wiki engines article on the Tech Corner web site (http://www.benh.org/techblog/) ▪ OSS Watch (http://www.oss.watch.ac.uk) can be a great resource for obtaining advice and guidance on the use, development and licensing of open source software applications
24 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 24 25 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Evaluation
Factors to consider • Reputation: What is the application's reputation for performance and reliability ▪ Some of the sites on the previous slide provide information on what others have experienced with the application ▪ Reputation of an application is often directly related to its popularity − While it's hard to gauge the popularity of a particular open source application there are some metrics that can be used to anecdotally determine the popularity such as download counters for the product as well as site counters for Internet based applications • Ongoing effort: Is there an ongoing effort to continue development of the application including both defect resolution as well as incorporating enhancements and new functionality? ▪ Places to look for this information include the application web site ▪ Another good place to look would be the sourceforge web site • Standards: Does the application adhere to a documented set of standards? ▪ Is there a requirement for the solution to be able to integrate with other applications? − What are the integration capabilities of the application under review?
25 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 25 26 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions Evaluation
Factors to Consider • Documentation: What level of documentation is available for the open source application? ▪ Popular open source applications like the SaMBa file server and Apache web server have a wealth of documentation available including both open source as well as commercial resources ▪ For less popular open source applications it is possible that documentation may lag release of product versions • Versioning: When was the last stable version of the application released? ▪ Ensure that there is an ongoing development effort for the application ▪ Be aware that version numbers in open source don't necessarily follow the conventions of commercial applications − As an example, often times when evaluating commercial applications customers will tend to stay away from those applications whose version numbers end in a zero (0) as that represents the first release of a new version of the application − This may not be the case in open source – where 1.0 usually represents the first release of a commercial application, version 0.1 may represent the first version of an open source application − Keep in mind there is no “standard” for version numbers in the open source space. • Licensing: Review and understand the licensing aspects of the product and that it meets the requirements ▪ Not all open source licenses are created equal
26 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 26 27 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions Evaluations
• Review of application literature should provide information necessary for the comparison of functions provided to those required ▪ Keep in mind – this comparison will only be as good as the initial definition of the requirements to be satisfied • Functional review should result in a subset (typically one or two) candidate applications migrating to the top of the “desired” list • Here's the fun part!! Establish a test environment and kick the tires of those application(s) that you want to further review ▪ Leverage the virtualization capabilities of the Power system to establish a Linux test LPAR (or KVM guest, or even a Docker container) for further evaluation of the application(s) • Once the application face-off is complete, put the winning application into production.
27 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 27 28 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Overview
• 78% of business are now operating on open source software • More than 50% find open source to be more secure than proprietary software Open Source Software has an inherent business value • With open source as the foundation of a business an enterprise is able to remove barriers that encourage innovation and entrepreneurialism. All permissions required to deploy, run, migrate, collaborate, and innovate have been provided in advance
28 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 29 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source on IBM i: Timeline
5733-OPS
1988 2002 2006 2013 2014 2015 2016 2017 2018 JTOpen Apache Zend PHP PowerRuby Node.JS Python Tools Nginx RPMs
29 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 30 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions on IBM i
Legacy Offerings 5733OPS Integration Community
Apache Python XMLService Many I OSS projects OpenSSL Node.JS Language Toolkits Active collaboration OpenSSH Git IBM I Services Blogs, articles JTOpen Orion (SQL) etc. etc etc. etc.
30 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 31 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com First Stop – Basic Infrastructure
Assimilation
Sustainable Revenue
Real Business Applications
Ecosystem & Documentation
PASE Languages Basic Infrastructure SSL, SSH slice compilers
31 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 32 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Basic Infrastructure
• The basic infrastructure of support has to be in place to allow us to build upon it with real-world solutions • For IBM i, the basic infrastructure includes items such as: ▪ SLIC ▪ PASE ▪ Programming Languages ▪ Compilers ▪ Protocol support – such as SSL and SSH
32 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 33 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Next Stop – Ecosystem & Documentation
Assimilation
Sustainable Revenue
Real Business Applications
Frameworks Language modules API libraries Ecosystem & Documentation Integration pieces Workable examples GNU tools
Basic Infrastructure
33 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 34 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com XML Service / aka XML Toolkit
• Set of classes that access IBM i native artifacts ▪ All program objects − RPG, COBOL, CL, etc. • Others ▪ Command processor Data Queues ▪ Spooled File ▪ More… • Access naturally from PHP code. ▪ No SPs required • Easy to use
34 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 35 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Two Parts
• XML Service ▪ Developed by IBM and contributed to the community as open source ▪ Made up of RPG, CL and DB2 stored procedures (SQL) • Toolkit (the PHP side) ▪ Series of classes that "wrap" the calls to the ILE Toolkit ▪ All PHP but Object Oriented (but easy to learn and use) ▪ Zend built the PHP wrapper ▪ No OO training required to use them • Other open source languages have their own wrapper (toolkit)
http://yips.idevcloud.com/wiki/index.php/XMLService/XMLService
35 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 36 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Capabilities/Benefits of ILE Access
• Leverage existing business logic – no need to re-write • Provide new interfaces for existing data objects and artifacts Use Case: Need to convert an existing RPG application to run in web browsers.
Solution: When using PHP as the "vehicle" to "convert" the application it's unlikely that 100% of the application will have to be rewritten. With PHP, the existing RPG code can be encapsulated into smaller programs or service programs containing modules. The programs and modules can be easily called using the ILE Toolkit.
Use Case: Leverage complex pricing routine written in RPG in a new web presence
Solution: Extract code from existing application and create an API in RPG. The resulting API can then be called from the PHP script thereby leveraging the investment in the existing application while using PHP to concentrate on the user interaction portion of the desired solution.
36 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 37 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com DB2 Storage Engine for MySQL and MariaDB
• Supports open source applications (i.e., LAMP-based) while simplifying data management ▪ Applications written to MySQL or MariaDB can store their data in DB2 without any code change ▪ One database to manage, backup, and protect ▪ RPG applications, DB2 Web Query have access to MySQL/MariaDB generated data.
• PHP, Apache, and MySQL/MariaDB enable 1000s I A M P of applications on IBM I including: B p y H – CRM, ecommerce, Portals, Wikis, blogs, etc a S P M c Q i h L e
37 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 38 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Final Stop (for now) – Ecosystem & Documentation
Assimilation
Sustainable Revenue
Real Business Applications
Ecosystem & Documentation
Basic Infrastructure
38 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 39 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com IBM i – enabled for Open Source Solutions
• IBM i runs many of the industry standard technologies, such as PHP, Apache, and MySQL/MariaDB that enables web solutions like Jenkins and Wordpress to be run natively on the platform • These applications can be easily deployed and leveraged
39 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 40 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com The Toolkits – integrating with IBM i
• Each Open Source Language has a toolkit that provides the ability to integrate the language with Db2 as well as ILE programs and artifacts.
• Db2 access provides the ability to execute SQL statements from the language against Db2.
Examples from Node.js, python, and PHP will be covered in the Open Source Programming Languages materials
40 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 41 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Toolkit links • Toolkits provide the ability to integrate with various features/functions/programs of IBM I • Node.js toolkit: ▪ https://bitbucket.org/litmis/nodejs-itoolkit • PHP Toolkit for IBM i: ▪ http://yips.idevcloud.com/wiki/index.php/XMLSERVICE/Python ▪ https://bitbucket.org/litmis/python-itoolkit • Python itoolkit-lite ▪ https://bitbucket.org/litmis/nodejs-itoolkit • Ruby itoolkit NOTE: Documentation for ▪ https://bitbucket.org/litmis/ruby-itoolkit Ruby, Swift, and .NET is not sufficient to complete the • Swift capability information ▪ https://bitbucket.org/litmis/swift-itoolkit provided on the following • .NET slides ▪ https://github.com/richardschoen/IbmiXmlserviceStd
41 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 42 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Databases • MariaDB – Major part of the LAMP-stack ecosystem ▪ Includes a storage engine for access to Db2 • MongoDB – NoSQL Document store database • Redis – In-memory key/value store
42 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 43 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Businesses are "dying" of thirst in an ocean of data" 90% 80% 20% of the world’s of the world’s is the amount of data was data today is available data created in the unstructured traditional last two years systems leverages
1 in 2 83% 54% business leaders don’t of CIO’s cited BI and of companies use have access to data they analytics as part of their analytics for competitive need visionary plan advantage Source: GigaOM, Software Group, IBM Institute for Business Value"
43 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 44 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Background • Relational Database Management Systems (RDBMS) have been, and continue to be, the primary method for storage of data • Open Source RDBMS offerings such as MySQL/MariaDB, and PostgreSQL augment proprietary solutions such as DB2 and Oracle • Popular web sites linking to smaller web sites can cause significant increases in traffic • This is sometimes referred to as the 'Slashdot Effect' • RDBMS tends to be geared to support OLTP. • Hooking these database to web-based applications can can cause database content from hundreds of thousands of visitors in a short span of time • Fronting the RDBMS with read-only cache, like memcached and Redis, can offload a considerable amount of the read traffic • The growth of datasets has shown limitations in the RDBMS model
44 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 45 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Background (continued)
• RDBMS were not intended/designed to be distributed • Attempting to scale up when the datasets become large causes issues • Limitations in scaling led technologists to look at multi- node database solutions • This has become to be known as "scaling out" or "horizontal scaling" • Different approaches to scaling include • Master-Slave • Sharding • The best way to provide ACID and rich query model is to have the database on a single host • There are limits to vertical scaling (although one could argue this point when it comes to IBM i) • The CAPP theorem provides an alternative to ACID – more on this in a moment
45 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 46 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
NoSQL Defined • Originally stood for Not SQL • Has come to be known as Not Only SQL • Provides a class of non-relational data storage systems • Usually does not require a fixed table schema • NoSQL databases do not use the concept of joins
• NoSQL relaxes one or more of the ACID properties [implements CAP, more on that in a moment] • As a reminder: • Atomicity – each transaction is treated as a single unit which either succeeds or fails completely • Consistency – a transaction can only bring the database from one valid state to another • Isolation – ensures that concurrent transactions will leave the database in the same state as if the transactions had been executed sequentially • Durability – once a transaction has been committed it will remain committed
46 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 47 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Document Database Example – What is MongoDB?
• Developed by in 2007 (based in New York). • Released as open source in 2009 under the GNU Affero General Public License (AGPL) and Apache License (language drivers)
• Community free edition has been downloaded 30 million times 1 • NoSQL Database: Uses collections of documents instead of rows in a table. • Written in C++, C and JavaScript • Dynamic schema design: Provides flexibility and changes usually have minimal impact on code. Be agile.
1 https://techcrunch.com/2017/09/21/database-provider-mongodb-has-filed-to-go-public/ 47 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 48 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
REDIS Overview • REDIS (Remote Dictionary Server) is an in-memory data structure store that can be used as: • Database • Cache • Message Broker • Has support for data structures such as: • Strings • Hashes • Lists • Sets • Sorted sets • Bitmaps • Hyperloglogs • Geospatial index
48 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 49 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Use Cases
• A replacement for memcached • A work-queue (sets & lists) • Fast auto completion (persistent, queryable) • Caching for Nginx via the Nginx HTTP Redis module • Simple full-text search for multiple data stores redis-textsearch
Memcached is a general-purpose distributed memory caching system that is often used to speed up dynamic database-driven websites by caching data and objects in RAM
49 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 50 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com NoSQL in the IBM i ecosystem
• Cassandra is implemented via Java – as such it should work "out of the box" • An AIX binary of Redis is available and has been used by a number of IBM i shops • MongoDB enables numerous Node.JS applications so just like MySQL was ported to enable PHP-based applications (i.e., LAMP) it is likely that MongoDB will be ported to IBM I to enable Node.JS-based applications
• Remember that 80% of new data being generated is unstructured – new solutions to support the explosion of non-structure data needs/will be available on IBM i
50 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 51 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com ActiveMQ • Message-oriented middleware • Apache project ▪ http://acivemq.apace.org/ • Apache license • Goal: ▪ Standards-based, message-oriented application integration across many languages and platforms
51 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 52 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com git • Git provides Source Code Control / Version Control • Version control is a system that keeps records of all changes • Enables collaborative development • Enables auditing – knowing who did what and when • Facilitates rolling back changes to a previous state
52 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 53 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Orion • Some refer to Orion as an IDE on the cloud
• Browser-based open tools integration platform ▪ Focused on developing for the web, on the web ▪ Focus on web developers working on client-side JavaScript, CSS, and HTML
• Orion is a set of server and client-side components/code that when brought together make up an extensible browser-based development platform.
• Supports integration with Git, as well as shell access, site definition, dealing with various tasks, etc.
53 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 54 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Nginx • Pronounced at "Engine X"
• Open Source web and reverse proxy server
• High-performance HTTP, HTTPS, SMTP, iMAP, and POP3 server
• Supports load balancing and HTTP caching
• Architecture is asynchronous and event-driven
54 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 55 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Nginx – Why Use It • Lightweight and small memory footprint • Uses predictable memory under load • Provides high level of concurrency • Serves static content quickly • Handles connections asynchronously • Uses single thread
55 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 56 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Comprehensive support solutions
56 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 57 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com TSS Support Offering
With a continuously evolving product list, TSS can support virtually the entire ecosystem Support Products List – March 2018
57 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 58 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com More than just Break / Fix
58 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 59 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Significant Supported OSS Solutions for IBM i • Git • Jenkins • rsync • Node.js • Apache Tomcat • WordPress • Python • Redis • Ansible
• For more resources, see http://ibm.biz/ibmi-oss-support
59 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 60 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Is Open Source Strategic
"Development on IBM i continues to evolve, and open source development tools are common."
https://www.helpsystems.com/resources/guides/ ibm-i-marketplace-survey-results
60 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 61 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
61 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 62 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Core Infrastructure Initiative (CII)
A project managed by The Linux Foundation whose intent is to enable technology companies, stake holders, and developers to collaboratively identify, fund and improve the security of critical open source project.
Members of CII include
https://www.coreinfrastructure.org/
62 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 63 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source vs. Closed Source (proprietary)
Open Source Closed Source/Proprietary Malicious people have easy/ready Malicious people can't see source code access to source code but can still reverse engineer Funded by a community, coalition, or Funded by the owner foundation Zero to many developers Traditional development team Code quality can be verified – peer Code quality assured by owner – review typically internal review Anyone can fix defects Owner fixes defects Community maintains update schedule Owner maintains update Testing community driven including Owner-staffed test team "bounty hunters" Generally quick to adopt new protocols. Requirements submitted to product [OpenSSL support for TLSv1.3 since owner for new protocols. April 5] [NSS support since before then]
63 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 64 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
64 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 65 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Better availability through load balancing
NGINX is an open source package that supports functions such as web serving, reverse proxy, caching, load balancing, media streaming and more.
And YES, NGINX runs on IBM i!
65 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 66 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
66 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 67 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com IBM addresses modernization with every new version
• FastCGI • Allows fast connection from HTTP server to backend PASE environment • Instrumental in PHP performance • ILE Object Toolkit • Allows connects from various languages (PHP, Python, perl, ruby, etc) to ILE objects and IBM I native artifacts • SQL Connector • Easily integrated way to transfer data to and from Db2 for i leveraging SQL.
67 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 68 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com The Web is driven by Open Source
• Languages • Application Framework/Servers • PHP – Apache Tomcat / TomEE • Python – Jboss EAP – Greenfish • Ruby – Rails • Javascript – Epxress.js • Packages – Salis.js • JSON / XML – Django • Swagger API – Bottle framework – Flask • HTTP Servers • SOAP libraries – Apache HTTP Server • Web frameworks – nginx – Eclipse Jetty
68 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 69 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
69 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 70 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Skills in High Demand • Open Source skills are the skills being sought after • Universities and trade schools offer wide-range of open-source related courses
https://www.tiobe.com/tiobe-index/
70 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 71 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Skills
• Acquisition • Easy to find developers to write applications on your IBM I
• Utilization • Open Source technologies/languages provide opportunity for current staff to learn new skills
71 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 72 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
72 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 73 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Analytics/Business Intelligence • Many packages available in open source communities
• Often these packages can run on the IBM i, or integrate with IBM i with the help of open source technologies
73 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 74 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Integration with other data sources • Most new/modern languages have OSS packages to interact with social media • Various OSS packages facilitate the consumption of XML or JSON feeds • Many databases have open source clients available
• Open Source DBMS enable new applications • MySQL/MariaDB • PostgeSQL • Open Source DBMS enable new streams of data • MongoDB Another area where open source on IBM i will facilitate handling • REDIS the growth in data source and data feeds is the implementation of NoSQL databases as a method of aggregating data stream input into DB2.
74 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 75 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Solutions address many of the top concerns of IBM i shops
https://www.helpsystems.com/resources/guides/ibm-i-marketplace-survey-results
75 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 76 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Free? • Sort of, but… consider • The "glue" required to integrate various technologies together • OSS licensing – not all are created equal • Staying current • Getting support
76 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 77 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Meets Business Needs
• latest security protocols, cipher suites, etc Security • Secure software trusted by virtually the entire industry
• Application-level availability High solutions, including pm2, Availability nginx, gunicorn, etc.
77 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 78 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Meets Business Needs
• Solutions for web development • Easy ways to achieve SOA, web Modernizing services, microservices • New languages • Unprecedented capabilities (IoT, Applications quantum computing, 3d visualization, etc etc etc)
• Well-known languages and frameworks IBM i Skills • Industry-standard tools (a “normalization” of the application developer's environment)
78 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 79 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Open Source Meets Business Needs
• Several business intelligence and reporting tools (BIRT) • A convenient conduit to cloud-based Data solutions like Watson Growth/Analytics/BI • Machine learning • Connectivity to social media or other databases
Reduction of IT • Sometimes…. FREE! • Often…. not free, but cost-competitive Spending • Affordable support options
79 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 80 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Greatest cybersecurity challenges
80 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 81 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Greatest cybersecurity challenges (top five) • Lack of security knowledge and skills
• Threats are changing constantly
• Balancing Security Controls and Business Efficiency
• Evolving technology (mobile, IoT, cloud, etc)
• Lack of budget for security projects
81 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 82 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Greatest cybersecurity challenges (top five) • Lack of security knowledge and skills ▪ OSS brings “googlable” solutions
• Threats are changing constantly ▪ OSS constantly evolves as new threats arise
• Balancing Security Controls and Business Efficiency ▪ OSS brings cost-effectiveness with industry-standard technology
• Evolving technology (mobile, IoT, cloud, etc) ▪ OSS thrives in all these areas and solutions already exist
• Lack of budget for security projects ▪ OSS has multimillion-dollar investments from many companies
82 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 83 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Evolution of the Platform • What’s Came Before – A few highlights ▪ CISC to RISC ▪ POWER Convergence ▪ LPARs (VMs) ▪ Db2 Enhancements
• The Evolution Continues ▪ PHP for Application Modernization ▪ 3rd Party Solutions (like phpGrid) for rapid modernization ▪ Open Source Support delivering languages and solutions
83 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 84 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Popular Languages on IBM i
https://spectrum.ieee.org/static/interactive-the-top-programming-languages-2018
84 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 85 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Machine Learning enabled on IBM i • RPM enablement • Python enablement • BLAS enablement • Db2 connection
• Most famous ML packages available ▪ Numpy, Pandas for data processing ▪ Scipy, Scikit Learn for ML and scientific analysis ▪ ipython, interactive python language support ▪ nltk, natural language toolkit for natural language ML process. ▪ matplotlib, jupyter notebook for visual/interactive ML/data analysis
85 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 86 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Demo-scikit-learn • Multi-label classification. ▪ Each sample has 20 features ▪ We try to use Support Vector Machine (SVM) based on the chosen features(2 features selected) by canonical correlation analysis (CCA) or principal component analysis (PCA), which make it easier for us to show on plot.
refs: https://scikit-learn.org/stable/auto_examples/plot_multilabel.html#sphx-glr-auto-examples-plot-multilabel-py
86 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 87 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Demo-scikit-learn • visualizing the stock market structure. ▪ try to get the stock open/close prices for some companies as training data. ▪ Using the GraphLassoCV to train the data. ▪ Using the trained model’s covariance matrix to calculate the classes and labels. ▪ Giving out the clusters output based on assinged labels. ▪ Meanwhile, visualizing the results into a 2d plane.
refs: https://scikit-learn.org/stable/auto_examples/applications/plot_stock_market.html#sphx-glr-auto-examples-applications-plot-stock-market-py
87 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 88 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Demo-scikit-learn
88 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 89 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com ActiveMQ • Open source message broker (Java-based) • Supported protocols: ▪ OpenWire ▪ STOMP ▪ MQTT ▪ AMQP ▪ REST ▪ WebSockets
89 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 90 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com .NET Available on IBM i!! • Mono has been ported to IBM i!!
• Community Effort
• Available via third-party RPM repository
https://bitbucket.org/ibmi/opensource/src/master/docs/yum/3RD_ PARTY_REPOS.md
90 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 91 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com DevOps-enabling tools with IBM i • git • Ansible • Jenkins • Jira • BarryCI • Nagios (monitoring)
91 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 92 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Git • Open Source distributed version control system • Source control for virtually any language • Powerful and flexible • Can have hooks to perform custom actions when new code is integrated • Already in use in the IBM i community
92 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 93 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Ansible with IBM i • Ansible comes with integration for RPM (yum) management!! ▪ (after all, Ansible, RPMs, and Yum are all RedHat technology!) • https://docs.ansible.com/ansible/latest/modules/yum_module.htm l
93 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 94 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Ansible+Jenkins in production use • Node.js CI system ▪ Ansible used to provision test environments and systems ▪ Jenkins oversees tests, provides dashboards, etc
94 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 95 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com PASE - Overview • By 2000, AIX and OS/400 were able to run on the same POWER processors. • This created the possibility for executables which are MI-based and AIX-based to run on the same hardware in the same partition • PASE makes it possible for those binaries to run in the same process • PASE is not a release of AIX but rather a set of AIX libraries ▪ Fitted to talk to SLIC rather than directly to the AIX kernel • PASE gets the memory from the same SLIC teraspace pools used by ILE ▪ For program run stack, heap, and shared memory ▪ PASE can ONLY see memory that PASE acquired through its own syscall APIs
95 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 96 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com What PASE isn’t! • PASE is not an emulated environment
• PASE is not a separate environment from IBM I ▪ As an example the same Integrated File System (IFS) is accessed from PASE as from any other part of IBM I ▪ A further example, applications/tools/programs started in PASE can access Db2 data and ILE programs/artifacts
96 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 97 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com PASE Architecture
• PASE provides a set of AIX shared libraries that run directly on the POWER processor ▪ Applications in PASE enjoy the same performance as applications running an AIX partition • A syscall interface provides ability for applications in PASE to call ILE applications and access Db2 resident data
97 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 98 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
PASE Applications IBM i Applications
IBM i Operating System (above TIMI)
PASE Shared Libraries
Technology Independent Machine Interface syscall SLIC (IBM i below TIMI) POWER Hardware
98 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 99 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com IBM Speak
• PASE supports the application binary interface (ABI) of AIX and provides a broad subset of the support provided by AIX shared libraries, shells, and utilities. • PASE also supports the direct processing of IBM PowerPC machine instructions, so it does not have the drawbacks of an environment that only emulates the machine instructions • PASE applications: ▪ Can be written in C, C++, Fortran, or PowerPC assembler ▪ Use the same binary executable format as AIX PowerPC applications ▪ Run in an IBM i job ▪ Use IBM i system functions, such as file systems, security, and sockets
https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzalf/rzalfwhatispase.htm
99 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 100 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com IBM Speak • PASE run-time runs on the Licensed Internal Code (LIC) kernel on the IBM i operating system. • The system provides integration of many common IBM i functions across PASE and other runtime environments including Integrated Language Environment (ILE) and Java. • PASE implements a broad subset of AIX system calls • System support for PASE enforces system security and integrity by controlling what memory a PASE program can access and restricting theprogram to use only unprivileged machine instructions
https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzalf/rzalfwhatispase.htm
100 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 101 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Integration – Getting from Here to There (and back again) • Call PASE from IBM i QSH CMD('ls /home/Erwin') • Call IBM i from PASE system "WRKOBJLCK OBJ(MYFILE) OBJTYPE(*FILE)"
• More robust integrations such as accessing Db2 and ILE programs from open source languages is possible – more on that later
101 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 102 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Working in PASE
• To be effective with working in the PASE environment requires a certain knowledge and/or affinity with working in *NIX-line environments • Shell concepts is a good skill to acquire.
Mini Agenda
• What is a Shell?
• Working with the shell • Working in PASE
102 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 103 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Agenda • Setting up the OSS ecosystem support in PASE • Setting up the user environment • Defining a container (chroot) • Installing OSS packages • Python – an example • Node.js - an example
103 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 104 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Setting up the OSS Ecosystem in PASE
104 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 105 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com RPM Pile Overview
• IBM has released an RPM pile that provides the packages from 5733-OPS (plus a lot more) in a RPM-based repository • This allows for installation of open source packages in a Linux- like fashion with PASE • Currently available on IBM i 7.2 and newer • The RPM pile contains many packages including: ▪ Node.js version 8 ▪ Python 3.6 ▪ The 'less' utility The RPMs are not AIX RPMs. They ▪ git are IBM i RPMs shipping IBM i ▪ The 'updated' and 'locate' utilities Software. Built on IBM i, for IBM i. ▪ GCC 6.3.0 and other development tools ▪ GNU Nano ▪ Others…
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20 Technology%20Updates/page/Open%20Source%20RPM's
105 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 106 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Installing the RPM/YUM support
• Step 1: Download the bootstrap file to your PC: ftp://public.dhe.ibm.com/software/ibmi/products/pase/rpms/bootstrap.sql • Step 2: Run the SQL script against the system you want to install RPMs on
106 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 107 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com RPM pile bootstrap
create or replace table qtemp.ftpcmd(cmd char(240)); create or replace table qtemp.ftplog(line char(240)); --delete from qtemp.ftpcmd with nc; --delete from qtemp.ftplog with nc; insert into qtemp.ftpcmd(CMD) values ('anonymous [email protected]') ,('namefmt 1') ,('lcd /tmp') ,('cd /software/ibmi/products/pase/rpms') ,('bin') ,('get README.md (replace') ,('get bootstrap.tar.Z (replace') ,('get bootstrap.sh (replace') with nc ;
107 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 108 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com RPM pile bootstrap (continued)
CL:OVRDBF FILE(INPUT) TOFILE(QTEMP/FTPCMD) MBR(*FIRST) OVRSCOPE(*JOB); CL:OVRDBF FILE(OUTPUT) TOFILE(QTEMP/FTPLOG) MBR(*FIRST) OVRSCOPE(*JOB); CL:FTP RMTSYS('public.dhe.ibm.com'); CL:QSH CMD('touch -C 819 /tmp/bootstrap.log; /QOpenSys/usr/bin/ksh /tmp/bootstrap.sh > /tmp/bootstrap.log 2>&1');
Select case when (message_tokens = X'00000000') then 'Bootstrapping successful! Review /tmp/README.md for more info' else 'Bootstrapping failed. Consult /tmp/bootstrap.log for more info' end as result from table(qsys2.joblog_info('*')) x where message_id = 'QSH0005' order by message_timestamp desc fetch first 1 rows only;
108 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 109 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com RPM/YUM installation steps continued • Step 3: Once the installation of the bootstrap is complete, start a terminal session ▪ This can be done via 5250 command 'call qcmd' ▪ Better yet, an SSH session can be established to the system • Step 4: Modify the PATH to include the bin directory for the packages installed by the bootstrap PATH=/QOpenSys/pkgs/bin:$PATH export PATH
109 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 110 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Useful Commands
Command Description bash A shell typically available on Linux systems. Features include command/file completion, and command recall. gcc GNU c Compiler rpm Used to install/manage packages built using the Redhat Package Manager. yum Yellowdog Updated, Modified – a wrapper around RPM that uses package repositories to simplify package installation and dependency resolution
110 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 111 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Setting up the User Environment
111 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 112 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Creating the User Environment
• A number of steps should be accomplished to create the user environment • Step 1: Create the user's home directory mkdir /home/
PATH=/QOpenSys/pkgs/bin:$PATH export PATH
bash – The first two lines update the path statement to include the location of the programs installed both by the bootstrap as well as subsequent 'yum install' commands – The third line causes the bash shell to be executed • NOTE: by default a PASE terminal session starts the 'ksh' shell
112 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 113 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Changing the default shell
• The default shell in the PASE environment is ksh (a favorite in the AIX space) • A better alternative to ksh is bash – a favorite in the Linux space especially for it’s features like command recall (up- arrow) and file-name completion (tab). • The ‘qsys2.set_pase_shell_info’ function can be used to change the default shell either for an individual user or for all users:
call qsys2.set_pase_shell_info(‘*DEFAULT’, ‘/QOpenSys/pkgs/bin/bash’);
• To check the default shell:
select authorization_name, pase_shell_path from qsys2.user_info where pase_shell_path is not null;
113 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 114 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Repository Definition • The RPM packages reside in a repository that is publicly accessible • The definition of the repository is located in the /QOpenSys/etc/yum/repos.d directory ▪ The repository file for the IBM RPM pile is ibm.repo [ibm] name=ibm baseurl=http://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo enabled=1 gpgcheck=0
Note: it is possible to use a local repository by downloading the files from the indicated FTP site and then uploading them to a directory on the system. The 'baseurl' would change to indicate 'file' and the path to the directory of RPMs.
Additional note: ACS has support for cloning the repository to a local server.
114 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 115 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Environment Variables Overview
• An environment variable is a name/value pair that can affect running process within a computing environment • The current environment can be output with the 'env' command
# env SSH_CONNECTION=10.165.1.229 49163 10.68.240.201 22 TZ=
115 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 116 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Environment Variables Variables of Note Variable Description USER Current user logged into system and using the current shell PWD The current working directory. This is the 'focus' of any command run on the system HOME The user's home directory. This is the directory that is typically used for storage of configuration files that affect a user's login environment as well as shell characteristics. NOTE: This directory does not exist by default in the IBM i environment – more on that in a moment. SHELL The current shell. The environment supports multiple shells including bash and ksh LOGNAME The login name of the user OLDPWD The previous (n-1) working directory
116 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 117 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com What is a Shell? • The command line used on “Unix™” systems (as well as Unix-like systems) as well as PASE • Like CL it can be used interactively, or run as a program • Like CL most commands are actually programs that get called – There are some “built in” commands • Unlike CL there are a number of varieties of shell – sh bourne shell – csh c shell – ksh korn shell – bash bourne again shell – qsh Q shell • There are some similarities and some differences • Most of the discussion here is not operating system specific – Will work on AIX, Linux, QSH in OS/400, other nasty Unix variants, etc
117 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 118 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Why do we care about the shell?
• All system configuration operations can be done through the shell – often more quickly then through a GUI
• Shell scripts can automate routine tasks such as backups, scheduled emails, etc.
• GUI can be used for a great amount of admin activities – However, the shell tends to be a comfort zone providing ability to fix things in case something goes wrong
118 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 119 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Different Types of Shells
• A number of shells are available each providing function/usability customized to a particular type of user:
• Popular shells include: – BASH (Bourne Again Shell) – PDKsh (Public Domain Korn Shell) – csh (C shell) – mc (Midnight Commander) – QSHELL (PASE shell) – ksh (Korn shell, default on AIX)
• Difference tends to be in scripting capabilities and user interface – Items such as command recall and file name completion are typically different
119 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 120 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Starting with bash
• bash stands for Bourne Again Shell Available in PASE! – Developed by Brian Fox in 1987 – One of the most popular shells available in Linux
• Bash incorporates features of the Korn and C shell (ksh and csh)
• Bash configuration files: /bin/bash – Bash executable /etc/profile – System wide initialization file for login shells ~/.bash_profile – Personal initialization file for login shells ~/.bashrc – Personal per-interactive-shell startup file ~/.bash_logout – Login shell clean file that executes when shell exits
120 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 121 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com A little more on BASH • Default Linux shell • This can be changed in a variety of ways • /etc/profile – login shell • $HOME/.profile • As we saw earlier, can be set as the default shell for PASE • Very powerful as a command line shell • Recall previous commands • Command and file completion with the
• Many programming features • Loops and conditionals
121 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 122 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Shell Environment
• The shell is an environment where commands can be entered and the Operations system can respond to them
• A key concept to the environment is environment variables – There are a large number of environment variables – HISTFILE: points to file containing the shell history, defaults to ~/.bash_history – HISTFILESIZE: how man last commands you wish to have in history – HOME: points to your home directory – PATH: set of directories to search when trying to execute a command – PS1: Prompt variable NOTE: All of these environment – USER: username variables are available when running bash in the PASE environment.
122 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 123 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Exploring the shell (command history/recall)
• The shell is the command-interpreter and as such there are a number of features that make it easier to work in and traverse the environment • The shell keeps a history of previous commands that have been executed. The 'history' command can be used to display a list of those commands: -bash-4.4# history 1 pwd 2 ls -l • NOTES: – The commands are shown preceded with a number. Any command in the history can be re-executed simply by entering ! Followed by the number from the command history list – Previously executed command can be recalled through use of the up-arrow key. This allows you to scroll through previously executed commands.
123 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 124 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Exploring the shell (file-name completion)
• A powerful feature of the bash shell is file-name completion • File-name completion s accomplished by pressing the
type ls /QO Press
Notice that the shell completes /QOpenSys/
Press
Notice that the shell provides a list of items under /QOpenSys/
124 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 125 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Shell metacharacters Symbol Meaning > Output redirection >> Output redirection (append) < Input redirection * File substitution wildcard; zero or more characters ? File substitution wildcard; one character [ ] File substitution wildcard; any character between brackets `cmd` Command substitution $(cmd) Command substitution | The pipe (connect output of command on right to input on command on left) ; Command Sequence || OR conditional execution && AND conditional execution
125 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 126 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Shell metacharacters (continued)
Symbol Meaning ( ) Group commands & Run command in the background # Comment $ Expand the value of a variable \ Prevent or escape interpretation of the next character << Input redirection " $val " Literal with variable substitution ' $val ' Literal without variable substitution
126 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 127 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Why are metacharacters important • The shell has two primary responsibilities • Walk the command-line looking for tokens • Cause a command string to be sent to the kernel • Tokens are identified by white-space • The metacharacters are considered tokens ls –l /home/erwin > /tmp/listing
127 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 128 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Useful Shell Constructs
• Arrow Up & Down: Scroll through recent commands used
• &&: command is only executed if preceding command was successful: command1 && command2
• alias: sets a command alias or prints defined aliases alias wrklnk=ls
• bg[jobid]: Resumes the suspended job in the background
• cd: changes current directory to directory indicated cd /home
128 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 129 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Useful Shell Constructs (continued)
• echo: Outputs the arguments echo "hello world"
• find [path][expression]: searches the directory indicating looking for files that match expression: find / -name passwd –print
• pwd – Prints the absolute pathname of the current working directory
• unalias – Removes an alias
• history – displays command history with line numbers
• umask – User file creation mask
• logout: exits the shell environment
• exit [n]: exits shell environment with exit status n
129 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 130 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com "bash"ing PASE – yes I went there ;-) • The bash shell is available for PASE • It is part of the RPM pile • Step 1: Install the RPM pile bootstrap • Step 2: Install bash yum install bash
130 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 131 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Input and Output • *nix programs start with three open files – Input (called stdin) (#0) STDIN (0) STDOUT (1) – Output (called stdout) (#1) program – Error output (called stderr) (#2) STDERR (2) • Our shell program uses the "echo" command to write to stdout • We can redirect the output to go to a file by using ">"
ls –l > output.txt
This will take the output of the "ls –l" command and write it into a file called "output.txt"
131 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 132 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Input and Output
• You can also specify which output goes to a file
ls –l 1> output.txt this is the same as before.
• Redirecting only error output
grep fred * 2> grep.err
this will redirect only the error output to the file grep.err
132 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 133 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Input and Output
• Use the "<" operator to redirect input – Equivalent to typing at the keyboard
• For example
sed "s/erwin/Erwin/g" < my.txt
This runs the command "sed" (an editor) changing "erwin" to "Erwin" and taking its input from a file called "my.txt"
• Input and output
sed "s/erwin/Erwin/g" < my.txt > my2.txt
133 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 134 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Common *nix commands • Unix is built around the idea of lots of little programs that all do one thing well – ls lists files – sed an editor – grep a searcher – cat a file outputer All of these are available in the – find a file locator IFS and can be run from an – sort sorts files IBM i shell (i.e., qsh, ssh – tr translates characters session) – ps list processes – seq print a sequence of numbers – … a thousand others Many available on PASE and more coming!
• Help for all these is in the “man” (for “manual”) command – e.g. man sed Not currently available in PASE
• Shell programs generally involve stringing lots of these together – Which leads us to “pipes”
134 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 135 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Pipes • Standard output (STDOUT) of one command/program is used as the standard input (STDIN) for the next command/program
STDOUT (1) STDIN (0)
ps –x | grep java • List processes, search for any involving java • Only the 'STDOUT' from the last command is actually output to the screen (unless re-directed) – NOTE: Any output to STDERR by the commands will be output to the screen – again unless re-directed.
NOTE: pipes are not limited to two commands… any number of pipes can be used to build a pipeline: cmd1 | cmd2 | cmd3 | …. | cmdx
135 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 136 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
SAMBA Deeper Dive
136 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 137 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com What is SaMBa
Briefly… SaMBa is… •SaMBa is an Open • Free Source file server and • Popular client compatible with Microsoft Networking, • Licensed under the GPL that comes with many • Andrew Tridgell’s current distributions of brainchild Linux • Highly configurable •SaMBa is the UNIX • Constantly being implementation of the enhanced SMB protocol (CIFS) • Vendor independent
137 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 138 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Protocol
SMB – CIFS • Server Message Block ▪ This protocol defines ways for a client to request files from a server ▪ The SMB protocol was created by IBM in 1985. − It was named the IBM PC Network SMB Protocol ▪ There are numerous SMB dialects − The dialect defines the level of Windows integration that the client making the file/print request supports − SAMBA 3.0 implements the NT-level dialect which allows for Domain Controller support and integration with Active Directory − SaMBa 4.0 implements the AD-level dialect which allows for Active Directory Services • Common Internet File System (CIFS) ▪ CIFS is an extension of Microsoft’s SMB protocol. It is optimized to run over the Internet or Intranets and implements file-level security mechanisms ▪ CIFS allows users to share documents over the Internet in the same way they share documents when running peer networking services on their internal SMB networks.
138 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 139 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Capabilities
• Allows users to share files • Allows users to share printers • Allows users to time sync their machines off of the SAMBA server • Allows the system administrator to perform remote password administration • A NetBIOS name server which, among other things, gives browsing support. ▪ SAMBA can be the master browser on your LAN if you wish • A tar extension to the client for backup of PCs • Ability to integrate with existing Active Directory domains • Ability to provide Active Directory services (new with version 4.0 of SAMBA)
139 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 140 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
SAMBA Configuration – The Basics
140 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 141 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Components of SMB
• SAMBA provides a number of programs and utilities that implements the SMB protocol support as well as providing Active Directory services.
Component Description smbd A daemon process that provides file and print services to SMB clients. nmbd A daemon process that provides NetBIOS name service and browsing support winbindd A daemon process used for integrating authentication and the user database into UNIX
141 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 142 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Programs & Tools
Program / Description Tool smbclient The smbclient program implements a simple ftp-like client. Used to access SMB shares testparm Simple syntax checker for Samba’s configuration file. smbstatus Provides access to information about the current connections to the smbd daemon. nmblookup Allows NetBIOS name queries to make from a UNIX host. smbpasswd Tool for changing LanMan and Windows NT password hashes on SAMBA servers. smbcacls A tool to set ACL’s on remote CIFS servers smbtree Text-based network neighborhood tool smbtar Make backups of data on CIFS/SMB servers smbspool A helper utility for printing on printers connected to CIFS servers
142 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 143 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Programs & Tools
Program / Description / Usage Tool smbcontrol Change the behavior of running smbd, nmbd and winbindd daemons rpcclient Used to execute RPC commands on remote CIFS servers pdbedit Used to maintain the local user database on a samba server findsmb Used to find SMB servers on the local network swat Web-based interface for configuring SAMBA wbinfo Retrieves and stores information related to winbind profiles Command-line utility that can be used to replace all occurrences of a certain SID with another SID ntlm_auth Helper-utility for external programs waiting to do NTLM- authentication smbcquoatas Used to set remote QUOTA’s on server with NTFS 5.
143 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 144 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Configuration – The Basics
• Configuration is broken down into sections • Sections, introduced by [sectionName]
Basic Sections include [global] • Specifies information such as workgroup, password checking, etc • Settings apply to all shares [homes] • Can be used to share users home directories [printers] • Used to specify printers to share Share Section • Settings apply to this share
144 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 145 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA – Defining Shares
• Sharing of file resources via SAMBA [public] is supported at the directory level – It is not possible to share individual files available = yes • A “file share” is denoted in the path = /public SAMBA configuration file with the public = yes “share name” in brackets guest only = no • Guest support for file shares is supported – this share allows users writable = yes who do not have accounts on the browseable = yes system to access the file share • File shares can be hidden such that it NOTE: On IBM i, SaMBa shares are is available but not included in the defined in /opt/samba/lib/smb.conf browse (windows network neighborhood) list • ‘public = yes’ is the same as ‘guest ok = yes’ – No password is required to connect to the service
145 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 146 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Sharing Users Home Directories
[homes] path = /home/%u browseable = no writable = yes
• The %u is a variable substitution that is replaced with the name of the user that connected to the share. • The ‘browseable = no’ line prevents the users’ home directory from being displayed as a share in the share list. • The ‘writable = yes’ line ensures that Samba will allow the user to write into his/her own directory (assuming they have the necessary Linux file permissions)
146 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 147 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Installation
• The SAMBA that is currently available in the PASE environment is actually an AIX build. • Documentation on the installation of SAMBA can be found at: ▪ http://www-01.ibm.com/support/docview.wss?uid=nas8N1020089
• Step 1: Extract the SAMBA package (commands are run from the PASE environment) mkdir samba cd samba jar xvf /QIBM/ProdData/OS/Samba.zip
$ mkdir samba cd samba jar xvf /QIBM/ProdData/OS/Samba.zip inflated: README.txt inflated: rpm_setup.sh inflated: samba-3.6.25a-1.aix7.1.ppc.rpm inflated: samba-3.6.25a.tar.gz inflated: samba_setup.sh
147 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 148 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Installation • Step 2: The installation of SaMBa utilizes the rpm and wget utilities. Ensure that these are installed: which rpm $ which rpm which wget /QOpenSys/pkgs/bin/rpm which wget no wget in /QOpenSys/pkgs/bin /QOpenSys/usr/bin /usr/ccs/bin /QOpenSys/usr/bin/X11 /usr/sbin . /usr/bin
• Step 3: If either of the packages are not available, install it with the yum command: – yum install wget
148 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 149 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Installation
$ yum install wget ibm | 1.5 kB 00:00:00 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package wget.ppc64 0:1.17.1-1 will be installed --> Finished Dependency Resolution
Dependencies Resolved
======Package Arch Version Repository Size ======Installing: wget ppc64 1.17.1-1 ibm 808 k
Transaction Summary ======Install 1 Package
Total download size: 808 k Installed size: 3.0 M Is this ok [y/N]: y Downloading Packages: wget-1.17.1-1.ibmi7.1.ppc64.rpm | 808 kB 00:00:00 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction Installing : wget-1.17.1-1.ppc64 1/1
Installed: wget.ppc64 0:1.17.1-1
Complete!
149 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 150 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com SAMBA Installation
• Step 4: Run the installer: ▪ chmod 755 samba_setup.sh ▪ ./samba_setup.sh install • Step 5: SAMBA and NetServer cannot both be running at the same time. Execute the following commands (from a 5250 session) to end NetServer ▪ CRTUSRPRF NOBODY ▪ ENDTCPSVR *NETSVR • Step 6: From a PASE terminal session start the SaMBa daemon process ▪ smbd &
150 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 151 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
Git "A 'git' is a cranky old man" Linus Torvalds – creator of git
151 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 152 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Overview • Created by Linux Torvalds in 2005 ▪ Came out of the Linux development community ▪ Designed for version control of the Linux kernel
• Goals of git ▪ Speed ▪ Support for non-linear development (thousands of parallel branches ▪ Fully distributed ▪ Able to handle large projects efficiently
152 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 153 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Checking for / Installing git
• The which command can be used to see if git is installed ▪ which git
• The yum install command can be used to install git ▪ yum install git
153 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 154 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Centralized vs. Distributed Version Control Systems
Centralized VCS Distributed VCS • Centralized VCS like Subversion, • Distributed VCS like git doesn't support a central server repository "checkout" from a local repo (repo) holds the "official copy" – Code is "clone"d and changes are of the code "pull"ed ▪ The server maintains the sole • The local repo is a complete copy of version history of the repo everything on the remote server • "checkouts" of the code are – Local copy is "just as good" as made to a local copy the remove server copy ▪ Local modifications are made • Many options are local ▪ Changes are not versioned – Check in/out from local repo • When complete, the code is – Commit changes to local repo "checked in" back to the server – Local repo keeps version history ▪ The checkin increments the repo's • When ready changes can be "pushed" version back to the server
154 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 155 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Local git areas
• In a local copy on git, files can be: ▪ In the local repo (committed) ▪ Checked out and modified, but not yet committed (working copy) ▪ In-between, in a "staging" area − Staged files are ready to be committed − A commit saves a snapshot of all staged states.
155 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 156 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Initial git configuration
• Set the name and email for git to use when commits are made: ▪ git config --global user.name "Erwin Earley" ▪ git config --global user.email [email protected] • Set the editor used for writing commit messages (default is vim) ▪ git config --global core.editor nano Review the settings: # git config --list • user.name=Erwin Earley ▪ git config --list [email protected] core.editor=nano
156 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 157 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Cloning a git repo (local git repo)
• Creating a local git repository in the # cd python-scripts/ bash-4.4# git init current directory Initialized empty Git repository in ▪ git init /home/QSECOFR/python-scripts/.git/ − bash-4.4# git add *.py The above creates a .git directory in your bash-4.4# git commit -m "initial commit" current directory [master (root-commit) 4263c42] initial − Files can then be committed in that directory commit 14 files changed, 167 insertions(+) into the repo create mode 100644 Ex01hello.py ▪ To stage all changes in a file (or directory) for the next create mode 100644 Ex02Indent.py commit: create mode 100644 Ex04Strings.py create mode 100644 Ex05Lists.py git add filename create mode 100644 Ex06tuples.py ▪ Commit the staged changes using the –m argument as create mode 100644 Ex07Dictionary.py the commit message create mode 100644 Ex10ifs.py create mode 100644 Ex11Fors.py git commit –m "commit message" create mode 100644 Ex12While.py create mode 100644 Ex15Functions.py create mode 100644 Ex16Functions2.py create mode 100644 Ex17Functions3.py create mode 100644 datatype.py create mode 100644 variables.py
157 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 158 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Cloning a git repo (remote repo)
• To clone a remote repository to the local directory ▪ git clone url localDirectory − Ex: git clone https://github.com/zendtech/ibmiToolkit toolkit
The above will create the specified directory (toolkit) that will contain a working copy of the files from the repo (https://github.com/zendtech/ibmiToolkit) as well as a .git directory that will be used to hold the staging area as well as the local repository.
158 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 159 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com git commands Command Description git clone url Copy a git repository so files can be added [dir] to it git add file Adds file contents to the staging area git commit Records a snapshot of the staging area git status View the status of files in the working directory and staging area git diff Shows the diff of what is staged and what is modified but unstaged git pull Fetch from a remote repo and try to merge into the current branch git push Push new branches and data to a remote repository Good git cheatsheet https://www.atlassian.com/git/tutorials/atlassian-git-cheatsheet
159 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 160 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Hands-on Exericse - Overview • An ssh client is needed to access the lab environment (5250 should not be used) ▪ PuTTY is the typical client for Windows (ssh web browser from google could also be used) ▪ MAC has ssh built in ▪ NOTE: ssh should be used in favor of QSH or CALL QP2TERM • A token is provided with connection details including credentials
• If you run into problems/questions with the lab contact the instructor at [email protected]
160 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 160 161 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com
COMMON Open Source on IBM i – common99
• System idevphp.idevcloud.com (port 22 – ssh) • User Name/Team common99 • Password labpass • Port 10801
Latest version of presentations and hands-on Questions/problems with hands-on exercises: exercises available at: Send mail to [email protected] http://idevphp.idevcloud.com/webinar
161 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 162 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Lab Outline • Establish connection to the lab environment • Get familiar with working at the CLI of the PASE environment • Work with text editors ▪ nano ▪ joe • Explore file system concepts ▪ Permissions ▪ Ownership ▪ Links • Work with processes
162 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 162 163 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com Day 2 Preview • Exploring Containers on IBM i
• Managing Open Source on IBM i
• Exploring Open Source Programming Languages
163 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 163 164 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com The /QOpeSys directory structure • /QOpenSys is where all of the open source files reside
/
dev home lib QOpenSys sbin tmp usr var
bin etc lib pkgs sbin tmp usr var
container
bin dev home QOpenSys lib sbin tmp usr var
bin etc lib pkgs sbin tmp usr var
164 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 164 165 | COMMON Webinar Series: Open Source on IBM i | November 2019 zend.com The /QOpeSys directory structure
Directory Contents bin Commands dev Device Files etc Configuration files home User Home Directories lib Libraries pkgs Package files / commands sbin Privileged commands tmp Temporary files usr Utilities & Applications var Variable files
165 | Zend by Perforce © 2019 Perforce Software, Inc. zend.com 165