Install Binaries
Total Page:16
File Type:pdf, Size:1020Kb
ZenPack Development without a running Zenoss Prerequisites Setup pip and common pip packages Clone common github repo dependencies Make changes to certain git repos zenoss-protocols Zope zenoss-prodbin Virtual Environment (Optional) Create virtual environment Install python packages Zenoss is dependent on Modify requirements.txt Developing a ZenPack Creating the ZenPack files and folders Running the ZenPack with a basic unit test Building the ZenPack Moving these test and build steps into scripts Running the ZenPack with advanced unit tests Standalone unit test Integration unit test The following are instructions on how to setup an environment that allows you to develop a ZenPack locally Prerequisites The following documentation assumes the following is already setup / available OS Linux (like Cent OS 7 which is what was used in developing these instructions) a user with sudo privs python 2.7 git Here's an example of checking your system for the prerequisites [zdeveloper@localhost ~]$ python --version Python 2.7.5 [zdeveloper@localhost ~]$ git --version git version 1.8.3.1 Install binaries Install the following binaries (if not already installed). We install a subset (we avoid mariadb-devel-5.5.37 and openldap-devel ) from what is listed in Dockerfile.in at https://github.com/zenoss/zenoss-py-deps python-devel python-pip (NOTE: Pip is part of Extra Packages for Enterprise Linux (EPEL), which is a community repository of non-standard packages for the RHEL distribution. You may have to set that up in your package manager before installing pip) gcc make libxml2-devel libxslt-devel libffi-devel protobuf-compiler NOTE: On centos we used the package manager yum (modified output showing these are already installed) Example listing of installed binaries [zdeveloper@localhost ~]$ sudo yum install python-devel ... Package python-devel-2.7.5-69.el7_5.x86_64 already installed and latest version [zdeveloper@localhost ~]$ sudo yum install python-pip ... Package python2-pip-8.1.2-6.el7.noarch already installed and latest version [zdeveloper@localhost ~]$ sudo yum install gcc ... Package gcc-4.8.5-28.el7_5.1.x86_64 already installed and latest version [zdeveloper@localhost ~]$ sudo yum install make ... Package 1:make-3.82-23.el7.x86_64 already installed and latest version [zdeveloper@localhost ~]$ sudo yum install libxml2-devel ... Package libxml2-devel-2.9.1-6.el7_2.3.x86_64 already installed and latest version [zdeveloper@localhost ~]$ sudo yum install libxslt-devel ... Package libxslt-devel-1.1.28-5.el7.x86_64 already installed and latest version [zdeveloper@localhost ~]$ sudo yum install libffi-devel ... Package libffi-devel-3.0.13-18.el7.x86_64 already installed and latest version Setup pip and common pip packages Still using Dockerfile.in at https://github.com/zenoss/zenoss-py-deps as a reference Change the version of pip to 9.0.1 (ignore that it's an old version) [zdeveloper@localhost ~]$ pip install --upgrade pip==9.0.1 Requirement already up-to-date: pip==9.0.1 in /usr/lib/python2.7/site-packages You are using pip version 9.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Install the following common python packages using pip wheel virtualenv setuptools [zdeveloper@localhost ~]$ pip install wheel Requirement already satisfied: wheel in /usr/lib/python2.7/site-packages [zdeveloper@localhost ~]$ pip install virtualenv Requirement already satisfied: virtualenv in /usr/lib/python2.7/site-packages [zdeveloper@localhost ~]$ pip install setuptools Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages Clone common github repo dependencies For the below git hub repos we cloned from within within parent folder "/home/zdeveloper/Documents/commongithubrepos" https://github.com/zenoss/zenoss-py-deps https://github.com/zenoss/zenoss-prodbin https://github.com/zenoss/zenoss-protocols https://github.com/control-center/service-migration https://github.com/zenoss/ZenPacks.zenoss.ZenPackLib https://github.com/zenoss/ZenPacks.zenoss.PythonCollector https://github.com/zenoss/pynetsnmp https://github.com/zopefoundation/Zope For example git clone https://github.com/zenoss/zenoss-py-deps Make changes to certain git repos zenoss-protocols The zenoss-protocols project is missing some files and requires you to build some python package(s) e.g. python/zenoss/protocols/protobufs. Navigate to zenoss-protocols/python and compile make GNUmakefile build Zope The Zope branch used by Zenoss is 2.13 Navigate to Zope and change the git branch to 2.13 git checkout 2.13 zenoss-prodbin The Zenoss Product has many different versions. You can choose which version to test against by changing the branch. Let's use a known stable branch like "support/5.3.x" NOTE: You can use github to browse what other branches are available. git checkout support/5.3.x Virtual Environment (Optional) Before we pip install a lot of Python packages Zenoss uses into your general python, we recommend you create a virtual environment to prevent interference with other programs using python and visa-versa Create virtual environment In the below steps/examples we start within parent folder "/home/zdeveloper/Documents/virtualenvs" 1. Create the virtual environment "testlatestzenossenv" [zdeveloper@localhost virtualenvs]$ virtualenv testlatestzenossenv New python executable in /home/zdeveloper/Documents/virtualenvs/testlatestzenossenv/bin/pyth on2 Also creating executable in /home/zdeveloper/Documents/virtualenvs/testlatestzenossenv/bin/pyth on Installing setuptools, pip, wheel... done. 2. Activate the virtual environment (the beginning of your command prompt should now indicate you are in that virtual environment) [zdeveloper@localhost virtualenvs]$ source testlatestzenossenv/bin/activate (testlatestzenossenv) [zdeveloper@localhost virtualenvs]$ 3. You can exit the virtual environment with "deactivate" (testlatestzenossenv) [zdeveloper@localhost virtualenvs]$ deactivate [zdeveloper@localhost virtualenvs]$ Install python packages Zenoss is dependent on Install what is listed in requirements.txt https://github.com/zenoss/zenoss-py-deps IF YOU ARE USING VIRTUAL ENVIRONMENT MAKE SURE IT IS ACTIVE!!!!! You can do this one at a time or can feed the file into pip like so (this assumes you have navigated to the cloned git repo zenoss-py-deps where requirements.txt resides) NOTE: This capture of the failing is on purpose, read on (testlatestzenossenv) [zdeveloper@localhost zenoss-py-deps]$ pip install -r requirements.txt Collecting AccessControl==2.13.7 (from -r requirements.txt (line 1)) Collecting Acquisition==2.13.8.1 (from -r requirements.txt (line 2)) Could not find a version that satisfies the requirement Acquisition==2.13.8.1 (from -r requirements.txt (line 2)) (from versions: 2.11.0b1, 2.11.1, 2.11.2, 2.11.3, 2.12.0a1, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.13.0, 2.13.1, 2.13.2, 2.13.3, 2.13.4, 2.13.5, 2.13.6, 2.13.7, 2.13.8, 2.13.9, 2.13.10, 2.13.11, 2.13.12, 4.0a1, 4.0, 4.0.1, 4.0.2, 4.0.3, 4.1, 4.2, 4.2.1, 4.2.2, 4.3.0, 4.4.0, 4.4.1, 4.4.2, 4.4.3, 4.4.4, 4.5) No matching distribution found for Acquisition==2.13.8.1 (from -r requirements.txt (line 2)) (testlatestzenossenv) [zdeveloper@localhost zenoss-py-deps]$ Modify requirements.txt The default requirements.txt should have failed in a similar way for you, so here’s changes that were made (may need to be tweaked as requirements,txt is tweaked for the Zenoss Product) Acquisition==2.13.8.1 changed to Acquisition==2.13.8 MySQL-python==1.2.3 Removed python-ldap==2.4.6 Removed PyXML==0.8.4 Removed RelStorage==2.0.0.1.dev14 changed to RelStorage==2.0.0 Yapps==2.1.1 changed to Yapps==2.2 zenpacksupport==1.0 Removed zope-app-compat==1.0 Removed Developing a ZenPack The previous steps covered setting up an environment to develop a ZenPack locally. Those steps should only have to be done once (assuming you continue to work in the same envrionment) These next steps are repeatable steps to create a ZenPack from scratch and develop it in an iterative cycle We use the Zenoss SDK WeatherUnderGround zenpack example (https://zenpack-sdk.zenoss.com/en/latest/tutorial-http-api/wunderground-api.h tml) to demonstrate local development. Creating the ZenPack files and folders Since this is a local env, we cannot create a base zenpack using the command "zenpacklib --create" provided in a running Zenoss The 3 files in the ZenPacks.zenoss.ZenPackLib git repo, containing code that is run by "zenpacklib --create" are... ZenPacks/zenoss/ZenPackLib/bin/zenpacklib ZenPacks/zenoss/ZenPackLib/zenpacklib ZenPacks/zenoss/ZenPackLib/lib/libexec/ZPLCommand.py Using that python code, you can construct a script to mimic the boiler plate build out of files and folders for a ZPL based ZenPack. With the example script called build_zenpack.py as a starting point, you can customize it further to automate repetitive steps you encounter during zenpack development As an example we can create the Zenoss SDK WeatherUnderGround zenpack called ZenPacks.training.WeatherUnderground NOTE: In the below execution the script was located in "/home/zdeveloper/Documents/scripts" but run from the projects folder "/home/zdeveloper/Documents/projects" (testlatestzenossenv) [zdeveloper@localhost projects]$ python ../scripts/build_zenpack.py ZenPacks.training.WeatherUnderground - making directory: ZenPacks.training.WeatherUnderground/ZenPacks/training/WeatherUndergroun d - creating file: ZenPacks.training.WeatherUnderground/setup.py - making directory: ZenPacks.training.WeatherUnderground/tests - making init file: ZenPacks.training.WeatherUnderground/tests/__init__.py