bmon Documentation Release 0.0.1
Alan Mitchell
Sep 16, 2021
Users
1 User Introduction 3
2 System Administrator Introduction5
3 Developer Introduction 7
4 Contact Information 9 4.1 User Introduction...... 9 4.2 System Administrator Introduction...... 10 4.3 How to Install BMON on a Web Server...... 13 4.4 How to Install BMON on a Local Web Server...... 18 4.5 Add Buildings and Sensors...... 27 4.6 Sharing BMON across Multiple Organizations...... 40 4.7 Setting Up Sensors to Post to BMON...... 41 4.8 Multi-Building Charts...... 60 4.9 Sensor Alerts...... 68 4.10 Creating a Dashboard...... 74 4.11 Transform Expressions...... 76 4.12 Calculated Fields...... 80 4.13 Periodic Scripts...... 92 4.14 How to Create Custom Jupyter Notebook Reports...... 107 4.15 Custom Reports...... 108 4.16 Backing Up and Analyzing Data from the System...... 111 4.17 System Performance with High Loading...... 113 4.18 Using CSV Transfer...... 115 4.19 Developer Introduction...... 120 4.20 BMON Architecture...... 120 4.21 Writing Periodic Scripts...... 126 4.22 Accessing Data in BMON: API and Python Library...... 129 4.23 Contact Information...... 135
i ii bmon Documentation, Release 0.0.1
Copyright (c) 2014, Alaska Housing Finance Corporation. All Rights Reserved. BMON is a web-based software application that stores and analyzes sensor data coming from buildings or other facilities. The application was developed by the Alaska Housing Finance Corporation (AHFC) to find ways to reduce the energy use and improve the maintenance of their buildings. The application presents a simple user interface for viewing data, but includes informative charts such as Histograms and Hourly Profile charts for analyzing the data. Alerts can be set up to text or email individuals if sensor values move outside normal ranges. Here is a link to the AHFC BMON Web Site Here is a screenshot of the application being used to look at the electricity usage of the AHFC Headquarters building (the green bands indicate the building’s occupied periods):
Note that a major upgrade to Python 3.7 occurred on May 1, 2019. Upgrade of existing systems running the Python 2.7 version will require assistance from the developer, Alan Mitchell, [email protected]. Please contact him before attempting an upgrade. The documentation for the BMON software is divided into three main sections, described below.
Users 1 bmon Documentation, Release 0.0.1
2 Users CHAPTER 1
User Introduction
This section contains documentation for users who are viewing and analyzing the data in the system. This documen- tation does not address any issues related to installing and configuring the system.
3 bmon Documentation, Release 0.0.1
4 Chapter 1. User Introduction CHAPTER 2
System Administrator Introduction
This section describes how to install the BMON application on a web server, which will require some basic skills with Linux system administration. This section also describes how to setup and configure the specific buildings and sensors for your system. The setup of sensors and buildings does not require any sophisticated IT skills; it is accomplished through a web interface after logging on to the system as an administrator.
5 bmon Documentation, Release 0.0.1
6 Chapter 2. System Administrator Introduction CHAPTER 3
Developer Introduction
This section provides documentation for developers who want to modify the code of the BMON system. The source code of the project is internally documented with comments, but the documentation in this section explains the overall structure of the application. The GitHub repository holding the source code is located here
7 bmon Documentation, Release 0.0.1
8 Chapter 3. Developer Introduction CHAPTER 4
Contact Information
Contact information for key BMON personnel and developers is available here.
4.1 User Introduction
A number of training videos have been developed to explain the features of the BMON system. The table below summarizes the available videos.
9 bmon Documentation, Release 0.0.1
Video Title Description Basic Features of This video shows how to select various various reports and graphs for each building. The the AHFC BMON video also explains a number of graphing features, such as zooming, highlighting points, and Web Application hiding data series on the graph. These features are common to all the graphs on the web site, (9 minutes) and this video is an important one to watch prior to watching the videos below. Multiple Building This video explains the graphs that are available to compare energy use across multiple build- Comparisons and ings that are being monitored. It also describes the “Current Values” report, which shows the Current Values current sensor readings for any one building. Report (13 Minutes) “Plot over Time” This video explains the “Plot over Time” graph, also known as the “Time Series” plot. This Chart (Time Series graph lets you display how one or more sensor values change over time. This is the workhorse Graph) graph of the system, so this video is important. (8 minutes) Hourly Profile This video explains the “Hourly Profile” chart. This graph lets you chart how a sensor’s value Chart changes throughout the day (hourly). It also allows you to select particular days to look at, (8 minutes) such as Monday through Friday or just Saturday. Histogram Chart This video explains the “Histogram” chart. This graph is good for learning about the range of (6 minutes) values measured by a sensor. It also shows you the values that are recorded most frequently and least frequently. Download Sensor This video explains how to download sensor data into an Excel spreadsheet. Data as Excel Spreadsheet (5 minutes)
4.2 System Administrator Introduction
This documentation explains how to install and configure the system for your particular buildings and sensors. These activities should be the task of one or two system administrators who have more experience with BMON than the general Users who are simply interested in viewing and analyzing the data. The separate system administrator topics are summarized below, and the associated documents are available through the sidebar on the left or by clicking on the section headings below.
4.2.1 BMON Installation
This document gives instructions for installing BMON on a web server. Linux skills and preferably some experience with Python and Django are necessary to perform this task.
4.2.2 Local BMON Installation
This document gives instructions for installing BMON on a local web server. Linux skills and preferably some expe- rience with Python and Django are necessary to perform this task.
4.2.3 Adding Buildings and Sensors
After installing BMON, you will need to configure your Buildings and Sensors in the system. This document shows how to do that using the web-based Admin interface of the BMON system.
10 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
4.2.4 Sharing BMON Across Multiple Organizations
A single BMON server can be shared across multiple organizations. This document describes how that is accom- plished.
4.2.5 Setting Up Sensors to Post to BMON
A few different sensor types have been set up to work with BMON (e.g. Monnit Wireless Sensors). Other sensors that have access to the Internet can be configured to work with BMON. This document gives details on how to set up sensors in the system.
4.2.6 Multi-Building Charts
BMON can produce some reports and graphs that compare data across buildings. These multi-building charts need to be configured through the web-based Admin interface. Single building reports and charts work out-of-the-box without additional configuration.
4.2.7 Sensor Alerts
You can have BMON send you a text message or email if important sensor conditions occur, such as Domestic Hot Water temperatures that are too low. This document details how to configure this feature.
4.2.8 Creating A Dashboard
A summary “Dashboard” can be created for any of the buildings in the BMON system. Here is an example:
This document describes how Dashboards are created.
4.2. System Administrator Introduction 11 bmon Documentation, Release 0.0.1
4.2.9 Custom Reports
This document explains how to create Custom Reports. A Custom Report allows you to combine any number of graphs, dashboards, or current value reports onto one page. The various elements can even come from different buildings.
4.2.10 Transform Expressions
Some sensors do not send data in a displayable format, some sensors have small errors that need correction. “Trans- form” expressions allow you to convert units or transform values before storage and display in BMON. This document explains how to set up transform values for your sensors.
4.2.11 Calculated Fields
This document explains how to work with calculated fields. Occasionally, you may want to calculate a value from multiple different sensors or have one sensor display its value in two different ways. “Calculated Fields” serve this need. Also, Calculated Fields can be used to acquire data from Internet weather services instead of installing your own meteorological sensors.
4.2.12 Periodic Scripts
Sometimes a process needs to occur repeatedly on a periodic basis. Often this is used to acquire data from a piece of equipment or a server connected to the Internet. It also could be used to create and send a report or perform a maintenance operation. The Periodic Script feature of BMON can be used for this purpose. There are some periodic scripts that are already available in BMON (such as to acquire data from Ecobee thermostats), and it is possible for a developer to create new scripts that will be periodically run by BMON, this document explains the basics of Periodic Scripts while a later document details writing custom scripts.
4.2.13 Backing Up and Analyzing Data from the System
This document explains how key BMON databases are backed up on the server and gives suggestions on off-server backup strategies. The document also explains how sensor data is stored in BMON and how it can be archived or exported from the system for analysis elsewhere (basic knowledge of database systems is required). The document is also useful if you need to clean-up or remove data from the system.
4.2.14 System Capacity
A stress test was done on the BMON system using high rates of sensor reading posts and chart/report requests. Also, large amounts of historical data were present in the test. This document describes the results of that testing.
4.2.15 Importing Data from CSV Files
This document explains how to use the csv-transfer utility to import csv data into BMON.
12 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
4.3 How to Install BMON on a Web Server
This document describes how to install the BMON application on a web server. The specific steps shown here apply to installation on the Webfaction Hosting Service, which has been used for most of the installations of the BMON application. The general goals of each step of the installation are also described, to assist in installation on other hosting providers or a standalone server. The assumption here is that BMON will be installed on a server running the Linux operating system, although instal- lation on a Windows server is also possible (the backup_db() method in the bmsdata.py file uses the gzip Linux utility, which can be installed on Windows through use of Cygwin). The skills needed for installation are primarily: • Linux command line skills. • Some experience with Python and the Django web framework would be helpful, as the BMON application is a Python application built using the Django web framework. If you are using the Wefaction hosting services, these skills are not absolutely necessary, as step-by-step installation instructions for that hosting service are detailed below. If you are not using the Webfaction hosting service, more Django and Python skills will be required, as the instructions below will need to be modified to fit your configuration. • Some knowledge of the Git version control system would be helpful.
4.3.1 Create the Website and Install the Django Application
The BMON application needs to have Django 2.2.1 or greater and Python 3.7 or greater installed. On the Webfaction hosting service, Python 3.7 is already installed, so none of the steps below perform that task. If installing on a system without Python 3.7, you will first need to install Python. The rest of the steps below install Django 2.2.1 or greater, and configure the server to use Django to serve requests being made on the Domain you will use for the BMON application. The steps in this section are patterned after the general Webfaction documentation for installing Django. 1. Log into the Webfaction Control Panel and click DOMAINS/WEBSITES from the menu bar. 2. Click the button Add new website. 3. In the Name field, type in bmon. 4. It is highly recommended that an Encrypted website (https) be created, because the BMON application utilizes a password-protected log in screen for administrative access to the application. Those passwords will travel the Internet encrypted if an https site is used. Select Encrypted website (https). You will need to install an SSL certificate in order for users to not receive a security warning when they access the site. For the Webfaction host, see their SSL documentation. 5. In the Domains field, select or type in the domain or subdomain where you want to access the BMON site. For this example, we are accessing the site on the subdomain bmon.ahfctest.webfactional.com, so we type that into the field. If your domain does not exist, you will get a prompt that will allow you to create the domain or subdomain. 6. Click the Add an application button and select Create a new application. • For the Name of the application, use bmon_django; the App category is Django; the App type is a Django release of 2.2.1 or greater, paired with Python 3.7. Leave the other fields at their default values. Click the Save button to complete the Application setup. 7. Now we need to create another application to serve static files for the BMON application such as JavaScript files, stylesheets, and images. As before, click the Add an application button and select Create a new application.
4.3. How to Install BMON on a Web Server 13 bmon Documentation, Release 0.0.1
• For the Name of the application, use bmon_static; the App category is Static; the App type is Static only (no .htaccess); enter static into the URL field (for the example, the final value in that field reads https://bmon.ahfctest.webfactional.com/static). Click the Save button to complete the Application setup. 8. The website configuration is now complete. The website screen in this example looks like:
9. Click the Save button on the website screen to complete the application setup. 10. Test progress by going to your website URL (https://bmon.ahfctest.webfactional.com in this example) and you should get an “It Worked” page in response. This may not function immediately, as some time is required for the server to process website configuration.
4.3.2 Set Up Email on the Web Server
The BMON application has the ability to send out notifications when sensor values are outside of normal ranges. In order to do this, BMON needs the ability to send emails. For the Webfaction hosting service, email setup is described on this page. The minimum requirements for the BMON application are to set up one mailbox and one email address on the Webfaction system. For example, with a Webfaction user account name of ahfc, you could set a mailbox with the name of ahfc and an email address of [email protected]. In a subsequent section, the mailbox name, password, and email address will be needed for entry in the BMON Settings File.
4.3.3 Installing the BMON Project
In this step, the Python code for the BMON project is installed and connected to the Django application. 1. Open an SSH session to the web host.
14 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
2. Navigate to the Django application directory by entering cd $HOME/webapps/bmon_django and pressing Enter. 3. We first need to remove the sample project that was created when we installed Django. Enter rm -rf ./ myproject and press Enter. 4. Clone the BMON project into this directory by executing:
git clone https://github.com/alanmitchell/bmon.git
You should now have a bmon directory containing the BMON software. 5. Changes are required so that Django runs this project instead of the myproject project you just removed. Execute the cd $HOME/webapps/bmon_django/apache2/conf command to move to that directory. Then, change all the occurrences of myproject to bmon in the file httpd.conf by executing the following command:
sed-i"s/myproject/bmon/g" httpd.conf
Alternatively, you can use an editor to replace occurrences of myproject with bmon in the httpd.conf file.
4.3.4 Install Python Packages
The BMON application uses some code libraries that do not come with Python. In this section, these libraries will be installed with Python’s pip package manager using a requirements.txt file that lists the required packages. For installing on Webfaction, first change into the BMON directory, cd $HOME/webapps/bmon_django/bmon, and then run the following command:
pip3.7 install--user-r requirements.txt
This command can take a few minutes to complete. There is some chance the installation process will be killed due to excessive memory usage (probably while installing the pandas package, and you will receive an email to that effect from Webfaction. The email will contain a link that will take you to a Webfaction Support ticket. Inform Webfaction that you are temporarily installing Python packages, and they will temporarily relax the memory limit.
4.3.5 Creating the BMON Settings File
Django requires a settings.py file to provide essential information for running a project. We will start with a sample settings file and make necessary changes. • Move to the $HOME/webapps/bmon_django/bmon/bmon directory (yes, the bmon folder inside the bmon folder) and create a settings.py file from the sample located there:
cp settings_example.py settings.py
• Open settings.py in an editor, e.g. nano settings.py. Each one of the settings is documented within the file. Examine each setting carefully for any needed changes. Write down the BMSAPP_STORE_KEY that you created, as this will be needed when you are setting up sensors to post to the BMON system.
4.3. How to Install BMON on a Web Server 15 bmon Documentation, Release 0.0.1
4.3.6 Create the Django Database, Install Initial Data, and Prepare Static Files
1. Go to the main project directory cd $HOME/webapps/bmon_django/bmon or cd .. if you just finished the prior step. 2. Create the Django database file by executing:
./manage.py migrate
3. Some initial data for common sensor units, sensor categories, and a sample building and sensor should be loaded into the database by:
./manage.py loaddata init_data.yaml
4. Copy the static files (images, stylesheets, JavaScript files, etc.) into the folder where they will be served by the Static Application you created. Do this by executing:
./manage.py collectstatic
5. In order to use the Admin site for setting up sensors and buildings, we need to create an admin user. To do this, execute:
./manage.py createsuperuser
Enter your desired username, email, and password to complete the setup. The username and password created here will be the credentials needed to log into the Admin side of the BMON site. 6. The final stage of the process is to restart the Apache web server. Change to Apache’s bin directory by executing cd $HOME/webapps/bmon_django/apache2/bin. Enter ./restart and press Enter. You should now be able to view the new BMON site by browsing to the Domain URL that you set up in the initial section of this installation. You should also be able to access the Admin portion of the site by browsing to
4.3.7 Cron Jobs
One cron job is necessary for the BMON application. To edit the your crontab file, execute crontab -e. Then, add the following line to the file, making changes to the path if BMON was installed in a non-standard directory:
*/5 **** cd~/webapps/bmon_django/bmon&&./manage.py runscript main_cron>/dev/ ˓→null2>&1
This cron job: • creates calculated reading values and stores Internet weather data in the reading database every half hour • checks for active Alert Conditions every five minutes • runs any Periodic Scripts that been configured in the BMON system • creates a daily status line in the log file indicating how many sensor readings were stored in the database during the past day (viewable by browsing to
16 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Note that the command executed by the cron job has two parts: 1) first, it changes into the base BMON directory, and then it executes the manage.py script. This two step process is necessary on the Webfaction server because the Django Python package may only be available for scripts executed from a BMON directory. If you choose to implement an off-server backup strategy, as described in Backing Up and Analyzing Data from the System you may have an additional cron job task.
4.3.8 Redirecting HTTP to HTTPS
If you want to automatically redirect users who browse to the unsecured HTTP domain name for your site, you can set up HTTP to HTTPS redirection. For the Webfaction hosting service, this process is described on this page. However, it is important to modify the .htaccess lines to the following:
RewriteEngine On RewriteCond %{HTTP:X-Forwarded-SSL} !on RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,NE,L]
The NE parameter on the last line ensures that the redirection does not distort certain query strings that may appear in the URLs.
4.3.9 Time Savers
It is useful, but optional, to define a couple aliases in your .bashrc file located in the home directory. These will allow for quick navigation to the BMON application directory and easy restarting of the Django application: alias cddj='cd ~/webapps/bmon_django/bmon' alias rsdj='~/webapps/bmon_django/apache2/bin/restart'
4.3.10 Steps for Upgrading the BMON Software
Developers are frequently upgrading the BMON software on its main GitHub repository. In order to update your installation to the latest version, follow these steps: 1. Open an SSH shell session to the server. 2. Stop the Django application by navigating to the Apache bin directory, /home/
# ------Generally shouldn't need to change anything beyond here ------
If there are new settings required, edit your bmon/settings.py file to include the new settings with appro- priate values.
4.3. How to Install BMON on a Web Server 17 bmon Documentation, Release 0.0.1
8. Restart the Django application by navigating to the Apache bin directory, /home/
4.3.11 Maintaining the Sensor Reading Database
The sensor readings posted to the BMON system are stored in a SQLite database dedicated to that purpose. For information on the structure and location of that database, see Backing Up and Analyzing Data from the System. Occasionally, maintenance operations, outlier removal, data archival and other database tasks need to be performed. One approach is to open a secure shell connection (SSH) to the web server and use the sqlite3 command line tool to manipulate the database. Alternatively, a web-based database administration tool can be installed onto the server, so that manual database operations can be performed through a web interface. One such tool is phpLiteAdmin. The tool allows viewing the sensor data, executing SQL statements, and exporting sensor reading tables. Installation of the tool is straight-forward and documented on the web page link above. When using the Webfaction hosting service, installation of the Static/CGI/PHP application is required to run the phpLiteAdmin tool, as this tool is a PHP web application.
4.3.12 Next Step: Add Buildings and Sensors
The next step for configuring the BMON system is to use the Admin interface available at
4.4 How to Install BMON on a Local Web Server
This document describes how to alternatively install the BMON application on a local web server. The specific steps shown here apply to installation on a locally housed Linux server running NGINX and UWSGI. This installation process has been tested and reproduced multiple times in a test environment, however your mileage may vary. There are many ways to configure a web server, this is just a method that worked for us - install at your own risk. The assumption here is that BMON will be installed on a server running the Linux operating system. The skills needed for installation are primarily: • Linux command line skills - it is assumed you are familiar so explanations of basic linux command line opera- tions are not included in this documentation. • Some experience with Python and the Django web framework would be helpful, as the BMON application is a Python application built using the Django web framework. If you are using the Wefaction hosting services, these skills are not absolutely necessary, as step-by-step installation instructions for that hosting service are detailed below. If you are not using the Webfaction hosting service, more Django and Python skills will be required, as the instructions below will need to be modified to fit your configuration. • Some knowledge of the Git version control system would be helpful.
4.4.1 Install and Configure the Server Operating System
1. Download the current LTS (long-term support) Version of Ubuntu Server from the Ubuntu Website. This docu- mentation was written for Ubuntu Server 18.04.2 LTS.
18 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
2. Install the operating system on a suitable network capable physical server or virtual machine Follow the self guided installation instructions for Ubuntu, optionally installing OpenSSH for remote access. 3. Run updates using sudo apt-get update and sudo apt-get upgrade
4.4.2 Installing Necessary Packages
In order to run BMON on a local web server you must first install some required packages
Python Dependencies sudo apt install build-essential checkinstall sudo apt-get install -y build-essential checkinstall libreadline-gplv2-dev python3.7 python3.7-dev libpython3.7-dev uwsgi uwsgi-src uuid-dev libcap-dev libpcre3-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev zlib1g-dev openssl libffi-dev python3-dev python3-setuptools wget
SQL Server sudo apt-get install mysql-server sudo mysql_secure_installation
Python Pip cd $home sudo apt-get install python3-pip sudo -H pip3 install --upgrade pip
4.4.3 Installing the Virtual Environment
It is assumed we are working from within our $home directory
Virtualenv & Virtualenvwrapper sudo -H pip3 install virtualenv virtualenvwrapper
Adjust your .bashrc file to incorporate the virtual environment variables
You may need to run whereis virtualenvwrapper.sh and document the path. sudo nano ~/.bashrc
Add the following to the end of the file: export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3 export WORKON_HOME=~/Env source /usr/local/bin/virtualenvwrapper.sh
4.4. How to Install BMON on a Local Web Server 19 bmon Documentation, Release 0.0.1
PYTHON=python3.7
Save the file and then run: source ~/.bashrc
If the parameters added to your bashrc are correct - virtualenvwrapper scripts will be created like this: virtualenvwrapper.user_scripts creating /home/cchrc/Env/premkproject virtualenvwrapper.user_scripts creating /home/cchrc/Env/postmkproject virtualenvwrapper.user_scripts creating /home/cchrc/Env/initialize virtualenvwrapper.user_scripts creating /home/cchrc/Env/premkvirtualenv virtualenvwrapper.user_scripts creating /home/cchrc/Env/postmkvirtualenv virtualenvwrapper.user_scripts creating /home/cchrc/Env/prermvirtualenv virtualenvwrapper.user_scripts creating /home/cchrc/Env/postrmvirtualenv virtualenvwrapper.user_scripts creating /home/cchrc/Env/predeactivate virtualenvwrapper.user_scripts creating /home/cchrc/Env/postdeactivate virtualenvwrapper.user_scripts creating /home/cchrc/Env/preactivate virtualenvwrapper.user_scripts creating /home/cchrc/Env/postactivate virtualenvwrapper.user_scripts creating /home/cchrc/Env/get_env_details
Create your virtual environment cd home mkvirtualenv bmon -p python3.7
You will know you are successful if an Env directory is created within your home directory. Once you make your virtual environment, while it’s active, your prompt will change to indicate you are working in the virtual environment, below is an example of how it will look
Note: To exit your virtual environment type deactivate to resume working in your virtual environment type workon bmon for the instructions below continue working in your virtual environment until instructed to exit.
4.4.4 Installing the Required Python Packages & BMON Project
Install BMON & Required Packages sudo git clone https://github.com/alanmitchell/bmon.git
20 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
A directory named bmon will be created in your $home directory
cd bmon
pip3 install -r requirements.txt
4.4.5 Creating BMON Settings File
cd bmon (you should be in bmon/bmon now) Django requires a settings.py file to provide essential information for running a project. We will start with a sample settings file and make necessary changes.
Move to the $home/bmon/bmon directory (yes, the bmon folder inside the bmon folder) and create a settings.py file from the sample located there: sudo cp settings_example.py settings.py
Open settings.py in an editor, e.g. sudo nano settings.py. Each one of the settings is documented within the file. Examine each setting carefully for any needed changes.
In this example, we modify the following attributes: BMSAPP_STORE_KEY - per the settings file, visit https://bms.ahfc.us/make-store-key to generate a key BMSAPP_TITLE_TEXT - purely cosmetic, change XYZ to the name of your organization/facility/etc. BMSAPP_HEADER - purely cosmetic, change XYZ to the name of your organization/facility/etc. ALLOWED_HOSTS - change to the server IP address or the URL depending on your setup ex. [‘172.20.127.167’] (brackets and single quotes necessary) SECRET_KEY - per the settings file, visit https://www.miniwebtool.com/django-secret-key-generator/ to generate a key BMSAPP_STATIC_APP_NAME - indicate the full path of your project to the first level, then add static ex. /home/cchrc/bmon/static
4.4.6 Configuring the Manage.py file
Unlike in the How to Install BMON on a Web Server documentation, we need to modify the manage.py file to point to the correct python location type which python3.7 and note the path ex. /home/cchrc/Env/bmon/bin/python3.7 cd $home/bmon or cd .. if you just finished the prior step.
Modify manage.py sudo nano manage.py Change #!/usr/local/bin/python3.7 to whatever path came up when you typed which python, but the line must begin with #! before the path to the Python executable.
4.4. How to Install BMON on a Local Web Server 21 bmon Documentation, Release 0.0.1
Test the manage.py file for errors sudo ./manage.py check
You’re looking for System check identified no issues (0 silenced) If you get a permission denied error make sure your path is typed correctly in the manage.py file
4.4.7 Create the Django Database, Install Initial Data, and Prepare Static Files
Create the Django database file by executing: sudo ./manage.py migrate
Some initial data for common sensor units, sensor categories, and a sample building and sensor should be loaded into the database by: sudo ./manage.py loaddata init_data.yaml
Copy the static files (images, stylesheets, JavaScript files, etc.) into the folder where they will be served by the Static Application you created. Do this by executing: sudo ./manage.py collectstatic
In order to use the Admin site for setting up sensors and buildings, we need to create an admin user. To do this, execute: sudo ./manage.py createsuperuser
Enter your desired username, email, and password to complete the setup. The username and password created here will be the credentials needed to log into the Admin side of the BMON site.
Test the development server by running the following: sudo ./manage.py runserver SERVERIP OR URL:8000
Then go to SERVERIP OR URL:8000 in your web browser and see if you see a poorly formatted version of BMON (the CSS doesn’t load in development). End the test by pressing Ctl-C to kill the process
4.4.8 Configuring the Webserver
Note: The steps in this section are patterned after the general instructions from Digital Ocean’s How To Serve Django Applications with uWSGI and Nginx on Ubuntu 18.04
Note: We are still working in the virtualenvironment
Install & Compile UWSGI
22 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 cd $home sudo wget https://projects.unbit.it/downloads/uwsgi-latest.tar.gz tar zxvf uwsgi-latest.tar.gz sudo rm -rf uwsgi-latest.tar.gz cd to the newly made uwsgi directory python3.7 uwsgiconfig.py --build make PROFILE=nolang
Create the python 3.7 plugin
PYTHON=python3.7 ./uwsgi --build-plugin "plugins/python python37"
Link the file sudo ln -s /home/cchrc/[uwsgi directory]/uwsgi /usr/local/bin/uwsgi
Now, end your virtual session by typing deactivate sudo mkdir -p /etc/uwsgi/sites sudo nano /etc/uwsgi/sites/bmon.ini example ini file
[uwsgi] plugins-dir = /home/cchrc/[uwsgi directory] plugin = python37 project = bmon uid = cchrc base = /home/%(uid) chdir = %(base)/%(project) home = %(base)/Env/%(project) module = %(project).wsgi:application pythonpath = %(base)/Env/%(project)/bin/python3.7 master = true processes = 5
4.4. How to Install BMON on a Local Web Server 23 bmon Documentation, Release 0.0.1 socket = /run/uwsgi/%(project).sock chown-socket = %(uid):www-data chmod-socket = 660
Explanation: plugins-dir = the location of your uwsgi install plugin = tells uwsgi to use python 3.7 as default chown-socket = YOURUSERNAME:www-data home = /path/to/home/Env/bmon chdir = /path/to/project pythonpath = /path/to/home/Env/bmon wsgi-file = /path/to/home/bmon/bmon/wsgi.py
Create a Service File sudo nano /etc/systemd/system/uwsgi.service example service file
[Unit] Description=uWSGI Emperor service
[Service] ExecStartPre=/bin/bash -c ‘mkdir -p /run/uwsgi; chown cchrc:www-data /run/uwsgi’ ExecStart=/usr/local/bin/uwsgi –emperor /etc/uwsgi/sites Restart=always KillSignal=SIGQUIT Type=notify NotifyAccess=all
[Install] WantedBy=multi-user.target
The only portion of the service file that needs to be modified in your installation is ExecStartPre=/bin/bash -c 'mkdir -p /run/uwsgi; chown cchrc:www-data /run/uwsgi'
24 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 where chown should indicate YOURUSERNAME:www-data cd $home
Install NGINX sudo apt-get install nginx sudo nano /etc/nginx/sites-available/bmon example bmon settings file
server { listen 80; server_name 172.20.127.197; location = /favicon.ico { access_log off; log_not_found off; } location /static/ { root /home/cchrc/bmon; } location / { include uwsgi_params; uwsgi_pass unix:/run/uwsgi/bmon.sock; } }
The only portion of this file that needs to be changed is server_name which should be changed to either your server IP address or URL and root should reflect your own directory structure.
Enable the Site sudo ln -s /etc/nginx/sites-available/bmon /etc/nginx/sites-enabled
Create a uwsgi run directory sudo mkdir /run/uwsgi sudo chown -R cchrc:www-data /run/uwsgi sudo chmod -R 774 /run/uwsgi
Change some owners and permissions to make sure the files are accessible
4.4. How to Install BMON on a Local Web Server 25 bmon Documentation, Release 0.0.1
sudo chown -R cchrc:www-data /home/cchrc sudo chmod -R 774 /home/cchrc/
Here, you would substitute YOURUSERNAME where cchrc is and /your/home/path where /home/cchrc is Create an override file (this was done to fix some errors)
sudo mkdir /etc/systemd/system/nginx.service.d
sudo nano /etc/systemd/system/nginx.service.d/override.conf
Put a space in the file and save Change owners and permissions for the override file
sudo chmod 666 /etc/systemd/system/nginx.service.d/override.conf sudo chown cchrc:www-data /etc/systemd/system/nginx.service.d/override.conf
sudo printf "[Service]\nExecStartPost=/bin/sleep 0.1\n" > /etc/systemd/system/nginx.service.d/override.conf
4.4.9 Start the Server
sudo systemctl daemon-reload sudo systemctl restart nginx
sudo nginx -t you want to see the following nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful
sudo systemctl start uwsgi
To check the status of any process type the following: sudo systemctl status SERVICENAME (ex. nginx) You should now be able to reach your project by going to its respective domain name or IP address from your preferred web browser.
If you are unable to access the site through your web browser you can test by entering sudo uwsgi --http SERVERIP OR URL:8080 --home /home/cchrc/Env/bmon --chdir /home/cchrc/bmon -w bmon.wsgi
26 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
and visiting the URL. | reviewing the nginx error log may also help troubleshoot sudo tail -30 /var/log/ nginx/error.log If you see messages about /run/uwsgi/bmon.sock failed (2: No such file or directory) while connecting to upstream it usually means you need to rerun the permission settings for the /run/uwsgi folder.
Note: change the IP address in the line above with either your server’s ip address or URL specified in your configu- ration
If everything works, do the following to have nginx uwsgi start automatically
sudo systemctl enable nginx sudo systemctl enable uwsgi
4.4.10 Cron Jobs
One cron job is necessary for the BMON application. To edit the your crontab file, execute crontab -e. Then, add the following line to the file:
*/5 **** cd/home/cchrc/bmon&&./manage.py runscript main_cron>/dev/null2>&1
This cron job: • creates calculated reading values and stores Internet weather data in the reading database every half hour • checks for active Alert Conditions every five minutes • creates a daily status line in the log file indicating how many sensor readings were stored in the database during the past day (viewable by browsing to
4.5 Add Buildings and Sensors
To start using the BMON system, Sensors and Buildings must be added to the system. A Sample building and a Sample sensor were added to BMON when it was installed on the web server. This document will show how to add your actual buildings and sensors to the system. The sample building and sensor can be deleted once at least one actual building and sensor are added to the system.
4.5.1 Logging into BMON’s Admin Interface
Adding sensors and the general configuration of BMON occurs through the Admin interface. To access the Admin interface, navigate to
4.5. Add Buildings and Sensors 27 bmon Documentation, Release 0.0.1
4.5.2 Adding a Building
Once logged in, the following screen should appear; click the Buildings link as shown below to access the part of the Admin interface that allows you to configure buildings.
Once in the Buildings section of the Admin interface, you will see the list of buildings that have already been entered into the system. If the system is new, only the Sample Building will be shown. In the screenshot below, a number of buildings have already been configured. As the image shows, click the Add building button to add another building.
28 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
An Add building page will appear, the top portion of the page contains the essential inputs for adding a new building:
Here are the inputs for entering a new building: Title: Enter the name of this building as you want it to appear in all of the reports and graphs. Current Operating Mode: (Optional) This input indicates the current operating mode of the building. The input is only used when evaluating Sensor Alerts, which are discussed elsewhere. Until you start using Sensor Alerts, you can leave this input at its default value, ‘——‘. Additional Building Documentation: (Optional) This input allows for images, links, and other additional building- wide notes to be entered using Markdown syntax. These images, links, or notes will be displayed at the bottom of the page on the Current Sensor Values report as well as the Dashboard report, if configured. There are many great resources online for learning to use Markdown syntax, including this tutorial. The two most important features for building documentation are adding in web-links and links to images. To add a web link, simply type in:
[link text](http://www.enter_your_web_link_here.com)
Note that you must include the url prefix (e.g. http://) in your links.
4.5. Add Buildings and Sensors 29 bmon Documentation, Release 0.0.1
Adding a link to a picture is similar, but with an added exclamation point before the link syntax:
If your documents and images aren’t already hosted somewhere on the web, google drive offers an easy and free way to host things online. First, if you don’t already have an account, you will need to create one. Then, you will need to upload the pictures or documents to google drive. Lastly, you will need to make the link public, share it, and paste it to gdurl.com as detailed in this 1-minute how-to video Latitude and Longitude: The BMON system includes the ability to show monitored buildings on a Map. Enter the Latitude and Longitude of the building, in decimal format, so it is properly placed on the map. You can determine the lat/lon of a building from Google Maps using these instructions. Time Zone of Facility: Enter the timezone where the building is located, for example “US/Alaska”. The list of valid time zones comes from the Olson Timezone database and is available here. Occupied Schedule of Facility: (Optional) Some buildings are not occupied 24/7. A number of the BMON graphs and reports highlight sensor values during the occupied periods of the building. If this building is not continually occupied, enter its occupied schedule here. Here is an example of how you would enter the schedule of a building that is occupied 8a - 5p on Mon - Fri and Noon - 4p on Saturday:
M-F:8a-5p Sa: 12p-4p
Here is a more complicated schedule:
M-F:8a-5p Tu, Th :6:30p-7p,8p-9:45p
The input allows for multiple lines of entry, each line being a different set of occupied periods. Days are separated from time ranges with a colon. Days can be abbreviated or spelled out. Day ranges like Mon-Fri are valid, or individual days can be separated by commas, like M, W, F. Start and end times are separated by a -, and multiple time ranges can be separated by a comma as shown in the prior example. Annotations for events in the building’s timeline: (Optional) If significant changes have occurred in a building’s timeline (i.e. energy retrofits, additions, changes to mechanical system, etc.), this input gives you the ability to annotate those dates on the BMON timeseries charts. The input allows for multiple lines of entry, with each line representing an event that will show up on the timeseries. On each line, enter the text that will be displayed on the chart followed by a colon followed by the date where the annotation will appear. The date can include also include a time of day if desired. Here is a sample entry:
Roof Insulation Retrofit:6/12/2016 Boiler Replaced:1/1/20173:45PM
There are a couple more Input sections beyond these essential inputs. The one labeled “Bldg to sensors” is used to associate particular sensors with this building. These inputs will be discussed in the Adding Sensors section below. Another input section is “Dashboard items”. These inputs will be discussed in the Creating a Dashboard document. After completing the essential inputs discussed above, Save the new building by clicking either the Save button or the Save and add another button (if you want to add another building). These buttons are found in the lower right corner of the screen and look like:
30 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
4.5.3 Building Groups
You can organize Buildings into Groups. Users can filter the list of buildings that they are analyzing down to a particular Building Group. A Building Group could represent a geographical region or a group of buildings that are the responsibility of a particular staff member. There are no restrictions on the meaning of each Building Group, and any one Building can be a member of multiple different groups. It is not necessary to set up any Building Groups. If this feature is not of interest, skip to the next section titled Adding Sensors. To set up a Building Group, you need to go to the main Admin Screen. If you just finished adding a building, you return to the Home screen of the Admin interface by clicking the Home link in the upper left corner:
Then, on the Home screen click the Building groups link in the left-hand column. In the upper right corner of the next screen, click the Add building group button. The Building Group editing screen is shown. Below is a screenshot after a Building Group titled Residential has been set up. There are 7 buildings shown in the right hand box that are in the Residential group; the two buildings on the left are not in the group.
4.5. Add Buildings and Sensors 31 bmon Documentation, Release 0.0.1
4.5.4 Adding Sensors
This section describes how to add sensors into the BMON system. If you just finished adding a building, you first need to return to the Home screen of the Admin interface by clicking the Home link in the upper left corner:
Once back to the main Home screen, click the Sensors link to add or edit sensors:
And then to add a new Sensor, click the Add sensor button in the upper right:
32 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Here are the key inputs for configuring a new sensor:
4.5. Add Buildings and Sensors 33 bmon Documentation, Release 0.0.1
34 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Monnit Sensor ID, or Calculate Field ID: Each sensor that posts its data to the BMON system needs to have a unique ID, which is entered here. For Monnit wireless sensors, the ID is the Monnit Sensor ID found on the label attached to the sensor:
An example of a Monnit Sensor ID is 29938. If the sensor is a 1-Wire temperature sensor attached to a Mini-Monitor, the ID is the 1-Wire ID for the sensor, such as 28.7CED4B050000. If this “sensor” is actually a calculated field or a value retrieved from an Internet weather service, you must create an ID that is not in use by any other sensor. The ID can contain numbers and letters and cannot be more than 30 characters long. An example of such as ID is anc_merril_temp (Anchorage Merril Field temperature). Title: The Title input is the descriptive name of the sensor and is the name displayed to users of the system. An example is Boiler Supply Temp. The Title must be 50 characters or less. You do not need to include the building name in the Title, as that will be displayed separately. Unit: Select the units for this sensor’s values, such as temperature: deg F. A number of different Unit types are available, but you can easily add others by clicking the green plus symbol to the right of the dropdown box. When adding a new Unit, you are asked for the Unit Label (deg F in the above example) and the Measurement Type (temperature in the above example). Any Units you add will be permanently available for other sensors. Please enter descriptive notes about the sensor: This section allows descriptive notes to be added for each sensor. These notes will be shown to the user on the Current Sensor Values page when the mouse hovers over the sensor, as shown below.
4.5. Add Buildings and Sensors 35 bmon Documentation, Release 0.0.1
This section should be used to give additional information to the user, such as the specific location of the sensor, the type of sensor, and any details about the measurements. The default text that will be shown to the user if nothing is entered in this field is “No sensor notes available.” Calculated Field: For a sensor that is posting values to the BMON system, this checkbox should not be checked. If the “sensor” you are adding is actually calculated from other sensor values or is acquired from the Internet, then this box should be checked and you should refer to the calculated-fields document for information on how to create Calculated Fields. Transform or Calculated Field Function Name: If the value that is posted by the sensor needs to be converted to a different value before being stored and displayed to the user, this input needs to be filled in. The simplest way of using this input is to type in a math expression that performs the conversion. Use the variable val to represent the sensor’s raw value before conversion. For example, if the sensor posts in degrees C and you want to convert to degrees F, you would use the following conversion expression:
1.8 * val+ 32
As another example, let’s say that you know that a temperature sensor is reading 1.5 degrees too high. You can use a
36 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
conversion expression to correct the reading:
val- 1.5
There are more complicated uses of this input. For example pulse counter sensor values can be converted to engineer- ing units like kW or Btu/hr through use of a Transform Expression. Read about these uses here. This input is also used for calculated-fields. Function Parameters in YAML form: For the more sophisticated uses of Transform Expressions and calculated- fields, extra information is sometimes needed and is entered here. Refer to those documents for more information. For simple sensors, this input should be left blank. Calculation order: This input is used with calculated-fields and affects which calculations are done first. For normal sensors or calculated fields where calculation order does not matter, the default value of 0 should be used. Formatting Function Name: Sensors always report their values as numbers. Sometimes these numbers really repre- sent a state that is better displayed as a word or phrase. For example, the alarm contacts on a Boiler will report a value of 1 when the Boiler has alarmed or gone off line and will report a 0 when the boiler is functioning properly. It is more informative to see those values as “Alarm” and “OK”. This input can be used to provide this formatting. There are number of functions available to convert reported values into more readable phrases. Some of the most commonly used ones are: • ‘‘alarm_formatter‘‘: Displays a 0 as OK and anything else as Alarm. To use this function, simply type alarm_formatter into the Formatting Function Name input box. • ‘‘on_off_formatter‘‘: Converts a 0 into Off and a 1 to On. Type on_off_formatter into the Formatting Function Name input box to use this formatting function. • ‘‘occupied_formatter‘‘: To be used with occupancy sensors. Converts a 0 from the sensor to Vacant and a 1 to Occupied • There are a number of special status codes reported by the Burnham Alpine Boilers and the AERCO boilers, which are monitored by the Mini-Monitor setups. These status codes have special formatting functions that are described in the Mini-Monitor documentation.
Once these key inputs have been filled out for the Sensor, you can associate the sensor with a Building (or multiple buildings). To do this, you use the inputs that appear just below those described above:
First, select the Building from the dropdown box (AHFC Headquarters in the example above). Next, users see sensors grouped into categories for a more organized display. Choose a Sensor Group from the dropdown (Domestic Hot Water in the example above). If an appropriate Sensor Group is not available, you can add one by clicking the green plus symbol to the right of the dropdown; just supply a Title for the new Sensor Group and a Sort Order value that will be used to order your new Sensor Group relative to the other groups used for this building. Finally, enter a Sort order for this sensor, which will be used to order the sensor within the Sensor Group. Leaving space between your Sort order values (e.g. 10, 20, 30, etc.) makes sense so a future sensor can be fit in between existing sensors. In some situations, it makes sense to associate a Sensor with multiple buildings. An example would be an outdoor temperature sensor that is near to multiple buildings. Seeing the temperature value when viewing each building is useful. To associate the Sensor with another building, click the Add another Bldg to sensor link that you
4.5. Add Buildings and Sensors 37 bmon Documentation, Release 0.0.1
see at the bottom of the image above. An additional row of inputs will appear allowing you add another building association. Note that you also can set up associations between Sensors and Buildings by going to the Edit screen for a particular building, and midway down that screen is an area where sensors can be selected and associated with the Building. Linking Sensors and Buildings can be done either on the Sensor edit screen or the Building edit screen; they are equivalent. Below the Bldg to Sensors inputs are inputs related to Sensor Alerts. Please see the Sensor Alerts document to learn how to setup notifications that will be sent out via Text or Email if this sensor reads outside of a normal range of values. To Save the input values you have just entered for this Sensor, click one of the Save buttons at the bottom right of the screen:
4.5.5 Filtering the List of Sensors
The full sensor list for the BMON system can become quite long, making it difficult to find a particular sensor to edit. However, a Filter feature is available to improve this situation. When you are on the main Sensor screen, there is a Filter box present at the right side of the screen:
By clicking on one of the Building names, you will filter the Sensor list down to just sensors associated with that building. There is also a Filter available to just show Calculated Fields or non-Calculated sensors.
4.5.6 Deleting Sensors, Building Associations, and Alerts
If you want to delete one or more sensors from the system, you can do so from the main Sensor screen:
38 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Check the boxes next to the sensors to be deleted, select Delete selected sensors from the Action drop- down, and then click the Go button. You can also delete an entire sensor while you are on the Editing screen for that sensor. Click the Delete button in the lower left corner of the screen:
When you need to delete an association that you have set up between a Sensor and a Building, you need to go to the edit screen for the Sensor and perform the following steps:
4.5. Add Buildings and Sensors 39 bmon Documentation, Release 0.0.1
1. Click the checkbox to delete the building association 2. Click save Do not use the red Delete button in the lower left corner of the screen as that will delete the entire sensor, not just the association between the sensor and the building. This same general approach is used when deleting other list items like Alerts–check the Delete box for the Alert and then click the Save button for the Sensor.
4.6 Sharing BMON across Multiple Organizations
One BMON server can be used to serve multiple Organizations. If this multiple-organization feature is used, an Organization drop-down selector will appear at the top of the BMON page, as shown in the screenshot below:
To set up multiple organizations, add Organizations using the link shown in the screenshot below:
40 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
When you add a new Organzation, you will be presented with inputs that require you to identify which Buildings, Building Groups, Multi-Building Charts, and Custom Reports belong to the Organization. If the BMON user chooses this Organization, only these items will be displayed in the BMON user interface.
4.7 Setting Up Sensors to Post to BMON
This document gives specific instructions on how to set up a couple different types of sensors to post their data to BMON system. In addition, general documentation is given to allow other types of sensors to post to BMON if they have Internet access ability and the ability to use the HTTP protocol to export data. Specific instructions are provided for: • Monnit Wireless Sensors • The Mini-Monitor, a Raspberry Pi data acquisition system designed for the Alaska Housing Finance Corporation (AHFC) Building Monitoring project. • Kors HTTP Poster for Niagara AX, a software module that allows building control systems utilizing the Niagara AX software framework to export data via HTTP Internet transactions.
4.7.1 Monnit Wireless Sensors
Monnit wireless sensors transfer their collected data to the iMonnit Internet site, where the data can viewed and the configuration of the sensor can be changed. Fortunately, the iMonnit software has a feature that allows the wireless sensor data to be sent to a separate Internet site, such as a BMON web site. To configure this feature for use with BMON, log in to the iMonnit system and perform the following steps to navigate to the page that allows you to configure a “Webhook”:
4.7. Setting Up Sensors to Post to BMON 41 bmon Documentation, Release 0.0.1
• On the home page after log in, click the cog icon in the upper right to access a drop-down menu. Select “API/Webhook” on that menu. • A horizontal top menu bar is present on the resulting page. Click “Webhook”. • On the next page, click the “Configure Webhook” button near the top right corner. You are then presented with the form shown here:
On this screen, only one input needs to the entered, the Base URL (required) input. Here is a specific example of the input:
https://bms.ahfc.us/readingdb/reading/store/?format=monnit&storeKey=123xYz
and here is the general form:
[Base URL of BMON site]/readingdb/reading/store/?format=monnit&storeKey=[Store Key]
The two parameters that are specific to your BMON site are the Base URL of the BMON site and the Store Key associated with your BMON site (found in the settings.py file). Configuring this Webhook for the iMonnit site will cause all subsequent sensor readings to be sent to the BMON site and stored there. No configuration at the sensor level is required. You need to substitute the web URL for your BMON site in the [URL of BMON site] placeholder, and you need to substitute your Store Key in the [store key] placeholder. When you installed BMON on the web server, you created and entered the Store Key as the BMSAPP_STORE_KEY setting in the BMON Settings File. After Saving the dialog box, sensor data should be forwarded to the BMON site after it arrives at the iMonnit site. On the previous configuration dialog, there is a Last Result field that shows the result from the last sensor post to BMON. It should say X readings stored successfully, O rejected if the process is working. If not, an error message will appear in this field. In the BMON system, each Monnit wireless sensor will appear with a BMON Sensor ID that is the same as the Monnit sensor ID; for example the Monnit Sensor with ID 139883 will appear in the BMON system with a Sensor ID of 139883.
4.7.2 Mini-Monitor
The Mini-Monitor data acquisition system based on the Raspberry Pi computer was developed for the AHFC Building Monitoring project. It directly posts data to the BMON system. As described in the Mini-Monitor Documentation, there are two critical entries for data posting in the Mini-Monitor settings file:
42 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
POST_URL='[URL of BMON site]/readingdb/reading/store/' POST_STORE_KEY='[store key]'
[URL of BMON site] and [store key] have the same meaning as they did for the Monnit Wireless sensors. More information on setting up the Mini-Monitor is provided in its documentation.
4.7.3 Kors HTTP Poster for Niagara AX
If you have a building DDC control system that utilizes the Niagara AX software platform (e.g. the Tridium Jace system), you can add a $100 software module from Kors Engineering that will allow you to post point values to the BMON system. Documentation that comes with the Kors Poster module will assist in its install and configuration. You should use the PeriodicTrackerExtension for the Kors Poster, and the two key inputs in that extension are the Url, which should be:
[URL of BMON site]/readingdb/reading/[Sensor ID]/store/
[URL of BMON site] has the same meaning as before. For the [Sensor ID] placeholder, you need to create a unique Sensor ID for this DDC point. This will become the Sensor ID for this point in the BMON system. The second important input is Format, which should be entered as:
storeKey=[store key]&val=%newValue.value%
The [store key] placeholder has the same meaning as before. The Kors Poster does not send any time information when it posts the point values. BMON will timestamp the readings when they arrive at the BMON system.
4.7.4 General Method for Gathering Data from Building Automation Systems
Most Building Automation/DDC Systems (BAS) can be configured to periodically create a file containing trend data for selected sensors and control points on the system. This file can then be used as a source of data to post to the BMON system. This section gives a general outline of how to set up a process for accomplishing this. The general steps necessary are: 1. Install the file-sharing software Dropbox on the PC running the Building Automation System Management Software, or install it on a file server accessible to the Automation System. Create a folder in the Dropbox account that will hold the trend files created by the BAS. 2. Configure the Management Software for the Building Automation System to periodically create a trend file containing data from the sensors and control points of interest. Often, a useful time interval between recorded points is 10 to 15 minutes. Also, configure the software to create the trend file at least once daily, although for fresher data in BMON, creating the file every 4 hours is better. The best format for the trend files is Comma- Separated-Values (CSV). Configure the BAS to store these files in the Dropbox folder created in the prior step. 3. Install Dropbox on the BMON Server. Here are Instructions for a “headless” install of Dropbox on a Linux server. For the server, you can use the same Dropbox account as you set up in Step 1 or you can create a new Dropbox account associated with the BMON server; if you use a different account, Share the Dropbox folder you are using for the trend files with the Dropbox account used in the BMON server. After completion of this step, the BAS trend files will be readily accessible to the BMON server. 4. Finally, a script needs to be run on the BMON server to load the sensor data from the trend files into BMON. The csv-transfer utility is one such script that can continually monitor the Dropbox folder for new or changed files, and then load data present in those files into BMON. See the above linked documentation for that script. This script can actually reside on any PC or server that has access to the trend files and access to the Internet,
4.7. Setting Up Sensors to Post to BMON 43 bmon Documentation, Release 0.0.1
including the BAS Management PC. The advantage of putting it on a cloud server running Dropbox is that it can be shared across multiple BAS systems and the owner of the BAS system does not need to install and configure the script; they have the simpler task of installing and configuring Dropbox. This procedure has been implemented for a Siemens building automation system running the Insight APOGEE man- agement software. This document explains how to create trend files on this system. The csv-transfer configura- tion file for posting the data from these files to BMON is shown below:
csv_files: - file_glob:"/home/amitchell2/Dropbox/cchrc_siemens/ *.csv" file_type: siemens chunk_size: 10 ts_tz: US/Alaska
consumers: - type: bmon poster_id: an-bmon-01 # unique ID for this posting object bmon_store_url: https://bmon.analysisnorth.com/readingdb/reading/store/ bmon_store_key: xyz1234567
Integrating with Siemens Building Management Systems
This document describes how to create data files (CSV) in the Siemens Building Management System for integration into the BMON system. The specific steps shown here are specific to the Siemens Insight APOGEE Building Au- tomation Software Program version 3.7.0 (2005). The general goals of each step of the report building process are described here to assist in integrating your Siemens Building Data with BMON. See General Method for Gathering Data from Building Automation Systems for one approach to pushing this data into BMON. * A few notes on Siemens-specific terminology used in this document. • Point refers to a sensor or data point. • Trend refers to a collection of points and their values over time.
Configure Sampling Rates for Each Sensor
The first step to automatically exporting data from a Siemens system to BMONs web-based monitoring tool is to select the sensors you would like to export, configure them to the correct time interval, and ensure that they are available for collection. On the computer running the Siemens program open the Trend Definition Editor
From the Trend menu, select Open and use the object selector to find the data sensor (referred to as points) you want to monitor. Use the asterisk wildcard * to have the object selector list all potential points in the system. If you already know the names of the points you would like to add you can also use the wildcard to narrow down your results. For instance, to locate all points with “boiler” in their name use the following format: *boiler* note that the wildcard should be added on both sides of the term unless you know the name starts with the term (format term*) or ends with the term (*term) otherwise your search will yield no results.
44 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Select a point by double clicking on it and define your parameters - depending on the sensor you will have a number of options which may include COV (change of value), Interval (time-based sample), and wizard (user assistance tool). For the purposes of integrating with BMON systems, we will focus on the Interval option.
4.7. Setting Up Sensors to Post to BMON 45 bmon Documentation, Release 0.0.1
Click add. Select Interval and click OK
Select a sampling interval, and make sure Enable PC Collection is checked.
46 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
While not specified above, Panel Samples Desired and PC Buffer Size may need to be adjusted based on the interval you select. Panel Samples Desired refers to the number of samples taken per interval, while PC Buffer Size indicates the number of days worth of space the computer is available to hold. If adjusting these variables, take note of the Maximum Samples available at Panel as this indicates the maximum number of samples that can be collected per interval.
Build the Trend Collection Report
In order to build a report, data must first be retrieved from the Siemens field panel and saved to the computer. This is done by creating a Trend Collection Report. Open the Report Builder
From the Definition menu, select New Select Trend Collection Report
4.7. Setting Up Sensors to Post to BMON 47 bmon Documentation, Release 0.0.1
Click Add and select the points you configured in the previous section Save the report by clicking the Definition menu and selecting Save. To verify your report is functioning correctly, run a test report by selecting Run Report from the Definition menu.
Build the Trend Interval Report
Once the system has been configured to obtain data from the Siemens field panel using the Trend Collection Report, the data can be extracted and written to a CSV file. Open the Report Builder
From the Definition menu, select New Select Trend Interval Report
48 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Configure your Trend Points by clicking Configure
4.7. Setting Up Sensors to Post to BMON 49 bmon Documentation, Release 0.0.1
Click Add and select the points you configured in the previous section
After you’ve added your points to the trend points list click OK
50 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Configure the Reporting Interval by clicking the Configure option and selecting your preferred interval
Configure the Output by clicking the Configure option Click the ellipses . . . next to File and browse to the location you wish to save your file, if you are using Dropbox to store your files for access from the BMON system you will want to select a location inside your Dropbox folder. Give the file a descriptive name and confirm that System Date and Time is checked.
Then click File Format... and select Delimit Text and Comma.
4.7. Setting Up Sensors to Post to BMON 51 bmon Documentation, Release 0.0.1
Click OK until you are back to the Trend Interval Report screen and check that the time format is 24 hours. Your report should now look similar to the example below:
52 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Run the report to verify the information is correct, then select Save As from the Definition menu. This will save your report.
Schedule Automatic Reports
Finally, the system needs to be configured to automatically collect the data from the Siemens field panel by scheduling your reports to run and save the data to a CSV file. Open the Scheduler
From the Schedule menu select New > Trend Collection Use the object selector to find the Trend Collection Report you created earlier.
4.7. Setting Up Sensors to Post to BMON 53 bmon Documentation, Release 0.0.1
Set up the report to run daily. If you want reports to be generated more often you’ll need to set up a separate instance run for each time you want to generate a report, for instance, one to run at 12:00PM and another to run at 1:00PM.
Once you’ve scheduled the Trend Collection Report, which only pulls data from the Siemens field panel and stores it, you’ll want to do the same for your Trend Interval Report which compiles the data into a readable format for the BMON system. From the Schedule menu select New > Trend Collection Use the object selector to find the Trend Interval Report you created earlier. Set up the report to run daily. If you want reports to be generated more often you’ll need to set up a separate instance run for each time you want to generate a report, for instance, one to run at 12:00PM and another to run at 1:00PM.
54 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
* Note that the Collection Time in this example is set to run 10 minutes after the Trend Collection Report runs, we do this to give the system time to pull the data from the Siemens field panel, only after this is performed can the Trend Interval Report be generated with current data. Please note that an oftentimes better alternative to csv-transfer is available: file-to-bmon. This utility can load sensor readings from a set of files into BMON. There is more detail in the section below.
4.7.5 General Utility for Loading Sensor Data from Files
The file-to-bmon project provides a script that can be used to load sensor reading data from a set of files. The script is structured so that custom file Reader modules can be written in Python to parse any type of text file containing sensor readings. The required code in the Reader module is focused exclusively on parsing sensor reading data from file, so the complexity of the code is relatively low. This script is often used to process data that is periodically uploaded to a server, such as 15-minute electric utility data or data from a conventional datalogger such as the Campbell Scientific datalogger. Please look at the documentation at the above link for more information.
4.7.6 Particle WiFi and Cellular Microcontroller Boards
Particle makes microcontroller boards with built-in WiFi or Cellular data connectivity. Firmware on the board manages communication with a Particle Internet server that provides a simple and robust Application Programming Interface (API) for sending and receiving data from the board. These boards make it relatively simple to implement custom data collection systems that are able to post data to BMON. This section describes how a configure a Particle board to deliver data to a BMON server. The essential steps involve: • Creating a Webhook in your Particle account that forwards data from events to the BMON server.
4.7. Setting Up Sensors to Post to BMON 55 bmon Documentation, Release 0.0.1
• Writing code for the Particle board that publishes an event with data in the correct format for consumption by the BMON server.
Create a Webhook on the Particle Server
• Assuming you are logged into your Particle account, select the CONSOLE item from the right side of the top menu bar. • From the resulting page, select the Integrations icon at the bottom of the left sidebar menu. • Click the large plus sign to create a NEW INTEGRATION, and then choose Webhook for the type of Integra- tion. The top part of the resulting form should look like the following, except the following includes sample inputs:
For the Event Name input, create a name for the event that will be published by your Particle board. In this sample, the event name was called bmon_store. In the URL input box, enter a URL with the format:
[Base URL of BMON site]/readingdb/reading/store/
In the sample shown, the Base URL of BMON site is https://bmon.analysisnorth.com. Next, expand the Advanced Settings portion of the form to reveal the following inputs:
56 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
As shown in the above screenshot, select the JSON option in the Send Custom Data section. Then, enter a JSON object with two keys. As the value of the storeKey, enter the BMON store key inside the quotes (from your BMON Settings File). As the value of the format key, enter the word particle inside the quotes. Leave other inputs at their defaults and click the Create Webhook button at the bottom of the page to complete creation of the Webhook. Whenever a bmon_store event (in this example) is received by the Particle server, its data will be be forwarded to the BMON server.
Particle Event Format
The Particle server is now prepared to forward data to the BMON server. However, the Particle board must publish an event of the proper name in order for this transfer to occur. Given the example above where we set the Event Name to bmon_store, here is an example of a Publish statement coded on a Particle board that will send two sensor values to the BMON server:
Particle.publish("bmon_store","Ttank=103.4 Tdhw=98.4", 60, PRIVATE);
The sensor names and values appear in the second parameter. The sensor name and the sensor value are separated by an equal sign, with no embedded spaces. The different sensors are separated by spaces. These two sensor readings will show up at the BMON server with the following two Sensor IDs, assuming the ID number of the Particle board is 260034000c47343432313031:
260034000c47343432313031_Ttank 260034000c47343432313031_Tdhw
The sensor readings will be timestamped with the time they were received by the Particle server. If you want the readings to be timestamped at some offset prior to the time they were published, you can cause that to occur by special formatting in the publish statement. Here is an example:
Particle.publish("bmon_store","Ttank=103.4a Tdhw=98.4 a=300")
In this example, the timestamp for the Ttank reading will be 300 seconds prior to the publish time, because an a is appended to the Ttank reading. Since no a is appended to Tdhw, its timestamp will be the publish time with no
4.7. Setting Up Sensors to Post to BMON 57 bmon Documentation, Release 0.0.1
adjustment. The a=300 part of the message is where the adjustment in seconds is set. The adjustment should be an integer value in seconds. This feature is particularly useful when the Particle board is averaging a number of readings over a set interval, and the average value is being published. The most accurate point in time to associate the average value with is the middle of the averaging interval, not the time when the value is published. If the averaging interval is 10 minutes (600 seconds), moving the timestamp back 300 seconds from the publish time is desirable and can be accomplished using the feature described above.
4.7.7 General Instructions for Posting Data from Other Systems
If your sensors are not one of the above types, you may still be able to use the BMON system if your sensor system can make HTTP requests containing the sensor data. There are currently two methods for storing sensor data into the BMON system. The first method stores one sensor reading. The second method allows you to store multiple sensor readings from one or more sensors. Below are descriptions of those two methods.
Storing One Sensor Reading
To store one sensor reading, you can make either a GET or a POST HTTP request. The URL that the request should be made to is:
[URL of BMON site]/readingdb/reading/[Sensor ID]/store/
Substitute the URL of the BMON site for [URL of BMON site], and substitute the unique BMON Sensor ID for this sensor for the [Sensor ID] placeholder. Here is a valid URL:
https://bms.ahfc.us/readingdb/reading/gview_dhw_temp/store/
In this example, the BMON site URL is https://bms.ahfc.us, and the BMON Sensor ID for for this sensor is gview_dhw_temp; in order to see this data in the BMON system, you need to have a Sensor configured with the gview_dhw_temp Sensor ID. If sensor data is posted to the BMON system prior to configuring the Sensor, it will still be stored and will be visible once the Sensor is configured in BMON. Next, you need to add a query string containing additional data about this sensor reading. The query string should be formatted like:
storeKey=[store key]&val=[sensor value]&ts=[date/time of reading]
The [store key] has been discussed before and is the BMSAPP_STORE_KEY setting you created in the BMON Settings File. [sensor value] is the sensor’s reading value that is being posted to the BMON database; it needs to be a number. [date/time of reading] is the time of the sensor reading expressed in Coordinated Universal Time (UTC). The date/time should be entered as a string value, such as Dec 3, 2014 10:03:45 PM, and it must be properly URL encoded. The prior date string, encoded properly, would be transmitted as: Dec+3%2C+2014+10%3A03%3A45+PM. The format of the date/time string is very flexible as the Python dateu- til parser is used. All of the following strings are valid and parse to the same date/time.
December3, 2014 10:03:45PM Dec3, 2014 10:03:45PM 12/3/2014 22:03:45
Finally, inclusion of the ts query parameter just discussed is optional. If it is not included, BMON will timestamp the sensor reading at the time it arrives at the BMON system. Here is a valid GET request that posts a gview_dhw_temp temperature reading of 125.3 occurring at Dec 3, 2014 10:03:45 PM UTC:
58 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
https://bms.ahfc.us/readingdb/reading/gview_dhw_temp/store/?storeKey=123abc&val=125.3&
˓→ts=Dec+3%2C+2014+10%3A03%3A45+PM
The same request can be made with the POST method by sending a POST request to this URL:
https://bms.ahfc.us/readingdb/reading/gview_dhw_temp/store/
and including this query string in the message body of the POST request:
storeKey=123abc&val=125.3&ts=Dec+3%2C+2014+10%3A03%3A45+PM
The response from a successful HTTP request will be 1 readings stored successfully. If unsuccessful, an error message will be returned.
Storing Multiple Sensor Readings
It is possible to post multiple sensor readings to BMON from one or more sensors. To do so, an HTTP POST request must be made. The URL to send the request to is:
'[URL of BMON site]/readingdb/reading/store/'
where [URL of BMON site] is filled in with your BMON URL, e.g. https://bms.ahfc.us. The message body of the POST request should be a JSON string following the pattern of this example:
{"storeKey":"123abc", "readings":[ [1432327040,"28.FFFA1C031400", 71.788], [1432327042,"test_cpu_temp", 45.527], [1432327040,"28.FF1A2D021400", 65.859] ] }
The JSON object has two name/value pairs. The first name is the storeKey, and the associated value must contain the BMSAPP_STORE_KEY setting you created in the BMON Settings File. The next name is readings, the value of which is an array of sensor readings that you would like to store in BMON. Each element of that array is in turn a 3-element array representing one reading. In the example above, the first sensor reading is:
[1432327040,"28.FFFA1C031400", 71.788]
The 1432327040 element is the UTC timestamp of the sensor reading, expressed as a Unix timestamp (number of seconds since January 1, 1970). If you send a timestamp with fractional seconds, BMON will truncate to an integer value. If the timestamp is the JSON value null, the current time will be used for the timestamp. The "28. FFFA1C031400" element is the unique BMON Sensor ID surrounded in double-quotes. The 71.788 is the actual sensor reading and must be a number. There can be as many sensor readings (each a 3-element array) as you need. If the Post was entirely successful, you receive a response like:
3 readings stored successfully
which tells you how many readings were successfully stored. If any errors occurred while storing readings, error messages separated by newline characters are appended to the above success phrase. If no readings were stored successfully, the response will be “0 readings stored successfully” plus an error message for each unsuccessful reading.
4.7. Setting Up Sensors to Post to BMON 59 bmon Documentation, Release 0.0.1
4.8 Multi-Building Charts
The BMON system can create reports and charts that display data across multiple buildings. However, each one of these multi-building reports needs to be configured in the Admin interface to the system.
4.8.1 Current Sensor Values Report
The first type of Multi-Building Report that will be discussed is the “Current Sensor Values” report. Here is a sample report that shows the Domestic Hot Water temperatures being delivered to a number of buildings:
The Value column shows the last value each sensor has reported. Two of the values are highlighted red because they have values that would trigger an Alert. The When column indicates how long ago the sensor reported in. The Graph column presents a simple graph of the values so it is easier to see where each reading falls within the range of values presently occurring. To create this type of report, you first need to write down the Sensor IDs and the associated Building for each sensor that you want to include in the report. Log into the Admin interface of BMON as described in Add Buildings and Sensors, and click on the Sensors link on the left side of the page. All of the Sensors currently entered into BMON will be shown. Find the sensors that you want to include in your report, perhaps using the Filter feature shown on the right side of the page, which allows you to narrow the list to a particular building. Once you have found a sensor to include, write down it’s Sensor ID, which is the text that appears prior to the colon. Also write down the Building name associated with the Sensor. In the sample shown below, two different Sensor IDs are highlighted in yellow: 28.EFED4C050000 and Burt158_alarm_code.
Once you have collected the Building names and Sensor IDs to include in your report, return to the Admin Home screen by clicking the Home link in the upper left of the page. Then click the Multi building charts link as shown below:
60 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Then click the Add multi building chart button shown below to add a new Multi-Building report.
Here are the inputs for creating a multi-building Current Sensor Values report:
4.8. Multi-Building Charts 61 bmon Documentation, Release 0.0.1
You fill in the Title field with a name for this report. This will be the name that the User sees when they are select- ing the report. From the Type of Chart dropdown, select Current Sensor Values. General Chart Parameters in YAML Form is left blank for this type of report. Sort Order is a number used to sort all of the multi-building reports. Pick a number that puts this report in the desired position relative to other multi-building reports. Counting by 10s with these numbers is recommended so other reports can be added to the list later. Next, move on to the Chart building infos section of the inputs. If you just started creating the report, there will be a blank Chart building infos item to fill out. You fill out one of these for each Building that will appear in your report. In the sample shown above, the Glacierview building was selected from the dropdown as the first Building to show in report. Next, the large box in the Chart Parameters in YAML Form needs to be filled out with the Sensor IDs from the building that you want to appear in the report. For each sensor you want to include, enter it on a separate line with a dash and a space preceding it, -. Here is the exact format for two Sensors having the IDs 29931 and 29929:
- 29931 - 29929
The Sort Order column affects where this building will appear in the list of buildings that are included in the report. Once you complete one Building, use the Add another Chart building info link to bring up another blank form for the next building to include in your report. Continue this process until all desired Buildings/Sensors have been entered for the report. Click the Save button in the lower right corner to complete the report. It will now be available for Users to view.
62 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
4.8.2 Energy/ft2 Chart
This chart allows the User to compare the Energy Use per square foot of building across multiple buildings. Here is a sample chart:
To use this type of chart, each building that you want to include in the chart must have one sensor (it can be a Calculated Field) that measures the rate of energy use you want to compare across buildings. Also, the rate of energy use must be expressed in the same units for each building. In the example chart above, the chart compares electricity use per square foot across three buildings. Each one of buildings has a sensor that measures the rate of electricity use in kW for the building. So, the prerequisites for this chart type are satisfied. Here are the Multi-Building Chart inputs that created the chart above:
4.8. Multi-Building Charts 63 bmon Documentation, Release 0.0.1
Many inputs have the same meaning as they did in the Current Values Report, but there some important new inputs. You must select the Energy/ft2 choice in the Type of Chart dropdown. There are also two important entries in the General Chart Parameters in YAML Form box that depend on the units used for the sensors that are measuring energy use and the total energy units you want to display on the chart. For this example, the chart will display energy use in terms of kWh per year. This unit is entered as one line in the General Chart Parameters box as:
value_units: kWh/year
This entry is only used as part of the label for the vertical axis of the chart. referring back to the chart image above, you will see that the vertical axis label is kWh/year/ft2; the /ft2 was added to the value_units entry to come up with this final chart label. The other necessary entry into the General Chart Parameters box is the multiplier entry. In this example a multiplier value of 8766 is entered as:
multiplier: 8766.0
64 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
This multiplier value is the amount that you need to multiply the rate of energy use by in order to calculate the desired energy units given by value_units. In our example, we explained above that each of the three buildings has a sensor that measures the rate of electricity in kW. Since we have decided this chart will display total energy use in kWh/year, we need to multiply the rate of electricity use in kW by the hours in a year to arrive at kWh/year of total electricity use. Thus, the multiplier must be 8766 (average hours in a year including leap years). As another example, assume our building sensors measured heating fuel use and reported it in Btu/hour units. For the multi-building chart to compare fuel use across buildings, assume that we wish to express total fuel use in kBtu/year (1,000s of Btus per year). To calculate total kBtu/year from an average Btu/hour value, you need to multiply by 8766 to arrive at Btus/year and then divide by 1,000 to convert to kBtu. So, the final multiplier would be 8766/1000 = 8.766. Our value_units entry would be kBtu/year. To finish this chart configuration, additional information needs to be entered for each building that will be compared. In the inputs screenshot above, you can see that each building has a floor_area and id_value entry. The floor area entry should be the floor area of the building in square feet. The id_value entry is the Sensor ID of the sensor that is measuring energy use in this building (the instructions above for the Current Sensor Values Report for finding Sensor IDs apply here as well). The format of the entry is important; there must be a space after the colon in each line: floor_area: 77900 id_value: 38511
4.8.3 Energy/Degree-Day/ft2 Chart
The Energy/Degree-Day/ft2 chart compares energy use across buildings normalized for both the size of the building and the number of heating degree-days experienced by the building. This allows better comparisons of buildings of varying sizes located in different climates. Here is an example of the chart:
4.8. Multi-Building Charts 65 bmon Documentation, Release 0.0.1
As before with the Energy/ft2 chart, each building that you want to include in the chart must have one sensor (it can be a Calculated Field) that measures the rate of energy use you want to compare across buildings. Also, the rate of energy use must be expressed in the same units for each building. Finally, there must be an outdoor temperature sensor available for each building in the chart. These “sensors” can be weather stations accessible over the Internet, as explained in calculated-fields. Here are the main inputs and the inputs for the first two buildings that are present in the above chart:
66 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
From the Type of Chart dropdown, Energy/Degree-Day/f2 is selected. There are three important inputs in the General Chart Parameters box. The base_temp input allows you to specify the base temperature used when calculating heating degree days. Choosing a base temperature near the “balance point” of the buildings is best; the balance point is the outdoor temperature above which no heat is typically needed in the building. 65 degrees F is the most typical base temperature, but in this chart 60 degrees F is used because these buildings typically do not need heat until outdoor temperatures drop below 60 F. value_units specifies the Energy units that you want to use in the chart. In this example, Btus are entered, so the chart vertical axis will be Btus/ft2/degree-day. Finally, the multiplier input can be specified (it defaults to 1.0 if not provided). As stated above, every building must have a sensor that reports the rate of energy use using units that are consistent across all the buildings. The multiplier converts the rate of energy use into the amount of energy consumed in one hour expressed in value_units. As an example, assume the sensors report in Btu/hour and the value_units are Btus. The multiplier in this case is 1.0; if the rate of energy use is 100,000 Btu/hour there will be 100,000 Btus used in one hour. If value_units were kBtu (1,000s of Btus), the multiplier would need to be 0.001, since a rate of use of 100,000 Btu/hour would use 100 kBtus in one hour. For each building that is presented in the chart, three different Chart Parameter inputs are required. As in the prior chart, a floor_area input giving the building floor area in square feet is required. Also similar to the prior chart, an id_value input is needed to provide the Sensor ID of the sensor that measures the rate of energy use for the building. The new input for this chart type is id_out_temp. For this input you need to supply the Sensor ID of an outdoor temperature sensor that is appropriate for the building. For the example above, the outdoor temperature sensor for the AHFC Headquarters building is bon_tudor_temp. This is a Weather Underground weather station that was setup as a Calculated Field. For the Chugach Manor building, the outdoor temperature sensor has the Senor ID of anc_merril_temp, which comes from the Merril Field, Anchorage, Alaska National Weather Service site.
4.8. Multi-Building Charts 67 bmon Documentation, Release 0.0.1
After entering information for each of the buildings to be presented in the chart, click Save in the lower right corner, and the chart will be available to users.
4.9 Sensor Alerts
You can have BMON send you a text message, email, or a Pushover notification (explained later) if important sensor conditions occur, such as Alarm conditions or too low/high of temperatures. To configure Alerts, you first must log into the Admin interface, as explained in Add Buildings and Sensors. Once into the Admin interface, there are two main tasks–adding Alert Recipients (people that receive alert notifications), and adding the actual Sensor Alert Conditions. These tasks are described in the sections below.
4.9.1 Adding Alert Recipients
An Alert Recipient is a person who will receive a notification when an Alert occurs. When the actual Alert Condition is set up, you will choose which Alert Recipients are notified for that Alert. From the Home screen of the Admin interface, click the Alert recipients link shown below to add an Alert Recipient or change information for an existing recipient:
Then on the next screen, click on Add alert recipient as shown below to add a recipient:
68 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
The following screenshot shows the inputs used to describe an Alert Recipient:
Enter a Name for the Alert Recipient and make sure the Active checkbox is checked (you can uncheck the Active box to temporarily disable sending messages to this recipient). There are three different ways a Recipient can be notified: Email, cell phone text message, and a Pushover notification. Pushover is a special notification application that you can install on your smart phone for a one-time $5 charge. Benefits of the application are discussed later. One or more of these notification methods can be enabled for a Recipient. The screenshot above shows the checkboxes for each method (Send Email?, Send Text Message?, and Send Pushover Notification?). Simply check the boxes for the methods you would like to use. If multiple boxes are checked, the Recipient will be notified by all of the enabled methods. Each notification method has some associated inputs; there is no need to fill out inputs for notification methods that are not used. The inputs are for the most part self explanatory. Do note that when you enter a cell phone number for the text message method, enter all 10 digits and do not use any dashes or spaces in the number. Select the cell phone carrier from the dropdown box; if a carrier is not available in the list, please contact the developer to have it added. The Pushover notification service has some advantages relative to Email and Text: • Messages from BMON will not be mixed in with your normal Email and Text messages. • You can set up Quiet Hours in Pushover so that normal priority notifications will be quieted during those hours.
4.9. Sensor Alerts 69 bmon Documentation, Release 0.0.1
• BMON allows you to assign priorities to notifications. Pushover can treat those priorities differently, producing more noticeable alert sounds for higher priorities.
4.9.2 Adding Alert Conditions
You add an Alert Condition to a specific sensor in the BMON system. So, you first need to access the screen that allows you to edit information about the particular Sensor. To do this, return the Home screen by clicking the Home link in the upper left corner of whatever Admin screen you are currently on. Next, click the Sensors link in the left column as shown below:
Next, you need to find the Sensor that you want to configure an Alert condition for. Remember that you can Filter sensors by Building by using the Filter box at the right hand side of the screen. Once you locate the Sensor, click it to bring up its Edit screen. The Alert Conditions section is at the bottom of the Sensor edit screen, as shown in the screenshot below:
70 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
After clicking the link to Add another Alert condition, you are taken to the Alert conditions page as seen below, to allow for configuration of the Alert. In the screenshot below, the inputs have already been filled in but will be described next:
4.9. Sensor Alerts 71 bmon Documentation, Release 0.0.1
This alert is being set up to notify someone if the Boiler Supply temperature drops below 140 degrees F. The first thing to ensure is that the Active checkbox is checked; it can be unchecked if you want to temporarily disable this Alert. Next, choose the correct comparison to make in the Notify when the Sensor value is dropdown. In this case, we chose the less than comparison because we want to know when the sensor drops below or is less than a certain value. The This value: input lets us set the trigger value, which in this case is 140 deg F. We will discuss the next two inputs related to Building mode later. In this example they are not used and are left with no selections. BMON will automatically create a relevant Alert message, or you can type your own message in the Alert message box. We have left the message blank in this example and will use the one created by BMON. You can select a priority for this Alert in the Priority of this Alert Situation dropdown. For text messages and Emails, this has little effect other than affecting the text in the Subject line of the message. For Pushover
72 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
notifications, the priority setting will affect the alert tones, selecting Emergency will force the User to acknowledge the Alert or the Alert will continue to repeat every 5 minutes. BMON checks for Alert conditions every 5 minutes. However, you may not want to receive a notification every 5 minutes for the same Alert. the Hours to Wait before Notifying Again input allows you to specify how long BMON should wait before notifying you again about the same Alert Condition. In this example we have accepted the default value of 4 hours, so if this Alert Condition is still present 4 hours after the first Alert notification is sent, BMON will send out another notification. The final section allows you to select the Alert Recipients who will receive notifications for this Alert. Double-clicking a name in the left Available recipients box will move the name into the Chosen recipients box. The arrow icons between the two boxes can also be used to select and unselect recipients. Finally, click the Save button in the lower right corner to Save the Alert setup and any other changes you may have made to this Sensor. There is no limit to the number of Alert Conditions that you can create for one sensor. Before clicking the Save button, we could have added another Alert Condition for this Boiler Supply temperature sensor by having clicked the Add another Alert condition link at the bottom of the screen. Or after Saving the first condition, we can go back to the Sensor edit screen and add another condition. It is common to want an Alert if a sensor has too low of a value and also want an Alert if the sensor value is too high. For the Boiler Supply temperature example, you could also set up a second Emergency priority Alert if the temperature drops below 120 deg F.
Note: Alerts can be created for calculated fields as well as normal sensor readings. Since a calculated field can combine the values of multiple different sensors (see Mathematical Calculated Fields), complex alert conditions can be created that include tests for multiple different sensor values. For example, a calculated field can be created that will evaluate to True if a particular flow rate is zero at the same time that outdoor temperature is less than 38 degrees. An Alert can then be set on that field to notify recipients when this condition occurs.
4.9.3 Inactive Alerts
You may want to know when a sensor is failing to post new readings. You can set up a special “Inactive” Alert for this purpose. The Notify when the Sensor value is input contains a number of choices, including less than, greater than, etc. The last choice in the dropdown is inactive. Selecting this choice will create an Alert that will notify recipients if the sensor has not posted a reading during the last two hours (or the value for the BMSAPP_SENSOR_INACTIVITY setting in the BMON Settings File. If inactive is chosen, the trigger value input is ignored.
4.9.4 Alerts Affected by Building Mode
Sometimes, Alert conditions only matter during certain seasons of the year or when a facility is in a particular operating mode. A particular circulation pump may need to be On for the entire Winter, but need not be On during the Summer. You can add a “Building Mode” check to your Alert Condition by filling out the two Alert Condition inputs shown below:
For our Boiler Supply temperature example, the above settings would cause the Alert to be issued only if the Glacierview building is in Winter mode. To set the particular operating mode of a building, you need to manu- ally go to the Edit screen for the Building and set the Current Operating Mode input. There is no provision in BMON for automatic determination of the building operating mode.
4.9. Sensor Alerts 73 bmon Documentation, Release 0.0.1
BMON comes pre-configured with the Winter and Summer operating mode choices. On the main Admin screen you will find a link to Building Modes where you can add and delete Building Operating Modes so they fit your needs.
4.10 Creating a Dashboard
A dashboard provides an easy-to-understand display of the current values from key sensors in a building. Here is an example:
This document explains how to create Dashboards. You can create one Dashboard for each building in the BMON system. Any or all of the sensors associated with the building can be displayed in the Dashboard. This screenshot shows the three different types of Dashboard Widgets:
To create a Dashboard for a building, log into the Admin interface and navigate to the screen that allows you to edit or configure the building. More details are provided in Add Buildings and Sensors but the basic steps are: 1. Go to the main Home screen for the Admin Interface by clicking the Home link in the upper left corner of your current screen. 2. Click the Buildings link in the left column of the Home screen.
74 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
3. Click on the Building that you wish to add the Dashboard to. At the very bottom of the edit page for the building should be a section for Dashboard items and the first blank Dashboard item is shown:
A Dashboard is made up of individual Widgets arranged in rows and columns. For the sample Dashboard shown above, there are three Widgets in the first row (three Graphs), and four Widgets in the second row (an LED and three Graphs). So, to create this Dashboard, you would need to enter seven different Dashboard items for the building. A blank Dashboard item is provided to start, and additional items are created by clicking the Add another Dashboard item link at the bottom of the page. When finished entering items, click the Save button for the Building found in the lower right corner of the page. If you would like to save your work before finishing all Dash- board items, click the Save and continue editing button, which will leave you on the editing screen. Here are sample inputs for a Graph Widget:
The Widget Type input is a dropdown showing the three different Widget types, Graph being the selection here. Next, you need to specify where this Widget is to be located in the Dashboard. Row and column numbering starts with 1, and this Graph is to be located in the second column of the first row. It is best to sketch out your Dashboard before entering values so it’s easy to determine row and column locations of the Widgets. Next, the Sensor that you wish to display in this Graph is selected in the Sensor dropdown. If you want to title the Graph with something other than the Sensor’s title, you can enter the new title in the optional Widget Title box. If you look at the pictures of the Graph control (the first image in this document), you see that a portion of the vertical scale is colored Green. The Green area designates the normal range of sensor values. You enter the normal range by filling in the Minimum normal value and Maximum normal value inputs for the Dashboard item. For this example, the normal range goes from a minimum value of 50 kW to a maximum value of 180 kW. You will also note in the pictures above that if the most recent value of the sensor falls outside the Normal range, the dot on the graph and the sensor value label are colored Red as opposed to the normal black. Finally, the Graph Y-Axis has a starting value and ending value. BMON will make reasonable estimates for these given what you put in for the Normal range. But, if you want to explicitly specify the full range of the vertical axis, you can enter values for Minimum axis value and/or Maximum axis value. In this example, the Minimum axis value is 0 kW and the Maximum axis value is 250 kW. In the sample Dashboard at the top of the page, one of the Graphs has two horizontal dashed red lines. These lines indicate that Sensor Alerts have been set to trigger at these values. Here are sample inputs for the Red/Green LED Widget:
4.10. Creating a Dashboard 75 bmon Documentation, Release 0.0.1
The first two rows of inputs have the same meaning as for the Graph Widget. The important settings for the LED Widget are the Minimum normal value and Maximum normal value. If the sensor value is within this range, the LED will be colored Green. If the sensor value is outside of this range, the LED will be Red. In this example, the sensor selected reads a set of alarm contacts on a boiler. The dry contact sensor reports a 0 when the alarm contacts are open and a 1 when they are closed and an alarm is occurring. So, Minimum and Maximum normal value inputs are both set to 0, because 0 is the only value that indicates no Alarm. Finally, here are sample inputs for a Label Widget:
This Widget simply shows a label in the Dashboard; the Widget is not linked to any sensor. The only inputs that are used are the Widget type, Row number, Column number, and Widget Title, which is the label that will be displayed. To delete any Dashboard items, check the Delete checkbox on the right side of the inputs for the item (not shown in the image above). Then click the Save button in the lower right corner of the page. Do Not click the red x Delete button in the lower left corner of the page, as that will delete the entire Building, not just the selected Dashboard items. If there are any Dashboard items configured for a Building, the Dashboard will automatically be added as the first item in the list of Reports/Charts that a User can select and view for the Building.
4.11 Transform Expressions
Transform Expressions are used for a number of purposes, including: • Sensor values may arrive expressed in units that are not desirable. A Transform Expression can be used to convert the units to the type desired for display. • Sensor values may have known errors that can be corrected with a Transform Expression before storage into the system. • Pulse counts generally should be converted into a rate value (units/second or units/hour), as counts do not always arrive at equal intervals due to wireless transmission issues and due to users changing the time interval that a sensor reports. Special Transform Expressions can be used to convert the pulse counts into a rate value.
4.11.1 Simple Transform Expressions
Transform Expressions are entered on the Sensor editing screen. The screenshot below shows a Transform Expression to adjust the reading from a Light Sensor, correcting for light reducing paint that covers the sensor.
76 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
In the Transform or Calculated Field Function Name box you can see the transform expression val*35.0 - 140.0. The variable val is always available in a Transform Expression and it contains the raw sensor value that was posted. This transform multiplies that raw value by 35.0 and then subtracts 140.0. When creating these expressions, you have all the built-in functions (e.g. abs() for absolute value, ) from the Python pro- gramming language plus the functions available in the Python math module (e.g. sqrt(), sin(), log10()). So, an expression such as sqrt(val)/(val - 8.6) is a valid transform expression. In the above screenshot, note that the Calculated Field box is not checked. This is not a new calculated field, created from other sensor values or the gathered from the Internet. Instead, this is just a conversion of an incoming sensor value.
4.11.2 Pulse Counter Transforms
Electric meters, gas meters, water meters, and fuel meters often produce pulses as a means of electronically com- municating flow through the meter. Each pulse represents a fixed amount of energy or volume of material flowing through the meter. The pulse output of the meter can be connected to a pulse counter that reports total pulse counts to the BMON system. It is generally desirable to convert these pulse count totals into a rate of flow of energy, fluid, or gas. By converting to a rate (units/second), variations in the length of the interval measured by the pulse counter are factored out of the measured value. Special pulse counter Transform features were developed to address this situation. The recommended setting for a pulse counter is to have it continually accumulate pulse counts, only resetting to zero when a maximum rollover pulse count is reached. If your pulse counter is set up in this way, below is an example of how you can transform the total pulse count values into a usable rate value. In this example, a natural gas meter is being read, and a pulse occurs for each CCF of gas passing through the meter. Our objective is to convert the pulse count readings into a gas flow measured in Btu/hour.
4.11. Transform Expressions 77 bmon Documentation, Release 0.0.1
The BMON Transform feature has a special variable named rate. If a Transform expression uses this variable, BMON knows that a pulse count is being measured, and BMON automatically calculates the pulse rate per second indicated by the last pulse count reading received relative to the prior pulse count reading. This pulse rate is stored in the variable rate. As an example, assume that the last pulse count received was a total count of 10,435. The prior pulse count reading from the sensor was 9,623 and it was received 605 seconds prior to the current reading. The pulse rate that occurred between the two readings was (10435 - 9623) / 605 = 1.342 pulses/second. This value of 1.342 is automatically stored in the rate variable. So continuing with the example, we know that BMON has already calculated pulse rate per second for the gas meter. We also know that one CCF of gas contains 1,010 Btus. To convert the pulse rate into Btus/hour we would do the following:
rate (pulses/sec) * 1,010 Btus/pulse * 3,600 sec/hour
which is rate * 3636000
rate * 3636000 is the expression shown in the Transform or Calculated Field Function Name box. There are couple other optional but often important parameters that can be entered in the Function Parameters in YAML form box: max_rate (default value = 5.0, expressed as pulses/second): Pulse count reporting errors (one common issue de- scribed below) can sometimes lead to an erroneous high pulse rate calculation. BMON will not save any sensor values if the pulse rate is above this max_rate value. For the example above, a maximum pulse rate 1.5 pulses per second is set. This corresponds to a 5,454,000 Btu/hour rate. Reported sensor values will be rejected if they exceed this rate. Note that there is a default max_rate value of 5.0 pulses/second if you do not provide a value in the Function Parameters box. rollover (default value = 65536, largest 16 bit value): Pulse counters usually have a maximum pulse count that they record before rolling over to zero. BMON will account for this rollover when calculating the pulse rate. Year 2014 and prior Monnit wireless pulse counters roll over at a count of 65,536, the default value for this parameter. Newer Monnit pulse counters are 32 bit and roll over at 4,294,967,296. If you expect your pulse counter to eventually roll over, you need to ensure that BMON is using the correct rollover value.
78 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
ignore_zero (default value = True, the other valid value is False): Monnit Wireless pulse counters occasionally reset to a pulse count of zero if there are transmission problems or an accidental reset of the sensor. So, the pulse count value of zero is usually an erroneous value. By leaving this ignore_zero parameter at its default value of True, these zero pulse count readings will be ignored and no value will be stored until the next valid reading. ignore_negative (default value = True, the other valid value is False): If set to True a backward movement of the counter is ignored (it could be due to a reset). If this parameter is False, a backward movement of the counter is reported as a negative rate of change. min_interval (default value = 60, measured in seconds): Monnit wireless pulse counters sometimes send the same pulse count value twice but with slight separation in time, resulting in a zero calculated pulse rate. To filter out these erroneous readings you can set a min_interval between valid readings, measured in seconds. If two successive readings are separated by less than this number of seconds, the second reading will be rejected. The default value is 60 seconds. Also note that all readings that use this pulse count transform are time-stamped at the midpoint between the current reading time and the previous reading time, since the rate reported is derived from the interval spanning those two time points.
If your pulse counter is setup to reset to zero after it reports its pulse count (Monnit wireless pulse counters 2013 and earlier), a different Transform function must be used to convert the counter values into rates. This is not the preferred method for setting up a pulse counter but may be needed for those counters that do not have a count accumulation mode, as described in the prior section. The screenshot below is an example of configuring a counter that resets to zero after each sensor report.
The text count_rate must be entered into the Transform or Calculated Field Function Name box. Then, additional parameters are entered into the Function Parameters in YAML form box. Each parameter has a default value, so they are not required to be entered, but you will likely need to override some of the default values. slope (default value = 1.0) offset (default value = 0.0): BMON will automatically convert the incoming pulse count into a rate of pulses per second. The slope and offset parameters are used to convert this pulse rate into the
4.11. Transform Expressions 79 bmon Documentation, Release 0.0.1
desired engineering units, such as Btu/hour or kW. The final value stored in the sensor database is: (pulse rate per second) * slope + offset typical_minutes (default value = 30.0, measured in minutes): Sensor transmissions are sometimes missed or occur multiple times due to poor signal strength. The BMON attempts to correct some of these problems but needs to know what the typical spacing is between sensor transmission. Enter that value expressed in minutes for this parameter. no_zero_after_link (default value = True, other possible value is False): If a Monnit wireless sensor is having difficulty communicating with its gateway, it will sleep for two hours and then try to reestablish contact. When it does this, it will send an initial count of zero. By setting no_zero_after_link to True (the default value), these zero readings will not be stored in the sensor database.
4.12 Calculated Fields
Calculated Fields are used for two purposes: • to calculate new sensor readings from other readings that are in the sensor database. • to add sensor readings gathered from the Internet. The current implementation allows for acquisition of temper- ature and wind speed data from Internet weather services. A Calculated Field uses the same editing form in BMON as a standard Sensor does. So, to add a Calculated Field, you follow the normal procedure for adding a Sensor, as described in Add Buildings and Sensors. The important configuration differences relative to a standard sensor are: • For a Calculated Field, you need to create your own Sensor ID for the calculated field. The Sensor ID must be unique across the entire BMON set of Sensors. The Sensor ID needs to be 30 characters or less, using numbers and letters but no spaces. • In the Sensor editing form, you need to make sure that the Calculated Field box is checked. The BMON system processes Calculated Fields every one-half hour. If the calculated fields involve sensors that report more frequently than every half hour, multiple calculated sensor readings will be generated at each half-hour processing interval to match up with the frequency of the source sensors.
4.12.1 Mathematical Calculated Fields
It is possible to create a calculated field by doing mathematical operations on one or more other sensor fields (a limit of five sensor fields can be involved in the calculation). For example, if you want to plot and analyze the difference in temperature between two sensors, you can create a new calculated “sensor” that is the difference in value between two other sensors. Below is a screen shot of how you would configure that calculated field.
80 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
This new calculated field has the Sensor ID of hp_temp_diff, and the values in that field are calculated as the difference between sensors with the IDs of hp_outlet_temp and hp_inlet_temp. Not only are these calculated fields useful for plotting and analysis, they also extend the capabilities of the Alerting system, as described in Sensor Alerts. Complex alerts can be created by configuring a calculated field involving multiple sensors. Then, an Alert condition can be added to that calculated field. Thus, the Alert will depend on the state of multiple sensors, since multiple sensors feed the calculated field. These calculated fields involving general mathematical expressions must have genericCalc entered in the “Trans- form or Calculated Field Function Name” box. This is the name of the function that allows for general calculations to be done using one or more existing sensor fields. Remaining configuration of the calculated field occurs in the “Function Parameters in YAML form” input box. Each one of the possible function parameters is described below. A (required: a Sensor ID) The value of this parameter is the Sensor ID of the sensor that will be used as the A variable in the mathematical expression. At least one sensor needs to be involved in the calculation of the new
4.12. Calculated Fields 81 bmon Documentation, Release 0.0.1
field, so that is why the A parameter is required. B, C, D, E (optional: a Sensor ID for each parameter used) Four additional sensors can be involved in the calcula- tion. Use any of the parameter names of B through E to give the Sensor IDs of those additional sensors involved in the calculated field. expression (required: a math or boolean expression, described below) Use this parameter to write out the math expression for the calculated field. You can use any of the sensor variables A through E that were in- cluded as parameters, and the values from these sensors will be used in the calculation. The expression must be a valid Python expression, and any of the functions in the Python math library can be included; see the math library documentation. Here are some valid expressions: An example of an expression using function from the Python math library, using three sensors as inputs (A, B and E):
expression: 3.4 * A * sin(B)+ sqrt(E)
An example of a Boolean expression. A True value from this expression is represented as a sensor value of 1.0 and a False value is represented as 0.0. An important note, when testing for equality with a Python expression, you must use a double equal sign (==) and not a single equal sign. The expression below will result in a calculated field value of 1.0 if Sensor A has a value equal to 1.0 and Sensor B has a value greater than 34.3; otherwise, the calculated value will be 0.0:
expression: (A == 1.0) and (B> 34.3)
averaging_hours (optional: a number of hours, fractional hours allowed; default is no averaging) If you include this parameter, it will cause sensor values to be averaged over the requested time interval before being used in the expression. So, if averaging_hours is 0.5, any sensor values will be averaged over half-hour intervals before they are used in the calculation. rolling_average (optional: the value True or False, default is False) When this parameter is False, the default, each averaging interval for the sensor values is distinct and does not overlap with other intervals. For example, if averaging_hours is set to 4.0, sensors will be averaged over 6 different periods across the day: Midnight to 4 am, 4 am - 8 am, etc. And, 6 calculated values will be produced corresponding to each one of those intervals. However, if rolling_average is set to True, a new calculated value will be computed for each and every timestamp that is present for sensor A. For example, if sensor A has a reading at 10:30 am, the averaging_hours is set to 2.0, and rolling_average is True, all sensors involved in the calculation will be averaged over the period of 8:30 am - 10:30 am (the 2 hours preceding sensor A’s timestamp), and a new calculated value will be created for that time interval. If another sensor A reading is present for 10:40 am, and new calculated value will be computed for the 8:40 am - 10:40 am period. So, there will be overlap in the time periods used for computing the new calculated field values, and many calculated readings will be generated even if averaging_hours is set to a large value. time_label (optional: the value left, right, or center, default is center) If time averaging is being used in the calculations (i.e. the parameter averaging_hours has a value), then this time_label pa- rameter determines where the timestamp for the new calculated reading will be placed. The default is center, which places the timestamp at the center of the time interval encompassing the averaged readings. For exam- ple, if the averaging period is Midnight to 4 am and center placement is being used, the timestamp for the calculated reading will be at 2 am. If left is specified, the timestamp is at the earliest edge of the interval, Midnight in this example. If right is specified, the timestamp will be at the latest edge of the interval, 4 am in this example. This parameter is also relevant rolling averages are being computed.
Note: If time averaging is not being used in the calculation, here is the procedure for determining the values and timestamps used in the calculated field. First, timestamps for the calculated field are aligned with the timestamps for Sensor A; i.e there will be a reading generated for every timestamp present for Sensor A. Next, other sensor timestamps
82 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
may not perfectly align with those from Sensor A; for those other sensors, their values are linearly interpolated to match up with Sensor A timestamps before being used in the calculation.
4.12.2 Acquiring Weather Data from the Internet
BMON can currently access outdoor dry-bulb temperature, wind speed, and relative humidity data from the National Weather Service and dry-bulb temperature and wind speed from the Weather Underground service. Here is an example of the needed configuration for the National Weather Service:
In the first box, a Sensor ID has been created, in this example: elmendorf_temp. Title and Unit entries are filled out as they are for standard sensors. The Calculated Field box must be checked. For gather- ing outdoor dry-bulb temperature, the Transform or Calculated Field Function Name must contain the value getInternetTemp (correct capitalization is critical and must be as shown). Finally, the Function Parameters in YAML form box must have an entry of stnCode: plus a 4 character National Weather Service station code, in this example (there must be a space after the colon):
stnCode: PAED
The only changes necessary to acquire a wind speed value in miles per hour is to enter getInternetWindSpeed into the Transform or Calculated Field Function Name box, change the Unit to velocity: mph, and enter an appropriate Sensor ID and Title. Acquiring relative humidity data in % RH requires entering getInternetRH into theTransform or Calculated Field Function Name box, and making appro- priate unit and title changes elsewhere.
The MesonetAPI service includes a larger set of weather stations. To gather temperature or wind data from this service, you must first acquire a Mesonet API Token and enter that key into the BMON Settings File as the BMSAPP_MESONET_API_TOKEN setting (restarting the Django web application after changing a setting is nec- essary).
4.12. Calculated Fields 83 bmon Documentation, Release 0.0.1
There is currently no charge for limited use of the API up to 5,000 requests and 5 million service units per month. Beyond that there is charge of 5 cents per thousand requests, and 15 cents per million Service Units. If either your Requests or Service Units exceed the free tier levels, you will be charged a $5.00 monthly service fee, in addition to the rated charges for any usage above the free tier levels. See the Mesonet Pricing Page for more information. Here is an example configuration for acquiring temperature data from the service:
Calculated Field
Transform or getAllMesonetTemperature Calculated Field Function Name:
Function stn: F2072 Parameters in request_interval_hours: 2 YAML form: since: 6/1/2019
The key differences from the National Weather Service configuration are: • getAllMesonetTemperature must be entered into the Transform or Calculated Field Function Name box. If you are acquiring wind speed data, then the correct entry is getAllMesonetWindSpeed. Capitalization must be as shown. • The Function Parameters box must contain a stn entry for the weather station you want data from. To find station codes, refer to the Mesonet map. • The Function Parameters box may contain an additional entry for the request_interval_hours which specifies the minimum interval at which data is updated. To stay within the limit of 5,000 requests per month, the interval can be 0.5 for up to three calculated sensors or 2.0 for up to 13. To estimate the minimum interval you can take the total number of fields that will use the mesonet API and multiply by 0.15. The default is two hours. • The Function Parameters box may also contain an additional since entry which specifies the earliest date or date/time to retieve data for.
4.12.3 Converting On/Off Events into Runtime Fraction
Some sensors record the precise time of On and Off events. An example of such a sensor is a Monnit Dry Contact sensor. This sensor posts a reading every time its two contacts are closed or are opened, and the sensor is often used to record when a device turns on and turns off. In addition to seeing the exact times a device turned on and turned off, it is often useful to record the percentage of time that the device was on during evenly spaced intervals. To provide this additional information, a special Calculated Field function is provided in BMON. The function will create a separate “sensor” in the BMON system that shows the fraction of time that a device was On for every half-hour interval (or other user-configurable interval). This function is called runtimeFromOnOff, and here is an example of its use:
84 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
The Unit entry generally should be runtime: Runtime Fraction or fraction: Occupied Fraction. runtimeFromOnOff must be entered as the Transform or Calculated Field Function Name. Finally, you need to provide the Sensor ID of the sensor that records the precise On and Off times (that sensor needs to report a value of 1 when the device turns on and a value of 0 when the device turns off). That Sensor ID is entered as the onOffID parameter in the Function Parameters box: onOffID: 29631
In this example, the Sensor ID is 29631, an ID of a Monnit Dry Contact sensor. By default, this function will calculate the runtime fraction for every half-hour interval. If you would like to use a different interval, add a second line to the Function Parameters box. For the above example, the following would be the entry for calculating 15 minute runtime fractions: onOffID: 29631 runtimeInterval: 15
This special runtime function is also useful with Motion or Occupancy Sensors and 1-Wire Motor Sensors used with the Mini-Monitor.
4.12.4 Storing the Raw Count Values from a Rate-of-Change Sensor
Counter type sensors generally use a Transform function to transform the cumulative count registered by the sensor into a rate-of-change of the quantity being sensed. For example, a fuel meter will register the total cumulative gallons of fuel consumed. A Transform function is usually applied to the cumulative gallon value to convert it to a rate of use per hour or per day. See the “Pulse Counter Transforms” section on the Transform Expressions page for further information. However, somestimes it is desirable to also store the cumulative count registered by the sensor, in addition to the rate of change. The lastCount calculated function described in this section meets that objective. The screenshot below shows a typical configuration of the calculated function.
4.12. Calculated Fields 85 bmon Documentation, Release 0.0.1
The only parameter necessary is the sensorID (Sensor ID) of the rate-of-change sensor. This calculated function will then acquire and store the last raw count that was used by the sensor to determine the rate of change. Note that calculated functions only run every 1/2 hour, so the time resolution for these count values will be 1/2 hour, even if the counter reports at a more frequent interval.
4.12.5 Acquiring Building Energy Usage Information from ARIS
BMON can import building energy usage information from AHFC’s Alaska Retrofit Information System (ARIS). Configuring a sensor for the imported data is very similar to the process for acquiring weather data from the internet described above. Using the administration interface, create a new Sensor ID. Title and Unit entries are filled out as they are for standard sensors. The Calculated Field box must be checked. The Transform or Calculated Field Function Name must contain the value getUsageFromARIS (correct capitalization is critical and must be as shown). Finally, the Function Parameters in YAML form box must have an entry of building_id: (there must be a space after the colon) with a valid building id number from the ARIS database, and an entry of energy_type_id: with a valid energy type value as described below. Required Function Parameters in YAML form:
building_id:1 energy_type_id:1
86 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Additional Optional Function Parameters in YAML form: energy_parameter:'EnergyQuantity' energy_multiplier:1 expected_period_months:1 building_id Parameter The easiest way to find a building_id value is to look on the ‘Commercial REAL Form’ in the ARIS user interface. When you select a building the building_id should show up in the upper left corner of the form. energy_type_id Parameter Possible values for the energy_type_id parameter: • 1 Electric • 2 Natural Gas • 3 Propane • 6 Coal • 7 Demand - Electric • 8 Demand - Nat Gas • 10 Steam District Ht • 11 Hot Wtr District Ht • 12 Spruce Wood • 13 Birch Wood • 14 #1 Fuel Oil • 15 #2 Fuel Oil energy_parameter Optional Parameter The energy_parameter specifies which value will be read from the ARIS database: • EnergyQuantity: The amount of energy used • DollarCost: The cost of energy for the given month • DemandUse: The amount of energy demand • DemandCost: The cost of energy demand for the given month, in dollars A value of ‘EnergyQuantity’ will be used by default if you don’t include this parameter. energy_multiplier Optional Parameter The energy_multiplier is a multiplier that is used to scale the value that is read from the ARIS database. If you don’t include the parameter, a value of 1.0 will be used by default. The value that is stored is calculated as: • For EnergyQuantity: [stored value] = [value from ARIS] * energy_multiplier / [total hours in the read period] • For Costs: [stored value] = [value from ARIS] * energy_multiplier / [standard length months in the read period] • For DemandUse: [stored value] = [value from ARIS] * energy_multiplier
4.12. Calculated Fields 87 bmon Documentation, Release 0.0.1 expected_period_months Optional Parameter In rare cases where the normal read period for the energy usage is something other than one month, you can enter a different number of months using this parameter. This value is used for estimating the previous read date when the date wasn’t set for the previous entry in ARIS, and for detecting missing data when the previous read date is more than 1.75 * [expected period months] earlier than the current read date.
Additional Required Settings
To use the BMON ARIS functionality you need to enter the URL, Username and Password in your installation’s settings.py file. The required settings parameters are: • BMSAPP_ARIS_URL • BMSAPP_ARIS_USERNAME • BMSAPP_ARIS_PASSWORD
4.12.6 Estimating Pellet Consumption and Heat Output of an Okofen Pellet Boiler
A Periodic Script is available to collect data from Okofen Wood Pellet Boilers. One of the Sensors indicates the Status of the boiler (the P241 sensor). If the Boiler Status is in state 5 or 6, then the boiler is firing, consuming pellets, and producing heat. A special calculated field has been created, OkoValueFromStatus, that allows you to create a new field showing the pellet consumption rate or the heat output rate of the boiler for every 5 minute interval. Here is an example of the function in use:
88 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
There are the two critical parameters that should be provided for the function, shown here with example values:
statusID: HainesSrCtr_P241 value: 127.17
The statusID parameter gives the Sensor ID of the boiler’s Status sensor. For the example, the Sensor ID is HainesSrCtr_P241. When this sensor reads a value of 5 or 6, the Okofen boiler is firing. The value parameter is the pellet consumption rate or heat output rate that occurs when the boiler is firing. For this example, that rate is 127.17 pounds per day of pellets (the units were specified in the Unit entry of the sensor). The calculated field will generate pellet consumption rates or heat output rates for each 5 minute interval spanning the available Status data set. It is often useful to the use the Data Averaging feature of the Plot Sensor Values over Time graph to see the average rates across day, week, or monthly periods.
4.12.7 Deprecated Calculated Field Functions
Warning: Deprecated functions are described below and are present for backward compatibility. Instead, use the genericCalc feature, described earlier in this document, for new work.
4.12. Calculated Fields 89 bmon Documentation, Release 0.0.1
Prior to development of the genericCalc function described above, calculated fields were only possible for a few different types of mathematical expressions. These specific types of calculated fields are described in this section, however, the genericCalc approach should be used in their place; the functions below are left available for back- ward compatibility reasons. The table below shows these functions and use of the functions is explained in the section following the table.
Function Name Expression Performed linear slope * val + offset
slope default is 1.0 offset default is 0.0
AminusB A-B
AplusBplusCplusD A + B + C + D
C default is 0.0 D default is 0.0
fluidHeatFlow flow * (Thot - Tcold * multiplier * (1.0-heat_recovery)
heat_recovery default is 0.0
Each one of these functions can create a Calculated Field based by applying a mathematical expression to a number of variables. The mathematical expression that is used is shown in the Expression Performed column of the table above. Each expression has a number of variables. Each variable can either be a number or Sensor ID (at least one of the variables must be a Sensor ID). Variables may have default values, as indicated in the table above. If a variable has a default value, it does not need to appear in the Function Parameters configuration box. Here is an example for the linear function:
90 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
In this example, there already is a sensor that reports the firing rate of a boiler as a percentage value varying from 0 to 100. We now want to create a Calculated Field that displays the rate of natural gas use of the boiler, expressed in Btu/hour. Because the gas use and the firing rate of the boiler are linearly related, we can use the linear Calculated Field function to create this gas usage field. Multiplying the firing rate by 1500 will give the gas usage in Btu/hour since the maximum gas usage of the boiler is 150,000 Btu/hour; a 100 firing rate times 1500 gives a gas usage of 150,000. The linear function has three variables: val, slope, and offset. For our example, our conversion multiplier of 1500 is the slope variable, and you can see its entry in the Function Parameters in the above screenshot. The offset variable is not needed in this application; BMON has a default value of 0.0 for this variable, which is correct for our application, so therefore we need not provide the variable in the Function Parameters box. Finally, the val variable will be used for the Firing Rate sensor values that we are using to calculate gas usage. Since this variable needs to be filled in with sensor values, we need to preface the variable with ‘‘id_‘‘ to indicate that this variable is a set of sensor values. Then, the value provided for the variable in the Function Parameters box is a Sensor ID: id_val: Burt158_firing_rate
The id_ prefix on the variable val indicates that the variable will be taken from an existing sensor. Burt158_firing_rate is the Sensor ID of the firing rate sensor. So, every 30 minutes BMON will gather up all of the Burth158_firing_rate sensor readings that have not already been used previously in this calculation, and BMON will multiply the by 1500 to create additional sensor readings for the Burt158_boiler_gas sensor. Here is a more complicated example that creates a Calculated Field that estimates the natural gas usage of a sidewalk snowmelt system based on measuring supply and return temperatures and the runtime of a circulating pump:
4.12. Calculated Fields 91 bmon Documentation, Release 0.0.1
The Calculated Function being used here is the fluidHeatFlow function, as described in the table above. You can see in the Function Parameters box that the heat_recovery variable is not provided in the configuration of this Calculated Field. Therefore, the heat_recovery variable will assume its default value of 0.0. Three of the variables in the math expression for the fluidHeatFlow function come from existing sensor values: flow, Thot, and Tcold. In the Function Parameter box, these variable names are prefaced by the id_ prefix, indicating the values provided are Sensor IDs. The multiplier variable is not a sensor value but instead the constant 14960.0. Finally, you can see that the flow variable appears in the Function Parameter box as id_flow_sync. As explained before, the id_ prefix indicates that the variable comes from a Sensor. The _sync suffix indicates that the final calculated values for the new sensor (manor_snw1_gas) should be synchronized on the timestamps of this sensor. The other input sensor values (Thot and Tcold) will be interpolated to these timestamp values when the calculation occurs. If you have multiple sensor values entering into a Calculated Field, you can add the suffix _sync to the variable whose timestamp values should be used for the resulting calculated values. If you do not append _sync to one of the variable names, one of the inputs sensors will be used for synchronization, but it will not be easy to determine which one.
4.13 Periodic Scripts
There are a number of tasks that are useful to run on a periodic basis, such as: • Acquiring a batch of new sensor readings or data to store in BMON from a piece of equipment or server connected to the Internet. • Creating and sending a report on a periodic basis. • Sending data from the BMON database to a different database.
92 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
• Performing a maintenance task. The Periodic Script feature of BMON can be used for this purpose. There are currently a number of Periodic Scripts available in BMON: • a script to collect temperatures and other data from Ecobee thermostats, • a script to collect data from a Modbus TCP device accessible on the Internet, • a script to collect data from a Sensaphone IMS-4000 monitoring device, • a script to collect data from an Okofen wood pellet boiler, • a script to collect data from a Solar PV system using Enphase microinverters, • a script to send data to an InfluxDB database. Those Scripts are discussed later in this document. To set up a periodic script, go to the Home screen in the BMON Admin interface and select “Periodic Scripts”:
On the next screen, click the “ADD PERIODIC SCRIPT +” button in the upper right corner of the screen. Below are the inputs required to configure a Periodic Script, with example values:
4.13. Periodic Scripts 93 bmon Documentation, Release 0.0.1
Each of these inputs is described below: File name of script The name of the file containing the Periodic Script is entered here. Each of the Periodic Scripts is contained in a separate Python file. Enter the name of that file here, but leave off the “.py” file extension. In this example, the script is contained in Python file named ecobee.py. This is a script that retrieves temperature, humidity, occupancy and other data from an Ecobee thermostat. For the Periodic Scripts that are already available in BMON, the proper name to fill in here will be provided in the specific documentation for the script. Optional Description In this optional field, you can add comments to document the purpose of the script. For this example, the field is used to indicate that the script is retrieving data from the Ecobee thermostat at the Mitchell home. How often should script run Choose a time interval indicating how often the script should run. Intervals range from every 5 minutes to every 24 hours. You can also select Disabled to stop the script from running at all. Script Parameters in YAML form Each script may allow the System Administrator to set additional con- figuration parameters through use of script parameters. In the example here, an include_occupancy pa- rameter is given with the value of true. Multiple parameters can be entered on separate lines in the box, using YAML notation. The valid script parameters are documented for the specific script being run. Script results in YAML form This is not a User input but instead displays results generated by the script the last time it was executed. For all scripts, the script_start_time is provided, indicating when the script last ran. script_execution_time shows how long in seconds it took for the script to run. If the script is one that inserts new sensor readings into BMON, the reading_insert_message will indicate how many readings were successfully inserted. The developer of the script can return any number of values to be displayed here. Also, if errors occur running the script, they will display here. After configuring the Script, click the “SAVE” button in the lower right corner of the screen to finalize your script. You can return to this screen to examine the most current Script Results produced by the script. The rest of this document is used to discuss the currently available Periodic Scripts in BMON. If you are a Developer and want to develop custom Periodic Scripts, see the Periodic Scripts document for additional information.
94 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
4.13.1 Collect Data from Ecobee Thermostats
The ecobee Periodic Script collects sensor data from a set of Ecobee Thermostats belonging to one Ecobee account. The script is capable of collecting the following sensor data: • Actual Temperature at the main Thermostat • Desired Heating Temperature (the heating “setpoint” temperature) • Relative Humidity at the main Thermostat • Occupancy Status at the main Thermostat (reads “Occupied” if there was motion detected in the last half hour) • The fraction of time that the Thermostat was calling for heat during each 5 minute interval • The Actual Temperatures at any remote sensors associated with the thermostat • The Occupancy status at any remote sensors associated with the thermostat The script collects and stores this data in the BMON sensor reading database for display through the normal BMON application.
Ecobee Authorization Process
Before BMON is allowed to pull data from an Ecobee account, two things must happen: • An API Key must be obtained by an Ecobee-registered developer. This key must be entered into the BMON Settings File as the BMSAPP_ECOBEE_API_KEY settings entry. This is a one-time procedure done by the BMON System Administrator, not the owner of the Ecobee account. • For each Ecobee account where data extraction is desired, the owner of the account must authorize and add the BMON application to their Ecobee account. To do this: – Navigate to the “ecobee-auth” web page in the BMON application. For example, here is the URL to the page for BMON on the Alaska Housing Finance server: https://bms.ahfc.us/ecobee-auth/ . Substitute the correct base URL for your server. – The page will generate a 4-character PIN that needs to be entered into the Ecobee account. Follow the explicit instructions on the ecobee-auth BMON page to complete the process. Here is an example of the ecobee-auth page that generates the 4-character PIN:
– After clicking the button for Step 2, a new page appears with a box containing two important token strings:
4.13. Periodic Scripts 95 bmon Documentation, Release 0.0.1
– Use your mouse to select and then copy the access_token and refresh_token text in the box. Paste this text into the “Script Parameters in YAML form” input box for the Periodic Script you are setting up. Here is an example of how the inputs should look for the Periodic Script:
The script above will collect data from all of the thermostats in the Ecobee Account that went through the authorization process. Currently, there is no limit on the number of thermostats that can be added to one Ecobee account. If you wish to collect data from other Ecobee accounts, you need to set up a separate Periodic Script for each account. Each account will have to go through the authorization process described above, and separate tokens will be created for each account. The initial authorization tokens that were acquired through the above process will expire. BMON knows how to acquire fresh tokens and will do so without any intervention by a user. Because the initial tokens acquired above are quickly invalid, BMON deletes those tokens out of the “Script Parameters” input box. If you re-open the Periodic Script configuration screen, you will see that the access_token and refresh_token entries are gone. This is not cause for alarm, as BMON internally has the new tokens stored. If the Account owner removes the BMON Application from the Ecobee account, or if some other authorization prob- lem occurs, you will have to use the ecobee-auth page again to acquire new tokens. Enter the new tokens into the “Script Parameters” input box to restore BMON’s access to the Ecobee account.
96 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
Other Notes on Ecobee Configuration
Here are some additional notes on configuring Ecobee Periodic Scripts. For the File name of script input, the value must be ecobee. A good use for the Optional Description input is to put the name of the Ecobee Account that is accessed by the script. For the How often should script run input, the best value is “10 min”, because the Ecobee API server returns data from three sequential 5-minute intervals, totalling 15 minutes of data. Polling the server every 10 minutes ensures that the server will be polled at least once during every 15-minute interval. For the Script Parameters in YAML form input, there is currently only one possible parameter other than the initial access_token and refresh_token parameters that are entered during the authorization process de- scribed above. That one parameter is: include_occupancy Possible values are True or False. If the parameter is set to True, the script will collect occupancy values reported by the occupancy sensor in in the main thermostat and the occupancy values reported by any remote thermostats. For Ecobee3 Lite thermostats, no occupancy sensors are present, so no occupancy values can be collected, irrespective of the include_occupancy parameter value. After the Ecobee Periodic Script is properly configured and saved, collection of Ecobee data will begin. Successfully collected sensor readings will be stored in the BMON sensor reading database. However, these readings will not be displayed in the BMON graphing and report interface until each individual sensor is configured in the BMON system. See the Adding Sensors section for the process of adding a sensor into the BMON system. This process assigns an understandable name to the sensor, puts the sensor in an appropriate Sensor Group, and assigns the sensor to one or more buildings. When adding a sensor to the BMON system, the sensor’s ID must be entered into the Sensor ID, or Calculated Field ID input. For each of the sensor readings collected by the Ecobee script, a Sensor ID is created. The table below shows an example of the Sensor ID’s that are created by the script. This table is based on an Ecobee Thermostat serial number of 311015614158 and a Remote Sensor code (found on the back of the remote sensor) of CK2W. Your actual Sensor IDs will be based on your Thermostat Serial Number and your Remote Sensor Codes.
Sensor ID Reading Type 311015614158_temp Actual Temperature at Main Thermostat, deg F 311015614158_heat_setpoint Desired Temperature (heating setpoint), deg F 311015614158_heat1_run Fraction of the time that the thermostat called for heat during the 5 minute interval 311015614158_rh Relative Humidity measured at the main thermostat, % 311015614158_occup Occupancy at main thermostat. 1 if motion was sensed during the last half hour, 0 if no motion. 311015614158_CK2W_temp Actual Temperature at the remote sensor, deg F 311015614158_CK2W_occupOccupancy at remote sensor. 1 if motion was sensed during the last half hour, 0 if no motion.
The occupancy readings are only included if include_occupancy is set to True in the Script Parameters. The Remote Sensor values are repeated for every remote sensor associated with the thermostat. Remote Sensors are not supported by the Ecobee3 Lite thermostat.
4.13.2 Collect Data from Devices with a MODBUS TCP Inteface
A BMON Periodic Script is available to collect data from devices that have a MODBUS TCP interface, such as many Programmable Logic Controllers (PLC). The periodic script can collect multiple holding register values from one MODBUS device.
4.13. Periodic Scripts 97 bmon Documentation, Release 0.0.1
Note: If you have upgraded from an older installation of BMON, you will need to install the modbus-tk==0.5.7 Python library to use this periodic script.
Below is a screenshot shot showing example inputs for one MODBUS Periodic Script:
The File name of script entry must be modbus in order to invoke the MODBUS periodic script. The Optional Description and How often should script run entries have been previously described in this document. The rest of this section will describe the Script Parameters in YAML form entry. site_id (required) The site_id is used to create a BMON Sensor ID for each of the holding registers collected by the script. The site_id is used as the first part of the Sensor ID; the latter part of the Sensor ID is the sensor name, which is entered for each holding register collected. The sensor name entry is described in more detail below. host (required) This is the IP Address or the host name of the target MODBUS device. device_id (optional, defaults to 1) Some MODBUS devices are composites of several MODBUS devices; for example, a number of devices behind a MODBUS gateway. This this case, the device_id identifies which device is the target of the MODBUS command. holding_registers (required) As can be seen in the example, this parameter is entered as a YAML list; each item in the list describes one holding register that will be read and stored under one Sensor ID in BMON. Each holding register is described on one line and is in turn a YAML list of either three or four items. Four examples will be described here. The first holding register example is:
98 Chapter 4. Contact Information bmon Documentation, Release 0.0.1
-[550, 2084, heat_rate]
The line must start with a dash - and there must be a spaced between the dash and the bracketed list items. The items in this exmaple are - [port, MODBUS address, sensor name]. The port (550 in this example) is the TCP/IP port at the Host which is used to access the MODBUS device. The MODBUS address (2084 in this example) is the MODBUS address of the holding register to read. It can be a number from 0 through 9998. The sensor name is the appended to the site_id, described earlier, to make a BMON Sensor ID. The site_id and sensor_name are separated by an underscore to make the Sensor ID. In this example, the final Sensor ID would be abc_heat_heat_rate, since the site_id is abc_heat. The second holding register example is:
-[550,[2087, 2086], total_heat]
The only thing different about this example is that a list of MODBUS addresses are given. When this is done, the values from the specified holding registers are combined into one number. The value from each register is considered to be a 16-bit digit in the final number; the most-signicant holding register is specified first in the list. In this example, assume that register 2087 contained the number 7 and that register 2086 contained the value 14345. The final value stored in BMON would be:
7 * 65536+ 14345 which equals: 473,097
The value 65536 is 2 raised to the 16 power. The third holding register example also involves a list of MODBUS addresses, but a type identifier is added as the last element in the list of addresses:
-[550,[2087, 2086,'f'], total_heat]
Currently, the only supported identifier is ‘f’, which indicates that the two 16-bit words read from the registers should be interpreted as a single-precision floating point number. The first address listed holds the most-significant 16-bit word and the second address holds the least-signficant word. Some devices encode floating point values using this method. The final holding register example uses the optional fourth descriptive parameter:
-[550, 2102, temp_return, val/10]
The fourth parameter is a transform function that is applied to the holding register value before being stored in BMON. The convention used is that the variable val in the function holds the raw holding register value. In this example, the transform function is val/10, so the holding register value is divided by 10 before being stored in BMON. All Python math operators are available, including all of the functions present in the ‘math 4.13.3 Collect Data from a Sensaphone IMS-4000 Monitoring System A Sensaphone IMS-4000 is a monitoring system with the capability of supporting multiple remote units. A BMON periodic script is available that retrieves sensor values from the IMS-4000, including sensors installed on remote units. The Sensaphone must have its SNMP port accessible to the BMON server. Below is a screenshot of an example Periodic Script configuration to read an IMS-4000: 4.13. Periodic Scripts 99 bmon Documentation, Release 0.0.1 To use this script the File name of script must be sensaphone. The Optional Description and How often should script run entries have been previously described in this document. The rest of this section will describe the Script Parameters in YAML form entry. There are two required parameters: site_id (required) The site_id is used to create a BMON Sensor ID for each of the sensor values collected by the script. The site_id is used as the first part of the Sensor ID; the latter parts of the Sensor ID are formed from the Sensaphone Node name and the Sensaphone Sensor name. Each component is sepa- rated by an underscore character. In the example above, the site_id is ABCD; for a Sensaphone Node named Kiana and a Sensaphone Sensor name of room_temperature, the final BMON Sensor ID will be ABCD_Kiana_room_temperature. Note that any spaces present in the site_id, Sensaphone Node name, or Sensaphone Sensor name are replaced with the underscore character. host (required) The IP Address or Host name of the Sensaphone. 4.13.4 Collect Data from Okofen Wood Pellet Boilers Wood pellet boilers manufactured by Okofen have a data collection system that can be accessed from the Internet. A BMON Periodic Script is available to collect data from such a boiler. The boiler creates a daily CSV file containing one-minute resolution values of various boiler parameters. The Okofen web interface to the boiler allows you to determine which boiler parameters are collected and stored in the CSV file. Please see Okofen documentation for the process of selecting and enabling those parameters. The rest of this section describes the process for configuring the Periodic Script in BMON that acquires and stores the data from the Okofen CSV files. As well as this documentation, there is also a Powerpoint Presentation that explains the configuration process. The screenshot below shows an example of a configured Periodic Script for collecting data from an Okofen boiler: 100 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 The File name of script must be okofen. The Okofen boiler creates a new CSV file once per day, presum- ably shortly after Midnight. There is little harm in allowing the BMON Periodic Script to run much more frequently, as it simply exits if it has already downloaded the newest file. 30 min is a good choice. If Internet access to the boiler is unavailable for a period of time, BMON will automatically retrieve multiple days of missed CSV files when Internet connectivity is restored. The Script Parameters in YAML form input can include the following parameters: url (required) This is the URL for accessing the Okofen boiler. The boiler access is usually on port 8888, so the URL must include this port number. Here is an example: http://64.182.156.18:8888 site_id (required) This string ID is used to create unique Sensor IDs for the values extracted from the boiler. A short descriptor of the site is best, with no spaces in the string. tz_data (optional, defaults to US/Alaska) The controller/data-collector in the boiler has a timezone setting that affects the timestamps of the values it records in the CSV files. The timezone may not be correctly set to the location of the boiler, for example the Hawaii timezone has been used for Alaskan boilers. Set this parameter to a timezone string found in this timezone database list. last_date_loaded (optional, normally determined from the last run of the script) The script internally keeps track of the last CSV file it successfully loaded, and then will request files after that date in subsequent runs of the script. If the script has never been run before, the first file it attempts to load is from a day two weeks ago. This is rarely necessary, but you can override this behavior by entering in a last_date_loaded 4.13. Periodic Scripts 101 bmon Documentation, Release 0.0.1 in a form like ‘2016-11-23’. When the script next runs, it will start loading CSV files one day after that date. You might use the parameter to avoid loading two weeks of older CSV files on the initial run of the script. It is difficult to determine a case where you would want to leave this parameter for more than one run of the script, as the script internally tracks the last CSV file loaded. The okofen Periodic Script collects all of the sensor values that have been enabled in the Okofen boiler. Each of these sensor values is assigned a Sensor ID by BMON. The table below gives examples of how the Okofen sensor names are translated into BMON Sensor IDs. The example in the table assumes that the site_id in the Script Parameters input box was set to HainesSrCtr: Okofen Sensor Name BMON Sensor ID P107 Flue/Flame temp HainesSrCtr_P107 P112 Burner Startups HainesSrCtr_P112 Boiler 1 HainesSrCtr_boiler_1 Here is the general rule. All BMON Sensor IDs start with the site_id followed by an underbar character. Then, if the Okofen sensor name contains a 3-digit parameter number in the form Pxxx, that parameter number (including the leading ‘P’) is added to the BMON Sensor ID. If there is no parameter number in the Okofen sensor name, a translated version of the Okofen name is added to the BMON Sensor ID. The translation converts all spaces and slashes to the underbar character, removes all period characters, and converts all letters to lower case. In order to see these sensors in the normal data display and analysis portion of BMON, they must be added to the Sensors table in the BMON Admin interface. This process is documented in the “Adding Sensors” section of the Add Buildings and Sensors document. There is also a special Calculated Field function available that can create a Sensor that estimates the pellet consumption or heat output of the boiler by examining the Status (P241 parameter number) sensor of the boiler. See the calculated-fields document for more details. 4.13.5 Collect Data from Enphase Solar PV Systems A Periodic Script is available to collect power production data from Solar PV Systems that utilize Enphase microin- verters. Before using this script, two tasks must be completed: • A developer API key must be obtained and an Enphase application must be set up, according to the “Creating Your Account and Getting Your API Key” section on this page. This only needs to be done once. • For each Enphase system that you want to collect data from, you must obtain permission from the owner that of that system to receive the data. The process of receiving permission is described on the same web page in the “Getting Access to Enlighten Systems” section. Here is a screenshot of a sample Periodic Script configuration that collects data from an Enphase solar system: 102 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 The File name of script must be enphase. The Script Parameters in YAML form input has the following parameters: api_key (required) This is the API key that you obtained in the prerequisite task described above. first_date (optional) When the script runs for the first time, you can tell it the earliest date/time of data that you want collected. Do so by filling out this parameter with a date or date/time string in any format. The Enphase API will not provide data that occurred more than one year ago, so ensure that this parameter is more recent than one year ago. After the script runs once, this parameter and the associated timezone parameter described below are deleted from the parameters box. If you don’t fill out this first_date parameter, the script will start acquiring data from one day prior to the script run time. first_date_tz (optional) If you provide a first_date parameter, you can also provide the timezone that should be used to interpret that date/time. If you do not provide a timezone the date is assumed to be in the Alaska timezone. A valid list of timezone labels can be found here. systems (required) The systems to acquire data from are listed here. The format, including indentation must be exactly as shown. Each system has two fields associated with it that are required: system_id - the Enphase System ID for the system, and user_id the Enphase ID of the user that owns the system; (user_id is indented two spaces). Any number of systems can be listed. Each time the Periodic Script runs, data from each listed system will be acquired. The script acquires the total system production with 5 minute resolution, measured in Watts. The values are measured by the microinverters and may differ somewhat from measurements done by a system production meter connected to the Enphase communication module. For each run of the Periodic Script, the API limits collection to one day’s worth of data. The next run of the Script will pick up where it left off, acquiring any data available after the last data point collected from the prior run, subject to the one day limitation. 4.13. Periodic Scripts 103 bmon Documentation, Release 0.0.1 BMON Sensor IDs The BMON Sensor ID for each system collected will be of the form: enph_ So, for the two systems shown in the example configuration above, the BMON Sensor IDs would be: enph_4e71437334d7a98730a_1023441 enph_4123a67abc7a51330a_1049390 Prior to setting up the Sensors in BMON, you will see these Sensor IDs in the “Find Unassigned Sensors” report found on the “Sys Admin” menu. Enphase API Rate Limits for Free Plan If you signed up for the free “Watt” Enphase API Plan, your use of the API is limited to 10 API calls per minute and also limited to 10,000 API calls per month. If you only use your API key in one Periodic Script (remember that you can collect data from multiple systems with one script), BMON ensures that you will not violate the 10 calls / minute limit. However, it is up to you to configure the script run frequency in order to satisfy the 10,000 calls per month limit. Here is a formula you can use to set the frequency of the script run: Minimum Minutes between Script Runs= 4.464x( # of systems collected) So, if you are collecting data from 5 systems, you need to wait at least 4.464 x 5 = 22.3 minutes between Periodic Script runs. The “30 min” choice on the “How often should script run:” drop-down will satisfy this requirement. The only disadvantage of choosing a longer period between runs of the script will be a delay in the availability of the data in BMON. The script will collect all available data since it’s prior run (subject to the limit of 1 day’s worth of data). So, no data will be lost by running the script infrequently (up to 24 hour spacing between script runs). 4.13.6 Send BMON Data to an InfluxDB Time-Series Database InfluxDB is a database optimized for storing and querying Time Series data, such as the data collected from sensors. This database is also supported as a data source for the Grafana Time Series Analytics package, which is a leading software package for creating graphical dashboards displaying time-series data. Both InfluxDB and Grafana are open source software. A BMON Periodic Script is available that will periodically send all new BMON sensor data to an InfluxDB database. An installation of Grafana can then display that data in a dashboard. To understand the configuration of this Periodic Script, you need to have a reasonable understanding of the use of an InfluxDB database, as described on the InfluxDB documentation page. Here is a screenshot of a sample Periodic Script configuration that sends data to an InfluxDB database: 104 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 The File name of script must be influxdb. The Script Parameters in YAML form input can include the following parameters: influx_url (required) This is the HTTP URL that allows writing to the InfluxDB database. If the write does not occur on the standard HTTP(s) port, then the port number should be included in the URL, as shown in this example. database_name (required) An InfluxDB server can contain multiple databases. In this parameter, specify the name of the InfluxDB database to write the BMON sensor readings into. username (required if InfluxDB is using authentication) The InfluxDB username to authenticate with, which must have write privileges on the target database. password (required if InfluxDB is using authentication) The password associated with the above username. measurement (defaults to ‘reading’) Every point in the InfluxDB database has a “measurement” type associated with it. This Periodic Script stores all of the BMON sensor readings under one measurement, which defaults to ‘reading’. However, you can change the measurement type with this parameter. value_field (defaults to ‘value’) Every point in the InfluxDB database has one or more field values. All of the BMON sensor readings sent by the Periodic Script are stored in one field, and the default name of that field is ‘value’. That field name can be changed with this parameter. reach_back (default = 14, measured in days) The first time the Periodic Script runs, it decides how much histor- ical data to send to the InfluxDB database. The reach_back parameter determines the amount of historical data sent, and it is measured in days. In subsequent runs of this Script, only new data is sent to the InfluxDB database. ignore_last_rec (default = False) If this parameter is set to True, the script will send all historical data up to the reach_back limit, even if the script has run before and sent that data. This can be used to repopulate a database that has been reset or cleaned of old data. This parameter should not be set to True on a continual basis, as that will cause the Script to send all historical data (up to the reach_back limit) every time the script runs. 4.13. Periodic Scripts 105 bmon Documentation, Release 0.0.1 InfluxDB Tags Created during Export InfluxDB allows each point in the database to be described by any number of “tags”. A tag is a key/value pair that adds metadata to the database record. An example tag would be “sensor_id=451897”, which gives the Sensor ID associated with this particular sensor reading. These tags are used to extract particular data points from the Database. Grafana knows how to build queries using these tags. When BMON sends data to the InfluxDB database, it automatically adds a number of tags to every sensor reading sent. Below is the list of tags that are automatically created by the BMON Periodic Script. Also note that all tag values sent by BMON are all “cleaned” by replacing spaces, the equals sign, and commas with the dash/hyphen character, “-“. Further, if the replacement results in more than one dash in a row, the string of dashes is reduced to one. building_title The value of this tag is the title of the Building associated with the sensor reading (as entered in the ‘Buildings’ table of the BMON Admin interface), e.g. ‘building_title=Kaluza-House’. Note that if a sensor reading is associated with more than one building, the sensor reading is sent multiple times to the InfluxDB database, once for each associated building. latitude The latitude of the associated Building (again, as entered in the Building table of the BMON Admin interface). longitude The longitude of the associated building. sensor_group The Sensor Group chosen when the sensor was associated with the Building in BMON. sensor_id The Sensor ID for the sensor, as entered in the Sensors table in the BMON Admin interface. sensor_title The Sensor Title, as entered in the Sensors table in the BMON Admin interface. unit The measurement unit type for the sensor, such as ‘deg-F’ or ‘kWh’. The above tags are always included with every sensor reading point sent to InfluxDB. In addition, you can have other tags sent with the readings by filling in Additional Properties input boxes found in both the Buildings BMON Admin table and in the Sensors BMON Admin table. Any key/value pairs that you enter into these areas will also be sent along with the readings to InfluxDB. Here is an example of two additional properties that were entered for a particular Building: For all sensors associated with that building, these properties will be made into key/value pairs and sent as tags with the sensor readings. Tag Names will be “cleaned” by substituting the underbar “_” character for spaces and Tag Values will be “cleaned” by substituting the dash character “-” for spaces. Also, if any of the Additional Properties for a Building or Sensor use the same Tag Name as an automatically-created Tag Name, the user-entered Tag will override the automatically-created Tag. When creating dashboards in Grafana, all of the Tag Names and Values will automatically be displayed as choice lists when creating data queries to feed your dashboard. 106 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 4.14 How to Create Custom Jupyter Notebook Reports BMON has a top-level Menu Item titled “Energy Reports” (a BMON version of March 24, 2020 or later must be present, and the Energy Reports feature must be configured in the BMON Settings file). A number of special reports that analyze the energy use of individual Buildings and analyze the energy use of Organziations are available to show on this BMON page. These reports satsify many analysis needs, but you also have the ability to create your own custom report developed in a Jupyter Notebook using the Python Programming Language. Note that these reports fall under the “Energy Reports” menu item, your custom reports do not have to be related to energy use; they can use any sensor data available in BMON and perform any form of data analysis. A Jupyter Notebook is a document that contains live programming code, equations, visualizations, and formatted text. Through use of the Python Programming language within the Notebook, data from BMON can be acquired, analyzed, visualized and reported on. The BMON Jupyter Notebook reporting is structured to run these reports nightly and convert the output from the report into an HTML document that is viewable through the “Energy Reports” menu item in BMON. The purpose of this How-To Guide is to show you how to create your own custom Jupyter Notebook report that is automatically run nightly and viewable through BMON. Your report can use BMON data and data from anywhere on the Internet. The Notebook has the full power of the Python programming language available to it, so the range of available processing and data analysis tools is very large. To successfully create a custom report, you will need to have some basic level of skill with the following tools and services: • The Python programming language. • Jupyter Notebooks • GitHub, a file sharing and version control system, which is used BMON to store the Jupyter Notebooks used for the creation of Energy Reports. Note that BMON for a number of years has had a “Custom Reports” feature that allows you to combine a number of BMON Graphs onto one page, and have that page accessible as a Report. This feature is described here: Custom Reports. This feature is simpler to use than the Custom Jupyter Report feature, but it lacks the power and flexibility offered by Jupyter Notebooks and the Python programming language. This guide provides some introductory text, but the bulk of the guide is in the form of videos that walks through creation of a simple custom report. These videos are available in the last section of this page. 4.14.1 Prerequisites This Guide assumes that your BMON system is already configured to run Energy Reports. There are a number of steps required to reach that point. This Guide is not meant to provide detail on those steps, but they are briefly listed here: • Create a GitHub repository to hold the all of the Jupyter Notebooks that are used to create the Energy Reports, including your custom report. Here is a Sample GitHub Repository. – That repository must include a config.yaml configuration file in the root directory. This file indicates which BMON Servers the reports will run against, and it also specifies the formatting theme used for the Jupyter Notebooks. • Ensure that the bmonreporter software is setup to run nightly and that the configuration file for that software lists the above GitHub repository. That software stores the created HTML reports in an Amazon AWS S3 bucket configured as a static website. This website is accessed by BMON to display the created the reports. • The ENERGY_REPORTS_URL setting in the settings.py file for your BMON server must have the URL of the AWS S3 static website. Further, that settings file must have an Energy Reports item in the BMSAPP_NAV_LINKS setting. 4.14. How to Create Custom Jupyter Notebook Reports 107 bmon Documentation, Release 0.0.1 Again, if your BMON install is already set up to run Energy Reports, none of the above needs to be performed prior to creating a Custom Jupyter Notebook Report. 4.14.2 Creating the Custom Jupyter Notebook Report This section contains two videos to explain how the Custom Jupyter Report is created. In this first video, the difference between Building reports and Organization reports is explained. Also, the video explains where to store your Custom Juptyer Notebook file in the GitHub repository. (Hint: In the YouTube Settings menu–cog symbol on the lower right–you can increase Playback speed to 1.25x, which may improve your listening experience!) This next video walks through creation of an actual Jupyter Notebook report. Please not some of the reference links below the video for important resources. As indicated in the video, creation of Custom Jupyter Notebook reports requires installation of a couple of special Python packages: • bmondata allows access to data (e.g. sensor readings, building characteristics) on BMON servers. • scrapbook is a special library for attaching arbitrary information to Jupyter notebooks. This is used to assign title and report order information to the notebook, which is then used by the report creation script. To install these packages, execute the following commands: pip install bmondata pip install nteract-scrapbook Here is some documentation for the bmondata Python package, which allows for easy access to BMON data from within your Jupyter Notebook (or any Python script for that matter). Example usage of the library is shown here. The Jupyter Notebook report that was created in the video above is available here. The Jupyter Notebook Reports that were used to create the primary Energy Reports for BMON are available in this repository. Find the reports in the templates directory. 4.15 Custom Reports Custom Reports give you the ability to combine any number of BMON charts and reports together into a single page, and gives you the ability to add explanatory text and include images and content from other websites. In the BMON interface, Custom Reports are accessed from the navigation link in the page header. After clicking the navigation link you’ll be presented with a list of all the reports that have been configured on your system. The reports are grouped under headings, and the title of each report is a clickable link that takes you to the report page. To create a Custom Report, log into the Admin interface and navigate to the screen that allows you to edit or configure Custom Reports: • Go to the main Home screen for the Admin Interface by clicking the Home link in the upper left corner of your current screen. • Click the Custom reports link in the left column of the Home screen. Once in the Custom Reports section of the Admin interface, you will see the list of Custom Reports that have already been entered into the system. If the system is new, none will be shown. Click the ADD CUSTOM REPORT button to add a new report. Here are the inputs for entering a new Custom Report: 108 Chapter 4. Contact Information bmon Documentation, Release 0.0.1 • Group: The name of the group header that you want this report to appear under in the list of custom reports. In the list, the groups are sorted alphebetically, but if you want them to appear in a different order you can prefix the group names with numbers (ex. 1 - First Group, 2 - Another Group). • Title: The name of this report as you want it to appear in the list. • Sort Order: A Sort order for this report, which can be used to order the report appears in the list of Custom Reports. Reports with lower Sort Order numbers will appear higher on the list. Leaving space between your Sort order values (e.g. 10, 20, 30, etc.) makes sense so a future report can be fit in between existing reports. If the order of the reports in unimportant you can leave the default value. • Report Content: This is where you define the content that will appear in the report. It allows for text, embedded BMON charts and reports, images, links, and other additional content to be entered using Markdown syntax. There are many great resources online for learning to use Markdown syntax, including this tutorial. There is more information on adding links and images under the BMON documentation for Add Buildings and Sensors. The easiest way to embed a BMON chart, report, or dashboard is to navigate to the chart on your BMON system (in the regular interface, not the Admin interface), and to click the -- Get Link -- link that is below the bottom right corner of the chart. Here is an example of where the – Get Link – option appears: 4.15. Custom Reports 109 bmon Documentation, Release 0.0.1 Clicking the link will bring up a dialog that shows the text that you need to include in the custom report to embed the chart. Click the Copy to Clipboard button and then paste (Ctrl+V) the text into the Report Content for the Custom Report that you are building. The text will look something like this: The text on the first line, between is just a label to help you keep track of what you’ve embedded. Feel free to edit that portion to create a more meaningful label. The text inside the