Custom Command Agent User Guide

1.6 VMC-Mxx VISUAL Message Center Custom Command Agent User Guide The software described in this book is furnished under a license agreement and may be used only in accordance with the terms of the agreement.

Document date: August 2012

Document version: 1.01

Product version: 1.6

No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any language or computer language, in any form or by any means, electronic mechani- cal, magnetic, optical, chemical, manual, or otherwise, without the prior written permission of Tango/04.

Trademarks Any references to trademarked product names are owned by their respective companies.

Technical Support For technical support visit our web site at www.tango04.com.

Tango/04 Computing Group S.L. Avda. Meridiana 358, 5 A-B Barcelona, 08027 Spain

Tel: +34 93 274 0051 Table of Contents

Table of Contents

Table of Contents...... iii How to Use this Guide...... vi

Chapter 1

Introduction ...... 1

1.1. What You Will Find in this Document...... 2

Chapter 2

Before You Begin ...... 3

Chapter 3

Windows Host Requirements...... 4

3.1. Checking for Administrative Shares in the Remote Host ...... 4 3.2. Checking the Remote Service Manager Access in the Remote Host...... 5

Chapter 4

Linux/Unix-based Host Requirements ...... 7

4.1. Checking a SSHv2 Connection Availability with the Putty Client...... 7 4.1.1. Checking Command Permissions and Availability ...... 8

Chapter 5

Configuration of Data Sources ...... 10

5.1. General Settings ...... 10 5.1.1. Main Information ...... 11 5.1.2. Target Host Settings ...... 11 5.1.3. Connection Settings...... 12 5.1.4. Linux/Unix Specific Settings...... 13 5.1.5. General Settings ...... 13 5.2. Command Execution Settings (Custom Command Data Source) ...... 14 5.3. Command Execution Settings (Advanced Custom Command Data Source)17 5.4. Command Execution Settings (Script Runner Data Source) ...... 20 5.5. Variables (Advanced Custom Command Data Source)...... 23 5.6. Command Output Settings...... 25

Chapter 6

Custom Command ThinAgent Configuration...... 29 6.0.1. General Settings ...... 29 6.0.2. Parser Options ...... 30 6.0.3. Table Settings ...... 36 6.0.4. Default Health Settings ...... 39 6.0.5. Default Message Templates ...... 40 6.0.6. Common Variables...... 40 6.0.7. Field Map SmartConsole – ThinkServer ...... 40

Chapter 7

Advanced Custom Command ThinAgent Configuration ...... 42 7.0.1. General Settings ...... 42 7.0.2. Parser Options ...... 43 7.0.3. Table Settings ...... 44 7.0.4. Default Health Settings ...... 44 7.0.5. Default Message Templates ...... 44 7.0.6. Common Variables...... 44 7.0.7. Field Map SmartConsole – ThinkServer ...... 45

Chapter 8

Script Runner ThinAgent Configuration...... 46 8.0.1. General Settings ...... 46 8.0.2. Parser Options ...... 47 8.0.3. Table Settings ...... 47

8.0.4. Default Health Settings ...... 47 8.0.5. Default Message Templates ...... 48 8.0.6. Common Variables...... 48 8.0.7. Field Map SmartConsole – ThinkServer ...... 48

Appendices

Appendix A: Some Execution and Parsing Examples ...... 50 A.1. Delimited Fields: Checking the Network Interface Connections on Windows.50 A.1.1. Checking the Requirements ...... 50 A.1.2. Configuring the Parser...... 51 A.1.3. Creating the Monitor ...... 52 A.1.4. Done! ...... 58 A.2. XML Tags: Reading the Solaris Audit Trail...... 59 A.2.1. Checking the Requirements ...... 59 A.2.2. Configuring the Parser...... 60 A.2.3. Creating the Monitor ...... 60 A.2.4. Done! ...... 65

Appendix B: Regular Expressions Syntax Reference...... 68 B.1. Characters ...... 68 B.2. Characters Classes or Character Sets [abc] ...... 69 B.3. Dot ...... 70 B.4. Anchors ...... 70 B.5. Word Boundaries...... 70 B.6. Alternation ...... 71 B.7. Quantifiers ...... 71

Appendix C: Reference...... 73

Appendix D: Contacting Tango/04...... 75

About Tango/04 Computing Group ...... 77 Legal Notice...... 78

How to Use this Guide

This chapter explains how to use Tango/04 User Guides and understand the typographical conventions used in all Tango/04 documentation.

Typographical Conventions The following conventional terms, text formats, and symbols are used throughout Tango/04 printed documentation:

Convention Description

Boldface Commands, on-screen buttons and menu options.

Blue Italic References and links to other sections in the manual or further documentation containing relevant information.

Italic Text displayed on screen, or variables where the user must substitute their own details.

Monospace Input commands such as System i commands or code, or text that users must type in. Keyboard keys, such as CTRL for the Control key and F5 for the UPPERCASE function key that is labeled F5.

Notes and useful additional information.

Tips and hints that will improve the users experience of working with this product.

Important additional information that the user is strongly advised to note.

Warning information. Failure to take note of this information could potentially lead to serious problems.

Chapter 1 1 Introduction

The Custom Command ThinAgents are a set of monitors that allow you to execute remote commands and scripts, and process the output and exit codes. The platforms it supports are Linux/Unix systems, and Microsoft Windows.

Regarding compatibility, the connections engine uses the most common technologies like RPC (Remote Procedure Call) for Windows, and SSHv2 (Secure Shell protocol version 2) for Linux/Unix. That means that the requirements of remote hosts are all standard.

The Custom Command suite of monitors includes the following ThinAgents:

• Custom Command

• Advanced Custom Command

• Script Runner

Figure 1 – The Custom Command ThinAgents shown in ThinkServer

The Custom Command ThinAgent allows you to execute a single command against a remote machine and retrieve its output in a very flexible way. It’s possible to parse the entire output with a powerful parser in many ways, including multiline processing.

The Advanced Custom Command ThinAgent is inherently like the Custom Command ThinAgent, however, you have the possibility to execute multiple commands on only one data source (up to 99 commands, in fact).

The Script Runner ThinAgent is much like the Custom Command one, but it replaces the command by a much more complex script that you can store directly on ThinkServer’s host. It automatically sends the script to the remote machine and executes it. You can then remove the script after the execution. It is also possible to define the interpreter that will run the script (like Python, Perl, and cmd.exe). 1.1 What You Will Find in this Document

This user guide describes the Custom Command ThinAgents, all the configuration settings for every component, and all the pre-configured variables for the monitors.

It also explains the minimum configuration settings required in the remote hosts in order to get the Custom Command ThinAgents running.

For a full description of VISUAL Message Center ThinkServer functionality see the VISUAL Message Center ThinkServer User Guide.

Chapter 2 2 Before You Begin

The Custom Command ThinAgents are multiplatform, which means that, depending on your remote target platform, requirements may change.

Normally one can create a monitor against a Windows or Linux/Unix host without needing to set anything on the remote server; however, as a general rule these two requirements apply:

• You MUST have network access to the remote host, allowing the platform specific protocol to be used.

• The platform specific service MUST be enabled and available in the remote host (SSHv2, RPC, etc.).

• You MUST have user permissions on the remote host to execute the requested commands/ scripts, and on Windows platforms you MUST be administrator.

More about platform specific protocols and services will be explained in the following chapter. But in almost any case, you won’t need to configure anything, because they are standard technologies that should be enabled in any modern file server.

Chapter 3 3 Windows Host Requirements

The currently supported Windows Operating Systems are almost all NT based technologies (Windows NT, Windows 2000, Windows 2003, Windows XP, Windows Vista, Windows 7 and Windows 2008).

When connecting to a LOCAL Windows host (where ThinkServer is installed), you don’t need anything special to execute commands: if the ThinkServer service is running with “localsystem” account (and by default it is), then you should already have access to the command execution service with administrator privileges. If the ThinkServer is running with another user account, then you will have to grant administrator permissions for that user account.

To connect to a REMOTE Windows host (a file server, a domain controller, a workstation, etc.), the Custom Command ThinAgents use RPC (Remote Procedure Call). Also, a service must be installed on the remote computer through SMB (Server Message Block), so you MUST provide an administrator account in order to automatically install this service. You also must enable administrative share on the remote host, and access to the remote service manager has to be granted. 3.1 Checking for Administrative Shares in the Remote Host

Administrative shares are a special feature of Windows NT servers that allow access to local drives as “hidden” shared resources by default, but they are limited only to administrative accounts. And for security policies, sometimes administrative shares are disabled.

The Custom Command ThinAgents need to access the ADMIN$ share, which represents the Windows installation path on the remote machine (by default it is C:\Windows). To check if the administrative share is enabled, try to log on to the remote admin folder from the ThinkServer host using Windows Explorer.

Figure 2 – Accessing the ADMIN$ folder of a remote machine

Although administrative share may be disabled by default in your remote host, if enabled, you don’t really need to do anything else other than having an administrator account. 3.2 Checking the Remote Service Manager Access in the Remote Host

The Service Manager of the remote host needs to be accessed from the ThinkServer host. To check if the remote Service Manager is accessible, just open your local service manager from the ThinkServer host (you can do this by running the services.msc command), then right click on the services tree and select “connect to another computer”:

Figure 3 – Attempting to connect to a remote machine

After entering the credentials, you should be able to see the services tree of the remote machine.

Figure 4 – Connected to the Service Manager of a remote machine

Chapter 4 4 Linux/Unix-based Host Requirements

When the desired remote host is a UNIX based platform, configuration is limited to some standard requirements. In fact, most UNIX systems will already have these services enabled and running. So normally you won’t have to do anything at the remote host side.

The connections engine uses the SSH (Secure Shell) protocol version 2 to communicate by default. Protocol 1 is NOT supported for security reasons (it’s a vulnerable protocol). It’s also possible to use a TELNET connector, but it’s only recommended for legacy hosts that don’t implement SSH.

You may check for SSH service availability by using a Windows SSH client from the ThinkServer host. In the following section we will be using the (free) Putty client to verify the service (you can find Putty at http://www.chiark.greenend.org.uk/~sgtatham/putty/ ). 4.1 Checking a SSHv2 Connection Availability with the Putty Client

To verify that you have full access to the server using SSH version 2, you’ll need to configure Putty first: In the Category field, click Session. Then enter the IP address of the remote host, and select the Connection Type SSH.

Figure 5 – Using Putty to check connection to a remote host via SSH

Then, under Connection, expand SSH, and select the Preferred SSH protocol version 2 only.

Figure 6 – Setting SSHv2 as the protocol to connect to a remote host

Then click Open to open the connection. If the protocol is available, you will be prompted for your credentials. The Custom Command ThinAgents support three types of authentication methods:

• Password authentication (the most common method).

• Keyboard interactive authentication.

• Private-key authentication (the most secure method).

In this server, we have used Keyboard interactive authentication over SSHv2.

Once you’ve connected to the server with these settings, you can then verify the commands you want to run with the Custom Command ThinAgents. 4.1.1 Checking Command Permissions and Availability

It is important to use the same user account that you plan to use with the Custom Command ThinAgents because you’ll want to check if you have the required permissions.

In this example, we use the netstat ‐lt command to list all TCP connections that are in LISTEN state.

Figure 7 – Determining that a Custom Command ThinAgents user has command permissions

As you can see in this example, we have checked out the user permissions, and we’re ready to use this command from the Custom Command ThinAgents.

Chapter 5 5 Configuration of Data Sources

This chapter describes the configuration of the three data sources: the Custom Command, the Advanced Custom Command, and the Script Runner data sources.

All data sources share the General Settings tab configuration, as connectivity works in exactly the same way, and all data sources also share the same Command output settings. But more specific options will remain separate in each data source. 5.1 General Settings

The general settings tab is used to configure the different connection options to the target host; here you can set the host IP, user credentials, and some Linux/UNIX-specific connection settings. The General Settings tab has five sections:

• Main information

• Target host settings

• Connection Settings

• Linux/UNIX-specific settings

• General Settings

Figure 8 – Entering a target remote host on a Windows domain 5.1.1 Main Information

By default, the names “Custom Command DataSource,” “Advanced Custom Command DataSource,” and “Script Runner DataSource” are used for the data sources, but you can change the names here to suit your monitoring needs and add a more detailed description of the data sources if required.

Configuration variables and default Description settings

Custom Command Datasource Use the default provided or enter a Advanced Custom new name for the data source. Name Command Data- Tip: Add the host name you are moni- source toring to help quickly identify where problems occur. Script Runner Data- source

Description Enter a description of the data source

5.1.2 Target Host Settings

In this group of options, you have to define the type of platform you’re connecting to, and the user credentials for the authentication.

In the Platform option, define the type of platform your target host is. At this moment, Windows and Linux/UNIX are supported. Each connection type will use a different technology—Windows uses the

SMB and RPC protocols and Linux/UNIX uses SSHv2 or TELNET. These technologies are completely agentless (although Windows automatically installs a service on the remote host). If you select Linux/ UNIX, please take a look at section 5.1.4 - Linux/Unix Specific Settings on page 13 to define some platform-specific connection options.

The remaining options are common to most of ThinkServer’s ThinAgents, and are well explained in the following table.

Configuration variables and default Description settings

Select the target host platform type. Valid Platform Windows options are Windows or Linux/Unix.

Enter the IP Address or use the DNS IP / DNS Name name of the host.

The port of the remote service, depending on the platform you’re connecting to. Port *DEFAULT The *DEFAULT parameter makes the ThinAgent use the default port for the type of service you’re using.

Host domain (only valid for Windows Domain platforms).

User User to be used to connect to host.

Password of the user connecting to the Password host. 5.1.3 Connection Settings

For all platforms, you can define how connections will be treated and managed from this data source.

Dedicated session The Dedicated session option allows you to make an independent connection for each data source, even if multiple data sources share the same host and credentials. This is good for performance reasons; if you are working with high volumes of data, then we recommend that you use dedicated sessions only.

The only downside of this method is that you will have one logged-in user in the remote host for every connected data source. So if you’re working with poor volumes of data and you have multiple data sources working with the same host, then you should disable this option.

Keep in mind that a shared session will only work when you have the exact hostname, user name, port and password associated to the data source.

Keep connection open If this option is enabled, the data source will connect to the remote host the first time, and then remain connected during the entire lifetime of the data source. This is useful for monitors with short refresh times, as it avoids authenticating at each refresh. If this option is not enabled, then the data source will

connect to the remote host before each refresh, and disconnect from the remote host after each refresh; this is useful for monitors with long refresh times.

Configuration variables and default Description settings

Use a dedicated socket or share the con- Dedicated session true nection with other data sources.

Keep the connection open between Keep connection true refreshes, or open/close the connection open at every refresh. 5.1.4 Linux/Unix Specific Settings

If you select Linux/UNIX specific settings in the Platform options of the Target Host settings group, then you may want to define some more advanced connection settings for the data source...

Connections Type The Connection Type option allows you to define the protocol you want to use to connect to the remote Linux/UNIX host. At this moment, SSHv2 and TELNET are supported. TELNET is provided as a compatibility option for legacy systems, although it’s not recommended because it’s not secure and it is very slow compared to SSH.

Private Key The Private Key option gives you the possibility to use the private key authentication method available in OpenSSH-based daemons. This kind of authentication uses a public and private key pair of files that are used when connecting to the server. You only need the private one, as the public is “deduced” in runtime by the connections engine. Once you have your private key stored in ThinkServer, put the full path to the file inside this option. If your private key also uses password protection, then put the password in the user Password box in the Target Host Settings group.

The private key must be a cryptographic RSA or DSA key that’s compatible with OpenSSH and associated with the remote host. There are several tools for generating and associating key pairs for authentication. The most common one is ssh-keygen, which should be available in your remote host. If you have a private key in .ppk format (from the popular tool Putty), you can export it to OpenSSH format by using the Putty Key Generator.

Remember that private key authentication is just an option, not a requirement for the Custom Command ThinAgents.

Configuration variables and default Description settings

Connection Type SSHv2 You have to select the connection driver.

The path of a SSH private key file Private Key (optional) 5.1.5 General Settings

The general settings group is the most common set of options in ThinkServer’s ThinAgents. Normally you won’t need to modify anything here, but feel free to take a look at the different options. Perhaps you may need to modify the Refresh Time according to your Custom Command monitor needs.

Configuration variables and default Description settings

Refresh Time 60 The data source’s refresh time

The amount of tries to perform when an Number of Tries 1 error occurs

Interval Between The time-delay between tries when an 10 Tries error occurs

The amount of time to wait until the next Error Retry Time 60 refresh after an error has been reported

The number of milliseconds you wish to Connection 5000 be considered as an internal connection Timeout timeout

Note The connection timeout is the most critical value here. We strongly recommend that you keep the default value (5000 milliseconds), and change it only if you are sure of potential outcomes.

5.2 Command Execution Settings (Custom Command Data Source)

This tab allows you to define some specific settings regarding command execution. That means, which command you want to run, how you want to run it, and some miscellaneous options, like logging and buffering.

Figure 9 – Defining the custom command we wish to be run on this data source

Command execution The command execution section allows you to define your command, the working directory, priority, and some other parameters related to remote execution. Normally you only have to define a command to get your monitor running; the remaining parameters are purely optional.

• Command: Define a valid command to be executed on the remote host. The command can be an absolute or relative executable program (if it’s relative, it will be relative to the Working Dir parameter), like “/bin/ls”, “C:\tools\runme.exe”, “netstat”, etc. You can also call the program with all the parameters that you want, like “ls –lh”. It is possible to define some variables (the definition of variable is explained in this chapter) to be replaced in the command, by putting a ‘$’ symbol in front of the variable.

• Working Dir: You can define the working directory for your program. For example, running the ls command with no parameters, will list the current working directory, normally the home of the user account. If you change the Working Dir with, let’s say, the root directory “/”, then the ls command without parameters will list the root directory. This parameter is optional.

• Priority: This option is only valid for Windows platforms. You can change the priority of the process created by your command by changing this value. If unsure, just use the NORMAL priority.

• Interactive execution: By default, all programs are executed in batch mode. That means a process is instantiated against two named pipes: one for standard output and one for standard errors. The process is executed and we wait until it finishes processing the output and exit code. This is the fastest and most stable method to use, except for those programs that don’t support named pipes; some programs just write output to the terminal device (tty), like some

versions of the top program, and some network devices with a very limited shell. In said cases, you can enable the Interactive execution option, and the output will be processed directly from the terminal device instead of the named pipes.

• Impersonate: By default on Windows platforms the remote process is created with the localsystem account, and it runs under the localsystem credentials. If you need to run the process with the same user account you’re using to connect to that server, then you should enable this option.

Enable variables Variables allow you to change some parts of your command directly from the attached monitors by using standard Python scripting.

To use variables, first you need to enable this option. Then you need to define the variables you want in the variables table, and finally, introduce those variables directly into the command, as shown in the previous figure.

Table structure:

VariableName InitialValue ReadOnly Persist

If enabled, the most recent If read only, then value will persist the monitors won’t A unique variable ID, on disk, and be be able to change with a pre-defined Defines the initial restored when this value. An name, starting with value for this vari- the monitor error message will CMDVAR01 up to able. restarts. If not, be shown if a CMDVAR20 the initial value monitor fails to will be used write the variable. when the monitor restarts.

Table explanation:

Variables are optional; you can use a variable to modifiy some parts of a command directly from the monitor’s health script, by changing the variable value directly in Python. A common use of this feature is to perform a sort of incremental search, where you need to update a timestamp from the monitor to the command.

• VariableName: You have up to 20 variables to define here. Valid names are predefined names with the following format: CMDVARXX, starting from CMDVAR01 up to CMDVAR20.

• InitialValue: Define an initial value for this variable. This value will be used the first time the data source runs, and every time the data source restarts if the Persist field is disabled.

• ReadOnly: If enabled, then this variable won’t be editable by the attached monitors. In order to change a variable’s value from the attached monitors, the variable can’t be in read-only mode. If a monitor attempts to write a read-only variable, it will send a warning message to the ThinkServer configurator.

• Persist: If this option is enabled, the changes made to a variable will persist on disk, which means that all changes will be remembered, even on monitor restarts (including the ThinkServer service restart). If this option isn’t enabled, then the InitialValue will be used on monitor restarts.

Buffer size You can define the size of the buffer the engine will use to retrieve data from the remote host. Buffers are used to request packets inside read loops. This is a low level configuration, and we strongly recommend that you use the Automatic option. Only change this option if you are sure of possible outcomes; take into account that, depending on your target platform, the driver you’re using to connect, and the amount of information you’re working with, optimal buffer sizes may vary. Hence, it’s always better to use the Automatic option.

Trace options Trace options are used to enable/disable internal engine logging. This is useful to detect connection errors, filtering problems, and events with low level warning messages you won’t see in ThinkServer. Although it might be useful to debug filtering tasks, we strongly advise you NOT to use this unless a Tango/04 technical consultant asks you to.

• The Enable tracing option enables/disables the internal logging of the data source and its attached monitors.

• The Overwrite at start option defines whether the log will be overwritten each time the data source is started (after it was stopped, or ThinkServer was restarted). Keep in mind that disabling this option will make the log file grow indefinitely.

• The Log level option defines the tracing level. Currently, the possible values are DEBUG, INFO, WARNING, and ERROR. The recommended value is WARNING.

Resulting trace files are stored inside the logs folder of the ThinkServer installation path.

Note Enabling the DEBUG option may produce enormous amount of data and consume a lot of computer resources in the ThinkServer’s host; please use it only if a Tango/04 consultant asks you to.

5.3 Command Execution Settings (Advanced Custom Command Data Source)

This tab allows you to define some specific settings regarding command execution. This means which commands you want to run, how you want to run them, and some miscellaneous options, like logging and buffering.

Figure 10 – You can define many commands to be run on the Advanced Custom Command data source

Command settings The command settings section allows you to define a list of commands, a working directory, a priority, and some other parameters related to the remote execution. Normally you only have to define a list of commands to get your monitor running. The remaining parameters are purely optional.

Table structure:

CommandValu Interacti CommandName ExecuteIf Impersonate e ve

Run the A unique Specifies if the com- command ID, with Define a spe- command will mand in Defines the a pre-defined cial condition be imperson- interac- command to be name, starting with for this com- ated or not tive mode executed. CMD01 up to mand to run. (Windows instead of CMD99 only). batch mode.

Table explanation:

The Advanced Custom Command ThinAgent allows you to run a set of commands by following some basic conditions. This table allows you to define those commands and conditions.

• CommandName: Define a unique command ID with a pre-defined name, starting with CMD01 up to CMD99. You need at least ONE command in order to get the monitor working.

• CommandValue: Define a valid command to be executed on the remote host. The command can be an absolute or relative executable program: if it’s relative, it will be relative to the Working Dir parameter), like “/bin/ls”, “C:\tools\runme.exe”, “netstat”, etc. You can also call the program with all the parameters that you want, like “ls –lh”. It is possible to define some variables (the definition of variable is explained in the variables chapter) to be replaced in the command, by putting a ‘$’ symbol in front of the variable.

• ExecuteIf: You can specify a condition to be met in order for this command to be run. The value Always is set by default, which means that this command will always be executed. You can deactivate the command execution by using the Never condition. You can decide to run this command only if the last command in the list returned OK with the LastSucceeded condition, or, inversely, running the command only if the last command in the list failed with the LastFailed condition.

• Impersonate: By default, on Windows platforms the remote process is created with the localsystem account, and it runs under the localsystem credentials. If you need to run the process with the same user account you’re using to connect to that server, then you should enable this option.

• Interactive: By default, all programs are executed in batch mode. That means a process is instantiated against two named pipes: one for standard output and one for standard errors. The process is executed and we wait until it finishes processing the output and exit code. This is the fastest and most stable method to use, except for those programs that don’t support named pipes; some programs just write output to the terminal device (tty), like some versions of the top program, and some network devices with a very limited shell. In said cases, you can enable the Interactive execution option, and the output will be processed directly from the terminal device instead of the named pipes.

• Priority: This option is only valid for Windows platforms. You can change the priority of the process created by your command by changing this value. If unsure, just use the NORMAL priority.

Enable variables Variables allow you to change some parts of your command directly from the attached monitors by using the standard Python script.

To use variables, first you need to enable this option. Then, you need to define the variables you want in the variables table, and finally, introduce those variables directly into the command, as shown in the previous figure.

Table structure:

VariableName InitialValue ReadOnly Persist

If enabled, the If read-only, then last value will the monitors won’t persist on disk, A unique variable ID, be able to change and be restored with a pre-defined Defines the initial this value. An when the moni- name, starting with value for this vari- error message will tor restarts. If CMDVAR01 up to able. be shown if a not, the initial CMDVAR20 monitor fails to value will be write the variable. used when the monitor restarts.

• The Enable tracing option enables/disables the internal logging of the data source and its attached monitors.

• The Log level option defines the tracing level. Currently, the possible values are DEBUG, INFO, WARNING, and ERROR. The recommended value is WARNING.

Resulting trace files are stored inside the logs folder of the ThinkServer installation path.

Note Enabling the DEBUG option may produce enormous amount of data, and consume a lot of computer resources in the ThinkServer’s host; please use it only if a Tango/04 consultant asks you to.

5.4 Command Execution Settings (Script Runner Data Source)

This tab allows you to define some specific settings regarding the script execution, which means what script you want to run, how you want to run it, and some miscellaneous options, like logging and buffering.

Figure 11 – Defining the path of a desired script to be run on the Script Runner data source, and entering and enabling the variables involved with the script

Script execution The script execution section allows you to define your script, working directory, priority, and some other parameters related to the remote execution. Normally you only have to define a script to get your monitor running, the remaining parameters are purely optional.

• Script path: Specify a local path to the script you want to execute on the remote host. The language of the script is not important here (you will define that in the parameter Interpreter ).

• Keep on remote host: The script you defined in the Script path parameter will be sent to the remote host before executing it. By default, the same script will be removed after its execution is complete. You can change this behavior if you enable this option. Keeping your script on the remote host may increase performance between refreshes but decrease security, as anyone could modify the script on the remote host.

• Arguments: If your script uses arguments at runtime, you can define those arguments directly here. It is possible to use some variables (the definition of variable is explained in this chapter) to be replaced in the argument’s parameter by putting a ‘$’ symbol in front of the variable.

• Interpreter: By default, the interpreter is the platform-specific default one. In Windows, it’s cmd.exe, in Linux, it’s sh. These interpreters will be called if you leave the *DEFAULT

parameter. If your script uses a different language, like Python or PowerShell, then you can change the default calling by writing your custom interpreter here. For example, on Windows using Python, C:\Python27\Python.exe will call the Python interpreter (assumed to be installed on the remote host); your script would have to be compatible with Python, of course. If you need to call the interpreter with some special parameters, this is the right place to put those parameters, for example: PowerShell ‐ExecutionPolicy Unrestricted ‐File.

• Priority: This option is only valid for Windows platforms. You can change the priority of the process created by your command by changing this value. If unsure, just use the NORMAL priority..

Enable variables Variables allow you to change some parts of your script’s arguments directly from the attached monitors by using the standard Python script.

Table structure:

VariableName InitialValue ReadOnly Persist

Table explanation:

• VariableName: You have up to 20 variables to define here. Valid names are predefined names with the following format: CMDVARXX, starting from CMDVAR01 up to CMDVAR20.

• InitialValue: Define an initial value for this variable. This value will be used the first time the data source runs, and every time the data source restarts if the Persist field is disabled.

• Persist: If this option is enabled, the changes made to a variable will persist on disk, which means that all changes will be remembered, even on monitor restarts (including the

ThinkServer service restart). If this option isn’t enabled, then the InitialValue will be used on monitor restarts.

Trace options Trace options are used to enable/disable internal engine logging. This is useful to detect connection errors, filtering problems, and event low level warning messages you won’t see in ThinkServer. Although it might be useful to debug filtering tasks, we strongly advise you NOT to use this at least a Tango/04 technical consultant asks you to.

• The Enable tracing option enables/disables the internal logging of the data source and its attached monitors.

• The Log level option defines the tracing level. Currently, the possible values are DEBUG, INFO, WARNING, and ERROR. The recommended value is WARNING.

Resulting trace files are stored inside the logs folder of the ThinkServer installation path.

Note Enabling the DEBUG option may produce enormous amount of data, and consume a lot of computer resources in the ThinkServer’s host; please use it only if a Tango/04 consultant asks you to.

5.5 Variables (Advanced Custom Command Data Source)

The variables tab of the Advanced Custom Command data source is equal to the Variable tables of the Custom Command and Script Runner data sources. The only difference is that it increases the number of available variables to 50. You can use any of these variables on any of the 99 possible commands.

Figure 12 – Defining variables that are to be used in a command run on the Advanced Custom Command data source

Enable variables Variables allow you to change some parts of your script’s arguments directly from the attached monitors by using standard Python scripting.

Table structure:

VariableName InitialValue ReadOnly Persist

If enabled, the If read-only, then last value will the monitors won’t persist on disk, A unique variable ID, be able to change and be restored with a pre-defined Defines the initial this value. An when the moni- name, starting with value for this vari- error message will tor restarts. If CMDVAR01 up to able. be shown if a not, the initial CMDVAR50 monitor fails to value will be write the variable. used when the monitor restarts.

Table explanation:

Variables are optional. You can use a variable to modifiy some parts of a command directly from the monitor’s health script by changing the variable’s value directly in Python. A common use of this feature

is to perform a sort of incremental search, where you need to update a timestamp from the monitor to the command.

• VariableName: You have up to 50 variables to define here. Valid names are predefined names with the following format: CMDVARXX, starting from CMDVAR01 up to CMDVAR50.

• InitialValue: Define an initial value for this variable. This value will be used the first time the data source runs, and every time the data source restarts if the Persist field is disabled.

The command output settings tab allows you to define some low level command output filters and to publish multiple line information as one event for the attached monitors.

The most interesting control here is the table of filters. This table allows you to add as many filters as you want, and even disable them without the need to delete.

Figure 13 – Filtering command output

Enable filters table This option just enables/disables the filters table. If disabled, any filter you’ve defined in the table won’t be processed at all. This is useful when testing filter sets.

Filters table This table allows you to define a set of filters to include/exclude the data retrieved by your commands. Also, you can enable the multiline settings to format multiple lines into single events to be published to the attached monitors. If the table is disabled or if no filter has been defined, then all events will be published to the attached monitors.

Table structure:

RegularExpression FilterType Multiline Enabled

A perl-compatible Defines if the reg- regular expression to ular expression is If enabled, treat this Enables/dis- match the incoming an Inclusion or filter as multiline. ables the filter. data. Exclusion filter.

Table explanation:

RegularExpression: This is actually the filter you want to create. You just have to create a perl- compatible regular expression, and ensure that it matches only the data you want.

FilterType: Define wheter the lines matching this filter will be included or excluded. Remember that when you add an exclusion filter, ALL the lines that don’t match that filter will be included, and when you add at least one Inclusion filter, it means that ALL the lines that don’t match the filter will be excluded. You can set as many inclusion/exclusion filters as you want. Priority is always given to the exclusion filters (if one line matches both exclusion and inclusion filters, then it will be excluded).

Multiline: Enable this option to process the data in multiline mode. When multiline is enabled, the regular expression filters will be evaluated against blocks of data instead of lines. This changes the logic of filtering a little bit, because instead of allowing or discarding lines, it will extract pieces of data from the main text each time it finds a match.

Example

You want to match these three lines in a single event:

‐‐Access violation‐‐ Application: someapp.exe Exception Details: Index out of range.

You may write an inclusion filter like this:

‐‐Access violation‐‐[\r\n]*Application:.*[\r\n]*Exception Details:.*[\r\n]*

Enable the multiline option (otherwise it won’t work at all). All attached monitors will receive a unique event each time this pattern matches those three lines in a file. You can create as many multiline filters as you want; this means that multiple stanza definitions are supported when reading multiline based logs.

Once again, remember that enabling at least one multiline filter for a file will disable exclusion filters for that file.

Enabled: You can enable/disable the filter at any time. This allows you to test for regular expressions’ effectiveness when you’re still configuring your monitor. It means that you can switch each one of the filters defined in this table to enabled or disabled at any time. Just remember to enable it again when you’ve finished performing your tests!

Regular expression settings This group of options allows you to define how the regular expressions of the filters you’ve defined in the table will be processed. This means that any option you set here will affect ALL the filters defined in the table, including the file patterns.

Match exact pattern: This option basically means that the incoming lines have to exactly match the same regular expression you’ve defined in the filters (from start to end). If the matching pattern is part of a bigger string, then it won’t match the filter; to change this, just disable this option.

Case sensitive: This option just makes the patterns you’ve defined case sensitive. Disabling this will ignore the case when matching, but take into consideration that this may not always be the best option, especially when dealing with case-sensitive platforms (almost all UNIX based systems are case sensitive).

Ungreedy: By default, pattern matching in almost any regular expressions engine is greedy, which means that the matcher returns the longest match possible. For example, applying the pattern A.*c to AbcAbcA matches AbcAbc rather than the shorter Abc.

This option inverts the "greediness" of the quantifiers so that they are not greedy by default, but become greedy if followed by a "?". It is not compatible with Perl regular expressions. It can also be set by a (?U) option setting within the pattern.

Dot matches all: With this option enabled, a dot metacharacter in the pattern matches a character of any value, including one that indicates a new line. However, it only ever matches one character, even if new lines are coded as CRLF. Without enabling this option, a dot does not match when the current position is at a new line. This option is equivalent to Perl's /s option, and it can be changed within a pattern by a (?s) setting. A negative class such as [^a] always matches new line characters, independent of the setting of this option.

Multiline settings This group of options is only valid when you have defined at least one filter as multiline. Not having any multiline filters invalidates all of these parameters.

Split one event per line: By default, all Custom Command derived monitors will send a unique event with the entire command output data, even if processing multiline filters. If you enable this option, the output will be split in lines, and each line will be published as a standalone event. If you’re working with multiline filters, then this option will send one event per matching inclusion filter.

Unbuffered processing: This option allows for the retrieval of the entire command output before processing any filters. Use this option ONLY when the content of a matching event is so big that one match may exceed the buffer size defined in the buffer configuration (normally 64 KB). If not sure, just keep it disabled, as enabling it may considerably reduce performance.

Warning This feature may be very expensive in terms of performance and memory usage. Use it only for small command outputs.

Join text after exclusion: In multiline mode, exclusion filters are processed first, and inclusion filters are processed at the end, exactly like in single-line mode. But in multiline mode, it’s a little more complex. If you have a multiline filter that matches a complex paragraph, when that paragraph may be contaminated by some portions of undesirable text (a log header for example), you may define an exclusion filter matching the undesirable text, and then enable this option. This option causes exclusion filters to be processed first, and then joins the portions of resulting text as a new text, and finally processes the inclusion filters against the new text.

Example

You have the following multiline command output:

[printer message: The tonner level is low, please contact your administrator] ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ New ticket: 15678 Name: John Smith Date: 20120131 Time: 10:30:00 ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ New ticket: 15679 Name: Pierre Joffre [printer message: The tonner level is low, please contact your administrator] Date: 20120131 Time: 10:30:22 ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

As you can see, the multiline paragraph is divided by a disturbing printer message. So we can use this regular expression as an inclusion filter:

 New ticket:[^\r\n]+[\r\n]+Name:[^\r\n]+[\r\n]+Date:[^\r\n]+[\r\n]+Time:[^\ r\n]+[\r\n]+ .

and this regular expression as an exclusion filter:

\[printer message:.*.

Then, select Join text after exclusion, and all printer messages will be removed before processing the inclusion filters’ regular expressions.

Chapter 6 6 Custom Command ThinAgent Configuration

The Custom Command ThinAgent is a powerful tool that lets you read and parse data from command output, exactly like files. The ThinAgent isn’t involved in command execution; it relies on all the work done on the data source, it only processes command output.

It includes a parser engine, an additional per-event filter table (apart from the lower-level table included in the data source), and a custom table that allows you to define names, per-field filters, and data types for every matched filter.

Once you have selected a data source, the Monitor Configuration window opens.

Monitor configuration consists of the following tabs:

• General settings

• Parser options

• Table settings 6.0.1 General Settings

The general settings tab allows you to set the name of the monitor, and optionally, a description.

Figure 14 – Naming a Custom Command Monitor

Configuration variables and default Description settings

Custom Command Enter a name for your Custom Com- Name Monitor mand Monitor

Enter a description for your Custom Description Command Monitor

Data source If you need to make any changes to the data source you can open do it in this tab. To select a desired data source click the Select Data Source button and select a data source form the list that appears. To edit the current data source click the Edit Data Source button. 6.0.2 Parser Options

The parser is the most relevant component of this monitor. It allows you to parse the incoming lines (or multiple lines) per event, and split them into fields. These fields may then be processed separately with more filters, or converted to other types of values (numeric, boolean, etc). The original line (or paragraph if working with multiple lines) will always remain untouched; all resulting fields will be stored in additional variables created especially for the parser.

Figure 15 – The Custom Command Monitor Configuration Parser options tab

Entry Format In the Entry Format section you define how to parse the command output. Remember that each event will be processed separately. The data arrives at the monitor exactly as it has been published by the data source. If multiline is enabled, then a unique event may have several lines. If not, then each event will be a line. The parser does take lines into account; it just splits the entire string according to your settings.

The different parsing methods available are:

• None: the parser is disabled. The lines are stored exactly as they arrive from the data source, and no field is extracted or converted.

• Delimited fields: in a character (or pattern) separated values log, an individual field can contain any number of characters; by definition, a field contains all the characters that precede a separator. This option allows you to cut the string each time it finds a defined separator. You may define only one separator, or a comma separated list of separators. If you define more than one separator, then you can also specify: whether the string is to be cut by following the order of the patterns, or that any of the patterns can be cut at any time. The separators may be a single char or a string pattern; there is also a special character list named with a special convention:

Separator Description

Defines a tabulator char.

\t Another way to define a tabulator char.

Separator Description

Defines a space char.

NOTE: It is also possible to use a literal space char (an empty char), but just in the middle of a separator list; you can’t use a literal space at the beginning or end of a separator list.

\s Another way to define a space char.

Defines a comma char, ‘,’. Because commas are used to separate multiple pattern definitions, defining a literal comma separator is needed. NOTE: If you define only one char separator and it is a comma, then you can use the normal comma char ‘,’.

Remember that all of these separators refer to a single char. If you want to define a double space separator, then you should use or \s\s as a separator pattern.

Example 1 (only one separator): We have the following line to be parsed:

“Peter is a consultant – Julia is a teacher – Marty is a lawyer.”

If you define the Delimited fields format with just one separator: the parser will then create the following fields:

Field01: “Peter is a consultant ”

Field03: “ Julia is a teacher ”

Field04: “ Marty is a lawyer.”

As you can see, spaces are kept by default, as they weren’t defined as separators. You can trim the resulting fields if you want.

Example 2 (multiple separators, not ordered): Now imagine we have a more complex string, and we have more than one possible separator, but those separators may be at any place in the string, in no special order. Our test string will be:

“Beautiful is better than ugly / Explicit is better than implicit ‐ Simple is better than complex / Complex is better than complicated”

But as you can see, we have more than one separator, and they are repeated everywhere, perhaps without a defined order, so we set the Delimited fields format like this:

‐,/

The parser will then create the following fields:

Field01: “Beautiful is better than ugly ”

Field03: “ Explicit is better than implicit ”

Field04: “ Simple is better than complex ”

Field04: “ Complex is better than complicated”

As you can see, we’ve defined two separators: “‐” and “/” and we defined no special order, which means that each time the parser finds one of those separators, it will cut the string into a new field.

Example 3 (multiple separators, ordered): Now imagine we want to cut the string by following a special order of separators, in this example we’ll also show you how to define a multi-char separator. Our test string will be:

“Logon event: Tom Parker – presales consultant ; result : success”

We then set the Delimited fields format like this:

:,‐,;

and we also select the Ordered matching check box. The parser will then create the following fields:

Field01: “Logon event”

Field03: “ Tom Parker ”

Field04: “ presales consultant ”

Field04: “ result : success”

As you can see, even by adding a “: “ to the separators list, the last field ignored it, because we have defined an ordered matching of the pattern, so only the first : has been split. Finally let’s modify our separator list to show how to use multi-char patterns:

:,‐,; result :

This redefines the delimiters, but it remains on three separators, the only difference is that the last one is a multi-char pattern: ;result, so the entire string will be used as a delimiter. The parser will then create the following fields:

Field01: “Logon event”

Field03: “ Tom Parker ”

Field04: “ presales consultant ”

Field04: “ success”

Multi-char separators are allowed in both ordered and non-ordered matching; you can even define a unique separator with multiple chars.

• Fixed width fields in a fixed-width length text log, fields are delimited by length rather than by a comma or other character separators. This option allows you to cut each string by a specific amount of characters. If only one value is provided, then it will cut the string every time that value has been reached. If more than one value is indicated (comma separated values), then it will cut the string by following that pattern, and making an entire field with the remaining string.

Example 1 (only one value): Imagine you’ve set the Fixed width fields option with only one value:

Then the monitor receives a new line with this value:

“This is a test for the File Reader Monitor”

The parser will then create the following fields:

Field01: “This is a ”

Field03: “test for t”

Field04: “he Log Rea”

Field05: “der Monito”

Field06: “r”

As you can see, the string splits until there are not enough characters, and the last field has the remaining chars.

Example 2 (comma separated values): We will use the same line we used in the previous example:

“This is a test for the File Reader Monitor”

But this time the Fixed width fields option is defined with this value:

10,5,2,8

The parser will then create the following fields:

Field01: “This is a “

Field03: “test ”

Field04: “fo”

Field05: “r the Lo”

Field06: “g Reader Monitor”

These examples don’t make much sense in practice, as nobody would want to split this string in this way, but it explains in an understandable way how it works.

• Regular expression: this is the most powerful parsing method available. It allows you to define a custom regular expression and define captured substrings with parentheses. All captured substrings will then be stored in the resulting fields, in the same order as they appear in your regular expression.

Example 1 (singe line): We want to parse the following line of the command output:

“[2011‐03‐10 08:25:32] ‐ username: tparker primary group: operators – result failure return code(‐10)”

We then define the Regular expression field:

\[([0‐9]{4}‐[0‐9]{2}‐[0‐9]{2}\s[0‐9]{2}:[0‐9]{2}:[0‐9]{2})\]\s‐ \susername:\s(\S+)\sprimary\sgroup:\s(\S+)\s‐ \sresult\s(\S+)\sreturn\scode$(‐?[0‐9]+)$

This regular expression using parentheses, captures the: timestamp; user name; primary group; result; and the return code. The parser will then create the following fields:

Field01: “2011‐03‐10 08:25:32”

Field03: “tparker”

Field04: “operators”

Field05: “failure”

Field06: “‐10”

As you can see, the approach here is not focused on separators, but just in the context of the grammar defined with Perl-compatible regular expressions.

Example 2 (multi line): If you selected a multiline filter in the data source, and the monitor receives multiline events, then you might want to figure out how to parse it with regular expressions. The parser does not actually take into account if the string is single or multiline (even with the delimited and fixed width fields options). So, in this example we will use the same lines we used in the multiline filter example of the data source:

‐‐Access violation‐‐ Application: someapp.exe Exception Details: Index out of range.

In the data source, we used this filter to retrieve the entire paragraph as a unified event:

‐‐Access violation‐‐[\r\n]*Application:.*[\r\n]*Exception Details:.*[\r\n]*

Now, for the parser, we will define this regular expression:

‐‐(Access violation)‐‐[\r\n]*Application:(.*)[\r\n]*Exception Details:(.*)[\r\n]*

The parser will then create the following fields:

Field01: “Access violation”

Field03: “ someapp.exe”

Field04: “ Index out of range.”

Regular expressions are very powerful for these kinds of operations, so you may practically parse any line you want with them. You may need to use a regular expression tool to debug your filters and parsing expressions before creating a monitor.

• XML Tags: This parser allows you to process an XML opening tag and its attributes. This parser is not intended to be a full XML reader, as it has some important limitations, but it is a useful tool for reading single XML elements with the corresponding set of attributes. An event will be published for each element found in command output. The condition for this parser to work is that the output format can't have more than one element per line. What this parser does: It sends an event with the element name, and one captured field per attribute found.  What this parser does NOT: It doesn't publish any information about child elements (all elements are treated as single tags), XML text, or comments. If you need to fully process XML output, please use the Generic XML ThinAgent.

Misc settings The Misc settings dialog box allows you to define some special settings for the parser:

• Header lines: define how many lines will be ignored by the monitor at the top of the file. This is useful for command output using headers with column names, or references. 0 means disabled. If you chose for example a value of 2, then only lines 3 and above will be processed by the monitor.

• Max process lines: defines a threshold for processing output. This allows you to avoid processing more than X lines per refresh; for example, a value of 10000 will ignore lines 10001 and above in a same refresh. This option is useful to avoid performance issues when command output grows more than it is expected to. 0 means disabled.

• Trim parsed fields: when enabled, blank spaces will be trimmed from the beginnings and endings of fields generated by the parser, as well as the original line.

• Ignore Empty lines: empty lines will NOT be reported when this option is enabled (we recommended enabling this option).

Line filters Line filters are a monitor level filter table, with the same features as the data source filters table. This table allows you to define some Perl-compatible regular expression inclusion and exclusion filters.

RegularExpression FilterType CaseSensitive ExactPattern Enabled

Defines if If enabled, the If enabled, the A Perl-compatible the regular lines will regular expres- regular expression expression match only if Enables/dis- sion will be to match the incom- is an Inclu- they are fully ables the fil- case sensitive, ing events (lines or sion or compliant ter. if not it will multi-lines). Exclusion with the regex ignore case. filter. pattern.

• RegularExpression: the actual filter you want to create. Create a Perl-compatible regular expression, and ensure that it matches only the data you want to.

• FilterType: select whether the lines (or multi-lines) matching this filter are included or excluded:  when you add an exclusion filter, ALL the lines that do NOT match that filter will be included. when you add at least one inclusion filter, it means that ALL the lines that do NOT match the filter will be excluded. You can set as many inclusion/exclusion filters as you want per command output. Priority always goes to the exclusion filters (if the same line matches both exclusion and inclusion filters, then it will be excluded).

• CaseSensitive: select this option to enable case comparisons in the regular expression

• ExactPattern: select this option to enable the strict matching of the exact pattern you’ve defined in the regular expression. If enabled the pattern will need to match the entire event.

• Enabled: select this check box to enable the filter. This allows you to test for regular expressions’ effectiveness while you are still configuring your monitor as you can enable or disable each of the filter lists defined in this table at any time. 6.0.3 Table Settings

After you have defined the parsing options, the fields are automatically generated. These generated fields have predefined names, starting with RecordFieldvalue01 to 20. Remember that the fields will ONLY be filled with data if you defined a parsing option, and the parser actually matches some valid data.

If you didn’t define a parser, then the event will be stored anyway, but the RecordFiledValueXX variables won’t be filled at all.

This table allows you to do some special things with the resulting fields generated by the parser:

• Define some more fine grain filtering, by defining inclusion/exclusion filters at a field level.

• Define a name for each field (useful for reports).

• Define the cast conversion type of the parsed data.

Note This table provides extended features which can be used with the parser, but it is an optional tool. You can, if you prefer, just use the parser and disable the table for most command output. Only use this table if you need the features it provides.

Figure 16 – The Custom Command Monitor Configuration Table settings tab

Enable table Select the Enable table check box to enable the use of the table at any time. By default it is disabled (don’t forget to enable it after defining your table entries).

Discard fields not defined in table If the parser generates more fields than the ones defined in the table, this option allows you to ignore those fields. Select the Discard fields not defined in table check box in order to not store fields not defined in the table with the record sent to SmartConsole. If disabled, those fields will be kept, but not processed by the table (no filtering, and no cast conversion).

Autodetect field names If you're using the Regular expression or XML Tags parsers, then you can automatically set the field names from the parser itself instead of defining it on the table. The Regular expression parser needs to use named, captured groups in order to correctly replace the fields defined in the table. The XML Tags parser uses the attribute name as the name of the field. Remember that the detected field names will be used instead of those defined in the table (if any), but the item is still valid in the table for filtering and type conversion.

Table

You can add up to 20 entries to the table; each one is one of the predefined RecordFieldValueXX variables.

FieldName Description Filter FilterType DataType Defines if Defines a Set the name of A Perl compat- the regular type cast One of the pre- the field, it will be ible regular expression conver- defined Record- stored in the corre- expression is an Inclu- sion to be FieldValueXX sponding Record- defining a filter sion or made with variables. FieldNameXX for this field. Exclusion the parsed filter. values.

• FieldName: the field name must be one of the predefined RecordFieldValueXX variables; you should make sure this variable is generated in the incoming events. If you define less fields than the number of generated fields, then the remaining fields won’t be processed by the table.

• Description: the description you set here will provide the field name. Field names are stored in the corresponding variables RecordFieldValueXX, where XX is the number of field. This is an optional value, useful for reports.

• Filter: a Perl-compatible regular expression defining the Inclusion/exclusion filter to be matched with this field only (not the line). Useful for performing fine grain filtering at field level.

• FilterType: select whether the lines (or multilines) matching this filter are included or excluded:  when you add an exclusion filter, ALL the lines that do NOT match that filter will be included. when you add at least one inclusion filter, it means that ALL the lines that do NOT match the filter will be excluded. You can set as many inclusion/exclusion filters as you want for this ThinAgent. Priority is always given to the exclusion filters (if the same line matches both exclusion and inclusion filters, then it will be excluded).

• DataType: you can execute a type cast conversion of the parsed data. This allows you to use the resulting fields with their native types. For example, an Integer data type may be used to evaluate arithmetic conditions, such as RecordFieldValue02 >= 23. Valid data types are: Integer Real Boolean DateTime (needs to be defined) Date (needs to be defined) Time (needs to be defined) String By default, all fields are created as String. Keep in mind that if an error happens when attempting to convert a data type, then the field will remain as a String type, but a warning message will be logged in the trace log (if enabled).

Date, Time, and DateTime format When selecting at least one Date, Time, or DateTime fields in the table, you need to define how the timestamp is formatted. This option allows you to define the date and/or time format used in the parsed

field. For this, we use POSIX conventions; the following list is a reference of possible formats (defined by the C language standard):

Directive Meaning

%a Locale’s abbreviated weekday name.

%A Locale’s full weekday name.

%b Locale’s abbreviated month name.

%B Locale’s full month name.

%c Locale’s appropriate date and time representation.

%d Day of the month as a decimal number [01,31].

Microsecond as a decimal number [0,999999], zero-padded on the %f left

%H Hour (24-hour clock) as a decimal number [00,23].

%I Hour (12-hour clock) as a decimal number [01,12].

%j Day of the year as a decimal number [001,366].

%m Month as a decimal number [01,12].

%M Minute as a decimal number [00,59].

%p Locale’s equivalent of either AM or PM.

%S Second as a decimal number [00,61].

Week number of the year (Sunday as the first day of the week) as a %U decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.

%w Weekday as a decimal number [0(Sunday),6].

Week number of the year (Monday as the first day of the week) as a %W decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0.

%x Locale’s appropriate date representation.

%X Locale’s appropriate time representation.

%y Year without century as a decimal number [00,99].

%Y Year with century as a decimal number.

UTC offset in the form +HHMM or -HHMM (empty string if the object %z is naive).

%Z Time zone name (empty string if the object is naive).

%% A literal '%' character.

For example, to match a date time field with this format:

date 2009‐03‐12 time 08:22:10

you will need to use this format for the DateTime data type:

date %Y‐%m‐%d time %H:%M:%S 6.0.4 Default Health Settings

The default Health script detects whether the monitor was able to retrieve data form the data source. By default Health is set to:

• Critical if ExitCode <> 0 and isn’t CommandOutput.

• Warning if ExitCode <> 0 and len(ErrorOutput) > 0.

• Success in all other cases.

Change the default health rules to meet your monitoring needs. As you see, Health is set to success by default because the Custom Command ThinAgent is very generic, so it is impossible to know when an event will be critical or not.

Normally you will want to set the criticalness with your parsed fields, as shown here:

RecordFieldValue02 == “failure” 6.0.5 Default Message Templates

By default, the monitor always sends the raw line (or multilines) that the data source published. These lines are never modified; the parser uses it as read-only to generate the RecordFieldValueXX variables. Feel free to modify this template according to your needs. 6.0.6 Common Variables

The variables list of the Custom Command ThinAgent.

Variable Description

Command The executed command.

CommandOutput The output of the executed command.

ErrorOutput The error output of the executed command.

ExitCode The program exit code.

LineNumber The line position in the raw output, this option is only valid when the Split one event per line option is enabled.

Host The host where the command is executed.

NumberOfFields The number of fields captured for the line.

RecordFieldValu A custom field value obtained by the parser. The e01...60 fields are labeled as: RecordFieldValue01, RecordFieldValue02, RecordFieldValue20...

RecordFieldName A custom field name defined in the parser table. The 01...60 fields are labeled as: RecordFieldName01, RecordFieldName02, RecordFieldName20...

RecordFieldType A custom field data type defined in the parser table. 01...60 The fields are labeled as: RecordFieldType01, RecordFieldType02, RecordFieldType20...

CMDVAR01...20 A custom variable used to modify the command from the attached monitor’s script. 6.0.7 Field Map SmartConsole – ThinkServer

ThinkServer sends a message to SmartConsole every time the monitor is run. By default it sends the following variables:

SmartConsole ThinkServer Description

The host where the command is exe- Var02 Host cuted.

Var03 Command The executed command.

Var04 CommandOutput The output of the executed command.

The error output of the executed com- Var05 ErrorOutput mand.

Var06 ExitCode The program exit code.

The field name for the RecordField‐ Var07 RecordFieldName01 Value variable.

A custom field value obtained with the Var08 RecordFieldValue01 parser.

Var09 RecordFieldNameXX (*)

Var10 RecordFieldValueXX (*)

(*) The pattern repeats until field 60. Variables sent to SmartConsole are configured in the Event Variables tab of the Templates section, available in the Health and Actions wizard.

In the Event Variables tab you can configure a maximum of 99 variables.

Variable 1 cannot be changed. You can configure one variable in the fields Variable 2 through variable 7. In the Variables 8 field and more you can add many variables as a comma separated list. Each variable will be sent to SmartConsole in the order they appear in this list, as Var08, Var09…Var99.

Chapter 7 7 Advanced Custom Command ThinAgent Configuration

The Advanced Custom Command ThinAgent is a powerful tool that lets you read and parse data from command output, exactly like files. The ThinAgent isn’t involved in command execution. It relies on all the work done at the data source, and just processes command output.

Once you have selected a data source, the Monitor Configuration window opens.

Monitor configuration consists of the following tabs:

• General settings

• Parser options

• Table settings 7.0.1 General Settings

The general settings tab allows you to set the name of the monitor, subscribe to a specific command executed by the data source, and optionally, a description.

Figure 17 – Filtering for the command executed at the data source so output can be read

Configuration variables and default Description settings

Advanced Custom Enter a name for your Custom Com- Name Command Monitor mand Monitor

Enter a description for your Custom Description Command Monitor

A regular expression, in order to sub- Command filter CMD.* scribe the monitor to a specific command(s) executed by the data source.

Command filter As the Advanced Custom Command monitor allows you to execute several commands, you might want to process each command with a different monitor (for example, because each command output has a different format). This filter allows you to define a regular expression to subscribe to the command names you want to process with this monitor. By default, all names are processed with CMD.*.

Data source If you need to make any changes to the data source you can open do it in this tab. To select another data source click the Select Data Source button and select a data source form the list that appears. To edit the current data source click the Edit Data Source button. 7.0.2 Parser Options

All the parser options for this ThinAgent are exactly the same as those of the Custom Command ThinAgent. Please refer to section 6.0.2 - Parser Options on page 30 to read the parser documentation.

7.0.3 Table Settings

All the table options are exactly the same as those of the Custom Command ThinAgent. Please refer to section 6.0.3 - Table Settings on page 36 to read the table documentation. 7.0.4 Default Health Settings

The default Health script detects whether the monitor was able to retrieve data form the data source. By default Health is set to:

• Critical if ExitCode <> 0 and isn’t CommandOutput.

• Warning if ExitCode <> 0 and len(ErrorOutput) > 0.

• Success in all other cases

Change the default health rules to meet your monitoring needs. As you see, Health is set to success by default because the Advanced Custom Command ThinAgent is very generic, so it is impossible to know when an event will be critical or not.

Normally you will want to set the criticalness with your parsed fields, as shown here:

RecordFieldValue02 == “failure” 7.0.5 Default Message Templates

The variables list of the Advanced Custom Command ThinAgent.

Variable Description

Command The executed command.

CommandName The name of the command executed (CMDXX).

CommandOutput The output of the executed command.

ErrorOutput The error output of the executed command.

ExitCode The program exit code.

The line position in the raw output. This option is only LineNumber valid when the Split one event per line option is enabled.

Host The host where the command is executed.

NumberOfFields The number of fields captured for the line.

A custom field value obtained with the parser. The RecordFieldValue01..60 fields are labeled as: RecordFieldValue01, RecordFieldValue02, RecordFieldValue20...

A custom field name defined in the parser table. The RecordFieldName01..60 fields are labeled as: RecordFieldName01, RecordFieldName02, RecordFieldName20...

Variable Description

A custom field data type defined in the parser table. RecordFieldType01..60 The fields are labeled as: RecordFieldType01, RecordFieldType02, RecordFieldType20...

A custom variable used to modify the command from CMDVAR01..50 the attached monitor’s script. 7.0.7 Field Map SmartConsole – ThinkServer

ThinkServer sends a message to SmartConsole every time the monitor is run. By default it sends the following variables:

SmartConsole ThinkServer Description

The host where the command is exe- Var02 Host cuted.

The name of the command executed Var03 CommandName (CMDXX).

Var04 Command The executed command.

Var05 CommandOutput The output of the executed command.

The error output of the executed com- Var06 ErrorOutput mand.

Var07 ExitCode The program exit code.

The field name for the RecordFieldValue Var08 RecordFieldName01 variable.

A custom field value obtained with the Var09 RecordFieldValue01 parser.

Var10 RecordFieldNameXX (*)

Var11 RecordFieldValueXX (*)

(*) The pattern repeats until field 60. Variables that are sent to SmartConsole are configured in the Event Variables tab of the Templates section, available in the Health and Actions wizard.

In the Event Variables tab you can configure a maximum of 99 variables.

Variable 1 cannot be changed. You can configure one variable in the fields Variable 2 through Variable 7. In the field Variables 8 and more you can add many variables as a comma separated list. Each variable will be sent to SmartConsole in the order they appear in this list, as Var08, Var09…Var99.

Chapter 8 8 Script Runner ThinAgent Configuration

The Script Runner ThinAgent is a powerful tool that lets you read and parse data from scripts’ output, exactly like files. The ThinAgent isn’t involved with the actual script execution. It relies on all the work done at the data source, and just processes script output.

Once you have selected a data source, the Monitor Configuration window opens.

Monitor configuration consists of the following tabs:

• General settings

• Parser options

• Table settings 8.0.1 General Settings

The general settings tab allows you to set the name of the monitor, and optionally, a description.

Figure 18 – Creating a monitor to retrieve script output from our Script Runner data source

Configuration variables and default Description settings

Script Runner Moni- Enter a name for your Script Runner Name tor Monitor

Enter a description for your Script Run- Description ner Monitor

Data source If you need to make any changes to your data source you can do it in this tab. To select another data source click the Select Data Source button and select a data source form the list that appears. To edit the current data source click the Edit Data Source button. 8.0.2 Parser Options

All the parser options for this ThinAgent are exactly the same as those of the Custom Command ThinAgent. Please refer to section 6.0.2 - Parser Options on page 30 to read the parser documentation. 8.0.3 Table Settings

All the table options are exactly the same as those of the Custom Command ThinAgent. Please refer to section 6.0.3 - Table Settings on page 36 to read the table documentation. 8.0.4 Default Health Settings

The default Health script detects whether the monitor was able to retrieve data form the data source. By default Health is set to:

• Critical if ExitCode <> 0 and isn’t CommandOutput.

• Warning if ExitCode <> 0 and len(ErrorOutput) > 0

• Success in all other cases

Change the default health rules to meet your monitoring needs. As you see, Health is set to success by default because the Script Runner ThinAgent is very generic, so it is impossible to know when an event will be critical or not.

Normally you will want to set the criticalness with your parsed fields, as shown here:

RecordFieldValue02 == “failure” 8.0.5 Default Message Templates

By default, the monitor always sends the raw line (or multilines) that the data source published. This lines are never modified. The parser uses it as read-only to generate the RecordFieldValueXX variables. Feel free to modify this template according to your needs. 8.0.6 Common Variables

The variables list of the Script Runner ThinAgent:

Variable Description

The full command executed (interpreter + scriptpath Command + args).

LocalScript The local path of the script.

CommandOutput The output of the executed command.

ErrorOutput The error output of the executed command.

ExitCode The program exit code.

The line position in the raw output, this option is only LineNumber valid when the ‘Split one event per line’ option is enabled.

Host The host where the command is executed.

NumberOfFields The number of fields captured for the line.

A custom field value obtained with the parser. The RecordFieldValue01..60 fields are labeled as: RecordFieldValue01, RecordFieldValue02..20

A custom field name defined in the parser table. The RecordFieldName01..60 fields are labeled as: RecordFieldName01, RecordFieldName02..20

A custom field data type defined in the parser table. RecordFieldType01..60 The fields are labeled as: RecordFieldType01, RecordFieldType02..20

A custom variable used to modify the command from CMDVAR01..20 the attached monitors script. 8.0.7 Field Map SmartConsole – ThinkServer

ThinkServer sends a message to SmartConsole every time the monitor is run. By default it sends the following variables:

SmartConsole ThinkServer Description

The host where the command is exe- Var02 Host cuted.

Var03 LocalScript The local path of the script.

Var04 Command The executed command.

Var05 CommandOutput The output of the executed command.

The error output of the executed com- Var06 ErrorOutput mand.

Var07 ExitCode The program exit code.

The field name for the RecordField‐ Var08 RecordFieldName01 Value variable.

A custom field value obtained with the Var09 RecordFieldValue01 parser.

Var10 RecordFieldNameXX (*)

Var11 RecordFieldValueXX (*)

(*) The pattern repeats until field 60. Variables sent to the SmartConsole are configured in the Event Variables tab of the Templates section, available in the Health and Actions wizard.

In the Event Variables tab you can configure a maximum of 99 variables.

Variable 1 cannot be changed. You can configure one variable in the fields Variable 2 through variable 7. In the field Variables 8 and more you can add many variables as a comma separated list. Each variable will be sent to the SmartConsole, in the order they appear in this list, as Var08, Var09…Var99.

Appendix A Appendix A: Some Execution and Parsing Examples

This section will show you some examples of how to use the Custom Command ThinAgents in practice. You may have several options to execute and parse a single command. Here we’ll chose the best option we think applies for specific situations.

Each example will show you a different parsing method:

• Delimited Fields: Checking the network interface connections on Windows.

• Regular Expressions: Reading the Linux audit trail

• XML Tags: Reading the Solaris audit trail. A.1 Delimited Fields: Checking the Network Interface Connections on Windows.

In this example, we’ll check for the active connections for all the interfaces of a Windows host. We’ll use the netstat command without parameters. A.1.1 Checking the Requirements

We just need to open a command line interface on the remote Windows host, and execute the command so we can analyze its output. This will then help us configure the parser.

Figure 19 – This command line output at a remote host has four headers A.1.2 Configuring the Parser

Our command output is something like this:

Active Connections

Proto Local Address Foreign Address State

TCP 127.0.0.1:443 BATCAVE:2326 ESTABLISHED

TCP 127.0.0.1:443 BATCAVE:2337 ESTABLISHED

TCP 127.0.0.1:2326 BATCAVE:https ESTABLISHED

TCP 127.0.0.1:2327 BATCAVE:2328 ESTABLISHED

TCP 127.0.0.1:2328 BATCAVE:2327 ESTABLISHED

TCP 127.0.0.1:2337 BATCAVE:https ESTABLISHED

TCP 127.0.0.1:2342 BATCAVE:2343 ESTABLISHED

TCP 127.0.0.1:2343 BATCAVE:2342 ESTABLISHED

TCP 127.0.0.1:4952 BATCAVE:4953 ESTABLISHED

TCP 127.0.0.1:4953 BATCAVE:4952 ESTABLISHED

TCP 127.0.0.1:6228 BATCAVE:6245 ESTABLISHED

TCP 127.0.0.1:6245 BATCAVE:6228 ESTABLISHED

TCP 192.168.0.192:6778 development:3690 ESTABLISHED

TCP 192.168.0.192:6805 jgalak:50161 ESTABLISHED

TCP 192.168.1.164:2331 a23‐56‐227‐51:https CLOSE_WAIT

TCP 192.168.1.164:2869 HERNAN‐PC:53987 TIME_WAIT

TCP 192.168.1.164:5194 BCNSRV01:microsoft‐ds ESTABLISHED

TCP 192.168.1.164:6227 sn1msg3010817:msnp ESTABLISHED

TCP 192.168.1.164:6499 157.56.52.13:http ESTABLISHED

TCP 192.168.1.164:6521 213.146.189.203:http ESTABLISHED

TCP 192.168.1.164:6618 154:pptp ESTABLISHED

TCP 192.168.1.164:6791 lhr08s02‐in‐f22:https ESTABLISHED

TCP 192.168.1.164:6795 bru01m01‐in‐f125:5222 ESTABLISHED

TCP 192.168.1.164:6796 channel‐id‐12‐01‐snc7:https ESTABLISHED

TCP 192.168.1.164:6801 www‐15‐08‐prn1:https ESTABLISHED

TCP 192.168.1.164:6807 190‐201‐108‐110:http ESTABLISHED

TCP 192.168.1.164:6808 139.30.123.188:http ESTABLISHED

TCP 192.168.1.164:6814 h‐91‐126‐102‐200:10649 ESTABLISHED

TCP 192.168.1.164:6850 77‐240‐125‐115:http ESTABLISHED

TCP [::1]:2332 BATCAVE:8307 ESTABLISHED

TCP [::1]:2338 BATCAVE:8307 ESTABLISHED

TCP [::1]:8307 BATCAVE:2332 ESTABLISHED

TCP [::1]:8307 BATCAVE:2338 ESTABLISHED

As you can see, the command output is formatted with space-delimited fields. Before deciding to use this kind of parser, you should check that there is no field with a “literal” delimiter inside its value (in this case, we have to make sure no field includes a space inside). For this output, the proper delimiter will be a space character.

This will capture four groups into the corresponding monitor’s fields (RecordFieldValueXX variables):

Field Description

RecordFieldValue01 Proto

RecordFieldValue02 Local Address

RecordFieldValue03 Foreign Address

RecordFieldValue04 State

A.1.3 Creating the Monitor

Once we have verified all the connectivity and authorization requirements with the host and the parser has been defined, we can then create our monitor.

General settings First we have to create the data source, and set the IP address and user credentials to connect with the host; remember to define the platform as Windows.

Figure 20 – Configuring a data source for the netstat command

Command execution settings Then we set the command we want to execute: netstat. We won’t use variables in this example, so we disable the variables table.

Figure 21 – Entering the desired command in the Command execution settings tab

Command output settings We won’t define any filter here, but we will change the default behavior of the monitor. By default, the data source sends a unique event to the attached monitors with the entire command output. In this situation, we want to process each line as a different event so that we can later parse the fields for each connection. With this in mind, we have to select the option Split one event per line.

Figure 22 – Multiline output will be split by the monitor so that the user sees one event per line

Monitor general settings The monitor doesn’t need much work in this tab; we just change the name of the monitor to a more descriptive one.

Figure 23 – The Monitor Configuration window opens after saving data source configuration

Monitor parser options Here we have to configure the parser. We just have to define our space delimiter with the wildcard . Then, we enable Trim parsed fields to remove the trailing space characters. Also, we enable Ignore empty fields to remove all NULL fields created between the multiple spaces on the command output. And finally, we set header lines to 4 so that we can discard the text on the top of the command output.:

Active connections

Proto Local Address Foreign Address State

Figure 24 – Parsing command output

Table settings (optional step) Finally, we’ll define a table to set custom names for the fields.

Figure 25 – Events shown in SmartConsole will display information for the fields Protocol, Address, etc.

A.1.4 Done!

After we’ve got the monitor configured, we can then start monitoring the network connections of the host in real time. If one of the connection parameters is not OK, or you don’t have enough permission to work with this command, then you’ll see the error report immediately in the ThinkServer Configurator interface. Another way to query for some possible error messages related with wrong parser configurations is to see the log file generated for this monitor; the log file is stored inside the logs folder in the ThinkServer installation path.

Note  The trace must be enabled in the data source in order to have information logged.

Figure 26 – Verifying in ThinkServer Configurator that the monitor is producing events as expected

To see the events in SmartConsole, just create a new Business View, filter for agent code CMD, and check that the variables are registered with the separated fields you defined with the parser:

Figure 27 – Viewing command output as desired in SmartConsole

If we take a look into an event properties window, the variables are listed exactly as we wanted, by following the order: Field Name, Field Value.

Figure 28 – Variables are listed alongside their field values A.2 XML Tags: Reading the Solaris Audit Trail.

The Oracle Solaris operating system allows you to query the audit trails generated by the audit daemon in two formats: a line-by-line format expressed in tokens, and a higher level format printed as an XML document. A.2.1 Checking the Requirements

Solaris auditing won’t be covered here, as it is out of this topic. The only thing we need to know is that audit trails are queried by a command line tool named auditreduce, and the output needs to be converted and formatted by a higher level tool named praudit.

The praudit tool lets you get output in XML format by passing the –x parameter. One more thing is needed: The XML tags parser is limited to one element per line, so elements need to be listed in one line. This can be done with the option –l.

The final sample command will be something like this:

auditreduce –a ‐b ‐c | praudit –lsx

Where and are posix timestamps indicating the range of time you want to query, and are the actual categories you want to list (lo, ex, etc…).

Figure 29 – Reading Solaris audit output with the Putty tool A.2.2 Configuring the Parser

We only have to select XML Tags and the parser will do the rest: parsing the XML contents, naming the fields with the attributes, and publishing an event per token.

The only additional thing we need to do here is to remove those tokens we don’t need: the token, the token, and of course, the XML header. In the previous figure, the important tokens (one per line), are:

successful login

A.2.3 Creating the Monitor

Once we have our command defined, we’re ready to create the monitor. Bear in mind that a static query with a timestamp isn’t actually a monitoring solution (we should perform an incremental reading command, by using variables). We’re using a static command here just to explain how the XML tags parser works.

General settings

First, create the data source, setting the IP address and user credentials to be used to connect to the host; and remember to define the platform as Linux/Unix (Solaris is a Unix based OS).

Figure 30 – Choosing a Solaris data source

Command execution settings Then we set the command to be executed: auditreduce ‐a 20120416150000 ‐c lo | praudit – lsx. We might use variables to modify the timestamp query between refreshes, but for this example we’ll keep it static.

Figure 31 – Setting the command we wish to be run at the Solaris data source

Command output settings The most important thing here is to enable the option Split one event per line, as we want one record for each audit event.

To remove the unnecessary lines mentioned before (the and tokens), we might use this general table with an exclusion filter regular expression, but we’ll keep it simple for this example and just exclude them from the monitor table (higher level and easier to define).

Figure 32 – Adjusting the output of the command

Monitor general settings The monitor doesn’t need much work in this tab; we just change the name of the monitor to a more descriptive one.

Figure 33 – Naming a Custom Command monitor

Monitor parser options Here we have to configure our XML Tags parser. Nothing else is required.

Figure 34 – Command output will be shown in XML

Table settings Finally, we’ll define our table, but not define the field names, as they will be automatically filled by the parser. We’ll exclude the , and tags. We exclude these names in Field01, because this field is always the XML element name when working with the XML Tags parser.

Also, don’t forget to select the option Autodetect field names.

Figure 35 – Defining SmartConsole event fields for this monitor A.2.4 Done!

After we’ve got the monitor configured, we can then start reading the Solaris audit trails in real time. If one of the connection parameters is not OK, or you don’t have enough permission to run these audit commands, then you’ll see the error report immediately in the ThinkServer Configurator interface. Another way to query for some possible error messages related with wrong parser configurations is to see the log file generated for this monitor; the log file is stored inside the logs folder of the ThinkServer installation path.

Note The trace must be enabled in the data source in order to have information logged.

Figure 36 – Viewing messages as one event per token

To see the events in SmartConsole, just create a new Business View, filter for the agent code CMD, and check that the variables are registered with the separated fields you defined with the parser:

Figure 37 – Viewing our Solaris command’s output in SmartConsole

If we take a look into an event properties window, the variables are listed exactly as we wanted, by following the order: Field Name, Field Value.

Figure 38 – You can see each variable values in a single event

That’s all for the XML Tags parser. You can now try to implement a dynamic command to change the timestamp from monitors after each refresh is performed.

Appendix B Appendix B: Regular Expressions Syntax Reference

The following table shows a basic reference of regular expression syntax. You can view this reference online at http://www.regular-expressions.info. B.1 Characters

Character Description Example

All characters except for these listed special Any character characters match a single instance of them- except selves, and are literal characters, unless a matches a [\^$.|?*+( they're part of a valid regular expression ) token (e.g. the {n} quantifier).

\ (backslash) followed by A backslash suppresses the meaning of any of \+ matches + special characters. [\^$.|?*+( ){}

Matches the characters between \Q and \E, \Q+‐*/\E matches \Q...\E literally, suppressing the meaning of special +‐*/ characters.

\xFF where Matches the character with the specified \xA9 matches © FF are 2 ASCII/ANSI value, which depends on the when using the hexadecimal code page used. Can be used in character Latin-1 code page. digits classes.

Match an LF character, CR character and a \r\n matches a \n, \r and \t tab character respectively. Can be used in DOS/Windows character classes. CRLF line break.

Match a bell character (\x07), escape char- \a, \e, \f and acter (\x1B), form feed (\x0C) and vertical \v tab (\x0B), respectively. Can be used in character classes.

Match an ASCII character Control+A \cM\cJ matches a \cA through through Control+Z, equivalent to \x01 DOS/Windows through . Can be used in character \cZ \x1A CRLF line break. classes.

Character Description Example

Starts a character class. A character class matches a single character out of all the pos- sibilities offered by the character class. Inside a character class, different rules (opening [ apply. The rules in this section are only valid square inside character classes. The rules outside bracket) this section are not valid in character classes, except for a few character escapes that are indicated by "can be used inside character classes".

Any character except ^-]\ add that character to All characters except the listed special char- [abc] matches a, b the possible acters. or c matches for the character class.

\ (backslash) followed by A backslash suppresses the special mean- [\^\]] matches ^ any of these: ing of special characters. or ] ^-]\

- (hyphen) except Specifies a range of characters. (Specifies a [a‐zA‐Z0‐9] immediately hyphen if placed immediately after the open- matches any letter after the ing [). or digit opening [

^ (caret) Negates the character class, causing it to [^a‐d] matches x immediately match a single character not listed in the (any character after the character class. (Specifies a caret if placed except a, b, c or d) opening [ anywhere except after the opening [)

Shorthand character classes matching digits, word characters (letters, digits, and [\d\s] matches a \d, \w and \s underscores), and white space (spaces, character that is a tabs, and line breaks), respectively. Can be digit or white space used inside and outside character classes.

Negated versions of the above. Should be \D matches a char- \D, \W and \S used only outside character classes. (Can acter that is not a be used inside, but that is confusing.) digit

[\b\t] matches a Inside a character class, \b is a backspace [\b] backspace or tab character. character

Character Description Example

Matches any single character except line . matches x and break characters \r and \n. Most regex fla- . (dot) (almost) any other vors have an option to make the dot match character line break characters too. B.4 Anchors

Character Description Example

Matches at the start of the string the regex pattern is applied to. Matches a position ^. matches a in rather than a character. Most regex flavors abc\ndef. Also ^ (caret) have an option to make the caret match after matches d in multiline breaks (i.e. at the start of a line in a file) line mode. as well.

Matches at the end of the string the regex pattern is applied to. Matches a position rather than a character. Most regex flavors .$ matches f in have an option to make the dollar sign match abc\ndef. Also $ (dollar) before line breaks (i.e. at the end of a line in matches c in multi- a file) as well. Also matches before the very line mode. last line break if the string ends with a line break.

Matches at the start of the string the regex pattern is applied to. Matches a position \A. matches a in \A rather than a character. Never matches after abc line breaks.

Matches at the end of the string the regex pattern is applied to. Matches a position .\Z matches f in \Z rather than a character. Never matches abc\ndef before line breaks, except for the very last line break if the string ends with a line break.

Matches at the end of the string the regex pattern is applied to. Matches a position .\z matches f in \z rather than a character. Never matches abc\ndef before line breaks. B.5 Word Boundaries

Character Description Example

Matches at the position between a word character (anything matched by \w) and a non-word character (anything matched by .\b matches c in \b [^\w] or \W) as well as at the start and/or abc end of the string if the first and/or last characters in the string are word characters.

Matches at the position between two word characters (i.e the position between \w\w) \B.\B matches b in \B as well as at the position between two non- abc word characters (i.e. \W\W).

Character Description Example

Causes the regex engine to match either the abc|def|xyz part on the left side or the part on the right (pipe) matches abc, def or | side. Can be strung together into a series of xyz options.

The pipe has the lowest precedence of all abc(def|xyz) | (pipe) operators. Use grouping if you wish to alter- matches abcdef or nate only part of the regular expression. abcxyz B.7 Quantifiers

Character Description Example

Makes the preceding item optional. Greedy, (question abc? matches ab or ? so the optional item is included in the match mark) abc if possible.

Makes the preceding item optional. Lazy, so the optional item is excluded from the match, abc?? matches ab ?? if possible. This construct is often excluded or abc from documentation because of its limited use.

Repeats the previous item zero or more times. Greedy, so as many items as possible ".*" matches will be matched before trying permutations (star) "def" "ghi" in abc * with less matches of the preceding item, up "def" "ghi" jkl to the point where the preceding item is not matched at all.

Repeats the previous item zero or more times. Lazy, so the engine first attempts to ".*?" matches *? (lazy star) skip the previous item, before trying permu- "def" in abc "def" tations with ever increasing matches of the "ghi" jkl preceding item.

Repeats the previous item once or more. Greedy, so as many items as possible will be ".+" matches matched before trying permutations with less (plus) "def" "ghi" in abc + matches of the preceding item, up to the "def" "ghi" jkl point where the preceding item is matched only once.

Repeats the previous item once or more. Lazy, so the engine first matches the previ- ".+?" matches +? (lazy plus) ous item only once, before trying permuta- "def" in abc "def" tions with ever increasing matches of the "ghi" jkl preceding item.

{n} where n is Repeats the previous item exactly n times. a{3} matches aaa an integer >= 1

{n,m} where Repeats the previous item between n and m a{2,4} matches n >= 0 and m times. Greedy, so repeating m times is tried aaaa, aaa or aa >= n before reducing the repetition to n times.

Character Description Example

{n,m}? where Repeats the previous item between n and m a{2,4}? matches n >= 0 and m times. Lazy, so repeating n times is tried aa, aaa or aaaa >= n before increasing the repetition to m times.

Repeats the previous item at least n times. Greedy, so as many items as possible will be {n,} where n matched before trying permutations with less a{2,} matches >= 0 matches of the preceding item, up to the aaaaa in aaaaa point where the preceding item is matched only n times.

Repeats the previous item n or more times. Lazy, so the engine first matches the previ- where a{2,}? matches aa {n,}? ous item n times, before trying permutations n >= 0 in aaaaa with ever increasing matches of the preceding item.

Appendix C Appendix C: Reference

For documentation about regular expressions, the http://www.regular-expressions.info Web site is a very good start, with plain of tutorials and references.

For more information about the drivers used by the data sources, you can visit the sites of the different providers:

Perl www.perl.org

Python www.python.org

SMB: • Wikipedia’s definition: http://en.wikipedia.org/wiki/Server_Message_Block

• Microsoft MSDN: http://msdn.microsoft.com/en-us/library/aa365233%28v=vs.85%29.aspx

SSHv2: • Wikipedia’s definition: http://en.wikipedia.org/wiki/Secure_Shell#Version_2.x

• RFC publications by the IETF "secsh" working group document SSH-2 as a proposed Internet standard:

http://tools.ietf.org/html/rfc4250

http://tools.ietf.org/html/rfc4251

http://tools.ietf.org/html/rfc4252

http://tools.ietf.org/html/rfc4253

http://tools.ietf.org/html/rfc4254

http://tools.ietf.org/html/rfc4255

http://tools.ietf.org/html/rfc4256

http://tools.ietf.org/html/rfc4335

http://tools.ietf.org/html/rfc4344

http://tools.ietf.org/html/rfc4345

RPC: • Wikipedia’s definition: http://en.wikipedia.org/wiki/Remote_procedure_call

• Microsoft MSDN: http://msdn.microsoft.com/en-us/library/windows/desktop/ aa378651(v=vs.85).aspx

Appendix D Appendix D: Contacting Tango/04

North America EMEA

Tango/04 North America Tango/04 Computing Group S.L. PO BOX 3301 Avda. Meridiana 358, 5 A-B NH 03458 Peterborough  08027 Barcelona  USA Spain   Phone: 1-800-304-6872 / 603-924-7391 Phone: +34 93 274 0051 Fax: 858-428-2864 Fax: +34 93 345 1329 [email protected] [email protected] www.tango04.com www.tango04.com

Italy Sales Office in France

Tango/04 Italy Tango/04 France Viale Garibaldi 51/53 La Grande Arche 13100 Vercelli  Paroi Nord 15ème étage Italy 92044 Paris La Défense   France Phone: +39 0161 56922  Fax: +39 0161 259277 Phone: +33 01 40 90 34 49 [email protected] Fax: +33 01 40 90 31 01 www.tango04.it [email protected] www.tango04.fr

Sales Office in Switzerland Latin American Headquarters

Tango/04 Switzerland Barcelona/04 Computing Group SRL (Argentina) 18, Avenue Louis Casaï Avda. Federico Lacroze 2252, Piso 6 CH-1209 Genève 1426 Buenos Aires Capital Federal Switzerland Argentina   Phone: +41 (0)22 747 7866 Phone: +54 11 4774-0112 Fax: +41 (0)22 747 7999 Fax: +54 11 4773-9163 [email protected] [email protected] www.tango04.fr www.barcelona04.com

Barcelona/04 PERÚ Barcelona/04 Chile Centro Empresarial Real Nueva de Lyon 096 Oficina 702, Av. Víctor A. Belaúnde 147, Vía Principal 140 Providencia Edificio Real Seis, Piso 6 Santiago L 27 Lima Chile Perú   Phone: +56 2 234-0898 Phone: +51 1 211-2690 Fax: +56 2 2340865 Fax: +51 1 211-2526 [email protected] [email protected] www.barcelona04.com www.barcelona04.com

Tango/04 Computing Group is one of the leading developers of systems management and automation software. Tango/04 software helps companies maintain the operating health of all their business processes, improve service levels, increase productivity, and reduce costs through intelligent management of their IT infrastructure.

Founded in 1991 in Barcelona, Spain, Tango/04 is an IBM Business Partner and a key member of IBM's Autonomic Computing initiative. Tango/04 has more than a thousand customers who are served by over 35 authorized Business Partners around the world.

Alliances

Partnerships IBM Business Partner IBM Autonomic Computing Business Partner IBM PartnerWorld for Developers Advanced Membership IBM ISV Advantage Agreement IBM Early code release IBM Direct Technical Liaison Microsoft Developer Network Microsoft Early Code Release

Awards

Legal Notice

The information in this document was created using certain specific equipment and environments, and it is limited in application to those specific hardware and software products and version and releases levels.

Any references in this document regarding Tango/04 Computing Group products, software or services do not mean that Tango/04 Computing Group intends to make these available in all countries in which Tango/04 Computing Group operates. Any reference to a Tango/04 Computing Group product, software, or service may be used. Any functionally equivalent product that does not infringe any of Tango/04 Computing Group's intellectual property rights may be used instead of the Tango/04 Computing Group product, software or service

Tango/04 Computing Group may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents.

The information contained in this document has not been submitted to any formal Tango/04 Computing Group test and is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility, and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. Despite the fact that Tango/04 Computing Group could have reviewed each item for accurateness in a specific situation, there is no guarantee that the same or similar results will be obtained somewhere else. Customers attempting to adapt these techniques to their own environments do so at their own risk. Tango/04 Computing Group shall not be liable for any damages arising out of your use of the techniques depicted on this document, even if they have been advised of the possibility of such damages. This document could contain technical inaccuracies or typographical errors.

Any pointers in this publication to external web sites are provided for your convenience only and do not, in any manner, serve as an endorsement of these web sites.

The following terms are trademarks of the International Business Machines Corporation in the United States and/or other countries: iSeries, iSeriese, iSeries, i5, DB2, e (logo)®Server IBM ®, Operating System/400, OS/400, i5/OS.

Microsoft, SQL Server, Windows, Windows NT, Windows XP and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and/or other countries. UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group. Oracle is a registered trade mark of Oracle Corporation.

Other company, product, and service names may be trademarks or service marks of other companies.