Owl Monitoring System Manager Installation Manual

4. Adding Owl Sensors to an Existing Owl Installation

	4.1.		Create Directories and Files for Sensor's Data
	4.2.		Firewall Configuration
	4.3.		SSH Set-up
	4.4.		Configuration Settings
	4.4.1.		Provide Configuration Information to Sensor Administrator
	4.4.2.		Receive Configuration Information from Sensor Administrator
	4.5.		Test Transfer from Sensor
	4.6.		Wait for Sensor Data
	4.7.		Build Nagios Sensor Objects
	4.8.		Nagios Modifications
	4.8.1.		nagios.cfg
	4.8.2.		owl-hostgroups.cfg
	4.9.		Graphing Modifications
	4.10.		Restart Nagios

Whenever a new Owl sensor is added to those handled by an Owl manager, there are a number of actions that must take place on both the sensor and the manager. These actions should be followed consecutively within each section. However, there is some amount of necessary interleaving of actions. For example, a particular sensor action may be required before a particular manager action.

It is acceptable for the Owl sensor and Owl manager to be under completely different administrative control. Each host may even be owned by different commercial, educational, governmental, or other such entity. All that is required is that there be some initial coordination between administrators when the sensor is first configured, along with (probably) aperiodic support from time to time later.

The discussions on adding Owl sensors assumes that the sensor and manager are under different administrative control. Required actions will not change if they are under unified administrative control, but they may be easier to accomplish.

This section describes the various actions that must be performed on the Owl manager in order to configure a new sensor for it. This assumes the installation and configuration procedures detailed in section 2 have been completed.

4.1. Create Directories and Files for Sensor's Data

A set of directories and files must be created for a new sensor. These are the sensor directory, the sensor's data directory, the history directory, the data archive directory. The heartbeat file, if the sensor will be providing heartbeats, must also be created.

These files are best created by the owl-initsensor command, but they may also be created manually. If they are created manually, you must use the names and organization as given in the examples below. The locations of the directories may change (i.e., the /owl/data and /owl/archive portions), but that's it.

For example, when adding the new sensors helsinki and canberra, you might create the following sets of directories:

The archive directory must be writable by the manager's Owl user, but the others must be writable by the manager's Owl user and the manager's web server.

4.2. Firewall Configuration

The Owl manager and the Owl sensor communicate by way of SSH and HTTP. If the Owl manager is protected by a firewall (either on the manager host itself or by an enterprise-level firewall) then the firewall must be configured to allow data to be transferred between the manager and the sensor. The direction of this transfer (initiated by sensor or initiated by manager) depends on the Owl configuration. These firewall modifications are far beyond the scope of this document.

4.3. SSH Set-up

First, you must generate a new SSH key for the manager's Owl user.

If the manager will be pulling data from the sensor, then the you must provide the sensor administrator with your Owl user's public key. This key will allow owl-transfer-mgr to retrieve Owl sensor data. You must generate your key in compliance with particular key requirements (e.g., length and type) specified by the sensor administrator.

If the sensor will be transferring data to the manager, then the sensor administrator must provide you with the public key of their Owl sensor user. This key will allow owl-transfer to pass Owl sensor data to the your manager. You must add this key to your .ssh/authorized_keys file. The key must be generated with key characteristics (e.g., length and type) that are acceptable to you. See the example below.

The authorized_keys file restricts access to known sensor hosts and controls where each Owl sensor's data are stored. This file should only have one entry per sensor, and each sensor should have a unique data directory.

Example entries (with abbreviated keys):

    command="/usr/local/bin/rrsync /owl/data/sensor1" ssh-rsa AAAA...Qw== sensor@sensor1.example.com
    command="/usr/local/bin/rrsync /owl/data/sensor8" ssh-rsa AAAA...gM== sensor@sensor8.example.au

4.4. Configuration Settings

With Owl's capability of having either the sensor or the manager transfer sensor data to the manager, there are configuration settings that must be made to allow the transfer. These are described in the subsections below. Regardless of how your Owl installation handles data transfer, you should read both subsections to ensure you are making all the required settings.

It would be a good idea for you and the sensor administrator to agree on a name for the sensor. This isn't required, but it will probably make things easier for you both in the long run to refer to the sensor by the same name.

The sensor name can be very generic, such as sensor42, sensor-d, or owl-us-east. It can also be very specific, such as washdc-1600-penn-ave-nw or cheltenham-bldg4. The intent is to provide distinguishing information to the intended audience of the DNS monitoring data. You should use names that easily distinguish sensors and are acceptable to the manager and sensor administrators.

4.4.1. Provide Configuration Information to Sensor Administrator

If the Owl sensor will be transferring data to the manager, then you must provide the sensor administrator with an SSH user. All Owl sensors may use this single SSH user, and the data will be distinguished by the SSH key used to connect from the sensor.

The sensor will use the heartbeat URL to provide a heartbeat to the manager. If this is to be used, it must be set on the sensor regardless of who initiates sensor data transfer.

The data values will be added to the sensor's configuration file.

The following example data will allow the sensor to work with the Owl manager as expected. These data are used, in conjunction with the remote keyword, in the sensor's configuration file.

Configuration Field		Purpose		Example
ssh-user		user on Owl manager with which owl-transfer will connect via ssh		sensor-ottawa@owl-manager.example.com
heartbeat		URL to provide "heartbeat" data to manager		http://owl.example.com/cgi-bin/owl-sensor-heartbeat.cgi

4.4.2. Receive Configuration Information from Sensor Administrator

If the Owl manager will be pulling data from the sensor, then the sensor administrator must provide you with an SSH user. The owl-transfer-mgr program will use this SSH user to copy data from the sensor.

The following data will allow the manager to connect to the Owl sensor as expected. These data are used, in conjunction with the remote keyword, in the sensor's configuration file.

Configuration Field		Purpose		Example
ssh-user		user on Owl sensor with which owl-transfer-mgr will connect via ssh		sensor-meatcove@owl-sensor.example.com

4.5. Test Transfer from Sensor

At this point, you are ready to test your Owl manager. When the sensor administrator reaches section 4.5 in their installation instructions, both sensor and manager are ready to test the data transfer.

Coordinate with the sensor administrator to test data transfer. The sensor administrator must put a data file (of any sort, it doesn't have to be an Owl data file) in their data directory. If the manager will be transferring data, you must run owl-transfer-mgr to attempt to retrieve the test file. If the sensor will be transferring data, the sensor administrator must run owl-transfer to attempt to retrieve the test file. After the transfer command appears to have transferred the file without error, you must check that sensor's data directory on the manager to ensure the file has arrived as expected.

Once the test file successfully appears in the new sensor's data directory, the sensor is ready to start collecting data and transferring it to the manager. You must inform the sensor administrator of this, so they can start the Owl sensor daemons.

4.6. Wait for Sensor Data

The sensor should now be in the process of collecting DNS response data. Whichever host will be transferring data must have the appropriate transfer daemon executing. You must now wait for sensor data to show up in the sensor's data directory. The time this takes will depend on how frequently the data is transferred to the manager. This is set in the configuration file.

You will know when the sensor is transferring data because the new sensor's data directory will start holding files whose names reflect the queries you are expecting. You may proceed when you have found files for all the queries the sensor will be performing.

4.7. Build Nagios Sensor Objects

Once all the sensor's data directory contains files for all the queries expected from the new sensor, Nagios objects must be built for those queries. This may be done automatically using the owl-newsensor command. A file containing these Nagios objects must be created, and it will be added to the Nagios environment in section 4.8.1.

owl-newsensor can be used like this to create the Nagios objects:

    $ owl-newsensor -out sensor8.cfg /owl/data/sensor8/data

Passing owl-newsensor the -heartbeat option will cause an object to be created that will allow Nagios to display heartbeat information about the new sensor.

4.8. Nagios Modifications

Several Nagios configuration files must be modified to account for the new sensor. These changes will allow Nagios to start reporting current status of the new sensor as well as saving historical data to be used in graphing.

4.8.1. nagios.cfg

The following modifications must be made to the nagios.cfg file to prepare Nagios for monitoring Owl sensor data.

• Add the following entries to nagios.cfg:

      cfg_file=/owl/nagios/etc/objects/owl-contacts.cfg
      cfg_file=/owl/nagios/etc/objects/owl-hosts.cfg
      cfg_file=/owl/nagios/etc/objects/owl-commands.cfg
      cfg_file=/owl/nagios/etc/objects/owl-services.cfg
  

  These files contain basic Nagios object definitions used by the Owl sensor objects. They should follow all the other standard cfg_file lines.

• Once Nagios object configuration files have been created for your sensors (as described in section 4.7), nagios.cfg must be modified to include the new files. Add a new cfg_file entry for each of the sensor object files. These entries will look something like this:

      cfg_file=/owl/nagios/etc/objects/owl-sensor21.cfg
      cfg_file=/owl/nagios/etc/objects/sensor-london.cfg
  

  The sensor lines should follow all four of the cfg_file lines listed in the previous point.

• Add the following line after the sensor cfg_file lines described in the previous point.

      cfg_file=/owl/etc/objects/owl-hostgroups.cfg
  

  Modification of the owl-hostgroups.cfg file is described below.

• Find the service_perfdata_command line and change it to: "service_perfdata_command=service-perfdata-for-owl"

• Find the service_perfdata_file_processing_command line and change it to: "service_perfdata_file_processing_command=service-perfdata-for-owl"

Examples for the cfg_file modifications may be found in nagios.cfg-owl.mods in the Owl manager distribution.

4.8.2. owl-hostgroups.cfg

This file contains the "DNS Response Time Sensors" Nagios hostgroup object that lists the Owl sensor hosts. All your Owl sensors should be listed in the "DNS Response Time Sensors" object.

• Add the new sensor name to the members field in the "DNS Response Time Sensors" object. The names must be separated by commas.

You may add your own hostgroup objects to this file. Use the "DNS Response Time Sensors" object as a guide to create custom groups. This can be used, for example, to group all the sensors in a geographical region or those that are running a particular operating system. The hostgroup object allows you to group a set of hosts in a manner that makes sense for your purposes.

4.9. Graphing Modifications

The new sensor's data must be made available to the drraw.cgi script. To do this, the new sensor's data sources must be added to the %datadirs hash in the drraw.conf file. See 2.1.2.4. drraw.cgi for more details.

4.10. Restart Nagios

Nagios must be restarted in order for it to see your new sensor's objects.

Prior to the restart, verify that your object modifications won't cause a problem for Nagios. Execute this command (assuming you are in the directory containing the Nagios files):

    $ nagios -v nagios.cfg

Once the Nagios configuration file is validated and without problems, Nagios may be restarted:

    $ nagios stop
    $ nagios start

It will probably take several minutes for Nagios to check all its services. Clicking on the "Services" link in the left-hand sidebar will bring up the configured services and you can watch the status of your new sensor.