Using Scout

How Scout Works

• How do I use the Scout Agent?
• What are the Scout Agent system requirements?
• What impact does the Scout Agent have on performance?
• How does the Scout Agent execute plugins?
• Scout retrieves plugins via HTTP. What happens if the resource is no longer available?
• What happens if a new version of a plugin is released?
• How does Scout approach security?
• Is Scout open source?
• How long does it take for data to appear on my Scout account?

Using Cron

• How do I setup Scout to run using cron?
• How do I setup Scout to run using the system-wide crontab file?
• How do I setup Scout to run using my current user's crontab file?
• Scout doesn't run in cron. I'm seeing a "Permission denied" error.
• Scout doesn't run in cron. I'm seeing a "ruby: command not found" error.
• I'm still having problems running Scout through cron.

Using Launchd

• How do I setup Scout to run using launchd?
• Scout doesn't run in launchd. I'm not seeing any errors.

Security

• What is Scout's approach to security?
• How does Scout's architecture make it secure?
• Are there any additional security measures I need to take with Scout?

Troubleshooting

• How do I Solve 'scout: command not found'?
• Scout Prints the Error: 'no such file to load -- net/https'
• Scout Prints the Error: 'no such file to load -- openssl'
• Updating Rubygems
• Common Ruby Ubuntu problems
• Scout Prints the Error: 'zlib(finalizer): Zlib::GzipWriter object must be closed explicitly'
• Does the Scout Agent generate a log file?
• I'm stuck. How do I get help?

Cloud Monitoring

• How do I enable cloud monitoring?
• How does Scout determine if a new server should be created?
• What if I delete the original server in Scout?
• Which cloud hosting providers does Scout support?

Copy & Paste

• I have a large cluster of servers. Can I update monitoring across the entire cluster quickly?
• When updating existing plugins, what happens to triggers and notification settings?
• When updating existing plugins, are there any attributes that aren't copied?
• Can I choose to only copy some plugin-specific settings?

Inserting Report Data Into Scout

• Can I associate report data with an arbitrary time?
• Scout isn't graphing fields it should be (or vice versa). Does report data need to fit a certain format?
• Is there a maximum size for a report field name?

Everything Else

• I'm having problems authenticating to retrieve RSS Feeds.
• Will Scout start automatically when my server is rebooted?
• I have a number of servers I'd like to install Scout on. Are there any auto-deployment options?

How Scout Works

How do I use the Scout Agent?

Installing the agent is just a simple Ruby gem install:

Note that the gem requires Ruby 1.8 or higher.

Once the gem is installed, you need to identify yourself with the agent key (which looks like a7349498-bec3-4ddf-963c-149a666433a4) that you get from scoutapp.com. Just issue this command and have your key ready when it asks for it:

At this point, you should be all set to run the agent. Just add Scout to your crontab file so it runs every minute (see our section on Cron):

With the agent running, you should be able to log into your account on scoutapp.com to setup your list of plugins and see the agent delivering data.

What are the Scout Agent system requirements?

The Scout Agent has been tested on the following operating systems: Linux, Apple OSX, and Solaris. Ruby 1.8.6 or higher and Rubygems 1.3.1 or higher must be installed.

What impact does the Scout Agent have on performance?

The Scout Agent is very lightweight. It only uses resources when it runs through Cron. It isn't a long-running process so it won't leak memory. Most plugins scrape basic Linux commands, so the plugins themselves usually don't load heavy libraries. On a typical system, the agent usually consumes less than 15 MB of memory when running.

How does the Scout Agent execute plugins?

The Scout Agent retrieves the plugin plan from the scoutapp.com server when it runs. The plugin plan includes the plugin code plus any plugin-specific options you've set. The agent then executes the code, with the provided options, and reports back the results to Scout.

Plugins are not manually installed on the agent. You configure all plugins on scoutapp.com. Once the Scout Agent is installed, you're done!

Scout retrieves plugins via HTTP. What happens if the resource is no longer available?

Scout caches the plugin code upon installation, so you'll be able to execute the plugin code whether or not the resource still exists.

What happens if a new version of a plugin is released?

Scout will never update the code running on your server without your explicit instructions to do so. If a new version of a plugin is released, the plugin won't be updated with the new code. If you wish to update a plugin to a new version, just click the "Update Code" button on the Plugin Settings page.

Is Scout open source?

The Scout agent is completely open source. The gem is a normal Ruby gem, open for development, available on GitHub and distributed under the MIT and/or Ruby License (whichever you prefer). We share a collection of open-source Scout Plugins surrounded and fostered by a community that encourages branching, fixes, and general open-ness. Scout plugins can be accessed via GitHub.

The Scout Server -- which handles the data collection, analysis, trending, and notifications -- is not open-source. We maintain the server, and keep all your data safe and sound.

How long does it take for data to appear on my Scout account?

Typically it takes less than 5 minutes for data to appear on your account.

Using Cron

How do I setup Scout to run using cron?

You need to configure Scout to run automatically after it is installed. Cron is the easiest way to have Scout run at a scheduled interval on a Linux, Solaris or BSD system.

Cron jobs are configured using the crontab file (also called the cron table). Each user can have their own crontab file, and there is also a system-wide crontab file located here:

If you have root access, find out: "How do I setup Scout to run using the system-wide crontab file?"

If you do not have root access, find out "How do I setup Scout to run using my current user's crontab file?"

How do I setup Scout to run using the system-wide crontab file?

As the root user (or using sudo), open the crontab file for editing:

We will need to add a line to the bottom of the file with our cron job.

To run the Scout agent every minute, we'll use the following format:

For example:

The above cron job runs every minute as the deploy user.

Save the file, and it will automatically run every minute.

How do I setup Scout to run using my current user's crontab file?

We will need to use the crontab command to edit our current user's crontab.

Run the crontab edit command:

If you do not have any jobs already setup, you should see a blank file. If you get an error, see this Unix Crontab Reference for more help.

To run the Scout agent every minute, we'll use the following format:

For example:

The above cron job runs every minute as the current user.

Save the file, and it will automatically run every minute.

Scout doesn't run in cron. I'm seeing a "Permission denied" error.

Scout probably doesn't have access to the data file it needs to store information locally. You may be scheduling the job to run as a different user than the one you installed Scout with, and that user may not be able to write to the data file.

You can get around this by specifying a path to a data file (doesn't matter if it exists yet) that the user in the cron job has access to using the "-d option":

Scout doesn't run in cron. I'm seeing a "ruby: command not found" error.

Scout can't find the Ruby executer. Your cron path may not contain the full path to Ruby. Be sure to update your PATH in /etc/crontab if you have installed Ruby in an alternate location. For example, if Ruby is installed in "/usr/local/bin", your PATH line my look like:

I'm still having problems running Scout through cron.

Here are a few tips for debugging Scout in the crontab file. We can tell Scout to log to a file:

Using the -v option tells it to be verbose. Using the -l debug option tells it to use the debugging log level.

For more help on cron and the crontab file, see this Unix Crontab Reference.

Using Launchd

How do I setup Scout to run using launchd?

Scout can be configured to run at a scheduled interval using launchd, or as it sometimes called, launchctl or LaunchDaemon. Launchd can be used by Mac OS X systems to run Scout regularly.

To create a LaunchDaemon, simply create a file, named com.scoutapp.ScoutAgent.plist with the following properties:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
          <key>Label</key>
          <string>com.scoutapp.ScoutAgent</string>
          <key>ProgramArguments</key>
          <array>
                  <string>[PATH TO RUBY]</string>
                  <string>[PATH TO SCOUT]</string>
                  <string>[AGENT KEY]</string>
                  <string>-d [DATA FILE]</string>
          </array>
          <key>StartInterval</key>
          <integer>[INTERVAL IN SECONDS]</integer>
  </dict>
  </plist>
  

Here's an example file with all the details filled in:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
          <key>Label</key>
          <string>com.scoutapp.ScoutAgent</string>
          <key>ProgramArguments</key>
          <array>
                  <string>/usr/local/bin/ruby</string>
                  <string>/usr/local/bin/scout</string>
                  <string>8c6dc00d-af3a-4606-a384-85f276ecbbb5</string>
                  <string>-d /tmp/scout_data.yml</string>
          </array>
          <key>StartInterval</key>
          <integer>60</integer>
  </dict>
  </plist>
  

Next, copy this file to /Library/LaunchDaemons :

To install it, you can simply restart the system, or load it instantly:

The daemon will run every minute (60 seconds) using the /tmp/scout_data.yml file for storage.

Scout doesn't run in launchd. I'm not seeing any errors.

In order to view errors produced by launchd, you'll need to visit the Console, or tail the /var/log/system.log file. Launchd logs messages and problems running the Scout Client there.

Security

What is Scout's approach to security?

Security is important to us, and we are committed to ensuring that Scout meets the same high standards you maintain for your own servers.

First, we ensure that only our server is sending you plugins. The agent does ensure this by validating the SSL certificate of the Scout server. The agent will not send any data to or take any data from a source that fails this validation. This makes a man-in-the-middle-attack practically impossible.

After that, we ensure our server doesn't send you bad plugins. We code-review each plugin that is accepted into our directory. We also don't think you should just take our word for it, which is why we show you the code when you add a plugin. This isn't nearly as it involved as it sounds since the majority of plugins are fairly short and straight forward.

In addition to the plugins being available to you, we also keep the agent open source. One of our major reasons for doing that is to ensure that your security team has full access to the code we are asking you to use on your server.

How does Scout's architecture make it secure?

• You install the Scout Agent (which is a Ruby gem) on your server.
• The agent connects over https (always) through a 256-bit secure, encrypted connection (the same encryption your bank uses).
• Scout never logs in to any of your servers.
• All communication is initiated by the agent — it is impossible for a 3rd party to initiate any operations by contacting the Scout agent.
• The agent downloads a pre-loaded plugin plan, consisting only of plugins of your choosing, so it cannot run plugins you didn’t explicitly authorize.
• The server also uses that same secure encryption for all communication. Individual accounts are protected.
• Agent keys (uniquely generated) can be revoked at any time, disabling the agent.

Are there any additional security measures I need to take with Scout?

As long as you are running approved plugins from the Scout plugin repository, there is nothing more you need to do. If you are running plugins from outside our repository, treat them like any other code you introduce into your production environment: make sure it comes from a credible source, and review the source code if you have an doubts. Most plugins are relatively few lines of code and easy to review.

Can plugin code be updated on my server without my knowledge?

No. Even if new code is checked into the plugin repository for a plugin you have installed, your plugin will not be updated unless you explicitly upgrade it.

Troubleshooting

How do I Solve 'scout: command not found'?

Assuming you've successfully installed the Scout gem, this indicates a problem with your path. Specifically, your path doesn't include executables installed via RubyGems. To locate the Scout executable, run:

The EXECUTABLE DIRECTORY contains the Scout executable. So in this case, you could run scout with:

For a longer-term solution, add the value of EXECUTABLE_DIRECTORY to your PATH. Depending on your system configurations, you may need to set PATH separately for crontab. Or, just use the full path to the scout executable in your crontab.

Scout Prints the Error: 'no such file to load -- net/https'

This happens most often on Debian installs, because Debian does not install LibOpenSSL by default with Ruby. To fix:

Advanced: if you are compiling Ruby from scratch, make sure you install (through aptitude) openssl libssl libssl-dev before compiling Ruby.

Scout Prints the Error: 'no such file to load -- openssl'

We've seen this issue on Centos installs. To fix:

Updating Rubygems

There are variety of issues that are resolved by upgrading the latest version of Rubygems. These issues can arise as:

• When attepting to install a gem: ERROR: While executing gem ... (Gem::GemNotFoundException)
• ... or Could not find elif (> 0) in any repository
• When running the Rails Analyzer plugin: Could not find RubyGem rspec (>= 1.2.4) (Gem::LoadError)

To fix, first try:

The gem --version command should return at least 1.3.4. If this doesn't succeed, try the following:

if your current version of Rubygems is very old, this approach may not work either. Ubuntu 8 (Hardy) in particular gives you an ancient version of rubygems through apt-get. So, first, uninstall any existing rubygems installation:

Common Ruby Ubuntu problems

Ruby on Ubuntu can present missing library errors. The blog post Troubleshooting common Ruby Ubuntu problems solves a number of these problems.

zlib(finalizer): Zlib::GzipWriter object must be closed explicitly

If you run the Scout agent at the debug log level and see the following error:

Update to the latest Scout Agent release. Versions 5.1.3 and higher should address this issue. To update:

Does the Scout Agent generate a log file?

No, but you can enable verbose logging when running the agent and store the output in a file:

The above line saves the output of the scout command to the /home/[USER]/scout.log file.

I'm stuck. How do I get help?

Contact us at support@scoutapp.com. We usually respond in a couple of hours during the business day. Please provide your operating system, account name and any error messages you received.

Cloud Monitoring

How do I enable cloud monitoring?

Once you have a server setup and monitored in Scout, select the 'Cloud Image' checkbox in the server settings: When you deploy another instance that reports to Scout using the same server key, a new server will automatically appear in Scout with the same plugins, triggers, settings, etc. as the original server.

Watch a 3 min 42 sec video of the functionality.

How does Scout determine if a new server should be created?

Scout creates a new server if the hostname of the reporting instance is different than the original server.

What if I delete the original server in Scout?

If there are other instances of that cloud image, all remaining instances will continue to check in normally. The oldest remaining instance will become the new cloud image.

If there are no remaining instances of a cloud image, the key no longer work for any checkins, and you must create a new cloud image from scratch.

Which cloud hosting providers does Scout support?

Scout's cloud monitoring works across all cloud providers on Linux-based instances.

Copy & Paste

I have a large cluster of servers. Can I update monitoring across the entire cluster quickly?

Yes. With Scout's Copy & Paste functionality, you can install, update, and delete plugins across your server cluster with 1 click.

Read more on our blog or watch a video.

When updating existing plugins, what happens to triggers and notification settings?

The triggers and notification settings on the plugin being copied replace those on existing plugins. The only exception: if a person doesn't have access to a server they won't be added to the list of users notified when the plugin generates alerts.

When updating existing plugins, are there any attributes that aren't copied?

The names of existing plugins are not overwritten.

Can I choose to only copy some plugin-specific settings?

Yes. If the plugin has specific settings, you can choose to only copy the selected settings. By default, all plugin-specific settings are copied. An example:

Can I associate report data with an arbitrary time?

No - you can't insert historical data into Scout.

Scout isn't graphing fields it should be (or vice versa). Does report data need to fit a certain format?

If a field has a nil value, it won't be graphed. Instead, return zero.

Is there a maximum size for a report field name?

Yes - the maximum size of a report field name is 30 characters. If the field name exceeds 30 characters, then it will be truncated to 30 characters. For example:

# use a report field that is longer than 30 characters
field_name = 'This is a very long report field name'
field_name.size => 37
# This becomes the following in Scout:
field_name[0..29] => "This is a very long report fie"

I'm having problems authenticating to retrieve RSS Feeds.

Scout's RSS Feeds are protected with Basic Authentication and use the same credentials as your Scout login (email & password). However, you can't type the '@' symbol of your email address in a URL. Instead, replace '@' with '%'. For example, if my email address was "dog@highgroove.com": http://dog%highgroove.com:pass@scoutapp.com/highgroove/activities.rss.

Will Scout start automatically when my server is rebooted?

We suggest running Scout regularly through Cron. This means you don't have to do anything special to ensure Scout runs when your server is restarted.

I have a number of servers I'd like to install Scout on. Are there any auto-deployment options?

The Scout Community has contributed a Chef Cookbook and a Moonshine Script for auto-deploying Scout to multiple servers.

Status Updates via Twitter

Google Group

Creating a Plugin