Server Monitoring Through DD-WRT

Happy New Year!  I’m kicking off my 2012 blog entries with a fun little hack for Linksys routers.

There are plenty of articles on the web about using DD-WRT to enable router monitoring.  I decided to turn this idea on its head and use my router for server monitoring!  When I realized DD-WRT comes with a sendmail command, I knew this was going to be quick and easy to set up.

This is great for anyone who would like their celly to light up as soon as something goes wrong with an important computer or website.  All of the needed software is already built in to compatible routers, so there is no need to purchase or install a dedicated monitoring system on a separate computer.

By following these easy steps, you can create your own reliable monitoring service.

Step 1: Install DD-WRT on your router.  This is simply a prerequisite.  The “mini” version is adequate.

Step 2: Log in to DD-WRT and click on the Administration tab.  Then click the Commands sub-tab.

Step 3: Write the monitoring script in the Commands text area.  Mine looks like this:

#!/bin/sh

date > /tmp/root/mylog

if ! ping -c1 -W1 192.168.1.2 > /dev/null; then
  sendmail -parameters go here
else
  echo "All tests passed." >> /tmp/root/mylog
fi

The part that says 192.168.1.2 is whatever address you would like to monitor.

The sendmail command requires several parameters that will be specific to your e-mail server. There is a good explanation of the sendmail parameters at the DD-WRT Forum.  Just use common sense to select an SMTP relay that isn’t on the same server being monitored and isn’t blocked by port restrictions from your ISP. There is also a workable syntax for changing the mail port to other than 25.

I configured my sendmail script so that a message is sent to my cell phone informing me of any failure of the server ping.

Step 4: Click the Save Custom Script button.  This takes care of storing your script in the appropriate location.

Step 5: Click the Management sub-tab under Administration.

Step 6: Scroll down to the area labeled Cron.  Make sure Cron is set to Enabled.

Step 7: Paste this into the Additional Cron Jobs text area:

0 * * * * root /tmp/custom.sh

This particular command will schedule your ping test to run every hour on the hour.  You can easily adjust the schedule using conventional cron syntax.

Step 8: Click the Apply Settings button.

That’s all you have to do.  Now you will know if the server is down before you have to find out the hard way.  🙂

Update 25 March

Unfortunately, through vigorous testing, I have confirmed that the DD-WRT cron service is unreliable due to the documented bug, “CRON Service falling asleep in V24.”  The available workaround involves manually restarting the service, which is not an adequate solution in this case.

My workaround is automated, but a bit involved.  In “Step 3” above, I added a line at the top of the router script that will cause it to create a file named ‘mylog’ containing the date when it runs.

Below is a PHP script that will check the log file and then restart the DD-WRT cron service when necessary. I hope it helps anyone who needs it.  This does have to run on a PHP server, so the implementation details are beyond the scope of this article.

<?php
/**
 * @author Robert Chapin
 * @link https://www.miqrogroove.com/
 * @license GPL
 */

$ocron = new CronCheck();
$result = $ocron->run();
if (!$result) exit(1);

class CronCheck {
    private $host     = 'host.name.goes.here';
    private $password = 'password.goes.here';
    private $logfile  = '/tmp/root/mylog';
    private $max_age  = 3600; // Time in seconds before cron considered failed.

    private $port       = 23;
    private $username   = 'root';
    private $connection = FALSE;

    public function run() {
        $this->connection = fsockopen($this->host, $this->port);
        if (FALSE === $this->connection) {
            return FALSE;
        }
        stream_set_blocking($this->connection, FALSE);
        sleep(1);

        if (FALSE === strpos($this->get(), 'login:')) {
            $this->disconnect();
            return FALSE;
        }

        $this->send($this->username);

        if (FALSE === strpos($this->get(), 'Password:')) {
            $this->disconnect();
            return FALSE;
        }

        $this->send($this->password, FALSE);

        if (FALSE === strpos($this->get(), "Enter 'help' for a list")) {
            $this->disconnect();
            return FALSE;
        }

        $this->send("more $this->logfile");
        $s = $this->get();

        $date = substr($s, 0, strpos($s, "\r"));
        $date = str_replace(' UTC', '', $date);
        $diff = time() - strtotime($date);

        $minutes = floor($diff / 60);
        echo "Cron last ran $minutes minutes ago.";

        if ($diff > $this->max_age) {
            $this->send('stopservice cron && startservice cron');
            $this->get();
        }

        $this->disconnect();

        return TRUE;
    }

    private function send($cmd, $ignore_echo=TRUE) {
        fwrite($this->connection, $cmd."\r");
        if ($ignore_echo) {
            $this->skip(strlen($cmd) + 2);
        } else {
            $this->skip(2);
        }
    }

    private function skip($limit) {
        $count = 0;
        sleep(1);
        while($count < $limit and !feof($this->connection)) {
            $line = fread($this->connection, $limit - $count);
            $count += strlen($line);
            if ($count < $limit) sleep(1);
        }
    }

    private function get() {
        $data = '';

        if(!feof($this->connection)) $data = fread($this->connection, 4096);

        return $data;
    }

    private function disconnect() {
        if (FALSE !== $this->connection) {
            $this->send('exit');
            fclose($this->connection);
            $this->connection = FALSE;
            return TRUE;
        } else {
            return FALSE;
        }
    }
}
?>