Tag Archives: php

Harddrive failure and monitoring

Posted by on October 11, 2011 1 comment | 3,334 views

Broken harddiskI have two Ubuntu servers running at home which both have large RAID volumes on them, set up via mdadm. This summer I had a total disk failure in one of my RAID5s which luckily didn’t result in data loss. Thanks you RAID5!

In any case, it caused me to write a script that logs SMART-data to a MySQL database. I also wrote an admin webpage that displays this data for me in an easy to follow way. The monitoring script itself is written in php5 and so are the admin pages. I used php5 because it is easy to communicate with MySQL from it, and it has the needed string manipulation commands. It could probably be done as easily in Python though. The script is called as a cron job every 2 hours on the servers and every hour on the desktops when they’re running. Examples of the code I’m using is attached below and includes the cron-ed script and the code generating the log output and plot.

What to look for?

Well, that is the big question. How do you know if a drive is about to fail? Google Labs has looked into this topic back in 2007 in this paper: «Failure Trends in a Large Disk Drive Population». An interesting read if you are at all concerned about harddrive failure in servers. The results for how temperature affects the lifetime and failure rate in harddrives are especially interesting. It turns out, at least in their data, that low temperature isn’t such a good thing for the drives contrary to what many people seem to assume. I have up until now been concerned that my drives get too hot, but in fact they seem to be almost overcooled the way I have things set up now.

When I wrote these scripts this summer I decided to log temperature and reallocated sector count primarily, which is what is emphasized in the log display scripts. Seems now I also should be including scan errors as well after reading that paper. The colour coding I use in the temperature plot below is loosely based on Figure 4 in the paper and reflects what seems to be the optimal operating temperature for harddrives.

Screenshots
HD-Mon Screenshot 1HD-Mon Screenshot 2HD-Mon Screenshot 3
Screenshot 1: The overview page.
Screenshot 2: Details of one of the RAID-drives.
Screenshot 3: Details of one of the drives with reallocated sectors.

Code

The php source code I wrote is available in this file: hd-mon.tar.gz

It is specifically designed to work with my setup and hardware and probably isn’t universal, but it gives an idea of how I set it up. There are probably better ways of doing this though. I just call shell commands from php and parse the returned text-string and do simple search on it and input the data into a MySQL database.

I also included php-snippets showing how the admin page is generated. These are not standalone php files, they need to be wrapped in a template. However they reproduce what is seen in the screenshots above.

Packages needed for these scripts to run:

  • php5-cli for the php5 command line.
  • mysql-client, php5-mysql for the database connection.
  • smartctl to access the SMART-data.
  • mdadm to access RAID-data (assuming you use mdadm for RAIDs in the first place)

All are available in the Ubuntu repository.

Getting Torrent Name from File in php

Posted by on January 24, 2011 No comments | 1,979 views

Wrote a script today that parses a torrent file and retrieve the name of the torrent. The reason I made this is that I have a small web page running at home that parses an RSS feed and upload selected torrents to another computer. Since the name of the torrent on disk (the data folder) doesn’t always match the name of the actual .torrent file, I need to get the name from the file itself. It is quite simple really, and it falls back to the .torrent file name if it fails.

// Find torrent name
$sTorrent     = "torrent_file_name.torrent";
$sTName       = trim(substr($sTorrent, 0, strrpos($sTorrent, ".")));
$sTorrentData = file_get_contents("path/to/torrent/file/".$sTorrent);
preg_match("(:name[0-9]+:)",$sTorrentData,$aTemp,PREG_OFFSET_CAPTURE);
if(count($aTemp) == 1) {
  $sTemp = str_replace(":","",$aTemp[0][0]);
  $sTemp = str_replace("name","",$sTemp);
  if(is_numeric($sTemp)) {
    $sTName = substr($sTorrentData,$aTemp[0][1]+strlen($aTemp[0][0]),(int)$sTemp);
    $sMessage = "<div>Torrent successfully parsed.<br>";
    $sMessage = "Name: ".$sTName."</div>";
  } else {
    $sError = "<div>Parseing failed.<br>Using name: ".$sTName."</div>";
  }
} else {
  $sError = "<div>Parseing failed.<br>Using name: ".$sTName."</div>";
}

How to stop the damn commentspam!

Posted by on November 18, 2009 No comments | 2,872 views

captchaI got fed up with the comment spam I get on my blog. I googled a bit and found a few solutions, but ended up with a plugin that adds CAPTCHA to the comment form.This little gem can be downloaded from here.

It is fun to observe the spammer-scripts bang their virtual heads against this simple little tool. I can observe all the spam attempts in my website log. :)

Meh, WordPress will do…

Posted by on May 17, 2009 No comments | 1,047 views

After spending most the afternoon coding php scripts for a new website, I decided to just stick with WordPress afterall. There are many things I don’t like about it, but I managed to find plugins for the stuff I was missing. Not to mention the animated Tag Cloud I saw the other day. Looks fancy, just need more, uhm, tags… :)

The current theme is fine too, spent a lot of time yesterday cleaning it up. It had several bugs. I also modified a lot of the code. I may just replace the banner (have a few made) and change the colors too, at which point there won’t be much left of the original theme. I have kept the link to it on the footer tho, but removed the ads.

Tweaking is half the fun of a website!