I am very pleased to announce the release of two new tools that I hope will prove to be useful additions to your operational arsenal:
- Senedsa is a small utility and library that wraps around the Nagios
- Elesai is a wrapper around LSI’s
Both of them are distributed as Ruby gems.
Senedsa is something I have been meaning to write for quite some time. A fair amount of the tools I have written over the last few years (most notably Zettabee and Theia) have had monitoring capabilities built-in, mostly in the form of a Nagios passive monitor. I have moved to developing in Ruby essentially full-time (I still do some Python and some shell scripting, but that’s mostly in maintenance mode), and after I went down the path of writing Elesai, I simply had to write Senedsa, since the ridiculous cut-and-paste maneuver was getting old (and frankly, embarrasing).
I pondered using Kevin Bedell‘s
send_nsca gem, but in the end I decided to implement the Senedsa wrapper, primarily because we have other non-Ruby code (shell scripts) that use
send_nsca in its native form, and I suspected that would be the case for most other shops. While that implies a fork whenever it is used, it is not something we are doing at high frequency.
Senedsa be be used both as cli utility (which is handy to test your
send_nsca installation or perhaps to be called from shell scripts, tho that implies you are be forking both
send_nsca) or a library from within your Ruby scripts.
Elesai is a wrapper around LSI’s
MegaCli utility that provides access to common types of information about LSI RAID controllers (currently physical and virtual disks) without the need to “speak martian” (run
MegaCli -h to see what I mean). It is a line-oriented tool so that it can be combined with other Unix command-line tools to process and manipulate the data (i.e.,
awk, and friends). It also provides a
check action (currently as a Nagios plugin in both active and passive modes) which monitors the health of the array and its components and reports it accordingly (this is not yet configurable).
The exercise of developing Elesai has be useful in a number of ways. Perhaps the most significant one has been the realization of something I have actually known for quite some time but had not fully solidified in my mind: each and every tool monitoring if it is doing any regular work in production. Full-stop. Clearly this is something I had been doing already (Zettabee being the primary example), but it has now been elevated to full requirement.
The second aspect was the use of state machines to parse pseudo-structured output like that of
MegaCli, where identifying the current element being processed becomes trickier. The usual (or perhaps the one I have seen implemented more often) approach is a set of nested
case) statements matching the appropriate regular expressions with state variables sprinkled around. This normally works, up until the point where the parser needs changes or additions six months after the code was written. Elesai itself will need several additions in the not-too-distant future, as it currently only shows information about physical and virtual disks (and does not, for instance, take spans into account): information about the adapters themselves and batteries are high in the list of new features.
So there had to be a better way. I first looked into the excellent Parslet library, but it really wasn’t the right tool for this job. I had had state machines lingering in my mind for quite some time, and that turned out to be a incredibly good fit, especially when using the wonderful workflow state machine implementation.
If you need to deal with LSI RAID controllers, I hope you find Elesai a worthy tool to add to your toolbox. It’s easy to install:
gem install elesai. Do please report problems and feature enhancements in the issue tracker, or better yet, and if you’re up to it, fork it and contribute. Ditto Senedsa.
It’s hard to believe it has almost a year since we started the process of open sourcing tools, but it has indeed been that long, and it picked up steam a few weeks ago, when pushed out nddtune, which is admittedly a very simple tool. Today we’re continuing that effort with a couple of more significant tools: Zettabee and Theia.
A Little History
About four years ago, we had a very real need to have fairly detailed performance metrics for NetApp filers. At the time, the available solutions relied on SNMP (NetApp’s SNMP support has historically been weak) or were NetApp’s own, which, asides from expensive, were hard to integrate with the rest of our monitoring infrastructure (which is comprised of Nagios and Zenoss). As such, we set out to write a tool that would both perform detailed filer monitoring (for faults and performance) and that would be able to interface with those systems. Theia was born.
In more recent times, as we were looking at beefing up our DR strategy, we found ourselves needing a good ZFS-based replication tool, and set out to write Zettabee, which gave us an opportunity to dive deeper into ZFS capabilities.
Let the Games Begin
Today we’re very excited to be releasing those two tools into the open. Theia has been in production for the last four years, dutifully keeping an eye on our filers, while Zettabee has been pushing bits long-distance for well over nine months. We are working on putting together a roadmap for future work, but are happy to have them out in the open for further collaboration. Tim has written a good post on some of the work he has done to make this happen, and I am grateful for his help on this endeavor.