Draconis Software Blog

Installing Subversion on MacOS

Recently, I needed to upgrade the Subversion client I had installed on my Mac (my personal development machine) to a more recent version. The issue that came up was an incompatibility between my IDE’s Subversion plugin and the command-line version I had previously installed.

When checking out files within the IDE (or even doing updates, for that matter), the sandbox would have its version flags switched, causing my command-line client to no longer work. It got frustrating, as different parts of the sandbox were done in different editors (a Java IDE for one project, a text editor for another, etc), and keeping my sandbox up-to-date was proving tedious.

So, the solution was to update the Subversion client on the command line. Doing so was very easy, though it required building from source (I haven’t been able to find a recent binary installer for the Mac).

(Read the article)

Sysadmin Certifications

I’ve generally been a fan of certification programs for systems administrators as a means for providing at least a basic idea of the competency of a potential hire. After reading this article at Linux.com, I’m not particularly surprised to see the number of certifications expected to increase (especially for GNU/Linux programs), though I have to wonder about it: the best sysadmins I’ve known didn’t have a single certification and weren’t particularly interested in getting one.

I see the whole certification process as having two main flaws (as least on the part of companies certifying their own products): (1) little pressure on the part of the certifying company to make the tests difficult or otherwise accurately prove a taker’s skills, and (2) lots of pressure on the part of the company to test the applicant’s knowledge of vendor-specific aspects. It seems to me that it’s in the interest of the certifying company to have lots of certified engineers out there who know the ins-and-outs of that company’s products and little about any competing products.

So, as someone who needs to hire a competent sysadmin, how does this help me? A potential sysadmin who’s certified as an MCSE or RHCE or whatever shows they can take a test and know the basics of that particular company’s product, but what about the millions of other things that sysadmin would be responsible for? Really, it seems these certifications are good for large companies interested in a sysadmin to manage many exact-same boxes and little else. For the majority of employers, especially startups and growing companies, I’d think someone who is more well-rounded in things they’d need to manage is much more useful (for instance, can a sysadmin fill in for a network admin should the need arise? do they understand infrastructure needs to make recommendations for expansion? etc). The best skill a sysadmin can have is the ability to learn as they go and adapt to changes. Having a certification in a particular OS doesn’t particularly help if its a heterogeneous network.

I think certification is a useful process but I’d like to see programs that are more comprehensive, easier to afford, and focus on general skills regardless of environment. What do you think? Have you gone through a certification process and, if so, how has it helped you? In the past, I’ve thought about getting certified myself, but never went through with it primarily for these reasons (as well as the cost).

Top 7 Things System Administrators Forget To Do

The O’Reilly Sysadmin Blog has a nice piece on things sysadmins forget to do. Most of them relate to common tasks that fall by the wayside. I think programmers and sysadmins alike have a tendency to prefer interesting, more intellectually challenging activities over things like managing users or root access. The good thing about this is that these kinds of menial tasks can often be automated by programs like cfengine or puppet.

Even for those tasks that cannot be fully automated, such as keeping documentation, there are plenty of applications out there to assist in the process. For example, we’ve found that keeping documentation, notes and fixes in a wiki is a great way to keep information organized.

Personally, one of the tasks I often forget is to keep packages managed and up-to-date. Luckily, this is a lot easier than it used to be with programs like apt-get and yum.

Finally, I think we can all appreciate the reminder to always keep a friendly attitude: “If you want support from management, consider remembering that the user you offend today could wind up on the board of directors. Regardless of that possibility, system administrators should always remember that their clients are internal and if you want to keep your job, be good to your clients.”

Updated to Ubuntu 7 (Feisty Fawn)

Ubuntu LogoYesterday, we updated our workgroup server to the latest version of Ubuntu - Feisty Fawn - and so far we’ve been happy with it. We’re a little late in updating, though that was primarily us being careful on a production machine. The main reason for updating, actually, was due to an issue with PHP.

We’ve been doing some work in PHP lately that makes use of the GD library to do some interesting graphics manipulations. One of the components of the project made use of a transparency color for an image resource, and it seemed like there may have been a bug in the imagefill function: filling with a color of -1 caused the php processor to segfault. What was odd, was a different version (on my Mac), wasn’t crashing at all. This prompted the update to the latest Ubuntu. The latest version, 5.2.1, solved the problem and we were good to go.

Doing an Update (from Edgy to Feisty):
A distribution upgrade with Ubuntu is cake: simply edit your /etc/apt/sources.list file and replace “edgy” with “feisty”. Next, update the sources list (apt-get update will do it), and finally do a dist-upgrade (apt-get dist-upgrade). Check out this article for more details, as well as how to do a dist-upgrade using the GUI.

Backing up a Subversion repository

SubversionWe maintain a lot of projects in version control, both internal stuff (like our monitoring/management app RSP, and our website), as well as client projects (some of whom have direct access). Originally, we used CVS to handle versioning, but over time, migrated to Subversion, a great replacement. Since moving to Subversion, however, we’ve had to rethink how backups are done, as one of the key differences with Subversion is how it stores information on disk. Instead of storing flat files with versioning information inside each file, it instead uses either the Berkeley database or a custom file system (called FSFS, for a filesystem within the host’s filesystem). It gives the application a lot more flexibility (directories are properly versioned as well as files), but can make backups a bit tricky. Let’s take a look at how to backup a Subversion repository (it’s not too difficult).

The trick here is dealing with a live repository: since users could be making changes to this repository at any given time, doing a recursive filesystem copy isn’t ideal. If a user were making a change while this copy was taking place, the backup could be missing data, or could simply be corrupt, which defeats the purpose of making the backup in the first place! A better way is to either freeze the Subversion server temporarily, until the backup completes, or to do a hot-copy.

(Read the article)

System Administration with Ruby

Ruby gets a lot of (well deserved) press because of Ruby on Rails, but recently I’ve found it to also be an excellent choice for scripting tasks, jobs that I otherwise would have used Perl for.

The first benefit of using Ruby is that its built-in code blocks allow for simpler, shorter code. A Ruby block is basically an anonymous function that can be passed to another function as a parameter. For example, here’s how you could take an array of numbers and square each one:

RUBY:
  1. nums = [1, 2, 3, 4, 5]
  2. nums.collect { |x| x*x }    # produces [1, 4, 9, 16, 25]

Blocks are used for all sorts of purposes, like looping, iterating, sorting and mapping. Once you get used to them, they become quite intuitive and save a lot of coding time.

In particular I’ve found code blocks to very useful with database programming. Here’s an example that uses the mysql Ruby plugin to do a query:

RUBY:
  1. require 'mysql'
  2.  
  3. db = Mysql.new("localhost", "root", "password")
  4. db.select_db("a_database")
  5.  
  6. result = db.query "select id,name from users"
  7. result.each do |row|
  8. puts “ID: #{id}, Name: #{name}”
  9. end

Note that “do/end” is a multi-line alternative to brackets.

In Ruby, everything is an object, even primitive types like numbers and strings. The language makes it quite easy to create your own classes as well, and because of this I find myself creating classes more often than I normally would while writing scripts. Currently in Perl, OO programming is still rather obtuse, so it’s often simpler just to use scalars, arrays and hashes in various combinations (for example, “a hash where the keys are names, and the values are arrays where the first element is a number”, etc). The problem with this is it can create brittle code that can make bugs common.

Ruby classes can be written inline with procedural statements, and are usually very short:

RUBY:
  1. # A simple class to represent a person
  2. class Person
  3. attr_accessor :first_name, :last_name, :age</code>
  4.  
  5. def initialize(first_name, last_name, age)
  6. @first_name = first_name
  7. @last_name = last_name
  8. @age = age
  9. end
  10. end
  11.  
  12. joe = new Person(“Joe”, “Schmo”, 25)

The “attr_accessor” line is a useful declaration that creates a private member variable along with public reader and writer methods.

Two common tasks of scripts are working with operating system calls, and string parsing/manipulation. In these aspects, Ruby works just like Perl. You can use hash marks (the ` character) to run a command return its output. For working with strings, Ruby regular expressions work just like Perl. Here’s an example that uses both:

RUBY:
  1. # Look for a resolve error
  2. resolve = `resolveip #{address}`
  3. puts “Host not found” if resolve =~ /host not found/

All of this being said, there are still many places where I feel that Perl is a more appropriate choice for sysadmins. While Ruby has a growing set of plugins, nothing can beat the breadth of Perl modules available through CPAN. For a very specific task (say, interfacing with the /proc filesystem) Perl is often a better choice simply because of the amount of publicly available code out there.

The next time you have to do some administrative scripting, consider giving Ruby a try. Even if you decide to stick with Perl (or whatever your language of choice), it’s a great way to try out Ruby if you’re not familiar with it.

How to Deal with Log Overload

Even a relatively inactive server can produce quite a large amount of logs, and a high use machine serving multiple functions will probably have so much log activity that it becomes impossible to discover anomalies, resource problems, or potential attacks. Luckily, this copious amount of data can be made manageable without too much trouble.

Consolidate logs across the network

If you have more than one machine to keep track of (and what sysadmin doesn't) the log problem only multiplies. One thing that can help in this situation is remote logging. For example, syslog makes it easy to send logs over the network to a single source. In large networks this may be a dedicated logging machine. In any case, this at least somewhat reduces the complexity of the situation, although does not help with the sheer amount of information produced.

I'll usually leave a window open with a connection to the central logging machine with a running output of logs (this can be done with `tail -f'). This helps me catch some issues as they occur, but it's easy to miss entries if I'm not at the computer or not paying attention to the running log.

(Read the article)

Complete, free Mac backup

Apple Leopard: Time MachineThere's a nice article on Lifehacker about backing up your Mac using several free tools. As I've said time and again, backups are key! The article gives a good introduction to the various tools available for Mac backups.

As pointed out in the article, the next iteration of the MacOS will see a new feature, Time Machine, which will provide built-in revisioning services to documents in the MacOS (very cool), as well as the ability to backup to a different hard drive (including a remote-mounted disk). I'm looking forward to this (and the Spaces feature) especially, but until then, another tool needs to fill the need of backing up your Mac.

(Read the article)

The rise of the local web application

I do a lot of data backups on CDs and DVDs, and I recently realized that I had no organized way of figuring out which files were located on which discs. Rather than looking for a piece of management software, I decided it would be easier to just write an app myself in Ruby on Rails.

Nowadays, agile development methodologies like such as Ruby on Rails allow you to development applications so quickly, that it's often easier to write code yourself, even if a similar tool already exists.

Imagine if I had tried to use an existing tool to manage my backups, rather than do it myself. I would needed to take the time to:

  • Research available tools and determine which one fits my needs
  • Setup and install the program
  • Customize it for my needs and environment
  • Learn to use it

During the time it takes to complete these tasks, it's just as easy (and frankly more fun) to code an app myself, which will be customized to my exact needs from the beginning, and requires no learning time.

As agile development becomes more and more used, I think this "do-it-yourself" practice will also become more common. Certainly for system administrators, who already spend time writing customized scripts and GUI apps. This is the topic of a recent IBM developerWorks article, Develop Web applications for local use, which encourages web application development in place of GUI or command-line applications.

An added benefit of this practice is that the developer might discover that the application fills a need felt by others, and can turn their local webapp into an actual website or product. That's how RSP started. It started development as a tool to use internally, before we discovered that it could have a use to the outside world.

Backup and Restore Ubuntu System using Sbackup

Backups, as always, are an important part of any sysadmin's job. If you've been following this blog for some time, we've taken a look at doing automated backups on Linux, Solaris, Windows, and the MacOS, but each of these assumed a sysadmin would be setting up the backups (or at least someone with knowledge of cron, rsync, etc). As many first-time Linux users are getting started using Ubuntu, this article gives a step-by-step to users who may not be ready to setup an rsync system for themselves.

SBackup is a simple backup solution intended for desktop use. It can backup any subset of files and directories. Exclusions can be defined by regular expressions. A maximum individual file size limit can be defined. Backups may be saved to any local and remote directories that are supported by gnome-vfs. There is a Gnome GUI interface for configuration and restore.

If you're looking for an easy way to backup your Ubuntu desktop machine, sbackup may be the way to go.

« Older PostsNewer Posts »