(KDevelop with the Class View on the left and console on the bottom)

But since switching to Xubuntu, starting up KDevelop was sluggish starting up. I also had “the process for the file protocol died unexpectedly” error (appearing twice). Since KDevelop is a KDE application, I knew it probably needed to KDE libraries for different file protocols, kioslaves. So although not a dependency (which it probably should be), installing the kio-locate package solved my problems.


$ sudo apt-get install kio-locate


We also helped by migrating the content from a legacy CMS (content management system) to their current one. To do this, used the power of Ruby with some help by the popular ActiveRecord library to quickly extract information stored in a blend of MS-SQL and Oracle databases to a nice single Oracle database.

Chekout the new website at http://www.csoonline.com and the new site tour at http://csoonline.com/sitetour.

Gruff’ing It

We first came across Gruff by Geoffrey Grosenbach, which is a nice plugin that uses ImageMagick to create PNG-based graphs. For our purposes, it did not seem to be flexible enough when there is a variable number of data pairs. Here are two examples, notice the legend is stripped.

CSS Graphs

Next, we looked at css_graphs, also by Geoffrey, but the limitations as stated on the site didn’t fit our needs:

Limitations of the First Release    * Only one graph can be used per page.    * Values must be in the range 0-10

CSS Graphy

So we wrote our own plugin with the features of auto-scaling so you can enter in any range of numbers. Also we can put down as many bar charts on our page as we would like.

Full Size

More Information

You can download the plugin and read about how it works on our project site: http://oss.vermonster.com/css_graphy

In Vermonster news, there are a few noteworthy points from 2007 we are proud to share:

• We successfully launched and are continuing to maintain an enterprise-level warehouse shipping application, which has processes literally millions of pieces of mail. We squeezed all the power and validated Ruby on Rails as a stable technology choice.
• Tech Dispenser, a completely custom-build application for Computerworld launched early 2007. There are tens-of-thousands of articles being processed using a highly-usable backend (with a few ajax tricks along the way).
• Continued to support the ever-growing data appliance company Dataupia. We expect to hear some big things from them this year and applaud their efforts.
• RallyClock finally launched. It is a market breaking time tracking application that fits into your work day featuring a rich API and an Instant Messenger interface for non-disruptive use.
• We had one of our best years on record and look forward to continued growth in 2008

We’ve also started a few projects in late 2007 that we are heads-down working on right now. Look for announcements soon!

First an informative article on how we used the standard message composing notification feature found in most IM (instant messenger) clients to add critical usability to our jabber bot. Second, we have an article exploring a cool mashup with the RallyClock API and how it can be integrated into SVN (subversion) to automatically capture messages to your active timers right as you commit code.

We are really delighted to support the site and their efforts!

• IPC::Shareable
• subs::parallel
• perl threads
• simple perl fork and pipes

As the title of this article hints, I went with the last one.

Try, Try and Try Again

To preface all of this, my target environment is quite constrained. It is a hardware appliance, with the day-to-day operations being anything but running my code. So I didn’t have the flexibility to install tons of CPAN modules and also couldn’t tweak the environment (basic out-of-the-box Fedora). I had to “just get it done” with the hand I was dealt.

First, I tried using the IPC::Sharable module. But I found that the process I was trying to run was consuming too much memory, especially when a number in parallel. I kept getting an error “Could not create semaphore set: No space left on device”. I think this had to do with the variable being tied could be a fairly complex object.

The next attempt was subs::parallel. In theory, this module should have worked great (it is a really nice module). But my application has a fairly robust ORM (Object Relationship Model) as it follows a MVC pattern, so my code was calling the “parallelized” return variable almost immediately following the subs::parallel call. As the module blocks once this occurs, it was also waiting for the return almost immediately and ended up not being any faster (in fact the overhead was making it a wallclock or two slower). To use this module, I would have to drastically change the existing (and working) application. So I kept looking.

I do have threaded perl available, so I thought I would give it a shot. Every few tests (I am running my own unit tests based on PerlUnit), I was getting core dumps. Not a good thing. When I gdb’d the core dump, I got an invalid format error, so I decided to pass on threads.

This lead me to look at using pipes and simple fork() to communicate from the child processes to the parent.

Basic Concept

The general idea is you can use IO pipes for interprocess communications. That is, much like opening a file handle (which is a form of a pipe) you can open a pipe that connects one handle to another. For instance, consider this:

pipe(FROM_SENDER, TO_READER);

# now you can write to the first handle
print TO_READER "Hello";

# and it will be able to be read from the other
my $message = <FROM_SENDER>;

This is great, but if you combine this with fork(), you will notice it is blocking, which isn’t all that interesting if you want to launch these processes in parallel. You can use IO::Handle to make the pipe non-blocking, but then if two (or more) of the child processes try to write to the pipe at the same time, you can loose messages (yea, I saw this in action).

So to implement a non-blocking, but non-dropping mechanism, the idea it to dynamically create the IO handles. The implementation I used is borrowed from the perl fork documentation, under the Forking pipe open() not yet implemented section.

sub pipe_from_fork ($) {
    my $parent = shift;
    pipe $parent, my $child or die;
    my $pid = fork();
    die "fork() failed: $!" unless defined $pid;
    if ($pid) {
        close $child;
    } else {
        close $parent;
        open(STDOUT, ">&=" . fileno($child)) or die;
    }
    $pid;
}

Now execute the fork process like so:

if (pipe_from_fork('BAR')) {
    # parent
    while (<BAR>) { print; }
    close BAR;
} else {
    # child
    print "pipe_from_forkn";
    exit(0);
}

This allowed me to fork off x-number of processes and make sure the responses are returned without losing any. This is great, but there was one more issue…the messages were strings.

Encoding/Decoding

I had the ability to fork off processes and read the return, but I could only send back simple strings. This was not adequate, as my processes mostly returned objects (or arrays or objects). I decided to use the infamous Data::Dumper to handle my encoding and decoding. So in the child section, the modification is something like this:

use Data::Dumper;

if (pipe_from_fork('BAR')) {
    # parent, read our "string"
    chomp( my $encoded_return = <BAR> );

    # Eval the the string to rebuild
    my $complicated_return = eval($encoded_return);

    close BAR;
} else {
    # child, assume we want to return a complicated thing
    my $complicated_thing = some_thing_that_made_it();

    # Encode it here.
    my $encoded_thing = Data::Dumper->new( [ $complicated_thing ] );
    $encoded_thing->Purity(1)->Terse(1)->Deepcopy(1)->Indent(0);

    # Then print, like before.
    print $encoded_thing;

    exit(0);
}

Putting it Together

This all is working fine, but I wanted more general usage. So I whipped together a quick and dirty perl module with a neat interface. First the interface looks something like this:

#!/usr/bin/perl -w

use SimpleParallel;

my $parallel = SimpleParallel->new;

my ($return_variable1, $return_variable2);

$parallel->processes($return_variable1, sub { sleep(5); return ["hi", "one"]; });
$parallel->processes($return_variable2, sub { sleep(5); return ["hi", "two"]; });

$parallel->execute;

use Data::Dumper;
print "From Process 1n"; print Dumper $return_variable1;
print "n----n";
print "From Process 2n"; print Dumper $return_variable2;
print "n----n";

The idea being to add processes with return variables (references, bound the each process). Then when ready, execute them in parallel. Running this with time show us the process takes roughly 5 seconds (as it should):

$ time perl ./parallel.pl
From Process 1:
$VAR1 = [
          'hi',
          'one'
        ];

----
From Process 2:
$VAR1 = [
          'hi',
          'two'
        ];

----

real    0m5.042s
user    0m0.036s
sys     0m0.000s

The entire library, including POD, is only 150 lines of code. You can download it here: simpleparallel.pm

We have a few new announcements and interesting articles about RallyClock, so stay tuned.