Dev: April 2008 Archives

Those of you playing along at home may have gotten the impression that I kinda like this whole Erlang thing. Correct. Those of you playing along at home (and wayward surfers as well) from outside of Europe may be wondering why there's a bothersome 'u' lurking in the spelling of 'behaviour'. Erlang came from Ericsson (who denies the connection between 'erlang' and 'ERicsson LANGuage'. They claim it was named after a mathematician of the same name.) and Ericsson just so happens to reside in Scandanavia.

Linguists theorize that they are allowed to put the letter 'u' wherever they please as a trade-off for having to use that weird o-with-a-line-through-it letter. Or maybe they just want to show the word how much they love empty sets. Regardless, that being said let's hope into gen_server.

gen_server is a component of OTP, the Open Telecom Platform. OTP can be thought of as a sort of application framework for Erlang, much as Python has Pylons, Ruby has Rails, and Windows has Problems Crashing. OTP is hilariously powerful, and being the relative new-comer that I am, I've only just started to really grasp just how powerful it is. To quote Joe Armstrong, author of the fantastic book "Programming Erlang":

    The power of OTP comes from the fact that properties such as fault tolerance, scalability, dynamic-code upgrade, and so on, can be provided by the behavior itself. In other words, the writer of the callback does not have to worry about [those things] because this is provided by the behavior.

Okay Philip, what's a behaviour? A behaviour is just what it sounds like - a bucket containing all the common behaviors of a certain kind of system or platform.

In the great tradition of contrived examples, imagine a movie store that would only let you check out one movie at a time. The code for that might look something like what I've included below. Just like last time, first the code and then the explanation. My eternal thanks go out to Jan Lehnardt of the CouchDB project for his assistance, patience, and friendship. In the near future, he's going to be posting a plok entry on this same subject, so by all means go see if that's there. And give him money.

  -module(movie_store).
  -behaviour(gen_server).
 
  -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
 
  -export([checkout/2, lookup/1, start_link/0]).
 
  start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
  checkout(Customer, Movie) -> gen_server:call(?MODULE, {checkout, Customer, Movie}).
  lookup(Customer) -> gen_server:call(?MODULE, {lookup, Customer}).

  init([]) ->
       Tab = ets:new(?MODULE, []),
       {ok, Tab}.
 
  handle_call({checkout, Customer, Movie}, _From, Tab) ->
       Response = case ets:lookup(Tab, Customer) of
         [] -> 
               ets:insert(Tab, {Customer, Movie}),
               {ok, Movie};
         [{Customer, OtherMovie}] ->
               {already_checked_out, OtherMovie}
       end,
       {reply, Response, Tab};
 
  handle_call({lookup, Customer}, _From, Tab) ->
        Reply = case ets:lookup(Tab, Customer) of
               [{Customer, Movie}] ->
                   Movie;
               [] ->
                   none
        end,
        {reply, Reply, Tab}.
 
  handle_cast(_Msg, State) -> {noreply, State}.
  handle_info(_Msg, State) -> {noreply, State}.
  terminate(_Reason, _State) -> ok.
  code_change(_OldVersion, State, _Extra) -> {ok, State}.

Okay - it looks terrifying (which incidentally, is Erlang's super power) but it's not. The first line declares the module name (and 'movie_store.erl' should also be the filename) and is a standard part of every Erlang program.

-behaviour(gen_server).can be thought of a line that declares which 'template' we'll be using for this program. The gen_server behaviour mandates that we define the functions specified in the first set of exports.

  init([]) ->
       Tab = ets:new(?MODULE, []),
       {ok, Tab}.

  handle_call({checkout, Customer, Movie}, _From, Tab) ->
       Response = case ets:lookup(Tab, Customer) of
         [] -> 
               ets:insert(Tab, {Customer, Movie}),
               {ok, Movie};
         [{Customer, OtherMovie}] ->
               {already_checked_out, OtherMovie}
       end,
       {reply, Response, Tab};

  handle_call({lookup, Customer}, _From, Tab) ->
        Reply = case ets:lookup(Tab, Customer) of
               [{Customer, Movie}] ->
                   Movie;
               [] ->
                   none
        end,
        {reply, Reply, Tab}.

  handle_cast(_Msg, State) -> {noreply, State}.
  handle_info(_Msg, State) -> {noreply, State}.
  terminate(_Reason, _State) -> ok.
  code_change(_OldVersion, State, _Extra) -> {ok, State}.

One thing I hate doing in Ruby (or any language for that matter) is setting up a web service or anything that requires a read/write socket. That's one of the reasons I like Erlang so much - the messaging takes a lot of the hassle out of the equation.

You build a module, register it (so you can address the module by name instead of PID), and send messages (see my tutorial a few entries ago). I've also become interested in the message bus idea. All concerned processes are tied into a communication circuit wherein they listen for messages addressed to them and send replies as appropriate.

Erlang offers fantastic stability (one project that involved 2 million lines of Erlang code achieved a nine nines reliability, which is absolutely unheard of) but is somewhat counter-intuitive when you're getting started. Ruby (or Python, PHP, etc) is very fast to write and generally makes sense but I find it's a pain in the ass to do network I/O. God help you people who do it in C or Java.

So I got to thinking that having an Erlang-powered 'spine' may be just the thing for a SOA. If one bites the bullet and writes an Erlang back-bone to handle the interconnectivity between machines and act as a central nerve fiber between interpreted language processes.

The hard part in my mind would be adding a smooth interface between Ruby and Erlang but once that's done it's trivial to hang another Erlang endpoint off the bus to do the talking.

I'm probably not describing this well and as such it may come out sounding really, really stupid. But I think it will at the very least be a good learning experience. The more I work with Erlang the more I like it and the more potential I see in it. If you haven't looked at it, I strongly suggest you at least give it a try. It can look very strange and will challenge some of the basic ways you think about coding, but it is very worth it.

Erlang is a language that has fascinated me for lo, these last few weeks. For the unfamiliar, it was developed to implement high-availability, distributed, fault-tolerant systems for the telecom world decades ago and in recent years has witnessed a resurgence as multi-core processors become the norm, rather than the exception.

At first, Erlang is bit daunting. Variables aren't, line endings vary by location, and recursion is common-place, among others. After a few hours of playing with the language however, these 'problems' turn into 'features' and you begin to see the benefits of the language.

A key component of Erlang are processes. Processes, unlike threads in other languages, have no shared environment or data. The only manner of communicating things between processes is through messaging. Processes listen for messages addressed to them and take appropriate action. I've been playing with inter-processes messaging and wanted to share a simple example. First, I'll list the code and then walk you through it. The code is based on the 'Getting Started' tutorial at http://www.erlang.org

The Code:

-module(messagetest).
-export([start/0, first/2, second/0]).

first(N, SecondPID) ->
  SecondPID ! {N + 1, self()},
  receive
    continue ->
      io:format("First Continuing... ~n", []),
      first(N + 1, SecondPID);
    stop ->
      io:format("First Received 'Stop' command.  Stopping.~n", []),
      SecondPID ! done
  end.

second() ->
  receive
    done ->
      io:format("Second Received 'done'~n", []);
    {N, FirstPID} ->
      case (N < 10) of
        false -> FirstPID ! stop;
        true -> FirstPID ! continue
      end,
      io:format("Second got ~p ~n", [N]),
      second()
  end.

start() ->
  SecondPID = spawn(messagetest, second, []),
  spawn(messagetest, first, [0, SecondPID]).

The Explanation:

The first two lines define the module name and give a directory of the functions that are externally callable along with their argument counts. The last three lines are where the fun begins. Save the code in a file named 'messagetest.erl' and from the Erlang interpreter, compile it by typing:

c(messagetest). 

Next, run the 'start' function by typing:

messagetest:start().

You'll see output similar to:

<0.71.0>Second got 1 

First Continuing... 
Second got 2 
First Continuing... 
Second got 3 
First Continuing... 
Second got 4 
First Continuing... 
Second got 5 
First Continuing... 
Second got 6 
First Continuing... 
Second got 7 
First Continuing... 
Second got 8 
First Continuing... 
Second got 9 
First Continuing... 
Second got 10
First Received 'Stop' command.  Stopping. 
Second Received 'done'

The <0.71.0> is the process id (not to be confused with a *nix PID) and will very likely be different on your machine. Let's look at the 'start' function.

start() ->
  SecondPID = spawn(messagetest, second, []),
  spawn(messagetest, first, [0, SecondPID])

The first line is the function definition. Simple enough. The second line executes the spawn command which kicks off an Erlang process that executes the function called 'second' in the 'messagetest' module. No parameters are passed ('[]'). The return value of spawn is the process ID of the started process.

The second line also spawns another Erlang process, passing it a value of 0 as well as the process ID of the process that was just spawned.

Ironically enough, we'll examine the function called 'second' first as it's a bit easier to understand, I think. 'second' consists entirely of a receive/end block who's job is to receive messages passed to the function. Inter-process messages are sent via:

process_id_of_recipient ! message

When a process receives a message, it is placed in a FIFO queue and the next message is examined when a receive/end is encountered. Message processing is very similar to a case/switch in other languages in that the message is compared against a list of expected values and when a match is found the corresponding code is evaluated. If no match is found, the next message is examined while keeping the first message in the queue. If the second doesn't match, the third is examined and so on until either a message matches at which point it is removed from the queue, or there are no more messages. In the case of the latter, the process blocks and waits for a message it knows how to process.

second() ->
  receive
    done ->
      io:format("Second Received 'done'~n", []);
    {N, FirstPID} ->
      case (N < 10) of
        false -> FirstPID ! stop;
        true -> FirstPID ! continue
      end,
      io:format("Second got ~p ~n", [N]),
      second()
  end.

Here we can see that the function is written to respond to either the word 'done' or a tuple matching a {x,y} pattern. In the case of the word 'done' for example, a notice is printed and the function ends.

In the event the second pattern matches the message, the value of 'N' is evaluated to see if it's less than 10. If it isn't, a 'stop' message is sent to whatever process has a PID of 'FirstPID' - I'll show where this comes from in a moment. If it is, the message 'continue' is sent instead.

In either case, the number it received in the variable 'N' is printed and the function calls itself to begin listening again. Now over to the 'first' function.

first(N, SecondPID) ->
  SecondPID ! {N + 1, self()},
  receive
    continue ->
      io:format("First Continuing... ~n", []),
      first(N + 1, SecondPID);
    stop ->
      io:format("First Received 'Stop' command.  Stopping.~n", []),
      SecondPID ! done
  end.

'first' expects two parameters, a number and a PID. As you recall from the 'start' function, this information was supplied by calling:

spawn(messagetest, first, [0, SecondPID]).

'first' uses:

SecondPID ! {N + 1, self()},

This matches the

continue ->
  io:format("First Continuing... ~n", []),
  first(N + 1, SecondPID);

block, so a message is printed and 'first' calls itself passing N + 1 and the PID of the 'second' function's process. This cycle repeats until 'first' tells 'second' it's reached the number 10.

At this point, 'second' send a message consisting of 'stop' back to 'first'.

stop ->
  io:format("First Received 'Stop' command.  Stopping.~n", []),
  SecondPID ! done

'first' processes the message and prints that it is shutting down. Finally, the message 'done' is sent back to 'second'. 'first' shuts itself down as it has nothing left to do.

done ->
  io:format("Second Received 'done'~n", []);

'second' receives the 'done' message from 'first', prints an acknowledgment, and decides it's going out for a beer or something so it shuts down. The program terminates.

There's a lot going on and it can take some time to wrap your head around it, but probably significantly less time than you might think. I find Erlang to be really enjoyable to write, although I think the error messages could use some work. The people who wrote Erlang apparently founded the 'Obscure Error Message College' (note: in the 1990s, it was renamed to 'The Javascript "Has No Properties" Academy').

I hope this was useful to someone and that you enjoy working in Erlang.

Finally got all the zones configured properly and transferred to my DNS server. In celebration, here are some additions to the Vector class. Enjoy.

require 'matrix'

class Vector
 
  def include?(search_term)
    self.to_a.include?(search_term)
  end
   
  def to_float
    new_elements = []
    self.to_a.each do |element|
        element = element.to_f if element.class == Fixnum
        new_elements << element
      end
          
      Vector.elements(new_elements)
  end

  def pretty  
    counter = 0 
    self.map do |element|
      puts "[#{counter}] = #{element}"
      counter += 1
    end
  end

  def empty?
    return true if self.nil?  
    empty = true
    self.map do |element|
      if case(element)
           when Fixnum
             element == 0
           when nil
             true
           when String
             element.empty? or element == ' '
           when Float
             element == 0.0
           end
        next
      else
        empty = false
        break
      end
    end

    empty
  end

  def self.random(size, max_value = 50)
    elements = []
    (1..size).each do |i|
      elements << rand(max_value) + 1
    end

    Vector.elements(elements)
  end

end

Taken from http://whois.domaintools.com/wordpress.com

Server Type: nginx/0.6.29

So it appears that Wordpress, which gets about 4 million hits a day has switched to an Nginx front-end! I really wish that tomorrow someone at work would say that no high-traffic sites uses Nginx. Of course I'd settle for an ignorant quip about how no there are no enterprise-level uses for Ruby or Rails.

It appears that the project I was on at work which was killed has risen from the proverbial ashes and is once again active.

I've been having DNS troubles since yesterday. First the serials weren't getting updated, then a full zone transfer wouldn't complete without an error, and now I've FINALLY eliminated all the errors in the DNS side of things. Not that it's working, of course. Now the trouble is that the external IPs of two of my boxes have been switched. So while it's resolving the domain names to the correct IP, that IP is tied to the wrong box. I should have it fixed by morning.

Apparently, it's tantamount to heresy to propose to use Nginx for anything Ruby-related other than Rails. I've scoured the internet (read: a good half-hour in between bites of Spaghettios) and I can't find anything on the subject. So, it looks like I'm going to have to 'figger it out my own sef'.

I doubt if either one of my readers has any experience on this front, feel free to chime in at any time.

Failing that, Ezra, on the off-chance you're reading this, send help. And sandwiches.

I had great plans for work this evening, but the better part of my night went to helping Roomie.male track down the reason why our zones weren't propagating (we think we have it, but we've got to wait and see).

I'm going to spend some time tomorrow digging into 'revactor', a gem which "...is an application framework for Ruby which uses the Actor model to simplify the creation of high-performance network services". In my unending attempt to get as low-level as possible with this stuff, I've started reading about π calculus. Initial impressions are that things like "∈ R/apple ∑ ramalama-ding-dong ⊗ 5% ♣ that's what she said" roughly translate to "Introduction". Still looks pretty cool, though.

I can't figure this out for the life of me...

The code below is for a simple logger class I wrote to go along with a personal project. To avoid having to write a ton of code I overrode method_missing thereby allowing calls like "logger_instance.emergency('emergency error text')".

For whatever reason, when method_missing-based calls are made, nothing happens. The code doesn't hang or crash, it just seems to skip right over the logging code and return. No errors are thrown that I can see, and it's really starting to bother me. Ideas?

class Utility::Log

  LOG_FILE = './operator_log.txt'

  def initialize
  end

  def method_missing(level, *mesg)
    log(mesg[0], level.capitalize)
  end

  def log(mesg, level = 'NOTICE')
    _new
    @logger.write(Time.now + "::#{level}::" + mesg + "\n")
    _close
  end

  private

  def _new
    if File.file?(LOG_FILE)
      @logger = File.open(LOG_FILE, 'a')
    end
  end

  def _close
    @logger.close
  end

end