Recently in Dev Category

Tkinter

|
As part of my continuing effort to hate Python less, I decided to do some fiddling with the Python Tkinter library.  For those of you playing along at home, it's a series of libraries (or whatever Python calls them) for performing Tk/Tcl tasks.  

I've never done GUI coding before.  Up to this point, all my code's execution space was limited to browsers and the occasional CLI tool.  Like one would expect from an object-oriented language, the GUI behaves in much the same fashion, with 'widgets' as they are called (a button, a menu, text field, etc.) descend from a root frame.

from Tkinter import *  # Get the library

class App:

    # Draw GUI and init
    def __init__(self, master):

        # Parent frame
        frame = Frame(master)

        # The pack() method returns an object of type None so Frame(master).pack() will only work if you don't want to reference 'frame' again.
        frame.pack()
   
        # Labels are simply output areas
        self.display_text = Label(frame,text="")
   
        # side=TOP - Sets the widget's placement
        # TOP, BOTTOM, LEFT, and RIGHT are constants set by Tk
        self.display_text.pack(side=TOP)

        # Create a button
        # Args:
        #  frame - Set 'frame' as the parent of the button
        #  text - an attribute setting the text on the button
        #  command - the callback which will fire when the button is clicked.
        self.button = Button(frame, text="This is a button", command=self.a_method)
        self.button.pack(side=LEFT)

    # A callback method
    def a_method(self):
   
        # the config() (or configure() ) method is used to modify widget options
        self.display_text.config(text="Button pressed")

#Yet another thing I hate about Python - if the next four lines are at the top of the file, this doesn't work.
root = Tk() #Instantiate the root widget
root.title('Test App')  # ...and give the window a title
app = App(root)  #Instantiate the App class, which contains all the logic
root.mainloop() # Start the event handling loop

Much like in CSS, object placement seems to get exponentially more troublesome as the number of objects grow. 

While so far the only thing I've done with this is write a utility for work (and thereby getting some more practice with Python's xmlrpclib methods) I think I'll definitely have to keep messing around with this.

Play around with it a bit and let me know what you end up creating!

Python *sigh*

|

After years of dislike, I may have to admit that I'm not entirely filled with hatred for Python. This did not come voluntarily, mind you. My love of laziness naturally breeds a lot of small projects; why repeat a task over and over when I can code something once and not have to think about it again?

My love of laziness was thrilled when I introduced it to web services some years ago. Now there was a place for my apathetic side as well - why bother learning what over people have coded, when all I have to do it point my code to a web service (ReST was a natural choice *rimshot*)?

A situation arose where there only real way to get something done without reinventing the wheel (avoiding a joke about things being chowned and chgrped to 'wheel') was to consume a web service. Finding a chunk of code where the consumption was already written was easy, but it happened to be in Python. I swallowed hard and decided to use it anyway, creating a Frankensteinian script wherein PHP passed arguments to a Python script which made the web service call and passed the results to a Ruby script (I never managed to work in Erlang).

As the benefit of that particular script became apparent, I made use of the Python section again and again. I finally realized that it was impossible to do any sort of sane version control since there were at least three script files to be updated per 'script'.

On one particularly slow day I decided to try to rewrite one of the aging scripts in pure Python. It worked and was a lot cleaner. I've written some subsequent scripts in pure Python as well, and a lot of my dislikes remain:

  • I hate converting every single number to a string before concatenating it to another string
  • Python, your implementation of lists is hideous
  • I don't want to have to track eleventy billion import statements
  • The whitespace is just annoying. I know how to write clean-looking code. Sometimes I just don't want to.
  • Get a damn case command. Virtually every other language in common use has one.
  • As I've said before: Twisted Python (which admittedly is not part of the Python core) tries way too hard to behave like Erlang. Just learn Erlang and be done with it.
  • String manipulation is the most counter-intuitive crap I've ever seen. x86 assembly makes more sense

That being said, it's just possible that perhaps Python isn't quite as evil as I once thought. Python, at least you're not Java.

  • FINALLY a language (that I'm aware of) where True !== 1 and False !== 0. I'd have to take off YOUR shoes to count the number of times that's bitten me in the ass. For people who don't know me, I wear sandals so no shoe-taking-offing is ever needed.

  • The it primitive is pretty neat and has possibilities:

    True || False
    >>True
    it
    >>True
    not it
    >>False


  • I like the /= representation of !=

  • Hmmm...lists are interesting.
    let x = [1,2]
    [1,2]
    0:x
    [0,1,2]

    So clearly the : primitive appends elements to the beginning of a list. I thought that perhaps [0,1]:x might produce [[0,1],[1,2]]. It turns out this is not the case, and instead throws an error that essentially amount to a data type mismatch. To achieve the desired result, one must type:

    [0,1]:[x]

    ...such that both arguments are lists, I assume. Weird.

  • putStrLn "Hi"
    Hi
    putStrLn 'Hi'

    :1:11:
    lexical error in string/character literal at character 'i'

    D'oh. I prefer to use single quotes with my strings. Not a deal breaker though.

  • Well, as thrilling as I'm sure this has been for you, I'm going to stop and actually get to coding.

So after a full reinstall of Leopard, my Mac seems to be behaving a bit more like it should. I wonder what the odds are of getting a corrupt OS from the factory.

MacPorts seems like a nice idea, and the list of software is decent. My two complaints (what, you thought I wouldn't have any?):

  • 1) It's slow as hell. I've been installing Haskell for the last twenty minutes.
  • The .dmg-based installer forgets one minor detail, namely changing the $PATH variable. Looking through the instructions for compiling from source, it seems that adding

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

to .profile or .bashrc fixes the issue. I'm sure there's a more elegant approach, but this allowed me to get started on the lengthy process of installing something that would be a single command on Ubuntu.

Erlang Spine Operational

|

I can't remember if I mentioned this earlier or not, and quite frankly I'm too lazy to look. So here's the idea. My servers do their internal communication on separate gigabit backbone, leaving the 100Mb NICs free to handle the rest of the traffic. I'd love to have a 10Gb backbone, but I can't afford a Cisco 6500 at the moment. Or possibly ever.

Anyway, the spine takes advantage of Erlang's ability to interconnect nodes running on different machines virtually transparently. The code is nothing more than a complicated wrapper around Erlang's '!' primitive, which is used to send messages between nodes.

My initial use for it is to send relevant entries from the BIND logs on the DNS server to the web server for easy display on the secondary DNS hosting site that's nearing completion. While this is a very simple application, there are plenty of other uses. After a few security-related tweaks, a node will be placed on our back-up cluster in Dallas/Fort Worth for doing backups.

The nice thing about having an intelligent spine is the each node can report back on the status of it's host machine. If, for example, a web server was experiencing difficulties, it could communicate that to a fail-over server which could take over until the problem could be resolved. It's sort of like a cloud concept - there are merely servers that behave however they have to.

Since Erlang nodes can pass instructions to other nodes, the network could essentially re-structure itself if needed.

To get a better idea of what it's like, Kevin Smith has a fabulous series of screencasts on the Pragmattic Programmers site. Go and buy them.

Kevin, if you're reading this and would like to team up on a screencast, let me know.

Quick update:

|

I mentioned in the last entry that Jan Lehnardt was also talking about gen_server, and his article is now up:
Read it here.

Those of you playing along at home may have gotten the impression that I kinda like this whole Erlang thing. Correct. Those of you playing along at home (and wayward surfers as well) from outside of Europe may be wondering why there's a bothersome 'u' lurking in the spelling of 'behaviour'. Erlang came from Ericsson (who denies the connection between 'erlang' and 'ERicsson LANGuage'. They claim it was named after a mathematician of the same name.) and Ericsson just so happens to reside in Scandanavia.

Linguists theorize that they are allowed to put the letter 'u' wherever they please as a trade-off for having to use that weird o-with-a-line-through-it letter. Or maybe they just want to show the word how much they love empty sets. Regardless, that being said let's hope into gen_server.

gen_server is a component of OTP, the Open Telecom Platform. OTP can be thought of as a sort of application framework for Erlang, much as Python has Pylons, Ruby has Rails, and Windows has Problems Crashing. OTP is hilariously powerful, and being the relative new-comer that I am, I've only just started to really grasp just how powerful it is. To quote Joe Armstrong, author of the fantastic book "Programming Erlang":

    The power of OTP comes from the fact that properties such as fault tolerance, scalability, dynamic-code upgrade, and so on, can be provided by the behavior itself. In other words, the writer of the callback does not have to worry about [those things] because this is provided by the behavior.

Okay Philip, what's a behaviour? A behaviour is just what it sounds like - a bucket containing all the common behaviors of a certain kind of system or platform.

In the great tradition of contrived examples, imagine a movie store that would only let you check out one movie at a time. The code for that might look something like what I've included below. Just like last time, first the code and then the explanation. My eternal thanks go out to Jan Lehnardt of the CouchDB project for his assistance, patience, and friendship. In the near future, he's going to be posting a plok entry on this same subject, so by all means go see if that's there. And give him money.

  -module(movie_store).
  -behaviour(gen_server).
 
  -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
 
  -export([checkout/2, lookup/1, start_link/0]).
 
  start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
  checkout(Customer, Movie) -> gen_server:call(?MODULE, {checkout, Customer, Movie}).
  lookup(Customer) -> gen_server:call(?MODULE, {lookup, Customer}).

  init([]) ->
       Tab = ets:new(?MODULE, []),
       {ok, Tab}.
 
  handle_call({checkout, Customer, Movie}, _From, Tab) ->
       Response = case ets:lookup(Tab, Customer) of
         [] -> 
               ets:insert(Tab, {Customer, Movie}),
               {ok, Movie};
         [{Customer, OtherMovie}] ->
               {already_checked_out, OtherMovie}
       end,
       {reply, Response, Tab};
 
  handle_call({lookup, Customer}, _From, Tab) ->
        Reply = case ets:lookup(Tab, Customer) of
               [{Customer, Movie}] ->
                   Movie;
               [] ->
                   none
        end,
        {reply, Reply, Tab}.
 
  handle_cast(_Msg, State) -> {noreply, State}.
  handle_info(_Msg, State) -> {noreply, State}.
  terminate(_Reason, _State) -> ok.
  code_change(_OldVersion, State, _Extra) -> {ok, State}.

Okay - it looks terrifying (which incidentally, is Erlang's super power) but it's not. The first line declares the module name (and 'movie_store.erl' should also be the filename) and is a standard part of every Erlang program.

-behaviour(gen_server).can be thought of a line that declares which 'template' we'll be using for this program. The gen_server behaviour mandates that we define the functions specified in the first set of exports.

-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). are the functions that gen_server expects to find (and will loudly complain if they are absent). I'll delve into more detail on these in the next tutorial, as this is going to be a huge entry as it is. For now, know that all of these have to be defined, even if they do nothing. As you can see at the bottom of the code, handle_cast/2, handle_info/2, terminate/2, and code_change/3 all do precisely nothing.

-export([checkout/2, lookup/1,start_link/0]). makes the custom functions that the server uses available to the outside world. They are in a separate -export statement only for clarity and can be combined with the one above.

start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

The starting point for the program. This function as you can see, serves as a wrapper for the command that spins up the server. {local, ?MODULE} shows that the server will be run locally and will not be available to other Erlang nodes in the cluster (as opposed to global). ?MODULE is a stand-in for the name for the current program name and specifies what we'll use to reference the server. The second ?MODULE tells the server where to find the callbacks (in this case, the same file). The remaining options allow you to turn debugging on, log to a file, and a few other things not used in this example.

checkout(Customer, Movie) -> gen_server:call(?MODULE, {checkout, Customer, Movie}).
lookup(Customer) -> gen_server:call(?MODULE, {lookup, Customer}).

These lines tie in the custom functions and tell gen_server what to do when (in this case) movie_store:checkout(...) and movie_store:lookup(....) are called. ?MODULE indicates where the function is, and {some_name,Arg1, Arg2,...} gives the function name and the arguments to be passed.

  init([]) ->
       Tab = ets:new(?MODULE, []),
       {ok, Tab}.
creates an database table using ets, the built-in memory database and returns it. In a deployed program, this would probably be something a bit more...permanent. A key component of a gen_server program is the program state. In this example, you'll see Tab getting thrown around into everything as it contains the present state of the movie store (who has which movie checked out).

  handle_call({checkout, Customer, Movie}, _From, Tab) ->
       Response = case ets:lookup(Tab, Customer) of
         [] -> 
               ets:insert(Tab, {Customer, Movie}),
               {ok, Movie};
         [{Customer, OtherMovie}] ->
               {already_checked_out, OtherMovie}
       end,
       {reply, Response, Tab};
Remember the gen_server:call(....) statements a little bit ago? This is what gen_server calls. You might wonder how there can be two handle_call functions defined with the same arity (the number of arguments, represented by the /x as in "functioname/2"). Look at the first argument, the three element tuple. Now look again at

checkout(Customer, Movie) -> gen_server:call(?MODULE, {checkout, Customer, Movie}).

The first elements match, which is how gen_server knows which handle_call to call to handle...the..call. Waay too many 'handle's and 'call's in that last bit. The important thing is that, like a lot of other things in Erlang, is based on pattern matching.

Notice that the table, Tab, has also been passed in. We go right into a case/of block that checks the table for the customer to see if they already have a movie checked out. If nothing is found ([] ->...) the movie is checked out to them and {ok, moviename} is returned by the block. If they already have a movie checked out, {already_checked_out, moviename} is returned by the block and they are not allowed to check out another movie (note: for this example, there is no way for a customer to return a movie. That is left as an exercise for the reader.).

In either case, the block's return value is stuffed into a tuple and shot back at whoever called the function. Before moving on to the next handle_call(...), notice the last line: {reply, Response, Tab};. You'd probably expect a period and not a semi-colon. The clauses of the handle_call(...) block are semi-colon delimited and will not work if they are terminated like other functions. I found that one out the hard way.

  handle_call({lookup, Customer}, _From, Tab) ->
        Reply = case ets:lookup(Tab, Customer) of
               [{Customer, Movie}] ->
                   Movie;
               [] ->
                   none
        end,
        {reply, Reply, Tab}.
This handle_call/3 defines a function to check and see if someone has a movie checked out. If so, it returns a tuple containing the customer's name and the movie title. Otherwise, the word 'none' is returned. I'm not going to spend a lot of time on this since it's so similar to the handle_call/3 we just looked at.

  handle_cast(_Msg, State) -> {noreply, State}.
  handle_info(_Msg, State) -> {noreply, State}.
  terminate(_Reason, _State) -> ok.
  code_change(_OldVersion, State, _Extra) -> {ok, State}.
The block of functions that need to be there for gen_server to work, but we're not using this time around. I'll cover these in a later tutorial. A quick note though - code_change/3 is really, really cool. It allows the server's code to be hot-swapped while the server is running. After all, why bother bringing a server down just because the code is changing, right?

Finally, here's how to run it:
2> c(movie_store).
{ok,movie_store}
3> movie_store:start_link().
{ok,<0.42.0>}
4> movie_store:checkout(phil, "Sneakers").
{ok,"Sneakers"}
5> movie_store:lookup(phil).
"Sneakers"
6> movie_store:checkout(phil, "Koyaanisqatsi").
{already_checked_out, "Sneakers"}
7> movie_store:lookup(phil).
"Sneakers"

One problem is that the program doesn't compliment you on your choice of movie if you try to check out 'Koyaanisqatsi'. Well, I've been long-winded enough for one day. I hope this was at the very least, interesting for you. If you improve on this, change it around, or whatever, by all means let me know.

Rerlang

| | Comments (2)

One thing I hate doing in Ruby (or any language for that matter) is setting up a web service or anything that requires a read/write socket. That's one of the reasons I like Erlang so much - the messaging takes a lot of the hassle out of the equation.

You build a module, register it (so you can address the module by name instead of PID), and send messages (see my tutorial a few entries ago). I've also become interested in the message bus idea. All concerned processes are tied into a communication circuit wherein they listen for messages addressed to them and send replies as appropriate.

Erlang offers fantastic stability (one project that involved 2 million lines of Erlang code achieved a nine nines reliability, which is absolutely unheard of) but is somewhat counter-intuitive when you're getting started. Ruby (or Python, PHP, etc) is very fast to write and generally makes sense but I find it's a pain in the ass to do network I/O. God help you people who do it in C or Java.

So I got to thinking that having an Erlang-powered 'spine' may be just the thing for a SOA. If one bites the bullet and writes an Erlang back-bone to handle the interconnectivity between machines and act as a central nerve fiber between interpreted language processes.

The hard part in my mind would be adding a smooth interface between Ruby and Erlang but once that's done it's trivial to hang another Erlang endpoint off the bus to do the talking.

I'm probably not describing this well and as such it may come out sounding really, really stupid. But I think it will at the very least be a good learning experience. The more I work with Erlang the more I like it and the more potential I see in it. If you haven't looked at it, I strongly suggest you at least give it a try. It can look very strange and will challenge some of the basic ways you think about coding, but it is very worth it.

Messaging in Erlang

|

Erlang is a language that has fascinated me for lo, these last few weeks. For the unfamiliar, it was developed to implement high-availability, distributed, fault-tolerant systems for the telecom world decades ago and in recent years has witnessed a resurgence as multi-core processors become the norm, rather than the exception.

At first, Erlang is bit daunting. Variables aren't, line endings vary by location, and recursion is common-place, among others. After a few hours of playing with the language however, these 'problems' turn into 'features' and you begin to see the benefits of the language.

A key component of Erlang are processes. Processes, unlike threads in other languages, have no shared environment or data. The only manner of communicating things between processes is through messaging. Processes listen for messages addressed to them and take appropriate action. I've been playing with inter-processes messaging and wanted to share a simple example. First, I'll list the code and then walk you through it. The code is based on the 'Getting Started' tutorial at http://www.erlang.org

The Code:

-module(messagetest).
-export([start/0, first/2, second/0]).

first(N, SecondPID) ->
  SecondPID ! {N + 1, self()},
  receive
    continue ->
      io:format("First Continuing... ~n", []),
      first(N + 1, SecondPID);
    stop ->
      io:format("First Received 'Stop' command.  Stopping.~n", []),
      SecondPID ! done
  end.

second() ->
  receive
    done ->
      io:format("Second Received 'done'~n", []);
    {N, FirstPID} ->
      case (N < 10) of
        false -> FirstPID ! stop;
        true -> FirstPID ! continue
      end,
      io:format("Second got ~p ~n", [N]),
      second()
  end.

start() ->
  SecondPID = spawn(messagetest, second, []),
  spawn(messagetest, first, [0, SecondPID]).

The Explanation:

The first two lines define the module name and give a directory of the functions that are externally callable along with their argument counts. The last three lines are where the fun begins. Save the code in a file named 'messagetest.erl' and from the Erlang interpreter, compile it by typing:

c(messagetest). 

Next, run the 'start' function by typing:

messagetest:start().

You'll see output similar to:

<0.71.0>Second got 1 

First Continuing... 
Second got 2 
First Continuing... 
Second got 3 
First Continuing... 
Second got 4 
First Continuing... 
Second got 5 
First Continuing... 
Second got 6 
First Continuing... 
Second got 7 
First Continuing... 
Second got 8 
First Continuing... 
Second got 9 
First Continuing... 
Second got 10
First Received 'Stop' command.  Stopping. 
Second Received 'done'

The <0.71.0> is the process id (not to be confused with a *nix PID) and will very likely be different on your machine. Let's look at the 'start' function.

start() ->
  SecondPID = spawn(messagetest, second, []),
  spawn(messagetest, first, [0, SecondPID])

The first line is the function definition. Simple enough. The second line executes the spawn command which kicks off an Erlang process that executes the function called 'second' in the 'messagetest' module. No parameters are passed ('[]'). The return value of spawn is the process ID of the started process.

The second line also spawns another Erlang process, passing it a value of 0 as well as the process ID of the process that was just spawned.

Ironically enough, we'll examine the function called 'second' first as it's a bit easier to understand, I think. 'second' consists entirely of a receive/end block who's job is to receive messages passed to the function. Inter-process messages are sent via:

process_id_of_recipient ! message

When a process receives a message, it is placed in a FIFO queue and the next message is examined when a receive/end is encountered. Message processing is very similar to a case/switch in other languages in that the message is compared against a list of expected values and when a match is found the corresponding code is evaluated. If no match is found, the next message is examined while keeping the first message in the queue. If the second doesn't match, the third is examined and so on until either a message matches at which point it is removed from the queue, or there are no more messages. In the case of the latter, the process blocks and waits for a message it knows how to process.

second() ->
  receive
    done ->
      io:format("Second Received 'done'~n", []);
    {N, FirstPID} ->
      case (N < 10) of
        false -> FirstPID ! stop;
        true -> FirstPID ! continue
      end,
      io:format("Second got ~p ~n", [N]),
      second()
  end.

Here we can see that the function is written to respond to either the word 'done' or a tuple matching a {x,y} pattern. In the case of the word 'done' for example, a notice is printed and the function ends.

In the event the second pattern matches the message, the value of 'N' is evaluated to see if it's less than 10. If it isn't, a 'stop' message is sent to whatever process has a PID of 'FirstPID' - I'll show where this comes from in a moment. If it is, the message 'continue' is sent instead.

In either case, the number it received in the variable 'N' is printed and the function calls itself to begin listening again. Now over to the 'first' function.

first(N, SecondPID) ->
  SecondPID ! {N + 1, self()},
  receive
    continue ->
      io:format("First Continuing... ~n", []),
      first(N + 1, SecondPID);
    stop ->
      io:format("First Received 'Stop' command.  Stopping.~n", []),
      SecondPID ! done
  end.

'first' expects two parameters, a number and a PID. As you recall from the 'start' function, this information was supplied by calling:

spawn(messagetest, first, [0, SecondPID]).

'first' uses:

SecondPID ! {N + 1, self()},
to send a message to the process with a PID of 'SecondPID' consisting of the number it received plus one and it's own PID. It then enters a receive/end loop to wait for a response from the 'second' function. As you remember, 'second' checks to see if this number is greater than 10 and sends either 'stop' or 'continue' accordingly. In this case, the number is 1 so 'second' tells 'first' to continue.

This matches the

continue ->
  io:format("First Continuing... ~n", []),
  first(N + 1, SecondPID);

block, so a message is printed and 'first' calls itself passing N + 1 and the PID of the 'second' function's process. This cycle repeats until 'first' tells 'second' it's reached the number 10.

At this point, 'second' send a message consisting of 'stop' back to 'first'.

stop ->
  io:format("First Received 'Stop' command.  Stopping.~n", []),
  SecondPID ! done

'first' processes the message and prints that it is shutting down. Finally, the message 'done' is sent back to 'second'. 'first' shuts itself down as it has nothing left to do.

done ->
  io:format("Second Received 'done'~n", []);

'second' receives the 'done' message from 'first', prints an acknowledgment, and decides it's going out for a beer or something so it shuts down. The program terminates.

There's a lot going on and it can take some time to wrap your head around it, but probably significantly less time than you might think. I find Erlang to be really enjoyable to write, although I think the error messages could use some work. The people who wrote Erlang apparently founded the 'Obscure Error Message College' (note: in the 1990s, it was renamed to 'The Javascript "Has No Properties" Academy').

I hope this was useful to someone and that you enjoy working in Erlang.

Ruby Vector extensions, DNS

|

Finally got all the zones configured properly and transferred to my DNS server. In celebration, here are some additions to the Vector class. Enjoy.


require 'matrix'

class Vector
 
  def include?(search_term)
    self.to_a.include?(search_term)
  end
   
  def to_float
    new_elements = []
    self.to_a.each do |element|
        element = element.to_f if element.class == Fixnum
        new_elements << element
      end
          
      Vector.elements(new_elements)
  end

  def pretty  
    counter = 0 
    self.map do |element|
      puts "[#{counter}] = #{element}"
      counter += 1
    end
  end

  def empty?
    return true if self.nil?  
    empty = true
    self.map do |element|
      if case(element)
           when Fixnum
             element == 0
           when nil
             true
           when String
             element.empty? or element == ' '
           when Float
             element == 0.0
           end
        next
      else
        empty = false
        break
      end
    end

    empty
  end

  def self.random(size, max_value = 50)
    elements = []
    (1..size).each do |i|
      elements << rand(max_value) + 1
    end

    Vector.elements(elements)
  end

end

About this Archive

This page is a archive of recent entries in the Dev category.

Linguistics is the next category.

Find recent content on the main index or look in the archives to find all content.