I can't remember if I mentioned this earlier or not, and quite frankly I'm too lazy to look. So here's the idea. My servers do their internal communication on separate gigabit backbone, leaving the 100Mb NICs free to handle the rest of the traffic. I'd love to have a 10Gb backbone, but I can't afford a Cisco 6500 at the moment. Or possibly ever.
Anyway, the spine takes advantage of Erlang's ability to interconnect nodes running on different machines virtually transparently. The code is nothing more than a complicated wrapper around Erlang's '!' primitive, which is used to send messages between nodes.
My initial use for it is to send relevant entries from the BIND logs on the DNS server to the web server for easy display on the secondary DNS hosting site that's nearing completion. While this is a very simple application, there are plenty of other uses. After a few security-related tweaks, a node will be placed on our back-up cluster in Dallas/Fort Worth for doing backups.
The nice thing about having an intelligent spine is the each node can report back on the status of it's host machine. If, for example, a web server was experiencing difficulties, it could communicate that to a fail-over server which could take over until the problem could be resolved. It's sort of like a cloud concept - there are merely servers that behave however they have to.
Since Erlang nodes can pass instructions to other nodes, the network could essentially re-structure itself if needed.
To get a better idea of what it's like, Kevin Smith has a fabulous series of screencasts on the Pragmattic Programmers site. Go and buy them.
Kevin, if you're reading this and would like to team up on a screencast, let me know.
