Philip Ratzsch: January 2009 Archives
In the spirit of previous primers, I thought I'd add one regarding CSS (Content Services Switch) content rules. But first, a caveat: this was written as an introductory guide only. I may have missed something, gotten something incorrect, or in some other way be responsible for the end of the world. I am not a CCNA, CCNP, CCIE, or CC-anything else. Apply at your own risk. Also, forgive my ASCII art - I couldn't get it looking quite the way I wanted.
Say you have a network arranged like this:
[Blogosphere/Intertubes]
|
|
[Intelligent weasel serving as a firewall]
|
[web server]
Suppose that you run a wildly successful website that allows you to purchase domesticated weasels (though not the one acting as your firewall - that one's worth millions). A single server may be fine initially, but eventually the demand for weasels overloads your server on a regular basis. Additionally, your fireweasel gets very tired.
You could take your server offline and add RAM, additional disk space, and a faster CPU to improve performance, but that's a solution that doesn't scale very well. Eventually, your server will be as fast as it can be, with as much RAM as it can hold and you'll still eventually run into the same problem as weasel demand grows.
After consulting a series of floor tiles in a local bar while mulling the problem over, it hits you. A boot. After spending several weeks recovering in a hospital, the SOLUTION hits you. A load balancer. The premise of a load balancer is that requests for content are spread over several computers in a server farm. A common load balancer is the Cisco 11500 family (11501, 11503, and 11506).
A basic layout might look something like this:
[Blogosphere/Intertubes]
|
|
[Ultra-wise vole acting as perimeter security]
|
[CSS]
|
__________ | ________
| | |
web1 web2 web3
A web application or service is associated with a virtual IP address (VIP) in your DNS records. This IP address is assigned to the CSS, which acts as a proxy for the servers sitting logically behind it.
In the example above, when HTTP requests come in, they are first confronted by the firevole. Once the vole has allowed the requests to proceed, they are intercepted by the CSS. The CSS then passes the traffic to one of the servers behind it by consulting a set of content rules; configuration commands specifying how the requests should be handled.
Talk about why round-robin DNS wouldn't do the same thing? Sure. In general, DNS resource records have TTLs of at least several hours, sometimes higher. Making a change to add a new server during a high-load period couldn't take affect until those TTLs had expired and remote caches had updated. Additionally, A records don't have one key piece of information that a CSS possesses; server status.
A CSS uses a series of keepalive requests to the server that allow it to remove a failed server from rotation. These keepalives can range from simple, periodic socket connections to more complex verifications of URI content - ie, does domain.com/test.html still say 'OK'. DNS is not able to determine server status, so requests could be routed to a dead server.
Additionally, round-robin DNS is precisely that; round-robin. Requests will be passed sequentially to the next server in line. Sometimes that's not what you want at all. If you have three servers that are all equally well-configured that may work fine, but suppose the servers are configured differently. web1 and web2 might be powerhouse servers while web3 is an older machine that can't handle as much load. The CSS can be configured to assign 'weights' to the servers allowing you to force web1 and web2 to each handle 40% of the requests while web3 servers only 20%. You get the idea - load balancers = good.
On to configuration. Each application or service to be load balanced is defined in a 'content rule', a set of configurations dictating exactly how the load balancing should work, and which servers (called 'services') are participating. Each content rule is an independent entity on the load balancer, allowing you to control it separately.
Since you might very well be hosting multiple clients behind a single load balancer, content rules can be grouped under 'owners'. Each content rule is a series of directives such as to use a leastconn load balancing algorithm (the server with the fewest active connections gets the next request), perform keepalive checks using a socket connection to port 80 on the servers, etc. Say your machines are IPed as follows (assume the servers and the internal IP of the CSS are on a 192.168.1.x/24 network. Here's how a sample request gets processed.
{step 1} {step 7}
|
[Ultra-wise vole acting as perimeter security] {step 2} {Step 6}
[CSS] (.1) {step 3} {step 5}
|
__________ | ________
| | |
web1(.2) web2 (.3) web3 (.4) {step 4}
Step 1: A request comes in to purchase a weasel on 9.9.9.4, a VIP on the CSS.
Step 2: The firevole performs NAT translations on the request, converting the IP addresses.
Step 3: The CSS intercepts the requests on the internal VIP and checks the content rules. The rules tell the CSS that the next request should be sent to web3, so the request is transferred there.
Step 4: web3 logs the purchase request and sends back a response.
Step 5: The CSS sees that the traffic is outbound and passes it back to the firevole.
Step 6: The firevole performs NAT translations again and shoots the response back to the client.
Step 7: One freshly-minted weasel owner.
Something may have jumped out at you. The CSS is not performing any type of NAT translations, and both it's inside and outside interface are part of the same network, using the same IP address. This normally wouldn't work in a standard network. However, here the CSS is operating in dispatch mode wherein it acts as a transparent bridge between two VLANs that have the same IP address scheme. The CSS can also operate in direct mode, but that in my experience is fairly rare.
Operating as a transparent bridge has both an up and a down side. The upside is that any new servers can be behind the CSS, even if they're not being load balanced. The CSS will bridge traffic and ARP for the server, acting like a switch. No additional configuration is required.
The downside is that acting a bridge automatically means the CSS must participate in spanning-tree protocol (STP) which is used to prevent layer 2 loops and the resultant frame storms which can bring a network to it's knees. Care must be taken to configure the CSS accordingly. I've seen cabinets in a test environment be brought down because this wasn't done.
Keep in mind that this is just one possible deployment of a CSS. There are many other ways of doing it, but this seemed to me to be the easiest to understand for the unfamiliar reader.
Before signing off, a quick thank you to those of you who have posted comments and sent emails from all the continents of the world, save one (I apparently am not popular in Antarctica). It's always good to hear from people who take the time not only to read but to respond. To those of you who are silently reading along, thanks to you too.
Say you have a network arranged like this:
[Blogosphere/Intertubes]
|
|
[Intelligent weasel serving as a firewall]
|
[web server]
Suppose that you run a wildly successful website that allows you to purchase domesticated weasels (though not the one acting as your firewall - that one's worth millions). A single server may be fine initially, but eventually the demand for weasels overloads your server on a regular basis. Additionally, your fireweasel gets very tired.
You could take your server offline and add RAM, additional disk space, and a faster CPU to improve performance, but that's a solution that doesn't scale very well. Eventually, your server will be as fast as it can be, with as much RAM as it can hold and you'll still eventually run into the same problem as weasel demand grows.
After consulting a series of floor tiles in a local bar while mulling the problem over, it hits you. A boot. After spending several weeks recovering in a hospital, the SOLUTION hits you. A load balancer. The premise of a load balancer is that requests for content are spread over several computers in a server farm. A common load balancer is the Cisco 11500 family (11501, 11503, and 11506).
A basic layout might look something like this:
[Blogosphere/Intertubes]
|
|
[Ultra-wise vole acting as perimeter security]
|
[CSS]
|
__________ | ________
| | |
web1 web2 web3
A web application or service is associated with a virtual IP address (VIP) in your DNS records. This IP address is assigned to the CSS, which acts as a proxy for the servers sitting logically behind it.
In the example above, when HTTP requests come in, they are first confronted by the firevole. Once the vole has allowed the requests to proceed, they are intercepted by the CSS. The CSS then passes the traffic to one of the servers behind it by consulting a set of content rules; configuration commands specifying how the requests should be handled.
Talk about why round-robin DNS wouldn't do the same thing? Sure. In general, DNS resource records have TTLs of at least several hours, sometimes higher. Making a change to add a new server during a high-load period couldn't take affect until those TTLs had expired and remote caches had updated. Additionally, A records don't have one key piece of information that a CSS possesses; server status.
A CSS uses a series of keepalive requests to the server that allow it to remove a failed server from rotation. These keepalives can range from simple, periodic socket connections to more complex verifications of URI content - ie, does domain.com/test.html still say 'OK'. DNS is not able to determine server status, so requests could be routed to a dead server.
Additionally, round-robin DNS is precisely that; round-robin. Requests will be passed sequentially to the next server in line. Sometimes that's not what you want at all. If you have three servers that are all equally well-configured that may work fine, but suppose the servers are configured differently. web1 and web2 might be powerhouse servers while web3 is an older machine that can't handle as much load. The CSS can be configured to assign 'weights' to the servers allowing you to force web1 and web2 to each handle 40% of the requests while web3 servers only 20%. You get the idea - load balancers = good.
On to configuration. Each application or service to be load balanced is defined in a 'content rule', a set of configurations dictating exactly how the load balancing should work, and which servers (called 'services') are participating. Each content rule is an independent entity on the load balancer, allowing you to control it separately.
Since you might very well be hosting multiple clients behind a single load balancer, content rules can be grouped under 'owners'. Each content rule is a series of directives such as to use a leastconn load balancing algorithm (the server with the fewest active connections gets the next request), perform keepalive checks using a socket connection to port 80 on the servers, etc. Say your machines are IPed as follows (assume the servers and the internal IP of the CSS are on a 192.168.1.x/24 network. Here's how a sample request gets processed.
{step 1} {step 7}
|
[Ultra-wise vole acting as perimeter security] {step 2} {Step 6}
[CSS] (.1) {step 3} {step 5}
|
__________ | ________
| | |
web1(.2) web2 (.3) web3 (.4) {step 4}
Step 1: A request comes in to purchase a weasel on 9.9.9.4, a VIP on the CSS.
Step 2: The firevole performs NAT translations on the request, converting the IP addresses.
Step 3: The CSS intercepts the requests on the internal VIP and checks the content rules. The rules tell the CSS that the next request should be sent to web3, so the request is transferred there.
Step 4: web3 logs the purchase request and sends back a response.
Step 5: The CSS sees that the traffic is outbound and passes it back to the firevole.
Step 6: The firevole performs NAT translations again and shoots the response back to the client.
Step 7: One freshly-minted weasel owner.
Something may have jumped out at you. The CSS is not performing any type of NAT translations, and both it's inside and outside interface are part of the same network, using the same IP address. This normally wouldn't work in a standard network. However, here the CSS is operating in dispatch mode wherein it acts as a transparent bridge between two VLANs that have the same IP address scheme. The CSS can also operate in direct mode, but that in my experience is fairly rare.
Operating as a transparent bridge has both an up and a down side. The upside is that any new servers can be behind the CSS, even if they're not being load balanced. The CSS will bridge traffic and ARP for the server, acting like a switch. No additional configuration is required.
The downside is that acting a bridge automatically means the CSS must participate in spanning-tree protocol (STP) which is used to prevent layer 2 loops and the resultant frame storms which can bring a network to it's knees. Care must be taken to configure the CSS accordingly. I've seen cabinets in a test environment be brought down because this wasn't done.
Keep in mind that this is just one possible deployment of a CSS. There are many other ways of doing it, but this seemed to me to be the easiest to understand for the unfamiliar reader.
Before signing off, a quick thank you to those of you who have posted comments and sent emails from all the continents of the world, save one (I apparently am not popular in Antarctica). It's always good to hear from people who take the time not only to read but to respond. To those of you who are silently reading along, thanks to you too.
