NIC teaming and inexpensive switches

[OVERKILL]

Figured you guys might get a kick out of this.

One of my clients has a rather broad network topology with VPN links to remote sites at different locations throughout North America. Recently, I've rolled out DFS to create a common workspace share for all the offices.

One of the remote servers at one of our smaller sites is connected to a 24-port ASUS switch. We have an older Cisco 2924XL at the main office with four Intel NIC's teamed that has run flawlessly since it was deployed a number of years ago.

So, setting up the new server, with the same NIC setup (load balancing), I was getting an unreliable link to the server which would go up and down like a yo-yo. The VPN link however was solid.

I figured that perhaps the switch didn't support the NIC configuration, so I changed the configuration to link redundancy/fall-over. This immediately fixed the problem.

I love Cisco stuff, and use it when I can. Being a Cisco partner, it makes using their equipment a no-brainer.

However, there are situations where the price of a Cisco switch or router simply isn't in the budget for the client. Given the size of this deployment, we chose the less expensive route. And I learned something in the process

 

[javacontour]

Do you have to tell the switch that you are doing this?

I would think a switch that was looking at layers 2 and 3 would need to know if someone was attaching four NICs to the switch all using the same layer 3 address.

Don't know anything about the switch specifically. Just my general TCP/IP knowledge and some general switch awareness leads me to believe one would have to tell the switch you are bringing in N ports (where N is greater than one) that have the same IP address.

 

[simple_gifts]

Quote:

Don't know anything about the switch specifically. Just my general TCP/IP knowledge and some general switch awareness leads me to believe one would have to tell the switch you are bringing in N ports (where N is greater than one) that have the same IP address.


This was my believe also (but from years ago) For our mission critical apps, we are now plugging the teamed NICs into two separate switches. Not sure how this is configured.... I don't mind a single point of failure, I just don't want it to be my stuff 8)

 

[javacontour]

Solaris supports the same sort of thing. They've called it IPMP (IP multipathing) as well as other things. Trunking is also supported. I think trunking is analogous to teaming. If not, it's close.

Anyway, for IPMP, we have customers who use multiple switches, so if a switch goes down, the links all migrate to the other interfaces.

There must be some communication between the switches so they know about what is hanging off of each switch. One switch can be part of multiple VLANs, and I suppose a VLAN can be spread across multiple switches to provide redundancy.

I'm so far out of this aspect today, it's not funny.

I really do think CISCO equipment and IIRC, IOS, is the next thing I'll study to stay well rounded.

 

[OVERKILL]

Originally Posted By: javacontour
Do you have to tell the switch that you are doing this?

I would think a switch that was looking at layers 2 and 3 would need to know if someone was attaching four NICs to the switch all using the same layer 3 address.

Don't know anything about the switch specifically. Just my general TCP/IP knowledge and some general switch awareness leads me to believe one would have to tell the switch you are bringing in N ports (where N is greater than one) that have the same IP address.


No, you don't need to tell the switch. The Cisco is using an OOTB IOS configuration; just one giant VLAN right now.

There are a PILE of ways of configuring this setup. As was mentioned in the post above mine, teaming is very much like trunking. Of course trunking in the Cisco world also involves the mention of VLAN information passing between switches, but we don't need to get into that for this discussion.

Essentially, with four NIC's, you can set them to do:

1. Load balancing. As long as the two switches are trunked so the VLAN information passes between the two, you CAN use two switches for this setup. Otherwise, it is simply a way of giving the server more bandwidth. This is the setup I was using, since we only have one switch.

2. Load balancing with redundancy. You could use one or two switches in this situation as well. You have two NIC's plugged into each switch, one is a fall-over for the other in case a NIC goes down. Same info regardling VLAN's is relevant here, and trunking is necessary between the two switches.

3. Redundancy. ONE NIC is used as the LAN feed to the server. The others are all fall-overs. Again, you can use a trunked switch setup so that if the feeds to the one switch fail, the data will go through the trunk.

4. SLA/DLA. Both require 802.3ad support by the switch. Not even a consideration for consumer-grade switches.

 

[OVERKILL]

Originally Posted By: javacontour
Solaris supports the same sort of thing. They've called it IPMP (IP multipathing) as well as other things. Trunking is also supported. I think trunking is analogous to teaming. If not, it's close.

Anyway, for IPMP, we have customers who use multiple switches, so if a switch goes down, the links all migrate to the other interfaces.

There must be some communication between the switches so they know about what is hanging off of each switch. One switch can be part of multiple VLANs, and I suppose a VLAN can be spread across multiple switches to provide redundancy.

I'm so far out of this aspect today, it's not funny.

I really do think CISCO equipment and IIRC, IOS, is the next thing I'll study to stay well rounded.


VLAN information is trunked between switches for redundancy and distribution purposes. You can also use ROAS to route between VLAN's on the same switch if you enjoy or require that level of network separation.

 

[brianl703]

Originally Posted By: OVERK1LL

No, you don't need to tell the switch. The Cisco is using an OOTB IOS configuration; just one giant VLAN right now.


Seems to suggest that a bit of configuration is required:

http://www.cisco.com/en/US/docs/ios/12_2sb/feature/guide/sbcelacp.html#wp1053752

 

[OVERKILL]

Originally Posted By: brianl703
Originally Posted By: OVERK1LL

No, you don't need to tell the switch. The Cisco is using an OOTB IOS configuration; just one giant VLAN right now.


Seems to suggest that a bit of configuration is required:

http://www.cisco.com/en/US/docs/ios/12_2sb/feature/guide/sbcelacp.html#wp1053752



That is for SLA/DLA, which is what I said in point #4 in my post above yours. I'm not using SLA OR DLA.

 

[Brons2]

I find that server NIC teaming works much better with LACP configured on the switch to support it. $0.02.

Also, HP stuff is a good alternative to Cisco for the SMB market, HP's layer 3 switches are pretty cheap and they work well if you don't need ultra-advanced feature sets. If you just want to run a few subnets and VLANs, HP is a good option.

 

[brianl703]

One implementation does what could be called "round robin ARP", where the server replies to ARP requests with the MAC of each teamed NIC in turn. I can see where that wouldn't work so well for load-balancing incoming traffic if the incoming traffic is mostly coming from one device (a router).

 

[OVERKILL]

Originally Posted By: brianl703
One implementation does what could be called "round robin ARP", where the server replies to ARP requests with the MAC of each teamed NIC in turn. I can see where that wouldn't work so well for load-balancing incoming traffic if the incoming traffic is mostly coming from one device (a router).




Indeed. But would work well in a workgroup environment where most of the traffic is internal.

 

[OVERKILL]

Originally Posted By: Brons2
I find that server NIC teaming works much better with LACP configured on the switch to support it. $0.02.

Also, HP stuff is a good alternative to Cisco for the SMB market, HP's layer 3 switches are pretty cheap and they work well if you don't need ultra-advanced feature sets. If you just want to run a few subnets and VLANs, HP is a good option.


EtherChannel is in of itself a really interesting topic.

Indeed, PAgP and LACP are the "easiest" methods when using DLA or SLA; The link Brian posted earlier shows how to configure the switch for DLA or SLA using the latter.

In both of these environments I mentioned in this topic, switch bandwidth utilization is actually pretty low, so as I said, the IOS config on the 2924XL is pretty much OOTB other than being the most recent version and a few small configuration changes done. I'm not running SLA or DLA on the server, so I never saw a reason to bother.

I think (though am not sure) that Intel's implementations of dynamic load balancing and dynamic failover redundancy may be designed for use on non-managed hardware. Which is why it works with an OOTB config of IOS.

This would explain why they offer DLA and SLA as separate configuration options and explicitly make mention of 802.3ad support.

In environments that are more bandwidth intensive I find DLA or SLA to be a better choice over Intel's basic Balancing/Redundancy solutions. Of course this requires hardware that supports it.

Brons2: I have not used HP switches. Do you know who manufactures them for HP? With Cisco's recent push into the SMB market, I would find it very hard to deviate from their products.