Attack of the clones.

[hk0]

So we bought a bunch of SuperMicro NVidias motherboards, two of which were HD8CEs, which use the NForce 2200 MCP for gigabit ethernet.

And for some reason we can't ping one from the other. Spent about 4 hours trying to figure the damn thing out.
Finally we come to find out that arp isn't happy, and it keeps flushing it's cache.

So I get my coworker in there and ask him to sit at one, while I got the other, and we both bring up our MAC addresses.

And what do we see?
Machine 1:
Description . . . . . . . . . . . : NVIDIA nForce Networking Controller
Physical Address. . . . . . . . . : 04-4B-80-80-80-04
... snip ...
Description . . . . . . . . . . . : NVIDIA nForce Networking Controller #2
Physical Address. . . . . . . . . : 04-4B-80-80-80-03

Machine 2:
Description . . . . . . . . . . . : NVIDIA nForce Networking Controller
Physical Address. . . . . . . . . : 04-4B-80-80-80-04
... snip ...
Description . . . . . . . . . . . : NVIDIA nForce Networking Controller #2
Physical Address. . . . . . . . . : 04-4B-80-80-80-03

Well NO F**KING WONDER!

And a quick search of google reveals that we're not the only ones with this MAC address.
http://www.google.com/search?q=%2204+4B+80+80+80+03%22

Hilariously, the official linux PXEBOOT HOWTO has immortalized the MAC address as the MAC address of a PXEBOOT server in an example configuration. :rolleyes:

So quick show of hands, anyone else have an NForce (forcedeth) driver with a 04-4B-80-80-80-* address?

(also, of note, the "GUID" in the NVRAM of the adapter used for DHCP is 0002-0003-0004-0005-0006-0007-0008-0009, which is suspicious in it's own right. I think there was a large batch of chips that didn't get flashed correctly or left a step in the process out)

[retsam]

this doesnt sound right...if you have two of the same mac address, you wouldnt have any connectivty on the network.... is this linux of windows?....

[AthlonXP1800]

If your computers connected to a router, make sure you disable MAC clone on the router that should fix the issue.

[hk0]

this doesnt sound right...if you have two of the same mac address, you wouldnt have any connectivty on the network.... is this linux of windows?....

Well, no. What was happening was that each machine would periodically get connectivity at the other's expense (think unintentional MAC poisoning). The switch was probably pretty pissed that each machine kept trying to claim the same MAC as it's owned. It would explain why accessing the network from one of them would freeze the other. :-(

[hk0]

If your computers connected to a router, make sure you disable MAC clone on the router that should fix the issue.
?
These were two machines with identical MAC addresses (verified in the BIOS at boot time). It wasn't a router cloning an existing address. And that would only make the duplicate MAC visible on the WAN side (why would you connect the WAN side back to the NAT segment anyway?)

Besides, these are two machines out of many in a cluster in a switched office environment.