Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Computing > Cisco > Mystery "ghost" MAC addresses?

Reply
Thread Tools

Mystery "ghost" MAC addresses?

 
 
John Oliver
Guest
Posts: n/a
 
      09-12-2008
While troubleshooting some firewall issues, I started to see some
mystery broadcast traffic:

15:22:20.192899 02:02:0a:00:0a:c8 (oui Unknown) > Broadcast, ethertype
Unknown (0x886f), length 1510:
0x0000: bf01 dec0 0402 0000 0200 0000 0a00 0ac8
.................
0x0010: 0a00 0a15 0100 0100 0100 0200 0f01 8542
................B
0x0020: a737 0000 0000 0000 0000 0000 00f0 ff6f
..7.............o
0x0030: 0000 0000 0000 0000 0000 0000 0000 0000
.................
0x0040: 0000 0000 0000 0000 0000 0000 0000 0000
.................
0x0050: 0000


I log in to the 2960 that's at the "heart" of this particular network
and use sh mac-address-table to find that MAC. It says it's on Gi0/6 I
follow that cable to a Dell 2748, so I log in to it... and it tells me
that MAC is on port 47, which leads back to the 2960! To make matters
worse, 02:02:0a is showing up as unassigned.

I've had issues like this with Cisco switches before, where mysterious
traffic seems to originate in the middle of wires. And I know others
have seen the same issue, because once in a while I get an email from
someone who saw an old post I made about it, asking if I ever found an
answer because they're tearing their hair out!

--
* John Oliver http://www.john-oliver.net/ *
 
Reply With Quote
 
 
 
 
Al
Guest
Posts: n/a
 
      09-13-2008
On Sep 13, 4:44*am, "Thrill5" <(E-Mail Removed)> wrote:
> First thoughts....
>
> The fact that the source MAC is "weird", the traffic is a broadcast, and is
> a full length packet (1510 bytes) leads me to believe that you have a device
> on your network that has a virus.
>
> I did work on a problem more than 12 years ago where we had bridges
> (CrossComm ILAN's) that replicated a randomly generated packet, but this was
> due to a bug in the bridge software. *I have been working with Cisco
> switches and routers for more than 10 years and I have never seen or heard
> of a problem like this with Cisco equipment. *I would suspect the "other"
> vendors equipment before Cisco.
>
> "John Oliver" <(E-Mail Removed)> wrote in message
>
> news:(E-Mail Removed).. .
>
> > While troubleshooting some firewall issues, I started to see some
> > mystery broadcast traffic:

>
> > 15:22:20.192899 02:02:0a:00:0a:c8 (oui Unknown) > Broadcast, ethertype
> > Unknown (0x886f), length 1510:
> > * * * *0x0000: *bf01 dec0 0402 0000 0200 0000 0a00 0ac8
> > ................
> > * * * *0x0010: *0a00 0a15 0100 0100 0100 0200 0f01 8542
> > ...............B
> > * * * *0x0020: *a737 0000 0000 0000 0000 0000 00f0 ff6f
> > .7.............o
> > * * * *0x0030: *0000 0000 0000 0000 0000 0000 0000 0000
> > ................
> > * * * *0x0040: *0000 0000 0000 0000 0000 0000 0000 0000
> > ................
> > * * * *0x0050: *0000

>
> > I log in to the 2960 that's at the "heart" of this particular network
> > and use sh mac-address-table to find that MAC. *It says it's on Gi0/6 *I
> > follow that cable to a Dell 2748, so I log in to it... and it tells me
> > that MAC is on port 47, which leads back to the 2960! *To make matters
> > worse, 02:02:0a is showing up as unassigned.

>
> > I've had issues like this with Cisco switches before, where mysterious
> > traffic seems to originate in the middle of wires. *And I know others
> > have seen the same issue, because once in a while I get an email from
> > someone who saw an old post I made about it, asking if I ever found an
> > answer because they're tearing their hair out!

>
> > --
> > * John Oliver * * * * * * * * * * * * * * *http://www.john-oliver.net/*


One time I saw some similar traffic it turned out to be to do with
network load balancing enabled on two NICs on two different servers.
The core switch started reporting a flapping mac-address & sure
enough, 1 second it would be on one uplink - to server A - and the
next it would be on another - to server B. As neither server had dual
NICs we just disabled the feature as it was unnecessary.
Unfortunately, I do not remember the MAC address & whether it was b/
cast or other. All I remember is that it wasn't a registered OUI & I
couldn't find anything about it online...
 
Reply With Quote
 
 
 
 
Phillip Remaker
Guest
Posts: n/a
 
      09-14-2008
Well, let try this as a wild guess: if the switch itself emitted a malformed
packet due to a bug, this could be the problem. The packet is pretty
clearly malformed, looking only at the ethertype. And it is a maximum sized
packet, which is suspicious for a broadcast that is not part of a stream.
Some switch seems to have a pointer to memory which is hoarked.

Let's presume it came from the Dell for the moment. The Cisco switch will
flood it (since it is a broadcast) and mark the interface where it heard the
broadcast (Gi0/6) as the source. The packet will flood and the Dell will
ultimately hear the packet back on that same port (depending on the state of
the spanning tree or nature of the bug that caused this packet), and it will
mark the interface where it heard it from. Since it has no way to know that
it generated this malformed packet itself, it adds it to the MAC address
forwaring table, presuming it must be a distant host.

This is how you get a "synthesized" packet in the middle of the wire - One
of the two switches emitted a malformed broadcast packet and did not
recognize the packet as its own when it came back.

The partial hex dump of the packet has no obvious clues (to me) about what
it is. Sometimes, these are clearly legitimate packets shifted by a number
of bytes. A more motivated person might play around with bitshifting the
hex dump or looking at a more complete hex dump to see if any recognizable
patterns emerge. But as near as I can tell, this is probably some kind of
bug in one of the switches, and it seems harmless as long as these emissions
are rare.

Are the MAC address entries timestamped? If they are timestamped with high
precision and the switches are synchronized, the one with the oldest MAC
address entry is probably the innocent one (since that switch heard the
impossible packet first).

Does the packet appear regularly? If you can somehow diagnose which side
"shoots first" you will be closer to understanding where it comes from.


 
Reply With Quote
 
Stephen
Guest
Posts: n/a
 
      09-14-2008
On Sat, 13 Sep 2008 00:26:53 +0200 (CEST), John Oliver
<(E-Mail Removed)> wrote:

>While troubleshooting some firewall issues, I started to see some
>mystery broadcast traffic:
>
>15:22:20.192899 02:02:0a:00:0a:c8 (oui Unknown) > Broadcast, ethertype
>Unknown (0x886f), length 1510:
> 0x0000: bf01 dec0 0402 0000 0200 0000 0a00 0ac8
>................
> 0x0010: 0a00 0a15 0100 0100 0100 0200 0f01 8542
>...............B
> 0x0020: a737 0000 0000 0000 0000 0000 00f0 ff6f
>.7.............o
> 0x0030: 0000 0000 0000 0000 0000 0000 0000 0000
>................
> 0x0040: 0000 0000 0000 0000 0000 0000 0000 0000
>................
> 0x0050: 0000
>
>
>I log in to the 2960 that's at the "heart" of this particular network
>and use sh mac-address-table to find that MAC. It says it's on Gi0/6 I
>follow that cable to a Dell 2748, so I log in to it... and it tells me
>that MAC is on port 47, which leads back to the 2960! To make matters
>worse, 02:02:0a is showing up as unassigned.


that Ethertype 886f "belongs" to Microsoft.
AFAIR it is something to do with server load balancing.
http://standards.ieee.org/regauth/et.../type-pub.html

i think when this is used the server uses a different source address -
but it makes it up from the NIC address.

Look to see if 1 of the source adr MACs on the server is the same
apart from a couple of the high order bits.

>
>I've had issues like this with Cisco switches before, where mysterious
>traffic seems to originate in the middle of wires. And I know others
>have seen the same issue, because once in a while I get an email from
>someone who saw an old post I made about it, asking if I ever found an
>answer because they're tearing their hair out!

--
Regards

http://www.velocityreviews.com/forums/(E-Mail Removed) - replace xyz with ntl
 
Reply With Quote
 
Phillip Remaker
Guest
Posts: n/a
 
      09-14-2008
> that Ethertype 886f "belongs" to Microsoft.
> AFAIR it is something to do with server load balancing.
> http://standards.ieee.org/regauth/et.../type-pub.html



Well, I guess you can disregard my last post, then!

With your clue, I Googled some more: Google "0x886f load balancing"

Protocol 0x886f is described at
http://www.iss.net/security_center/a...6f/default.htm

Which says:

Microsoft's Windows NT Load Balancing Service (WLBS), a clustering service.
Packets sent using this Ethertype have the following properties:

a.. The destination MAC address is either the broadcast (FFFFFFFFFFFF) or
a multicast address that is set in the registry.
b.. The source MAC address is 00 BF followed by the IP address of the
machine sending the packet. That machine also has a corresponding MAC
address of 02 BF ... that you can use to ping the machine with.
c.. Status messages appear about once per second per machine in the
cluster.

so from the original post it looks like it belongs to a server with an IP
address 0A:00:0A:C8 or 10.0.10.200


 
Reply With Quote
 
Sam Wilson
Guest
Posts: n/a
 
      09-16-2008
In article <(E-Mail Removed)>,
John Oliver <(E-Mail Removed)> wrote:

> While troubleshooting some firewall issues, I started to see some
> mystery broadcast traffic:
>
> 15:22:20.192899 02:02:0a:00:0a:c8 (oui Unknown) > Broadcast, ethertype
> Unknown (0x886f), length 1510:
> [snip]
>
> ... To make matters
> worse, 02:02:0a is showing up as unassigned.


One snippet that hasn't come up in the thread so far is that the second
least significant bit (0x02) of the first octet of a MAC address is the
"locally administered bit". That means that any MAC address with that
bit set is (supposed) not to be globally assigned but generated locally.
Except for a few historical oddities any address with that bit set is
not a universally registered OUI.

Sam
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Image quality on .mac (dot mac) eyalnevo@gmail.com Digital Photography 4 05-16-2006 03:07 PM
Mac OSX Dashboard mystery ram Javascript 0 03-13-2006 05:10 PM
Vertical Gap between DIVs in Opera 8.5 Mac + IE 5.2 Mac mangm HTML 2 12-01-2005 12:48 PM
mac!! mac!!! * * * Y o u r . S h e p h e r d . A q u i l a . D e u s . ( d 2 0 0 5 x x , d 2 0 0 4 x x , d 2 0 0 Computer Support 7 06-03-2005 12:48 AM
Senseless rendering: Mac.Mozilla != Mac.Netscape6.01 ?!?! Roman =?ISO-8859-15?Q?Bl=F6th?= HTML 1 07-02-2003 10:23 AM



Advertisments