Image courtesy of Ambro / FreeDigitalPhotos.net
Here’s a quick test of your psychic troubleshooting skills. I’ll give you a real life scenario, and you try to figure out what’s wrong.
Sound like fun? Go on, give it a shot.
Feel free to ask questions via the comments, and I’ll update the ‘trouble ticket’ with your questions and any relevant answers. The scenario is pictured below and I’ve shared the details we had managed to gather in the first five minutes. If you need more information, the idea is that you ask for it!
Troubleshooting Scenario
Two Cisco routers have been connected to each other as follows:
- R1 is using a copper 10/100 Ethernet port hard coded for 100MB Full Duplex.
- R2 is using a copper 10/100/1000 port Ethernet port hard coded for 100MB Full Duplex.
- The media converters bridge between copper and fiber so that the routers can talk over a longer distance.
- If you unplug the Rx fiber on either media converter (i.e. it no longer sees light being received from the other end), it correctly ‘downs’ the local copper port and the connected router port goes down.
The Problem
Both routers’ Ethernet interfaces show UP UP, but they cannot ping each other.
- You may assume that the IP configuration is correct on both ends.
Side note: a colleague beat me to the solution on this one 🙂
Over to you – fire away!
UPDATED: The Solution
We had some good troubleshooting questions in the comments – very much following our own steps. Well done to Chuck Wellhausen though who saw something recently, as proposed this solution:
“I ran into something like this the other day. One side of the media converter had the speed set to 100Mb the other was 1000Mb. It would allow light to pass, the interfaces never went down and no MAC addresses ever populated. We had to change the DIP switches on one side from the default that was set up to a configuration that would force it to a different speed.”
Not exactly what happened here, but the actual problem was very similar:
In this case it turned out that somebody – presumably being helpful and matching the interface speeds – had installed a gigabit converter near R1 and a FastEthernet converter near R2.
The gig converter was able to do 10/100/1000 on the Ethernet side, but the optic was only able to support 1Gbps operation on the fiber side. The FastEthernet converter only did 100Mbps. Bingo – they see each other in light terms, but can’t actually talk to each other; nothing at all got across.
Swap out the gig converter for a FastEth, and we’re up!
Thanks to Andrew, Ben and Chuck for their quick participation!
The media converters are not hardcoded for 100M FD?
The router and it’s connection to the media converter are both matched; the port has come up correctly at 100mb FD.
Good guess though.
Late collisions? How long is the fibre?
Good thinking. The fiber turned out to be only about 400m. Light levels were within spec at both ends. There shouldn’t be any collisions we hope, as all links are full duplex.
Someone hit the loopback or test button on the media converter?
No, but we had the same thought and spent a while checking the various operation mode settings that the converters supported to make sure they were set up properly.
I ran into something like this the other day. One side of the media converter had the speed set to 100Mb the other was 1000Mb. It would allow light to pass, the interfaces never went down and no MAC addresses ever populated. We had to change the DIP switches on one side from the default that was set up to a configuration that would force it to a different speed.
It was because the media converters could not buffer the traffic down to a different speed.
Couldn’t talk, period, in this case!
Close enough!
It was indeed a speed mismatch on the fiber side of things. Fiber I interfaces, generally, are stupid. See light? I’m UP!
In this case, it turned out that somebody – presumably being helpful and matching the interface speeds had installed a gigabit converter near R1 and a FastEthernet converter near R2.
The gig converter was able to do 10/100/1000 on the Ethernet side, but the optic was only able to support 1Gbps operation on the fiber side. The FastEthernet converter only did 100Mbps. Bingo – they see each other in light terms, but can’t actually talk to each other; nothing at all got across.
Swap out the gig converter for a FastEth, and we’re up.
Well done!
In my case I looked for documentation on the model to figure out the dip switch settings (no help), so I started trying different combos until it came up. One of our symptoms looking at the media converter was that there was never any xmit on the mismatched side. Once I found the right combo, it fired right up.
I got my gold star for the day 🙂