One class of network problem is unusually expensive in engineer-hours: the symptom is intermittent, load testing reproduces nothing, and no amount of investigation in software finds a root cause.
That class is almost always in the physical layer.
The good news about physical layer work is that the order is fixed, and it starts with the cheapest part.
1. The optic goes in and the port stays dark
Symptom. The transceiver is seated and the port does not come up, or comes up and will not negotiate the expected rate.
The most common cause: vendor coding checks. Many switches and adapters read vendor information from the optic’s EEPROM and reject anything not on their list. This is a commercial policy, not a technical defect — which means “the module is fine” and “this device will accept it” are two independent questions.
Three other causes:
- Wavelength mismatch — 850nm (multimode), 1310nm or 1550nm (single-mode) do not interoperate
- Mode mismatch — a single-mode optic on multimode fibre, or the reverse
- Rate mismatch — the two ends cannot agree
What to do. Select from the device’s official compatibility list, or use a coded module programmed to match. When ordering, specify coding, wavelength, mode, rate and reach — not “10G optic”.
Worth noting. This looks like a failed NIC and the NIC is perfectly healthy. Before replacing an adapter, try a module you know is compatible. The cost difference is roughly two orders of magnitude.
2. A 10G port running at 1G
Symptom. The link is up and the rate is stuck at a lower tier.
Three causes, in the order to check them:
- Auto-negotiation failed and both ends fell back to the lowest common rate. Check the negotiation state at both ends first — most switches and adapters report both the negotiated result and local capability.
- Cable category or optic mismatch. Copper category, DAC length, and the optic pairing with the far end all matter, and any one of them below spec drops the tier.
- Driver or firmware too old to support the rate mode. This one is especially common on a newly fitted adapter: the card is new, the driver is whatever shipped with the OS.
The order. Check negotiation state (free), then swap cable and optic (cheap), and only then suspect the adapter (expensive).
Ruling out configuration before replacing hardware saves a lot of money.
3. Three days chasing packet loss, and it was a cable
Symptom. Intermittent loss and retransmits. The application is fast, then slow. Load testing reproduces nothing.
This is the expensive one, and the causes are usually mundane:
- Marginal connector — not fully seated, or worked loose under vibration
- Bend radius exceeded — fibre in particular attenuates once bent past its minimum radius
- Dirty fibre end face — a fingerprint or dust is enough to cause bit errors at 10G and above
- Cable ageing — patch cords in high-temperature or UV-exposed runs
Confirming it is the physical layer. Look at the port’s CRC error counter and error counters.
What matters is the increment: clear them, observe for a period, and if the counters climb steadily the fault is physical and there is no reason to keep reading application logs. That step alone can save days.
Order of replacement, cheapest first:
- Clean the fibre end face and reseat — free, and frequently effective
- Replace the patch cable — tens of dollars
- Replace the optic — hundreds
- Replace the adapter or move to another switch port — thousands
Patch cables and optics are cheap. Try them first. The large majority of intermittent loss is resolved in the first two steps.
Order of work
| Symptom | Check first | Then | Only then |
|---|---|---|---|
| Port dark | Optic compatibility, wavelength/mode | Swap in a known-good optic | Suspect NIC or port |
| Rate degraded | Negotiation state both ends | Cable category, optic pairing | Driver and firmware |
| Intermittent loss | CRC / error counter increment | Clean end face, swap patch cable | Swap optic, then card |
| No link at all | Config and VLAN at both ends | Physical path, optical power | Replace parts |
One general principle: a physical layer fault cannot be found in software. So before spending three days reading logs, spend three minutes looking at the CRC counter.
On spare parts
Optic coding, wavelength, rate and reach all have to match. Our catalogue records those fields against each transceiver part number along with the equipment it is qualified for.
Adapters, optics and patch cables are mostly physically stocked. Search by brand or part number in Parts Lookup, or send us a list for manual checking.
This describes common failure modes for this class of equipment, not a specific incident at a named site.