Short notes from thirty years of Cisco networks: the questions engineers keep asking, the mistakes that keep turning up in change reviews, and the answer I give each time. No dates to keep up with. Each one stands on its own.
An incident report from one of my own changes. The fix took a minute. The lesson took longer, and it is why every EtherChannel change I do now starts with a two-line check.
What happened
I was adding a VLAN for a new guest service across a client site. One of the trunks it needed was a two-member port-channel between the core switch and the server switch. I added the VLAN to one of the physical member interfaces instead of to the port-channel.
The switch did exactly what it is designed to do. The member no longer matched the bundle, so it was suspended, and the logs said so plainly:
%EC-5-CANNOT_BUNDLE2: Te1/1/1 is not compatible with Te2/1/1 and will be suspended (vlan mask is different)
%EC-5-CANNOT_BUNDLE2: Te1/1/1 is not compatible with Po100 and will be suspended (vlan mask is different)
The impact
Three application VLANs went down for fifteen minutes. Nothing else was touched, because every other VLAN on that site was trunked over more than one path. Those three were not: an earlier project had put them on that single port-channel as a temporary design, under time pressure, and the move to the new network had not happened yet.
The fix
Take the VLAN off the member, add it to the port-channel interface, and let IOS push it down to both members. The bundle came straight back.
Configure the port-channel interface, never the members. The only lines that belong on a member are its description and its channel-group.
Before any EtherChannel change, compare the bundle with its members in one command, and again afterwards:
show etherchannel summary
show running-config | section interface (TenGigabitEthernet1/1/1|TenGigabitEthernet2/1/1|Port-channel100)$
Do not let application VLANs depend on one trunk, even "for now". Temporary designs last for years and nobody remembers which links are the only path.
Write the incident report the same day and say plainly whose mistake it was. Mine said "I take responsibility for the error" in the first paragraph. Clients forgive a mistake that is owned and explained. They do not forgive one they have to dig out.
How I run a network implementation, start to handover
The process I have used on implementations for years, from the first design document to handing the network to the people who will support it. None of it is clever. All of it is there because something went wrong once without it.
1. High-level design
A short summary of the solution and the technologies in it, written so a manager can say yes or no to the approach.
2. Low-level design
The detail an engineer can build from: hardware and software with summary configuration, VLAN numbers, IP addressing, firewall rules, VRFs, diagrams and the kit list.
3. Plan the build on paper first
List every switch port and patch panel port you will use.
Draw the cabinets and connectivity, fibre and copper runs, patch panel use and switch port use.
Read the existing configuration and work out exactly what has to change on it.
Write the configuration scripts, and have another engineer review them.
Dry-run the scripts. Problems found here cost minutes, not a night.
4. The implementation plan
Submit it with everything above for approval. Mine always has the same sections: objective, success criteria, when and where, who is involved and how to reach them, pre-implementation tasks, the implementation steps, a back-out plan with how long it takes, post-implementation tasks, testing and a risk table.
Hold a go or no-go call before the window. If anything is missing, it is a no-go.
Tell the NOC, and any provider involved, before you touch anything.
5. Build it, and know when to stop
Small differences from the plan get written down as you go. A significant difference is a different rule entirely:
Stop all work.
Fall back to where you started.
Amend the design, the plan and the scripts.
Resubmit the plan.
Redesigning in the window, tired, with the change board asleep, is how a two-hour change becomes a two-day outage.
6. Update the documents
Every small change made during the build goes back into the design, the plan and the configuration documents, and into the CMDB.
7. Acceptance into service, then handover
Pull the designs, plans, diagrams and configuration into one acceptance-into-service document, then hand the solution to the live support team, with a walk-through if it is new to them. Finish with a short post-implementation review, even when it went perfectly.
The Change Window Kit MOP Builder writes steps 4 and 5 for ten common changes, go or no-go call and review included.
Routing policy · October 2026
ACL, prefix list, route-map, distribute-list: who does what
Four names that get used as if they were the same thing. They are not. Two of them match, one matches and changes, and one is just a place to plug the others in.
The short version
Access lists, prefix lists, AS-path lists and community lists are matchers. Give them a packet or a route and they answer one of three things: matched a permit, matched a deny, or matched nothing. Nothing else.
A route-map is policy. It calls the matchers, and for whatever matches it can permit, deny or change something: a metric, a next hop, local preference, a community, the path a packet takes.
Distribute-list, access-group, filter-list and the route-map keyword on a neighbor are attachment points. They are where you apply a matcher or a policy. A distribute-list is to routing updates what ip access-group is to packets.
Redistribution is the odd one out. On IOS and IOS XE it only takes a route-map. If you want an ACL or prefix list there, wrap it in a route-map.
Commands on the left use a policy. A route-map can match and change. The lists on the right only answer one question: does this match?
Rules that save you later
Everything ends in an implicit deny. ACLs, prefix lists and route-maps alike. A route-map made only of deny entries denies everything, including what you meant to keep. End it with an empty permit.
Use prefix lists for routes, not ACLs. A standard ACL can only see the network address, so 10.0.0.0/8 and 10.0.0.0/24 look the same to it. A prefix list matches the length too, and le and ge give you ranges of lengths.
An OSPF distribute-list in only filters your own routing table. The LSAs are still in the database and still flooded to your neighbours, so they still learn the route. To stop a route spreading inside OSPF, filter or summarise at the ABR with area filter-list or area range.
BGP policy changes do not touch routes you have already exchanged. Change a route-map, then soft clear that one neighbour in the direction you changed. Never a hard clear of everything.
Worked example: do not become a transit AS
Two internet providers, one company. Without an outbound filter, your router can advertise ISP A's routes to ISP B, and some of the internet's traffic tries to cross your office link. The AS path of a route you originate yourself is empty, so ^$ matches exactly your own routes.
ip as-path access-list 1 permit ^$
ip prefix-list OUR-PREFIXES seq 5 permit 198.51.100.0/24
!
router bgp 64500
neighbor 203.0.113.1 remote-as 64496
address-family ipv4
network 198.51.100.0 mask 255.255.255.0
neighbor 203.0.113.1 filter-list 1 out
neighbor 203.0.113.1 prefix-list OUR-PREFIXES out
The filter-list keeps out anything learned from another AS. The prefix list is the belt and braces: even if something odd ends up with an empty path, only your own block leaves. The same result can be had with a route-map that matches as-path 1, and that is the version to use if you also want to set something, such as prepending.
Check what you are actually sending before and after:
show ip bgp regexp ^$
show ip bgp neighbors 203.0.113.1 advertised-routes
clear ip bgp 203.0.113.1 soft out
[100-199] is three characters, not a hundred numbers
A BGP regex that looks right, passes a quick read in a change review, and matches the wrong AS numbers.
Someone wants to match any private AS from 65100 to 65199 in a path and writes this:
ip as-path access-list 10 permit _65[100-199]_
It reads naturally. It is wrong. Square brackets match one character from a set. [100-199] is the characters 1, 0, the range 0 to 1, 9 and 9, which boils down to a single 0, 1 or 9. So the expression matches AS 650, 651 and 659, and none of 65100 to 65199.
Write the digits position by position instead:
ip as-path access-list 10 permit _651[0-9][0-9]_
For ranges that do not line up so neatly, such as 64512 to 65534, the expression gets long. That is what the BGP regex tester and the number-range match in the CLI Filter Builder are for: they write the alternatives for you, and the tester now warns when it sees a multi-digit range in brackets.
Always prove an expression against the real table before it goes into a policy:
Two WAN providers, one VRF: moving sites across a prefix at a time
The design I use when a customer changes MPLS provider and the old and new networks have to run side by side for months. Each site moves one subnet at a time, and rolling back is one line.
The situation is common. The incumbent MPLS is handed off on one VLAN, the new provider's routers on another, and the new contract adds local internet breakout at each site, where before everything went to a central default route. The sites cannot all move on one night.
The design
One WAN VRF per site on the core switch. Both providers live in it, each on its own SVI with its own OSPF process, so neither provider ever learns the other's routes.
Nothing is advertised by accident. Internal subnets are static routes in the VRF towards the firewall, and each provider gets only the statics its prefix list allows.
Inbound, a distribute-list on each process accepts only the remote site ranges you expect from that provider.
A standard addressing block per site for the handoffs: a /24 split into /29s, with the switch SVI on .1 and the provider's two routers on .2 and .3.
The internet VLAN is layer 2 only on the switch, from the provider router straight to the firewall's outside interface. The switch never routes it.
vrf definition SITE_WAN
rd 10.100.11.1:10
address-family ipv4
exit-address-family
!
interface Vlan999
description Old provider MPLS
vrf forwarding SITE_WAN
ip address 10.100.11.20 255.255.255.248
interface Vlan1000
description New provider MPLS
vrf forwarding SITE_WAN
ip address 10.100.11.1 255.255.255.248
interface Vlan1002
description Firewall inside this VRF
vrf forwarding SITE_WAN
ip address 10.100.11.25 255.255.255.248
!
ip route vrf SITE_WAN 10.65.90.0 255.255.255.0 10.100.11.28 name VOICE
ip route vrf SITE_WAN 10.66.71.0 255.255.255.0 10.100.11.28 name PRINTERS
!
ip prefix-list TO-OLD seq 10 permit 10.66.71.0/24
ip prefix-list TO-NEW seq 10 permit 10.65.90.0/24
route-map TO-OLD permit 10
match ip address prefix-list TO-OLD
route-map TO-NEW permit 10
match ip address prefix-list TO-NEW
!
router ospf 100 vrf SITE_WAN
router-id 10.100.11.20
passive-interface default
no passive-interface Vlan999
network 10.100.11.16 0.0.0.7 area 0
redistribute static subnets route-map TO-OLD
router ospf 900 vrf SITE_WAN
router-id 10.100.11.1
passive-interface default
no passive-interface Vlan1000
network 10.100.11.0 0.0.0.7 area 0
redistribute static subnets route-map TO-NEW
Migrating, and rolling back
Moving a subnet is moving one line from TO-OLD to TO-NEW. Rolling it back is moving it again. The change board sees a one-line change per subnet, and the provider sees a clean, expected advertisement.
Three things that catch people
Two OSPF processes, one prefix. If both providers advertise the same remote subnet, both routes have a distance of 110 and their metrics are never compared, so whichever arrived first stays installed. Set distance on one process to make the choice deliberate during the overlap.
The passive exception must name the real interface. With passive-interface default, the no passive-interface line has to match the interface OSPF actually runs on: the SVI, not the physical port behind it. Get it wrong and the adjacency never forms, with no error.
mtu-ignore hides a problem. Provider handoffs often need ip ospf mtu-ignore to get past ExStart. It works, but large packets can still be dropped across the link, so ask the provider for their MTU and match it where you can.
IT and OT segmentation in plain English: zones, conduits and the two-firewall rule
How I lay out a power or industrial site network to follow IEC 62443 without drowning in the standard. Three zones, a firewall for each, and a register that says what crosses where.
Industrial networks are easy to attack for reasons that are not the engineers' fault: hard-coded passwords on controllers, software that cannot be patched without recertifying the plant, and historically flat /16 networks where every PLC can see every workstation. The standard's answer is segmentation into zones, joined only through controlled conduits.
The layout
Three zones: CORP for internal IT, DMZ for anything higher risk or shared with outsiders, and OT for the control systems: SCADA, HMIs, PLCs and RTUs.
Each zone has its own firewall cluster, always at least two nodes, with IPS and application control on. Separation is a VLAN at layer 2 and a VRF at layer 3, so a zone keeps its own routing table even on shared switches.
The two-firewall rule: nothing goes from CORP to OT directly. Traffic passes through the DMZ and through the OT firewall, so a compromise in IT has two policies to beat, not one.
Walled gardens inside OT. The flat OT network is then split further, VRF and VLAN per area, firewalled from each other, so a breach stays where it started.
The access policy, in one table
From
To
Allowed
Internet
Low-risk zone
Never
Internet
High-risk zone
Only with a business case, security approval, a specific rule and a vulnerability scan
Low-risk zone
Internet
Through content filtering
Between zones of different risk
Only with a business case, security approval and a specific rule
Within a zone
Allowed between VLANs of the same risk level
The zone register
The part most sites skip, and the part an assessor asks for first. For every zone and every conduit, write down:
Its name, and who is accountable for it.
Its logical boundary (subnets, VLANs, VRF) and its physical boundary (which cabinets and devices).
Whether it is safety-related.
Every logical and physical access point, and the data flows through each one.
The zones and conduits it connects to.
Its assets, with their criticality.
Its target security level, the requirements and policies that apply, and the assumptions it relies on.
Keep it next to the firewall rule base. When a rule request arrives, the register tells you whether the conduit exists at all.