Routing Tricks

The problem

A router has two internet links. Site-to-site VPN traffic should use one and general internet the other, each should fail over to the other link, and NAT has to follow whichever link the traffic actually leaves by.

The trick

• Policy-route the VPN traffic on the LAN interface with set ip next-hop verify-availability tied to an IP SLA track through ISP B. When the probe fails, the route-map stops matching and normal routing takes over.
• Default route: a tracked static to ISP A, with a floating static (distance 100) to ISP B behind it.
• NAT: two overload rules, each tied to a route-map that matches the inside ACL and match interface for one exit. Traffic gets the address of the link it really leaves by.
• A deny line in the NAT ACL keeps the VPN traffic out of PAT. Without it, the traffic is translated before encryption, no longer matches the crypto ACL, and goes to the internet in clear.
• Two plain overload rules that share one ACL do not split by exit: the first one wins and traffic leaving ISP B carries ISP A's address. That is why the route-maps match the interface.
• After a failover, old translations stay until they time out. Clear them in the failback procedure if it matters.
• Simpler alternative: a tracked static route for the remote VPN subnet via ISP B, no PBR. You lose per-source control. Cisco's own advice for this scenario is that two routers, one per ISP, is simpler still.

The problem

Hosts behind a pair of Nexus cores (vPC, peer-gateway, HSRP) must reach particular remote hosts over particular ISP links, with failover. Routing protocols cannot route by source, and the ISP links don't terminate on the cores, so the cores never see a link go down.

The trick

• Run IP SLA probes from both cores to the far end of each ISP link, with tracks and short up and down delays.
• Put the same PBR route-map on both cores: with peer-gateway, a host's traffic can land on either one.
• Each sequence matches one source and destination pair and lists two next hops in order, each with verify-availability and its track, plus force-order so the first available one in the list always wins.
• Add host routes so each probe target is reached through the right link, and mirror the policy at the far site for return traffic.

The problem

Traffic arriving on one interface belongs to different VRFs depending on its source, for example several departments or customers sharing one access link.

The trick

• Use Multi-VRF selection with policy routing: a route-map matches the source and sets the VRF and next hop with set ip vrf, or sends it back to the global table with set ip global next-hop.
• Define each VRF before the route-map that names it.
• Add ip vrf receive for each VRF on the interface. Without it, packets selected into a VRF and addressed to the router itself are dropped.

The problem

Engineers expect a policy to drop traffic when its next hops are unavailable. It doesn't: the packet quietly falls back to the routing table and leaves by a path the policy was meant to avoid.

The trick

• Know the order IOS works through set clauses: next hop, recursive next hop, interface, default next hop, default interface.
• A recursive next hop can be any address reachable through the routing table, not just a connected one. Only one is allowed per route-map entry.
• If nothing in the list is available, the packet is routed normally. To force a drop, finish with set interface Null0.

The problem

A router with two ISPs must bring up a VPN, or send its own pings and management traffic, over one specific link. You add local policy routing, the tunnel comes up, and nothing passes through it.

The trick

• Router-generated traffic: local PBR chooses the exit for the packet before encryption (and for IKE), but the encrypted ESP packet is routed by the normal routing table. If that points the other way, encapsulation fails.
• Transit traffic: interface PBR is applied twice, before and after encryption, so no extra route is needed.
• Interface PBR is CEF-switched; local PBR is process-switched. Keep local policies narrow.
• Practical fix: local PBR for IKE and test traffic, plus a matching tracked host route to the VPN peer so the encrypted packets follow.
• Watch for uRPF at the provider: traffic sourced from the "other" link's address may be dropped. Cisco mentions a loopback bounce that makes local traffic look like transit, and does not advise it.

The problem

A site should announce its prefix to a backup provider only when the primary path has failed. A permanent backup announcement attracts traffic all the time, even with prepending.

The trick

• BGP conditional advertisement: neighbor x advertise-map ADVERTISE non-exist-map WATCH. The prefixes in ADVERTISE go to the backup only while the prefix in WATCH is missing from the BGP table. exist-map does the opposite.
• The trap: if the watched prefix can still be learned another way, for example through the backup provider from the wider internet, it never disappears and the backup announcement never happens.
• Fix it with a match as-path in the watch map anchored on the primary peer's AS, so it only counts the prefix when it comes from that peer. The AS-path match narrows a prefix match; it can't replace it.
• Check the state with show ip bgp neighbors: it shows the condition map, the advertise map and whether it is advertising or withdrawn.

The problem

Prepending is blunt, and some providers strip or ignore it. You want to choose which upstream announces a prefix, or tilt traffic between two upstreams, without a config change on every border router each time.

The trick

• Tag at the origin: where a prefix enters BGP, give it an internal community meaning "upstream 1 only", "upstream 2 only" or none for both. Each upstream session's route-map announces, drops or alters by community. Moving a prefix becomes one tag change.
• The soft version: announce to both, but set origin incomplete towards the less preferred upstream. Origin is compared after local preference and AS-path length, so it tilts traffic without removing the backup path.
• Provider action communities: many providers accept communities that prepend or lower local preference further from you, which is often more effective than prepending yourself. Check each provider's community guide; not all support them.
• Tag peering and transit routes differently inside your network, and decide deliberately whether a broken peering link should fail over to transit.

The problem

Two exits towards the same provider have different bandwidths. Once BGP multipath is on, traffic splits 1:1 and the smaller link fills first.

The trick

• Turn on multipath first: maximum-paths is 1 by default on IOS.
• Add DMZ link bandwidth: neighbor x dmzlink-bw on each eBGP session attaches a link-bandwidth extended community, and bgp dmzlink-bw uses it to set traffic shares. The routing table then shows unequal "traffic share count" values.
• Across iBGP, an edge router with several links advertises the sum of its link bandwidths, so routers further in balance across edges in proportion to their capacity. Send extended communities to iBGP peers and enable iBGP multipath.

The problem

An edge link is congested. Changing local preference prefix by prefix moves unpredictable amounts of traffic, and what worked on Monday is wrong by Friday.

The trick

• Move groups, not single prefixes: match an origin AS or an AS-path pattern and lower local preference for that group on the congested exit. Big, popular origin ASes carry stable volumes; single prefixes swing week to week.
• Use NetFlow to find which destinations fill the link, and pick large, stable ones with one best path across all exits.
• Make path selection deterministic with bgp deterministic-med. Use bgp always-compare-med only if you understand what it does across different neighbour ASes.
• Change policy without resetting sessions: route refresh (clear ip bgp x soft in), or inbound soft reconfiguration if you also want to inspect what was received.

The problem

Shutting a BGP session for maintenance drops traffic while the internet reconverges, because the routes are withdrawn before alternatives have spread.

The trick

• Use the well-known GRACEFUL_SHUTDOWN community, 65535:0 (RFC 8326). Before the work, tag routes you send on that session with it and set local preference 0 on routes you receive over it. Wait for convergence, then shut the session.
• Receiving side: an inbound policy on every eBGP session that sets local preference 0 for routes carrying 65535:0, so neighbours can drain you the same way.
• Many large networks honour it, including NTT, Telia, GTT, Cogent and PCCW on the NLNOG list. Check your provider.
• Raising MED or the IGP metric only works when the alternatives are otherwise equal, which is why the community is preferred.
• Check whether your IOS XE release has a built-in graceful-shutdown command; otherwise use route-maps as shown.

The problem

After a merger or a redesign, a router must move to a new AS number, but its eBGP neighbours can't all change their configuration on the same night.

The trick

• Configure neighbor x local-as <old AS> on each eBGP session. Neighbours keep peering with the old AS while the router runs the new one.
• The options no-prepend, replace-as and dual-as control whether the old AS appears in paths and whether the neighbour can use either AS during the cutover.
• Plan the inside separately: in the worked example, the old eBGP link between the merging networks becomes iBGP between loopbacks, with the IGP extended over it and next-hop-self on the edge.

The problem

A DDoS attack needs dropping at every edge router within a minute, faster than an access list can be written and pushed.

The trick

• Destination RTBH: every edge router routes a reserved address to Null0. A trigger router announces the victim /32 by iBGP with a black-hole community and that reserved next hop, so every edge router forwards it into Null0.
• Source RTBH: announce the attacker's source address the same way, and enable uRPF loose mode on the edge interfaces. The source now resolves to Null0, so uRPF drops its packets.
• Lock down acceptance on the edge routers: /32s only, empty AS path (^$), add no-export, and drop anything else carrying the black-hole community.
• Customer-triggered RTBH is possible, but only for prefixes each customer is authorised to announce.
• Know uRPF: strict mode drops legitimate traffic on asymmetric paths; loose mode drops traffic from sources with no route at all.
• History: uRPF loose mode was created for this in 2000, to drop by source on 60 routers in 60 seconds.

The problem

Traffic must pass from one VRF to another with NAT in between, for example a customer VRF with overlapping addresses reaching a shared WAN or services VRF.

The trick

• Create a VASI pair: vasileft1 in one VRF and vasiright1 in the other, on a small shared subnet. Whatever leaves vasileft enters vasiright, so each VRF routes into the other with a static route via its VASI interface.
• Apply NAT where it fits: inside on vasiright with the WAN interface as outside (the usual case), or entirely in the left VRF with vasileft as outside.
• The TTL drops by two: once in each VRF.
• The trap: don't translate to the WAN interface's own address. The router treats the return traffic as its own and never forwards it back through the VASI pair. Use a separate address or pool.

The problem

A server with a public address sits in a service VRF (an SD-WAN service VPN, say) but must be reachable from the internet in the global table, bypassing the NAT everything else uses.

The trick

• Replicate routes both ways: in the VRF, route-replicate from vrf global unicast connected; in the global address family, route-replicate from vrf <VRF> unicast connected. Each table then holds the other's connected routes.
• Pair it with a data policy that matches the server's address and sets NAT bypass, so it keeps its real address.
• In the same policy, other groups can use different internet NAT methods: a pool shared across both ISPs for one group, one ISP's interface address for another.