Routing Tricks

The problem

A router has two internet links. Site-to-site VPN traffic should use one and general internet the other, each should fail over to the other link, and NAT has to follow whichever link the traffic actually leaves by.

The trick

• Policy-route the VPN traffic on the LAN interface with set ip next-hop verify-availability tied to an IP SLA track through ISP B. When the probe fails, the route-map stops matching and normal routing takes over.
• Default route: a tracked static to ISP A, with a floating static (distance 100) to ISP B behind it.
• NAT: two overload rules, each tied to a route-map that matches the inside ACL and match interface for one exit. Traffic gets the address of the link it really leaves by.
• A deny line in the NAT ACL keeps the VPN traffic out of PAT. Without it, the traffic is translated before encryption, no longer matches the crypto ACL, and goes to the internet in clear.
• Two plain overload rules that share one ACL do not split by exit: the first one wins and traffic leaving ISP B carries ISP A's address. That is why the route-maps match the interface.
• After a failover, old translations stay until they time out. Clear them in the failback procedure if it matters.
• Simpler alternative: a tracked static route for the remote VPN subnet via ISP B, no PBR. You lose per-source control. Cisco's own advice for this scenario is that two routers, one per ISP, is simpler still.

The problem

Hosts behind a pair of Nexus cores (vPC, peer-gateway, HSRP) must reach particular remote hosts over particular ISP links, with failover. Routing protocols cannot route by source, and the ISP links don't terminate on the cores, so the cores never see a link go down.

The trick

• Run IP SLA probes from both cores to the far end of each ISP link, with tracks and short up and down delays.
• Put the same PBR route-map on both cores: with peer-gateway, a host's traffic can land on either one.
• Each sequence matches one source and destination pair and lists two next hops in order, each with verify-availability and its track, plus force-order so the first available one in the list always wins.
• Add host routes so each probe target is reached through the right link, and mirror the policy at the far site for return traffic.

The problem

Traffic arriving on one interface belongs to different VRFs depending on its source, for example several departments or customers sharing one access link.

The trick

• Use Multi-VRF selection with policy routing: a route-map matches the source and sets the VRF and next hop with set ip vrf, or sends it back to the global table with set ip global next-hop.
• Define each VRF before the route-map that names it.
• Add ip vrf receive for each VRF on the interface. Without it, packets selected into a VRF and addressed to the router itself are dropped.

The problem

Engineers expect a policy to drop traffic when its next hops are unavailable. It doesn't: the packet quietly falls back to the routing table and leaves by a path the policy was meant to avoid.

The trick

• Know the order IOS works through set clauses: next hop, recursive next hop, interface, default next hop, default interface.
• A recursive next hop can be any address reachable through the routing table, not just a connected one. Only one is allowed per route-map entry.
• If nothing in the list is available, the packet is routed normally. To force a drop, finish with set interface Null0.

The problem

A router with two ISPs must bring up a VPN, or send its own pings and management traffic, over one specific link. You add local policy routing, the tunnel comes up, and nothing passes through it.

The trick

• Router-generated traffic: local PBR chooses the exit for the packet before encryption (and for IKE), but the encrypted ESP packet is routed by the normal routing table. If that points the other way, encapsulation fails.
• Transit traffic: interface PBR is applied twice, before and after encryption, so no extra route is needed.
• Interface PBR is CEF-switched; local PBR is process-switched. Keep local policies narrow.
• Practical fix: local PBR for IKE and test traffic, plus a matching tracked host route to the VPN peer so the encrypted packets follow.
• Watch for uRPF at the provider: traffic sourced from the "other" link's address may be dropped. Cisco mentions a loopback bounce that makes local traffic look like transit, and does not advise it.

The problem

A site should announce its prefix to a backup provider only when the primary path has failed. A permanent backup announcement attracts traffic all the time, even with prepending.

The trick

• BGP conditional advertisement: neighbor x advertise-map ADVERTISE non-exist-map WATCH. The prefixes in ADVERTISE go to the backup only while the prefix in WATCH is missing from the BGP table. exist-map does the opposite.
• The trap: if the watched prefix can still be learned another way, for example through the backup provider from the wider internet, it never disappears and the backup announcement never happens.
• Fix it with a match as-path in the watch map anchored on the primary peer's AS, so it only counts the prefix when it comes from that peer. The AS-path match narrows a prefix match; it can't replace it.
• Check the state with show ip bgp neighbors: it shows the condition map, the advertise map and whether it is advertising or withdrawn.

The problem

Prepending is blunt, and some providers strip or ignore it. You want to choose which upstream announces a prefix, or tilt traffic between two upstreams, without a config change on every border router each time.

The trick

• Tag at the origin: where a prefix enters BGP, give it an internal community meaning "upstream 1 only", "upstream 2 only" or none for both. Each upstream session's route-map announces, drops or alters by community. Moving a prefix becomes one tag change.
• The soft version: announce to both, but set origin incomplete towards the less preferred upstream. Origin is compared after local preference and AS-path length, so it tilts traffic without removing the backup path.
• Provider action communities: many providers accept communities that prepend or lower local preference further from you, which is often more effective than prepending yourself. Check each provider's community guide; not all support them.
• Tag peering and transit routes differently inside your network, and decide deliberately whether a broken peering link should fail over to transit.

The problem

You want a prefix announced with one community in office hours and another at night, perhaps to pull inbound traffic onto a cheaper link out of hours. A route-map that matches a time-based ACL looks right and does nothing: redistribution re-checks a route only when the routing table changes, and the clock moving is not a change.

The trick

• Turn time into reachability. Apply the time-based ACL inbound on a spare loopback and ping that loopback every second with an IP SLA probe. The probe succeeds only while the time-range permits.
• Track the probe, and build a second track that is its inverse with track list boolean and and object 1 not.
• Install two static routes for the prefix to Null0 with different tags, each bound to one track. Exactly one is in the table at any moment, and the swap at the boundary is a real routing change.
• Redistribute static into BGP with a route-map that sets the community by tag.
• Simpler alternative: an EEM cron applet (or kron) that runs clear ip route for the prefix just after each boundary, so the time-based route-map is checked again.
• The Null0 static originates the prefix. Only use it for a prefix you own and already announce.

The problem

Two exits towards the same provider have different bandwidths. Once BGP multipath is on, traffic splits 1:1 and the smaller link fills first.

The trick

• Turn on multipath first: maximum-paths is 1 by default on IOS.
• Add DMZ link bandwidth: neighbor x dmzlink-bw on each eBGP session attaches a link-bandwidth extended community, and bgp dmzlink-bw uses it to set traffic shares. The routing table then shows unequal "traffic share count" values.
• Across iBGP, an edge router with several links advertises the sum of its link bandwidths, so routers further in balance across edges in proportion to their capacity. Send extended communities to iBGP peers and enable iBGP multipath.

The problem

An edge link is congested. Changing local preference prefix by prefix moves unpredictable amounts of traffic, and what worked on Monday is wrong by Friday.

The trick

• Move groups, not single prefixes: match an origin AS or an AS-path pattern and lower local preference for that group on the congested exit. Big, popular origin ASes carry stable volumes; single prefixes swing week to week.
• Use NetFlow to find which destinations fill the link, and pick large, stable ones with one best path across all exits.
• Make path selection deterministic with bgp deterministic-med. Use bgp always-compare-med only if you understand what it does across different neighbour ASes.
• Change policy without resetting sessions: route refresh (clear ip bgp x soft in), or inbound soft reconfiguration if you also want to inspect what was received.

The problem

Shutting a BGP session for maintenance drops traffic while the internet reconverges, because the routes are withdrawn before alternatives have spread.

The trick

• Use the well-known GRACEFUL_SHUTDOWN community, 65535:0 (RFC 8326). Before the work, tag routes you send on that session with it and set local preference 0 on routes you receive over it. Wait for convergence, then shut the session.
• Receiving side: an inbound policy on every eBGP session that sets local preference 0 for routes carrying 65535:0, so neighbours can drain you the same way.
• Many large networks honour it, including NTT, Telia, GTT, Cogent and PCCW on the NLNOG list. Check your provider.
• Raising MED or the IGP metric only works when the alternatives are otherwise equal, which is why the community is preferred.
• Check whether your IOS XE release has a built-in graceful-shutdown command; otherwise use route-maps as shown.

The problem

After a merger or a redesign, a router must move to a new AS number, but its eBGP neighbours can't all change their configuration on the same night.

The trick

• Configure neighbor x local-as <old AS> on each eBGP session. Neighbours keep peering with the old AS while the router runs the new one.
• The options no-prepend, replace-as and dual-as control whether the old AS appears in paths and whether the neighbour can use either AS during the cutover.
• Plan the inside separately: in the worked example, the old eBGP link between the merging networks becomes iBGP between loopbacks, with the IGP extended over it and next-hop-self on the edge.

The problem

A DDoS attack needs dropping at every edge router within a minute, faster than an access list can be written and pushed.

The trick

• Destination RTBH: every edge router routes a reserved address to Null0. A trigger router announces the victim /32 by iBGP with a black-hole community and that reserved next hop, so every edge router forwards it into Null0.
• Source RTBH: announce the attacker's source address the same way, and enable uRPF loose mode on the edge interfaces. The source now resolves to Null0, so uRPF drops its packets.
• Lock down acceptance on the edge routers: /32s only, empty AS path (^$), add no-export, and drop anything else carrying the black-hole community.
• Customer-triggered RTBH is possible, but only for prefixes each customer is authorised to announce.
• Know uRPF: strict mode drops legitimate traffic on asymmetric paths; loose mode drops traffic from sources with no route at all.
• History: uRPF loose mode was created for this in 2000, to drop by source on 60 routers in 60 seconds.

The problem

Traffic must pass from one VRF to another with NAT in between, for example a customer VRF with overlapping addresses reaching a shared WAN or services VRF.

The trick

• Create a VASI pair: vasileft1 in one VRF and vasiright1 in the other, on a small shared subnet. Whatever leaves vasileft enters vasiright, so each VRF routes into the other with a static route via its VASI interface.
• Apply NAT where it fits: inside on vasiright with the WAN interface as outside (the usual case), or entirely in the left VRF with vasileft as outside.
• The TTL drops by two: once in each VRF.
• The trap: don't translate to the WAN interface's own address. The router treats the return traffic as its own and never forwards it back through the VASI pair. Use a separate address or pool.

The problem

A server with a public address sits in a service VRF (an SD-WAN service VPN, say) but must be reachable from the internet in the global table, bypassing the NAT everything else uses.

The trick

• Replicate routes both ways: in the VRF, route-replicate from vrf global unicast connected; in the global address family, route-replicate from vrf <VRF> unicast connected. Each table then holds the other's connected routes.
• Pair it with a data policy that matches the server's address and sets NAT bypass, so it keeps its real address.
• In the same policy, other groups can use different internet NAT methods: a pool shared across both ISPs for one group, one ISP's interface address for another.

The problem

A small edge router receives a large table and throws most of it away with an inbound prefix-list. Every update is still received, parsed and discarded, which costs CPU and memory and slows convergence.

The trick

• Outbound Route Filtering lets the receiving router send its inbound prefix-list to the neighbour inside BGP. The neighbour applies it outbound, so the unwanted prefixes are never sent.
• Receiving side: keep the inbound prefix-list and add capability orf prefix-list send. Sending side: capability orf prefix-list receive. The capability is agreed when the session opens, so it takes effect after the session re-establishes.
• The received list never appears in the sender's running configuration. Look for it in the neighbour detail instead.
• After editing the prefix-list, push the new version with clear ip bgp <neighbour> in prefix-filter.
• Defined in RFC 5291 and RFC 5292. Few public ISPs accept it, so it is mainly for your own PE-to-CE, hub or WAN aggregation sessions.

The problem

Every router in a PIM sparse-mode domain needs to know the rendezvous point. Static RP lists drift out of step, and BSR is one more protocol to run and troubleshoot.

The trick

• With embedded RP (RFC 3956) the RP address is part of the group address. Routers read it from the group, so only the RP needs any RP configuration.
• Group format: FF7S:0RPL:<64-bit prefix>:<32-bit group ID>. S is the scope, R is the RP interface ID (one hex digit, never 0) and PL is the prefix length in hex (40 for a /64).
• The RP address is the prefix with R as its last digit. Group FF7E:540:2001:DB8:10:20:0:1 means RP 2001:db8:10:20::5.
• Put the address on a loopback of the RP, make sure the IGP carries it, and point the RP at itself. Embedded RP is on by default, so the other routers need only multicast routing.
• Each RP prefix gives you 32 bits of group IDs. Hand groups out from it, and keep SSM (FF3x::/32) for sources the receivers can name.