User/Mutual internet backup between two small ISP
Contents
Motivation
Assume two small ISP which operate in surrounding area and have own physical link between them. Both network use dynamic routing OSPF with own backbone and have their own upstream internet connection with NATing. As they operate in different area they don't compete to each other and so they have settle on mutual supplying backup internet connection for free for case of failure of their main link or part of their network.
Realization
Starting conditions
Both networks have their own independent OSPF routing table and one default gateway. Both network are addressable form each other thanks to static routing. Internetwork neutral subnet is addressable too as connected subnets are imported to OSFP.
ABR1:
/ip address
add address=172.16.0.1/30 broadcast=172.16.0.3 comment="" disabled=no interface=wlan1 network=172.16.0.0
add address=1.0.0.1/8 broadcast=1.255.255.255 comment="" disabled=no interface=ether1 network=1.0.0.0
/ip route
(imported form OSPF) add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=1.0.0.2 scope=30 target-scope=10
add comment=ISP2 disabled=no distance=1 dst-address=2.0.0.0/8 gateway=172.16.0.2 scope=30 target-scope=10
/routing ospf instance
set default comment="" disabled=no distribute-default=no in-filter=ospf-in metric-bgp=20 \
metric-connected=20 metric-default=1 metric-other-ospf=auto metric-rip=20 metric-static=20 name=default \
out-filter=ospf-out redistribute-bgp=no redistribute-connected=as-type-1 redistribute-other-ospf=no redistribute-rip=\
no redistribute-static=as-type-1 router-id=0.0.0.0
/routing ospf area
set backbone area-id=0.0.0.0 comment="" disabled=no instance=default name=backbone type=default
/routing ospf network
add area=backbone comment="" disabled=no network=1.0.0.0/8
ABR2:
/ip address
add address=172.16.0.2/30 broadcast=172.16.0.3 comment="" disabled=no interface=wlan1 network=172.16.0.0
add address=2.0.0.1/8 broadcast=2.255.255.255 comment="" disabled=no interface=ether1 network=2.0.0.0
/ip route
(imported form OSPF) add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=2.0.0.2 scope=30 target-scope=10
add comment=ISP1 disabled=no distance=1 dst-address=1.0.0.0/8 gateway=172.16.0.1 scope=30 target-scope=10
/routing ospf instance
set default comment="" disabled=no distribute-default=no in-filter=ospf-in metric-bgp=20 \
metric-connected=20 metric-default=1 metric-other-ospf=auto metric-rip=20 metric-static=20 name=default \
out-filter=ospf-out redistribute-bgp=no redistribute-connected=as-type-1 redistribute-other-ospf=no redistribute-rip=\
no redistribute-static=as-type-1 router-id=0.0.0.0
/routing ospf area
set backbone area-id=0.0.0.0 comment="" disabled=no instance=default name=backbone type=default
/routing ospf network
add area=backbone comment="" disabled=no network=2.0.0.0/8
Setup secondary default gateway
As both network use OSPF routing protocol they can get advantage of automatic propagation of multiple default gateways with defined priority. So they have to inject each other primary default gateway as secondary default gateway to own routing table.
- We need to add to both routers new default gateway with destination to each other. But as static default gateway will take over dynamically imported primary default gateway we need to insert default gateway to different routing table.
ABR1:
/ip route add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=172.16.0.2 routing-mark=backup scope=30 target-scope=10
ABR2:
/ip route add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=172.16.0.1 routing-mark=backup scope=30 target-scope=10
- Then we need also to mark right packets using firewall mangle to our new routing table. All packets originating from other network with destination different than local network should be directed to backup routing table.
ABR1:
/ip firewall mangle add action=mark-routing chain=prerouting comment="" disabled=no new-routing-mark=backup passthrough=no \ src-address=2.0.0.0/8 dst-address=!1.0.0.0/8
ABR2:
/ip route add action=mark-routing chain=prerouting comment="" disabled=no new-routing-mark=backup passthrough=no \ src-address=1.0.0.0/8 dst-address=!2.0.0.0/8
- Default gateway is not yet imported to OSPF so we need to set OSPF to import default gateway and set higher metric value for example 5000. All default gateways should have to same metric type. If secondary default gateway have type1 and primary type2 than secondary will be elected even if primary would have lower metric.
Both ABR1 and ABR2:
/routing ospf instance
set default comment="" disabled=no distribute-default=if-installed-as-type-2 in-filter=ospf-in metric-bgp=20 \
metric-connected=20 metric-default=5000 metric-other-ospf=auto metric-rip=20 metric-static=20 name=default \
out-filter=ospf-out redistribute-bgp=no redistribute-connected=as-type-1 redistribute-other-ospf=no redistribute-rip=\
no redistribute-static=as-type-1 router-id=0.0.0.0
- You can check presence of default gateways in OSPF table /routing ospf lsa. You can use traceroute for connection route checking.
Checking if default gateway is working
We have two default gateways now but how we assure to be static default gateways entry properly disable on both routers in case of link failure? RouterOS allow to set check-gateway parameter to automatically check reachability of gateway using ping or arp. Then if gateway is not reachable entry is disabled and OSPF discard entry distributed routing table as parameter redistribute-default-gateway is setted to if-installed. So this can handle two cases if interface is down and if gateway is not responding to ping.
But this could be not enough for real life situation. We need to check some external high available address for reachability and on both routers use only proper gateway.
There more things to take into account. Because we need to disable/enable gateway we make sure that ping test will be able to reach selected host in internet even if default gateway is disabled. So we need setup another routing table with our static default gateway for purpose of ping checking.
Example is presented for single router. In fact all routers which propagate default route should have ping check of internet connection. In our example test should run on ABR1, ABR2, GW1 and GW2.
- Add new static default gateway for ping checking of internet state. For ABR1 address of ISP2 gateway is 172.16.0.2.
/ip route
add comment="" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=172.16.0.2 routing-mark=PingCheck scope=30 \
target-scope=10
- Add new script for ping testing.
Script t
Change pinged IP address 195.47.235.3 to your high available reachable internet address.
/system script
add name=PingCheck policy=read,write,test,sniff source=":local Count\
\_[/ping 195.47.235.3 routing-table=PingCheck count=5 size=1500];\r\
\n:if (\$Count = 0) do={\r\
\n/ip route set [find comment=backup] disabled=yes\r\
\n:log info \"backup gw down\"\r\
\n}\r\
\n\r\
\n:if (\$Count = 5) do={\r\
\n/ip route set [find comment=backup] disabled=no\r\
\n:log info \"backup gw up\"\r\
\n}\r\
\n"
/system scheduler
add comment="" disabled=no interval=10s name=schedule_default_gw_check on-event="/system script run PingCheck" \
policy=read,write,test start-date=jan/01/1970 start-time=00:00:00
As script duration is about 5 seconds of pinging than interval could be for example 10 seconds.
Script change route table entry labelled by comment backup. So set comment on static default gateway to this word or change script according to your default gateway comment.
- It is better to test multiple internet hosts for reachability. If checked host itself goes down this event cause false action. In general every dynamic checking could cause some trouble where static configuration could work well.
