Testwiki/IP routing
IP routing
Routing is the process of moving a packet of data from one network to another network based on the destination IP address. The Internet uses routing to forward data from one host across several networks, to reach a final destination host (like personal PC, web or mail server). The network devices that perform this routing function are referred to as routers. Routers maintain a list of routes which is often referred to as a routing table.'' A route may be defined as information that is used to make a decision where forward packet next. The route information is used to make decisions about which will be output interface by forwarding a packet destination.
Routers operate at the network layer OSI model (layer 3) therefore often its refered as layer 3 devices. Routers look up routes in the routing table to figure out how to move data from one network to another network.
Brief description of routing process happens as follows:
When router receives packet on network interface, it will look at destination address of IP packet and if the packet isn’t destined for the particular router it look up for appropriate destination network address in own routing table. Routing table contains list of destination networks with the appropriate output interfaces. Applying this routing table router finds an exit interfaces for particular packet and forward it through this interface to next router. This process is repeated at each router until packet reaches destination. If router can’t find route entry for the packet’s destination network in the routing table, router drops the packet.
Routing table on MikroTik router can be viewed using ip route print command:
[admin@MikroTik_CE1] /ip route> print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 ADo 0.0.0.0/0 192.168.2.1 110 1 ADo 10.0.1.0/30 192.168.2.1 110 2 ADo 10.0.1.4/30 192.168.2.1 110 3 ADo 10.0.1.8/30 192.168.2.1 110 4 ADo 10.255.255.1/32 192.168.2.1 110 5 ADo 10.255.255.3/32 192.168.2.1 110 6 ADo 10.255.255.4/32 192.168.2.1 110 7 ADC 10.255.255.5/32 10.255.255.5 lobridge 0 8 ADo 192.168.1.0/30 192.168.2.1 110 9 ADC 192.168.2.0/30 192.168.2.2 ether3 0 10 ADC 192.168.11.0/24 192.168.11.1 ether1 0
Static route
As we previously found a router sending packet to the remote network look at the routing table to find out how to get to remote network.
There are several ways how to configure the routing table for each router. First you can create it statically (manually) or to create dynamically by configuring some of the dynamic routing protocols. Process when static routes are used forwarding packets through the network is known as static routing. Static routing is not really a protocol, simply the process of manually entering routes into the routing table these routes don't change after they are configured unless a human changes them. Static routing are several benefits and also some disadvantage. At the benefits can mention:
- Low Router’s CPU usage, there aren’t necessary to maintain any additional routing table process as using dynamic routing.
- There aren’t routing protocol updates therefore no additional bandwidth usage between routers
Whereas at disadvantages can mention:
- Very complicated to configure large networks, because each route must be manually (by hand) entered on all routers.
- Network administrator need change network configuration every time when some network goes down or network topology has changed.
Here’s an example of how to configure static routes on MikroTik routers, network topology for this example is given below:
Configuration on MikroTik_CE1:
[admin@MikroTik_CE1] /ip route> add dst-address=192.168.2.0/24 gateway=10.0.0.2
This configuration means that all packets destined to network 192.168.2.0/24 will be send to the router 10.0.0.2. In this case gateway means next hop IP address for static route.
Configuration on MikroTik_CE2:
[admin@MikroTik_CE2] /ip route> add dst-address=192.168.1.0/24 gateway=10.0.0.1
Show routing table of MikroTik_CE1 router:
[admin@MikroTik_CE1] /ip route> print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 ADC 10.0.0.0/30 10.0.0.1 ether2 0 1 ADC 192.168.1.0/24 192.168.1.1 ether1 0 2 A S 192.168.2.0/24 10.0.0.2 1
Static route is marked with letter – S. Directly connected routes in routing table are marked with C – connected (ADC – means that route is active, dynamic, and directly connected to router).
DST-Address'' – the destination network that is placed in the routing table
Gateway'' – Specifies which host or interface packets should be sent to. Usually value of this property is a single IP address of a gateway that can be directly reached through one of router's interfaces
Distance'' – Value used in route selection. Distance is an integer form 0 to 255 where 0 means more trusted and 255 means that this route is not used for traffic forwarding. Routes with smaller distance value are given preference it means that if there are two possible routes for the same remote network than the routes with the lowest distance will be placed in the routing table. If there are two routes to the same network with the same distance, then routing metric (such as hop count, bandwidth of line) will be used to find the best path to the remote network. If value of distance is not specified, then the default values are used. Default value of distance depends on route protocol:
- connected routes: 0
- static routes: 1
- eBGP: 20
- OSPF: 110
- RIP: 120
- MME: 130
- iBGP: 200
Default route
A default route is also referred to as the “route of last resort”. This is the route a router uses when no other known route exists for a given IP packet's destination address. Routing process which using default route is called as default routing. Default works as follows: All the packets received by router with IP destination address that not to conform to any destination networks in routing table can’t be forwarded and will be discarded. But if there is specified default route packet will be sent via this default route.
Configuration example of default route:
[admin@MikroTik] /ip route> add dst-address=0.0.0.0/0 gateway=10.0.0.1
Gateway can be also specified by name of interface, for example:
[admin@MikroTik] /ip route> add dst-address=0.0.0.0/0 gateway=ether1
Route with dst-address 0.0.0.0/0 specifies any destination address. A route lookup that doesn't match anything else in routing table will naturally fall back onto this route. If routing table contains an active default route, then routing table lookup in this table will never fail.
For computer on your local network also is needed to specify default route which refered as default gateway. In IP based networks, a default gateway address is needed to reach all hosts that are not on the same local IP subnet.
Simple load-balancing – ECMP
Equal-Cost Multi-Path (ECMP) is a forwarding mechanism for routing packets along multiple paths of equal cost with the goal to achieve almost equally traffic load sharing among multiple links. Static routes and also most of the routing protocols support ECMP load balancing mechanism.
Consider the following network layout:
In this example we have connections on two internet service providers (ISP) and we can to introduce 1:1 load balancing between both links. This can be done only adding simple static (default) route with two specified gateways:
[admin@MikroTik_PE1] /ip route> add dst-address=0.0.0.0/0 gateway=10.10.1.1,12.12.1.1 \ check-gateway=ping
This is typical ECMP (Equal Cost Multi-Path) gateway with check-gateway. In this case check-gateway send periodic ping request and as soon as one of the gateways will not be reachable, check-gateway will remove it from gateway list. ECMP is "persistent per-connection load balancing" – load balancing occur among the individual connections or "per-src-dst-address combination load balancing".
Show routing table from router:
[admin@MikroTik] /ip route> print
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 AS 0.0.0.0/0 10.10.1.1 1
12.12.1.1
1 ADC 192.168.11.0/24 192.168.11.1 ether2 0
There is to remember that not all case ECMP can offer real advantage, for example if your network physical topology of the system differs from the logical topology (system that use s VLANs), you have multiple logical paths to destination without real bandwidth improvement.
There also would need such router feature as Network Address Translation (NAT), which translate private IP addresses to public (ISP given). This feature will be explained later in chapter “Network address translation”.
NAT example for this case may be as follows:
[admin@MikroTik]/ ip firewall nat [admin@MikroTik]/ ip firewall nat> add chain=srcnat out-interface=ether1 action=masquerade [admin@MikroTik]/ ip firewall nat> add chain=srcnat out-interface=ether2 action=masquerade
You can also use asymmetric bandwidth links, for example, one link is 2Mbps other 10Mbps. Just use this command to make load balancing 1:5.
[admin@MikroTik]/ ip route add dst-address=0.0.0.0/0 gateway=10.10.1.1,12.12.1.1,12.12.1.1,12.12.1.1,12.12.1.1,12.12.1.1 \ check-gateway=ping
Dynamic routing
Unlike static routing where network administrator manually configures routing table entries (routes) dynamic routing allows create these entries dynamically by using some of routing protocols.
Major task of routing protocol is to determine the path of packet through network. A routing protocol is used by routers to dynamically find all the destination networks in the network and ensure that routing table is updated. Routing protocol provides collaboration among routers where one router communicates with the same protocol running on other (neighbor) routers. Some examples of routing protocols RIP, RIPv2 and OSPF there may be others protocols also implemented by different producers. Neighbor router – that reachable with one forwarding step, for example two directly connected routers. If occur some change in network (interface goes down, network topology change and so on), dynamic routing protocol automatically inform neighbor routers about all changes by sending updates massage. Update massages of routing protocol are used to help maintain and update entries of routing table on routers. Update process may be organized differently by each of routing protocols, for example RIP protocol send periodical updates whereas OSPF send updates when to occur some network changes. One router can maintains more than one routing protocols but each of routing protocol has it’s own internal table. This is where per-protocol routing decisions are made and best routes (route what algorithm of routing protocol is calculated as best) for each of destination networks are kept in the main (global) routing table, in MikroTik main table can be viewed with ip route print command.
More information about routing process organization in MiktroTik router can found here.
Note: Remember that process which found the gateway address where packet is routed next is called the nexthop lookup that is part of best route selection process.
Two major types of IGPs (Interior Gateway Protocol) protocols:
- Distance-vector routing protocols
- Link-state routing protocols
The distance-vector protocol calculates the best route to a remote network based only on link cost. The link cost to reaching destination is calculated using various route metrics. RIP protocol as metric uses the hop count to destination. One hop means when packet goes from one router to next and the route with the lowest number of hops to destination network is determined as the best route. The distance-vector means that routers are advertised as vector of distance and direction. The distance-vector protocol sends complete routing table contents to all its neighbors that are configured to use the same routing protocol. Receiver router combine the received table with own routing table to create the new updated routing table. RIP uses only hop count to determine the best path to a destination network, it can to perform round-robin load balancing when there are more than one link with the same hop count to the same remote destination. But there is also one drawback in case when we have more than one link to remote network with the same hop count but different bandwidths. For example if we have two links, first 1Mbps and second 10Mbps, then these links by RIP protocol would be seen as equal and can to perform one to one load balancing. Example of distance-vector protocols is RIPv2 too.
The link-state routing protocol send update massages that contain information about their own links not only to neighbors routers but to all other routers on the network. This type of routing protocol requires router to maintain network topology map (table) that provide connectivity map, showing which router are connected to which other. Connectivity map is like graph with all possible links between routers, maintained independently by each router and used for calculate the best next hop to every destination (called shortest path tree). The collection of all these best hops is populated to the routing table on each router.
OSPF and IS-IS are examples of link-state routing protocols.
Open Shortest Path First (OSPF)
OSPF concept
Open Shortest Path First (OSPF) is an open standard routing protocol defined RFC 2328 and available almost on every network router today. As other alternative is RIPv2 (Routing Information Protocol version 2), but it isn’t intended for large network, so really, your option for large network can be OSPF. OSPF also is one of most used protocols today because it provides more advantage in comparison with RIPv2 such as:
- Provides large network to divide into areas
- Low bandwidth usage (optimized updates sending)
- Unlimited hop count
- Fast convergence
OSPF support also multiple, equal-cost routes to the same destination and support ECMP (Equal-Cost Multi-Path) load balancing.
OSPF allow routers to dynamically learn routes from other routers and to advertise routes to other routers. As I mentioned previously that OSPF is link-state routing protocol therefore each OSPF router maintain own local routing topology map (link-state database) that is updated via link-state advertisements (LSAs). Are many types of link state advertisements that contain link-state and routing information.
OSPF areas and router types
OSPF uses areas to organize a network into a hierarchical structure that allow to divide one the large network into more smaller called areas. Figure 6.2 shows a typical OSPF simple design.
Autonomous system
An autonomous system is a network or group of networks that are under a common administration and that have common routing policies. It basically means that all routers sharing the same routing information as in the same AS. External routing protocols (EGP) are used to communicate between AS.
Area
OSPF allows a single autonomous system AS divided into smaller groups called areas. An area is a set of routers that will share routing information among themselves therefore all routers within an area have identical topological databases. An area provides to reduce updates of OSPF protocol into single AS and also reduce the size of the topological database that each router must maintain. Routers are used by OSPF to maintain routing information within an area by sending Link State Advertisements (LSA) to other areas. Each area is identified by 32-bit Area ID. There are several types of routing information:
- intra-area routes - routes generated from within an area (destination belongs to the area).
- inter-area routes - routes originated from other areas, also called Summary Routes.
- external routes - routes originated from other routing protocols and that are injected into OSPF by redistribution
Backbone area
The backbone is the first (main) area you should always build in any network using OSPF and its area ID is always 0.0.0.0. All areas are connected directly to the OSPF backbone area that distributes routing information between others areas.
Backbone router
The routers that are part of the OSPF backbone. The backbone routers are connected only to others routers in the same backbone, and is therefore not part of any other area (non-backbone area).
Area border router (ABR)
Routers that belong to more than one area are called area border routers or ABR. An ABR is used to connect non-backbone areas to the backbone. They maintain a separate topological database for each area to which they are connected. You can create virtual links between any two area border routers that have an interface to a common non-backbone area. Virtual links are used for connecting the two non-backbone areas.
Autonomous system boundary router (ASBR)
Routers that exchange routing information with routers in other ASs are called autonomous system boundary routers. Any router from one AS can be an AS boundary router. They advertise externally learned routes throughout the AS.
OSPF operation
Link State Advertisments are flooded to all other routers in routing domain and each router generates link-state database from received LSAs. The link-state protocol's flooding algorithm ensures that each router has identical link-state database. Each router is calculating routing table based on this link-state database. Looking at the link-state database each routing domain router knows how many other routers are in the network, how many interfaces routers have, what networks link between routers connects, cost of each link and so on. There are several steps before OSPF network becomes fully functional:
- Neighbor discovery
- Database Synchronization
- Best routes calculation
Neighbor discovery
Routers that share the same network segment can become neighbors. There are several types of OSPF packets: Hello packet, Database Description (DD) packet, Link state request packet, link State Update packet and Link State Acknowledgment packet. Most of them are used link-state database synchronization except hello packet that is used for dynamic neighbor discovery. Neighbors are discovered by periodically sending OSPF Hello packets out of configured interfaces. Hello packets are sent periodically out of each interface using IP multicast address 224.0.0.5. By default Hello packets are sent out with 10 second interval. This interval can be configured by hello interval command under RouterOS. Router learns the existence of a neighboring router when it receives the neighbor's Hello in return.
Therefore there is important time interval how long router can wait for hello response from neighbor this called dead time interval. If Hello packets are not received within Dead interval (which by default is 40s) router assume that link to neighbor router is dead (DOWN). OSPF also requires that hello and dead time intervals must be the same between two neighbors.
OSPF protocol allows authentication process between OSPF neighbors before them can to become neighbors. MikroTik RouterOS provides two authentication methods, simple and MD5:
Simple authentication is plain text authentication method. Method is vulnerable to passive attacks, anybody with packet sniffer can easily get password. Method should be used only to protect OSPF from mis-configurations.
MD5 is a cryptographic authentication and is more preferred. Authentication-key, key-id and OSPF packet content is used to generate message digest that is added to the packet. Unlike the simple authentication method, key is not exchanged over the network.
But keep in mind that routers can to become neighbors, only when they used the same authentication method and password.
Two routers do not become neighbors unless the following conditions are met:
- Bidirectional communication is needed between two routers
- Interface on both routers are attached to the same area (equal area ID) with IP address from the same subnet.
- If authentication option are used it must be the same on both routers
- Hello and Dead time interval should be the same
Database Synchronization
The routing database synchronization is next step after the neighbor discovery. There are two types of database synchronizations:
- initial database synchronization
- reliable flooding.
Initial database synchronization will occur when neighbor discovery process is implemented for first time and OSPF neighborship is established between routers. Unsynchronized databases may lead to calculation of incorrect routing table. OSPF is using explicit database download when neighbor connections first come up. Instead of sending the entire database, OSPF router sends only its LSA headers in a sequence of OSPF Database Description (DD) packets. Router will send next DD packet only when previous packet is acknowledged. When entire sequence of DD packets has been received, router knows which LSAs it does not have and which LSAs are more recent. The router then sends Link State Request (LSR) packets requesting desired LSAs, and the neighbor responds by flooding LSAs in Link State Update (LSU) packets. After all updates are received neighbors are said to be fully adjacent.
Reliable flooding is another database synchronization method. It is used when adjacencies are already established and OSPF router wants to inform other routers about LSA changes. Router that received the new LSA update install it in own link-state database and sends an acknowledgement packet back to sender, as soon as sends information about new update out to all interfaces except the one that received the LSA in the first place.
Databases are not always synchronized between all OSPF neighbors, OSPF decides whether databases needs to be synchronized depending on network segment, for example, on point-to-point links databases are always synchronized between routers, but on Ethernet networks databases are synchronized between certain neighbor pairs.
On broadcast segment there are n*(n-1)/2 neighbor relations, it will be huge amount of Link State Updates and Acknowledgements sent over the subnet if OSPF router will try to synchronize with each OSPF router on the subnet.
In order to minimize the amount of OSPF update information exchange on the same subnet OSPF elects Designated Router and one Backup Designated Router for each broadcast subnet. BDR is as backup DR router in case when BDR goes down. The purpose of this selection is pointed at a central point of contact for information exchange. Instead of each router exchanging update with each other on the same subnet (segment) every router exchanges information with DR or BDR.
DR and BDR election is formed via hello packets. Router with the highest priority becomes the DR. If the all routers have the same priority then router with highest IP address will become the DR. The router that have priority to set to 0 (zero), can’t participate in the DR and BDR election process.
Best routes calculation
When router’s database is synchronized it can calculate the best route to each destination. OSPF router is using Dijkstra's Shortest Path First (SPF) algorithm to calculate shortest path. The algorithm places router at the root of a tree and calculates shortest path to each destination based on the cumulative cost (link metric) required to reach the destination. Each router calculates own SPF tree to reach all destination in the same area. This calculation process is based on information that is in the link-state database (topology database). The link-state database contain information about all available links among routers and associated cost (metric) of link which is value form 1 – 65535 where 1 mean this best link and 65535 that this link can’t be used for traffic forwarding. All router interfaces in OSPF domain are given a cost. OSPF selects the best routes (Shortest Path first – SPF) by finding the lowest cost paths to a destination. The full cost to destination is the sum of the cost of the outgoing interfaces along the path. Example how to calculate best route (SPF) is given below:
Example of network consists of four routers where cost values are assigned by each of outgoing interface of router. In this case R1 is root router and calculate the smallest cost to each destination. For example, there is only one path to directly connected network 192.168.1.0 with cost 0. The cost to 192.168.2.0 destination is sum of two cost (2+3)=5. As you can see there also are two equal cost paths to 215.15.15.0 network, one through R2 and R3 and other through R4, allowing load balancing of the traffic to this destination called Equal Cost Multi-Path. When this calculation is done router can starts to build the routing table accordingly.
The Cost in RouterOS is set to 10 on all interfaces by default. Value can be changed in ospf interface configuration menu, for example to add ether2 interface with cost of 20:
[admin@MikroTik]/routing ospf interface> add interface=ether2 cost=20
Each router can advertise a different cost for the router's own link direction, making it possible to have asymmetric links (packets to destination travels over one path, but response travels different path). Asymmetric paths are not recommended, because it makes harder to find routing problems.
OSPF multiple area design and operation
Large single area network can produce serious issues:
- Each router recalculates database every time whenever network topology change occurs, this takes CPU resources (larger network require more CPU performance).
- Each router holds entire link-state database, which shows the topology of the entire network, it takes memory resources.
- Updating large databases require more bandwidth.
To divide large OSPF area in smaller areas we achieve:
- Save CPU resources
- Save router memory resources
- Save bandwidth usage between routers
OSPF use two type of area, backbone area that is also known as area 0 (area-id always is 0.0.0.0), and sub-areas which are connected to backbone area. The communication between sub-areas occurs over backbone. Each router in the same area is maintained own topology database and knows full information how all routers within area are interconnected but doesn’t knows detailed information about others topology of areas.
Area Border Router (ABR) leak one area’s summary addresses information into another by using Summary-LSA advertisements.
Policy routing implementation example
Basically when you implement routing you have set of routes (static or dynamic) that tell the router how to reach certain destination. Policy routing does the same task, but may be different routes for various types of traffic. All routes by default are installed into main routing table which content can be displayed with following command:
[admin@MikroTik] /ip route> print where routing-mark=main
All routes with appropriate routing-mark are shown using command:
[admin@MikroTik] /ip route> print or [admin@MikroTik] /ip route> print where routing-mark=all
In this case routing-mark denote appropriate routing table.
When we implement the policy routing there can be used others routing tables that includes routes for specific traffic or destination network. In this example, are explored the requirements for setting up policy routing and is given simple implementation example.
Implementation of policy routing consists of 3 steps:
- The first step is to define the routes and which policies (routing table) will use those routes.
- The second step to define the routing rules, which will define how the policies apply to certain traffic.
- The third is to define the actual policies. We’ll look at each of these individually.
The network below is the one we will use for this example.
Task of this example is to create routing policy that route all traffic from LAN 192.168.1.0/24 over provider “ISP-A” and all traffic from 192.168.2.0/24 via provider ISP-B.
Set up routing policy using ip route rule
I assume that IP addresses already have set up on your router. We will add three “default” routes for each of routing table. These are below:
/ip route add dst-address=0.0.0.0/0 gateway=11.1.1.1 routing-mark=Table_A /ip route add dst-address=0.0.0.0/0 gateway=12.1.1.1 routing-mark=Table_B /ip route add dst-address=0.0.0.0/0 gateway=11.1.1.1
The first two routes will be used by our policies, one for routing table Table_A and one for Table_B. The third route will be used by any traffic does not have a policy defined and by traffic from the router itself (it refered to main routing table), to define routing-mark=main is not mandatory because it is by default.
Next, we need to define our routing rules (policy). There are several ways to accomplish this, and one of simplest way is given below.
/ip route rule>add src-address=192.168.1.0/24 action=lookup table=Table_A /ip route rule>add src-address=192.168.2.0/24 action=lookup table=Table_B
The first two rules mean that traffic that comes from LAN 192.168.1.0/24 lookup for next hop IP address in routing table Table_A and traffic from 192.168.2.0/24 using the routing table Table_B.
Note: If routes entries in specific routing table doesn’t satisfy particular traffic router looks for next hop (gateway) in the main routing table.
Policy basically means “which routing table use for this type of traffic”.
Set up routing policy using ip firewall mangle
Other way how to implement the same policy is by using such RouterOS feature as traffic marking. Packets marking in RouterOS is available under /ip firewall mangle from command line interface. More information about them will be later in chapter “Network Security” section “Packet Markin”.
The marking process marks incoming packets according to various conditions in this case condition will be source IP address of packet. How to mark traffic from the LAN 192.168.1.0/24 with routing-mark Table_A and all traffic form network 192.168.2.0/24 with routing-mark Table_A is given below.
Here is example how it is implemented it:
/ip firewall mangle> add chain=prerouting src-address=192.168.1.0/24 action=mark-routing \ new-routing-mark=Table_A passthrough=no /ip firewall mangle> add chain=prerouting src-address=192.168.2.0/24 action=mark-routing \ new-routing-mark= Table_B passthrough=no
Here is example how to create routing policy by traffic type.
For example, we could implement a policy like the following:
Route all HTTP, SMTP, POP3 and DNS traffic over to internet provider ISP-A and all other traffic over the ISP-B network.
;;;HTTP traffic /ip firewall mangle> add chain=prerouting dst-port=80 protocol=tcp action=mark-routing \ new-routing-mark=Table_A passthrough=no ;;;SMTP traffic /ip firewall mangle> add chain=prerouting dst-port=25 protocol=tcp action=mark-routing \ new-routing-mark=Table_A passthrough=no ;;;POP3 traffic /ip firewall mangle> add chain=prerouting dst-port=110 protocol=tcp action=mark-routing \ new-routing-mark=Table_A passthrough=no ;;;DNS traffic /ip firewall mangle> add chain=prerouting dst-port=53 protocol=udp action=mark-routing \ new-routing-mark=Table_A passthrough=no /ip firewall mangle> add chain=prerouting action=mark-routing new-routing-mark=Table_B \ passthrough=no
This was a brief overview how to implement policy routing here was not complete description of all the possible implementations, but it will give example to in implement it.