Manual:PCC: Difference between revisions

From MikroTik Wiki
Jump to navigation Jump to search
 
(42 intermediate revisions by 5 users not shown)
Line 1: Line 1:
==Introduction==
{{Versions|v3, v4}}


Starting from RouterOS version 3.24 there is a new option available in the firewall - '''PCC''' (Per Connection Classifier). This option was introduced to address the configuration issues with load balancing over multiple gateways with masquerade
== Introduction ==


Previous configurations:
PCC matcher will allow you to divide traffic into equal streams with ability to keep packets with specific set of options in one particular stream (you can specify this set of options from src-address, src-port, dst-address, dst-port)
* [[ECMP load balancing with masquerade]]
* [[NTH load balancing with masquerade]]
* [[NTH load balancing with masquerade (another approach)]]


PCC takes selected fields from IP header (you can choose from src-address, dst-address, src-port, dst-port) and with a help of hashing algorithm convert selected fields into 32-bit value. This value then is divided by a "Denominator", the remainder then is compared to a specified "Remainder", if equal then packet will be captured.  
===Theory===
PCC takes selected fields from IP header, and with the help of a hashing algorithm converts selected fields into 32-bit value. This value then is divided by a specified ''Denominator'' and the remainder then is compared to a specified ''Remainder'', if equal then packet will be captured. You can choose from src-address, dst-address, src-port, dst-port from the header to use in this operation. 


<pre>
<pre>
Line 15: Line 13:
   Remainder ::= 0..4294967295    (integer number)
   Remainder ::= 0..4294967295    (integer number)
   Denominator ::= 1..4294967295    (integer number)
   Denominator ::= 1..4294967295    (integer number)
   ValuesToHash ::= src-address|dst-address|src-port|dst-port[,ValuesToHash*]
   ValuesToHash ::= both-addresses|both-ports|dst-address-and-port|
  src-address|src-port|both-addresses-and-ports|dst-address|dst-port|src-address-and-port  
</pre>
</pre>


Example - this configuration will divide all connections into 3 streams based on source address and port
===Example===
This configuration will divide all connections into 3 groups based on source address and port
 
<pre>
<pre>
/ip firewall mangle add chain=prerouting action=mark-connection new-connection-mark=1st_conn per-connection-classifier=src-address,src-port:3/0
/ip firewall mangle add chain=prerouting action=mark-connection \
/ip firewall mangle add chain=prerouting action=mark-connection new-connection-mark=2nd_conn per-connection-classifier=src-address,src-port:3/1
new-connection-mark=1st_conn per-connection-classifier=src-address-and-port:3/0
/ip firewall mangle add chain=prerouting action=mark-connection new-connection-mark=3rd_conn per-connection-classifier=src-address,src-port:3/2
/ip firewall mangle add chain=prerouting action=mark-connection \
  new-connection-mark=2nd_conn per-connection-classifier=src-address-and-port:3/1
/ip firewall mangle add chain=prerouting action=mark-connection \
  new-connection-mark=3rd_conn per-connection-classifier=src-address-and-port:3/2
</pre>
</pre>
===Notes===
PCC is available in RouterOS since v3.24. This option was introduced to address configuration issues with load balancing over multiple gateways with masquerade
Previous configurations:
* [[ECMP load balancing with masquerade]]
* [[NTH load balancing with masquerade]]
* [[NTH load balancing with masquerade (another approach)]]
{{Note | PCC setups is not designed to work if [[Manual:IP/Settings | '''RP Filter''']] is enabled}}
==Application Example - Load Balancing==


Consider the following network layout:
Consider the following network layout:


[[Image:LoadBalancing.jpg]]
[[Image:LoadBalancing.png]]


==Quick Start for Impatient==
====Quick Start for Impatient====
Configuration export from the gateway router:
Configuration export from the gateway router:
<pre>
<pre>
/ ip address
/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=LAN
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=ISP1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=ISP2


/ ip firewall mangle
/ ip firewall mangle
add chain=input in-interface=wlan1 action=mark-connection new-connection-mark=wlan1_conn
add chain=prerouting dst-address=10.111.0.0/24  action=accept in-interface=LAN
add chain=input in-interface=wlan2 action=mark-connection new-connection-mark=wlan2_conn
add chain=prerouting dst-address=10.112.0.0/24  action=accept in-interface=LAN
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wla1   
add chain=prerouting in-interface=ISP1 connection-mark=no-mark action=mark-connection \
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wla2   
    new-connection-mark=ISP1_conn
add chain=prerouting dst-address-type=!local in-interface=Local per-connection-classifier=src-address,dst-address:2/0 \
add chain=prerouting in-interface=ISP2 connection-mark=no-mark action=mark-connection \
    action=mark-connection new-connection-mark=wlan1_conn passthrough=yes
    new-connection-mark=ISP2_conn
add chain=prerouting dst-address-type=!local in-interface=Local per-connection-classifier=src-address,dst-address:2/1 \
add chain=prerouting in-interface=LAN connection-mark=no-mark dst-address-type=!local \
    action=mark-connection new-connection-mark=wlan2_conn passthrough=yes
    per-connection-classifier=both-addresses:2/0 action=mark-connection new-connection-mark=ISP1_conn
add chain=prerouting connection-mark=wlan1_conn in-interface=Local action=mark-routing new-routing-mark=to_wlan1 passthrough=yes
add chain=prerouting in-interface=LAN connection-mark=no-mark dst-address-type=!local \
add chain=prerouting connection-mark=wlan2_conn in-interface=Local action=mark-routing new-routing-mark=to_wlan2 passthrough=yes
    per-connection-classifier=both-addresses:2/1 action=mark-connection new-connection-mark=ISP2_conn
add chain=prerouting connection-mark=ISP1_conn in-interface=LAN action=mark-routing \
    new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=LAN action=mark-routing \
    new-routing-mark=to_ISP2
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1   
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2


/ ip route
/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_wla1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_ISP1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_wla2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_ISP2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping


/ ip firewall nat  
/ ip firewall nat  
add chain=srcnat out-interface=wlan1 action=masquerade
add chain=srcnat out-interface=ISP1 action=masquerade
add chain=srcnat out-interface=wlan2 action=masquerade
add chain=srcnat out-interface=ISP2 action=masquerade
</pre>
</pre>


===Explanation===


Let's assume this configuration:


 
====IP Addresses====
 
 
 
 
==Explanation==
First we give a code snippet and then explain what it actually does.
===IP Addresses===
<pre>
<pre>
/ ip address  
/ ip address  
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=LAN
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=ISP1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=ISP2
</pre>
</pre>
The router has two upstream (WAN) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24.
The LAN interface has the name "Local" and IP address of 192.168.0.1/24.


The router has two upstream (ISP) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24.
The LAN interface has IP address of 192.168.0.1/24.


===NAT===
====Policy routing====
<pre>
<pre>
/ ip firewall nat
/ ip firewall mangle
add chain=srcnat out-interface=wlan1 action=masquerade
add chain=prerouting dst-address=10.111.0.0/24  action=accept in-interface=LAN
add chain=srcnat out-interface=wlan2 action=masquerade
add chain=prerouting dst-address=10.112.0.0/24  action=accept in-interface=LAN
</pre>
</pre>
With policy routing it is possible to force all traffic to the specific gateway, even if traffic is destined to the host (other that gateway) from the connected networks. This way routing loop will be generated and communications with those hosts will be impossible.
To avoid this situation we need to allow usage of default routing table for traffic to connected networks.


As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. if this packet will leave via wlan1 it will be NATed to 10.112.0.2/24,
<pre>
if via wlan2 then NATed to 10.111.0.2/24 
add chain=prerouting in-interface=ISP1 connection-mark=no-mark action=mark-connection \
    new-connection-mark=ISP1_conn
add chain=prerouting in-interface=ISP2 connection-mark=no-mark action=mark-connection \
    new-connection-mark=ISP2_conn
</pre>
First it is necessary to manage connection initiated from outside - replies must leave via same interface (from same Public IP) request came.
We will mark all new incoming connections, to remember what was the interface.


<pre>   
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \
    per-connection-classifier=both-addresses:2/0 action=mark-connection new-connection-mark=ISP1_conn
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \
    per-connection-classifier=both-addresses:2/1 action=mark-connection new-connection-mark=ISP2_conn
</pre>
Action mark-routing can be used only in mangle chain '''output''' and '''prerouting''', but mangle chain '''prerouting''' is capturing all traffic that is going to the router itself.
To avoid this we will use ''dst-address-type=!local''. And with the help of the new PCC we will divide traffic into two groups based on source and destination addressees.


===Routing===
<pre>
<pre>
/ ip route
add chain=prerouting connection-mark=ISP1_conn in-interface=LAN action=mark-routing \
add dst-address=0.0.0.0/0 gateway=10.111.0.1,10.112.0.1 check-gateway=ping
    new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=LAN action=mark-routing \
    new-routing-mark=to_ISP2
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1   
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2
</pre>
</pre>


This is typical ECMP (Equal Cost Multi-Path) gateway with check-gateway. ECMP is "persistent per-connection load balancing" or "per-src-dst-address combination load balancing".
Then we need to mark all packets from those connections with a proper mark. As policy routing is required only for traffic going to the Internet, do not forget to specify in-interface option.
As soon as one of the gateway will not be reachable, check-gateway will remove it from gateway list. And you will have a "failover" effect.
 


You can use asymmetric bandwidth links also  - for example one link is 2Mbps other 10Mbps. Just use this command to make load balancing 1:5
<pre>
<pre>
/ ip route  
/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1,10.112.0.1,10.112.0.1,10.112.0.1,10.112.0.1,10.112.0.1 check-gateway=ping  
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_ISP1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_ISP2 check-gateway=ping
</pre>
</pre>


===Connections to the router itself===
Create a route for each routing-mark


<pre>
<pre>
/ ip firewall mangle
add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add chain=input in-interface=wlan1 action=mark-connection new-connection-mark=wlan1_conn
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping
add chain=input in-interface=wlan2 action=mark-connection new-connection-mark=wlan2_conn
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wlan1   
add chain=output connection-mark=wlan1_conn action=mark-routing new-routing-mark=to_wlan2   
</pre>
</pre>
To enable failover, it is necessary to have routes that will jump in as soon as others will become inactive on gateway failure. (and that will happen only if check-gateway option is active)
====NAT====


<pre>
<pre>
/ ip route
/ ip firewall nat
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_wlan1
add chain=srcnat out-interface=ISP1 action=masquerade
add dst-address=0.0.0.0/0 gateway=10.111.0.2 routing-mark=to_wlan2
add chain=srcnat out-interface=ISP2 action=masquerade
</pre>
</pre>


With all multi-gateway situations there is a usual problem to reach router from public network via one, other or both gateways. Explanations is very simple -  
As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. If this packet will leave via wlan1 it will be NATed to 10.112.0.2,
Outgoing packets uses same routing decision as packets that are going trough the router. So reply to a packet that was received via wlan1 might be send out and masqueraded via wlan2.
if via wlan2 then NATed to 10.111.0.2  
 
To avoid that we need to policy routing those connections.
 
==Known Issues==
 
===DNS issues===
 
ISP specific DNS servers might have custom configuration that treats specific requests from ISP's network differently than requests from other network. So in case connection is made via other gateway those sites will not be accessible.
 
To avoid that we suggest to use 3rd-party (public) DNS servers, and in case you need ISP specific recourse, create static DNS entry and policy route that traffic to specific gateway.
 
===Routing table flushing===
 
Every time when something triggers flush of the routing table and ECMP cache is flushed. Connections will be assigned to gateways once again and may or may not be on the same gateway.(in case of 2 gateways there are 50% chance that traffic will start to flow via other gateway).  
 
 
If you have fully routed network (clients address can be routed via all available gateway), change of the gateway will have no ill effect, but in case you use
masquerade, change of the gateway will result in change of the packet's source address and connection will be dropped.  
 
 
Routing table flush can be caused by  2 things:
 
1) routing table change  (dynamic routing protocol update, user manual changes)


2) every 10 minutes routing table is flushed for security reasons (to avoid possible DoS attacks)
== Related Articles ==


'''So even if you do not have any changes of routing table, connections may jump to other gateway every 10 minutes'''
[[Manual:Hotspot_with_PCC | Hotspot server with PCC]]


[[Category: Manual]]
[[Category: Manual|PCC]]
[[Category: IP|PCC]]
[[Category: Firewall|PCC]]
[[Category: Examples|PCC]]

Latest revision as of 08:44, 26 May 2017

Version.png

Applies to RouterOS: v3, v4

Introduction

PCC matcher will allow you to divide traffic into equal streams with ability to keep packets with specific set of options in one particular stream (you can specify this set of options from src-address, src-port, dst-address, dst-port)

Theory

PCC takes selected fields from IP header, and with the help of a hashing algorithm converts selected fields into 32-bit value. This value then is divided by a specified Denominator and the remainder then is compared to a specified Remainder, if equal then packet will be captured. You can choose from src-address, dst-address, src-port, dst-port from the header to use in this operation. 

per-connection-classifier=
PerConnectionClassifier ::= [!]ValuesToHash:Denominator/Remainder
  Remainder ::= 0..4294967295    (integer number)
  Denominator ::= 1..4294967295    (integer number)
  ValuesToHash ::= both-addresses|both-ports|dst-address-and-port|
  src-address|src-port|both-addresses-and-ports|dst-address|dst-port|src-address-and-port

Example

This configuration will divide all connections into 3 groups based on source address and port 

/ip firewall mangle add chain=prerouting action=mark-connection \
 new-connection-mark=1st_conn per-connection-classifier=src-address-and-port:3/0
/ip firewall mangle add chain=prerouting action=mark-connection \
  new-connection-mark=2nd_conn per-connection-classifier=src-address-and-port:3/1
/ip firewall mangle add chain=prerouting action=mark-connection \
  new-connection-mark=3rd_conn per-connection-classifier=src-address-and-port:3/2

Notes

PCC is available in RouterOS since v3.24. This option was introduced to address configuration issues with load balancing over multiple gateways with masquerade

Previous configurations:


Icon-note.png

Note: PCC setups is not designed to work if RP Filter is enabled


Application Example - Load Balancing

Consider the following network layout:

LoadBalancing.png

Quick Start for Impatient

Configuration export from the gateway router: 

/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=LAN
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=ISP1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=ISP2

/ ip firewall mangle
add chain=prerouting dst-address=10.111.0.0/24  action=accept in-interface=LAN
add chain=prerouting dst-address=10.112.0.0/24  action=accept in-interface=LAN
add chain=prerouting in-interface=ISP1 connection-mark=no-mark action=mark-connection \
    new-connection-mark=ISP1_conn
add chain=prerouting in-interface=ISP2 connection-mark=no-mark action=mark-connection \ 
    new-connection-mark=ISP2_conn
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \
    per-connection-classifier=both-addresses:2/0 action=mark-connection new-connection-mark=ISP1_conn 
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \ 
    per-connection-classifier=both-addresses:2/1 action=mark-connection new-connection-mark=ISP2_conn
add chain=prerouting connection-mark=ISP1_conn in-interface=LAN action=mark-routing \ 
    new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=LAN action=mark-routing \
    new-routing-mark=to_ISP2
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1     
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_ISP1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_ISP2 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping

/ ip firewall nat 
add chain=srcnat out-interface=ISP1 action=masquerade
add chain=srcnat out-interface=ISP2 action=masquerade

Explanation

Let's assume this configuration:

IP Addresses

/ ip address 
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=LAN
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=ISP1
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=ISP2

The router has two upstream (ISP) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The LAN interface has IP address of 192.168.0.1/24.

Policy routing

/ ip firewall mangle
add chain=prerouting dst-address=10.111.0.0/24  action=accept in-interface=LAN
add chain=prerouting dst-address=10.112.0.0/24  action=accept in-interface=LAN

With policy routing it is possible to force all traffic to the specific gateway, even if traffic is destined to the host (other that gateway) from the connected networks. This way routing loop will be generated and communications with those hosts will be impossible. To avoid this situation we need to allow usage of default routing table for traffic to connected networks. 

add chain=prerouting in-interface=ISP1 connection-mark=no-mark action=mark-connection \
    new-connection-mark=ISP1_conn
add chain=prerouting in-interface=ISP2 connection-mark=no-mark action=mark-connection \ 
    new-connection-mark=ISP2_conn

First it is necessary to manage connection initiated from outside - replies must leave via same interface (from same Public IP) request came. We will mark all new incoming connections, to remember what was the interface. 

     
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \
    per-connection-classifier=both-addresses:2/0 action=mark-connection new-connection-mark=ISP1_conn 
add chain=prerouting  in-interface=LAN connection-mark=no-mark dst-address-type=!local \ 
    per-connection-classifier=both-addresses:2/1 action=mark-connection new-connection-mark=ISP2_conn

Action mark-routing can be used only in mangle chain output and prerouting, but mangle chain prerouting is capturing all traffic that is going to the router itself. To avoid this we will use dst-address-type=!local. And with the help of the new PCC we will divide traffic into two groups based on source and destination addressees. 

add chain=prerouting connection-mark=ISP1_conn in-interface=LAN action=mark-routing \ 
    new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=LAN action=mark-routing \
    new-routing-mark=to_ISP2
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1     
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2

Then we need to mark all packets from those connections with a proper mark. As policy routing is required only for traffic going to the Internet, do not forget to specify in-interface option. 

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_ISP1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_ISP2 check-gateway=ping

Create a route for each routing-mark 

add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping

To enable failover, it is necessary to have routes that will jump in as soon as others will become inactive on gateway failure. (and that will happen only if check-gateway option is active)

NAT

/ ip firewall nat 
add chain=srcnat out-interface=ISP1 action=masquerade
add chain=srcnat out-interface=ISP2 action=masquerade

As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. If this packet will leave via wlan1 it will be NATed to 10.112.0.2, if via wlan2 then NATed to 10.111.0.2

Related Articles

Hotspot server with PCC

Retrieved from "https://wiki.mikrotik.com/index.php?title=Manual:PCC&oldid=29301"