Testwiki/Quality of Service

Quality of Service

Queuing disciplines

Before we start to speak about queue and queuing disciplines (algorithms) we need to understand where queue is allocated, what is queuing and how it is implemented.

Buffer memory are typically used for queue allocation and it is used when there is a difference between the rate at which data is received and the rate at which it can be sent.

Queuing is the process of sequencing packets before they leave a router interface and it decides the order that packets leave any given interface. Normally, packets leave the router in the order they arrived. The queuing is only useful for packets in the outbound direction because packet arrives on an interface in the inbound direction it's already too late to queue it - it's already consumed network bandwidth. Which of queuing strategy to implement in your network is absolute your decision.

When we speak about queues we need understand two basic classes that related to traffic flows control algorithms in buffer memory on router interface:

1. Queue management – control buffer availability and length of queue as well as leads packet rejection (drops packet) when it is necessary.
2. Queuing (scheduling) algorithms – its describes which packet will be transmitted next in line

Queue interface (/queue interface)

Before sending data over an interface, it is processed with a queuing discipline. By default, queuing disciplines are set under /queue interface for each physical interface.

For each available interface type (Ethernet, wireless, bridge, PPTP and others) RouterOS by default is assigned some of default defined queue types that can be changed. You can to check assigned queue type for each interface using command:

[admin@Augsha] /queue interface> print 
Flags: D - dynamic 
 #   INTERFACE                               QUEUE                             
 0   ether1                                  ethernet-default                  
 1   ether2                                  ethernet-default                  
 2   ether3                                  ethernet-default                                          
 3   vlan3                                   default                           
 4   vlan3-bridge                            default                           
 5   vrrp1                                   default                           
 6   pptp-in1                                default                           
 7   etunels                                 default                           
 8  bridge101                                default   

You can change queue type for interface like this:

[admin@Augsha] /queue interface> set 3 queue=myqueue

Generally, we can set only one queue for an interface but as you can see later in this section, then using HTB we can to define many of queues (queue hierarchy) per interface, where each queue is for different client, client group, subnet, port or other parameter.

Simple Queue

The simplest way to limit data rate for specific IP addresses and/or subnets, is to use simple queues.

You can also use simple queues to build advanced QoS applications. They have useful integrated features:

• Peer-to-peer traffic queuing
• Applying queue rules on chosen time intervals
• Priorities
• Using multiple packet marks from /ip firewall mangle
• Shaping (scheduling) of bidirectional traffic (one limit for the total of upload + download)

Assume we have network topology like Figure 8.6 and we want to limited download and upload for private network (upload - 256kbps, and download – 512kbps).

File:Image8006.gif

Add a simple queue rule, which will limit the download traffic to 512kbps and upload to 256kbps for the network 10.1.1.0/24, served by the interface Ether2:

[admin@MikroTik] /queue simple> add name=private target-addresses=10.1.1.0/24 max-limit=256K/512K \
interface=ether2

In this case statement works right also if we indicate only one of parameters: "target-addresses=" or "interface=", because both of these define where and for which traffic this queue will be implemented.

Check your configuration:

[admin@Augsha] /queue simple> print 

Flags: X - disabled, I - invalid, D - dynamic 
 0    name="private" target-addresses=10.1.1.0/24 dst-address=0.0.0.0/0 
      interface=ether2 parent=none direction=both priority=8 
      queue=default-small/default-small limit-at=0/0 max-limit=256k/512k 
      burst-limit=0/0 burst-threshold=0/0 burst-time=0s/0s 
      total-queue=default-small

The max-limit parameter cuts down the maximum available bandwidth. The value max-limit=256k/512k means that clients from private network will get maximum of 512kbps for download and 256kbps for upload. The target-addresses allows to define the source IP addresses to which the queue rule will be applied.

Probably, you want to exclude the server from being limited, if so, add a queue for it without any limitation (max-limit=0/0 which means no limitation). Move this rule to the beginning of the list, because items in /queue simple are executed in order one by one if router finds rule that satisfy certain packet next rules aren’t compared:

[admin@MikroTik] /queue simple> add name=server target-addresses=10.1.1.1/32 max-limit=0/0 \
interface=ether2

All simple queue command references are available here.

Queue Tree

HTB (Hierarchical Token Bucket) is a classful queuing discipline that is useful for applying different handling for different kinds of traffic. When we create queue tree we have always followed three basic steps:

• Matching and marking traffic'' – classify traffic for further use. Consists of one or more matching parameters to select packets for the specific class.
• Create rules (policy) for marking traffic – put specific traffic class into specific queue and to define the actions that are taken for each class.
• Attach policy'for specific interface(-s) – append policy for all interfaces (global-in, global-out or global-total), for specific interface or for specific parent queue.

Queue tree creates only one directional queue in one of the HTBs. Hierarchical Token Bucket allows to create a hierarchical queue structure and to determine relations between queues. Hierarchy is implemented with different level of queues and new term is established like parent queue. Look at next Figure 8.7, for example of HTB hierarchy where Queue_A is parent queue for Queue_C and Queue_B, but Queue_B is parent queue for Queue_D and Queue_E. Queues without any sub-queues (children) like Queue_C, Queue_D, Queue_E in this case called as leaf queues. Leaf queues make actual traffic consumption whereas parent queues are responsible only for traffic distribution. All leaf queues are treated on equal basis. In RouterOS it is necessary to specify parent option to assign queue as a child to other queue.

File:Image8007.gif

For example, using HTB (/queue tree) we can set a maximum data rate for a workgroup and then distribute that amount of traffic between the members of that workgroup.

Each queue in the HTB hierarchy has three important parameters: There are 2 major parameters in RouterOS which refer to rates limit and one - to prioritizing:

• limit-at (Committed Information Rate - CIR) - data rate that is guaranteed to traffic flow (class) within specific queue (amount of available traffic rate regardless of other traffic flows)
• max-limit (Maximum Information Rate - MIR) - maximal data rate that is allowed for a traffic flow within specific queue if there is free any part of parent queue bandwidth.
• priority - order in which classes are served at the same level (8 is the lowest priority, 1 is the highest)

In other words, at first limit-at of the all queues will be satisfied, only then child queues will try to borrow the necessary data rate from their parents in order to reach their max-limit.

NOTE CIR will be assigned to the corresponding queue no matter what.

Priority is responsible for distribution of remaining parent queues traffic to child queues so that they are able to reach max-limit

The Queue with higher priority will reach its max-limit before the queue with lower priority.

For example (refer to Figure 8.7), if we assume that parent Queue_B (max-limit=limit-at=512kbps) and its child queues is set as follows: Queue_D (max-limit=512kbps limit-at=128kbps), Queue_E (max-limit=512kbps limit-at=384kbps) both of child queues have the same priority. It means that for Queue_D guaranteed bandwidth is 128kbps and for Queue_E 384kbps. Queue_D can to reach data rate 512kbps when Queue_E is empty (no data to send). Or if Queue_E uses only 256Kbps from allowed data rate, queue_D gets also 256Kbps (all remaining part of available bandwidth).

Sum of committed rates of all children must be less or equal to amount of traffic that is available to parent: CIR(parent)* ? CIR(child1) + CIR(child1)...+ CIR(childN). In case if parent is main parent CIR(parent)=MIR(parent)

Maximal rate (MIR) of any child must be less or equal to maximal rate of the parent.

HTB configuration example

Assume that our network diagram is as Figure 8.6, and we want to limit maximum download speed for subnet 10.1.1.0/24 to 2Mbps and distribute this amount of traffic between the server and workstations like Figure 8.9 using HTB. To limit upload to 2Mbps and distribute upload bandwidth like Figure 8.8. As remember that HTB work in one direction and is implemented on outbound interface. If we referring to Figure 8.6 then HTB for download is introduced on ether2 whereas HTB for upload is on ether1.

File:Image8008.gif

File:Image8009.gif

The first, we need to classify traffic.

Mark traffic form/to server. The first rule we will mark the outgoing connection from server and with the second one, all packets, which belong to this connection (download and upload packets for this connection):

/ip firewall mangle> add chain=prerouting src-address=10.1.1.1/32 action=mark-connection \
new-connection-mark=server_con

/ip firewall mangle> add chain=forward connection-mark=server_con action=mark-packet  \
new-packet-mark=server

Do the same for workstation too. Match all workstation connections, mark it with the same mark (new-connection-mark=workstation_con) and after that mark all packets which belong to these workstation.

/ip firewall mangle> add chain=prerouting src-address=10.1.1.2 
action=mark-connection new-connection-mark=workstation_con
/ip firewall mangle> add chain=prerouting src-address=10.1.1.3 
action=mark-connection new-connection-mark=workstation_con
/ip firewall mangle> add chain=prerouting src-address=10.1.1.4 
action=mark-connection new-connection-mark=workstation_con

/ip firewall mangle> add chain='''forward''' connection-mark=workstation_con  action=mark-packet \
new-packet-mark=workstations

At the end create /queue tree for upload and download based on figure 8.8 and figure 8.9.

Queue tree for upload limitation is implemented on ether1 interface.

 ;;; Queue_A1 creation 
/queue tree> add name=Queue_A1 parent='''ether1''' max-limit=2048k 

 ;;; Queue_B1 creation 
/queue tree> add name=Queue_B1 parent=Queue_A1 max-limit=2048k limit-at=1024k

 ;;; Queue_C1 creation 
/queue tree> add name=Queue_C1 parent=Queue_A1 max-limit=2048k limit-at=1024k priority=7 \
packet-mark=server
 
 ;;; Queue_D1, Queue_E1 and Queue_F1 creation 
 /queue tree> add name=Queue_D1 parent=Queue_B1 max-limit=2048k limit-at=340k priority=8 \
 packet-mark=workstations
 /queue tree> add name=Queue_E1 parent=Queue_B1 max-limit=2048k limit-at=340k priority=8 \
 packet-mark=workstations
 /queue tree> add name=Queue_F1 parent=Queue_B1 max-limit=2048k limit-at=340k priority=8 \
 packet-mark=workstations

Priority value by default is 8 so it is not specified here.    

Queue tree for download limitation is implemented on ether2 interface.

 ;;; Queue_A2 creation
/queue tree> add name=Queue_A2 parent='''ether1''' max-limit=2048k 

 ;;; Queue_B2 creation 
/queue tree> add name=Queue_B2 parent=Queue_A2 max-limit=2048k limit-at=1536k

 ;;; Queue_C creation 
 /queue tree> add name=Queue_C2 parent=Queue_A2 max-limit=2048k limit-at=512k priority=7 \
 packet-mark=server

 ;;; Queue_D2, Queue_E2 and Queue_F2 creation
 /queue tree> add name=Queue_D2 parent=Queue_B2 max-limit=2048k limit-at=512k priority=8 \
 packet-mark=workstations
 /queue tree> add name=Queue_E2 parent=Queue_B2 max-limit=2048k limit-at=512k priority=8 \
 packet-mark=workstations
 /queue tree> add name=Queue_F2 parent=Queue_B2 max-limit=2048k limit-at=512k priority=8 \
 packet-mark=workstations

Configuring PCQ

Per Connection Queue (PCQ) was introduced to optimize massive QoS systems, where most of the queues are exactly the same for different sub-streams. PCQ allows dynamically and equally share download and upload traffic (called also as “bandwidth equal distribution between users”). It means that if you have subnet with 3 hosts and maximum bandwidth for subnet is 6 Mbps, we can share it equally using PCQ. If one of host is downloading 1Mbps, all others share 5 Mbps and vice versa. There might be situation when all of the three hosts want to use maximum bandwidth (4 Mbps), then each of them receive 2 Mbps.

Look at previously example (see Figure 8.9) where we were limited download traffic equally among server and all workstation (see Figure 8.6) applying specific queue tree structure. We can perform it easy using PCQ rate limiting. In the same as configuring HTB also here is needed packet marking, it can be the similar as previously, but in this case we need to mark all packets that related to the network 10.1.1.0/24 with one common mark.

/ip firewall mangle> add chain=prerouting src-address=10.1.1.0/24
 action=mark-connection new-connection-mark=private_con

/ip firewall mangle> add chain=forward connection-mark=private_con 
action=mark-packet new-packet-mark='''private'''

Next think that we need to do is to define new PCQ type which will group all traffic by destination address. As I mentioned previous about PCQ then it allows to classify traffic flows (streams) by four parameters:

• Source address
• Destination address
• Source port
• Destination port

In this case we classify download traffic only by one parameter – destination address (dst-address), it will create a dynamic queue for each destination address.

 /queue type add name=pcq-download kind=pcq pcq-classifier=dst-address

Finally, make a queue tree for download traffic:

 /queue tree add name=Download parent=ether2 max-limit=2048
 /queue tree add parent=Download queue=pcq-download packet-mark='''private'''

It means, that PCQ can to perform equal bandwidth sharing on interface ether2, it will make one dynamic queue for each separate destination address. If we have network with four hosts (figure 8.6.) where maximum download is limited to 2Mbps and each of them want to use more than 512kbps, then all of them will receive data rate – 512Kbps.

In the same manner you can also equal distribute upload traffic from your LAN, but this case you create new queue tree and attach this queue to the public interface.