NetworkPro on Quality of Service

From MikroTik Wiki
Revision as of 12:41, 30 May 2009 by NetworkPro (talk | contribs)
Jump to: navigation, search

Here I will share some useful info on Quality of Service and packet queue management. Currently this WiKi page is an alpha version, not linked to. It needs fixing and evaluation before presented to the public.

Let's begin with several quotes:

http://en.wikipedia.org/wiki/Bandwidth_management TCP rate control - artificially adjusting TCP window size as well as controlling the rate of ACKs being returned to the sender (RouterOS can adjust TCP window by dropping packets out of bandwidth/queue size restrictions or rate of ACK packets if you mark them and set up a Queue to limit their rate)

http://en.wikipedia.org/wiki/ACK_(TCP)

Eugene http://forum.mikrotik.com/viewtopic.php?t=5947

A TCP window is the amount of data a sender can send on a particular TCP connection before it gets an acknowledgment (ACK packet) back from the receiver that it has gotten some of it.

For example if a pair of hosts are talking over a TCP connection that has a TCP window size of 64 KB (kilobytes), the sender can only send 64 KB of data and then it must stop and wait for an acknowledgment from the receiver that some or all of the data has been received. If the receiver acknowledges that all the data has been received then the sender is free to send another 64 KB. If the sender gets back an acknowledgment from the receiver that it received the first 32 KB (which could happen if the second 32 KB was still in transit or it could happen if the second 32 KB got lost or dropped, shaped), then the sender could only send another 32 KB since it can't have more than 64 KB of unacknowledged data.

The primary reason for the window is congestion control. The whole network connection, which consists of the hosts at both ends, the routers in between and the actual physical connections themselves (be they fiber, copper, satellite or whatever) will have a bottleneck somewhere that can only handle data so fast. Unless the bottleneck is the sending speed of the transmitting host, then if the transmitting occurs too fast the bottleneck will be surpassed resulting in lost data. The TCP window throttles the transmission speed down to a level where congestion and data loss do not occur.

Janis Inbound traffic for router - traffic that hits routers interfaces, no matter from what side - internet or local - it will be received by interface no matter what, even malformed packets and you cannot do anything with these. Outbound traffic for router - traffic that goes out of routers interfaces, no matter of direction, to your network or out of it. This is where you can set up queues and prioritize, and limit.

Macgaiver Priority is HTB feature. HTB allows to order and/or shape outgoing traffic (traffic that is leaving router via any interface).

Example1: Client sends out 10Mbps UDP traffic - this traffic will get to the routers local interface, but in one of the HTBs (global-in, global-total, global-out or outgoing interface) it will be shaped to (let’s say) 1Mbps. So only 1Mbps will leave the router. But in the next second the client can send 10Mbps once again and we will shape them again.

Example2: Client sends out 10Mbps TCP traffic - this traffic will get to the routers local interface, but in one of the HTBs (global-in, global-total, global-out or outgoing interface) it will be shaped to (let’s say) 1Mbps. So only 1Mbps will leave the router. Source gets ACK replies only for 1Mbps of 10Mbps, so source, in the next second, will send little more than 1Mbps of TCP traffic.

To make it clear - there are 4 traffics that we can talk about:

1) client upload that router receives on the local interface 2) client upload that router sends out to the Internet 3) clients download that router receives on the public interface 4) clients download that router receives sends out to the customer

1) and 3) - is Inbound traffic 2) and 4) - is Outbound traffic QoS can change (shape/prioritize) 1) into 2) and 3) into 4)

Connections can't be upload or download, packets are. For example TCP connections have traffic in both directions.

Simple packet flow diagram:

Normis QoS includes several facilities, in the following order:

1. Mangle chain prerouting 2. HTB global-in 3. Mangle chain forward 4. Mangle chain postrouting 5. HTB global-out 6. HTB out interface

So, in one router, you can shape twice if you use:

a) #1 and #2 for first marking and shaping, and #3+#5 for second b) #1 and #2 for first marking and shaping, and #3+#6 for second c) #1 and #2 for first marking and shaping, and #4+#5 for second d) #1 and #2 for first marking and shaping, and #4+#6 for second

Macgaiver 1) As you can see all traffic is going through prerouting and postrouting - so there is no need to mark traffic in 2 different places. 2) A packet can have only one packet-mark at the same time (in one chain) (refer to packet flow diagram)

http://wiki.mikrotik.com/wiki/Queue#Queue_Tree Queue tree creates an unidirectional queue in one of the HTBs. It is also The way to add a queue on a separate interface. This way (setting up queue tree on the interface HTB: parent:ether1/pppoe1…) it is possible to ease mangle configuration - you don't need separate marks for download and upload - only upload will get to Private interface and only download will get to Public interface. (If you are marking connections + their packets) Also it is possible to have double queuing (example: prioritization of traffic in global-in or global-out, limitation per client on the outgoing interface). If you have simple queues and queue tree in the same HTB - simple queues will get traffic first. Queue tree is not ordered - all traffic passes it together (unlike simple queue where the order matters).

priority (1..8) : Prioritize one child queue over other child queue. Does not work on parent queues (if queue has at least one child). One is the highest, eight is the lowest priority. Child queue with higher priority will have chance to reach its limit-at before child with lower priority and after that child queue with higher priority will have chance to reach its max-limit before child with lower priority. Priority has nothing to do with bursts.

Large queue sizes can increase latency, but utilize channel better. ?Parent queue have to have match all the traffic that will go for child queues.?

as for TCP, you may increase queue size if you have drops

in real-time UDP traffic queues should be smaller to drop packets out of bandwidth limit, because they almost useless when arrived with delay

so, see what exactly you need

Chupaka http://forum.mikrotik.com/viewtopic.php?f=2&t=12870

Because PCQ does not affect child queues, it really does not matter, what type you set for parent: parent is just to see total traffic. (type applies only to Child queues)

Janis In my presentation I told that creating priorities separately for each client is suicide - there is no hardware that can handle small queue tree for every user (if you have 1000 of them). So in my presentation I discuss next best thing, which is as close as possible to desired behavior.

The main Idea of the setup is to have two separate QoS steps:

1) In the first step we prioritize traffic, we are making sure that traffic with higher priority has more chance to get to the customers than traffic with lower priority.

Example: We have total of 100Mbps available, but clients at this particular moment would like to receive 10Mbps of Priority=1 traffic, 20Mbps of Priority=4 and 150Mbps of Priority=8. Of course after our prioritization and limitation 80Mbps of priority=8 will be dropped. And only 100Mbps total will get to the next step.

2) Next step is per-user limitation, we already have only higher priority traffic, but now we must make sure that some user will not overuse it, so we have PCQ with limits.

This way we get virtually the same behavior as "per user prioritization".

Macgaiver So the plan for you might be to mark by traffic type in prerouting and limit by traffic type in global-in. Then remark traffic by IP addresses in forward and limit them on the outgoing interface.

1) you need to mark all traffic at the same place (prerouting) (that would be managed by one particular Queue) 2) you must mark upload and download for every type of traffic separately (if you use global-total/in/out or Queue Simple or if you use Queue Tree and you do not mark connections first or if you do not set up the Queue in the interface HTB) 4) you must have a parent queue, that has max-limit and (let’s say) parent=global-in - all other queues parent=<parent> (for proper Queue Tree or in case simple PCQ – just one Queue can manage all) 5) you need 2 sets of those queues - one for upload, one for download

Priority doesn't work without limitation.

1) HTTP browsing connection usually is not more than 0,5MB (or 4Mb) 2) Mangle facility can mark only part of connection - you must use mark-packet directly without mark-connection (and mark separately upload and download traffic) 3) create two marks "first_bytes" and "last_bytes" 4) create queue structure in queue tree (one for download on local interface, one for upload on public interface) queue structure must have 3 queues: a) parent with max-limit b) queue for "first_bytes" with priority=1 and limit-at and max-limit specified c) queue for "last_bytes" wint priority=8 and limit-at and max-limit specified

That is it - I use it every day to prioritize normal HTTP over other traffic on port 80 (like downloads and any other large transfer that would use port 80)

Janis Each simple queue creates 3 separate queues: One in global-in (“direct” part) One in Global-out (“reverse” part) One in Global-total (“total” part) Simple queues are ordered - similar to firewall rules further down = longer packet processing further down = smaller chance to get traffic (so it’s necessary to reduce number of queues)

Queuing Placement Limitation for in mangle chain “forward” marked traffic can be placed in the “global-out” or interface queue (see packet flow diagram)

If queues will be placed in the interface queues queues on the public interface will capture only client upload queues on the local interface will capture only client's download

If queues will be placed in global-out download and upload will be limited together (separate marks needed)

dot-bot In the case of Simple Queues, the order is for 'catching traffic' (mangle) and the priority is for packet queue management in the HTB.


Create packet marks in the mangle chain “Prerouting” for traffic prioritization in the global-in Queue


A theoretic mangle and queue setup...:

/ ip firewall mangle 
add chain=prerouting action=mark-packet new-packet-mark=icmp_in passthrough=no \
   in-interface=Public protocol=icmp comment="icmp" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=icmp_out \
   passthrough=no out-interface=Public protocol=icmp comment="" disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=p2p_in passthrough=no \
   p2p=all-p2p in-interface=Public comment="p2p" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=p2p_out \
   passthrough=no p2p=all-p2p out-interface=Public comment="" disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=pop3_in passthrough=no \
   in-interface=Public src-port=110 protocol=tcp comment="pop3" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=pop3_out \
   passthrough=no out-interface=Public dst-port=110 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=smtp_in passthrough=no \
   in-interface=Public src-port=25 protocol=tcp comment="smtp" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=smtp_out \
   passthrough=no out-interface=Public dst-port=25 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=imap_in passthrough=no \
   in-interface=Public src-port=143 protocol=tcp comment="imap" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=imap_out \
   passthrough=no out-interface=Public dst-port=143 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=ssh_in passthrough=no \
   in-interface=Public dst-port=22 protocol=tcp comment="ssh" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=ssh_out \
   passthrough=no out-interface=Public src-port=22 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=winbox_in \
   passthrough=no in-interface=Public dst-port=8291 protocol=tcp \
   comment="winbox" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=winbox_out \
   passthrough=no out-interface=Public src-port=8291 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=dns_in passthrough=no \
   in-interface=Public src-port=53 protocol=udp comment="dns" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=dns_out \
   passthrough=no out-interface=Public dst-port=53 protocol=udp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=www_in passthrough=no \
   in-interface=Public src-port=80 protocol=tcp comment="www" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=www_out \
   passthrough=no out-interface=Public dst-port=80 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=ssl_in passthrough=no \
   in-interface=Public src-port=443 protocol=tcp comment="ssl" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=ssl_out \
   passthrough=no out-interface=Public dst-port=443 protocol=tcp comment="" \
   disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=udp_in passthrough=no \
   in-interface=Public protocol=udp comment="udp" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=udp_out \
   passthrough=no out-interface=Public protocol=udp comment="" disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=tcp_in passthrough=no \
   in-interface=Public protocol=tcp comment="tcp" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=tcp_out \
   passthrough=no out-interface=Public protocol=tcp comment="" disabled=no 
add chain=prerouting action=mark-packet new-packet-mark=other_in \
   passthrough=no in-interface=Public comment="other" disabled=no 
add chain=postrouting action=mark-packet new-packet-mark=other_out \
   passthrough=no out-interface=Public comment="" disabled=no


?step 1 satisfy limit-ats by priority step 2 satisfy limit-ats by hierarchy step 3 satisfy max-limits by priority step 4 satisfy max-limits by hierarchy

Sample Queue Tree for 12Mbps/1Mbps

Lets suppose we already mark packets in the proper mangle chains. In these samples I tried to manage all packets that could ever travel out a certain interface. They are not tested and need adjustment.

add comment="DNStcp_d" chain=forward action=mark-packet new-packet-mark=DNSToCl passthrough=no src-port=53 protocol=tcp dst-address-list=local-addr
add comment="ACK_d" chain=forward action=mark-packet new-packet-mark=ACKToCl passthrough=no protocol=tcp tcp-flags=ack packet-size=0-80 dst-address-list=local-addr
add comment="SYN_d" chain=forward action=mark-packet new-packet-mark=SYNToCl passthrough=no protocol=tcp tcp-flags=syn dst-address-list=local-addr
add comment="RST_d" chain=forward action=mark-packet new-packet-mark=RSTToCl passthrough=no protocol=tcp tcp-flags=rst dst-address-list=local-addr
add comment="WinBox from Internet" chain=forward action=mark=packet new-packet-mark=WinBoxToCl passthrough=no src-port=8291 protocol=tcp dst-address-list=local-addr
add comment="Games" chain=forward action=mark-packet new-packet-mark=GamesTCPToCl protocol=tcp passthrough=no src-address-list=GameServers dst-address-list=local-addr

add comment="High Priority Data" chain=forward action=mark-packet new-packet-mark=HighPriorityToCl passthrough=no src-port=443 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=HighPriorityToCl passthrough=no src-port=22 protocol=tcp packet-size=0-1400 dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=HighPriorityToCl passthrough=no src-port=80 protocol=tcp connection-bytes=0-600000 dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=HighPriorityToCl passthrough=no src-port=23 protocol=tcp dst-address-list=local-addr

add comment="Download Data" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=80 connection-bytes=600000-0 protocol=tcp dst-address-list=local-addr  
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=110 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=995 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=143 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=993 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=25 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=20 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=21 protocol=tcp dst-address-list=local-addr
add comment="" chain=forward action=mark-packet new-packet-mark=DownloadToCl passthrough=no src-port=22 protocol=tcp packet-size=1400-1500 dst-address-list=local-addr

add comment="Unrecognized Data" chain=forward action=mark-packet new-packet-mark=UnrecognizedTCPToCl passthrough=no protocol=tcp dst-address-list=local-addr

add comment="All else – ICMP and UDP" chain=forward action=mark-packet new-packet-mark=ICMPUDP passthrough=no dst-address-list=local-addr




add comment="ACK_u" chain=postrouting action=mark-packet new-packet-mark=ACKToInet passthrough=no protocol=tcp tcp-flags=ack packet-size=0-80 src-address-list=local-addr
add comment="SYN_u" chain=postrouting action=mark-packet new-packet-mark=SYNToInet passthrough=no protocol=tcp tcp-flags=syn src-address-list=local-addr
add comment="RST_u" chain=postrouting action=mark-packet new-packet-mark=RSTToInet passthrough=no protocol=tcp tcp-flags=rst src-address-list=local-addr
add comment="WinBox Server" chain=postrouting action=mark=packet new-packet-mark=WinBoxToInet passthrough=no src-port=8291 protocol=tcp src-address-list=local-addr
add comment="Games" chain=postrouting action=mark-packet new-packet-mark=GamesTCPToInet protocol=tcp passthrough=no src-address-list=GameServers src-address-list=local-addr

add comment="High Priority Data" chain=postrouting action=mark-packet new-packet-mark=HighPriorityToInet passthrough=no dst-port=443 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=HighPriorityToInet passthrough=no dst-port=22 protocol=tcp packet-size=0-1400 src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=HighPriorityToInet passthrough=no dst-port=80 protocol=tcp connection-bytes=0-600000 src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=HighPriorityToInet passthrough=no dst-port=23 protocol=tcp src-address-list=local-addr

add comment="Download Data" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=80 connection-bytes=600000-0 protocol=tcp src-address-list=local-addr  
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=110 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=995 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=143 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=993 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=25 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=20 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=21 protocol=tcp src-address-list=local-addr
add comment="" chain=postrouting action=mark-packet new-packet-mark=DownloadToInet passthrough=no dst-port=22 protocol=tcp packet-size=1400-1500 src-address-list=local-addr

add comment="Unrecognized Data" chain=postrouting action=mark-packet new-packet-mark=UnrecognizedTCPToInet passthrough=no protocol=tcp src-address-list=local-addr

add comment="All else – ICMP and UDP" chain=postrouting action=mark-packet new-packet-mark=ICMPUDP passthrough=no src-address-list=local-addr





/ queue tree
add name="HTB_Out_Local" parent=global-out limit-at=40000000 max-limit=40000000
add name="WinBox_OL" parent=HTB_Out_Local packet-mark=WinBox_OL priority=1 queue=pcqol limit-at=1000000 max-limit=1000000
add name="icmpdnsol" parent=HTB_Out_Local packet-mark=icmpol,dnsol priority=2 queue=pcqol limit-at=1000000 max-limit=30000000
add name="cl2clgntcpgol" parent=HTB_Out_Local packet-mark=cl2clg,ntcpgol priority=2 queue=pcqol limit-at=10000000 max-limit=30000000
add name="ntcpol" parent=HTB_Out_Local packet-mark=ntcpol priority=7 queue=pcqol limit-at=25000000 max-limit=25000000
add name="ToInternetTCP" parent=HTB_Out_Local limit-at=10000000 max-limit=10000000
add name="pr2ol" parent=ToInternetTCP packet-mark=inettcpg limit-at=531337 queue=pcqol priority=2 max-limit=8000000
add name="pr3ol" parent=ToInternetTCP packet-mark=hipr limit-at=8000000 queue=pcqol priority=3 max-limit=10000000
add name="pr6ol" parent=ToInternetTCP packet-mark=dl limit-at=1000000 queue=pcqol priority=6 max-limit=10000000
add name="pr7ol" parent=ToInternetTCP packet-mark=unrectcp limit-at=234331 queue=pcqol priority=7 max-limit=10000000
add name="pr8ol" parent=ToInternetTCP packet-mark=recp2p queue=pcqol priority=8 limit-at=234332 max-limit=10000000
add name="cl2cl" parent=HTB_Out_Local priority=8 queue=pcqol packet-mark=cl2cl limit-at=3000000 max-limit=40000000

/ queue tree
add name="ToInternetPostroutingQueue_QoS_PCQ" parent=global-out limit-at=999999 max-limit=999999
add name="wboxop" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=wboxop priority=1 queue=pcqop limit-at=131337 max-limit=691337
add name="icmpdnsop" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=icmpop,dnsop priority=2 queue=pcqop limit-at=131337 max-limit=691337
add name="ntcpgop" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=ntcpgop priority=2 queue=pcqop limit-at=691337 max-limit=696969
add name="ntcpop" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=ntcpop priority=7 queue=pcqop limit-at=131337 max-limit=131337
add name="tcpop" parent=ToInternetPostroutingQueue_QoS_PCQ limit-at=699999 max-limit=999999
add name="pr2op" parent=tcpop packet-mark=tcpgop limit-at=131337 queue=pcqop priority=2 max-limit=699999
add name="pr3op" parent=tcpop packet-mark=hiprop limit-at=631337 queue=pcqop priority=3 max-limit=999999
add name="pr6ol" parent=tcpop packet-mark=dlop limit-at=131337 queue=pcqop priority=6 max-limit=999999
add name="pr7ol" parent=tcpop packet-mark=unrectcpop limit-at=!#$%^& queue=pcqop priority=7 max-limit=999999
add name="pr8ol" parent=ToInternetTCP packet-mark=recp2p queue=pcqol priority=8 limit-at=234332 max-limit=999999

Sample Queue Tree for 20Mbps/20Mbps

/ queue tree
add name="ToClientsPostroutingQueue_QoS_PCQ" parent=global-out limit-at=35000000 max-limit=35000000
add name="WinBoxToCl" parent=ToClientsPostroutingQueue_QoS_PCQ packet-mark=WinBoxToCl priority=1 queue=dst-addr-PCQ limit-at=1000000 max-limit=2000000
add name="VeryImportantPacketsToCl" parent=ToClientsPostroutingQueue_QoS_PCQ packet-mark=ICMPToCl,DNSToCl,ACKToCl,SYNToCl,RSTToCl priority=2 queue=dst-addr-PCQ limit-at=2000000 max-limit=8000000
add name="CustomNotTCPToCl" parent=ToClientsPostroutingQueue_QoS_PCQ packet-mark=NotTCPGamesToCl priority=2 queue=dst-addr-PCQ limit-at=2000000 max-limit=30000000
add name="UnrecognizedNotTCPToCl" parent=ToClientsPostroutingQueue_QoS_PCQ packet-mark=UnrecognizedNotTCPToCl priority=6 queue=dst-addr-PCQ limit-at=2500000 max-limit=25000000
add name="ToClientTCP" parent=ToClientsPostroutingQueue_QoS_PCQ limit-at=20000000 max-limit=20000000
add name="2" parent=ToClientTCP packet-mark=GamesTCPToCl limit-at=1000000 queue=dst-addr-PCQ priority=2 max-limit=20000000
add name="3" parent=ToClientTCP packet-mark=HighPriorityToCl limit-at=18000000 queue=dst-addr-PCQ priority=3 max-limit=20000000
add name="5" parent=ToClientTCP packet-mark=DownloadToCl limit-at=5000000 queue=dst-addr-PCQ priority=5 max-limit=20000000
add name="7" parent=ToClientTCP packet-mark=UnrecognizedTCPToCl limit-at=1000000 queue=dst-addr-PCQ priority=7 max-limit=20000000
add name="8" parent=ToClientTCP packet-mark=RecognizedP2PToCl queue=dst-addr-PCQ priority=8 limit-at=512000 max-limit=20000000
add name="Client2Client" parent=ToClientsPostroutingQueue_QoS_PCQ priority=8 queue=dst-addr-PCQ packet-mark=Client2Client limit-at=2000000 max-limit=35000000

/ queue tree
add name="ToInternetPostroutingQueue_QoS_PCQ" parent=global-out limit-at=25000000 max-limit=25000000
add name="WinBoxToInet" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=WinBoxToInet priority=1 queue=src-addr-PCQ limit-at=1000000 max-limit=2000000
add name="VeryImportantPacketsToInet" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark= ICMPToInet,DNSToInet,ACKToInet,SYNToInet,RSTToInet priority=2 queue=src-addr-PCQ limit-at=2000000 max-limit=8000000
add name="CustomNotTCPToInet" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=NotTCPGamesToInet priority=2 queue=src-addr-PCQ limit-at=2500000 max-limit=20000000
add name="UnrecognizedNotTCPToInet" parent=ToInternetPostroutingQueue_QoS_PCQ packet-mark=UnrecognizedNotTCPToInet priority=6 queue=src-addr-pcq limit-at=3000000 max-limit=22000000
add name="ToInternetTCP" parent=ToInternetPostroutingQueue_QoS_PCQ limit-at=20000000 max-limit=20000000
add name="2" parent= ToInternetTCP packet-mark=GamesTCPToInet limit-at=1000000 queue=src-addr-PCQ priority=2 max-limit=20000000
add name="3" parent= ToInternetTCP packet-mark=HighPriorityToInet limit-at=18000000 queue=src-addr-PCQ priority=3 max-limit=20000000
add name="5" parent= ToInternetTCP packet-mark=DownloadToInet limit-at=18000000 queue=src-addr-PCQ priority=5 max-limit=20000000
add name="7" parent= ToInternetTCP packet-mark=UnrecognizedTCPToInet limit-at=1000000 queue=src-addr-PCQ priority=7 max-limit=20000000
add name="8" parent= ToInternetTCP packet-mark=RecognizedP2PToInet queue=src-addr-PCQ priority=8 limit-at=512000 max-limit=20000000

Upload QoS for ADSL

/ip firewall mangle
add action=mark-packet chain=postrouting comment=QoS dst-port=80,443 new-packet-mark=QoS_1_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=syn
add action=mark-packet chain=postrouting dst-port=80,443 new-packet-mark=QoS_1_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting dst-port=53,123 new-packet-mark=QoS_1_Up out-interface=ADSL1 passthrough=no protocol=udp
add action=mark-packet chain=postrouting connection-bytes=0-1000000 dst-port=80,443 new-packet-mark=QoS_2_Up out-interface=ADSL1 passthrough=no protocol=tcp
add action=mark-packet chain=postrouting dst-port=110,995,143,993,25,20,21 new-packet-mark=QoS_2_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=syn
add action=mark-packet chain=postrouting dst-port=110,995,143,993,25,20,21 new-packet-mark=QoS_2_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting new-packet-mark=QoS_3_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=syn
add action=mark-packet chain=postrouting new-packet-mark=QoS_3_Up out-interface=ADSL1 packet-size=0-666 passthrough=no protocol=tcp tcp-flags=ack
add action=mark-packet chain=postrouting dst-port=110,995,143,993,25,20,21 new-packet-mark=QoS_4_Up out-interface=ADSL1 passthrough=no protocol=tcp
add action=mark-packet chain=postrouting connection-bytes=1000000-0 dst-port=80,443 new-packet-mark=QoS_4_Up out-interface=ADSL1 passthrough=no protocol=tcp
add action=mark-packet chain=postrouting new-packet-mark=QoS_8_Up out-interface=ADSL1 p2p=all-p2p passthrough=no
add action=mark-packet chain=postrouting new-packet-mark=QoS_7_Up out-interface=ADSL1 passthrough=no
add action=mark-connection chain=prerouting dst-port=8291,58291,58292,58293,58294 new-connection-mark=WinBox passthrough=yes protocol=tcp
add action=mark-packet chain=postrouting connection-mark=WinBox new-packet-mark=QoS_2_Up out-interface=ADSL1 passthrough=no


/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=666k name=QoS_ADSL1_Up packet-mark="" parent=ADSL1 priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_1 packet-mark=QoS_1_Up parent=QoS_ADSL1_Up priority=1 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_2 packet-mark=QoS_2_Up parent=QoS_ADSL1_Up priority=2 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_3 packet-mark=QoS_3_Up parent=QoS_ADSL1_Up priority=3 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_7 packet-mark=QoS_7_Up parent=QoS_ADSL1_Up priority=7 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_8 packet-mark=QoS_8_Up parent=QoS_ADSL1_Up priority=8 queue=default
add burst-limit=0 burst-threshold=0 burst-time=0s limit-at=0 max-limit=0 name=QoS_4 packet-mark=QoS_4_Up parent=QoS_ADSL1_Up priority=4 queue=default