View source for PCQ and Hotspots, and exempting upstream resources from rate limit

Note: the below discusses Hotspots, but PPPoE with dynamic queues for rate limits functions the same way, and the same techniques work for exempting PPPoE users from resource access.

Hotspot rate limits fundamentally work based on simple queues. There is a simple queue based on the rate-limit configured on the Hotspot profile itself. This simple queue is dynamically created when the Hotspot is enabled, and is based on the interface the Hotspot runs on. Because of that it is applied to all traffic in and out of that router interface, and is consequently shared by all users behind the interface. This includes all unauthenticated users and users bypassed in IP bindings. 

When a user logs in - whether it is as a local user with a local user profile, or via RADIUS - the per user rate-limit is used to create a new dynamic simple queue. This time the user's IP address is used as the target address, and the queue is installed higher up in the list (the shared queues are always at the bottom of the dynamically created queue list). Therefore this new dynamic queue is applied to all traffic to and from the user on layer 3, based on the target IP address of the queue being specific to the IP address the Hotspot knows the user is using. On a sidenote, RADIUS users that are authenticated but for which the RADIUS server did not supply a rate limit attribute back to the router continue sharing the Hotspot profile pool since they do not get a specific queue created for them, and continue to fall through to the shared queue. 

All the nitty, gritty details about simple queues can be found in the [[Manual:Queue#Simple_Queues | manual]]. The most important detail is that really, simple queues are just a construct to make queues - well, simple. Depending on a simple queue's parameters creating it just really creates HTB objects attached to global-in, global-out, and global-total in the background. The interesting thing about this is that when you create queue trees attached to those parents they override the simple queues because the objects transparently installed via simple queues are evaluated later. Once a queue has processed a packet for a parent other queues attached to the same parent will not see the packet - that means that it is possible to get the packets that would usually be shaped by simple queues to be shaped by queue trees instead, allowing you to override the automatic Hotspot QoS. 

All the nitty, gritty details about queue trees can be found in the [[Manual:Queue#Queue_Tree | manual]]. The basics include that the queue trees MUST be attached to the global parents, interface HTB will not work as global parents are evaluated first - and since that is where the simple queues will grab the packets, you do not get a choice of queue parents. Because the global queues are global you will have to use packet marks (applied in IP firewall mangle) to let the router know which packets to treat in what manner.

Because the choice of queue parent has already been made I find it easiest to mark upload in prerouting and download in postrouting. This is sufficient for global queue parents, and ensures that all NAT has been done and undone in a manner that has the packets carry the real IP addresses for most scenarios, making marking based on IPs simple. Alternatively routing has also been decided on for download, making decisions based on in-interface and out-interface simple.

Details on packet flow can be found on what is easily the most useful [[Manual:Packet_Flow | manual]] page. 

PCQ queue types make it easy to do per user rate limiting by automatically creating subqueues based on some classifier. When creating automatic subqueues based on source address you can easily apply per user upload rate limits, destination addresses as classifiers work perfectly for download rate limits. All the nitty, gritty details about PCQ type queues can be found in the [[Manual:Queues_-_PCQ | manual]]. 

Simple queues are processed in order, so having lots and lots of simple queues because lots of users are logged into a Hotspot does not scale very well. PCQ type queues are evaluated rather quickly. It is relatively simple to use PCQ queue trees instead of simple queues.

This example assumes a simple router with two interfaces. One is called LAN and has a Hotspot configured on it, and the other is called WAN and connects the router to the Internet. The goal is to rate limit each user to 256 Kbps download and 128 Kbps upload using PCQ queue trees.

First, mark download and upload packets. This is easily done by interface: packets received by the LAN interface go from a client to the Internet and are upload, packets sent via the LAN interface are coming from the Internet to the client and are download. Depending on the complexity of the rule it can be beneficial to use connection marks, but for simple rules like below that is actually more expensive for the router to evaluate.
<pre>
/ip firewall mangle
add chain=prerouting in-interface=LAN \
  action=mark-packet new-packet-mark=upload
add chain=postrouting out-interface=LAN \
  action=mark-packet new-packet-mark=download
</pre>
Next, create the PCQ type queues. This is where the per user rate limits are set - edit accordingly. Do read up on the other parameters in the manual so you know what to tweak when there are many users.
<pre>
/queue type
add type=pcq pcq-classifier=src-address \
  pcq-rate=128000 name=pcq-up
add type=pcq pcq-classifier=dst-address \
  pcq-rate=256000 name=pcq-down
</pre>
The last thing that remains are the actual queues. You will usually want to apply some network rate limit here to match available WAN bandwidth, but could just go for wire speed. Below we will make sure all users combined cannot exceed 10 Mbps either way.
<pre>
/queue tree
add name=hotspot-up parent=global-in \
  packet-mark=upload queue=pcq-up \
  max-limit=10M
add name=hotspot-down parent=global-out \
  packet-mark=download queue=pcq-down \
  max-limit=10M
</pre>

One common question is how to except some traffic from being rate limited by a Hotspot. Usually there is some server on another network attached to the Hotspot router that provides some service, and users should not be rate limited when accessing content on that server. A simple solution is to use the above to make queue trees that mark that traffic and the use wire speed queues. You can stop there and let the simple queues do their magic for all other, non-exempted traffic - which we will do below - or make more mangle rules and queues that catch fallthrough traffic. 

Assuming the network users can access without a rate limit is 10.0.0.0/24:
<pre>
/ip firewall mangle
add chain=prerouting in-interface=LAN \
  dst-address=10.0.0.0/24 action=mark-packet \
  new-packet-mark=exempt-up
add chain=postrouting out-interface=LAN \
  src-address=10.0.0.0/24 action=mark-packet \
  new-packet-mark=exempt-down
/queue type
add name=exempt type=sfq
/queue tree
add name=hotspot-exempt-up parent=global-in \
  packet-mark=exempt-up queue=exempt max-limit=1G
add name=hotspot-exempt-down parent=global-out \
  packet-mark=exempt-down queue=exempt max-limit=1G
</pre>