Queue with Masquerading and Internal Web-Proxy

Introduction

This page will talk about how to make a QUEUE TREE in RouterOS that is also running Web-Proxy and Masquerading. Several topics in the Forum say that it's impossible to do.

In version 2.9.x, we can not determine which traffic is HIT and which traffic is MISS from web-proxy. Several people want to make a configuration, to let cache data in proxy (HIT traffic) deliver in maximum possible speed. In other word, if we already have the requested data, those process will not queued.

In ver 3.0 we can do this, using TOS header modification in web-proxy feature. We can set any TOS value for the HIT traffic, and make it as a parameter in mangle.

Basic Setup

First, let's set the basic setting first. I'm using a machine with 2 network interface:

admin@instaler] > in pr
#    NAME       TYPE    RX-RATE    TX-RATE    MTU  
0  R public     ether   0          0          1500 
1  R lan        wlan    0          0          1500

And this is the IP Address for each interface:

[admin@instaler] > ip ad pr
Flags: X - disabled, I - invalid, D - dynamic 
#  ADDRESS           NETWORK      BROADCAST      INTERFACE
0  192.168.0.217/24  192.168.0.0  192.168.0.255  public   
1  172.21.1.1/24     172.21.1.0   172.21.1.255   lan

Don't forget to set the transparant web-proxy. We set cache-hit-dscp: 4.

[admin@instaler] > ip proxy pr
                    enabled: yes
                src-address: 0.0.0.0
                       port: 3128
               parent-proxy: 0.0.0.0
          parent-proxy-port: 0
                cache-drive: system
        cache-administrator: "webmaster"
             max-cache-size: none
              cache-on-disk: yes
 maximal-client-connections: 600
 maximal-server-connections: 600
             max-fresh-time: 3d
      serialize-connections: yes
             cache-hit-dscp: 4

Firewall NAT

Make 2 NAT rules, 1 for Masquerading, and the other for redirecting transparant proxy.

[admin@instaler] ip firewall nat> pr
Flags: X - disabled, I - invalid, D - dynamic 
0   chain=srcnat out-interface=public 
    src-address=172.21.1.0/24 action=masquerade 
1   chain=dstnat in-interface=lan src-address=172.21.1.0/24 
    protocol=tcp dst-port=80 action=redirect to-ports=3128

Mangle Setup

And now is the most important part in this case.

If we want to make HIT traffic from web proxy not queued, we have to make a mangle to handle this traffic. Put this rule on the beginning of the mangle, as it will check first.

[admin@instaler] > ip firewall mangle print
Flags: X - disabled, I - invalid, D - dynamic 
0   ;;; HIT TRAFFIC FROM PROXY
    chain=output out-interface=lan
    dscp=4 action=mark-packet 
    new-packet-mark=proxy-hit passthrough=no 

As we will make Queue for uplink and downlink traffic, we need 2 packet-mark. In this example, we use "test-up" for uplink traffic, and "test-down" for downlink traffic.

For uplink traffic, it's quite simple. We need only one rule, using SRC-ADDRESS and IN-INTERFACE parameters, and using PREROUTING chain. Rule number #1.

But for downlink, we have to make sevaral rules. As we use masquerading, we need Connection Mark, named as "test-conn". Rule no #2.

Then we have to make 2 more rules. First rule is for non-HTTP connection / direct connection. We use chain forward, as the data traveling through the router. Rule no #3.

The second rule is for data coming from web-proxy to the client (MISS traffic). We use OUTPUT chain, as the data coming from internal process in the router itself. Rule no #4.

For both rules (no #3 and #4) we named it "test-down".

Please be aware, we use passthrough only for connection mark (rule no #2).

[admin@instaler] > ip firewall mangle print
Flags: X - disabled, I - invalid, D - dynamic 
1   ;;; UP TRAFFIC
    chain=prerouting in-interface=lan 
    src-address=172.21.1.0/24 action=mark-packet 
    new-packet-mark=test-up passthrough=no 

2   ;;; CONN-MARK
    chain=forward src-address=172.21.1.0/24 
    action=mark-connection 
    new-connection-mark=test-conn passthrough=yes 

3   ;;; DOWN-DIRECT CONNECTION
    chain=forward in-interface=public 
    connection-mark=test-conn action=mark-packet 
    new-packet-mark=test-down passthrough=no 

4   ;;; DOWN-VIA PROXY
    chain=output out-interface=lan 
    dst-address=172.21.1.0/24 action=mark-packet 
    new-packet-mark=test-down passthrough=no

Queue Tree Setup

And now, the queue tree setting. We need one rule for downlink and one rule for uplink. Be careful when choosing the parent. for downlink traffic, we use parent "lan", the interface name for local network. And for uplink, we are using parent "global-in".

[admin@instaler] > queue tree pr
Flags: X - disabled, I - invalid 
0   name="downstream" parent=lan packet-mark=test-down 
    limit-at=32000 queue=default priority=8 
    max-limit=32000 burst-limit=0 
    burst-threshold=0 burst-time=0s 

1   name="upstream" parent=global-in 
    packet-mark=test-up limit-at=32000 
    queue=default priority=8 
    max-limit=32000 burst-limit=0 
    burst-threshold=0 burst-time=0s

You can use those mangle also with PCQ.