Testwiki/Proxy (Cache) services
Proxy (Cache) services
Web proxy (/ip proxy)
MikroTik RouterOS performs proxying of HTTP and HTTP-proxy (for FTP, HTTP and HTTPS protocols) requests. Proxy server performs Internet object cache function by storing requested Internet objects, i.e., data available via HTTP and FTP protocols on a system positioned closer to the recipient in the form of speeding up customer browsing by delivering them requested file copies from proxy cache at local network speed. MikroTik RouterOS implements the following proxy server features:
- Regular HTTP proxy – customer (itself) specify what is proxy server for him
- Transparent proxy – customer does not know about the proxy being enabled and there isn’t need any additional configuration for web browser of client.
- Access list by source, destination, URL and requested method (HTTP firewall)
- Cache access list to specify which objects to cache, and which not.
- Direct Access List – to specify which resources should be accessed directly, and which - through another proxy server
- Logging facility – allows to get and to store information about proxy operation
- Parent proxy support – allows to specify other proxy server, ('if they don’t have the requested object ask their parents, or to the original server.)
A proxy server usually is placed at various points between users and the destination server (also known as origin server) on the Internet. (see Figure 10.1).
A Web proxy (cache) watches requests coming from client, saving copies of the responses for itself. Then, if there is another request for the same URL, it can use the response that it has, instead of asking the origin server for it again. If proxy has not requested file, it downloads that from the original server.
There can be many potential purpose of proxy server:
- To decrease access speed to resources (it takes less time for the client to get the object).
- Works as HTTP firewall (deny access undesired web pages),
Allows to filter web content (by specific parameters, like source address, destination address and port, URL, HTTP request method) scan outbound content, e.g., for data leak protection.
Note that it may be useful to have Web proxy running even with no cache when you want to use it only as something like HTTP and FTP firewall (for example, denying access undesired web pages or deny specific type of files e.g. .mp3 files) or to redirect requests to external proxy (possibly, to a proxy with caching functions) transparently.
Proxy configuration example
In MikroTik RouterOS proxy configuration are organized under /ip proxy menu. See below how to enable the proxy on port 8080 and set up 195.10.10.1 as proxy source address:
[admin@MikroTik] ip proxy> set enabled=yes port=8080 src-address=195.10.10.1
[admin@MikroTik] ip proxy> print
enabled: yes
src-address: 195.10.10.1
port: 8080
parent-proxy: 0.0.0.0:0
cache-drive: system
cache-administrator: "admin@mikrotik.com"
max-disk-cache-size: none
max-ram-cache-size: 100000KiB
cache-only-on-disk: yes
maximal-client-connections: 1000
maximal-server-connections: 1000
max-object-size: 2000KiB
max-fresh-time: 3d
When setting up regular proxy service, make sure it serves only your clients and prevent unauthorized access to it by creating firewall that allow only your clients to use proxy, otherwise it may be used as an open proxy.
Remember that regular proxy require also to configure client’s web browser settings.
- For example:
|
Explorer 8.x |
Firefox 3.x |
Opera 10.x |
|
Select Tools>Internet options. Click the Connections tab. Select the necessary connection and choose Settings button. Configure proxy address and port. |
Select Tools>Options. Click the Advanced tab. Open the Network tab. Click the Connection/Settings Select Manual proxy configuration' |
Select Tool>Preferences. Open the Advanced tab/Network. Click the Proxy servers. Enter proxy address and port. |
Transparent proxy configuration example
RouterOS can also act as a Transparent Caching server, with no configuration required in the customer’s web browser. Transparent proxy does not modify requested URL or response. RouterOS will take all HTTP requests and redirect them to the local proxy service. This process will be entirely transparent to the user (users may not know anything about proxy server that is located between them and original server), and the only difference to them will be the increased browsing speed.
To enable the transparent mode, firewall rule in destination NAT has to be added, specifying which connections (to which ports) should be transparently redirected to the proxy. Check proxy settings above and redirect us users (192.168.1.0/24) to proxy server.
[admin@MikroTik] ip firewall nat> add chain=dstnat protocol=tcp src-address=192.168.1.0/24 \ dst-port=80 action=redirect to-ports=8080 [admin@MikroTik] ip firewall nat> print Flags: X - disabled, I - invalid, D - dynamic 0 chain=dstnat protocol=tcp dst-port=80 action=redirect to-ports=8000 [admin@MikroTik] ip firewall nat>
The web proxy can be used as transparent and normal web proxy at the same time. In transparent mode it is possible to use it as standard web proxy, too. However, in this case, proxy users may have trouble to reach web pages which are accessed transparently.
Proxy based firewall – Access List (/ip proxy access)
Access list is implemented in the same way as MikroTik firewall rules. Rules are processed from the top to the bottom. First matching rule specifies decision of what to do with this connection. Connections can be matched by its source address, destination address, destination port, substring of requested URL (Uniform Resource Locator) or request method. If none of these parameters is specified, every connection will match this rule.
If connection is matched by a rule, action property of this rule specifies whether connection will be allowed or not (deny). If connection does not match any rule, it will be allowed.
In this example assume that we have configured transparent proxy server as given example above. Argument description:
- src-address - source address of the request
- dst-address - destination address of the request
- dst-port - destination port of the request
- path - the URL of the request. Can be regular expression.
- action - (allow / deny) action to take.
- method - (any | connect | delete | get | head | options | post | put | trace) - HTTP method used in the request
Block particular Websites.
/ip proxy access add dst-host=www.facebook.com action=deny
It will block website http://www.facebook.com, we can always block the same for different networks by giving src-address.
/ip proxy access add src-address=192.168.1.0/24 dst-host=www.facebook.com action=deny
Users from network 192.168.1.0/24 not be able to access to website www.facebook.com.
You can block also websites that contain specific words in URL:
/ip proxy access add dst-host=:mail action=deny
This statement will block all websites which contain word “mail” in URL. Like www.mail.com, www.hotmail.com, mail.yahoo.com etc.
We can also stop downloading specific types of files like .flv, .avi, .mp4, .mp3, .exe, .dat, …etc.
/ip proxy access add path=*.flv action=deny add path=*.avi action=deny add path=*.mp4 action=deny add path=*.mp3 action=deny add path=*.zip action=deny add path=*.rar action=deny.
Here are available also different wildcard characters, to creating specific conditions and to match it by proxy access list.
Wildcard properties (dst-host and dst-path) match a complete string (i.e., they will not match "example.com" if they are set to "example"). Available wildcards are '*' (match any number of any characters) and '?' (match any one character).
Regular expressions are also accepted here, but if the property should be treated as a regular expression, it should start with a colon (':').
To show that no symbols are allowed before the given pattern, we use ^ symbol at the beginning of the pattern.
To specify that no symbols are allowed after the given pattern, we use $ symbol at the end of the pattern.
Direct access (/ip proxy direct)
If parent-proxy property is specified, it is possible to tell proxy server whether to try to pass the request to the parent proxy or to resolve it connecting to the requested server directly. Direct Access List is managed just like Proxy Access List and determines access to proxy server.
Unlike the access list, the direct proxy access list has default action equal to deny. It takes place when no rules are specified or a particular request did not match any rule.
Cache management (/ip proxy cache)
In the same way as you implemented proxy access-list you can define also access-list for proxy that filter proxy content. Cache access list specifies, which requests (domains, servers, pages) have to be cached locally by web proxy, and which not. Default action (if no matching rule is found) is to store object into cache memory.
More information and /ip proxy command references are found here:
http://wiki.mikrotik.com/wiki/Manual:IP/Proxy
DNS proxy (/ip dns)
DNS (Domain Name System) cache facility on MikroTik router to provide domain name resolution for router itself as well as for the connected clients. This facility allows router works as default DNS resolver for clients on the LAN.
Router (resolver) receives local client DNS requests and forwards to public DNS only those requests that are not yet in the local DNS cache.
Domain name are useful as argument to a local resolver, which retrieves information associated with the domain name (basically IP address). A user may ask for the host IP address associated with particular domain name (For example: What is IP address of domain name “www.mikrotik.com”?' -> Resolver return answer that IP address for www.mikrotik.com is 159.148.147.196).
DNS query and response from “Wireshark” traffic analyzer:
Domain Name System (query) No. Time Source Destination Protocol Info 614 80.581710 172.16.16.243 172.16.16.1 DNS(53) Standard query A www.mikrotik.com Domain Name System (response) No. Time Source Destination Protocol Info 615 80.592878 172.16.16.1 172.16.16.243 DNS(53) Standard query response A 159.148.147.196
A special type of request is sent to the resolver on a port 53. If you want to receive IP addresses of all public domain names your local resolver must know at least one public name server. When the resolver processes a user query that is not in local cache it ask to public name server for required information. In this way resolver learns content of name server.
DNS cache is used to minimize DNS requests to an external DNS server as well as to minimize DNS resolution time.
DNS proxy (cache) setup
A MikroTik router with DNS feature enabled can be set as a DNS server for any DNS-compliant client. When the remote requests are enabled (allow-remote-requests: yes), the MikroTik router responds to TCP and UDP DNS requests on port 53.
DNS cache management can be accessed under the /ip dns submenu. To enable DNS cache, use the set command, for example:
[admin@MikroTik] /ip dns> set servers=10.5.8.1,159.148.60.20
allow-remote-requests=yes
[admin@MikroTik] /ip dns> print
servers: 10.5.8.1,159.148.60.20
allow-remote-requests: yes
max-udp-packet-size: 512
cache-size: 2048KiB
cache-max-ttl: 1w
cache-used: 144KiB
[admin@MikroTik] /ip dns>
Moreover, MikroTik router can be specified as a primary DNS server under its dhcp-server settings.
- Property Description:
|
|
|
|
servers (IP address list; default: 0.0.0.0) |
comma separated list of DNS server IP addresses |
|
allow-remote-requests (yes | no; default: no) |
specifies whether to allow network requests |
|
cache-size (integer: 512..10240; default: 2048KiB) |
specifies the size of DNS cache in KiB |
|
cache-max-ttl (time; default: 1w) |
Specifies maximum time-to-live for cache records. In other words, cache records will expire unconditionally after cache-max-ttl time. Shorter TTL received from DNS servers are respected |
|
cache-used (read-only: integer) |
displays the current cache size in KiB |
Static DNS entries
MikroTik RouterOS allows to add static DNS entries. It allows you to link the particular domain names with the respective IP addresses and advertize this address to the DNS clients. Two main properties that must be specify are address and domain name:
- address – (IP address) IP address to resolve domain name with
- name – (text) DNS name to be resolved to a given IP address. May be a regular expression
[admin@MikroTik] /ip dns static> add name=admin.example.com address=1.1.1.2
Remember that the meaning of a dot (.) in regular expressions is any character, so the expression should be escaped properly. For example, if you need to match anything within example.com domain but not all the domains that just end with example.com, use name=".example.com" and this condition accept domain names like www.another-example.com, ns1.another-exampe.com.
For example:
[admin@MikroTik] /ip dns static> print detail Flags: D - dynamic, X - disabled, R - regexp 0 name="admin.example.com" address=1.1.1.2 ttl=1d 1 R name=".example.com" address=1.2.2.3 ttl=1d
MikroTik /ip dns command references you can find: