Testwiki/Proxy (Cache) services

From MikroTik Wiki
Jump to navigation Jump to search

Proxy (Cache) services

Web proxy (/ip proxy)

MikroTik RouterOS performs proxying of HTTP and HTTP-proxy (for FTP, HTTP and HTTPS protocols) requests. Proxy server performs Internet object cache function by storing requested Internet objects, i.e., data available via HTTP and FTP protocols on a system positioned closer to the recipient in the form of speeding up customer browsing by delivering them requested file copies from proxy cache at local network speed. MikroTik RouterOS implements the following proxy server features:

  • Regular HTTP proxy – customer (itself) specify what is proxy server for him
  • Transparent proxy – customer does not know about the proxy being enabled and there isn’t need any additional configuration for web browser of client.
  • Access list by source, destination, URL and requested method (HTTP firewall)
  • Cache access list to specify which objects to cache, and which not.
  • Direct Access List – to specify which resources should be accessed directly, and which - through another proxy server
  • Logging facility – allows to get and to store information about proxy operation
  • Parent proxy support – allows to specify other proxy server, ('if they don’t have the requested object ask their parents, or to the original server.)


A proxy server usually is placed at various points between users and the destination server (also known as origin server) on the Internet. (see Figure 10.1).

File:Image10001.gif

A Web proxy (cache) watches requests coming from client, saving copies of the responses for itself. Then, if there is another request for the same URL, it can use the response that it has, instead of asking the origin server for it again. If proxy has not requested file, it downloads that from the original server.

There can be many potential purpose of proxy server:

  • To decrease access speed to resources (it takes less time for the client to get the object).
  • Works as HTTP firewall (deny access undesired web pages),

Allows to filter web content (by specific parameters, like source address, destination address and port, URL, HTTP request method) scan outbound content, e.g., for data leak protection.

Note that it may be useful to have Web proxy running even with no cache when you want to use it only as something like HTTP and FTP firewall (for example, denying access undesired web pages or deny specific type of files e.g. .mp3 files) or to redirect requests to external proxy (possibly, to a proxy with caching functions) transparently.


Proxy configuration example

In MikroTik RouterOS proxy configuration are organized under /ip proxy menu. See below how to enable the proxy on port 8080 and set up 195.10.10.1 as proxy source address:

 [admin@MikroTik] ip proxy> set enabled=yes port=8080 src-address=195.10.10.1

 [admin@MikroTik] ip proxy> print
                      enabled: yes
                  src-address: 195.10.10.1
                         port: 8080
                 parent-proxy: 0.0.0.0:0
                  cache-drive: system
          cache-administrator: "admin@mikrotik.com"
          max-disk-cache-size: none
           max-ram-cache-size: 100000KiB
           cache-only-on-disk: yes
   maximal-client-connections: 1000
   maximal-server-connections: 1000
              max-object-size: 2000KiB
               max-fresh-time: 3d


When setting up regular proxy service, make sure it serves only your clients and prevent unauthorized access to it by creating firewall that allow only your clients to use proxy, otherwise it may be used as an open proxy.

Remember that regular proxy require also to configure client’s web browser settings.

For example:

Explorer 8.x

Firefox 3.x

Opera 10.x

Select Tools>Internet options.

Click the Connections tab.

Select the necessary connection and choose Settings button.

Configure proxy address and port.

Select Tools>Options.

Click the Advanced tab.

Open the Network tab.

Click the Connection/Settings

Select Manual proxy configuration'

Select Tool>Preferences.

Open the Advanced tab/Network.

Click the Proxy servers.

Enter proxy address and port.


Transparent proxy configuration example

RouterOS can also act as a Transparent Caching server, with no configuration required in the customer’s web browser. Transparent proxy does not modify requested URL or response. RouterOS will take all HTTP requests and redirect them to the local proxy service. This process will be entirely transparent to the user (users may not know anything about proxy server that is located between them and original server), and the only difference to them will be the increased browsing speed.


To enable the transparent mode, firewall rule in destination NAT has to be added, specifying which connections (to which ports) should be transparently redirected to the proxy. Check proxy settings above and redirect us users (192.168.1.0/24) to proxy server.

[admin@MikroTik] ip firewall nat> add chain=dstnat protocol=tcp src-address=192.168.1.0/24 \
dst-port=80 action=redirect to-ports=8080

 [admin@MikroTik] ip firewall nat> print
 Flags: X - disabled, I - invalid, D - dynamic
  0   chain=dstnat protocol=tcp dst-port=80 action=redirect to-ports=8000
 [admin@MikroTik] ip firewall nat>


The web proxy can be used as transparent and normal web proxy at the same time. In transparent mode it is possible to use it as standard web proxy, too. However, in this case, proxy users may have trouble to reach web pages which are accessed transparently.


Proxy based firewall – Access List (/ip proxy access)

Access list is implemented in the same way as MikroTik firewall rules. Rules are processed from the top to the bottom. First matching rule specifies decision of what to do with this connection. Connections can be matched by its source address, destination address, destination port, substring of requested URL (Uniform Resource Locator) or request method. If none of these parameters is specified, every connection will match this rule.

If connection is matched by a rule, action property of this rule specifies whether connection will be allowed or not (deny). If connection does not match any rule, it will be allowed.

In this example assume that we have configured transparent proxy server as given example above. Argument description:

  • src-address - source address of the request
  • dst-address - destination address of the request
  • dst-port - destination port of the request
  • path - the URL of the request. Can be regular expression.
  • action - (allow / deny) action to take.
  • method - (any | connect | delete | get | head | options | post | put | trace) - HTTP method used in the request


Block particular Websites.

 /ip proxy access add dst-host=www.facebook.com action=deny

It will block website http://www.facebook.com, we can always block the same for different networks by giving src-address.

/ip proxy access add src-address=192.168.1.0/24 dst-host=www.facebook.com action=deny

Users from network 192.168.1.0/24 not be able to access to website www.facebook.com.


You can block also websites that contain specific words in URL:

 /ip proxy access add dst-host=:mail action=deny

This statement will block all websites which contain word “mail” in URL. Like www.mail.com, www.hotmail.com, mail.yahoo.com etc.


We can also stop downloading specific types of files like .flv, .avi, .mp4, .mp3, .exe, .dat, …etc.

 /ip proxy access
 add path=*.flv action=deny
 add path=*.avi action=deny
 add path=*.mp4 action=deny
 add path=*.mp3 action=deny
 add path=*.zip action=deny
 add path=*.rar action=deny.

Here are available also different wildcard characters, to creating specific conditions and to match it by proxy access list.

Wildcard properties (dst-host and dst-path) match a complete string (i.e., they will not match "example.com" if they are set to "example"). Available wildcards are '*' (match any number of any characters) and '?' (match any one character).

Regular expressions are also accepted here, but if the property should be treated as a regular expression, it should start with a colon (':').

To show that no symbols are allowed before the given pattern, we use ^ symbol at the beginning of the pattern.

To specify that no symbols are allowed after the given pattern, we use $ symbol at the end of the pattern.


Direct access (/ip proxy direct)

If parent-proxy property is specified, it is possible to tell proxy server whether to try to pass the request to the parent proxy or to resolve it connecting to the requested server directly. Direct Access List is managed just like Proxy Access List and determines access to proxy server.

Unlike the access list, the direct proxy access list has default action equal to deny. It takes place when no rules are specified or a particular request did not match any rule.


Cache management (/ip proxy cache)

In the same way as you implemented proxy access-list you can define also access-list for proxy that filter proxy content. Cache access list specifies, which requests (domains, servers, pages) have to be cached locally by web proxy, and which not. Default action (if no matching rule is found) is to store object into cache memory.

More information and /ip proxy command references are found here:

http://wiki.mikrotik.com/wiki/Manual:IP/Proxy


DNS proxy (/ip dns)

DNS (Domain Name System) cache facility on MikroTik router to provide domain name resolution for router itself as well as for the connected clients. This facility allows router works as default DNS resolver for clients on the LAN.

Router (resolver) receives local client DNS requests and forwards to public DNS only those requests that are not yet in the local DNS cache.

Domain name are useful as argument to a local resolver, which retrieves information associated with the domain name (basically IP address). A user may ask for the host IP address associated with particular domain name (For example: What is IP address of domain name “www.mikrotik.com”?' -> Resolver return answer that IP address for www.mikrotik.com is 159.148.147.196).

DNS query and response from “Wireshark” traffic analyzer:

Domain Name System (query)

No.   Time       Source          Destination    Protocol   Info

614   80.581710  172.16.16.243   172.16.16.1    DNS(53)    Standard query A www.mikrotik.com

Domain Name System (response)

No.    Time      Source        Destination     Protocol  Info

615   80.592878  172.16.16.1   172.16.16.243   DNS(53)   Standard query response A 159.148.147.196

A special type of request is sent to the resolver on a port 53. If you want to receive IP addresses of all public domain names your local resolver must know at least one public name server. When the resolver processes a user query that is not in local cache it ask to public name server for required information. In this way resolver learns content of name server.

DNS cache is used to minimize DNS requests to an external DNS server as well as to minimize DNS resolution time.


DNS proxy (cache) setup

A MikroTik router with DNS feature enabled can be set as a DNS server for any DNS-compliant client. When the remote requests are enabled (allow-remote-requests: yes), the MikroTik router responds to TCP and UDP DNS requests on port 53.

DNS cache management can be accessed under the /ip dns submenu. To enable DNS cache, use the set command, for example:

[admin@MikroTik] /ip dns> set servers=10.5.8.1,159.148.60.20 
allow-remote-requests=yes
[admin@MikroTik] /ip dns> print 
                servers: 10.5.8.1,159.148.60.20
  allow-remote-requests: yes
    max-udp-packet-size: 512
             cache-size: 2048KiB
          cache-max-ttl: 1w
             cache-used: 144KiB
[admin@MikroTik] /ip dns>

Moreover, MikroTik router can be specified as a primary DNS server under its dhcp-server settings.

Property Description:
Property
Description

servers (IP address list; default: 0.0.0.0)

comma separated list of DNS server IP addresses

allow-remote-requests (yes | no; default: no)

specifies whether to allow network requests

cache-size (integer: 512..10240; default: 2048KiB)

specifies the size of DNS cache in KiB

cache-max-ttl (time; default: 1w)

Specifies maximum time-to-live for cache records. In other words, cache records will expire unconditionally after cache-max-ttl time. Shorter TTL received from DNS servers are respected

cache-used (read-only: integer)

displays the current cache size in KiB


Static DNS entries

MikroTik RouterOS allows to add static DNS entries. It allows you to link the particular domain names with the respective IP addresses and advertize this address to the DNS clients. Two main properties that must be specify are address and domain name:

  • address – (IP address) IP address to resolve domain name with
  • name – (text) DNS name to be resolved to a given IP address. May be a regular expression
[admin@MikroTik] /ip dns static> add name=admin.example.com address=1.1.1.2

Remember that the meaning of a dot (.) in regular expressions is any character, so the expression should be escaped properly. For example, if you need to match anything within example.com domain but not all the domains that just end with example.com, use name=".example.com" and this condition accept domain names like www.another-example.com, ns1.another-exampe.com.

For example:

[admin@MikroTik] /ip dns static> print detail 

Flags: D - dynamic, X - disabled, R - regexp 
 0     name="admin.example.com" address=1.1.1.2 ttl=1d 
 1   R name=".example.com" address=1.2.2.3 ttl=1d 

MikroTik /ip dns command references you can find:

http://wiki.mikrotik.com/wiki/Manual:IP/DNS