How to make transparent web proxy

Overview

Web proxy is a service that is placed between a client and the internet, specifically for HTTP web surfing. It is normal that only HTTP traffic is cached. It is not so easy to cache and provide a web-proxy for HTTPS and FTP. Therefore the following example only shows how easy it is to enable a transparent web-proxy for HTTP traffic.

There are two main benefits to using a web-proxy.

• Raising Security for client and network
• Enhanced Performance and possibly lowering costs for client and network

Raising Security

Security is raised as the client is not directly connected to the website they are requesting data from. The client makes a connection request to the web-proxy and the web-proxy fetches the data on the client's behalf. Therefore the internet is connected to the web-proxy interface, not directly to the client. Using a web-proxy also allows the possibility of providing other services, such as anti-virus scanning, content filtering and monitoring or reports on the websites being requested.

Enhanced Performance

Performance is enhanced as it is very likely that the same identical website is being requested by many clients. If the web page is cached, then the web-proxy can deliver the content of that web page directly from it's own cache, rather than fetching it every single time, again and again, from the internet. This is very important for satellite links or on limited internet connections. If the network connection is metered by the service provider any means of reducing the traffic will bring cost benefits.

Normally when placing a web-proxy into the network, the client web browsers, such as IE, Firefox, Safari must be manually configured to point the web page requests through the web-proxy. However, it is more convenient to redirect the http web traffic on port 80 through to your web-proxy without needing any manual configuration of the client. This is called 'transparent web proxy'.

Hardware requirements

It is important to consider the level of traffic that will be handled by the web-proxy, which on large networks can be very high. Ensure that the hardware chosen is appropriate to the level of traffic you will expect to transport! Caching on medium to large networks will require some serious hardware as any bottleneck in the system will completely negate any speed improvement from using a local cache. Do not use NAND memory for caching. Always use a real hard drive or RAM. NAND will wear out after a finite number of read/write cycles and will also be slow.

Also ensure that the web-proxy cache is stored on a physically separate drive (store) than the Router OS. Placing the cache on a separate store to the ROS ensures maximum performance and reduces problems if the disk becomes full or fails as the OS will then still be OK!

Howto

By default, the web-proxy is listening on port 8080. Therefore we first need to redirect all traffic on port 80 to port 8080 with a DST-NAT firewall rule and ensure that the web-proxy service is enabled and listening to port 8080.

To enable a transparent web proxy on Mikrotik, perform the following:

ip firewall nat add in-interface=ether1 dst-port=80 protocol=tcp action=redirect to-ports=8080 chain=dstnat 
ip proxy set enabled=yes port=8080

Pay particular attention to locking down the security of the web-proxy. Read about the /ip proxy access command!

Webbox problem

By default the webbox user interface of the Mikrotik is also on port 80. By transparently redirecting all traffic on port 80 through to the web-proxy service running on port 8080, you will no longer be able to get access to the webbox server running on the router itself. You should either change the port from port 80 to something else, e.g. 8888 or use a firewall rule to NOT redirect the port 80 traffic bound for the router to the web-proxy. There are examples in the ROS manual.

It is also important to consider the size of the cache and all the various other parameters you can change on the web-proxy, however this outside the scope of this article.

Customizing error pages

To customize the page web proxy shows on error:

[admin@MikroTik] > /ip proxy reset-html
Current html pages will be lost! Reset anyway? [y/N]

Answer 'y'. Now HTML files are accessible for editing. (Currently there is only one file: error.html, that contains the error message.)

[admin@MikroTik] > /file print
 # NAME                 TYPE                 SIZE                 CREATION-TIME
 0 webproxy             directory                                 jul/28/2009 12:07:51
 1 webproxy/error.html  .html file           529                  jan/02/1970 00:03:4
[admin@MikroTik] > /file edit webproxy/error.html contents
...

You can also simply replace the file with your own. The syntax used in the file is similar to to that used in hotspot HTML files. Predefined variables (such as $error, $url, $admin), as well as $(if ...) statements can be used.

See also

http://www.mikrotik.com/testdocs/ros/3.0/pnp/proxy.php

http://wirelessconnect.eu/index.php?option=com_content&task=view&id=206&Itemid=454