Tiny Web Proxy and Content Filtering Appliance (version 1.2)

See the Updated version for CentOS 6, Squid 3 and qlproxy 1.4..

This small HOWTO will show you how to set up a small virtual machine to speed up and secure your home / small enterprise web surfing network using Debian 6, Squid 3.1 and QuintoLabs Content Security 1.2 applications deployed in a VMWare Virtual Player running on Windows 7 x64 as a host operating system. This howto is targeted at novice users and may sometimes seem too thorough for more advanced gurus.

While preparing this howto I have tried several different Linux distributions to act as the base for the virtual appliance we are going to create and found out that the one easiest to deploy, having smallest memory footprint and latest packages available is Debian Linux 6. In case you disagree please adjust the instructions of this howto for your favorite distribution and mention the needed changes in comments. I will try to integrate them into the next version of the howto.

Step 1. Download and Install Debian 6

Go to http://www.debian.org and get the smallest ISO image of the latest version of Debian 6 32-bit. The site says the recommended version for a modern server is x64 bit but as we are trying to create a small virtual machine the 32bit will suffice for our purposes.

Start up the VMWare Virtual Player and create a new virtual machine with the following hardware parameters: name – virtual-proxy, hard disk – 8Gb. Press the “Customize the hardware” button and delete the floppy, USB controller, printer and sound card, set the amount of memory to 256Mb. Switch the network adapter from “NAT” mode into “Bridged”. Point the virtual CDROM to the ISO image that you have downloaded earlier and start the virtual machine.

Follow the steps of the Debian install wizard mostly accepting the defaults. Deselect all packages in the “Software Selection” wizard step except for “SSH server” and “Standard system utilities”. Configure machine hostname as “proxy” and root password as “P@ssw0rd” (without quotation marks). Now wait a little until the installation is complete and then reboot the system.

The following screenshots of the installation show most important steps of the installation process.

After reboot check the amount of free memory available and note that it is around 200Mb (kind of cool comparing to default CentOS 5 installation with 140Mb of free memory available after similar install).

Step 2. Perform post install configuration of Debian

The IP address of the network interface card of our freshly installed Debian server is set to DHCP. While it is possible to leave it like this it means that every time the server is restarted it may be assigned a different IP address so accessing this server later by IP address may become a little irritating. The easiest way is to set the IP address to static one. To do it we must change the file located at /etc/network/interfaces by typing the following in the terminal:

#nano /etc/network/interfaces

find the line with iface eth0 inet dhcp in the file, remove or comment it out and add the following:


# iface eth0 inet dhcp
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

NOTE: The provided above settings (IP addresses) are valid for my environment that connects to the ISP using the LinkSys Wireless N Broadband Router (with custom dd-wrt firmware) that has a DHCP server build in that gives out internal IP addresses from the 192.168.1.* private subnet. Your router may give other addresses so please beware 🙂 !

One more thing left to do is adjust the DNS settings. Run the following command in the terminal #nano /etc/resolv.conf and add the IP address of the DNS server that runs on router
nameserver 192.168.1.1

Now restart your networking by typing #/etc/init.d/networking in the root terminal or just restarting the virtual machine. After restart confirm that the networking functions correctly by typing in the terminal (there should not be any errors in the outputs on these commands):
ping –c 3 192.168.1.1
nslookup google.com

Next small step prior to installing Squid is to update the system to the latest version of packages. This will require disabling Debian repositories that refer to CDROM that we used to install the system from and enabling the online repositories. Type the following in the root terminal #nano /etc/apt/sources.list then find and comment all lines that start with “deb cdrom:”. Save the file by pressing CTRL+o followed by CTRL+x and exit nano. Then perform system update by typing #apt-get update && apt-get upgrade in the root terminal. Wait until the system downloads all the updates.

Step 3. Install Squid Web Caching Proxy

Next we need to install the latest version of the Squid proxy server. In order to do that type the following in the root terminal #apt-get install squid3. The squid package is downloaded from the Internet and installed automatically. The only thing to do is to let the external users from our network access the squid. Open the squid configuration file by typing #nano /etc/squid3/squid.conf and uncomment the following lines: acl localnet src 192.168.0.0/16 (approximately line 651) and http_access allow localnet (approximately line 786). Save the file and restart the squid by typing #service squid3 restart. To verify that squid runs correctly point your browser to the IP address of the proxy server (192.168.1.2) and surf to some of your favorite websites.

Step 4. Install Apache 2

It is also a good idea to have a web server installed on the virtual machine. This web server will later host the status and report information for Squid and Content Security for Squid Proxy. In order to have the Apache2 installed issue the following commands in the root terminal # apt-get install apache2 libapache2-mod-php5 then open your browser and navigate to http://192.168.1.2. You should see the “It Works!” greetings from Apache.

Step 5. Install QuintoLabs Content Security 1.2

Next step would be to install the Content Security 1.2 for Squid from QuintoLabs (I will refer to it as qlproxy further in text). NOTE: this part was upgraded from the previous version of this howto. The 1.2 version of qlproxy now supports ICAP based integration with Squid and is much easier to install.

For those who do not know, qlproxy is a content filtering server to be used as a companion to the Squid web proxy that (citation) “allows the administrator to filter/block web downloads, remove advertisements and banners and control web site usage by the proxy clients” (i.e. prohibit explicit and adult content).

Unfortunately QuintoLabs does not yet have online package repository with qlproxy but I heard it will change in the future. So until it happens we have to get the Debian 6 package of qlproxy manually from the QuintoLabs web site at http://www.quintolabs.com/qlicap_download.php using your favorite browser (thorough your new Squid of course 🙂 and upload the package to the system using scp. Another (much easier) way would be to type the following commands in the root terminal:

#wget http://www.quintolabs.com/qlproxy/binaries/1.2.232.0/qlproxy-1.2-debian-1.2.232_i386.deb

Wait a little until the download completes (approx. 20Mb) and run the following command to install the downloaded package:

#dpkg –install qlproxy-1.2-debian-1.2.232_i386.deb

installer will run and after a short while the program will be installed into /opt/quintolabs/qlproxy. Now we need to configure it and integrate it with Squid. The configuration files are plain text and stored in /opt/quintolabs/qlproxy/etc/*.conf and rather simple to modify with a handful of comments inside. I am going to perform the following modifications:

  1. Change the default blocked template to the minimal by commenting out the blocked_page=/opt/quintolabs/qlproxy/redirect/blocked.html and uncommenting blocked_page=/opt/quintolabs/qlproxy/redirect/blocked_minimal.html. This is done to get the empty blank page on the screen when qlproxy blocks something. The idea for this is to have nothing on the browser screen instead of default text “this item was blocked…”. When QuintoLabs adds a default 1×1 pixel for blocked ads in the next version this step would be irrelevant… but for now it is ok.
  2. I personally do not like excessive advertizing on the web sites so as I often browse through russian and german web sites I will also enable adblock filtering by uncommenting the corresponding russian and german adblock subscriptions in /opt/quintolabs/qlproxy/etc/adblock.conf file. I also do not like sites tracking me so I usianlly uncomment easy_privacy subscription in the same file.
  3. My kids sometimes play online games on my computer so I prefer to set the level of adult blocking heuristics to high in the /opt/quintolabs/qlproxy/etc/adultblock.conf file – by changing from heuristics_level = normal to heuristics_level = high. If anything is falsely blocked by the qlproxy I can later add it to the exceptions.conf file to have it passed through.
  4. I’ve heard that worms, trojans and other malware related software often connect to the world by IP addresses so I put a magic regexp into the /opt/quintolabs/qlproxy/etc/httpblock.conf file to filter them out url = http://\d+\.\d+\.\d+\.\d+/.*

Good for now, let us issue a restart command to make the qlproxyd daemon reload the configuration #/etc/init.d/qlproxy restart.

Next we need to integrate it with Squid. As the qlproxy daemon now supports the shiny ICAP protocol this is a little bit different from the url_rewrite_program integration described in the previous version of this howto. By the way, README file in /opt/quintolabs/qlproxy/ contains instructions on how to do that. Anyway here are the steps required:

  1. Open the /etc/squid3/squid.conf in nano by typing #nano /etc/squid3/squid.conf in the root terminal
  2. Find and change the icap_enable off directive to icap_enable on
  3. Uncomment icap_preview_enable on directive
  4. Uncomment and change icap_preview_size -1 directive to icap_preview_size 4096
  5. Uncomment directive icap_persistent_connections on
  6. Uncomment and change directive icap_send_client_ip off to icap_send_client_ip on
  7. Uncomment and change directive icap_send_client_username off to icap_send_client_username on
  8. Find the icap_service section with commented out samples for different services and add somewhere there two lines icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod and icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod
  9. Find the adaptation_access section and add the following two lines adaptation_access qlproxy1 allow all and adaptation_access qlproxy2 allow all
  10. Write the changes to file and close nano by typing Ctrl+o and then Ctrl+x

Now restart the squid by typing #service squid3 restart in the root terminal. After restart try surfing the same sites with your browser and see how nicely ads are blocked. Another useful test is to go to the eicar.com web site and try to download a sample artificial eicar.com virus to see that com files are blocked by the download filter.

Note: for those of you who must stick with squid 2.6 for performance reasons the url rewriter integration is quite straightforward. Open /etc/squid3/squid.conf and find the url_rewrite_program section. Add the following line url_rewrite_program /opt/quintolabs/qlproxy/sbin/qlproxyd_redirector --config_path=/opt/quintolabs/qlproxy/etc/qlproxyd.conf.

The last thing to do is to integrate the qlproxy with Apache to be able to see the reports on user activities generated once a day. This is actually quite easy, open the /etc/apache2/sites-enabled/default file and add the following to it near the </VirtualHost> directive:


Alias /reports /opt/quintolabs/qlproxy/reports
<Directory "/opt/quintolabs/qlproxy/reports">
Options FollowSymLinks
AllowOverride None
</Directory>

Now reload the apache by typing in the terminal #service apache2 restart.

You can navigate to http://192.168.1.2/reports to see the generated reports. The funny thing is that qlproxy blocks access by the IP address according to our settings in httpblock.conf described earlier. Solution would be to add the 192.168.1.2 as entry to the /opt/quintolabs/qlproxy/etc/exceptions.conf or just tell the browser not to use proxy for this address.

Resume

Finally everything is in place to start the accelerated secure web surfing without adverts – point your browser to 192.168.1.2 port 3128 and browse to your favorite website and see the difference. The IP addresses in URLs are blocked and explicitly adult content sites too. The VMWare takes not more than 256 MB and surfing experience is quite acceptable. The system is automatically updated once a day for the latest url block list and advert subscriptions and requires minimal additional maintenance.

Useful documentation links

http://www.debian.org
http://www.squid-cache.org
http://www.quintolabs.com/qlicap_info.php
http://issues.quintolabs.com/trac/quintolabs_qlicap/wiki/QlicapDocs

About sichent

sichent
This entry was posted in Network. Bookmark the permalink.

2 Responses to Tiny Web Proxy and Content Filtering Appliance (version 1.2)

  1. Pingback: Build SELinux Policy for Your Next Daemon. Part 1 | sichent

  2. Pingback: Tiny Web Proxy and Content Filtering Appliance on CentOS 6 (version 1.4) | sichent

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s