Having a cluster of Squids in your Active Directory environment is a must have for more or less big organization where number of users and browsing habits have grown out of one instance of Squid. There are several ways to implement this each with its advantages and disadvantages.
Today we present one possible way to do that using HAPROXY as frontend with number of Active Directory integrated Squid’s supporting Negotiate/Kerberos, NTLM and Basic LDAP authentication schemes and performing HTTPS filtering and deep content inspection with Web Safety ICAP server.
We will use the virtual appliances from https://www.diladele.com/download_next_version.html (version 4.8) as HTTPS filtering Squids with integrated ICAP web filter, linked to Active Directory domain using instructions from https://docs.diladele.com/administrator_guide_4_8/active_directory/index.html
- Create new type A record for proxy.example.lan with IP address 192.168.178.10. This will be the haproxy frontend to our cluster.
- Create new type A record for node11.example.lan with IP address 192.168.178.11.
- Create new type A record for node12.example.lan with IP address 192.168.178.12.
- Deploy two virtual appliances of Web Safety and assign IP address of 192.168.178.11 for the first node and 192.168.178.12 for the second node. For the instructions on how to assign static IP for a virtual appliance see article How to Set Static IP Address in VA.
- Configure haproxy on proxy.example.lan with the following configuration /etc/haproxy/haproxy.cfg.
global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon defaults log global mode tcp option tcplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 frontend squid bind 192.168.178.10:3128 default_backend squid_pool backend squid_pool balance roundrobin mode tcp server squid1 192.168.178.11:3128 check server squid2 192.168.178.12:3128 check
On each node, configure NTLM and Basic LDAP authentication as usual but when configuring Kerberos authenticator provide SPN based on proxy.example.lan and check the Use GSS_C_NO_NAME checkbox. This will let the cluster node process requests for Kerberos authentication from browsers based on credentials contained in the request and not based on SPN (SPN still needs to be configured though).
Click Save and Restart on both nodes and voila – your HA proxy will happily distribute load between the nodes.
For now Web Safety Web UI is not cluster aware. It means after you click Save and Restart or Save and Reload in the Web UI it generates correct configuration in /opt/qlproxy/etc folder and restarts/reloads locally installed version of wsicapd, wsmgrd and squid. We need to somehow propagate the changes of configuration from local node to all nodes in the cluster.
One way to do it that comes to mind is:
- Select for yourself one node as master node. This is not a real master node in technical terms but just an imaginary concept that tells you to always connect to that node by IP address to access Web Safety Web UI.
- Make any change to the web filtering setting always from the master node. After a change is made click Save and Restart or Save and Reload. This will generate the web filter configuration (JSON files) into local folder /opt/qlproxy/etc. Web UI will then call /opt/qlproxy/bin/restart.sh or /opt/qlproxy/bin/reload.sh scripts respectively.
- Add some commands to sync the contents of /opt/qlproxy/etc folder to all other cluster nodes and also some command to restart/reload daemon instances on these nodes. Exactly what commands is up to the administrator but we recommend rsync for syncing folders and ansible to remotely restart services.
For example to upload changes from 192.168.178.10 node to 192.168.178.11 run the following rsync command. Please note in the example the password is provided in the command line. You need to setup passwordless auth in ssh in cluster nodes to rsync without password (i.e. automatically from script).
rsync -r -a -v -e "ssh -l root" --delete /opt/qlproxy/etc 192.168.178.11:/opt/qlproxy
After this call we have all our cluster nodes synced with configuration and need only to restart/reload daemons. Ansible script to do that may look like the following (see http://docs.ansible.com/ansible/service_module.html#examples):
# Example action to restart service httpd, in all cases - service: name=wsicapd state=restarted