Highly available Squid, HTTPS filtering and Active Directory

Having a cluster of Squids in your Active Directory environment is a must have for more or less big organization where number of users and browsing habits have grown out of one instance of Squid. There are several ways to implement this each with its advantages and disadvantages.

Today we present one possible way to do that using HAPROXY as frontend with number of Active Directory integrated Squid’s supporting Negotiate/Kerberos, NTLM and Basic LDAP authentication schemes and performing HTTPS filtering and deep content inspection with Web Safety ICAP server.

We will use the virtual appliances from https://www.diladele.com/download_next_version.html (version 4.8) as HTTPS filtering Squids with integrated ICAP web filter, linked to Active Directory domain using instructions from https://docs.diladele.com/administrator_guide_4_8/active_directory/index.html

  1. Create new type A record for proxy.example.lan with IP address 192.168.178.10. This will be the haproxy frontend to our cluster.
  2. Create new type A record for node11.example.lan with IP address 192.168.178.11.
  3. Create new type A record for node12.example.lan with IP address 192.168.178.12.
  4. Deploy two virtual appliances of Web Safety and assign IP address of 192.168.178.11 for the first node and 192.168.178.12 for the second node. For the instructions on how to assign static IP for a virtual appliance see article How to Set Static IP Address in VA.
  5. Configure haproxy on proxy.example.lan with the following configuration /etc/haproxy/haproxy.cfg.
global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    tcp
    option  tcplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000

frontend squid
    bind 192.168.178.10:3128
    default_backend squid_pool

backend squid_pool
    balance roundrobin
    mode tcp
    server squid1 192.168.178.11:3128 check
    server squid2 192.168.178.12:3128 check

On each node, configure NTLM and Basic LDAP authentication as usual but when configuring Kerberos authenticator provide SPN based on proxy.example.lan and check the Use GSS_C_NO_NAME checkbox. This will let the cluster node process requests for Kerberos authentication from browsers based on credentials contained in the request and not based on SPN (SPN still needs to be configured though).

gcc_c_no_name

Click Save and Restart on both nodes and voila – your HA proxy will happily distribute load between the nodes.

Limitations

For now Web Safety Web UI is not cluster aware. It means after you click Save and Restart or Save and Reload in the Web UI it generates correct configuration in /opt/qlproxy/etc folder and restarts/reloads locally installed version of wsicapd, wsmgrd and squid. We need to somehow propagate the changes of configuration from local node to all nodes in the cluster.

One way to do it that comes to mind is:

  • Select for yourself one node as master node. This is not a real master node in technical terms but just an imaginary concept that tells you to always connect to that node by IP address to access Web Safety Web UI.
  • Make any change to the web filtering setting always from the master node. After a change is made click Save and Restart or Save and Reload. This will generate the web filter configuration (JSON files) into local folder /opt/qlproxy/etc. Web UI will then call /opt/qlproxy/bin/restart.sh or /opt/qlproxy/bin/reload.sh scripts respectively.
  • Add some commands to sync the contents of /opt/qlproxy/etc folder to all other cluster nodes and also some command to restart/reload daemon instances on these nodes. Exactly what commands is up to the administrator but we recommend rsync for syncing folders and ansible to remotely restart services.

For example to upload changes from 192.168.178.10 node to 192.168.178.11 run the following rsync command. Please note in the example the password is provided in the command line. You need to setup passwordless auth in ssh in cluster nodes to rsync without password (i.e. automatically from script).

rsync -r -a -v -e "ssh -l root" --delete /opt/qlproxy/etc 192.168.178.11:/opt/qlproxy

After this call we have all our cluster nodes synced with configuration and need only to restart/reload daemons. Ansible script to do that may look like the following (see http://docs.ansible.com/ansible/service_module.html#examples):

# Example action to restart service httpd, in all cases
- service: name=wsicapd state=restarted

 

 

About sichent

sichent
This entry was posted in Linux. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s