Transparent Caching Proxy with Squid and Just One Ethernet Port
Tags: ARM, Efika, Linux, Network, Squid
Some time ago we had a project which needed a simple transparent caching proxy for use in classrooms in Latin America. The classroom desktops consisted of ten of our Efika's, and we thought why not use an extra one of those to act as the proxy. This is not the kind of high bandwidth environment of a typical installation, so the performance limitations were not really an obstacle. The goal was rather to use the limited internet connection available as efficiently as possible. The configuration of the system using Squid is detailed below.
The goals:
- Install and configure DHCP server on the system for the clients
- Install and configure Squid as a transparent caching proxy with one Ethernet port
- Setup of the network interface and iptables rules to route packages appropriately
The Network consists of:
- Internet gateway only visible by the Squid server (IP = 192.168.253)
- The Squid server (IP = 192.169.2.253)
- Several clients with IP addresses serviced by the same Squid server (Range 10.0.0.10/10.0.0.100)
The first step is to make sure the system (assuming a headless Debian or Ubuntu installation) is up to date (aptitude update followed by aptitude safe-upgrade). It would probably be a good idea to have OpenSSH running so the configuration can be done remotely.
We need the required Squid and DHCP server software installed:
aptitude install squid3 dhcp3-server
Note: there is also a package called squid (without the '3'). We opt to use Squid3 since it is easier to configure and subjectively runs better.
Now, we have to configure the network interface. Since we have only one, we are going to multi-host this interface to serve two networks: the 192.168.2.x and 10.0.0.x ones. Edit /etc/network/interfaces file so it looks like this:
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 192.168.2.253 netmask 255.255.255.0 network 192.168.2.0 broadcast 192.168.2.255 gateway 192.168.2.254 auto eth0:0 iface eth0:0 inet static name Ethernet alias LAN card address 10.0.0.1 netmask 255.255.255.0 network 10.0.0.0
Both eth0 and eth0:0 are configured static, with eth0 the part that will talk to the internet gateway, and eth0:0 the part that will talk to our local network. After we have configured the network interface, let's continue with the DHCP server so we can serve our clients with IP addresses. Edit /etc/dhcp3/dhcpd.conf and add a subnet as follows:
subnet 10.0.0.0 netmask 255.255.255.0 { range 10.0.0.10 10.0.0.100; option routers 10.0.0.1; }
This will provision our clients with IP addresses in the range of 10.0.0.1o to 10.0.0.100, more than enough for this application. Make sure that in the same configuration file you also configure the correct domain name servers for your internet. This should be done in the following directives:
option domain-name "example.org"; option domain-name-servers ns1.example.org, ns2.example.org;
Next comes the configuration of Squid itself. The config file is quite big, but we only need to make some small changes to make it work. Open /etc/squid3/squid.conf in an editor and locate the section that deals with access control lists (acl). There should already be a minimum section like this:
acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
To this section, add the following acl:
acl our_networks src 10.0.0.0/24
Next, locate the section that sets the http_port variable. It should look like this:
# Squid normally listens to port 3128 http_port 3128
change the value of the http_port variable so that it reads:
# Squid normally listens to port 3128 http_port 10.0.0.1:3128 transparent
The reason for this is twofold: it enables transparent mode so we don't have to configure proxy settings on each client, and it makes sure we only listen on the internal network. Next, we have to configure to allow the clients access to the Squid server. Search for a section that configures http_access rules (or search for "INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS"). There, add the following line:
http_access allow our_networks
Finally, turn on the disk cache mechanism. Search for "Uncomment and adjust the following to add a disk cache directory" and change the line that reads:
#cache_dir ufs /var/spool/squid3 100 16 256
into:
cache_dir ufs /var/spool/squid3 2000 16 256
That's all for Squid. To tie this all together, we have to instruct Linux that it is ok to route packets, and how they have to be routed. For that, a script was written with inline comments explaining the directives which looks like this:
#!/bin/sh # Squid server IP SQUID_SERVER="10.0.0.1" # Interface connected to Internet INTERNET="eth0" # Address connected to LAN LOCAL="10.0.0.0/24" # Squid port SQUID_PORT="3128" # Clean old firewall iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X # Enable Forwarding echo 1 > /proc/sys/net/ipv4/ip_forward # Setting default filter policy iptables -P INPUT DROP iptables -P OUTPUT ACCEPT # Unlimited access to loop back iptables -A INPUT -i lo -j ACCEPT iptables -A OUTPUT -o lo -j ACCEPT # Allow UDP, DNS and Passive FTP iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT # set this system as a router for Rest of LAN iptables -t nat -A POSTROUTING -o $INTERNET -j MASQUERADE iptables -A FORWARD -s $LOCAL -j ACCEPT # unlimited access to LAN iptables -A INPUT -s $LOCAL -j ACCEPT iptables -A OUTPUT -s $LOCAL -j ACCEPT # DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy iptables -t nat -A PREROUTING -s $LOCAL -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT # if it is same system iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT #open everything iptables -A INPUT -i $INTERNET -j ACCEPT iptables -A OUTPUT -o $INTERNET -j ACCEPT # DROP everything and Log it iptables -A INPUT -j LOG iptables -A INPUT -j DROP
Make this script executable and run it at startup, for example within /etc/rc.local. That's it! Make sure you restart all the services and double check the clients get the correct IP address, gateway and name servers. The full configuration files and script are attached. Remove the .txt extension from 'interfaces.txt' and 'firewall.sh.txt'
Comments
Can we do this with Linux Namespace
I have installed linux container namespace or Debian ARM architecture .. can you help me to run this conig in linux namespace
Re: Can we do this with Linux Namespace
It's probably possible, but keep in mind that this article is quite old by now - things might have changed.
It's been a while since I've played around wtih things like proxies, so I don't think I'll be able to guide you through the process (I also have very limited free time). However the principles are pretty much the same, so you should be able to get there. I think there should be plenty of other resources available online that can help you as well.
Hi
I am using ubuntu 18.04.3tls and I have two network cards one local
and one in bridge mode as I can implement the squid in transparent mode.
Network card in bridge mode has the network 192.168.1.0/24.With this I have internet access and the card that is in the local network 172.168.1.0/24
Sorry, my English is that I'm Latino and I don't know much.
I am using ubuntu 18.04.3tls and I have two network cards one local and one in bridge mode as I can implement the squid in transparent mode.
Hi, This entry is quite old,
Hi,
This entry is quite old, so I'm not sure if all the info is still relevant. However, do keep in mind that with https (TLS) being used more and more, the effectiveness of a proxy becomes less.
Hope you can find the info in my post useful, but I think you might need to look at some other sites for more up-to-date info as well.
Johan.
configuration
first of all
can you please explain for my configuration
dhcp server settup via wifi
option routers is 192.168.1.1
dynamic allocation is given for clients
configuration
Hi.
It's been a while since I wrote this entry, so this is mostly from (distant) memory. You need to have control over what is serving the IP addresses. The reason I used a single device for that is because of simplicity. You now have a router which is sending out IP addresses, but that means that none of that traffic is going through the proxy. You probably have to configure your wireless access point to route through the proxy. In other words, put the access point behind the proxy and let the proxy communicate to the internet through a switch between your WAN network and the access point.
How you would do this exactly depends on your equipment though. The principle stays the same, except that instead of having a switch connected to the Ethernet port of the proxy and connect the PC's there, you have a wireless access point on that Ethernet port and have the PC's connect to that.
Hope this helps.
Johan.
Routing Issue
Hello,
Thank you so much for your guide and configuration files.
I applied everything on my configuration but the problem is my dhcp provider is core switch. Configuration is like the below scenario
Core Router > Firewall > Core Switch > Access Switch (With 22 VLan) > Users.
> DMZ Switch > proxy, dns, mail, web.
DMZ zone is 10.100.222.0/24
Users 10.10.0.0/16
Now what will be routing (IPtables Default route or anything)
Thanks in advance.
Hi. If I understand you
Hi.
If I understand you correctly, you only have to change the IP addresses on the two Ethernet interfaces (eth0 and eth0:0) so that eth0 is connected to the DMZ and eth0:0 serves your local clients with IP addresses. Except for the IP addresses, nothing else changes.
Johan.
directive in squid.conf
Hi,
Nice tutorial you've made here. Could u explain to me regarding the squid.conf. By referring to line 'http_access deny manager'. Why the config deny manager? I'm confuse. Help me
Best regards,
Tyro
Hi.Glad you like the
Hi.
Glad you like the tutorial!
http_access deny manager controls access rights to the cache manager: http://wiki.squid-cache.org/Features/CacheManager. The reason to deny it in that line is to make sure not just anyone can access the manager. The line right above that one allows for manager access form localhost only, not every device connected to the network the squid proxy is serving.
Johan.
Hi Johan, Nice setup. I'm
Hi Johan,
Nice setup. I'm curious - given that this is an small embedded system, how much throughput were you getting with this setup? Did you run any benchmarks comparing bandwidth before and after the proxy was placed inbetween?
Best regards,
John
Hi John, No, I did not run
Hi John,
No, I did not run any benchmarks since the bottle neck for the set-up was the main internet connection to begin with, and the proxy itself was fast enough to make that experience orders of magnitude better. I imagine that the device won't be the best choice for a corporate environment of course, but for small amounts of users with basic internet needs it works great.
Johan.
Https sites not blocking
how to block https sites
Blocking https
Hi,
You can block https with iptables, e.g.:
Keep in mind that in the set-up discussed in the post I did not make provisions for https in the first place. In order to use squid as a transparent proxy for https as well, complement my post with this one here: http://ajayadas.com/e110body-anchor/
Johan.
doubts in lines of the scrip located in /etc/rc.local and others
Hello Johan, thanks for such a fantastic tutorial, I'm doing a project with a raspberry pi 2b + and 3b +
and I thought it was great to do it with them, but on the way taking the tutorial out,
I came across that the DHCP3 SERVER version did not It was useful so I chose to use ISC-DHCP-SERVER
and everything started to work according to what you propose in the tutorial, the router is an internet service with 2MB of bandwidth
and generates its own ips range so that disable the DHCP of the router, so that my server ISC-DHCP-SERVER deploy in the LAN and WLAN
the ips to my devices and if indeed when observing in the devices the TCP / IP works and I have the transparent proxy with its due gateway ,
I can browse some http sites such as google, youtube, netflix, but wanting to see some other sites I can not through the proxy either http or https,
not if you have to add something to the script in /etc/rc.local I have doubts with some lines for example in the next line:
# DNAT port 80 request comming from LAN systems to squid 3128 ($ SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -s $ LOCAL -p tcp --dport 80 -j DNAT --to $ SQUID_SERVER: $ SQUID_PORT
Do we have to add another line for port 443 with the same characteristics as with port 80? like the other line that says:
# if it is same system
iptables -t nat -A PREROUTING -i $ INTERNET -p tcp --dport 80 -j REDIRECT --to-port $ SQUID_PORT
Would we have to add another line with port 443?
and also I have doubts my router generates ipv6 and in the scritp you also have a line that says:
# Enable Forwarding
echo 1> / proc / sys / net / ipv4 / ip_forward
Would we have to add for example #Enable Forwarding
echo 1> / proc / sys / net / ipv6 / ip_forwar? I do not know what may be missing and because I do not see some sites with specific domains,
doing more research I found the topic of intercepting https by means of ssl certificates and I see that the configuration becomes more complex,
I do not want to have to be installing certificates on the computers browsers, tablets and smartphones
I would like the transparent proxy to work as the non-transparent works that can see any site, either http or https,
I hope you can enlighten me and terminate my project, in advance very grateful for your help.
Hi, Let me preface this by
Hi,
Let me preface this by saying that this tutorial was written six years ago or so, so several things changed in the mean time. Most importantly is probably the push for SSL everywhere, which is a good thing. Unfortunately, SSL is inherently incompatible with proxy caching software unless you intercept this traffic, decrypt it, and then re-encrypt with your local LAN certificate. I would say that this is not just inefficient, but also leads to security and privacy issues. Sadly, if you want to use a proxy in this day and age, it's the only option and pretty much mandatory since there relatively few non-SSL sites that would benefit from the proxy cache in the first place.
I think the changes you mention in your post should get you to a working proxy, but I won't manage to verify that for you due to lack of time. If I were you however, I would investigate one of the more recent tutorials on the web that include SSL. It's more complex, but if you take them step by step you should be able to get there.
Johan.