Using a Raspberry Pi as a Squid proxy cache – updated for 2018

I was looking to build *something* that would possibly benefit several customers that have slow or laggy internet access (ie 2mb broadband or satellite).

The solution needed to be cost effective and unobtrusive.

I did some research and decided to use Squid.

Squid (amongst other things) can cache web objects (such as images and executables), speeding up page load times and download times.

 

Next, I needed some hardware to run Squid on.

Squid can run on Linux and Windows computers, but i ruled out Windows as thats a paid licenced product, thus not cost effective.  I also didnt want to install Squid on a desktop computer, i’d either have to buy a new computer, or re-purpose an old computer – (which can be noisy and unsightly). Both options would also consume significant amounts of energy per year, again not very cost effective.

 

Enter the Raspberry Pi!

pi pic

I decided on the Raspberry Pi (in this case a Pi3 – Pi 3B+ now available).  It fitted my requirements exactly.  Low inital outlay, low running annual costs, no OS licence fee, small and quiet!

For this guide, you will need:

  • Raspberry Pi 2, 3 or 3b+
  • 16GB micro SD card minimum (faster the better)
  • 5v Micro USB charger
  • CAT5e cable connected to the Pi and your Router
  • Temporary use of a usb keyboard and mouse
  • Temporary use of a HDMI TV / Monitor

Once you have completed the inital network configuration, you can connect to your Raspberry Pi via SSH using Putty (default login for the Raspberry Pi is pi and raspberry).

So, you’ve got your Pi plugged in, and you’ve installed the latest version of Rasperian OS (if you’re new to Linux and the Pi family, download the NOOBs installer!), now its time to get to work.

 

NOTE:  I’m not going to show you how to cache encrypted SSL traffic (port 443). This guide will show you how to cache non encrypted (80) traffic only!

 

Configure a static IP on the Pi.

Remember, unlike Windows, Linux commands are case sensitve!

This guide is based on Raspbian Jessie!

Open a shell window and type:

sudo nano /etc/dhcpcd.conf

Scroll to the end of doc and enter (use a static ip from your subnet, and change the router address to that of your own router).

# Set a static ip address
interface eth0
static ip_address=192.168.5.250/24        
static routers=192.168.5.254
static domain_name_servers=208.67.222.222 208.67.220.220


Press Ctrl + X to quit. Hit Y to save

dhcp

Reboot the Pi for the changes to take effect

At the shell window type:

sudo reboot

Tweaking the Pi:

Open a new shell window and enter:

sudo raspi-config

Option 1, expand filesystem

Option 9, advanced config. A3 memory split. 8.

Option 3. Boot options – select boot to CLI.

If your running a Raspberry Pi2, you’ll be able to overclock it as well!

Option 5. Make sure SSH is enabled.

Finish / Exit

For the changes to take effect, type:

sudo reboot

At this stage , you will be able to SSH onto your Raspberry Pi using Putty.  This will let you remove the keyboard / mouse + monitor, and let you copy & paste the remaining commands if you want.

Time to update:

Next, we want to update our Raspberry Pi with the latest patches.  Open a new shell window

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

Hit Y if prompted to reboot.

Now we have a fully patched and up-to-date version of Rasparian; its time to install Squid.

Install Squid:

Enter the following in the shell

sudo apt-get install squid

Configuring Squid:

Backup the original Squid config file:

sudo cp /etc/squid/squid.conf /etc/squid/squidoriginal.conf.bak

Edit the config file:

sudo nano /etc/squid/squid.conf

use Ctrl + W to find each section:

http_access allow localnet = remove the # symbol

squid conf 1.png

Find: acl localnet section

add the following:
acl localnet src YOUR CIDR IP RANGE # Description

ie: 

acl localnet src 192.168.5.0/24 # Home Network

Make sure the ip range/cidr matches your networks range

squid conf 2.png

Find: # dns_v4_first off remove the # symbol and change off to on.

squid conf 3.png

Cache_mem 256 MB

squid conf 4.png

Maximum_object_size 4096 MB

squid conf 5.png

Maximum_object_size_in_memory 8192 KB
squid conf 6

Cache_dir ufs /var/spool/squid3 = 8192 (1st variable - this is 8192 MB)
squid conf 7.png
Ctrl + X and Y to save & exit.

Backup your altered squid config file and restart the Squid service:

sudo cp /etc/squid/squid.conf /etc/squid/mysquid.conf.bak
sudo service squid restart

squid conf 8.png

 

Make managing Squid easier with Webmin:

First, install webmins prereqs; open a shell and enter:

sudo apt-get -f install
sudo apt-get -y install apache2 apache2-suexec-custom libnet-ssleay-perl libauthen-pam-perl libio-pty-perl apt-show-versions samba bind9 webalizer locate mysql-server
sudo apt-get install squid-cgi

Enter a secure password for MySQL when prompted.:

mysql1.png

From the shell enter these commands in turn:

cd
check your current path should read as /home/pi
pwd
sudo mkdir installed-packages
cd installed-packages
Download the Webmin interface package:
sudo wget http://www.webmin.com/download/deb/webmin-current.deb
Install Webmin:
sudo dpkg -i webmin-current.deb

 

Once Webmin has been installed; open a browser on your pc https://192.168.5.250:10000

Login using the raspberry pi login (default is pi and raspberry).

webmin1

In webmin; you’ll be able to adjust Squid settings through webmin.  Look under servers; Squid proxy server.

At this stage, its highly likely your webmin config isn’t configured for Squid Pi (should be fine if your following this guide on a Linux ‘Intel / AMD’ PC…

You’ll find SQUID hidden under the In-used modules menu.

webmin2

Click on the edit config button, change squid3 to squid where highlighted.

Hit save, then hit the orange apply config button. After a minute of so, the Squid services will be restarted and Webmin will work.

squid changes for

If you have an error relating to the cache manager statistic icon, ssh back onto the Pi, use sudo nano to edit the config file, make the required change and save the file. You may need to reboot the pi.

Configuring the client:

Set windows browser proxy: Enter the ip address of the Raspberry Pi (192.168.5.250) and port 3128. Restart browser.

client proxy 1.png

Clear your browser cache and restart the browser.  You should now be using the Squid Proxy server on your Raspberry Pi.

 

Check the cache log:

To check the squid cache logs, open a new shell window and enter:

sudo tail -f /var/log/squid/access.log

Hits are items being pulled from the Squid Cache rather than the internet.

hit log.png

Summary:

If your unlucky enough to have a slow or laggy internet connection,  one possible solution for you is to build and test a Squid proxy server.  However, bear in mind, your mileage may vary as not all objects are cacheable, and certainly any improvement is less noticiable on fast internet connections such as BT infinity.

I performed some “not very scientific” tests using OpenOffice.org.  I found that the download speed of the OpenOffice installer on the first try was 3.9mbs, jumping to 7.9Mb/s after caching once, then maxing out at 9.8Mb/s on the second and subsequent runs (likely a limitation of the Raspberry Pi’s network card – which is limited to 100mbs – UPDATE: Raspberry Pi3B+ has a much faster NIC card).

Sticker.png

 

tsn signoff

Unlimited business broadband from £15.99 a month

 

Advertisements

4 thoughts on “Using a Raspberry Pi as a Squid proxy cache – updated for 2018

  1. Static IP should be configured via /etc/network/interfaces!

    Example
    nano /etc/network/interfaces if root or sudo nano /etc/network/interfaces

    change ‘iface eth0 inet dhcp’ to ‘iface eth0 inet static’ then add configuration for IP, netmask, gateway and DNS as per example below.

    iface eth0 inet static
    address 192.168.3.3
    netmask 255.255.255.0
    gateway 192.168.3.1
    dns-search example.com sales.example.com dev.example.com
    dns-nameservers 192.168.3.45 192.168.8.10

    • Hi Shaun,

      Thank you for the feedback.

      The method you’ve posted isn’t valid in Raspbian Jessie (You are correct in that it is valid in other Linux distros, even the previous Raspbian build – Wheezy).

      If you edit the interfaces config in Jessie using: sudo nano etc/network/interfaces
      your informed that the config has moved to /etc/dhcpcd.conf instead.

      If your still running Wheezy, or another Linux distro; this is the equivalent method of setting up static ip:

      sudo nano /etc/network/interfaces

      Remove the line that reads: iface eth0 inet dhcp

      Add the following:

      iface eth0 inet static
      address 192.168.5.250
      netmask 255.255.255.0
      network 192.168.5.0
      broadcast 192.168.5.255
      gateway 192.168.5.254

      Check DNS:
      sudo nano /etc/resolv.conf
      set to: 208.67.222.222, 208.67.220.220

      Reboot the Pi using: sudo reboot

    • Hi. No, only HTTP. I’ve not looked into HTTPS but you would need to have certificates installed on the Squid Proxy, decrypt the HTTPS traffic, inspect and cache it, then re encrypt using your Squid certificate.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s