Goaccess web server analytics

GoAccess is a terminal-based log analyzer. It provided web server statistics in real time. We use it to write to an html file twice daily on order to get a clear idea of what is REALLY going on with the server rather than relying on google analytics. 

Install on web server

echo "deb https://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list;
wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/goaccess.gpg add -;
sudo apt-get update;
sudo apt-get install goaccess;

Configure

Edit the file /etc/goaccess/goaccess.conf and uncomment the following line. Note that these are standards for Apache and NGinx only:

time-format %H:%M:%S

date-format %d/%b/%Y

log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

Run (Apache example)

sudo goaccess -f /var/log/apache2/development-access.log -a >/var/www/mysite.com/public/acreprts/access_report_2021_04_09.html

Run (Nginx example)

sudo goaccess -f /var/log/nginx/access.log -a >/var/www/mysite.com/public/acreprts/access_report_2021_04_09.html

Crons

To set this up to run automatically create a shell script:

#!/bin/bash

# note you will need to install goaccess on the server first. 
# Once installed open the file /etc/goaccess/goaccess.conf and uncomment the following lines

# modify to match path for goaccess output
acreprtsPath="/var/www/mysite.com/public/acreprts"

logsfile="/var/log/apache2/my-website-access.log"

FILE=access_report.html 
FILE=${FILE%.*}-`date +%Y-%m-%d`.${FILE#*.}

goaccess -f $logsfile -a > $acreprtsPath/$FILE 

# fix the permissions
sudo chown $USER:www-data $acreprtsPath/$FILE 
sudo chmod 775 $acreprtsPath/$FILE 

Save as : /var/www/dev-tools/shell-scripts/goacces.sh and set the permissions to executable. Next add the cron as root user with this command:

sudo crontab -e -u root 

and add the following:

# use the following cron to run every 4 hours
# 0 */4 * * * /var/www/dev-tools/shell-scripts/goaccess.sh

Note that once we are on load balancers you will need to use the ip address to access these logs. 

We access these logs by ip add ress so make sure the first apache vhost fle is for mysite.com. That was we can access the live report with https://mysite.com/acreprts/access_report-2021-07-13.html