Docker LEMP Stack: Get Your Logs With Rsyslog

Wed, Feb 21, 2018
4-minute read

Warning: This post is over 365 days old. The information may be out of date.

In this post, i will explain howto collect your logs when you have a LEMP stack with Docker and rsyslog.

PHP: log to stdout

Nginx use php-fpm and in order to have PHP to log to stdout, you will have to change two parameters. My examples is based on a Debian Stretch installation, so files locations can change for you.

Change in /etc/php/7.0/fpm/php-fpm.conf :
```
error_log = /proc/self/fd/2
```
Change in /etc/php/7.0/fpm/pool.d/www.conf:
```
access.log = /proc/self/fd/2
```

Now restart your PHP container and you will see that all your PHP logs will display on your terminal.

Hints: if you use the official PHP docker image, no needs to change these parameters. They are already set

Nginx : log to stdout

The changes are more easy here.

Change in /etc/nginx/nginx.conf:

access_log /dev/stdout;
error_log /dev/stderr;

MariaDB: I will skip this step since the official Docker image already logs everything to stdout.

Docker : use the syslog driver

Now that all your containers log to stdout, we will tell Docker to send them to syslog. For this, you can add these lines to your docker-compose file:

	services:
	  web:
		image: "customer/nginx"
		container_name: web
		networks:
			- core
		ports:
      			- "80:80"
    		volumes:
      			- ./src/:/var/www/
    		logging:
      			driver: syslog
      			options:
        		 	syslog-address: "tcp://x.x.x.x:514"
        			syslog-format: rfc3164
        			tag: web

The import part is the last 6 lines :

driver: syslog : tell Docker which driver to use
syslog-address : put the address and port of your syslog server
syslog-format : the syslog message format to use
tag : A string that is appended to ‘APP-NAME’ in the syslog message. It’s important because we will use it to sort messages

As usual the Docker documentation is usefull.

We will do the same for our PHP container:

    logging:
      driver: syslog
      options:
        syslog-address: "tcp://127.0.0.1:514"
        syslog-format: rfc3164
        tag: php

For the MariaDB container :

    logging:
      driver: syslog
      options:
        syslog-address: "tcp://127.0.0.1:514"
        syslog-format: rfc3164
        tag: db

Configure rsyslog

Now all your container’s logs will go to syslog. That’s cool but we will probably sort them by type.

We will store our logs in ‘/var/log/docker/xxx’

Here is the rsyslog configuration file :

# Files locations
$template webaccess,"/var/log/docker/web/access/%timereported:0:10:date-rfc3339%.log"
$template weberror,"/var/log/docker/web/error/%timereported:0:10:date-rfc3339%.log"
$template php,"/var/log/docker/php/access/%timereported:0:10:date-rfc3339%.log"
$template phperror,"/var/log/docker/php/error/%timereported:0:10:date-rfc3339%.log"
$template mysql,"/var/log/docker/mysql/%timereported:0:10:date-rfc3339%.log"

# Nginx
if $programname == "web" and $syslogseverity-text == "info" then ?webaccess
& stop

if $programname == "web" and $syslogseverity-text == "error" then ?weberror
& stop

# PHP
if $programname == "php" and $syslogseverity-text == "info" then ?php
& stop
if $programname == "php" and $syslogseverity-text == "error" then ?phperror
& stop

# MySQL
if $programname == "db" then ?mysql
& stop

In the first part, I declare the files locations and their formats:

$template webaccess,"/var/log/docker/web/access/%timereported:0:10:date-rfc3339%.log"
$template weberror,"/var/log/docker/web/error/%timereported:0:10:date-rfc3339%.log"
$template php,"/var/log/docker/php/access/%timereported:0:10:date-rfc3339%.log"
$template phperror,"/var/log/docker/php/error/%timereported:0:10:date-rfc3339%.log"
$template mysql,"/var/log/docker/mysql/%timereported:0:10:date-rfc3339%.log"

More informations in the rsyslog documentation

The next step is to tell rsyslog how to filter the logs:

if $programname == "web" and $syslogseverity-text == "info" then ?webaccess
& stop

The syntax is clear and you can see that that’s here that we use the syslog tag defined in our docker-compose file. By default the tag (if not specified) is the container ID.. so it’s not very usefull because we cannot know it by advance.

Start your stack and get your logs

$ docker-compose up
Creating network "test_default" with the default driver
Creating network "test_core" with driver "bridge"
Creating db ... 
Creating php71 ... 
Creating php71
Creating php71 ... done
Creating web ... 
Creating web ... done
Attaching to db, php71, web
db      | WARNING: no logs are available with the 'syslog' log driver
php71   | WARNING: no logs are available with the 'syslog' log driver
web     | WARNING: no logs are available with the 'syslog' log driver

Go to the ‘/var/log/docker’ directory and check your logs:

$ cd /var/log/docker
$ ls -hlR
.:
total 12K
drwxr-xr-x 2 root root 4.0K Feb 23 15:23 mysql
drwxr-xr-x 4 root root 4.0K Feb 23 14:09 php
drwxr-xr-x 3 root root 4.0K Feb 23 14:02 web

./mysql:
total 24K
-rw-r+++-- 1 root adm 21K Feb 23 15:15 2018-02-23.log

./php:
total 8.0K
drwxr-xr-x 2 root root 4.0K Feb 23 15:23 access
drwxr-xr-x 2 root root 4.0K Feb 23 14:09 error

./php/access:
total 0

./php/error:
total 4.0K
-rw-r+++-- 1 root adm 561 Feb 23 15:15 2018-02-23.log

./web:
total 4.0K
drwxr-xr-x 2 root root 4.0K Feb 23 14:02 access

./web/access:
total 4.0K
-rw-r+++-- 1 root adm 820 Feb 23 14:09 2018-02-23.log

Everything is fine (not for me because I have a php error file).

Now you can use your LEMP stack and check your logs if you have some problems.

Enjoy 😉

Docker Rsyslog