From 181f172875bec7915dc06de949fc30681156544a Mon Sep 17 00:00:00 2001 From: andryyy Date: Tue, 11 Jul 2017 17:05:29 +0200 Subject: [PATCH] Change logging information --- docs/debug-logs.md | 5 ++- docs/firststeps-logging.md | 58 ++++++++++++++++++++++++ docs/firststeps-syslog.md | 90 -------------------------------------- mkdocs.yml | 2 +- 4 files changed, 63 insertions(+), 92 deletions(-) create mode 100644 docs/firststeps-logging.md delete mode 100644 docs/firststeps-syslog.md diff --git a/docs/debug-logs.md b/docs/debug-logs.md index 1926ce420..7b5eb694d 100644 --- a/docs/debug-logs.md +++ b/docs/debug-logs.md @@ -1,6 +1,9 @@ +!!! warning + This section only applies for Dockers default logging driver (Json). + To view the logs of all mailcow: dockerized related containers, you can use `docker-compose logs` inside your mailcow-dockerized folder that contains your `mailcow.conf`. This is usually a bit much, but you could trim the output with `--tail=100` to the last 100 lines per container, or add a `-f` to follow the live output of all your services. -To view the logs of a specific service you can use `docker-compose logs [options] $Service_Name` +To view the logs of a specific service you can use `docker-compose logs [options] $service_name` !!! info The available options for the command **docker-compose logs** are: diff --git a/docs/firststeps-logging.md b/docs/firststeps-logging.md new file mode 100644 index 000000000..47277861e --- /dev/null +++ b/docs/firststeps-logging.md @@ -0,0 +1,58 @@ +Logging in mailcow: dockerized consists of multiple stages, but is, after all, much more flexible and easier to integrate into a logging daemon than before. + +In Docker the containerized application (PID 1) writes its output to stdout. For real one-application containers this works just fine. + +Some containers log or stream to multiple destinations. + +No container will keep persistent logs in it. Containers are transient items! + +In the end, every line of logs will reach the Docker daemon - unfiltered. + +The **default logging driver is "json"**. + +### Filtered logs + +Some logs are filtered and written to Redis keys but also streamed to a Redis channel. + +The Redis channel is used to stream logs with failed authentication attempts to be read by fail2ban-mailcow. + +The Redis keys are persistent and will keep up to 5000 lines of logs for the web UI. + +This mechanism makes it possible to use whatever Docker logging driver you want to, without losing +the ability to read logs from the UI or ban suspicious clients with fail2ban-mailcow. + +Redis keys will only hold logs from applications and filter out system messages (think of cron etc.). + +### Logging drivers + +Here is the good news: Since Docker has some great logging drivers, you can integrate mailcow: dockerized into your existing logging environment with ease. + +Docker logging drivers can now be implemented as plugins, next to Dockers integrated drivers. +Logging driver plugins are available in Docker 17.05 and higher. + +Edit `docker-compose.yml` and append, for example, this block to use the "gelf" logging plugin: + +``` +logging: + log_driver: "gelf" + options: + gelf-address: "udp://graylog:12201" + gelf-tag: "mailcow-logs" +``` + +Linux users can also add or edit the Docker daemons configuration file `/etc/docker/daemon.json` to affect the global logging behavior. Windows users please have a look at the [docker documentation](https://docs.docker.com/engine/reference/commandline/dockerd//#windows-configuration-file): + +``` +{ +... + "log-driver": "gelf", + "log-opts": { + "gelf-address": "udp://graylog:12201" + "gelf-tag": "mailcow-logs" + } +... +} + +``` + +Restart the Docker daemon and run `docker-compose down && docker-compose up -d` to recreate the containers with the new logging driver. diff --git a/docs/firststeps-syslog.md b/docs/firststeps-syslog.md deleted file mode 100644 index 62675e4e1..000000000 --- a/docs/firststeps-syslog.md +++ /dev/null @@ -1,90 +0,0 @@ -!!! warning - In newer versions of mailcow: dockerized we decided to set a max. log size. You need to remove all "logging: xy" lines and options from docker-compose.yml to be able to start the stack. - - Example: - - ``` - logging: - options: - max-size: "5m" - ``` - -!!! info - If you prefere the udp protocol use: - - ``` - $ModLoad imudp - $UDPServerRun 524 - ``` - - at `rsyslog.conf` and `"syslog-address": "udp://127.0.0.1:524"` at `daemon.json`. - - -Enable Rsyslog to receive logs on 524/tcp at `rsyslog.conf`: - -``` -# This setting depends on your Rsyslog version and configuration format. -# For most Debian derivates it will work like this... -$ModLoad imtcp -$TCPServerAddress 127.0.0.1 -$InputTCPServerRun 524 - -# ...while for Ubuntu 16.04 it looks like this: -module(load="imtcp") -input(type="imtcp" address="127.0.0.1" port="524") - -# No matter your Rsyslog version, you should set this option to off -# if you plan to use Fail2ban -$RepeatedMsgReduction off -``` - -Restart rsyslog after enabling the TCP listener. - -Now setup Docker daemon to start with the syslog driver. -This enables the syslog driver for all containers! - -Linux users can add or change the configuration in `/etc/docker/daemon.json`. Windows users please have a look at the [docker documentation](https://docs.docker.com/engine/reference/commandline/dockerd//#windows-configuration-file) : -``` -{ -... - "log-driver": "syslog", - "log-opts": { - "syslog-address": "tcp://127.0.0.1:524" - } -... -} - -``` - -Restart the Docker daemon and run `docker-compose down && docker-compose up -d` to recreate the containers. - -### Fail2ban with Docker syslog logging driver - -**This only applies to syslog-enabled Docker environments.** - -Open `/etc/fail2ban/filter.d/common.conf` and search for the prefix_line parameter, change it to ".*": - -``` -__prefix_line = .* -``` - -Create `/etc/fail2ban/jail.d/dovecot.conf`... -``` -[dovecot] -enabled = true -filter = dovecot -logpath = /var/log/syslog -chain = FORWARD -``` - -and `jail.d/postfix-sasl.conf`: -``` -[postfix-sasl] -enabled = true -filter = postfix-sasl -logpath = /var/log/syslog -chain = FORWARD -``` - -Restart Fail2ban. - diff --git a/mkdocs.yml b/mkdocs.yml index 7867212b2..c7382c4b3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -30,7 +30,7 @@ pages: - 'Rspamd Web UI': 'firststeps-rspamd_ui.md' - 'Reverse Proxy': 'firststeps-rp.md' - 'Setup a relayhost': 'firststeps-relayhost.md' - - 'Log to Syslog': 'firststeps-syslog.md' + - 'Logging': 'firststeps-logging.md' - 'Local MTA on Docker host': 'firststeps-local_mta.md' - 'Sender and receiver model': 'firststeps-sender_rcv.md' - 'Debugging & Troubleshooting':