Merge pull request #9 from broedli/master

Added new chapters and had fun restructuring

docs/attach_service.md

@@ -0,0 +1,41 @@
+## Attaching a Container to your Shell
+
+To attach a container to your shell you can simply run
+
+```
+docker-compose exec $Service_Name /bin/bash
+```
+
+### Connecting to Services
+
+If you whant to connect to a service / application directly it is always a good idea to `source mailcow.conf` to get all relevant variables in your environment.
+
+#### MySQL
+
+```
+source mailcow.conf
+docker-compose exec mysql-mailcow mysql -u${DBUSER} -p${DBPASS} ${DBNAME}
+```
+
+#### Redis
+
+```
+docker-compose exec redis-mailcow redis-cli
+```
+
+## Service Descriptions
+
+Here is a brief overview of what container / service does what:
+
+| Service Name  | Service Descriptions                                                      |
+| --------------- | ------------------------------------------------------------------------- |
+| bind9-mailcow   | Local (DNSSEC) DNS Resolver                                               |
+| mysql-mailcow   | Stores SOGo's and most of mailcow's settings                                         |
+| postfix-mailcow | Receives and sends mails                                                  |
+| dovecot-mailcow | User logins and sieve filter                                              |
+| redis-mailcow   | Storage backend for DKIM keys, Rmilter and Rspamd                         |
+| rspamd-mailcow  | Mail filtering system. Used for av handling, dkim signing, spam handling  |
+| rmilter-mailcow | Integrates Rspamd into postfix                                            |
+| clamd-mailcow   | Scans attachments for viruses                                             |
+| sogo-mailcow    | Webmail client that handles Microsoft ActiveSync and Cal- / CardDav       |
+| nginx-mailcow   | Nginx remote proxy that handles all mailcow related HTTP / HTTPS requests |

docs/bl_wl.md

@@ -1,5 +1,7 @@
 To add or edit an entry to your **domain wide** filter table, login to your *mailcow UI* as (domain) administrator.
 
-![Black- and Whitelist configuration](images/bl_wl.png)
+![Black- and Whitelist configuration](images/mailcow-bl_wl.png)
 
-Beware that a mailbox user can login to mailcow and override a domain policy filter item.
+
+!!! info
+    Be aware that a user [may override](spamfilter.md) this setting by setting his own black- and whitelist!

docs/debug.md

@@ -1,34 +1,10 @@
-## Logs
+When a problem occurs, then always for a reason! What you want to do in such a case is:
 
-You can use `docker-compose logs $service-name` for all containers.
-
-Run `docker-compose logs` for all logs at once.
-
-Follow the log output by running docker-compose with `logs -f`.
-
-Limit the output by calling logs with `--tail=300` like `docker-compose logs --tail=300 mysql-mailcow`.
-
-## Reset admin password
-Reset mailcow admin to `admin:moohoo`:
-
-```
-cd mailcow_path
-bash mailcow-reset-admin.sh
-```
-
-## What container does what
-
-Here is a brief overview of what container does what:
-
-| Container Name  | Service Descriptions                                                      |
-| --------------- | ------------------------------------------------------------------------- |
-| bind9-mailcow   | Local (DNSSEC) DNS Resolver                                               |
-| mysql-mailcow   | Stores most of mailcow's settings                                         |
-| postfix-mailcow | Receives and sends mails                                                  |
-| dovecot-mailcow | User logins and sieve filter                                              |
-| redis-mailcow   | Storage backend for DKIM keys, Rmilter and Rspamd                         |
-| rspamd-mailcow  | Mail filtering system. Used for av handling, dkim signing, spam handling  |
-| rmilter-mailcow | Integrates Rspamd into postfix                                            |
-| clamd-mailcow   | Scans attachments for viruses                                             |
-| sogo-mailcow    | Webmail client that handles Microsoft ActiveSync and Cal- / CardDav       |
-| nginx-mailcow   | Nginx remote proxy that handles all mailcow related HTTP / HTTPS requests |
+1. Read your logs; follow them to see what the reason for your problem is.
+2. Follow the leads given to you in your logfiles and start investigating.
+3. Restarting the troubled service or the whole stack to see if the problem persists.
+4. Read the documentation of the troubled service and search it's bugtracker for your problem.
+5. Search our [issues](https://github.com/mailcow/mailcow-dockerized/issues) and [forum](https://forum.mailcow.email/) for topics related to your problem.
+6. [Create an issue](https://github.com/mailcow/mailcow-dockerized/issues) over at our GitHub repository if you think your problem might be a bug or a missing feature you badly need. But please make sure, that you include **all the logs** and a full description to your problem.
+7. If you need help with a problem on a configuration or a setup, or got a question please head over to our [forum](https://forum.mailcow.email/) and open a thread.
+8. Connect to IRC ([chat.freenode.net](https://webchat.freenode.net/)) and join our IRC channel [#mailcow](irc://chat.freenode.net:6667/mailcow).

docs/debug_logs.md

@@ -0,0 +1,11 @@
+To view the logs of all mailcow: dockerized related containers, you can use `docker-compose logs` inside your mailcow-dockerized folder that contains your `mailcow.conf`. This is usually a bit mutch but you could trim the output with `--tail=100` to the last 100 lines, or add a `-f` to follow the live output of all your services.
+
+To view the logs of a specific service you can use `docker-compose logs [options] $Service_Name`
+
+!!! info
+    The available options for the command **docker-compose logs** are:
+
+    - **--no-color**: Produce monochrome output.
+    - **-f**: Follow the log output.
+    - **-t**: Show timestamps.
+    - **--tail="all"**: Number of lines to show from the end of the logs for each container.

docs/deinstall.md

@@ -0,0 +1,11 @@
+To remove mailcow: dockerized with all it's volumes, images and containers do:
+
+```
+docker-compose down -v --rmi all --remove-orphans
+```
+
+!!! info
+    - **-v** Remove named volumes declared in the `volumes` section of the Compose file and anonymous volumes attached to containers.
+    - **--rmi <type>** Remove images. Type must be one of: `all`: Remove all images used by any service. `local`: Remove only images that don't have a custom tag set by the `image` field.
+    - **--remove-orphans** Remove containsers for services not defined in the compose file.
+    - By default `docker-compose down` only removes currently active containers and networks devinded in the `docker-compose.yml`.

docs/dns.md

@@ -11,7 +11,7 @@ Below you can find a list of **recommended DNS records**. While some are mandato
 
 ## Reverse DNS of your IP
 
-Make sure that the PTR record of your IP matches the FQDN hostname of your mailcow host: `mail.domain.tld`. This record is usually set at the provider you leased the IP (server) from.
+Make sure that the PTR record of your IP matches the FQDN of your mailcow host: `${MAILCOW_HOSTNAME}` [^1]. This record is usually set at the provider you leased the IP (server) from.
 
 ## The minimal DNS configuration
 
@@ -28,13 +28,13 @@ autoconfig          IN A       1.2.3.4
 
 ## DKIM, SPF and DMARC
 
-In the example DNS zone file snippet below, a simple **SPF** TXT record is used to only allow THIS server (the MX) to send mail for your domain. Every other server is disallowed but able to ("`~all`"). Please refer to [SPF Project](http://www.openspf.org).
+In the example DNS zone file snippet below, a simple **SPF** TXT record is used to only allow THIS server (the MX) to send mail for your domain. Every other server is disallowed but able to ("`~all`"). Please refer to [SPF Project](http://www.openspf.org) for further reading.
 
 ```
 @                   IN TXT     "v=spf1 mx ~all"
 ```
 
-It is highly recommended to create a **DKIM** TXT record in your mailcow UI and set the corresponding TXT record in your DNS records. Please refer to [OpenDKIM](http://www.opendkim.org).
+It is highly recommended to create a **DKIM** TXT record in your mailcow UI and set the corresponding TXT record in your DNS records. Please refer to [OpenDKIM](http://www.opendkim.org) for further reading.
 
 ```
 default._domainkey  IN TXT     "v=DKIM1; k=rsa; t=s; s=email; p=..."
@@ -76,3 +76,5 @@ If you are interested in statistics, you can additionally register with the [Pos
 ```
 @                   IN TXT     "google-site-verification=..."
 ```
+
+[^1]: A **Fully Qualified Domain Name** (**FQDN**) is the complete (absolute) domain name for a specific computer or host, on the Internet. The FQDN consists of at least three parts devided by a dot: the hostname (myhost), the domain name (mydomain) and the toplevel domain in short **tld** (com). In the example of `mx.mailcow.email` the hostname would be `mx`, the domain name 'mailcow' and the tld `email`.

docs/mc14_migrate.md

@@ -62,11 +62,9 @@ mysqldump --replace --no-create-info --default-character-set=utf8mb4 \
 ```
 
 !!! Info
-    **--replace**: Write `REPLACE` statements rather than `INSERT` statements
-
-    **--no-create-info**: Don't write `CREATE TABLE` statements.
-    
-    **--default-character-set** make sure our exported default charset is *utf8mb4*.
+    - **--replace**: Write `REPLACE` statements rather than `INSERT` statements
+    - **--no-create-info**: Don't write `CREATE TABLE` statements.
+    - **--default-character-set** make sure our exported default charset is *utf8mb4*.
 
 
 ## Prepare mailcow: dockerized

docs/mysql.md

@@ -1,10 +1,7 @@
-### Connect
-```
-source mailcow.conf
-docker-compose exec mysql-mailcow mysql -u${DBUSER} -p${DBPASS} ${DBNAME}
-```
+MySQL is used to store the settings and / or usertables of the whole mail-stack (mailcow UI, SOGo, dovecot, postfix).
+
+## Backup
 
-### Backup
 ```
 cd /path/to/mailcow-dockerized
 source mailcow.conf
@@ -12,7 +9,7 @@ DATE=$(date +"%Y%m%d_%H%M%S")
 docker-compose exec mysql-mailcow mysqldump --default-character-set=utf8mb4 -u${DBUSER} -p${DBPASS} ${DBNAME} > backup_${DBNAME}_${DATE}.sql
 ```
 
-### Restore
+## Restore
 
 You should redirect the sql dump without Docker-Compose to prevent parsing errors.
 
@@ -21,48 +18,3 @@ cd /path/to/mailcow-dockerized
 source mailcow.conf
 docker exec -i $(docker-compose ps -q mysql-mailcow) mysql -u${DBUSER} -p${DBPASS} ${DBNAME} < backup_file.sql
 ```
-
-### Reset MySQL passwords
-
-Stop the stack by running `docker-compose stop`.
-
-When the containers came to a stop, run this command:
-
-```
-docker-compose run --rm --entrypoint '/bin/sh -c "gosu mysql mysqld --skip-grant-tables & sleep 10 && mysql -hlocalhost -uroot && exit 0"' mysql-mailcow
-```
-
-**1\. Find database name**
-
-```
-MariaDB [(none)]> show databases;
-+--------------------+
-| Database           |
-+--------------------+
-| information_schema |
-| mailcow_database   | <=====
-| mysql              |
-| performance_schema |
-+--------------------+
-4 rows in set (0.00 sec)
-```
-
-**2\. Reset one or more users**
-
-Both "password" and "authentication_string" exist. Currently "password" is used, but better set both.
-
-```
-MariaDB [(none)]> SELECT user FROM mysql.user;
-+--------------+
-| user         |
-+--------------+
-| mailcow_user | <=====
-| root         |
-+--------------+
-2 rows in set (0.00 sec)
-
-MariaDB [(none)]> FLUSH PRIVILEGES;
-MariaDB [(none)]> UPDATE mysql.user SET authentication_string = PASSWORD('gotr00t'), password = PASSWORD('gotr00t') WHERE User = 'root' AND Host = '%';
-MariaDB [(none)]> UPDATE mysql.user SET authentication_string = PASSWORD('mookuh'), password = PASSWORD('mookuh') WHERE User = 'mailcow' AND Host = '%';
-MariaDB [(none)]> FLUSH PRIVILEGES;
-```

docs/redis.md

@@ -1,15 +1,45 @@
-### Client
+Redis is used as a key-value store for (some of) mailcow's and rspamd's settings and data. If you are unfamilliar with redis please read the [introduction to redis](https://redis.io/topics/introduction) and maybe visit this [wonderfull guide](http://try.redis.io/) on how to use it.
+
+## Client
+
+To connect to the redis cli execute:
 
 ```
 docker-compose exec redis-mailcow redis-cli
 ```
 
-## Remove persistent data
+### Debugging
 
-- Remove volume `mysql-vol-1` to remove all MySQL data.
-- Remove volume `redis-vol-1` to remove all Redis data.
-- Remove volume `vmail-vol-1` to remove all contents of `/var/vmail` mounted to `dovecot-mailcow`.
-- Remove volume `dkim-vol-1` to remove all DKIM keys.
-- Remove volume `rspamd-vol-1` to remove all Rspamd data.
+Here are some usefull commands for the redis-cli for debugging:
 
-Running `docker-compose down -v` will **destroy all mailcow: dockerized volumes** and delete any related containers.
+##### MONITOR
+
+Listens for all requests received by the server in real time:
+
+```
+# docker-compose exec redis-mailcow redis-cli
+127.0.0.1:6379> monitor
+OK
+1494077286.401963 [0 172.22.1.253:41228] "SMEMBERS" "BAYES_SPAM_keys"
+1494077288.292970 [0 172.22.1.253:41229] "SMEMBERS" "BAYES_SPAM_keys"
+[...]
+```
+
+##### KEYS
+
+Get all keys matching your pattern:
+
+```
+
+```
+
+##### PING
+
+Test a connection:
+
+```
+127.0.0.1:6379> PING
+PONG
+```
+
+If you want to know more, here is a [cheat sheet](https://www.cheatography.com/tasjaevan/cheat-sheets/redis/).

docs/relayhost.md

@@ -1,4 +1,4 @@
-Insert these lines to `data/conf/postfix/main.cf`. "relayhost" does already exist (empty), just change its value.
+Insert these lines into `data/conf/postfix/main.cf`. "relayhost" does already exist (empty), just change its value.
 ```
 relayhost = [your-relayhost]:587
 smtp_sasl_password_maps = hash:/opt/postfix/conf/smarthost_passwd

docs/reset_pw.md

@@ -0,0 +1,59 @@
+## Mailcow Admin Account
+
+Reset mailcow admin to `admin:moohoo`:
+
+```
+cd mailcow_path
+bash mailcow-reset-admin.sh
+```
+
+## Remove Two-Factor Authentication
+
+```
+
+```
+
+## Reset MySQL Passwords
+
+Stop the stack by running `docker-compose stop`.
+
+When the containers came to a stop, run this command:
+
+```
+docker-compose run --rm --entrypoint '/bin/sh -c "gosu mysql mysqld --skip-grant-tables & sleep 10 && mysql -hlocalhost -uroot && exit 0"' mysql-mailcow
+```
+
+### 1\. Find database name
+
+```
+MariaDB [(none)]> show databases;
++--------------------+
+| Database           |
++--------------------+
+| information_schema |
+| mailcow_database   | <=====
+| mysql              |
+| performance_schema |
++--------------------+
+4 rows in set (0.00 sec)
+```
+
+### 2\. Reset one or more users
+
+Both "password" and "authentication_string" exist. Currently "password" is used, but better set both.
+
+```
+MariaDB [(none)]> SELECT user FROM mysql.user;
++--------------+
+| user         |
++--------------+
+| mailcow_user | <=====
+| root         |
++--------------+
+2 rows in set (0.00 sec)
+
+MariaDB [(none)]> FLUSH PRIVILEGES;
+MariaDB [(none)]> UPDATE mysql.user SET authentication_string = PASSWORD('gotr00t'), password = PASSWORD('gotr00t') WHERE User = 'root' AND Host = '%';
+MariaDB [(none)]> UPDATE mysql.user SET authentication_string = PASSWORD('mookuh'), password = PASSWORD('mookuh') WHERE User = 'mailcow' AND Host = '%';
+MariaDB [(none)]> FLUSH PRIVILEGES;
+```

docs/rm_volumes.md

@@ -0,0 +1,13 @@
+You may want to remove a set of persistend data to resolve a conflict or to start over:
+
+```
+docker volume rm mailcowdockerized_${VOLUME_NAME}
+```
+
+- Remove volume `mysql-vol-1` to remove all MySQL data.
+- Remove volume `redis-vol-1` to remove all Redis data.
+- Remove volume `vmail-vol-1` to remove all contents of `/var/vmail` mounted to `dovecot-mailcow`.
+- Remove volume `dkim-vol-1` to remove all DKIM keys.
+- Remove volume `rspamd-vol-1` to remove all Rspamd data.
+
+Running `docker-compose down -v` will **destroy all mailcow: dockerized volumes** and delete any related containers and networks.

docs/rspamd.md

@@ -1,4 +1,6 @@
-### Learn spam and ham
+Rspamd is used for av handling, dkim signing and spam handling. It's a powerfull and fast filter system. For a more in-depth documentation on Rspamd please visit it's [own documentation](https://rspamd.com/doc/index.html).
+
+## Learn Spam & Ham
 
 Rspamd learns mail as spam or ham when you move a message in or out of the junk folder to any mailbox besides trash.
 This is archived by using the Dovecot plugin "antispam" and a simple parser script.
@@ -9,7 +11,7 @@ The bayes statistics are written to Redis as keys `BAYES_HAM` and `BAYES_SPAM`.
 
 You can also use Rspamd's web ui to learn ham and/or spam.
 
-### Learn ham or spam from existing directory
+### Learn Spam or Ham from existing directory
 
 You can use a one-liner to learn mail in plain-text (uncompressed) format:
 ```
@@ -25,7 +27,7 @@ Consider attaching a local folder as new volume to `rspamd-mailcow` in `docker-c
 for file in /data/old_mail/.Junk/cur/*; do rspamc learn_spam < zcat $file; done
 ```
 
-### CLI tools
+## CLI tools
 
 ```
 docker-compose exec rspamd-mailcow rspamc --help
@@ -33,4 +35,3 @@ docker-compose exec rspamd-mailcow rspamadm --help
 ```
 
 See [Rspamd documentation](https://rspamd.com/doc/index.html)
-

docs/spamalias.md

@@ -0,0 +1,5 @@
+These temporary email aliases are mostly used for places where we need to provide an email address but don't want future correspondence with. They are also called spam alias.
+
+To create, delete or extend a temporary email aliase you need to login to mailcow's UI as a mailbox user and navigate to the tab **Temporary email aliases**:
+
+![How to set spam- or temporary email aliases in mailcow](images/mailcow-spamalias.png)

docs/spamfilter.md

@@ -0,0 +1,8 @@
+A mailbox user may adjust the spam filter and black- / whitelist settings for his mailbox individually by navigating to the **Spam filter** tab in the users mailcow UI.
+
+![Where to adjust the users spam, black- and whitelist settings](images/mailcow-spamfilter.png)
+
+!!! info
+    For global adjustments on your spam filter please check our section on [Rspamd](rspamd.md).
+
+    For a domain wide black- and whitelist please check our guide on [Black / Whitelist](bl_wl.md)

docs/ssl.md

@@ -34,7 +34,8 @@ certbot certonly \
     --agree-tos
 ```
 
-**Remember to replace the example.org domain with your own domain, this command will not work if you dont.**
+!!! warning
+    Remember to replace the example.org domain with your own domain, this command will not work if you dont.
 
 4\. Create hard links to the full path of the new certificates. Assuming you are still in the mailcow root folder:
 ``` bash

docs/syslog.md

@@ -28,11 +28,13 @@ DOCKER_OPTS="--log-driver=syslog --log-opt syslog-address=tcp://127.0.0.1:524"
 ...
 ```
 
-**Caution:** For some reason Ubuntu 16.04 and some, but not all, systemd based distros do not read the defaults file parameters.
+!!! warning
+    For some reason Ubuntu 16.04 and some, but not all, systemd based distros do not read the defaults file parameters.
 
 Just run `systemctl edit docker.service` and add the following content to fix it.
 
-**Note:** If "systemctl edit" is not available, just copy the content to `/etc/systemd/system/docker.service.d/override.conf`.
+!!! info
+    If "systemctl edit" is not available, just copy the content to `/etc/systemd/system/docker.service.d/override.conf`.
 
 The first empty ExecStart parameter is not a mistake.
 

docs/tagging.md

@@ -1,6 +1,8 @@
 Mailbox users can tag their mail address like in `me+facebook@example.org`. They can control the taghandling in the users **mailcow UI** panel.
 
-![mailcow mail tagging settings](images/tagging.png)
+![mailcow mail tagging settings](images/mailcow-tagging.png)
+
+### Available Actions
 
 1\. Move this message to a subfolder "facebook" (will be created lower case if not existing)
 

docs/tfa.md

@@ -2,7 +2,6 @@ So far three methods for *Two-Factor Authentication* are implemented: U2F, Yubi
 
 - For U2F to work, you need an encrypted connection to the server (HTTPS) as well as a FIDO security key.
 - Both U2F and Yubi OTP work well with the fantastic [Yubikey](https://www.yubico.com).
-<!-- @andryyy, ich bin mir nicht sicher, was du mit diesem Absatz sagen möchtest, Yubi OTP oder U2F? -->
 - While Yubi OTP needs an active internet connection and an API ID + key, U2F will work with any FIDO U2F USB key out of the box, but can only be used when mailcow is accessed over HTTPS.
 - U2F and Yubi OTP support multiple keys per user.
 - As the third TFA method mailcow uses TOTP: time-based one-time passwords. Those psaswords can be generated with apps like "Google Authenticator" after initially scanning a QR code or entering the given secret manually.

mkdocs.yml

@@ -10,6 +10,7 @@ markdown_extensions:
   - pymdownx.tasklist(custom_checkbox=true)
   - pymdownx.mark
   - pymdownx.tilde
+  - footnotes
 pages:
 - 'Information & Support': 'index.md'
 - 'Prerequisites':
@@ -27,24 +28,35 @@ pages:
   - 'Log to Syslog': 'syslog.md'
   - 'Local MTA on Docker host': 'local_mta.md'
   - 'Sender and receiver model': 'sender_rcv.md'
+- 'Debugging & Troubleshooting':
+    - 'Introduction': debug.md
+    - 'Logs': 'debug_logs.md'
+    - 'Attach a Container': 'attach_service.md'
+    - 'Reset Passwords': 'reset_pw.md'
+    - 'Remove Persistent Data': 'rm_volumes.md'
 - 'Usage & Examples':
-  - 'Debugging & Troubleshooting': 'debug.md'
-  - 'mailcow UI Configuration': 'mailcow_ui.md'
+  - 'mailcow UI':
+      - 'Configuration': 'mailcow_ui.md'
+      - 'Blacklist / Whitelist': 'bl_wl.md'
+      - 'Spamfilter': 'spamfilter.md'
+      - 'Temporary email aliase': 'spamalias.md'
+      - 'Tagging': 'tagging.md'
+      - 'Two-Factor Authentication': 'tfa.md'
   - 'Redis': 'redis.md'
-  - 'MySQL': 'mysql.md'
   - 'Rspamd': 'rspamd.md'
+  - 'Postfix':
+      - 'Anonymize Headers': 'anonym_headers.md'
+      - 'Disable Sender Addresses Verification': 'disable_sender_verification.md'
   - 'Why bind9?': 'why_bind9.md'
+  - 'Backup & Restore':
+      - 'Backup Maildir': 'backup_maildir.md'
+      - 'MySQL': 'mysql.md'
+  - 'Autodiscover / Autoconfig': 'autodiscover_config.md'
+  - 'Redirect HTTP to HTTPS': '80_to_443.md'
   - 'Adjust Service Configurations': 'change_config.md'
   - 'Customize Dockerfiles': 'cust_dockerfiles.md'
   - 'Docker Compose Bash Completion': 'dc_bash_compl.md'
-  - 'Backup Maildir': 'backup_maildir.md'
-  - 'Two-Factor Authentication': 'tfa.md'
-  - 'Redirect HTTP to HTTPS': '80_to_443.md'
-  - 'Anonymize Headers': 'anonym_headers.md'
-  - 'Tagging': 'tagging.md'
-  - 'Blacklist / Whitelist': 'bl_wl.md'
-  - 'Autodiscover / Autoconfig': 'autodiscover_config.md'
-  - 'Disable Sender Addresses Verification': 'disable_sender_verification.md'
+  - 'Deinstall': 'deinstall.md'
 - 'Third party apps':
   - 'Roundcube': 'roundcube.md'
   - 'Portainer': 'portainer.md'