Tuesday, November 2, 2021

Docker + Logs + Fail2Ban: Originating IP addresses

 Fail2Ban is a great tool for securing our servers and relies heavily on the log files of various services. Recently, we've deployed apps using Docker containers which has been an interesting journey, but there was little in the way of documentation on how to get fail2ban working on logs produced from docker containers. So let's start with the problem:

The problem

I managed to get Docker to dump mongoDB logs to the host file system under /var/log using the driver syslog following this reference: https://techroads.org/docker-logging-to-the-local-os-that-works-with-compose-and-rsyslog/

However, any failed login attemps were logged with the local docker IP as follows:

 Nov 3 14:43:07 shiny2 docker-mongodb[18367]: {"t":{"$date":"2021-11-03T03:43:07.203+00:00"},"s":"I", "c":"ACCESS",  "id":20249,  "ctx":"conn10","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"root","authenticationDatabase":"admin","client":"172.18.0.2:61065","result":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}  
You can see that the IP address of 172.18.0.2 is a docker generated IP address and there's no point of having fail2ban blocking 172.18.0.2 from further access to the VM when the originating IP address is something completely different.

So how do we get docker to log the originating IP in the logs?

The solution

There's a little known configuration you can use in the docker-compose.yml that can tell docker to share the same network space as the host rather than trying to virtualise the network environment, shown as follows:

 version: "3.8"  
 services:  
  mongodb:  
  image : mongo  
  container_name: mongodb  
  command: [--auth]  
  environment:  
   - MONGODB_ROOT_PASSWORD=changeme  
   - MONGODB_USERNAME=someuser
   - MONGODB_PASSWORD= changeme  
   - MONGODB_DATABASE=someDB    
  volumes:  
   - /mnt/srv/databases/mongodb:/data/db  
  ports:  
   - 27017:27017  
  restart: unless-stopped  
  # Share the same network space as host to preserve IPs in log files  
  network_mode: host  
  logging:  
   driver: syslog  
   options:  
    tag: docker-mongodb  
 networks:  
  default:  
  external: true  
  name: db-net  
I hope this helps someone save hours of searching like I did!