Earlier this year I decided to start using the IPv6 connection that my hosting provider has been providing for the last few years. It had been something I'd been meaning to do for a while and for some reason it felt like the right time to look at it. As it turned out, it was as simple as adding a few lines to various configuration files and all was well.
I wasn't sure how quickly I would see any IPv6 traffic, but as soon as I had made the changes it started flowing. Since the change around 80% of the IMAP traffic I see is via IPv6 and a lot of SMTP connections are also using it.
The one app that didn't seem to be interested in IPv6 was fail2ban. The version I was using was too old so after some looking at various posts I upgraded and enabled the IPv6 support. Initial results weren't encouraging as the upgrade also meant a lot of work to get the configuration I had created working again. Once 100% working again there was a noticeable lack of IPv6 bans taking place, though there have been a few.
After using fail2ban for many years, it did make me wonder if there was anything else that might be a better replacement. A few searches revealed nothing at that time. However, a few days ago I saw a post about Crowdsec. There is a good explanation at https://danielmiessler.com/study/crowdsec/
Written in Go (a big plus from my point of view), it appeared to have a lot of current thinking around privacy embodied together with a very flexible approach. As it could be run alongside fail2ban I decided to install it and start experimenting.
Installation
After following the instructions on the website I had a running instance of crowdsec and could see it examining log entries. It was as straightforward as it could have been.
Configuration
One problem with the wizard approach to installation and configuration is that it then leaves you without an obvious answer to the "what now" question. Like most server installs I have logs that need to be monitored in non-standard locations, so how do I add them? How do I add monitoring for apps that aren't included in the provided options?
acquis.yaml
The answer to including additional logfiles was /etc/crowdsec/config/acquis.yaml
#Generated acquisition file - wizard.sh (service: sshd) / files : /var/log/auth.log
filenames:
- /var/log/auth.log
labels:
type: syslog
---
#Generated acquisition file - wizard.sh (service: mysql) / files : /var/log/mysql/error.log
filenames:
- /var/log/mysql/error.log
labels:
type: mysql
---
#Generated acquisition file - wizard.sh (service: linux) / files : /var/log/syslog /var/log/kern.log
filenames:
- /var/log/syslog
- /var/log/kern.log
labels:
type: syslog
---It's just a simple yaml file, so adding in additional files looked simple enough. Looking at the output from cscli metrics shows how the acquisitions have been processed.
INFO[0000] Acquisition Metrics:
+------------------------------+------------+--------------+----------------+-
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED |
+------------------------------+------------+--------------+----------------+-
| /var/log/auth.log | 6032 | 2233 | 3799 |
| /var/log/kern.log | 49 | 49 | - |
| /var/log/mail.log | 4938 | 267 | 4671 |
| /var/log/mysql/error.log | 21 | - | 21 |
| /var/log/syslog | 5460 | 316 | 5144 |Getting the correct labels was the only part that gave pause, but after looking at how the files are parsed it became clearer.
cat /etc/crowdsec/config/parsers/s00-raw/syslog-logs.yaml
#If it's syslog, we are going to extract progname from it
filter: "evt.Line.Labels.type == 'syslog'"
onsuccess: next_stage
name: crowdsecurity/syslog-logs
grok:
#this is a named regular expression. grok patterns can be kept into separate files for readability
name: "SYSLOGLINE"
#This is the field of the `Event` to which the regexp should be applied
apply_on: Line.Raw
#if the node was successfull, statics will be applied.
statics:
- parsed: "logsource"
value: "syslog"
# syslog date can be in two different fields (one of hte assignment will fail)
- target: evt.StrTime
expression: evt.Parsed.timestamp
- target: evt.StrTime
expression: evt.Parsed.timestamp8601
---
#if it's not syslog, the type is the progname
filter: "evt.Line.Labels.type != 'syslog'"
onsuccess: next_stage
name: crowdsecurity/non-syslog
#debug: true
statics:
- parsed: message
expression: evt.Line.Raw
- parsed: program
expression: evt.Line.Labels.type
---Adding in any other type of log simply needs either a type, which can be referenced later. Every line from a logfile will be passed as a types.Event object to the parsers. The event object has the attributes specified under the statics section before being passed on.
Parsers
Each parser is essentially just a yaml file with a given syntax. The various stages a log entry can pass through are
- s00-raw - presently this just does syslog or non-syslog and every message seems to pass through this layer
- s01-parsers - application specific parsers that look for events of interest.
- s02-enrich - parsers that can add information.
The syntax is very similar and simple enough to follow.
It's worth noting that the filter entries need to be correct or nothing will be parsed! The postfix-logs.yaml parser filter line wasn't correct for my configuration so initially I had no postfix messages being parsed. It took me a while to figure out why, so hopefully this will help others. Once I adjusted the filter entry it started parsing entries and all was well.
Scenarios
Once messages are parsed they are then filtered into buckets/scenarios. These are, once again, simple yaml files that contain details of which entries they are interested in and how long bans should last for.
The system is elegant and quite adaptable.
Dovecot
One big missing element in the log scanning for my server was dovecot. To add support I
- added an entry to acquis.yaml that specified the log file and set the labels.type to dovecot
- added a dovecot-logs.yaml with appropriate lines to filter and parse the log entries I was interested in
- added a scenario
After doing this and refining things a few times the dovecot log is now parsed, events recorded and bans created as required.
Simples!