Project

General

Profile

Syslog server » History » Version 3

Charles Atkinson, 12/08/2020 17:30
Added "bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968171 workaround" and "h2. Cisco Small Business series switches"

1 1 Charles Atkinson
h1. syslog (rsyslog) server implementation
2 1 Charles Atkinson
3 1 Charles Atkinson
{{toc}}
4 1 Charles Atkinson
5 1 Charles Atkinson
h1. Introduction
6 1 Charles Atkinson
7 1 Charles Atkinson
This page documents a syslog (rsyslog) server implementation that results in:
8 2 Charles Atkinson
* Each client having its own /srv/syslog/<FQDN> directory, including clients that identify themselves by bare hostname or by IP address
9 2 Charles Atkinson
* The files in that directory being named as they were on the client
10 1 Charles Atkinson
* Three year retention
11 1 Charles Atkinson
12 1 Charles Atkinson
h2. Related documents
13 1 Charles Atkinson
14 3 Charles Atkinson
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968171
15 1 Charles Atkinson
16 1 Charles Atkinson
h1. Server
17 1 Charles Atkinson
18 1 Charles Atkinson
h2. Directories and files
19 1 Charles Atkinson
20 1 Charles Atkinson
|_.Directory or file |_.Usage |
21 1 Charles Atkinson
|/etc/logrotate.d/rsyslog-imudp |Rotates /srv/syslog/<FQDN>/* log files |
22 1 Charles Atkinson
|/etc/rsyslog.conf |Primary configuration file; as installed by package |
23 1 Charles Atkinson
|/etc/rsyslog.d/00-modules.conf |Load and configure rsyslog modules imudp (input messages by UDP) and builtin:omfile (output files)|
24 1 Charles Atkinson
|/etc/rsyslog.d/10-FileFormat.template.conf |Similar to the version on all servers|
25 1 Charles Atkinson
|/etc/rsyslog.d/10-templates.conf |Sets /srv/syslog/<FQDN>/<log name> paths and sets same message format as 10-FileFormat.template.conf |
26 1 Charles Atkinson
|/etc/rsyslog.d/20-rulesets.conf |Equivalent to rsyslog.conf's rules; directs clients messages to individual /srv/syslog/<FQDN>/<log name> files.  Associates the rules with module imudp |
27 1 Charles Atkinson
|/etc/rsyslog.d/debug.conf.disabled |When .disabled removed, configures debug logging |
28 1 Charles Atkinson
|/etc/rsyslog.d/postfix.conf |Installed by the postfix package |
29 1 Charles Atkinson
|/srv/syslog/<FQDN>/ |Directory for all logs from a client |
30 1 Charles Atkinson
|/var/log/* |Local log files, same as on all servers |
31 1 Charles Atkinson
32 1 Charles Atkinson
/etc/logrotate.d/rsyslog-imudp
33 1 Charles Atkinson
<pre>
34 1 Charles Atkinson
# Rotation for logs created by the rsyslog configuration for imudp
35 1 Charles Atkinson
36 1 Charles Atkinson
/srv/syslog/*/syslog
37 1 Charles Atkinson
{
38 1 Charles Atkinson
    daily
39 1 Charles Atkinson
    rotate 731
40 1 Charles Atkinson
    dateext
41 1 Charles Atkinson
    dateyesterday
42 1 Charles Atkinson
    missingok
43 1 Charles Atkinson
    notifempty
44 1 Charles Atkinson
    delaycompress
45 1 Charles Atkinson
    compress
46 1 Charles Atkinson
    postrotate
47 1 Charles Atkinson
        invoke-rc.d rsyslog rotate > /dev/null
48 1 Charles Atkinson
    endscript
49 1 Charles Atkinson
}
50 1 Charles Atkinson
51 1 Charles Atkinson
/srv/syslog/*/mail.info
52 1 Charles Atkinson
/srv/syslog/*/mail.warn
53 1 Charles Atkinson
/srv/syslog/*/mail.err
54 1 Charles Atkinson
/srv/syslog/*/mail.log
55 1 Charles Atkinson
/srv/syslog/*/daemon.log
56 1 Charles Atkinson
/srv/syslog/*/kern.log
57 1 Charles Atkinson
/srv/syslog/*/auth.log
58 1 Charles Atkinson
/srv/syslog/*/user.log
59 1 Charles Atkinson
/srv/syslog/*/lpr.log
60 1 Charles Atkinson
/srv/syslog/*/cron.log
61 1 Charles Atkinson
/srv/syslog/*/debug
62 1 Charles Atkinson
/srv/syslog/*/messages
63 1 Charles Atkinson
{
64 1 Charles Atkinson
    weekly
65 1 Charles Atkinson
    rotate 109
66 1 Charles Atkinson
    dateext
67 1 Charles Atkinson
    dateyesterday
68 1 Charles Atkinson
    missingok
69 1 Charles Atkinson
    notifempty
70 1 Charles Atkinson
    compress
71 1 Charles Atkinson
    delaycompress
72 1 Charles Atkinson
    sharedscripts
73 1 Charles Atkinson
    postrotate
74 1 Charles Atkinson
        invoke-rc.d rsyslog rotate > /dev/null
75 1 Charles Atkinson
    endscript
76 1 Charles Atkinson
}
77 1 Charles Atkinson
</pre>
78 1 Charles Atkinson
/etc/rsyslog.d/00-modules.conf
79 1 Charles Atkinson
<pre>
80 1 Charles Atkinson
# rsyslog config fragment for module customisation
81 1 Charles Atkinson
82 1 Charles Atkinson
# provides UDP syslog reception
83 1 Charles Atkinson
module(load="imudp")
84 1 Charles Atkinson
input(type="imudp" port="514")
85 1 Charles Atkinson
86 1 Charles Atkinson
# Change omfile's default parameters (so they do not have to be set in every action)
87 1 Charles Atkinson
# Note: setting template MyMsgFormat here was not effective
88 1 Charles Atkinson
module(
89 1 Charles Atkinson
    load="builtin:omfile"
90 1 Charles Atkinson
    dirCreateMode="0750"
91 1 Charles Atkinson
    dirGroup="adm"
92 1 Charles Atkinson
    dirOwner="root"
93 1 Charles Atkinson
    fileCreateMode="0640"
94 1 Charles Atkinson
    fileGroup="adm"
95 1 Charles Atkinson
    fileOwner="root"
96 1 Charles Atkinson
)
97 1 Charles Atkinson
</pre>/etc/rsyslog.d/10-templates.conf
98 1 Charles Atkinson
<pre>
99 1 Charles Atkinson
# rsyslog config fragment for custom templates
100 1 Charles Atkinson
101 1 Charles Atkinson
# Log file paths
102 2 Charles Atkinson
template (name="AuthLog"   type="string" string="/srv/syslog/%FROMHOST%/auth.log")
103 2 Charles Atkinson
template (name="DaemonLog" type="string" string="/srv/syslog/%FROMHOST%/daemon.log")
104 2 Charles Atkinson
template (name="DebugLog"  type="string" string="/srv/syslog/%FROMHOST%/debug.log")
105 2 Charles Atkinson
template (name="KernLog"   type="string" string="/srv/syslog/%FROMHOST%/kern.log")
106 2 Charles Atkinson
template (name="MailError" type="string" string="/srv/syslog/%FROMHOST%/mail.error")
107 2 Charles Atkinson
template (name="MailInfo"  type="string" string="/srv/syslog/%FROMHOST%/mail.info")
108 2 Charles Atkinson
template (name="MailLog"   type="string" string="/srv/syslog/%FROMHOST%/mail.log")
109 2 Charles Atkinson
template (name="MailWarn"  type="string" string="/srv/syslog/%FROMHOST%/mail.warn")
110 2 Charles Atkinson
template (name="Messages"  type="string" string="/srv/syslog/%FROMHOST%/messages")
111 2 Charles Atkinson
template (name="Syslog"    type="string" string="/srv/syslog/%FROMHOST%/syslog")
112 2 Charles Atkinson
template (name="UserLog"   type="string" string="/srv/syslog/%FROMHOST%/user.log")
113 1 Charles Atkinson
114 1 Charles Atkinson
# Message format
115 1 Charles Atkinson
template (name="MyMsgFormat" type="string"
116 1 Charles Atkinson
    string="%TIMESTAMP% %HOSTNAME:R,BRE,0,FIELD:^[^.]*\\.[^.]*--end:% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
117 1 Charles Atkinson
)
118 3 Charles Atkinson
</pre>Notes
119 3 Charles Atkinson
* <notextile>%FROMHOST%</notextile> is required as shown above to make clients which identify themselves by bare hostname or by IP address log to /srv/syslog/<FQDN> directories.  It requires DNS reverse lookup
120 3 Charles Atkinson
* In case DNS reverse lookup is not available or there are no such clients, <notextile>%FROMHOST%</notextile> should be replaced by <notextile>%HOSTNAME%</notextile>
121 3 Charles Atkinson
* When <notextile>%FROMHOST%</notextile> is used, bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968171 may be triggered.  It can be worked around by installing
122 3 Charles Atkinson
** "/usr/local/etc/restart_rsyslog_on_new_dir.conf":https://redmine.auroville.org.in/attachments/download/7715/restart_rsyslog_on_new_dir.conf
123 3 Charles Atkinson
** "/usr/local/bin/restart_rsyslog_on_new_dir.sh":https://redmine.auroville.org.in/attachments/download/7716/restart_rsyslog_on_new_dir.sh
124 3 Charles Atkinson
** "/etc/systemd/system/restart_rsyslog_on_new_dir.service":https://redmine.auroville.org.in/attachments/download/7717/restart_rsyslog_on_new_dir.service
125 2 Charles Atkinson
126 2 Charles Atkinson
/etc/rsyslog.d/20-rulesets.conf
127 1 Charles Atkinson
<pre>
128 1 Charles Atkinson
# rsyslog config fragment for custom rulesets
129 1 Charles Atkinson
130 1 Charles Atkinson
# Ruleset based on package's rsyslog.conf's rules for local messages
131 1 Charles Atkinson
ruleset(name="imudp"){
132 1 Charles Atkinson
    auth,authpriv.*             action(type="omfile" dynaFile="AuthLog"   template="MyMsgFormat")
133 1 Charles Atkinson
    *.*;auth,authpriv.none      action(type="omfile" dynaFile="Syslog"    template="MyMsgFormat")
134 1 Charles Atkinson
    daemon.*                    action(type="omfile" dynaFile="DaemonLog" template="MyMsgFormat")
135 1 Charles Atkinson
    kern.*                      action(type="omfile" dynaFile="KernLog"   template="MyMsgFormat")
136 1 Charles Atkinson
    mail.*                      action(type="omfile" dynaFile="MailLog"   template="MyMsgFormat")
137 1 Charles Atkinson
    user.*                      action(type="omfile" dynaFile="UserLog"   template="MyMsgFormat")
138 1 Charles Atkinson
    mail.info                   action(type="omfile" dynaFile="MailInfo"  template="MyMsgFormat")
139 1 Charles Atkinson
    mail.warn                   action(type="omfile" dynaFile="MailWarn"  template="MyMsgFormat")
140 1 Charles Atkinson
    mail.err                    action(type="omfile" dynaFile="MailError" template="MyMsgFormat")
141 1 Charles Atkinson
    *.=debug;\
142 1 Charles Atkinson
        auth,authpriv.none;\
143 1 Charles Atkinson
        news.none;mail.none     action(type="omfile" dynaFile="DebugLog"  template="MyMsgFormat")
144 1 Charles Atkinson
    *.=info;*.=notice;*.=warn;\
145 1 Charles Atkinson
        auth,authpriv.none;\
146 1 Charles Atkinson
        cron,daemon.none;\
147 1 Charles Atkinson
        mail,news.none          action(type="omfile" dynaFile="Messages"  template="MyMsgFormat")
148 1 Charles Atkinson
}
149 1 Charles Atkinson
150 1 Charles Atkinson
# Use the ruleset to input module for UPD
151 1 Charles Atkinson
input(type="imudp" port="514" ruleset="imudp")
152 1 Charles Atkinson
</pre>/etc/rsyslog.d/debug.conf.disabled
153 1 Charles Atkinson
<pre>
154 1 Charles Atkinson
$DebugFile /var/log/rsyslog-debug.log
155 1 Charles Atkinson
$DebugLevel 2
156 1 Charles Atkinson
</pre>
157 1 Charles Atkinson
158 1 Charles Atkinson
h1. Clients
159 3 Charles Atkinson
160 3 Charles Atkinson
h2. Cisco Small Business series switches
161 3 Charles Atkinson
162 3 Charles Atkinson
<pre>
163 3 Charles Atkinson
snmp contact "<contact>" 
164 3 Charles Atkinson
snmp-server community <community name> ro view Default
165 3 Charles Atkinson
snmp-server location <FQDN>
166 3 Charles Atkinson
snmp-server server
167 3 Charles Atkinson
snmp-server source-interface traps vlan <VLAN>
168 3 Charles Atkinson
snmp-server host <rsyslog server IP address> traps version 2 <community name>
169 3 Charles Atkinson
</pre>
170 1 Charles Atkinson
171 1 Charles Atkinson
h2. Debian computers
172 1 Charles Atkinson
173 1 Charles Atkinson
/etc/rsyslog.d/00-rsyslog-server.conf
174 1 Charles Atkinson
<pre>
175 1 Charles Atkinson
# Format all messages with hostname as FQDN and send to <syslog server FQDN>
176 1 Charles Atkinson
177 1 Charles Atkinson
$PreserveFQDN on
178 1 Charles Atkinson
*.* @<syslog server FQDN>
179 1 Charles Atkinson
</pre>/etc/logrotate.d/rsyslog (as installed by package except for dateext and dateyesterday
180 1 Charles Atkinson
<pre>
181 1 Charles Atkinson
/var/log/syslog
182 1 Charles Atkinson
{
183 1 Charles Atkinson
	rotate 7
184 1 Charles Atkinson
	daily
185 1 Charles Atkinson
	dateext
186 1 Charles Atkinson
	dateyesterday
187 1 Charles Atkinson
	missingok
188 1 Charles Atkinson
	notifempty
189 1 Charles Atkinson
	delaycompress
190 1 Charles Atkinson
	compress
191 1 Charles Atkinson
	postrotate
192 1 Charles Atkinson
		invoke-rc.d rsyslog rotate > /dev/null
193 1 Charles Atkinson
	endscript
194 1 Charles Atkinson
}
195 1 Charles Atkinson
196 1 Charles Atkinson
/var/log/mail.info
197 1 Charles Atkinson
/var/log/mail.warn
198 1 Charles Atkinson
/var/log/mail.err
199 1 Charles Atkinson
/var/log/mail.log
200 1 Charles Atkinson
/var/log/daemon.log
201 1 Charles Atkinson
/var/log/kern.log
202 1 Charles Atkinson
/var/log/auth.log
203 1 Charles Atkinson
/var/log/user.log
204 1 Charles Atkinson
/var/log/lpr.log
205 1 Charles Atkinson
/var/log/cron.log
206 1 Charles Atkinson
/var/log/debug
207 1 Charles Atkinson
/var/log/messages
208 1 Charles Atkinson
{
209 1 Charles Atkinson
	rotate 4
210 1 Charles Atkinson
	weekly
211 1 Charles Atkinson
	dateext
212 1 Charles Atkinson
	dateyesterday
213 1 Charles Atkinson
	missingok
214 1 Charles Atkinson
	notifempty
215 1 Charles Atkinson
	compress
216 1 Charles Atkinson
	delaycompress
217 1 Charles Atkinson
	sharedscripts
218 1 Charles Atkinson
	postrotate
219 1 Charles Atkinson
		invoke-rc.d rsyslog rotate > /dev/null
220 1 Charles Atkinson
	endscript
221 1 Charles Atkinson
}
222 1 Charles Atkinson
223 1 Charles Atkinson
</pre>
224 1 Charles Atkinson
225 1 Charles Atkinson
h2. MikroTik routers
226 1 Charles Atkinson
227 1 Charles Atkinson
<pre>
228 1 Charles Atkinson
/system logging action
229 1 Charles Atkinson
set 3 bsd-syslog=yes remote=<syslog server address> src-address=<router address to send messages from> \
230 1 Charles Atkinson
    syslog-facility=local0 syslog-severity=alert
231 1 Charles Atkinson
</pre>