From time to time I run across an Exim mailserver whose mail queue has become corrupted. This can often happen if the server has unexpected reboots or crashes while processing a large queue and loses data as a result.

Say I notice exim is taking a lot of resources on the server and want to see what’s going on. First I may look to see how many messages are in the queue:

[root@mailserver ~]# exim -bpc
11078
[root@mailserver ~]#

Wow, that’s a lot of messages for most servers, even for shared hosts. (this server was generating a ton of bounces for various reasons, but that’s another story.) The large queue was becoming corrupt often, so that when I actually tried to drill down on queued messages I would get strange errors. Here I want to see what’s in the queue from the past hour.

[root@mailserver ~]# exiqgrep -y 3600
Line mismatch: 5h 1Qmr1H-0005ql-So
[root@mailserver ~]#

What’s this? I decide to freeze the offending message-id:

[root@mailserver ~]# exim -Mf 1Qmr1N-0005rj-BM
Spool data file for 1Qmr1N-0005rj-BM does not exist
[root@mailserver ~]#

Ahh, exim has lost the spool file for this message so I can remove the message:

[root@mailserver ~]# exim -Mrm 1Qmr1N-0005rj-BM
Spool data file for 1Qmr1N-0005rj-BM does not exist
Continuing, to ensure all files removed
Message 1Qmr1N-0005rj-BM has been removed or did not exist
[root@mailserver ~]#

But chances are if this happened to one, there are more to follow:

[root@mailserver ~]# exiqgrep -y 3600
Line mismatch: 5h 1Qmr1N-0005rm-Uq
[root@mailserver ~]#

To remove a larger number of these corrupt messages I can run the following command for any that are labeled ’5h’.

[root@mailserver ~]# exim -bpru | grep ’5h’ | awk ‘{print $2}’|xargs -n 1 -P 20 exim -Mrm
exim: malformed message id 886 after -Mrm option
-snip-
exim: malformed message id 886 after -Mrm option
Spool data file for 1Qmr1V-0005sc-9v does not exist
Continuing, to ensure all files removed
Message 1Qmr1V-0005sc-9v has been removed or did not exist
exim: malformed message id 886 after -Mrm option
-snip-
exim: malformed message id 1016 after -Mrm option
Spool data file for 1Qmr1z-0005y7-1x does not exist
Continuing, to ensure all files removed
Message 1Qmr1z-0005y7-1x has been removed or did not exist
exim: malformed message id 894 after -Mrm option
-snip-
exim: malformed message id 897 after -Mrm option
Spool data file for 1Qmr1S-0005sL-28 does not exist
Continuing, to ensure all files removed
Message 1Qmr1S-0005sL-28 has been removed or did not exist
exim: malformed message id 886 after -Mrm option
-snip-
exim: malformed message id 965 after -Mrm option
Spool data file for 1Qmr1N-0005rm-Uq does not exist
Continuing, to ensure all files removed
Message 1Qmr1N-0005rm-Uq has been removed or did not exist
exim: malformed message id 1.1K after -Mrm option
-snip-
exim: malformed message id 940 after -Mrm option
[root@mailserver ~]#

Notice all the lines that say ‘malformed message id’ (I snipped hundreds form the command output above). This is because only the corrupt messages have the message id as the 2nd position in the above commands output that is sent to the awk ‘print($2)’, all messages that are not corrupt will print the size of the message in this column, so only the corrupt messages are affected.

If I continue to run this command for any more messages that report ‘line mismatch’, I can remove the corrupt messages and get on with troubleshooting the real problem at hand, the ever growing mail queue on this server.