Page 2 of 2

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 1:11 pm
by phoenix
Ah yes, I remember you've mentioned that before. There should be no reason for ZeXtras to have those problems regardless of the underlying operating system, I've used it for years without problems. If you had those sorts of problems with ZeXtras it would suggest to me that you probably had problems with the your 'current' installation at the time you tried it. If ZeXtras is not your preferred solution how about just exporting accounts and importing them into a new server? I can't really comment about Ubuntu, I have used it and also done a migration of ZCS from one release to another and still didn't have problems but Ubuntu is not my favourite distribution.

If you don't have any outrageous modifications on ZCS (i.e. a fairly standard install) then it would appear to me that new new build would be the best option, preferably to CentOS7 purely from my point of view of course. :)

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 1:30 pm
by JDunphy
Labsy wrote:This is nuts! Users are getting mad, my phone will overheat of complaints.
Amavis discarding messages like crazy:

Code: Select all

amavis[32456]: (32456-19) Blocked SPAM {DiscardedInbound}, ..., Queue-ID: 7F317168EBB2, mail_id: O9Pr3MirlGCG, Hits: 31.534, size: 13480, 1367 ms
postfix/smtp[10848]: 7F317168EBB2: ..., relay=127.0.0.1[127.0.0.1]:10024, delay=2.9, delays=1.5/0/0/1.4, dsn=2.7.0, status=sent (250 2.7.0 Ok, discarded, id=32456-19 - spam)
I have no idea, how to determine, which filter bumped spam score to 30+, so I cannot adjust.
Please, desperatelly need ideas what to do.
Discarding is normal for high scoring spam but in case this wasn't that type.
Did you miss making sure that the default 15 is much higher while you investigate... say:

Code: Select all

$sa_kill_level_deflt = 150.0;
So you could investigate. Given that 31.534 points which is cumulative with all the rules is over the default 15... How sure are you that you were able to make this change?
Here is a link describing some of those important variables: https://blog.bravi.org/?p=683

Determining the rule is fairly simple once you have the email... Two methods:
1) Look at this header in the email

Code: Select all

X-Spam-Status: No, score=-104.21 required=4.8 tests=[BAYES_00=-1.9,
	DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
	HTTP_IN_BODY=0.1, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01,
	USER_IN_DKIM_WHITELIST=-100] autolearn=ham autolearn_force=no
And verify or adjust scores for those rules.
2) capture the email and run it through spamassassin in debug mode as I showed previously and verify the rules that way.

I guess the other thing is how did you tell amavis to restart after adjusting it parameters?

Code: Select all

zmamavisdctl restart

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 1:41 pm
by JDunphy
I should note this: viewtopic.php?t=144 I had discarded D_PASS because I thought it would disable spam checking... I am beginning to think it might work the same as raising the score really high... reference: https://www.ijs.si/software/amavisd/ama ... ml#actions ... Hmm lots of ways apparently.

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 1:44 pm
by Labsy
Thanx, JDunphy, I was now able to think a bit more (I am so stressed and under pressure about the issue, that I barely can think normally).
So first I did now is to D_PASS all messages through, so I will be able to see message headers and brake down the spam score.

Code: Select all

 zmprov ms `zmhostname` zimbraAmavisFinalSpamDestiny D_PASS
 zmamavisdctl restart
Now just wait and catch some e-mails.

BTW...as now all mail will pass, do you have idea how to catch only those, which othervise wouldn't?

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 1:55 pm
by JDunphy
Labsy wrote:Thanx, JDunphy, I was now able to think a bit more (I am so stressed and under pressure about the issue, that I barely can think normally).
So first I did now is to D_PASS all messages through, so I will be able to see message headers and brake down the spam score.

Code: Select all

 zmprov ms `zmhostname` zimbraAmavisFinalSpamDestiny D_PASS
 zmamavisdctl restart
Now just wait and catch some e-mails.

BTW...as now all mail will pass, do you have idea how to catch only those, which othervise wouldn't?
You will be looking at the headers and the score. Initially, I thought D_PASS would disable spam scoring but now I think it only disables spam discarding with amavisd-new. If that is the case, any email including this that is sent to you by the forum software should have an X-Spam-Status header.

Code: Select all

X-Spam-Status: No, score=1.567 required=5.0 tests=[BAYES_50=0.8,
	DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
	HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_IMAGE_ONLY_32=0.001,
	HTML_MESSAGE=0.001, HTTP_IN_BODY=0.1, J_IMG_NO_EXTENS=0.1,
	J_RCVD_IN_HOSTKARMA_YEL=0.003, RCVD_IN_DNSWL_NONE=-0.0001,
	SPF_HELO_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_GREY=0.424]
	autolearn=no autolearn_force=no
If you no longer have X-Spam-Status then this isn't the method you want. So in recap... score under 5 will be in your inbox or a folder by some user defined filter and anything higher will be in your junk folder. Any score higher than 15 would be normally discarded and not delivered to the junk folder. That last case is what you are most interested in.

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 2:07 pm
by ccelis5215
Labsy,

You can search zimbra log

Code: Select all

grep -i spammy /var/log/zimbra.log


Hope it help.

ccelis

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 2:30 pm
by JDunphy
Labsy wrote: BTW...as now all mail will pass, do you have idea how to catch only those, which othervise wouldn't?
Ohhh great tip from caccelis5215 about 'grep spammy /var/log/zimbra.log' ... learn something new every day. :-)

In the past, I have pulled the junk folder for a user that is having a problem. I'll see if I can find that program. I want to write a zimlet that a user could click and then it would provide a bunch of details about the email including why it was flagged or not flagged as spam and an option to forward on that email or parts of it to the admin for further analysis. I was going to use the zeta alliances unsubscribe zimlet as the base.

Darn... I can't find that script. Here is the general idea from what I have done in the past.

Code: Select all

zmmailbox -z -m user@example.net -t 0 getRestURL "/?fmt=tgz&query=in:junk"| tar -xz -O --wildcards '*.eml'
That would pull the junk folder so my idea was to do something like this.

Code: Select all

% zmmailbox -z -m user@example.net -t 0 getRestURL "/?fmt=tgz&query=in:junk" | tar -xz -O --wildcards '*.eml' | grep -A 10 X-Spam-Score
X-Spam-Score: 13.268
X-Spam-Level: *************
X-Spam-Status: Yes, score=13.268 required=4.8 tests=[BAYES_99=4,
	BAYES_999=0.2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
	HTML_FONT_FACE_BAD=0.981, HTML_IMAGE_ONLY_20=1.546,
	HTML_MESSAGE=0.001, HTML_SHORT_LINK_IMG_3=0.148, HTTP_IN_BODY=0.1,
	J_BELOW_FOLD=1.5, J_DNSBL_MILTER_META=0.3, J_IMG_NO_EXTENS=0.1,
	J_OBFUSCATED_URL=1, KAM_INFOUSMEBIZ=0.5, RCVD_IN_IVMSIP24=2,
	RDNS_DYNAMIC=0.982, T_REMOTE_IMAGE=0.01]
	autolearn=no autolearn_force=no

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 2:42 pm
by gbillat
Labsy wrote:This is nuts! Users are getting mad, my phone will overheat of complaints.

I have no idea, how to determine, which filter bumped spam score to 30+, so I cannot adjust.
Please, desperatelly need ideas what to do.
Hi,

Thanks for bringing this to our attention. We want to get your issue resolved asap, but it sounds complex. Please open a support ticket to get help directly from the Zimbra Support Team.

Thanks,
Gayle

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 5:05 pm
by JDunphy
ccelis5215 wrote:Labsy,
You can search zimbra log

Code: Select all

grep -i spammy /var/log/zimbra.log
Been playing around with this a little today. Going further with your suggestion.

Code: Select all

grep -i blocked /var/log/zimbra.log
and 
grep -i blocked /var/log/zimbra.log | awk '{print $22, $12}' | sort
to see what wasn't delivered.

Our lowest was DrOzzfatburner scored at 15.013 that didn't get delivered to my own junk folder and our highest was 68.885 to our noc from SerbianBeauties. They will be disappointed. :-)

Re: Help needed - Amavis deleting healthy mail items

Posted: Thu Mar 15, 2018 5:20 pm
by ccelis5215
ahhh.., those spammers..

Code: Select all

6.746, <MoneyNews@mysheddss.bid>
9.91, <5GMale@lawsuitss.bid>
:D

ccelis