Subject editing based on email headers...

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
User avatar
DavidMerrill
Advanced member
Advanced member
Posts: 126
Joined: Thu Jul 30, 2015 2:44 pm
Location: Portland, ME
ZCS/ZD Version: 8.8.15 P19
Contact:

Subject editing based on email headers...

Post by DavidMerrill »

Hi All,

I've got a client who is pretty interested in pre-pending "(External Sender)" to the subject of any email they receive that does not come from their own domain.

I've poked around with postfix header_checks and got a POC going (on an 8.8.15 deployment) on plain old subject-editing by following:

- https://wiki.zimbra.com/wiki/King0770-N ... der-Checks

I tried out the example for censoring bad words

Code: Select all

/^Subject:.*Bad Word*/ REPLACE Subject: Censored
and ended making mine like this:

Code: Select all

# 20191031 - DWM - Test subject replacement...
/^Subject:(.*)Work(.*)$/
        REPLACE Subject: Censored - "Work" is not an allowed word...
One caveat:

- The article states that in ZCS 8.5+ you do not use zmlocalconfig.
- I found that I HAD to make the suggested zmclocalconfig edits to get it to work

Long lead-in, here's my query for folks...

I think I'm at a loss though, from what I understand of postfix header_checks, there may not be a way to (bear w/my pseudo-code):

Code: Select all

IF ( (FROM DOES NOT CONTAIN abc.com) AND (TO CONTAINS abc.com) )
THEN PREPEND subject with "(External Sender) "
as Postfix's header checks go through the headers in the email line-by-line? Am I barking up the wrong tree or asking too much?

Thanks for taking a look!
___________________________________
David Merrill - Zimbra Practice Lead
OTELCO Zimbra Hosting, Licensing and Professional Services
Zeta Alliance
User avatar
DavidMerrill
Advanced member
Advanced member
Posts: 126
Joined: Thu Jul 30, 2015 2:44 pm
Location: Portland, ME
ZCS/ZD Version: 8.8.15 P19
Contact:

Re: Subject editing based on email headers...

Post by DavidMerrill »

Still chasing down possibilities, it might be possible to clear this nugget by figuring out how to invoke the header_checks on a per domain basis?
___________________________________
David Merrill - Zimbra Practice Lead
OTELCO Zimbra Hosting, Licensing and Professional Services
Zeta Alliance
User avatar
DavidMerrill
Advanced member
Advanced member
Posts: 126
Joined: Thu Jul 30, 2015 2:44 pm
Location: Portland, ME
ZCS/ZD Version: 8.8.15 P19
Contact:

Re: Subject editing based on email headers...

Post by DavidMerrill »

Still kicking this around. It occurred to me that a nice place to be able to do this would be w/in Zimbra's mail filtering.

There's no current convention for altering subjects, but I wondered if someone considered adding that functionality via a Zimlet. I've searched a bit & found this (which seemed promising - at least SOMEONE in the past and run into this):

- viewtopic.php?t=44524

it looks good (code get's kicked around and I see what looks like a reference to prepending the subject) but the thread dies off pretty quickly.
___________________________________
David Merrill - Zimbra Practice Lead
OTELCO Zimbra Hosting, Licensing and Professional Services
Zeta Alliance
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 899
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Subject editing based on email headers...

Post by JDunphy »

Hi David,

I am thinking about this but nothing yet pops out... A few idea and questions. I am leaning toward amavisd/SA to do this.

A few ideas.

1) Given you probably digitally sign your domains, it would be harder to spoof and an additional clause could strengthen against FN/FP's?
2) It would be possible to extend the interface later with more options fairly easily via SA

The problem is amavisd has some support already as does SA. With respect to SA, I am looking into both Plugin/WhiteListSubject.pm which pulls support from PerMsgStatus.pm as a starting point.

On the other hand, Amavisd looks like it could also do it but I find that code really hard to follow. They have the concept of: $sa_spam_subject_tag1 (commented out) which populates @spam_subject_tag_maps that is applied at various times to prepend the Subject. I think??? tag1 would work most times and not just spammy times.

If SA could do it then you could do it with a modified plugin which would allow you to further extend it with a simple function/rules in the future as more capability is required. You could even pass in the selector for inclusion for some use cases.

I think this is a useful feature myself and could use it for some cases where we are pretty sure but not sure enough to move it to the junk folder to further help our users. Currently we are using filters to move email from external addresses with attachments that are not in our contacts or who we have had previously correspondence with... ie. external email. I guess that would work to... a folder that is for external email not in contacts. Same concept to alert the user that this could be more dangerous that inbox mail and to be careful.

Still thinking but nothing concrete yet what or how to approach this problem.

I am also not discounting it could be a zimlet.

Just thinking out loud here. Maybe someone else has some ideas to offer that will get us thinking what would be possible both feature wise and what layer to do this at.

Jim
User avatar
DavidMerrill
Advanced member
Advanced member
Posts: 126
Joined: Thu Jul 30, 2015 2:44 pm
Location: Portland, ME
ZCS/ZD Version: 8.8.15 P19
Contact:

Re: Subject editing based on email headers...

Post by DavidMerrill »

Hi Jim,

Thanks for sharing your thoughts on this.

I wasn't clear on this statement?
Given you probably digitally sign your domains, it would be harder to spoof and an additional clause could strengthen against FN/FP's?
I like that there are some possibilities with Amavis/SA but was hoping (ha!) not to have to customize too much. Having said that if it could be done sanely and was reproducible that could be fine.

I also like the idea abandoning any kind of messing around with the subject line entirely & using the filters to move things into folders (i.e. shift-work-culture instead of paying for technology-fix that would then need to be supported).

Exploring the zimlet (that provides the functionality in the filtering dialog box) path could be "fun"? It'd be new ground for me (where is the "So you want to develop Zimlets?" page anyway?).

Best,
David
Last edited by DavidMerrill on Mon Nov 04, 2019 11:58 am, edited 1 time in total.
___________________________________
David Merrill - Zimbra Practice Lead
OTELCO Zimbra Hosting, Licensing and Professional Services
Zeta Alliance
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 899
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Subject editing based on email headers...

Post by JDunphy »

DavidMerrill wrote:Hi Jim,

Thanks for sharing your thoughts on this.

I wasn't clear on this statement?
Given you probably digitally sign your domains, it would be harder to spoof and an additional clause could strengthen against FN/FP's?
Hi David,

That was in reference to some pseudo-code you had written to strengthen those checks.

Code: Select all

header __RETURNPATH_FROM Return-Path =~ /\@examplel\.com|\@example\.net/i
header __SEARCHTERM ... list of stuff to search for
meta REPORT2USER_1 (!__RETURNPATH_FROM && !DKIM_VALID_AU && __SEARCHTERM)
score  REPORT2USER_1 0.001
describe REPORT2USER_1 Warning this not allowed
I wonder if a zimlet that reported based on specific rules listed in X-Spam-Status would provide enough flexibility without becoming a maintenance problem... you could add SA rules to trigger how or if you wanted it reported... ie. for subset of domains, foreign delivery, external urls, bad words, dangerous attachments, tracking, stealthy techniques like bayes busting text off the view-able screen, etc. In other words, anything you could write a SA rule for is potentially on the table to have the zimlet fire on and alert the user. The zimlet would then only need to check to see if its included rules are present and if so do something to alert the user. That would allow you to update the triggers without updating the zimlet. Just update the sauser.cf file. Could use a convention for the rules like 'Report2User_' that the zimlet searches for as the prefix to notify the zimlet to jump into action.

How would the zimlet report or alert the user to a trigger? Adding a tag to the email? adding some text before the message when the user viewed the email? Change the subject line, move it to a folder, lock it so it can 't be displayed. It would suffer from the problem that new rules would not be seen in previously processed email and that other MUA don't get the warnings. The benefit is you don't need an admin interface and the code for the zimlet would be much simpler as all it needs to do is search for X-Spam-Status header and perform some action should one of its rules be included.

I had written a zimlet 2-3 years ago that might be a starting point for me to experiment with. I don't remember where I left off other than this comment in the code. In fact, I feel like I have now forgotten everything I had learned about zimlets so that is par for the course these days with me. ;-)

Code: Select all

/**
 * This zimlet checks for X-Spam-Score message header. The X-Spam-Score is displayed below the message subject when the message is opened.
 *
 */
Still thinking out loud here or rambling a lot perhaps. :-)

Jim
tonyg
Advanced member
Advanced member
Posts: 51
Joined: Fri Mar 16, 2018 5:25 pm
Location: USA
ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
Contact:

Re: Subject editing based on email headers...

Post by tonyg »

From Mr Clueless Observer:

If I understand correctly, the challenge here is to modify the header/subject after reading through the header?

I've read in various places about :
1) pre-processing with a milter:
http://www.postfix.org/postconf.5.html# ... der_checks
2) using cleanup before mail enters the queue:
http://www.postfix.org/cleanup.8.html
3) using a different IP address and/or port for local vs remote SMTP:
http://www.postfix.org/BUILTIN_FILTER_R ... emote_only

I dunno if any of that rings a bell. I don't have immediate application for this, but I know a couple of my clients do something similar - they're in legal/medical/finance. Rather than modifying the subject, they add text to the top of the body:
** WARNING: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. **
or simply
[EXTERNAL]

Like with the Subject, that body text is very visible to the recipient. But to achieve another kind of visible cue, what about adding a Category to the email and then (not sure if this is possible), using a CoS to set the color of items with that flag? I think that presents the same challenge that you already have, sorry. :(

Idea spew ™

HTH
Post Reply