Share my work: How to setup SingleDrop mailbox with Getmail 4

Ask questions about your setup or get help installing ZCS server (ZD section below).
Post Reply
philpw99
Posts: 18
Joined: Sat Sep 13, 2014 12:37 am

Share my work: How to setup SingleDrop mailbox with Getmail 4

Post by philpw99 »

Hi, everybody, I am very new to linux and zimbra, but I was so impressed on the free spirit of the community, like this whole full feature AJAX webmail suite is for free. It's hard to believe until I see it. That's why I finally decided to share my work with you guys, because you guys help me a lot during the learning process.

I have been working on it for 3 weeks now. Many obstacles were overcome. Thanks to tutorials and guides online. Then I met the final challenge: singledrop mailbox.

I don't want to pay much, so I set up only the cheapest option in 1and1.com. For $3 a year, I got a domain name: digitalwonders.us and a 2gb email box. I created the email as *@digitalwonders.us . It's a catch-all email, means it will receive all emails with the domain name "digitalwonders.us"

The service of 1and1 doesn't provide any "envelope to" feature. So if I need to send a mail to philip@digitalwonders.us , I need a program that pull the mail from the pop server, get the name of recipient and drop it to Zimbra.

Thanks to people in this forum, I was able to get fetchmail up and running, and deliver the mail with the method:
/opt/zimbra/postfix/sbin/sendmail -t
to send mail into the box. Everything seems fine until I met an email with To: is SomeoneElse@example.com and CC as me: philip@digitalwonder.us

"sendmail -t" will send a copy to SomeoneElse@example.com, which is really not right because it was supposed to receive only, not bounce back a copy.
I have been thinking about how to solve the problem for a whole week. Tried to intercept the bounce emails, but always feel not right, why can't fetchmail just get the email filtered by the domain, and set the recipient, and send it to Zimbra? I looked up and down, nothing in fetchmail can do what I need. In a post, somebody said it's impossible. I was discouraged.
Then I met Getmail 4. It uses a much more structured and easier to understand configuration file, and it had a multiguesser feature. I worked on it again. It was up and running very quickly, thanks to the nice documentation of getmail 4. Here is a sample.


[retriever]

type = SimplePOP3SSLRetriever

server = pop.1and1.com

username = *@digitalwonders.us

password = nowayiwilltellyou
[destination]

type = MultiGuesser

default = [admin]

locals = (

('philip@digitalwonders.us' , '[philip]'),

)
[philip]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ( "-f", "%(sender)", "philip@digitalwonders.us" )

user = philip

group = admin
[admin]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ("-f", "%(sender)","admin@digitalwonders.us", )

user = philip

group = admin


The above showcases the multiguesser feature, it will match my email and send it to my zimbra account. For all others, it will send to admin account, so there is no bounce mails.
It seemed everything was perfect. However, then I realized that if I put this in my friend's server, every time they added or removed a user, they had to call me and ask me to do it. It was a big waste of the beautiful Admin console!
The question came back again: can Getmail 4 pull the catch-all emails from a single mailbox without envelops, guess the recipient's names, filter them by domain, and set them to %(recipient) variable, so the emails can be automatically delivered to different accounts in Zimbra without bouncing back to external addresses in headers?
The short answer is "Now you can".
Thanks again for Getmail's pure python implementation. I learned python in a hurry and mostly copied their code. I added a "SingleDropMultiGuesser" to the file "destinations.py" It was located at

/usr/lib/python2.5/site-packages/getmailcore/

not easy to find, but once you replace the script with the attached one, you have a new method of guessing and selecting emails by the domain, and the email address that fit the reg ex will be added to %(recipient) variable. So now getmail 4 became even stronger with this new addition. Do you still want to use fetchmail?
Here is the finished getmailrc:



[options]

read_all = false

delivered_to = false

received = false

delete_after = 30

max_message_size = 0

message_log = /var/log/getmail.log
[retriever]

type = SimplePOP3SSLRetriever

server = pop.1and1.com

username = *@digitalwonders.us

password = nowayiwilltellyou
[destination]

type = SingleDropMultiGuesser

default = [admin]

locals = (

('.*@digitalwonders.us' , '[mails]'),

)
[mails]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ( "-f", "%(sender)", "%(recipient)" )

user = philip

group = admin
[admin]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ("admin@poficonstruction.com", )

user = philip

group = admin

The power of this code is that if a email has headers like this:

To: philip@digitalwonders.us, steven@digitalwonders.us, nobody@digitalwonders.us

Cc: SomeoneElse@example.com
It will deliver all 3 emails to my zimbra in one copy only and ignore the example.com email. Meanwhile nobody@digitalwonders.us will be bounced back and tell the sender that he make a mistake in address. It couldn't be more perfect. Now my friends can setup as many account as they like without the need of asking my help.
I share my hard work with you, for the appreciation of all the hard work done by you.
P.S. Here is a little shell script. I use it to run getmail. It will make sure only one instance of getmail is running. I use cron to run this script every minute. If one process of getmail take longer time to download emails, the others will simply skip.


#!/bin/sh
# our tmpfile

LOCKFILE=/tmp/getmail-running
if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then

# echo "already running"

exit

fi
# make sure the lockfile is removed when we exit and then claim it

trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT

echo $$ > ${LOCKFILE}
# do stuff

/usr/bin/getmail
rm -f ${LOCKFILE}


A word of warning: because singledropmultiguesser is using method of guessing, so it's fallible. For example, I don't think it will work with Chinese email names. Use it at your own risk.
destinations.zip
bhuva25
Posts: 10
Joined: Tue Dec 31, 2019 10:29 am

Re: Share my work: How to setup SingleDrop mailbox with Getmail 4

Post by bhuva25 »

philpw99 wrote:Hi, everybody, I am very new to linux and zimbra, but I was so impressed on the free spirit of the community, like this whole full feature AJAX webmail suite is for free. It's hard to believe until I see it. That's why I finally decided to share my work with you guys, because you guys help me a lot during the learning process.

I have been working on it for 3 weeks now. Many obstacles were overcome. Thanks to tutorials and guides online. Then I met the final challenge: singledrop mailbox.

I don't want to pay much, so I set up only the cheapest option in 1and1.com. For $3 a year, I got a domain name: digitalwonders.us and a 2gb email box. I created the email as *@digitalwonders.us . It's a catch-all email, means it will receive all emails with the domain name "digitalwonders.us"

The service of 1and1 doesn't provide any "envelope to" feature. So if I need to send a mail to philip@digitalwonders.us , I need a program that pull the mail from the pop server, get the name of recipient and drop it to Zimbra.

Thanks to people in this forum, I was able to get fetchmail up and running, and deliver the mail with the method:
/opt/zimbra/postfix/sbin/sendmail -t
to send mail into the box. Everything seems fine until I met an email with To: is SomeoneElse@example.com and CC as me: philip@digitalwonder.us

"sendmail -t" will send a copy to SomeoneElse@example.com, which is really not right because it was supposed to receive only, not bounce back a copy.
I have been thinking about how to solve the problem for a whole week. Tried to intercept the bounce emails, but always feel not right, why can't fetchmail just get the email filtered by the domain, and set the recipient, and send it to Zimbra? I looked up and down, nothing in fetchmail can do what I need. In a post, somebody said it's impossible. I was discouraged.
Then I met Getmail 4. It uses a much more structured and easier to understand configuration file, and it had a multiguesser feature. I worked on it again. It was up and running very quickly, thanks to the nice documentation of getmail 4. Here is a sample.


[retriever]

type = SimplePOP3SSLRetriever

server = pop.1and1.com

username = *@digitalwonders.us

password = nowayiwilltellyou
[destination]

type = MultiGuesser

default = [admin]

locals = (

('philip@digitalwonders.us' , '[philip]'),

)
[philip]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ( "-f", "%(sender)", "philip@digitalwonders.us" )

user = philip

group = admin
[admin]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ("-f", "%(sender)","admin@digitalwonders.us", )

user = philip

group = admin


The above showcases the multiguesser feature, it will match my email and send it to my zimbra account. For all others, it will send to admin account, so there is no bounce mails.
It seemed everything was perfect. However, then I realized that if I put this in my friend's server, every time they added or removed a user, they had to call me and ask me to do it. It was a big waste of the beautiful Admin console!
The question came back again: can Getmail 4 pull the catch-all emails from a single mailbox without envelops, guess the recipient's names, filter them by domain, and set them to %(recipient) variable, so the emails can be automatically delivered to different accounts in Zimbra without bouncing back to external addresses in headers?
The short answer is "Now you can".
Thanks again for Getmail's pure python implementation. I learned python in a hurry and mostly copied their code. I added a "SingleDropMultiGuesser" to the file "destinations.py" It was located at

/usr/lib/python2.5/site-packages/getmailcore/

not easy to find, but once you replace the script with the attached one, you have a new method of guessing and selecting emails by the domain, and the email address that fit the reg ex will be added to %(recipient) variable. So now getmail 4 became even stronger with this new addition. Do you still want to use fetchmail?
Here is the finished getmailrc:



[options]

read_all = false

delivered_to = false

received = false

delete_after = 30

max_message_size = 0

message_log = /var/log/getmail.log
[retriever]

type = SimplePOP3SSLRetriever

server = pop.1and1.com

username = *@digitalwonders.us

password = nowayiwilltellyou
[destination]

type = SingleDropMultiGuesser

default = [admin]

locals = (

('.*@digitalwonders.us' , '[mails]'),

)
[mails]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ( "-f", "%(sender)", "%(recipient)" )

user = philip

group = admin
[admin]

type = MDA_external

path = /opt/zimbra/postfix/sbin/sendmail

arguments = ("admin@poficonstruction.com", )

user = philip

group = admin

The power of this code is that if a email has headers like this:

To: philip@digitalwonders.us, steven@digitalwonders.us, nobody@digitalwonders.us

Cc: SomeoneElse@example.com
It will deliver all 3 emails to my zimbra in one copy only and ignore the example.com email. Meanwhile nobody@digitalwonders.us will be bounced back and tell the sender that he make a mistake in address. It couldn't be more perfect. Now my friends can setup as many account as they like without the need of asking my help.
I share my hard work with you, for the appreciation of all the hard work done by you.
P.S. Here is a little shell script. I use it to run getmail. It will make sure only one instance of getmail is running. I use cron to run this script every minute. If one process of getmail take longer time to download emails, the others will simply skip.


#!/bin/sh
# our tmpfile

LOCKFILE=/tmp/getmail-running
if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then

# echo "already running"

exit

fi
# make sure the lockfile is removed when we exit and then claim it

trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT

echo $$ > ${LOCKFILE}
# do stuff

/usr/bin/getmail
rm -f ${LOCKFILE}


A word of warning: because singledropmultiguesser is using method of guessing, so it's fallible. For example, I don't think it will work with Chinese email names. Use it at your own risk.
destinations.zip

I have tried above solution but still mails are receiving in postmaster. Please suggest one solution to make it work in getmail

We have configured fetchmail to fetch mails from catchall server and delivering to domain ids. If we send mail to online domain user c@a.com along with local zimbra user a@a.com and b@a.com means online user receiving 3 copies (1 original copy+2 cc copy). How to avoid duplicate mails in fetchmail?

Below configuration is used to deliver mails

poll remote-host proto imap envelope 1 "Delivered-To"
localdomains example.com: user catchall-user with pass "password" to * here
bhuva25
Posts: 10
Joined: Tue Dec 31, 2019 10:29 am

Re: Share my work: How to setup SingleDrop mailbox with Getmail 4

Post by bhuva25 »

I have tried above solution but still mails are receiving in postmaster. Please suggest one solution to make it work in getmail

We have configured getmail to fetch mails from catchall server and delivering to domain ids. If we send mail to online domain user c@a.com along with local zimbra user a@a.com and b@a.com means online user receiving 3 copies (1 original copy+2 cc copy). How to avoid duplicate mails in zimbra postfix?

Below configuration is used to deliver mails. Is there any way to use mailing list with specific ids instead of wildcard?

[destination]
type = MDA_external
path = /usr/sbin/sendmail
#arguments = ( "-f", "%(sender)", "%(recipient)" )
arguments = ("-f%(sender)","-t","%(recipient)")
user = zimbra
group = zimbra
Post Reply