Right way to add option for new transport

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
postfixlover
Posts: 5
Joined: Fri Jan 24, 2020 11:56 am

Right way to add option for new transport

Post by postfixlover »

Hi all,
i'm new to zimbra but i'm using postfix since 1999.
I recently added to master.cf.in new transport to manage few slow-transport-rate destination domain.
To do this i have added in master.cf.in this:
slow unix - - n - 1 smtp
-o in_flow_delay=2s
-o syslog_name=slow
Transport will survive to restart of zmlocalconfig or server.
I have used zmlocalconfig for tuning ,
zmlocalconfig -e slow_delivery_slot_cost = 5
zmlocalconfig -e slow_delivery_slot_discount = 50
zmlocalconfig -e slow_delivery_slot_loan = 3
zmlocalconfig -e slow_destination_concurrency_failed_cohort_limit = 1
zmlocalconfig -e slow_destination_concurrency_limit = 1
zmlocalconfig -e slow_destination_concurrency_negative_feedback = 1
zmlocalconfig -e slow_destination_concurrency_positive_feedback = 1
zmlocalconfig -e slow_destination_rate_delay=5s
zmlocalconfig -e slow_destination_recipient_limit = 2
zmlocalconfig -e slow_extra_recipient_limit = 10
zmlocalconfig -e slow_initial_destination_concurrency = 1
zmlocalconfig -e slow_minimum_delivery_slots = 1
zmlocalconfig -e slow_recipient_limit = 10000
zmlocalconfig -e slow_recipient_refill_delay = 5s
zmlocalconfig -e slow_recipient_refill_limit = 10
zmlocalconfig -e slow_transport_rate_delay = 2s
.
Settng option for specific transport vanish after avery zmcontrol (or server) restart.
How can i do to permanently add option for new transport?

Thank you very much. Loris
User avatar
DualBoot
Elite member
Elite member
Posts: 1326
Joined: Mon Apr 18, 2016 8:18 pm
Location: France - Earth
ZCS/ZD Version: ZCS FLOSS - 8.8.15 Mutli servers
Contact:

Re: Right way to add option for new transport

Post by DualBoot »

Hello,

sometime Zimbra is a bit tricky and there is some attributes still present in the zmlocalconfig which are also present in the server configuration (stored into Zimbra LDAP).
So you need to identify which to modify at server level and to modify with zmprov ms server_name attribute_name value or at global level with zmprov mcf attibute_name value

Regards,
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2802
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.7 Network Edition
Contact:

Re: Right way to add option for new transport

Post by L. Mark Stone »

This wiki article should help clarify the Postfix settings that are now in LDAP, and whcih can therefore be set via a zmprov command (so that the changes you apply survive service restarts).

https://wiki.zimbra.com/wiki/New_Features_ZCS_8.5

Hope that helps,
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
postfixlover
Posts: 5
Joined: Fri Jan 24, 2020 11:56 am

Re: Right way to add option for new transport

Post by postfixlover »

L. Mark Stone wrote:This wiki article should help clarify the Postfix settings that are now in LDAP, and whcih can therefore be set via a zmprov command (so that the changes you apply survive service restarts).

https://wiki.zimbra.com/wiki/New_Features_ZCS_8.5

Hope that helps,
Mark
Thank you for suggestion, but ... while postfix show attribute due to new transport:

postconf | grep slow_
slow_delivery_slot_cost = $default_delivery_slot_cost
slow_delivery_slot_discount = $default_delivery_slot_discount
slow_delivery_slot_loan = $default_delivery_slot_loan
slow_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit
slow_destination_concurrency_limit = $default_destination_concurrency_limit
slow_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback
slow_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback
slow_destination_rate_delay = $default_destination_rate_delay
slow_destination_recipient_limit = $default_destination_recipient_limit
slow_extra_recipient_limit = $default_extra_recipient_limit
slow_initial_destination_concurrency = $initial_destination_concurrency
slow_minimum_delivery_slots = $default_minimum_delivery_slots
slow_recipient_limit = $default_recipient_limit
slow_recipient_refill_delay = $default_recipient_refill_delay
slow_recipient_refill_limit = $default_recipient_refill_limit
slow_transport_rate_delay = $default_transport_rate_delay

when i try to add attribute with zmprov

zmprov ms `zmhostname` slow_destination_concurrency_failed_cohort_limit 1
zmprov ms `zmhostname` slow_destination_concurrency_failed_cohort_limit '1'
zmprov ms `zmhostname` +slow_destination_concurrency_failed_cohort_limit 1

i always get error:

[] WARN: checkValue: no attribute info for: slow_destination_concurrency_failed_cohort_limit
ERROR: account.INVALID_ATTR_NAME (invalid attr name: invalid attr name - unable to modify attributes: ldap host=zm-prod-1.cineca.it:636: slow_destination_concurrency_failed_cohort_limit: AttributeDescription contains inappropriate characters) (cause: com.zimbra.cs.ldap.LdapException$LdapInvalidAttrNameException invalid attr name - unable to modify attributes: ldap host=zm-prod-1.cineca.it:636: slow_destination_concurrency_failed_cohort_limit: AttributeDescription contains inappropriate characters)

what i miss?

thank you in advance.
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2802
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.7 Network Edition
Contact:

Re: Right way to add option for new transport

Post by L. Mark Stone »

Please post the output from:

zmcontrol -v

...run as the zimbra user.
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
postfixlover
Posts: 5
Joined: Fri Jan 24, 2020 11:56 am

Re: Right way to add option for new transport

Post by postfixlover »

L. Mark Stone wrote:Please post the output from:

zmcontrol -v

...run as the zimbra user.
[zimbra]$ zmcontrol -v
Release 8.8.15_GA_3829.RHEL7_64_20190718141144 RHEL7_64 NETWORK edition.
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2802
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.7 Network Edition
Contact:

Re: Right way to add option for new transport

Post by L. Mark Stone »

Modifications via zmprov can only be made to the supported attributes listed in the wiki article. So like:

Code: Select all

zmprov ms `zmhostname` zimbraMtaFallbackRelayHost <whatver you want to set this to>
Zimbra then modifies all the Postfix and other configuration files accordingly. Indeed, most config files are rewritten at every service restart, so editing main.cf will not survive a zmmtactl restart command.

Since no concurrency variables are listed in the wiki article I referenced (nor in this one: https://wiki.zimbra.com/wiki/Postconf_keys), you'll need to use postconf -e to set the Postfix variables as you wish.

Please note that this is unsupported, may break things (depending on what you set) and may not survive service restarts nor upgrades. There is a wiki article that shows an unsupported method for endeavoring to ensure that custom changes like this are not overwritten during upgrades; basically you create a new localconfig variable prepended with "postfix_" and ending with the Postfix parameter you want to change for which there is no global config variable. https://wiki.zimbra.com/wiki/Tuning_Pos ... e_Settings

My approach is to avoid doing these kinds of things; I advise customers that Zimbra is an appliance we get to peek inside of, but that changing things outside of "the Zimbra Way" creates risk -- risk that in this case could be avoided perhaps by deploying a separate Postfix gateway/relay server for Zimbra to use. I'll also say that the most fubar'd Zimbra systems I have come across in my 14 years of dealing with Zimbra are the ones maintained by sysadmins with advanced Linux/Postfix/MySQL foo. They know what they are doing as regards the individual open source component, but customize it outside of the range expected by Zimbra, thereby creating unintended negative consequences.

Hope that helps,
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
postfixlover
Posts: 5
Joined: Fri Jan 24, 2020 11:56 am

Re: Right way to add option for new transport

Post by postfixlover »

Thank you very mush. I'm a little bit confused.
i can modify master.cf.in files, adding a new transport.
From postfix > 2.5 (cfr http://www.postfix.org/QSHAPE_README.html) every options for new transport need to be configured accordling to documentation.
So if i create a new "fragile" transport, i can and even i need, to modify fragile_ option available for postfix but with zimbra i can't.

I know this can be un unsupported practices, but it's impossibile to modify master.cf.in and are not able to modify new option.

I think Zimbra must reproduce postfix behavior. If a new transport is addedd to master.cf.in, every new variable must be modified by zmprov.
Btw, I really appreciate your suggestions and, aware of the risk, i do the job with postconf.

Thank you very very very much. my problem is solved and setting for new transport will survive to "zmcontrol restart".
I also alerted my team to become aware of the risk involved in a future update (minor or major) of Zimbra.
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2802
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.7 Network Edition
Contact:

Re: Right way to add option for new transport

Post by L. Mark Stone »

I'm glad your problem is solved!

*.in files are mostly left over from when Zimbra was first born... :-)

They hold a mix of hard-coded values and global config variables. Each time a Zimbra service is started, Zimbra's configd daemon destroys the existing .conf, .cf, (or whatever) file, and rebuilds it using the companion .in file.

Many users (myself included) make alterations to, for example, amavisd.conf.in to do things like whitelist the Daily Mail Report. When the MTA packages are upgraded, Zimbra may provision a new amavisd.conf.in file. A in-place Zimbra upgrade always will rewrite amavis.conf.in.

Postfix is more complex, because for a time Zimbra had a zmmta.cf.in file, which made what you are trying to do easier. But, Zimbra has been (like in 8.5) moving away from .in files in favor of global config (LDAP) variables and localconfig (localconfig.xml) variables as needed. New Global Config variables are not provisionable by users; they are typically part of the proprietary Zimbra LDAP Schema as I understand it. Users can provision new Local Config variables, but Zimbra isn't always aware that it should use them. Again, Postfix is a "special case".

All I can recommend is to keep a lookout, and be prepared to redeploy your customizations if a monthly Patch includes MTA package updates.

All the best,
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
postfixlover
Posts: 5
Joined: Fri Jan 24, 2020 11:56 am

Re: Right way to add option for new transport

Post by postfixlover »

thank you very much. your trick work like a charm.
Post Reply