IRONMAIDEN 8.0.3 ported to FreeBSD

Ask questions about your setup or get help installing ZCS server (ZD section below).
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

zcs-8.0.4-FreeBSD-9.1_amd64 packages, r5718, just download, extract and run "./FreeBSD_amd64.install.sh".
zcs-8.0.3-FreeBSD-9.1_amd64 packages, r5647, just download, extract and run "./FreeBSD_amd64.install.sh".
install.log, proof that it really works on fresh installation.
VMware Player compatible FreeBSD-9.1 amd64 image for anyone wishing a quick test. Login is 'root'. Password is 'zimbra'. Just 'cd /install && ./install.sh'. There is no swap, 4GB RAM is minimum.

I have been using zimbra on FreeBSD since 5.0.16, it is amazing piece of software. Thank you.
For all those "this is not a real FreeBSD port" people: it is not meant to be.
Have You seen "apt-get install zimbra" lately? No? See.. they (the Linux people) got official support and they still do it the old fashioned way. Why maintain separate installers?!
http://wiki.zimbra.com/wiki/Zimbra_on_F ... _.21.21.21"]
wiki with all the info. Building and Installation is now super easy. And you can now fetch dependencies remotely:

setenv PACKAGEROOT http://zimbra-pkg-proxy.imladris.sk[/url] &&

pkg_add -r zimbra-builddeps-8.0.3


Beat that, Linux :P.
Please, make this sticky as well.
Ary Firman
Posts: 8
Joined: Sat Sep 13, 2014 3:12 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by Ary Firman »

Wow, thank you for your hard work to provide ZCS on FreeBSD, i'am using your zcs 7.2.0 on production level and it has been working great. I'am just facing the memory consumption is increasing day by day but not to much (displayed using "top" command).
I hope this new packages will have better memory management..
Just wondering about 8.0.4 packages, since zimbra official do not release the 8.0.4 yet. Any explanation about it..?
cheers,
Ary Firman
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

[quote user="Ary Firman"]

I hope this new packages will have better memory management..

[/QUOTE]

my university mail server is running 7.2.0+zfs raid6 with current uptime of 198 days. what memory consumption exactly?

i don't use swap (yes, for a reason) on any of my production zimbra servers and i have never had any problems with only 10GB RAM. remember, FreeBSD (unlike Linux which insta-worries most Linux->BSD newcomers) will try to use almost all available memory: "Free memory is wasted memory." and I can only second that :-). "Free" memory is memory that is not used for anything right now, for most situations you should do "Inactive+Free"=free. So it's pretty much okay to have it as low as possible for most systems (depends on UFS/ZFS and other things).

that being said, using zimbra with less than 4GB RAM is a very bad idea.
[quote user="Ary Firman"]

Just wondering about 8.0.4 packages, since zimbra official do not release the 8.0.4 yet. Any explanation about it..?

[/QUOTE]

there is no official release yet but i wanted to prepare a patch beforehand. there are not many major changes in releases, only between them and 8.0.4 contains updated 3rd party.

besides i tend to forget about new releases since noone ever bugs me about them. i should create some 'nightly zimbra build' box.

but best answer to your question would be 8.0.4 pmweb. if any bug catches your eye.. you might want 8.0.4.
Ary Firman
Posts: 8
Joined: Sat Sep 13, 2014 3:12 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by Ary Firman »

Thanks for your response, I do not really like if Zimbra use swap memory, CMIIW it will make system slow, but after 61 days uptime i still have not seen my zimbra using Swap Memory
Perl, Java and mysqld tends to increase the memory usage


I'am using HP ML 150 G6 with 4GB RAM running FreeBSD 9.1 & Zimbra 7.2.0 and only run LDAP, Mailbox, MTA, AntiSPAM and Logger service (without antivirus, spell checker, snmp)


When I get new Hardware test, I will try your 8.0.4 packages soon, and give some report if there is bugs appear on that. One thing that makes me happy, there is builtin OpenDKIM on 8.0.3/8.0.4, since i dont have to install and configure it manually anymore
Again, thank you for your effort, i don't mind if you put Donation Button on your webpage....oops sorry if this is wrong
cheers,
Ary Firman
[zimbra@mail ~]$ zmcontrol -v

Release 7.2.0_GA_2678.FreeBSD_amd64 FreeBSD_amd64 FOSS edition.
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

[quote user="Ary Firman"]Thanks for your response, I do not really like if Zimbra use swap memory, CMIIW it will make system slow, but after 61 days uptime i still have not seen my zimbra using Swap Memory

[/QUOTE]

so as i see it.. there is no problem at all and what you are experiencing is utterly normal? :)
i wish that more people would actually report their "success stories". that's the only thing that matters really. proving that it works. besides, it makes me happy in the long run :).
dijichi2
Elite member
Elite member
Posts: 1133
Joined: Fri Sep 12, 2014 10:00 pm

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by dijichi2 »

great job solko, well done on keeping this port going. i used to maintain a solaris port but gave up years ago as it was so painful to keep synched with the main tree. why don't you ask zimbra to see if it's possible to merge your code into the main codebase, can only be a good thing for the project, right?
[QUOTE]i don't use swap (yes, for a reason) on any of my production zimbra servers and i have never had any problems with only 10GB RAM. remember, FreeBSD (unlike Linux which insta-worries most Linux->BSD newcomers) will try to use almost all available memory: "Free memory is wasted memory." and I can only second that :-). Free memory is memory that has never been used so it's pretty much okay to have it as low as possible for most systems (depends on UFS/ZFS and other things).[/QUOTE]
not sure what you mean by this - linux will use every bit of memory it can get it's hands on for cacheing, buffering etc. it often confuses newbies to linux as they think it's a memory hog. free memory (on linux, at least) isn't memory that's never been used - it just means it isn't currently allocated. if you kill/stop a process that was using a significant amount of memory, at least a portion of the memory allocation will be freed and show as free memory. linux, like BSD has had many public discussions over the years around balancing the VM part of the kernel related to memory and swap usage. it's not often that having no swap is a good idea - it's a vital safety net for most OS environments. for example if you have a sudden rush of incoming mail (say a spam run) that fires off a lot of amavis/SA/policyd/postfix/lmtp activity and you reach up to your memory limit, what happens if you have no swap? the kernel will be forced to start killing off vital processes. swap, although very slow, at least gives the OS a chance to get through the sudden rush of spam and then recover. the kernel VM also has settings to change the behaviour of swapping (sys.vm.swappiness in linux) so you can dissuade it from swapping unless absolutely necessary. in addition, swap is a good (cheap) way of getting large allocated processes out of expensive ram that are rarely used. i've had reasonable performance from very small VM or zones running zimbra where a large chunk of the main java and mysql processes are swapped out and most of the RAM is used for active processing.
anyway, great job on the port!
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

[quote user="dijichi2"]great job solko, well done on keeping this port going. i used to maintain a solaris port but gave up years ago as it was so painful to keep synched with the main tree. why don't you ask zimbra to see if it's possible to merge your code into the main codebase, can only be a good thing for the project, right?

[/QUOTE]

i actually asked, at least i think so. guess i was not direct enough? my thinking was.. if they want to integrate the patch, they will say so. and they might want to do it but the management might not. in either case, they know about the possibility for sure.

and i don't think the zimbra team would have trouble porting to FreeBSD by themselves if they decided to. policy. politics. whatever. we just have to endure.

maybe i should send the "zimbra contribution agreement" in and wait for a miracle to happen.
[quote user="dijichi2"]

not sure what you mean by this - linux will use every bit of memory it can get it's hands on for cacheing, buffering etc. it often confuses newbies to linux as they think it's a memory hog. free memory (on linux, at least) isn't memory that's never been used - it just means it isn't currently allocated. if you kill/stop a process that was using a significant amount of memory, at least a portion of the memory allocation will be freed and show as free memory. linux, like BSD has had many public discussions over the years around balancing the VM part of the kernel related to memory and swap usage.

[/QUOTE]

well, linux and bsd is different. and that is always confusing. i try not to use Linux much (because i am not proficient in it) but i have seen the "linux -> bsd memory" question too many times. "why is my bsd system using so much memory while my linux system is using way less memory?" etc.

bsd got this "inactive" memory which can be freed at any time so the really "free" memory is usually kept very low. and yes, i stand corrected, "free" memory on FreeBSD is the actual "free" memory and it definitely does not mean it has never been used. i should stop believing everything i read on the internet.
[quote user="dijichi2"]

it's not often that having no swap is a good idea - it's a vital safety net for most OS environments. for example if you have a sudden rush of incoming mail (say a spam run) that fires off a lot of amavis/SA/policyd/postfix/lmtp activity and you reach up to your memory limit, what happens if you have no swap? the kernel will be forced to start killing off vital processes. swap, although very slow, at least gives the OS a chance to get through the sudden rush of spam and then recover. the kernel VM also has settings to change the behaviour of swapping (sys.vm.swappiness in linux) so you can dissuade it from swapping unless absolutely necessary. in addition, swap is a good (cheap) way of getting large allocated processes out of expensive ram that are rarely used. i've had reasonable performance from very small VM or zones running zimbra where a large chunk of the main java and mysql processes are swapped out and most of the RAM is used for active processing.[/QUOTE]

thank you for your explanation.

while we both agree that swap is a great thing to have (generally), i can't agree that it is a great thing to actively use on most modern production FreeBSD servers. i mean, why would someone want that? a alot of active swapping is usually a strong system hint that something is very wrong. one should definitely try to tune one's system so that it does not need swap (if the one got the minimum required memory to do so - that is the one and only exception in my opinion).

as for the example you mentioned. the real answer would be.. nothing at all as you are not supposed to reach the memory limit. sudden rush of emails (i am thinking several thousands messages) will (at least from my experience with postfix+amavis+clamav but without zimbra) kill your system in disk activity (even if you put all your amavis temporary content into memory, hello, zfs raid6). amavis is preforked and will most probably not spawn new memory hungry processes and postfix is nothing short of amazing (and i have been using qmail for a very long time).

a little bit testing on my university mail server (zimbra7) just now..
i have sent 5000 identical 18KB (normal text, not garbage) message within 30 seconds via localhost (no SMTP submission). the messages got delivered within 3 minutes to the local mailbox.

before:

Mem: 1952M Active, 2333M Inact, 7098M Wired, 4084K Cache, 479M Free

after 3 minutes:

Mem: 3732M Active, 744M Inact, 5392M Wired, 7392K Cache, 1991M Free
i have sent 1000 identical message from a nonwhitelisted ip address via SMTP (took few minutes though) and the result is quite similar. so unless you have very low RAM, you should be okay.
i will be happy to pursue this topic further off forums so that people won't get under false impressions of starting some flame wars. i am not saying you are wrong, i am just saying that "it depends". that being said, i am not a conventional person in any way. if i was, i would install all my zimbra servers on linux :-).
[quote user="dijichi2"]

anyway, great job on the port![/QUOTE]

creating the "nightly builds" box right now. that should keep this project going. hopefully.
Ishtar
Posts: 2
Joined: Sat Sep 13, 2014 3:12 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by Ishtar »

Thanks a lot for this. All your hard work means so much less work for us!
I'm new to setting up Zimbra (have been using it as plain user for a while now) and I'm trying to set it up with IPv6 only. The problem is that the installer's looking for an MX record but doesn't seem to know how to parse IPv6. Is this the Zimbra installer? Like I said, I'm completely new to this, just googled for a freebsd howto.
Let me explain. Everything works fine until it checks for the domain name. The server's hostname obviously has no MX record, so I enter the domain I want to use, which only has an IPv6 MX record. So it runs its check and fails with the following error:

DNS ERROR - none of the MX records for domain.com

resolve to this host

It is suggested that the MX record resolve to this host

Re-Enter domain name? [Yes]

Create domain: [hostname.domain.com] domain.com

MX: mail.domain.com (2001:xxx:xxx:xxx:0:0:0:3a11)

It then prints all the IPs, which I won't for sanity's sake, saying that it's not on this server. I replaced parts with xxx but the obvious is there, it's looking for 0's since that's how it gets resolved for some reason. The IP on the server is actually 2001:xxx:xxx:xxx::3a11 so it doesn't detect it and fail.
I'm not sure about the result of either ignoring this error and proceeding, or using a valid IPv4 domain then changing it later... Just wanted to report this issue. I'll try something and update my findings. Thanks again for your work!


Update: One of the zimbra packages breaks portupgrade/pkgdb. It goes in an endless loop:

# pkgdb -fF

---> Checking the package registry database

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

etc...
Running on 9.1-RELEASE-p1 with latest port tree. I removed all the packages because I needed to do an upgrade, not sure which one causes the problem. I'll update again if I have more time to play around with it.
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

[quote user="Ishtar"]

Let me explain. Everything works fine until it checks for the domain name. The server's hostname obviously has no MX record, so I enter the domain I want to use, which only has an IPv6 MX record. So it runs its check and fails with the following error:

I'm not sure about the result of either ignoring this error and proceeding, or using a valid IPv4 domain then changing it later... Just wanted to report this issue. I'll try something and update my findings. Thanks again for your work![/QUOTE]

thanks for the info.

sadly, i have never tried ipv6 with zimbra (i know that it should work and that it is basically beta) and yes, i had to patch the installer so that it finds network interfaces on freebsd - obviously i have not paid much attention to ipv6, sorry about that :(.

if you are sure that your DNS is set up correctly, then yes, you should be able to go ahead and ignore the installer warning message completely. remember, freebsd version should not be (very) different from the linux version but ipv6 is really something i cannot guarantee (yet).
i will create an ipv6 enabled host and test it in a very near future.
solko
Advanced member
Advanced member
Posts: 93
Joined: Sat Sep 13, 2014 12:22 am

IRONMAIDEN 8.0.3 ported to FreeBSD

Post by solko »

[quote user="Ishtar"]

Update: One of the zimbra packages breaks portupgrade/pkgdb. It goes in an endless loop:

# pkgdb -fF

---> Checking the package registry database

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

[Rebuilding the pkgdb in /var/db/pkg ... - 966 packages found (-0 +966) error] Remove and try again.

etc...
Running on 9.1-RELEASE-p1 with latest port tree. I removed all the packages because I needed to do an upgrade, not sure which one causes the problem. I'll update again if I have more time to play around with it.[/QUOTE]

huh. i never thought about this. ever. the zimbra freebsd packages were only "a native delivery method" (with scripts and all that) and they have no dependencies specified so they should be pretty much safe. could you, please, try to estimate exact steps you have taken so that i could reproduce this issue?
i am having some troubles with deploying my zimbra test boxes. hyperv+freebsd is a new territory for me. so hang on, please :).
EDIT: how do you know that zimbra packages are responsible for the problem? i mean, you have 966 packages and portupgrade should not even consider zimbra-packages at all.
Post Reply