error in network service after one wrong login

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
stegbth
Advanced member
Advanced member
Posts: 141
Joined: Fri Sep 12, 2014 11:48 pm
Location: Bavaria in Germany

error in network service after one wrong login

Post by stegbth »

Hi,

i am running Zimbra Release 8.6.0.GA.1153.UBUNTU14.64 UBUNTU14_64 FOSS edition, Patch 8.6.0_P6.
The user get authenticated against a Windows 2008R2 DC (LDAP Port 3268).
This works most time.

Sometime random user try to login and get immediatly "error in network service"
After waiting some time the login is working again.

in the mailbox.log i see
2016-05-11 08:20:57,051 INFO [qtp509886383-1216983:http://127.0.0.1:80/service/soap/AuthRequest] [name=mgloss@mydomain.hq;oip=10.58.0.91;ua=zclient/8.6.0_GA_1194;] SoapEngine - handler exception: authentication failed for [mgloss@mydomain.hq], external LDAP auth failed, LDAP error: - unable to ldap authenticate: 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1
2016-05-11 08:20:57,051 INFO [qtp509886383-1216983:http://127.0.0.1:80/service/soap/AuthRequest] [name=mgloss@mydomain.hq;oip=10.58.0.91;ua=zclient/8.6.0_GA_1194;] soap - AuthRequest elapsed=5
2016-05-11 08:21:00,922 INFO [qtp509886383-1216982:http://127.0.0.1:80/service/soap/AuthRequest] [] misc - Access to IP 10.58.50.91suspended, for repeated failed login.

There are NO authentication errors before!

The interesting part is:

Code: Select all

zmprov gacf |grep -i InvalidLoginFilter
zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating: 15
zimbraInvalidLoginFilterMaxFailedLogin: 10
zimbraInvalidLoginFilterMaxSizeOfFailedIpDb: 7000
zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5
so the question is:
what is the timerange for the 10 invalid logins?

How can it happen, after the first invalid login to hit the CSFE_SVC_ERROR ?

best regards
Thomas
Last edited by stegbth on Wed May 11, 2016 2:50 pm, edited 1 time in total.
Release 8.6.0.GA.1153.UBUNTU14.64 FOSS edition, Patch 8.6.0_P6
User avatar
vavai
Advanced member
Advanced member
Posts: 174
Joined: Thu Nov 14, 2013 2:41 pm
Location: Indonesia
ZCS/ZD Version: 0
Contact:

Re: error in network service after one wrong login

Post by vavai »

Hi Thomas,
stegbth wrote:Hi,

i am running Zimbra Release 8.6.0.GA.1153.UBUNTU14.64 UBUNTU14_64 FOSS edition, Patch 8.6.0_P6.
The user get authenticated against a Windows 2008R2 DC (LDAP Port 3268).
This works most time.

Sometime random user try to login and get immediatly "error in network service"
After waiting some time the login is working again.

in the mailbox.log i see
2016-05-11 08:20:57,051 INFO [qtp509886383-1216983:http://127.0.0.1:80/service/soap/AuthRequest] [name=mgloss@mydomain.hq;oip=10.58.0.91;ua=zclient/8.6.0_GA_1194;] SoapEngine - handler exception: authentication failed for [mgloss@mydomain.hq], external LDAP auth failed, LDAP error: - unable to ldap authenticate: 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1
2016-05-11 08:20:57,051 INFO [qtp509886383-1216983:http://127.0.0.1:80/service/soap/AuthRequest] [name=mgloss@mydomain.hq;oip=10.58.0.91;ua=zclient/8.6.0_GA_1194;] soap - AuthRequest elapsed=5
2016-05-11 08:21:00,922 INFO [qtp509886383-1216982:http://127.0.0.1:80/service/soap/AuthRequest] [] misc - Access to IP 10.58.50.91suspended, for repeated failed login.

There are NO authentication errors before!

The interesting part is:
zmprov gacf |grep -i InvalidLoginFilter
zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating: 15
zimbraInvalidLoginFilterMaxFailedLogin: 10
zimbraInvalidLoginFilterMaxSizeOfFailedIpDb: 7000
zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5

so the question is:
what is the timerange for the 10 invalid logins?

How can it happen, after the first invalid login to hit the CSFE_SVC_ERROR ?

best regards
Thomas
It seems that the network service error are came from DoSFilter False Positive : https://wiki.zimbra.com/wiki/DoSFilter

DoSFilter itself not only bound to invalid login counts but also with maximum requests per second.

Actually, I'm using following command to prevent any DoSFilter false positive :D

Code: Select all

zmprov ms `zmhostname` zimbraInvalidLoginFilterMaxFailedLogin 0
zmprov mcf zimbraInvalidLoginFilterMaxFailedLogin 0
zmprov mcf zimbraHttpDosFilterDelayMillis 0
stegbth
Advanced member
Advanced member
Posts: 141
Joined: Fri Sep 12, 2014 11:48 pm
Location: Bavaria in Germany

Re: error in network service after one wrong login

Post by stegbth »

Hi vavai,

thank you for your response. I think the same.
But if possible i would appreciate to keep the DosFilter running for security reasons.

As wrote before the DosFilter is defined by default for 10 invalid logins. but sometimes it blocks after the first invalid login.
And the login comes from a normal user pc, no terminalserver.

best regards
Thomas
Release 8.6.0.GA.1153.UBUNTU14.64 FOSS edition, Patch 8.6.0_P6
User avatar
vavai
Advanced member
Advanced member
Posts: 174
Joined: Thu Nov 14, 2013 2:41 pm
Location: Indonesia
ZCS/ZD Version: 0
Contact:

Re: error in network service after one wrong login

Post by vavai »

Hi Thomas,
stegbth wrote:Hi vavai,

thank you for your response. I think the same.
But if possible i would appreciate to keep the DosFilter running for security reasons.

As wrote before the DosFilter is defined by default for 10 invalid logins. but sometimes it blocks after the first invalid login.
And the login comes from a normal user pc, no terminalserver.

best regards
Thomas
Is this possible that DoSFilter not only related with invalid logins as yours but also with zimbraHttpDosFilterMaxRequestsPerSec, so it still being blocked even if only one failed/invalid login attempt.
stegbth
Advanced member
Advanced member
Posts: 141
Joined: Fri Sep 12, 2014 11:48 pm
Location: Bavaria in Germany

Re: error in network service after one wrong login

Post by stegbth »

Hi,

Code: Select all

zmprov gacf |grep -i DosFilter
zimbraHttpDosFilterDelayMillis: -1
zimbraHttpDosFilterMaxRequestsPerSec: 30
nevertheless i will try to the MaxRequestperSec to 100 and test again.
Also i modified jetty-distribution-9.1.5.v20140505/webapps/zimbra/WEB-INF/classes/messages/ZMsg_de.properties
to notify the user to retry in 15 minutes if the get hit.

best regards
Thomas
Release 8.6.0.GA.1153.UBUNTU14.64 FOSS edition, Patch 8.6.0_P6
Hala
Posts: 31
Joined: Tue Apr 05, 2016 6:12 am

Re: error in network service after one wrong login

Post by Hala »

I face the same problem from some users randomly

my configuration
[zimbra@xxxx ~]$ zmprov gacf |grep -i InvalidLoginFilter
zimbraInvalidLoginFilterDelayInMinBetwnReqBeforeReinstating: 15
zimbraInvalidLoginFilterMaxFailedLogin: 10
zimbraInvalidLoginFilterMaxSizeOfFailedIpDb: 7000
zimbraInvalidLoginFilterReinstateIpTaskIntervalInMin: 5

[zimbra@xxxx ~]$ zmprov gacf |grep -i zimbraHttpDosFilter
zimbraHttpDosFilterDelayMillis: 20
zimbraHttpDosFilterMaxRequestsPerSec: 100

I dont want to change the default zimbraInvalidLoginFilterMaxFailedLogin: 10 for security purpose

How you can help us to sovle this problem
User avatar
vavai
Advanced member
Advanced member
Posts: 174
Joined: Thu Nov 14, 2013 2:41 pm
Location: Indonesia
ZCS/ZD Version: 0
Contact:

Re: error in network service after one wrong login

Post by vavai »

Hi Thomas,
stegbth wrote:Hi,

Code: Select all

zmprov gacf |grep -i DosFilter
zimbraHttpDosFilterDelayMillis: -1
zimbraHttpDosFilterMaxRequestsPerSec: 30
nevertheless i will try to the MaxRequestperSec to 100 and test again.
Also i modified jetty-distribution-9.1.5.v20140505/webapps/zimbra/WEB-INF/classes/messages/ZMsg_de.properties
to notify the user to retry in 15 minutes if the get hit.

best regards
Thomas
Sound good, please let us know if the above are correctly suspected source of problem. Modifying ZMsg_de.properties is a nice idea to suggest a temporary workaround for them :)
User avatar
altmas5
Posts: 7
Joined: Sat Sep 13, 2014 2:27 am
Contact:

Re: error in network service after one wrong login

Post by altmas5 »

Hello Folks,

I also got a similar problem with LDAP Auth.
AFAIK is not related to DoSFilter (No other messages from this module show up)

Code: Select all

2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [name=account@domain.com.ni;oip=192.168.1.204;ua=zclient/8.6.0_GA_1153;] SoapEngine - handler exception: authentication failed for [account@domain.com.ni], external LDAP auth failed, LDAP error:  - unable to ldap authenticate: 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 531, vece
2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [name=account@domain.com.ni;oip=192.168.1.204;ua=zclient/8.6.0_GA_1153;] soap - AuthRequest elapsed=9
2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [] misc - Invalid login filter, checking if this was an auth req and authentication failed.
2016-05-19 19:36:03,156 INFO  [qtp509886383-5377:https://127.0.0.1:7071/service/admin/soap/GetDomainInfoRequest] [ip=127.0.0.1;ua=ZCS/8.6.0_GA_1153;] soap - GetDomainInfoRequest elapsed=0
It randomly occurs on the first auth request.

Any idea?
User avatar
vavai
Advanced member
Advanced member
Posts: 174
Joined: Thu Nov 14, 2013 2:41 pm
Location: Indonesia
ZCS/ZD Version: 0
Contact:

Re: error in network service after one wrong login

Post by vavai »

Hi,
altmas5 wrote:Hello Folks,

I also got a similar problem with LDAP Auth.
AFAIK is not related to DoSFilter (No other messages from this module show up)

Code: Select all

2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [name=account@domain.com.ni;oip=192.168.1.204;ua=zclient/8.6.0_GA_1153;] SoapEngine - handler exception: authentication failed for [account@domain.com.ni], external LDAP auth failed, LDAP error:  - unable to ldap authenticate: 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 531, vece
2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [name=account@domain.com.ni;oip=192.168.1.204;ua=zclient/8.6.0_GA_1153;] soap - AuthRequest elapsed=9
2016-05-19 19:36:03,142 INFO  [qtp509886383-5376:http://127.0.0.1:80/service/soap/AuthRequest] [] misc - Invalid login filter, checking if this was an auth req and authentication failed.
2016-05-19 19:36:03,156 INFO  [qtp509886383-5377:https://127.0.0.1:7071/service/admin/soap/GetDomainInfoRequest] [ip=127.0.0.1;ua=ZCS/8.6.0_GA_1153;] soap - GetDomainInfoRequest elapsed=0
It randomly occurs on the first auth request.

Any idea?
Better to create new thread instead, because your problem are related to external auth, not with DoSFilter as op described on this thread.
User avatar
altmas5
Posts: 7
Joined: Sat Sep 13, 2014 2:27 am
Contact:

Re: error in network service after one wrong login

Post by altmas5 »

It's OK, I took a deeper look to the logs and found out that

Code: Select all

AcceptSecurityContext error, data 531
is a AD-specific error code which means "not permitted to logon at this workstation".
So the users we're restricted to the station were they can login and my zimbra server wasn't one of those stations.
Post Reply