very high cpu

[maumar]

Hello
on a Zcs 8.7.1 we suffered a power outage fault, /opt was very damaged but at last we was able to rescue.
But zimbra has got some damage
After a lot of works, after we migrated to a new server with rsync, with 8.8.9
now the situation is this:

Zimbra OS 8.8.9 on CentOS7
RAM 12 Giga
8 Cpu

we have cpu very high, load averae very low, a lot of thread

Code: Select all

zmlocalconfig | grep thread
compute_aggregate_quota_threads = 10
imap_thread_keep_alive_time = 60
ldap_common_threads = 8
ldap_common_toolthreads = 2
mailboxd_thread_stack_size = 256k
milter_thread_keep_alive_time = 60
pop3_thread_keep_alive_time = 60
thread_pool_warn_percent = 100
xmpp_server_outgoing_max_threads = 20
xmpp_server_processing_core_threads = 2
xmpp_server_processing_max_threads = 50
zimbra_index_threads = 10
zimbra_mailbox_lock_max_waiting_threads = 100
zimbra_reindex_threads = 10

Code: Select all

zmprov -l gs $(zmhostname) | grep -i Threads
zimbraAdminImapImportNumThreads: 20
zimbraClamAVMaxThreads: 10
zimbraHttpNumThreads: 2000
zimbraHttpSSLNumThreads: 100
zimbraImapNumThreads: 100
zimbraLmtpNumThreads: 20
zimbraMilterNumThreads: 100
zimbraPop3NumThreads: 100
zimbraScheduledTaskNumThreads: 20

this is top -H

Code: Select all

top - 11:51:39 up 5 days, 17:46,  2 users,  load average: 43.00, 38.61, 32.14
Threads: 674 total,  48 running, 626 sleeping,   0 stopped,   0 zombie
%Cpu(s): 98.1 us,  0.4 sy,  0.0 ni,  1.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.1 st
KiB Mem : 12137624 total,  2626144 free,  6561604 used,  2949876 buff/cache
KiB Swap:  3354620 total,  3307764 free,    46856 used.  4522868 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 4739 zimbra    20   0   10.2g   2.4g  21588 R 28.5 20.9   2:20.35 java
23118 zimbra    20   0   10.2g   2.4g  21588 R 20.1 20.9  14:50.86 java
31061 zimbra    20   0   10.2g   2.4g  21588 R 20.1 20.9   8:51.11 java
23019 zimbra    20   0   10.2g   2.4g  21588 R 19.7 20.9  15:45.97 java
 5486 zimbra    20   0   10.2g   2.4g  21588 R 18.8 20.9   1:51.61 java
23055 zimbra    20   0   10.2g   2.4g  21588 R 18.5 20.9  17:07.57 java
  830 zimbra    20   0   10.2g   2.4g  21588 R 18.5 20.9   2:06.42 java
23001 zimbra    20   0   10.2g   2.4g  21588 R 18.2 20.9  12:34.87 java
28833 zimbra    20   0   10.2g   2.4g  21588 R 18.2 20.9   0:06.65 java
30285 zimbra    20   0   10.2g   2.4g  21588 R 18.2 20.9   6:29.71 java
 2648 zimbra    20   0   10.2g   2.4g  21588 R 18.2 20.9   4:13.73 java
32393 zimbra    20   0   10.2g   2.4g  21588 R 17.9 20.9   6:53.49 java
 2649 zimbra    20   0   10.2g   2.4g  21588 R 17.9 20.9   3:08.99 java
 4091 zimbra    20   0   10.2g   2.4g  21588 R 17.9 20.9   2:49.81 java
29556 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9  11:44.02 java
29576 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9   6:03.02 java
29583 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9  11:15.79 java
 1319 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9   3:57.98 java
 4049 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9   1:39.76 java
 6172 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9   1:14.06 java
 6859 zimbra    20   0   10.2g   2.4g  21588 R 17.6 20.9   0:48.76 java
29584 zimbra    20   0   10.2g   2.4g  21588 R 17.2 20.9  10:30.39 java
29614 zimbra    20   0   10.2g   2.4g  21588 R 17.2 20.9  10:50.52 java
31109 zimbra    20   0   10.2g   2.4g  21588 R 17.2 20.9   4:57.22 java
31767 zimbra    20   0   10.2g   2.4g  21588 R 17.2 20.9   7:18.19 java
 6821 zimbra    20   0   10.2g   2.4g  21588 R 17.2 20.9   1:00.54 java
23127 zimbra    20   0   10.2g   2.4g  21588 R 16.9 20.9  12:50.75 java
31729 zimbra    20   0   10.2g   2.4g  21588 R 16.9 20.9   7:45.85 java
 2795 zimbra    20   0   10.2g   2.4g  21588 R 16.9 20.9   3:44.90 java
 2981 zimbra    20   0   10.2g   2.4g  21588 R 16.9 20.9   2:38.74 java
28157 zimbra    20   0   10.2g   2.4g  21588 R 16.6 20.9  13:43.64 java
  641 zimbra    20   0   10.2g   2.4g  21588 R 16.6 20.9   5:37.68 java
 6820 zimbra    20   0   10.2g   2.4g  21588 R 16.6 20.9   0:26.52 java
 6989 zimbra    20   0   10.2g   2.4g  21588 R 16.6 20.9   0:37.71 java
 8182 zimbra    20   0   10.2g   2.4g  21588 R 16.6 20.9   0:14.03 java
29615 zimbra    20   0   10.2g   2.4g  21588 R 16.3 20.9   9:53.72 java
 1248 zimbra    20   0   10.2g   2.4g  21588 R 16.3 20.9   4:34.18 java
 3421 zimbra    20   0   10.2g   2.4g  21588 R 16.3 20.9   3:23.81 java

we are wondering what is thread cosuming....
any advice on this ?

Best regards

[L. Mark Stone]

Something is very wrong; you shouldn't have that many Java processes running.

Normally I would suggest grepping the logs for errors and then fixing what you find. But, since you said you migrated to this new system via rsync, I suspect some of the corruption from your old system has carried over to the new system.

I'd still go through the logs to see what you can fix easily, but if the system is running, just slowly, your best bet might be to migrate to another new 8.8.9 system, by using the ZeXtras Suite to do so.

In that way you will have a very clean system, with no risk of any corruption from the old system coming over.

Hope that helps,
Mark