VirusTotal check for Zimbra emails

[maxxer]

I was mistaken. Content of /etc/systemd/system/amavis-vtd.service is

Code: Select all

[Unit]
Description=AmavisVTd

[Service]
ExecStart=/usr/bin/python3 /usr/local/lib/python3.5/dist-packages/amavisvt-0.5.3-py3.5.egg/amavisvt/amavisvtd.py
Restart=always
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=amavisvtd
# Run as different user?
User=zimbra

[Install]
WantedBy=multi-user.target


[zimico]

Thank Maxxer,
The service file seems to be ok. I have to change from python 3 to 3.6 and corresponding path as the following:

Code: Select all

ExecStart=/usr/bin/python3.6 /usr/local/lib/python3.6/site-packages/amavisvt-0.5.3-py3.6.egg/amavisvt/amavisvtd.py

The service starts successfully. However, unfortunately it can not connect to virustotal because of a lot of errors while execute .py file:

Code: Select all

Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: 2019-11-11 21:41:33,221 ERROR   [Thread-2] Error asking virustotal about files
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: Traceback (most recent call last):
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: (self._dns_host, self.port), self.timeout, **extra_kw
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: raise err
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: sock.connect(sa)
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: socket.timeout: timed out
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: During handling of the above exception, another exception occurred:
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: Traceback (most recent call last):
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: chunked=chunked,
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: self._validate_conn(conn)
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: conn.connect()
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 334, in connect
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: conn = self._new_conn()
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 164, in _new_conn
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: % (self.host, self.timeout),
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.VerifiedHTTPSConnection object at 0x7f3ffc1c6278>, 'Connection to www.virustotal.com timed out. (connect timeout=10.0)')
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: During handling of the above exception, another exception occurred:
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: Traceback (most recent call last):
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: timeout=timeout
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment
Nov 11 21:41:33 mail.zimilab.com amavisvtd[4273]: raise MaxRetryError(_pool, url, error or ResponseError(cause))

Do you have any idea?
Regards,
Minh.

[Peter Parker]

Hi Maxxer,

I successfully started services as your guide, and now, I'm getting the error as below, I can't figure out, could you please give me some advice.

Code: Select all


[root@mail ~]# journalctl -fu amavis-vtd
-- Logs begin at Mon 2019-11-11 00:05:42 +07. --
Nov 12 15:06:20 mail.zoholab.com amavisvtd[16553]: 2019-11-12 15:06:20,394 ERROR   [MainThread] Socket /opt/zimbra/data/clamav/amavisvtd.sock isn't working: [Errno 111] Connection refused
Nov 12 15:06:20 mail.zoholab.com amavisvt[16553]: MainThread - Socket /opt/zimbra/data/clamav/amavisvtd.sock isn't working: [Errno 111] Connection refused
Nov 12 15:08:16 mail.zoholab.com systemd[1]: Stopping AmavisVTd...
Nov 12 15:08:16 mail.zoholab.com systemd[1]: Stopped AmavisVTd.
Nov 12 15:08:16 mail.zoholab.com systemd[1]: Started AmavisVTd.
Nov 12 15:08:17 mail.zoholab.com amavisvtd[18057]: 2019-11-12 15:08:17,138 ERROR   [MainThread] Socket /opt/zimbra/data/clamav/amavisvtd.sock isn't working: [Errno 111] Connection refused
Nov 12 15:08:17 mail.zoholab.com amavisvt[18057]: MainThread - Socket /opt/zimbra/data/clamav/amavisvtd.sock isn't working: [Errno 111] Connection refused
Nov 12 15:09:08 mail.zoholab.com systemd[1]: Stopping AmavisVTd...
Nov 12 15:09:08 mail.zoholab.com systemd[1]: Stopped AmavisVTd.
Nov 12 15:09:20 mail.zoholab.com systemd[1]: Started AmavisVTd.


[maxxer]

From this line

Code: Select all

Nov 12 15:06:20 mail.zoholab.com amavisvtd[16553]: 2019-11-12 15:06:20,394 ERROR   [MainThread] Socket /opt/zimbra/data/clamav/amavisvtd.sock isn't working: [Errno 111] Connection refused


it looks like either the socket isn't created or it doesn't have permissions to access the location.

[zimico]

Just remove the .sock file (the path is in the conf file) and restart service. And then you will see the errors which relate to .py files.
Best regards,
Minh.

[maxxer]

Just remove the .sock file (the path is in the conf file) and restart service. And then you will see the errors which relate to .py files.

I have no idea how the plugin works, but based on this message Connection to www.virustotal.com timed out. it looks like your server cannot reach VT

[zimico]

Dear Maxxer,
Ping to http://www.virustotal.com is OK, I can manually carry out the request by:

Code: Select all

[root@mail ~]# curl -X POST https://www.virustotal.com/vtapi/v2/file/scan -F apikey=efe-------------------my key------------35 -F file=/root/myfile.sh
{"scan_id": "44e370512eadbcf17fdb925ac32157dc914be18c4d3c5caa3f606f575c9d3002-1573630626", "sha1": "c652a738c179fa72cc075ba27821ef4c5a62085e", "resource": "44e370512eadbcf17fdb925ac32157dc914be18c4d3c5caa3f606f575c9d3002", "response_code": 1, "sha256": "44e370512eadbcf17fdb925ac32157dc914be18c4d3c5caa3f606f575c9d3002", "permalink": "https://www.virustotal.com/file/44e370512eadbcf17fdb925ac32157dc914be18c4d3c5caa3f606f575c9d3002/analysis/1573630626/", "md5": "6f7e1dd90c4d6d6970cfbfb1adb3409f", "verbose_msg": "Scan request successfully queued, come back later for the report"}

Scan request successfully queued.
I am not a software engineer so I hope that someone can have a look at the code in case of using CentOS.
Best regards,
Minh.

[maxxer]

How strange. Unfortunately I don't know python so I cannot help further. You can try opening an issue on the project's github

[zimico]

Dear Maxxer,
Your are right about the connection. Our server has some advance firewall rules which block oubound connection to specific port. After turning off firewall, now the server can connect to virustotal site. I can see the increasement of quota usage in virustotal portal. However, there is no action log whenever we send or receive test mail.

Code: Select all

[root@mail cron.d]# journalctl -fu amavis-vtd
-- Logs begin at Thu 2019-11-14 00:06:57 +07. --
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: Traceback (most recent call last):
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: File "/usr/local/lib/python3.6/site-packages/amavisvt-0.5.3-py3.6.egg/amavisvt/client.py", line 529, in check_vt
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: raise Exception("API-Limit exceeded!")
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: Exception: API-Limit exceeded!
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: 2019-11-14 01:24:09,877 ERROR   [Thread-143] Error asking virustotal about files
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: Traceback (most recent call last):
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: File "/usr/local/lib/python3.6/site-packages/amavisvt-0.5.3-py3.6.egg/amavisvt/client.py", line 529, in check_vt
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: raise Exception("API-Limit exceeded!")
Nov 14 01:24:09 mail.zimilab.com amavisvtd[857]: Exception: API-Limit exceeded!
Nov 14 01:24:09 mail.zimilab.com amavisvt[857]: Thread-143 - Error asking virustotal about files
                                                Traceback (most recent call last):
                                                  File "/usr/local/lib/python3.6/site-packages/amavisvt-0.5.3-py3.6.egg/amavisvt/client.py", line 529, in check_vt
                                                    raise Exception("API-Limit exceeded!")
                                                Exception: API-Limit exceeded!

The status is always API-Limit exceeded at the time we restart the server. Can we see virustotal check in zimbra.log?
I am going to reinstall the server so we will have a clean enviroment to carry out this integration again.
Best regards,
Minh.

[maxxer]

I think there's no logging for normal operations, only for errors