com.zextras.lib.vfs.s3.S3Error: InvalidPartOrder

Forum to discuss, report issues, or provide feedback about Zimbra HSM Plus
LinU777
Posts: 10
Joined: Sat Sep 13, 2014 2:27 am

com.zextras.lib.vfs.s3.S3Error: InvalidPartOrder

Postby LinU777 » Tue Apr 26, 2022 11:49 am

Zimbra 9.0.0_GA_4258.NETWORK
Hello. We are testing HSM with secondary volume on custom S3.
After manually starting doMoveBlobs operation we see move blob errors in mailbox.log:

Code: Select all

2022-04-26 14:33:41,245 WARN  [HSM Blob Mover Copy] [tid=137;operationId=ea7856b6-63d3-4ffe-99de-44f5888ccdcc;module=ZxPowerstore;operation=MoveBlobs;] mailbox - Failed to copy 3461 of mailbox 232 from volume 1: Error occurred in underlying filesystem com.amazonaws.services.s3.model.AmazonS3Exception: The list of parts was not in ascending order. The parts list must be specified in order by part number. (Service: Amazon S3; Status Code: 400; Error Code: InvalidPartOrder; Request ID: a5b9bedac20d3bb8; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null Skipping.
2022-04-26 14:33:41,725 ERROR [HSM Blob Mover Copy] [tid=194;operationId=ea7856b6-63d3-4ffe-99de-44f5888ccdcc;module=ZxPowerstore;operation=MoveBlobs;] mailbox - com.zextras.lib.vfs.s3.S3Error: Error occurred in underlying filesystem com.amazonaws.services.s3.model.AmazonS3Exception: The list of parts was not in ascending order. The parts list must be specified in order by part number. (Service: Amazon S3; Status Code: 400; Error Code: InvalidPartOrder; Request ID: 06d91f2cb40bf913; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
        at com.zextras.lib.vfs.s3.S3FS.wrapException (S3FS.java:373)
        at com.zextras.lib.vfs.s3.S3StreamWriter$2.run (S3StreamWriter.java:248)
        at com.zextras.lib.activities.ActivityThread.run (ActivityThread.java:144)
Caused by: java.io.IOException: com.amazonaws.services.s3.model.AmazonS3Exception: The list of parts was not in ascending order. The parts list must be specified in order by part number. (Service: Amazon S3; Status Code: 400; Error Code: InvalidPartOrder; Request ID: 06d91f2cb40bf913; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
        at com.zextras.lib.vfs.s3.S3Error.<init> (S3Error.java:16)
        at com.zextras.lib.vfs.s3.S3FS.wrapException (S3FS.java:373)
        at com.zextras.lib.vfs.s3.S3StreamWriter$2.run (S3StreamWriter.java:248)
        at com.zextras.lib.activities.ActivityThread.run (ActivityThread.java:144)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The list of parts was not in ascending order. The parts list must be specified in order by part number. (Service: Amazon S3; Status Code: 400; Error Code: InvalidPartOrder; Request ID: 06d91f2cb40bf913; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse (AmazonHttpClient.java:1811)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse (AmazonHttpClient.java:1395)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest (AmazonHttpClient.java:1371)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper (AmazonHttpClient.java:1145)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute (AmazonHttpClient.java:802)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer (AmazonHttpClient.java:770)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute (AmazonHttpClient.java:744)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500 (AmazonHttpClient.java:704)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute (AmazonHttpClient.java:686)
        at com.amazonaws.http.AmazonHttpClient.execute (AmazonHttpClient.java:550)
        at com.amazonaws.http.AmazonHttpClient.execute (AmazonHttpClient.java:530)
        at com.amazonaws.services.s3.AmazonS3Client.invoke (AmazonS3Client.java:5062)
        at com.amazonaws.services.s3.AmazonS3Client.invoke (AmazonS3Client.java:5008)
        at com.amazonaws.services.s3.AmazonS3Client.completeMultipartUpload (AmazonS3Client.java:3490)
        at com.zextras.lib.s3.S3Client.completeMultipartUpload (S3Client.java:346)
        at com.zextras.lib.vfs.s3.S3StreamWriter$2.run (S3StreamWriter.java:237)
        at com.zextras.lib.activities.ActivityThread.run (ActivityThread.java:144)

In hsm monitor we see skipped blobs (about 2% of moved blobs). Also in S3 console we see failed multipart uploads.

1. Is there any way to force right part order?
2. Do this failed blobs stay in primary volume?
3. If we run doMoveBlobs again will zimbra try to move failed blobs again, creating another failed multipart uploads in S3?


Return to “Zimbra HSM Plus”

Who is online

Users browsing this forum: No registered users and 1 guest