Issue

Zend Server for IBM i, version 6  - 2019.x are 32 bit programs, which means they are limited to file sizes around 2 GB.  This example shows how to use the split command in PASE to break the file into smaller chunks and send them up to AWS.  The split command is a standard command in Linux, so this technique would also apply there.

Environment

Zend Server versions 6 - 8.5x, 9.1.x, 2018.x and 2019.x. 

Resolution

The way to accomplish this with 32 bit PHP is to 'chunk' the file. To 'chunk' the file means to break it up into smaller pieces, and transmit the pieces. PASE uses the 'split' command to break up the file. Then the AWS SDK provides for sending the pieces. You can read the AWS documentation here:

Upload a File in Multiple Parts Using the PHP SDK Low-Level API

Chunking the file is fairly straightforward.  Here is the format for the PASE command:

$ split -b 1000m /path/to/large/file /path/to/output/file/prefix

You end up with files having prefix_aa, prefix_ab, so you can then pass each chunk to uploadPart, and finally call completeUpload and you’re done.  Here is an example PASE session:

$ split -b 500m test testnew 
$ ls -la testnew* 
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:38 testnewaa
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:39 testnewab
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:39 testnewac
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:40 testnewad
-rw-r--r-- 1 maurice 0 314572800 Mar 10 14:40 testnewae

So you call Aws\S3\S3Client::createMultipartUpload() which will initialize the upload and give you an upload id.

Then call Aws\S3\S3Client::uploadPart()with a chunk and a part number, iteratively or in parallel for all the chunks.

Finally you call Aws\S3\S3Client::completeMultipartUpload() to finalize the upload.

For code examples and more information, please refer to the AWS documentation in the above link.