diff --git a/src/content/changelog/r2/2025-12-08-r2-multipart-list-apis.mdx b/src/content/changelog/r2/2025-12-08-r2-multipart-list-apis.mdx new file mode 100644 index 00000000000000..ddeedff6193918 --- /dev/null +++ b/src/content/changelog/r2/2025-12-08-r2-multipart-list-apis.mdx @@ -0,0 +1,39 @@ +--- +title: List multipart uploads and parts with the R2 Workers API +description: New Workers API methods to list in-progress multipart uploads and their uploaded parts +products: + - r2 +date: 2025-12-08 +--- + +The [R2 Workers API](/r2/api/workers/workers-api-reference/) now includes two new methods for managing multipart uploads: + +#### List multipart uploads + +Use [`listMultipartUploads()`](/r2/api/workers/workers-api-reference/#r2listmultipartuploadsoptions) on your R2 bucket binding to list all in-progress multipart uploads. This is useful for monitoring active uploads or cleaning up incomplete ones. + +```js +const uploads = await env.MY_BUCKET.listMultipartUploads({ + prefix: "uploads/", + limit: 100, +}); + +for (const upload of uploads.uploads) { + console.log(`Upload: ${upload.key}, ID: ${upload.uploadId}`); +} +``` + +#### List parts of a multipart upload + +Use [`listParts()`](/r2/api/workers/workers-api-reference/#r2listpartsoptions) on an `R2MultipartUpload` object to list all parts that have been uploaded. This enables you to resume uploads by checking which parts have already been uploaded. + +```js +const multipartUpload = env.MY_BUCKET.resumeMultipartUpload(key, uploadId); +const result = await multipartUpload.listParts(); + +for (const part of result.parts) { + console.log(`Part ${part.partNumber}: ${part.size} bytes`); +} +``` + +Both methods support pagination for handling large result sets. For more details and examples, refer to the [Workers API reference](/r2/api/workers/workers-api-reference/) and [multipart upload guide](/r2/api/workers/workers-multipart-usage/). diff --git a/src/content/docs/r2/api/workers/workers-api-reference.mdx b/src/content/docs/r2/api/workers/workers-api-reference.mdx index ac28aeb841de53..1151b7fe499a56 100644 --- a/src/content/docs/r2/api/workers/workers-api-reference.mdx +++ b/src/content/docs/r2/api/workers/workers-api-reference.mdx @@ -104,6 +104,12 @@ export default { - Returns an object representing a multipart upload with the given key and uploadId. - The resumeMultipartUpload operation does not perform any checks to ensure the validity of the uploadId, nor does it verify the existence of a corresponding active multipart upload. This is done to minimize latency before being able to call subsequent operations on the `R2MultipartUpload` object. +- `listMultipartUploads` + + - Returns an `R2MultipartUploads` containing a list of in-progress multipart uploads in the bucket. + - The returned list of multipart uploads is ordered lexicographically by key. + - Refer to [R2ListMultipartUploadsOptions](#r2listmultipartuploadsoptions) for available options. + ## `R2Object` definition `R2Object` is created when you `PUT` an object into an R2 bucket. `R2Object` represents the metadata of an object based on the information provided by the uploader. Every object that you `PUT` into an R2 bucket will have an `R2Object` created. @@ -229,6 +235,12 @@ A multipart upload can be completed or aborted at any time, either through the S - Completes the multipart upload with the given parts. - Returns a Promise that resolves when the complete operation has finished. Once this happens, the object is immediately accessible globally by any subsequent read operation. +- `listParts` + + - Returns an `R2UploadedParts` containing a list of parts that have been uploaded to this multipart upload. + - Parts are returned in ascending order by part number. + - Refer to [R2ListPartsOptions](#r2listpartsoptions) for available options. + ## Method-specific types ### R2GetOptions @@ -416,6 +428,114 @@ An object containing an `R2Object` array, returned by `BUCKET_BINDING.list()`. - For example, if no prefix is provided and the delimiter is '/', `foo/bar/baz` would return `foo` as a delimited prefix. If `foo/` was passed as a prefix with the same structure and delimiter, `foo/bar` would be returned as a delimited prefix. +### R2ListMultipartUploadsOptions + +- `limit` + + - The number of results to return. Defaults to `1000`, with a maximum of `1000`. + +- `prefix` + + - The prefix to match keys against. Keys will only be returned if they start with given prefix. + +- `cursor` + + - An opaque token that indicates where to continue listing multipart uploads from. A cursor can be retrieved from a previous list operation. + +- `delimiter` + + - The character to use when grouping keys. + +- `startAfter` + + - A key to start listing multipart uploads after. Used for pagination in combination with cursor. + +### R2MultipartUploads + +An object containing an array of `R2MultipartUploadListing` objects, returned by `BUCKET_BINDING.listMultipartUploads()`. + +- `uploads` + + - An array of multipart uploads matching the `listMultipartUploads` request. + +- `truncated` + + - If true, indicates there are more results to be retrieved for the current `listMultipartUploads` request. + +- `cursor` + + - A token that can be passed to future `listMultipartUploads` calls to resume listing from that point. Only present if truncated is true. + +- `delimitedPrefixes` + + - If a delimiter has been specified, contains all prefixes between the specified prefix and the next occurrence of the delimiter. + +### R2MultipartUploadListing + +An object representing a single multipart upload in the list returned by `listMultipartUploads`. + +- `key` + + - The key (object name) for this multipart upload. + +- `uploadId` + + - The unique identifier for this multipart upload. + +- `initiated` + + - A Date object representing when the multipart upload was initiated. + +- `storageClass` + + - The storage class associated with this multipart upload. + +### R2ListPartsOptions + +- `maxParts` + + - The maximum number of parts to return. Defaults to `1000`, with a maximum of `1000`. + +- `partNumberMarker` + + - The part number to start listing from. Parts with a part number greater than this value will be returned. Must be a positive integer. + +### R2UploadedParts + +An object containing an array of uploaded parts, returned by `multipartUpload.listParts()`. + +- `parts` + + - An array of parts that have been uploaded to this multipart upload. + +- `truncated` + + - If true, indicates there are more parts to be retrieved for the current `listParts` request. + +- `partNumberMarker` + + - A part number marker that can be passed to future `listParts` calls to resume listing from that point. Only present if truncated is true. + +### R2UploadedPartInfo + +An object representing detailed information about an uploaded part. + +- `partNumber` + + - The part number of this part. + +- `etag` + + - The etag of this part. + +- `size` + + - The size of this part in bytes. + +- `uploaded` + + - A Date object representing when this part was uploaded. + ### Conditional operations You can pass an `R2Conditional` object to `R2GetOptions` and `R2PutOptions`. If the condition check for `get()` fails, the body will not be returned. This will make `get()` have lower latency. diff --git a/src/content/docs/r2/api/workers/workers-multipart-usage.mdx b/src/content/docs/r2/api/workers/workers-multipart-usage.mdx index 40ddfd81263282..eff9524af37dea 100644 --- a/src/content/docs/r2/api/workers/workers-multipart-usage.mdx +++ b/src/content/docs/r2/api/workers/workers-multipart-usage.mdx @@ -266,6 +266,65 @@ def upload_part(filename, partsize, url, uploadId, index): upload_file(worker_endpoint, filename, partsize) ``` +## List multipart uploads + +You can list all in-progress multipart uploads in a bucket using the `listMultipartUploads` method. This is useful for managing and cleaning up incomplete uploads. + +```js +// List all in-progress multipart uploads +const uploads = await env.MY_BUCKET.listMultipartUploads(); + +for (const upload of uploads.uploads) { + console.log(`Upload: ${upload.key}, ID: ${upload.uploadId}, Started: ${upload.initiated}`); +} + +// Handle pagination for large results +let cursor = uploads.cursor; +while (uploads.truncated) { + const moreUploads = await env.MY_BUCKET.listMultipartUploads({ cursor }); + for (const upload of moreUploads.uploads) { + console.log(`Upload: ${upload.key}, ID: ${upload.uploadId}`); + } + cursor = moreUploads.cursor; + uploads.truncated = moreUploads.truncated; +} + +// Filter by prefix +const filteredUploads = await env.MY_BUCKET.listMultipartUploads({ + prefix: "uploads/", + delimiter: "/", +}); +``` + +## List parts of a multipart upload + +You can list all parts that have been uploaded to a multipart upload using the `listParts` method. This is useful for resuming uploads or verifying which parts have been uploaded. + +```js +// Resume an existing multipart upload +const multipartUpload = env.MY_BUCKET.resumeMultipartUpload(key, uploadId); + +// List all uploaded parts +const result = await multipartUpload.listParts(); + +for (const part of result.parts) { + console.log( + `Part ${part.partNumber}: etag=${part.etag}, size=${part.size}, uploaded=${part.uploaded}` + ); +} + +// Handle pagination for uploads with many parts +let partNumberMarker = result.partNumberMarker; +while (result.truncated) { + const moreParts = await multipartUpload.listParts({ partNumberMarker }); + for (const part of moreParts.parts) { + console.log(`Part ${part.partNumber}: size=${part.size}`); + } + partNumberMarker = moreParts.partNumberMarker; + result.truncated = moreParts.truncated; +} +``` + ## State management The stateful nature of multipart uploads does not easily map to the usage model of Workers, which are inherently stateless. In a normal multipart upload, the multipart upload is usually performed in one continuous execution of the client application. This is different from multipart uploads in a Worker, which will often be completed over multiple invocations of that Worker. This makes state management more challenging. @@ -274,4 +333,4 @@ To overcome this, the state associated with a multipart upload, namely the `uplo In the example Worker and Python application described in this guide, the state of the multipart upload is tracked in the client application which sends requests to the Worker, with the necessary state contained in each request. Keeping track of the multipart state in the client application enables maximal flexibility and allows for parallel and unordered uploads of each part. -When keeping track of this state in the client is impossible, alternative designs can be considered. For example, you could track the `uploadId` and which parts have been uploaded in a Durable Object or other database. +When keeping track of this state in the client is impossible, alternative designs can be considered. For example, you could track the `uploadId` and which parts have been uploaded in a Durable Object or other database. Alternatively, use `listMultipartUploads` and `listParts` to query the current state of uploads directly from R2.