Skip to content

Fix issue of blob batch request handling error when boundary includes '='#2421

Open
EmmaZhu wants to merge 1 commit intoAzure:mainfrom
EmmaZhu:batch_boundary_equal
Open

Fix issue of blob batch request handling error when boundary includes '='#2421
EmmaZhu wants to merge 1 commit intoAzure:mainfrom
EmmaZhu:batch_boundary_equal

Conversation

@EmmaZhu
Copy link
Copy Markdown
Member

@EmmaZhu EmmaZhu commented Jul 3, 2024 •
edited
Loading

JS storage SDK doesn't use boundary with '='. Cannot easily implement a test case to generate request with boundary with '='.
Manually tested the change with cases like:

  1. When request content type doesn't include 'boundary=' segment:
    Storage service response looks like:
HTTP/1.1 202 Accepted
Transfer-Encoding: chunked
Content-Type: multipart/mixed; boundary=batchresponse_a0858ed9-2c3b-40b0-a4eb-9fb5aa32d428
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 0981b5d3-601e-0024-62bd-cd6581000000
x-ms-version: 2024-08-04
x-ms-client-request-id: f0f82bd6-4822-4951-b908-a8452ec39073
Date: Thu, 04 Jul 2024 02:57:54 GMT
Connection: close

33B
--batchresponse_a0858ed9-2c3b-40b0-a4eb-9fb5aa32d428
Content-Type: application/http

HTTP/1.1 400 The value for one of the HTTP headers is not in the correct format.
x-ms-error-code: InvalidHeaderValue
x-ms-request-id: 0981b5d3-601e-0024-62bd-cd6581000000
x-ms-version: 2024-08-04
x-ms-client-request-id: f0f82bd6-4822-4951-b908-a8452ec39073
Content-Length: 335
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0

<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format.
RequestId:0981b5d3-601e-0024-62bd-cd6581000000
Time:2024-07-04T02:57:54.5495268Z</Message><HeaderName>Content-Type</HeaderName><HeaderValue>multipart/mixed;</HeaderValue></Error>
--batchresponse_a0858ed9-2c3b-40b0-a4eb-9fb5aa32d428--
0

 Azurite response looks like:

HTTP/1.1 202 Accepted
Server: Azurite-Blob/3.31.0
content-type: multipart/mixed; boundary=batchresponse_5caca79b-42c2-4dfa-ae2d-ce9bad5ade82
x-ms-request-id: e4614594-20ce-46db-b70a-27abc2ff96eb
x-ms-version: 2024-08-04
Date: Thu, 04 Jul 2024 02:51:05 GMT
Connection: close
Transfer-Encoding: chunked

2aa
--batchresponse_5caca79b-42c2-4dfa-ae2d-ce9bad5ade82
Content-Type: application/http

HTTP/1.1 400 The value for one of the HTTP headers is not in the correct format.
x-ms-error-code: InvalidHeaderValue
x-ms-request-id: e4614594-20ce-46db-b70a-27abc2ff96eb
content-type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidHeaderValue</Code>
  <Message>The value for one of the HTTP headers is not in the correct format.
RequestId:e4614594-20ce-46db-b70a-27abc2ff96eb
Time:2024-07-04T02:51:24.715Z</Message>
  <HeaderName>Content-Type</HeaderName>
  <HeaderValue/>
</Error>

--batchresponse_5caca79b-42c2-4dfa-ae2d-ce9bad5ade82--
0
  1. When request content-type includes more than one 'boundary' segment, like: Content-Type: multipart/mixed; boundary=batch==d88eec8e-68ce-450a-b029-b6e3072638d5; boundary=1111
    Storage service response looks like:
HTTP/1.1 202 Accepted
Transfer-Encoding: chunked
Content-Type: multipart/mixed; boundary=batchresponse_73428b91-e55d-47ea-8d70-b10acf6ec30d
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: c7246e06-f01e-0044-55bd-cd191e000000
x-ms-version: 2024-08-04
x-ms-client-request-id: 73b8342c-f3b7-4da5-ad9d-7e9d9c3103a5
Date: Thu, 04 Jul 2024 02:56:22 GMT
Connection: close

2A7
--batchresponse_73428b91-e55d-47ea-8d70-b10acf6ec30d
Content-Type: application/http

HTTP/1.1 400 One of the request inputs is not valid.
x-ms-error-code: InvalidInput
x-ms-request-id: c7246e06-f01e-0044-55bd-cd191e000000
x-ms-version: 2024-08-04
x-ms-client-request-id: 73b8342c-f3b7-4da5-ad9d-7e9d9c3103a5
Content-Length: 221
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0

<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidInput</Code><Message>One of the request inputs is not valid.
RequestId:c7246e06-f01e-0044-55bd-cd191e000000
Time:2024-07-04T02:56:23.3362534Z</Message></Error>
--batchresponse_73428b91-e55d-47ea-8d70-b10acf6ec30d--
0

Azurite response looks like:

HTTP/1.1 202 Accepted
Server: Azurite-Blob/3.31.0
content-type: multipart/mixed; boundary=batchresponse_a5b0c5d5-51c4-4110-b3c0-cba01b38b0f7
x-ms-request-id: 851019f5-bde8-4be2-814e-bd73f82b3b20
x-ms-version: 2024-08-04
Date: Thu, 04 Jul 2024 02:54:43 GMT
Connection: close
Transfer-Encoding: chunked

22d
--batchresponse_a5b0c5d5-51c4-4110-b3c0-cba01b38b0f7
Content-Type: application/http

HTTP/1.1 400 One of the request inputs is not valid.
x-ms-error-code: InvalidInput
x-ms-request-id: 851019f5-bde8-4be2-814e-bd73f82b3b20
content-type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidInput</Code>
  <Message>One of the request inputs is not valid.
RequestId:851019f5-bde8-4be2-814e-bd73f82b3b20
Time:2024-07-04T02:54:43.493Z</Message>
</Error>

--batchresponse_a5b0c5d5-51c4-4110-b3c0-cba01b38b0f7--
0
  1. No content-type specified:
    Storage service response looks like:
HTTP/1.1 202 Accepted
Transfer-Encoding: chunked
Content-Type: multipart/mixed; boundary=batchresponse_10248f9a-47e1-4f15-81c7-f5dd8111b46e
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 90f0055a-601e-0046-7ebe-cda7a6000000
x-ms-version: 2024-08-04
x-ms-client-request-id: 4fb573f0-9cfb-4393-8bc8-b4c493621262
Date: Thu, 04 Jul 2024 03:01:24 GMT
Connection: close

319
--batchresponse_10248f9a-47e1-4f15-81c7-f5dd8111b46e
Content-Type: application/http

HTTP/1.1 400 An HTTP header that's mandatory for this request is not specified.
x-ms-error-code: MissingRequiredHeader
x-ms-request-id: 90f0055a-601e-0046-7ebe-cda7a6000000
x-ms-version: 2024-08-04
x-ms-client-request-id: 4fb573f0-9cfb-4393-8bc8-b4c493621262
Content-Length: 299
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0

<?xml version="1.0" encoding="utf-8"?>
<Error><Code>MissingRequiredHeader</Code><Message>An HTTP header that&apos;s mandatory for this request is not specified.
RequestId:90f0055a-601e-0046-7ebe-cda7a6000000
Time:2024-07-04T03:01:25.1421605Z</Message><HeaderName>Content-Type</HeaderName></Error>
--batchresponse_10248f9a-47e1-4f15-81c7-f5dd8111b46e--
0

Azurite response looks like:

HTTP/1.1 202 Accepted
Server: Azurite-Blob/3.31.0
content-type: multipart/mixed; boundary=batchresponse_791220da-4050-4684-ad5e-7b46aea28f3d
x-ms-request-id: 193c5e97-435a-488a-a8a9-56655ecf2208
x-ms-version: 2024-08-04
Date: Thu, 04 Jul 2024 02:53:21 GMT
Connection: close
Transfer-Encoding: chunked

2c7
--batchresponse_791220da-4050-4684-ad5e-7b46aea28f3d
Content-Type: application/http

HTTP/1.1 400 The value for one of the HTTP headers is not in the correct format.
x-ms-error-code: InvalidHeaderValue
x-ms-request-id: 193c5e97-435a-488a-a8a9-56655ecf2208
content-type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidHeaderValue</Code>
  <Message>The value for one of the HTTP headers is not in the correct format.
RequestId:193c5e97-435a-488a-a8a9-56655ecf2208
Time:2024-07-04T02:53:30.621Z</Message>
  <HeaderName>Content-Type</HeaderName>
  <HeaderValue>multipart/mixed;</HeaderValue>
</Error>

--batchresponse_791220da-4050-4684-ad5e-7b46aea28f3d--
0

@EmmaZhu EmmaZhu mentioned this pull request Jul 3, 2024
Open
@EmmaZhu EmmaZhu force-pushed the batch_boundary_equal branch from 6a55b6d to 482ac28 Compare July 4, 2024 03:05
@EmmaZhu
Copy link
Copy Markdown
Member Author

EmmaZhu commented Jul 4, 2024

/check-enforcer reset

@blueww
Copy link
Copy Markdown
Member

blueww commented Jul 8, 2024

/azp run

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Jul 8, 2024

Azure Pipelines successfully started running 1 pipeline(s).

blueww
blueww approved these changes Jul 8, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@blueww blueww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Not easy to add test since Can't JS SDK will always has valid boundary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blueww blueww blueww approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@EmmaZhu @blueww