enhancement(aws_sqs sink): Batching Support

[jlambatl]

Summary

This change introduces batching to the SQS sink. If Vector is configured to use SQS as a sink, nothing will change; however, users who do use SQS will be able to configure a batch in the same way that they can with other sinks.

The batch logic conforms to other sinks, with max_events, max_bytes, and timeout.

The default max_bytes is 256 KB to be compatible with most existing SQS configurations; however, it can be set to 1 MB if users have higher limits.

This change allows Vector users to opt in to balance their SQS request costs and latency requirements. All other behaviours should remain the same, as this change introduces a parallel code path.

Vector configuration

Original Configuration

[sinks.my_sqs]
type = "aws_sqs"
queue_url = "..."

Batching

[sinks.my_sqs]
type = "aws_sqs"
queue_url = "..."

[sinks.my_sqs.batch]
max_events = 10      # SQS API limit
timeout_secs = 1.0

[sinks.my_sqs.batch]
max_events = 10
max_bytes = 1048576  # Explicit opt-in
timeout_secs = 1.0

How did you test this PR?

There are unit tests that are designed to test various edge cases to ensure the batching and error logic are correct. I've also added integration tests.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes
• No. A maintainer will apply the no-changelog label to this PR.

References

• Closes: AWS SQS batch mode #8550