PutBucketNotificationConfiguration operation: Configurations overlap Error

[ramji-ai]

Hello Team,

I am trying this tool for the first time. I am getting the following error while running matano deploy command.

An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configurations overlap. Configurations on the same
› bucket cannot share a common event type

My configurations:

========================================================================
log_sources/dummy_test/log_source.yml

name: "dummy_test"

ingest:
s3_source:
bucket_name: "matanodpcommonstack-matanoingestionbucket1eaea516-170y1mafidx91"
key_prefix: "dummy_test/jsonlogs"

transform: |
. = object!(parse_json!(string!(.line)))
.source.ip = del(.srcIpAddress)
del(.username)
.message = downcase(string!(.message))

schema:
ecs_field_names:

• source.ip
• message
  =============================================================================
  detections/my_detection/detection.yml

name: "my_detection"
tables:

• "dummy_test"

==============================================================================
and uploaded my json file to Ingest directory of matano:

matanodpcommonstack-matanoingestionbucket1eaea516-170y1mafidx91
dummy-logs/

===============================================================================
helloworld.json

{
"line":"{"status":200,"srcIpAddress":"1.1.1.1","message":"SUCCESS","username":"ub40fan4life"}"
}

===============================================================================

Kindly request you to help to debug this issue.

Best regards
Ramji

[Samrose-Ahmed]

Hi, Matano supports either using the default Matano created sources ingestion bucket or a custom bucket (Bring your own bucket).

The ingest.s3_source block is for bringing your own custom bucket.

When you want to just use the Matano provided ingestion source bucket, you don't have to include any ingest configuration in your log_source.yml.

So try to remove the entire ingest key from your log_source.yml.

[ramji-ai]

Thank you so much. I was successfully able to deploy it now. I tried pushing some files to source bucket. I don't find any files ingested in lakestorage and realtime buckets.

[shaeqahmed]

Make sure you are uploading the files under a subdirectory for the log source, in this case (s3://your-matano-ingest-bkt/dummy_test/*). In your example earlier, I see the object path begins with 'dummy-test' which is incorrect and will not match to the log source 'dummy_test' by the default prefix routing behavior for the matano managed source bucket.

Btw feel free to join our discord, happy to help you out there if it's easier 🙂
https://discord.gg/YSYfHMbfZQ