-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathmap2-auto-tagger-optimized.yaml
More file actions
3211 lines (2976 loc) · 160 KB
/
map2-auto-tagger-optimized.yaml
File metadata and controls
3211 lines (2976 loc) · 160 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
AWSTemplateFormatVersion: '2010-09-09'
Description: >
MAP 2.0 Auto-Tagger v21.0.6 - Auto-tags AWS resources with map-migrated for MAP 2.0
credit eligibility. 190+ resource types. Deploy once; tagging happens within 60-90 s
of resource creation. Daily reconciliation Lambda (RGTA-based) catches any tags the
live Lambda missed. Three-path error classifier + TagFailureByClass CloudWatch metric
distinguishes transient / permanent-ignorable / permanent-actionable failures.
Pipeline: EventBridge -> SQS (14-day retention) -> Lambda with DLQ + SNS alarm.
See CHANGELOG.md for full history.
Parameters:
MpeId:
Type: String
Description: MAP 2.0 Migration Program Engagement ID (e.g., mig1234567890)
AllowedPattern: ^mig[a-zA-Z0-9]+$
MaxLength: 20
ConstraintDescription: Must start with 'mig' followed by alphanumeric characters
AgreementStartDate:
Type: String
Description: MAP 2.0 agreement start date (YYYY-MM-DD)
AllowedPattern: ^(19|20)\d{2}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])$
AgreementEndDate:
Type: String
Description: MAP 2.0 agreement end date (YYYY-MM-DD). Resources created after this date will NOT be tagged.
AllowedPattern: ^(19|20)\d{2}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])$
Default: "2099-12-31"
ScopeMode:
Type: String
Description: How to determine which resources to tag
Default: account
AllowedValues:
- account
- vpc
ScopedAccountIds:
Type: CommaDelimitedList
Description: >
Comma-separated AWS account IDs in MAP scope (used when ScopeMode=account).
Leave as 'ALL' to tag all resources in this account.
Default: ALL
ScopedVpcIds:
Type: CommaDelimitedList
Description: >
Comma-separated VPC IDs in MAP scope (used when ScopeMode=vpc).
Only resources in these VPCs will be tagged.
Default: NONE
ReconciliationInterval:
Type: String
Description: >
Schedule for the reconciliation Lambda (daily safety-net that enumerates
in-scope resources via RGTA GetResources and re-enqueues any whose
map-migrated tag is missing or wrong). EventBridge schedule expression.
Default: 'rate(24 hours)'
AllowedPattern: '^(rate\(\d+ (hour|hours|day|days)\)|cron\(.+\))$'
Conditions:
IsAccountAll: !Equals [!Join [',', !Ref ScopedAccountIds], 'ALL']
IsVpcNone: !Equals [!Join [',', !Ref ScopedVpcIds], 'NONE']
Resources:
# SSM Parameter Store - pinned template version (read by ops, upgrade.sh, and
# the Lambda at cold start). Zero outbound network calls — version is written
# at deploy time and discovered locally via ssm:GetParameter.
MapVersion:
Type: AWS::SSM::Parameter
Properties:
Name: !Sub '/auto-map-tagger/${MpeId}/version'
Type: String
Description: MAP 2.0 Auto-Tagger template version pinned at deploy time
Value: v21.0.6
# SSM Parameter Store - single source of truth for config.
# Tier: Intelligent-Tiering leaves the parameter in the free Standard tier
# (4KB limit) until the Value crosses the threshold, at which point AWS
# auto-upgrades to Advanced (8KB limit, $0.05/parameter/month). Closes
# §1.60: customers with ~240+ accounts in `scoped_account_ids` generate a
# Value > 4KB; prior behavior (default Standard) silently failed stack
# create with `ParameterMaxSizeExceeded`.
MapConfig:
Type: AWS::SSM::Parameter
Properties:
Name: !Sub '/auto-map-tagger/${MpeId}/config'
Tier: Intelligent-Tiering
Type: String
Description: MAP 2.0 Auto-Tagger configuration
Value: !Sub
- |
{
"mpe_id": "${MpeId}",
"agreement_start_date": "${AgreementStartDate}",
"agreement_end_date": "${AgreementEndDate}",
"scope_mode": "${ScopeMode}",
"scoped_account_ids": ${AccountIdsArray},
"scoped_vpc_ids": ${VpcIdsArray}
}
- AccountIdsArray: !If
- IsAccountAll
- '["ALL"]'
- !Sub
- '["${Joined}"]'
- Joined: !Join ['","', !Ref ScopedAccountIds]
VpcIdsArray: !If
- IsVpcNone
- '[]'
- !Sub
- '["${Joined}"]'
- Joined: !Join ['","', !Ref ScopedVpcIds]
# Lambda execution role
AutoTaggerRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub 'map-auto-tagger-role-${MpeId}-${AWS::Region}'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: map-auto-tagger-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
# SECURITY NOTE: Resource: '*' is required for tagging actions.
# AWS Tag Editor API (tag:TagResources) does not support resource-level
# permissions — it must be scoped to '*'. This is consistent with
# AWS-managed tagging solutions (e.g., AWS Tag Editor console, MAP Taggr).
# The Lambda only applies the map-migrated tag and has no create/delete/update
# permissions outside of tagging. IAM Access Analyzer: 0 findings.
- Sid: UniversalTagging
Effect: Allow
Action:
- tag:TagResources
- tag:GetResources
Resource: '*'
- Sid: ServiceSpecificTagging
Effect: Allow
Action:
# --- Compute ---
- ec2:CreateTags
- autoscaling:CreateOrUpdateTags
- ecs:TagResource
- eks:TagResource
- lambda:TagResource
- elasticbeanstalk:AddTags
- emr-serverless:TagResource
- elasticmapreduce:AddTags
# AWS Batch (§1.27) — job queues, compute environments, job
# definitions. RGTA dispatches tag:TagResources on these to
# batch:TagResource per the AWS service-authorization matrix;
# without this grant every Batch CreateJobQueue / CreateComputeEnvironment
# / RegisterJobDefinition fell through to AccessDenied.
- batch:TagResource
# --- Storage ---
- s3:PutBucketTagging
- s3:GetBucketTagging
- elasticfilesystem:TagResource
- fsx:TagResource
- ecr:TagResource
- backup:TagResource
# --- Database ---
- rds:AddTagsToResource
- dynamodb:TagResource
- elasticache:AddTagsToResource
- memorydb:TagResource
- redshift:CreateTags
- redshift-serverless:TagResource
- es:AddTags
- kafka:TagResource
- dms:AddTagsToResource
- cassandra:TagResource
- mq:CreateTags
# --- Networking ---
- elasticloadbalancing:AddTags
- globalaccelerator:TagResource
- cloudfront:TagResource
- route53:ChangeTagsForResource
- network-firewall:TagResource
- directconnect:TagResource
- appmesh:TagResource
# VPC Lattice: RGTA dispatches to vpc-lattice:TagResource per
# AWS auth matrix. Without this grant, COVERAGE.md's VPC
# Lattice claim was backed by RGTA fallthrough that silently
# AccessDenied on every CreateServiceNetwork event (D7).
- vpc-lattice:TagResource
# --- Analytics ---
- kinesis:AddTagsToStream
- firehose:TagDeliveryStream
- kinesisanalytics:TagResource
- glue:TagResource
- glue:GetDatabase
- databrew:TagResource
- athena:TagResource
# --- Integration ---
- sns:TagResource
- sqs:TagQueue
- states:TagResource
- appsync:TagResource
# SECURITY NOTE: API Gateway v1 (REST APIs) tags via PUT /tags/{arn}.
# AWS maps this to 'apigateway:PUT' — no narrower IAM action exists for v1 tagging.
# Lambda code only calls tagging APIs; no other API GW mutation code paths exist.
- apigateway:PUT
# v1 REST APIs also use PATCH (AWS routes tag:TagResources
# through PATCH /tags/{arn} internally).
- apigateway:PATCH
# v2 HTTP APIs actually use POST /tags/{arn} (confirmed via
# AccessDenied testing on PR #8 iteration 2). The v1/v2
# routing for tagging is POST for v2, PATCH for v1.
- apigateway:POST
# --- Management & Monitoring ---
- logs:TagResource
- cloudwatch:TagResource
- ssm:AddTagsToResource
- secretsmanager:TagResource
- servicediscovery:TagResource
# --- Security & Identity ---
- kms:TagResource
- acm:AddTagsToCertificate
- acm-pca:TagCertificateAuthority
- cognito-idp:TagResource
- cognito-identity:TagResource
- securityhub:TagResource
- wafv2:TagResource
# --- Developer Tools ---
- codepipeline:TagResource
- codedeploy:TagResource
- cloud9:TagResource
# SECURITY NOTE: CodeBuild has no standalone TagResource IAM action.
# AWS requires UpdateProject to apply tags (confirmed via AccessDenied testing).
# BatchGetProjects resolves project ARNs. Lambda code only calls tagging APIs.
- codebuild:UpdateProject
- codebuild:BatchGetProjects
# SECURITY NOTE: AWS routes tag:TagResources on CF stacks through UpdateStack
# internally (confirmed via AccessDenied testing). Lambda code only calls
# tag:TagResources; UpdateStack is never invoked directly by the Lambda.
- cloudformation:TagResource
- cloudformation:UpdateStack
- cloudformation:UpdateStackSet
# SECURITY NOTE: AWS routes Service Catalog tagging through UpdateProduct /
# UpdatePortfolio internally (confirmed via AccessDenied testing in v14).
# Lambda code only calls tag:TagResources; Update* is never invoked directly.
- servicecatalog:TagResource
- servicecatalog:UpdatePortfolio
- servicecatalog:UpdateProduct
# --- Migration & Transfer ---
- transfer:TagResource
- datasync:TagResource
- storagegateway:AddTagsToResource
# --- ML / AI ---
- sagemaker:AddTags
- comprehend:TagResource
- kendra:TagResource
- bedrock:TagResource
- bedrock-agentcore:TagResource
- braket:TagResource
# --- IoT ---
- iot:TagResource
- iotanalytics:TagResource
- iotevents:TagResource
# EventBridge (events:*) — distinct service from IoT Events (iotevents:*).
# Needed to tag newly-created Event rules / buses / schedules / connections.
- events:TagResource
- iotsitewise:TagResource
# --- Media ---
- mediaconvert:TagResource
- medialive:CreateTags
- mediapackage:TagResource
# --- Other ---
- ram:TagResource
- appstream:TagResource
- workspaces:CreateTags
- workspaces-web:TagResource
- quicksight:TagResource
- connect:TagResource
- managedblockchain:TagResource
- gamelift:TagResource
- timestream:TagResource
- healthlake:TagResource
- omics:TagResource
- dax:TagResource
- drs:TagResource
- deadline:TagResource
- kinesisvideo:TagStream
- dsql:TagResource
- payment-cryptography:TagResource
- networkmanager:TagResource
# --- Tier 1 MAP services (PR #25) ---
# Keyspaces (Cassandra-compatible) — RGTA support is unreliable
# for cassandra resources, so Lambda calls keyspaces:TagResource
# directly. cassandra:TagResource already present above; adding
# cassandra:Alter (required by the service-authorization matrix
# for keyspaces:TagResource — AccessDenied at runtime without it,
# live-confirmed as §1.99 in the remediation plan) + ds + cloudhsm.
- cassandra:Alter
- ds:AddTagsToResource
- cloudhsm:TagResource
Resource: '*'
# NOTE: This template deploys a single-account Lambda. No sts:AssumeRole or
# cross-account permissions are present or required. For multi-account deployments,
# a separate CloudFormation StackSet deploys this same template to each account
# independently via CloudFormation SERVICE_MANAGED permissions (AWS service-linked
# roles), not direct cross-account role assumption by the Lambda function itself.
- Sid: MinimalReads
Effect: Allow
Action:
# VPC scope checking (only used when ScopeMode=vpc)
- ec2:DescribeInstances
- ec2:DescribeVolumes
# Determine local account ID for cross-account tagging logic
- sts:GetCallerIdentity
# SSM TagResource validates the param name via GetParameters
# internally (AccessDenied without it). Lambda code only
# calls tag:TagResources.
- ssm:GetParameters
# CFN TagResource validates the stack ARN via DescribeStacks
# internally (AccessDenied without it). Lambda code only
# calls tag:TagResources.
- cloudformation:DescribeStacks
# IAM role tagging (iam:TagRole not covered by tag:TagResources)
- iam:TagRole
Resource: '*'
- Sid: SqsEventSource
Effect: Allow
Action:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- sqs:GetQueueAttributes
- sqs:SendMessage
Resource:
- !GetAtt EventQueue.Arn
- !GetAtt EventDLQ.Arn
- Sid: AlertPublish
Effect: Allow
Action:
- sns:Publish
Resource: !Ref AlertTopic
- Sid: EmitClassifierMetrics
Effect: Allow
Action:
- cloudwatch:PutMetricData
# PutMetricData does not support resource-level IAM; '*' is
# standard. Constrained to the MapAutoTagger namespace via
# IAM Condition below.
Resource: '*'
Condition:
StringEquals:
cloudwatch:namespace: MapAutoTagger
- Sid: ReadConfig
Effect: Allow
Action:
- ssm:GetParameter
Resource: !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/auto-map-tagger/${MpeId}/config'
# Peer-tagger detection at cold start. Lists map-auto-tagger-mig*
# stacks in this account/region to surface concurrent taggers
# (§1.108 Phase 16). ListStacks has no resource-level IAM per AWS
# IAM Service Authorization Reference; scope is implicitly the
# caller's account. Read-only.
- Sid: PeerTaggerDetect
Effect: Allow
Action:
- cloudformation:ListStacks
Resource: '*'
- Sid: Logging
Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*
# ── Preflight: peer-tagger scope-overlap guard at stack-instance creation ──
# Runs BEFORE AutoTaggerFunction is created. If another map-auto-tagger-mig*
# stack exists in this account+region with an overlapping scope, fail the
# Custom Resource → CFN rolls back this stack instance → no AutoTaggerFunction
# is ever provisioned. Closes the §1.108 temporal race (Phase 16 Test 5,
# evidence 0/50) that configurator.html preflight cannot catch: StackSet
# AutoDeployment into newly-joined OU accounts, and member-account deploys
# via multi-account StackSet where the member already has a peer stack.
# Fail-open on any internal error (throttle / IAM propagation / region issue)
# so a legitimate deploy is never blocked by a transient AWS condition;
# PR #60's runtime detector + PeerTaggerDetectedAlarm remain as the second
# backstop if the preflight's fail-open path let a real conflict through.
PreflightRole:
Type: AWS::IAM::Role
Properties:
# Name kept short (≤64 chars worst-case: 21+20+1+14 = 56) to match
# AutoTaggerRole's budget. Regional suffix is necessary because IAM
# role names are account-global and same-MPE stacks may deploy to
# multiple regions in the same account.
RoleName: !Sub 'map-preflight-role-${MpeId}-${AWS::Region}'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: map-auto-tagger-preflight-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: ListOwnStacks
Effect: Allow
Action:
- cloudformation:ListStacks
Resource: '*' # ListStacks has no resource-level IAM per AWS auth matrix
- Sid: ReadPeerConfigs
Effect: Allow
Action:
- ssm:GetParameter
Resource: !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/auto-map-tagger/*/config'
- Sid: Logging
Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*'
PreflightLogGroup:
Type: AWS::Logs::LogGroup
DeletionPolicy: Delete
UpdateReplacePolicy: Delete
Properties:
LogGroupName: !Sub '/aws/lambda/map-auto-tagger-preflight-${MpeId}'
RetentionInDays: 14
PreflightFunction:
Type: AWS::Lambda::Function
DependsOn: PreflightLogGroup
Properties:
FunctionName: !Sub 'map-auto-tagger-preflight-${MpeId}'
Runtime: python3.12
Handler: index.handler
Role: !GetAtt PreflightRole.Arn
Timeout: 60
MemorySize: 256
Code:
ZipFile: |
import json, os, boto3
from urllib.request import urlopen, Request
def respond(event, context, status, reason=''):
body = json.dumps({
'Status': status,
'Reason': reason or 'See CloudWatch logs',
'PhysicalResourceId': context.log_stream_name,
'StackId': event['StackId'],
'RequestId': event['RequestId'],
'LogicalResourceId': event['LogicalResourceId'],
})
req = Request(event['ResponseURL'], data=body.encode(), method='PUT')
req.add_header('Content-Type', '')
req.add_header('Content-Length', str(len(body)))
urlopen(req)
def scope_overlap(new_mode, new_accts, new_vpcs, peer_mode, peer_accts, peer_vpcs, this_account):
"""Return conflict reason string if scopes overlap; '' otherwise.
Mirrors the 5-case logic at configurator.html preflight
(lines ~6473–6514). Both sides are already in the same
account+region; overlap decisions reduce to scope dimensions.
"""
new_accts = set(new_accts or [])
peer_accts = set(peer_accts or [])
new_vpcs = set(new_vpcs or [])
peer_vpcs = set(peer_vpcs or [])
if new_mode == 'account' and peer_mode == 'account':
if 'ALL' in peer_accts:
return f'peer scope=account/ALL dominates {this_account}'
if 'ALL' in new_accts:
return f'our scope=account/ALL dominates peer in {this_account}'
if this_account in peer_accts and (this_account in new_accts or 'ALL' in new_accts):
return f'peer scope includes {this_account}'
return ''
if new_mode == 'account' and peer_mode == 'vpc':
# Our side claims whole account; peer claims specific VPCs in same account.
if 'ALL' in new_accts or this_account in new_accts:
return 'our account-mode dominates peer VPC-scope on shared VPCs'
return ''
if new_mode == 'vpc' and peer_mode == 'account':
if 'ALL' in peer_accts or this_account in peer_accts:
return f'peer account-mode dominates our VPC-scope'
return ''
# Both VPC-scoped
overlap = new_vpcs & peer_vpcs
if overlap:
return f'shared VPC(s): {sorted(overlap)}'
return ''
def check_peers(props, account, region):
own_mpe = props['MpeId']
new_mode = props.get('ScopeMode', 'account')
new_accts = props.get('ScopedAccountIds') or ['ALL']
if isinstance(new_accts, str):
new_accts = [s.strip() for s in new_accts.split(',') if s.strip()]
new_vpcs = props.get('ScopedVpcIds') or []
if isinstance(new_vpcs, str):
new_vpcs = [s.strip() for s in new_vpcs.split(',') if s.strip() and s.strip() != 'NONE']
cfn = boto3.client('cloudformation')
ssm = boto3.client('ssm')
conflicts = []
active = ['CREATE_COMPLETE', 'UPDATE_COMPLETE', 'UPDATE_ROLLBACK_COMPLETE']
for page in cfn.get_paginator('list_stacks').paginate(StackStatusFilter=active):
for s in page.get('StackSummaries', []):
name = s.get('StackName', '')
if name.startswith('StackSet-map-auto-tagger-mig'):
peer_mpe = name[len('StackSet-map-auto-tagger-'):]
elif name.startswith('map-auto-tagger-mig'):
peer_mpe = name[len('map-auto-tagger-'):]
else:
continue
if peer_mpe == own_mpe:
continue # same MPE = in-place update, not a conflict
try:
p = ssm.get_parameter(Name=f'/auto-map-tagger/{peer_mpe}/config')
cfg = json.loads(p['Parameter']['Value'])
except Exception as e:
conflicts.append(f'peer {name} config unreadable ({e.__class__.__name__})')
continue
peer_mode = cfg.get('scope_mode', 'account')
peer_accts = cfg.get('scoped_account_ids') or ['ALL']
peer_vpcs = cfg.get('scoped_vpc_ids') or []
reason = scope_overlap(
new_mode, new_accts, new_vpcs,
peer_mode, peer_accts, peer_vpcs,
account,
)
if reason:
conflicts.append(f'{name} (MPE {peer_mpe}): {reason}')
return conflicts
def handler(event, context):
try:
rt = event.get('RequestType', '')
if rt in ('Update', 'Delete'):
return respond(event, context, 'SUCCESS', f'{rt}: no-op')
# Detect upgrade: CFN sends Create for a new resource, but if a
# tagger stack with this MpeId already exists then the peer-Lambda
# is already live — this is a template upgrade, not a fresh deploy.
# Uses ListStacks (already in PreflightRole IAM) instead of
# DescribeStacks to avoid needing an extra IAM grant.
props = event.get('ResourceProperties', {})
own_mpe = props.get('MpeId', '')
if own_mpe:
own_stack = f'map-auto-tagger-{own_mpe}'
cfn_client = boto3.client('cloudformation')
active = ['CREATE_COMPLETE', 'UPDATE_COMPLETE', 'UPDATE_ROLLBACK_COMPLETE', 'UPDATE_IN_PROGRESS']
for page in cfn_client.get_paginator('list_stacks').paginate(StackStatusFilter=active):
for s in page.get('StackSummaries', []):
if s['StackName'] == own_stack:
return respond(event, context, 'SUCCESS',
f'Upgrade detected ({own_stack} exists): skip preflight')
# stack not found — genuine first deploy, proceed to check
account = context.invoked_function_arn.split(':')[4]
region = os.environ.get('AWS_REGION', '')
conflicts = check_peers(props, account, region)
if conflicts:
reason = (
'Peer tagger scope conflict — stack creation blocked '
f'to prevent §1.108 cross-Lambda contamination. '
f'Conflicts: {"; ".join(conflicts)[:800]}. '
'Resolve via: delete the peer stack, OR narrow this '
'deploy\'s scope to not overlap, OR narrow the peer\'s scope.'
)
print(f'PreflightConflict: {reason}')
return respond(event, context, 'FAILED', reason)
return respond(event, context, 'SUCCESS', 'No peer tagger scope conflict')
except Exception as e:
# Fail-open: any internal error (throttle, IAM propagation,
# region transient) must NOT block a legitimate deploy. The
# runtime PeerTaggerDetected alarm (PR #60) remains as the
# second backstop if a real conflict slips through.
import traceback
print(traceback.format_exc())
return respond(event, context, 'SUCCESS', f'Preflight fail-open ({e.__class__.__name__}): {str(e)[:200]}')
PreflightTrigger:
Type: Custom::PeerTaggerPreflight
# No explicit DependsOn — the ServiceToken reference below already
# creates the implicit dependency on PreflightFunction. Adding an
# explicit DependsOn produces cfn-lint W3005 (redundant dependency).
Properties:
ServiceToken: !GetAtt PreflightFunction.Arn
MpeId: !Ref MpeId
ScopeMode: !Ref ScopeMode
ScopedAccountIds: !Ref ScopedAccountIds
ScopedVpcIds: !Ref ScopedVpcIds
# CloudWatch Log Group — explicit declaration sets retention and ensures the
# group is cleaned up when the stack is deleted, so a customer can redeploy
# with the same MpeId without hitting ResourceExistenceCheck.
AutoTaggerLogGroup:
Type: AWS::Logs::LogGroup
DeletionPolicy: Delete
UpdateReplacePolicy: Delete
Properties:
LogGroupName: !Sub '/aws/lambda/map-auto-tagger-${MpeId}'
RetentionInDays: 14
# Lambda function
AutoTaggerFunction:
Type: AWS::Lambda::Function
DependsOn:
- AutoTaggerLogGroup
- PreflightTrigger # Custom Resource must return SUCCESS before tagger is provisioned
Properties:
FunctionName: !Sub 'map-auto-tagger-${MpeId}'
Runtime: python3.12
Handler: index.handler
Role: !GetAtt AutoTaggerRole.Arn
Timeout: 60
MemorySize: 256
# No ReservedConcurrentExecutions — using SQS buffering instead of direct
# Lambda invocation means throttling delays processing but never drops events.
# Messages are retained in SQS for up to 14 days.
DeadLetterConfig:
TargetArn: !GetAtt EventDLQ.Arn
Environment:
Variables:
CONFIG_PARAM: !Sub '/auto-map-tagger/${MpeId}/config'
ALERT_TOPIC_ARN: !Ref AlertTopic
Code:
ZipFile: |
import json, os, boto3, re, time, random
from datetime import datetime, timezone
from botocore.exceptions import ClientError
# Template version pinned at deploy time. Surfaced in CloudWatch Logs on
# every cold start so ops can trace which version processed an event
# without reading the CFN stack or SSM parameter.
TEMPLATE_VERSION = 'v21.0.6'
print(f'auto-map-tagger {TEMPLATE_VERSION} cold start')
ssm = boto3.client('ssm')
ec2 = boto3.client('ec2')
tagging = boto3.client('resourcegroupstaggingapi')
_config = None
_config_ts = 0.0
# Warm containers live ~15 min; an MPE rotation via SSM wouldn't be
# seen until the container recycled, silently misattributing credit.
# 60s TTL bounds the window without adding meaningful SSM load.
_CONFIG_TTL_SECONDS = 60
# Peer-tagger detection (§1.108, partial). Runs once at cold-start and
# logs a WARN + emits a CloudWatch metric (MapAutoTagger /
# PeerTaggerDetected, dimensions MpeId + PeerMpeId) when another
# map-auto-tagger-mig* stack is found in this account/region.
#
# Why this exists: the configurator's Class-2 scope-intersection
# preflight (PR #24 + PR #38 Option D) hard-fails on overlap at
# deploy.sh run-time, but StackSet-engine provisioning (AutoDeployment:
# True or stack-instance-creation into new OU accounts) bypasses
# deploy.sh entirely. Phase 16 Test 5 confirmed: linked3 had
# migbfltest1 (own deploy, passed preflight) + migph2stack01 (arrived
# via StackSet AutoDeployment, no preflight) — 0/50 resources tagged
# with migbfltest1. This detector doesn't prevent the contamination;
# it surfaces it so customers find out from a CloudWatch alarm rather
# than during a MAP finance audit.
#
# Architectural fix (deterministic multi-Lambda routing) is tracked as
# plan-PR #59 / Wave 28, blocked on design decision with Jin.
def _detect_peer_taggers():
try:
own_mpe = None
cfg_param = os.environ.get('CONFIG_PARAM', '')
# CONFIG_PARAM is /auto-map-tagger/<mpe>/config
m = re.match(r'^/auto-map-tagger/([^/]+)/config$', cfg_param)
if m:
own_mpe = m.group(1)
if not own_mpe:
print('PeerTaggerDetect: could not derive own MpeId from CONFIG_PARAM; skipping')
return
cfn = boto3.client('cloudformation')
paginator = cfn.get_paginator('list_stacks')
peer_mpes = []
active = ['CREATE_COMPLETE', 'UPDATE_COMPLETE', 'UPDATE_ROLLBACK_COMPLETE']
for page in paginator.paginate(StackStatusFilter=active):
for s in page.get('StackSummaries', []):
name = s.get('StackName', '')
if name.startswith('StackSet-map-auto-tagger-mig'):
peer_mpe = name[len('StackSet-map-auto-tagger-'):]
elif name.startswith('map-auto-tagger-mig'):
peer_mpe = name[len('map-auto-tagger-'):]
else:
continue
if peer_mpe == own_mpe:
continue
peer_mpes.append(peer_mpe)
if not peer_mpes:
return
print(
f'PeerTaggerDetect: WARN — {len(peer_mpes)} peer tagger '
f'stack(s) found in this account/region alongside '
f'MpeId={own_mpe}: {peer_mpes}. Concurrent taggers race '
f'on the same CloudTrail events; last-writer wins on '
f'map-migrated tag value. See docs/ARCHITECTURE.md for '
f'deterministic-routing status.'
)
try:
cw = boto3.client('cloudwatch')
cw.put_metric_data(
Namespace='MapAutoTagger',
MetricData=[
*[
{
'MetricName': 'PeerTaggerDetected',
'Dimensions': [
{'Name': 'MpeId', 'Value': own_mpe},
{'Name': 'PeerMpeId', 'Value': peer},
],
'Value': 1,
'Unit': 'Count',
}
for peer in peer_mpes
],
{
'MetricName': 'PeerTaggerDetected',
'Dimensions': [
{'Name': 'MpeId', 'Value': own_mpe},
],
'Value': len(peer_mpes),
'Unit': 'Count',
},
],
)
except Exception as mx:
print(f'PeerTaggerDetect: could not emit PeerTaggerDetected metric: {mx}')
except Exception as e:
# Detector must never break cold start. Missing IAM
# (cloudformation:ListStacks) or throttle means we skip
# today and retry on next cold start.
print(f'PeerTaggerDetect: detector skipped ({e})')
_detect_peer_taggers()
def get_account_from_arn(arn):
"""Extract account ID from ARN."""
parts = arn.split(':')
return parts[4] if len(parts) > 4 else None
def ci_get(d, key):
"""Case-insensitive dict lookup.
Handlers read CloudTrail responseElements / requestParameters fields
that AWS has historically emitted in inconsistent casing — older
services (Kendra, Redshift, Elastic Beanstalk) emit camelCase or
lowercase keys while handlers written against the boto3 SDK
response shape assumed PascalCase. §1.91 / §1.97 / §1.103 document
three live-confirmed silent-miss handlers.
Only use on `responseElements` and `requestParameters` — not on
fixed-shape keys we control. On a response that legitimately has
both `Id` and `id`, this returns whichever sorts first by key
(stable across Python 3.7+ dict insertion order, so effectively
the one AWS emitted first).
"""
if not isinstance(d, dict) or not d:
return None
if key in d:
return d[key]
lk = key.lower()
for k, v in d.items():
if isinstance(k, str) and k.lower() == lk:
return v
return None
def get_config():
global _config, _config_ts
now = time.time()
if _config is None or (now - _config_ts) > _CONFIG_TTL_SECONDS:
try:
resp = ssm.get_parameter(Name=os.environ['CONFIG_PARAM'])
cfg = json.loads(resp['Parameter']['Value'])
except Exception as e:
# One SSM hiccup must not DLQ a burst. Fail closed: return a
# safe-default config where mpe_id is None — is_in_scope()
# hard-rejects that, so nothing tags until the next TTL
# refresh succeeds.
print(f'CONFIG_UNREACHABLE: {e}')
return {
'mpe_id': None,
'scope_mode': 'account',
'scoped_account_ids': ['ALL'],
'scoped_vpc_ids': ['NONE'],
'agreement_start_date': None,
}
# Whitespace strip for customer-edited SSM values. CFN
# CommaDelimitedList strips on deploy; SSM-stored config may
# carry customer-edit whitespace. Drop empties after strip.
if isinstance(cfg.get('scoped_account_ids'), list):
cfg['scoped_account_ids'] = [
s.strip() for s in cfg['scoped_account_ids']
if isinstance(s, str) and s.strip()
]
if isinstance(cfg.get('scoped_vpc_ids'), list):
cfg['scoped_vpc_ids'] = [
s.strip() for s in cfg['scoped_vpc_ids']
if isinstance(s, str) and s.strip()
]
_config = cfg
_config_ts = now
return _config
def is_after_agreement(config):
try:
start = datetime.strptime(config.get('agreement_start_date'), '%Y-%m-%d').replace(tzinfo=timezone.utc)
except (ValueError, TypeError):
print(f"Invalid agreement_start_date '{config.get('agreement_start_date')}', skipping event.")
return False
now = datetime.now(timezone.utc)
if now < start:
return False
end_str = config.get("agreement_end_date")
if end_str:
try:
end = datetime.strptime(end_str, "%Y-%m-%d").replace(tzinfo=timezone.utc)
if now > end:
print(f"Event after agreement_end_date {end_str}, skipping.")
return False
except (ValueError, TypeError):
pass
return True
def is_in_scope(config, account_id, detail):
# Fail-closed guards (§1.3 / §1.129 class). Safe-default config
# from CONFIG_UNREACHABLE has mpe_id=None; if that ever happens
# elsewhere, the scope decision is also False.
if not config.get('mpe_id'):
return False
asd = config.get('agreement_start_date')
try:
datetime.strptime(asd, '%Y-%m-%d')
except (ValueError, TypeError):
print(f"CONFIG_INVALID_AGREEMENT_DATE: '{asd}'")
return False
mode = config['scope_mode']
if mode == 'account':
scoped = config.get('scoped_account_ids', [])
if isinstance(scoped, str):
scoped = [s.strip() for s in scoped.split(',')]
if 'ALL' in scoped:
return True
return account_id in scoped
if mode == 'vpc':
scoped_vpcs = config.get('scoped_vpc_ids', [])
if isinstance(scoped_vpcs, str):
scoped_vpcs = [s.strip() for s in scoped_vpcs.split(',') if s.strip() and s.strip() != 'NONE']
vpc_id = None
resp_els = detail.get('responseElements') or {}
if 'instanceId' in detail:
try:
resp = ec2.describe_instances(InstanceIds=[detail['instanceId']])
vpc_id = resp['Reservations'][0]['Instances'][0].get('VpcId')
except: pass
elif resp_els.get('instancesSet', {}).get('items'):
vpc_id = resp_els['instancesSet']['items'][0].get('vpcId')
elif 'vpcId' in detail:
vpc_id = detail['vpcId']
elif resp_els.get('vpcId'):
vpc_id = resp_els['vpcId']
elif (detail.get('requestParameters') or {}).get('vpcId'):
vpc_id = detail['requestParameters']['vpcId']
elif 'volumeId' in detail:
try:
resp = ec2.describe_volumes(VolumeIds=[detail['volumeId']])
attachments = resp['Volumes'][0].get('Attachments', [])
if attachments:
inst_id = attachments[0]['InstanceId']
resp2 = ec2.describe_instances(InstanceIds=[inst_id])
vpc_id = resp2['Reservations'][0]['Instances'][0].get('VpcId')
except: pass
if vpc_id and vpc_id in scoped_vpcs:
return True
# §1.3 / U5: when scope_mode == 'vpc' and the resource has no
# VPC (S3, DDB, Lambda, SQS, IAM, …), DO NOT fall through to
# account-scope. The customer explicitly chose VPC scoping;
# silently including non-VPC resources defeats that intent
# (e.g. data-residency exclusion). Return False — the resource
# is out of scope.
return False
return False
def extract_arns_multi(detail, account_id, region):
"""Return a list of ARNs for events that create multiple resources.
Used for events where AWS does NOT emit separate CloudTrail events
for each child resource. Returns None if the event is not one of
these; callers should fall back to single-ARN extract_arn.
RunInstances: the CloudTrail responseElements contains instance IDs
and the primary ENI, but the block-device-mapping is usually empty
— AWS populates the attached EBS volume IDs AFTER the instance
reaches pending/running, not in the RunInstances API response.
We therefore describe_instances to fetch the volume IDs.
"""
event_name = detail.get('eventName', '')
if event_name != 'RunInstances':
return None
resp = detail.get('responseElements') or {}
items = resp.get('instancesSet', {}).get('items') or []
if not items:
return None
arns = []
# Gather instance IDs + any attached ENIs present in the event.
# Volumes come from describe_instances below.
inst_ids = []
for it in items:
inst_id = it.get('instanceId')
if inst_id:
arns.append(f"arn:aws:ec2:{region}:{account_id}:instance/{inst_id}")
inst_ids.append(inst_id)
for ni in (it.get('networkInterfaceSet', {}) or {}).get('items') or []:
ni_id = ni.get('networkInterfaceId')
if ni_id:
arns.append(f"arn:aws:ec2:{region}:{account_id}:network-interface/{ni_id}")
# Look up attached volume IDs via describe_instances. CloudTrail
# fires RunInstances milliseconds after the API call returns;
# volumes may not yet be attached. Poll briefly (up to 30s) for
# BlockDeviceMappings to populate. Every account has its own
# Lambda (StackSet architecture), so events are always processed
# in-account — no cross-account client needed.
if inst_ids:
try:
ec2_lookup = boto3.client('ec2', region_name=region)
if ec2_lookup:
deadline = time.time() + 30
volume_ids = set()
while time.time() < deadline:
try:
desc = ec2_lookup.describe_instances(InstanceIds=inst_ids)
all_have_vols = True
volume_ids.clear()
for resv in desc.get('Reservations', []):
for inst in resv.get('Instances', []):
bdm = inst.get('BlockDeviceMappings', []) or []
if not bdm:
all_have_vols = False
for m in bdm:
vid = (m.get('Ebs') or {}).get('VolumeId')
if vid:
volume_ids.add(vid)
if all_have_vols and volume_ids:
break
except ClientError as e:
# Malformed instance IDs never resolve — no
# point burning the 30s budget. NotFound is
# different: the instance may still be
# materializing, so keep polling.
if e.response.get('Error', {}).get('Code') == 'InvalidInstanceID.Malformed':
raise
print(f'describe_instances poll: {e}')
except Exception as e:
print(f'describe_instances poll: {e}')
time.sleep(3)
for vid in volume_ids:
arns.append(f"arn:aws:ec2:{region}:{account_id}:volume/{vid}")
except ClientError as e:
# Let malformed-ID propagate so _classify_error routes
# it to permanent_actionable instead of a silent log.
if e.response.get('Error', {}).get('Code') == 'InvalidInstanceID.Malformed':
raise
print(f'Could not resolve RunInstances attached volumes: {e}')
except Exception as e:
print(f'Could not resolve RunInstances attached volumes: {e}')
return arns or None
def extract_arn(detail, account_id, region):
event_name = detail.get('eventName', '')
event_source = detail.get('eventSource', '')
resp = detail.get('responseElements') or {}
req = detail.get('requestParameters') or {}
resources = detail.get('resources', [])
# Handle early exceptions BEFORE universal pattern scan
# (prevents universal scan from picking up wrong ARNs)
# Bedrock Agent (nested: agent.agentArn)
if event_name == 'CreateAgent' and event_source == 'bedrock.amazonaws.com':
agent_arn = resp.get('agent', {}).get('agentArn')
if agent_arn:
return agent_arn
# Bedrock Agent Action Group (no ARN in response — construct from IDs)
elif event_name == 'CreateAgentActionGroup' and event_source == 'bedrock.amazonaws.com':
ag = resp.get('agentActionGroup', {})
agent_id = ag.get('agentId')
agent_ver = ag.get('agentVersion', 'DRAFT')
ag_id = ag.get('actionGroupId')
if agent_id and ag_id:
return f"arn:aws:bedrock:{region}:{account_id}:agent/{agent_id}/agentversion/{agent_ver}/actiongroup/{ag_id}"