Use your company identities for analytics with Amazon EMR and AWS IAM Id Heart

To allow your workforce customers for analytics with fine-grained knowledge entry controls and audit knowledge entry, you might need to create a number of AWS Id and Entry Administration (IAM) roles with totally different knowledge permissions and map the workforce customers to a type of roles. A number of customers are sometimes mapped to the identical position the place they want related privileges to allow knowledge entry controls on the company person or group stage and audit knowledge entry.

AWS IAM Id Heart allows centralized administration of workforce person entry to AWS accounts and purposes utilizing a neighborhood id retailer or by connecting company directories by way of id suppliers (IdPs). IAM Id Heart now helps trusted id propagation, a streamlined expertise for customers who require entry to knowledge with AWS analytics providers.

Amazon EMR Studio is an built-in improvement setting (IDE) that makes it simple for knowledge scientists and knowledge engineers to construct knowledge engineering and knowledge science purposes. With trusted id propagation, knowledge entry administration could be based mostly on a person’s company id and could be propagated seamlessly as they entry knowledge with single sign-on to construct analytics purposes with Amazon EMR (EMR Studio and Amazon EMR on EC2).

AWS Lake Formation permits knowledge directors to centrally govern, safe, and share knowledge for analytics and machine studying (ML). With trusted id propagation, knowledge directors can instantly present granular entry to company customers utilizing their id attributes and simplify the traceability of end-to-end knowledge entry throughout AWS providers. As a result of entry is managed based mostly on a person’s company id, they don’t want to make use of database native person credentials or assume an IAM position to entry knowledge.

On this put up, we present methods to carry your workforce id to EMR Studio for analytics use instances, instantly handle fine-grained permissions for the company customers and teams utilizing Lake Formation, and audit their knowledge entry.

Resolution overview

For our use case, we need to allow a knowledge analyst person named analyst1 to make use of their very own enterprise credentials to question knowledge they’ve been granted permissions to and audit their knowledge entry. We use Okta because the IdP for this demonstration. The next diagram illustrates the answer structure.

This structure relies on the next elements:

Okta is answerable for sustaining the company person identities, associated teams, and person authentication.
IAM Id Heart connects Okta customers and centrally manages their entry throughout AWS accounts and purposes.
Lake Formation gives fine-grained entry controls on knowledge on to company customers utilizing trusted id propagation.
EMR Studio is an IDE for customers to construct and run purposes. It permits customers to log in instantly with their company credentials with out signing in to the AWS Administration Console.
AWS Service Catalog gives a product template to create EMR clusters.
EMR cluster is built-in with IAM Id Heart utilizing a safety configuration.
AWS CloudTrail captures person knowledge entry actions.

The next are the high-level steps to implement the answer:

Combine Okta with IAM Id Heart.
Arrange Amazon EMR Studio.
Create an IAM Id Heart enabled safety configuration for EMR clusters.
Create a Service Catalog product template to create the EMR clusters.
Use Lake Formation to grant permissions to customers to entry knowledge.
Check the answer by accessing knowledge with a company id.
Audit person knowledge entry.

Conditions

It is best to have the next conditions:

Combine Okta with IAM Id Heart

For extra details about configuring Okta with IAM Id Heart, consult with Configure SAML and SCIM with Okta and IAM Id Heart.

For this setup, we’ve got created two customers, analyst1 and engineer1, and assigned them to the corresponding Okta utility. You may validate the combination is working by navigating to the Customers web page on the IAM Id Heart console, as proven within the following screenshot. Each enterprise customers from Okta are provisioned in IAM Id Heart.

The next actual customers won’t be listed in your account. You may both create related customers or use an current person.

Every provisioned person in IAM Id Heart has a singular person ID. This ID doesn’t originate from Okta; it’s created in IAM Id Heart to uniquely determine this person. With trusted id propagation, this person ID will likely be propagated throughout providers and in addition used for traceability functions in CloudTrail. The next screenshot exhibits the IAM Id Heart person matching the provisioned Okta person analyst1.

Select the hyperlink beneath AWS entry portal URL and log in with the analyst1 Okta person credentials which can be already assigned to this utility.

If you’ll be able to log in and see the touchdown web page, then all of your configurations as much as this step are set appropriately. You’ll not see any purposes on this web page but.

Arrange EMR Studio

On this step, we exhibit the actions wanted from the info lake administrator to arrange EMR Studio enabled for trusted id propagation and with IAM Id Heart integration. This permits customers to instantly entry EMR Studio with their enterprise credentials.

Word: All Amazon S3 buckets (created after January 5, 2023) have encryption configured by default (Amazon S3 managed keys (SSE-S3)), and all new objects which can be uploaded to an S3 bucket are robotically encrypted at relaxation. To make use of a unique kind of encryption, to fulfill your safety wants, please replace the default encryption configuration for the bucket. See Defending knowledge for server-side encryption for additional particulars.

On the Amazon EMR console, select Studios within the navigation pane beneath EMR Studio.
Select Create Studio.

For Setup choices¸ choose Customized.
For Studio title, enter a reputation (for this put up, emr-studio-with-tip).
For S3 location for Workspace storage, choose Choose current location and enter an current S3 bucket (if in case you have one). In any other case, choose Create new bucket.

For Service position to let Studio entry your AWS assets, select View permissions particulars to get the belief and IAM coverage info that’s wanted and create a task with these particular insurance policies in IAM. On this case, we create a brand new position referred to as emr_tip_role.

For Service position to let Studio entry your AWS assets, select the IAM position you created.
For Workspace title, enter a reputation (for this put up, studio-workspace-with-tip).

For Authentication, choose IAM Id Heart.
For Person position¸ you may create a brand new position or select an current position. For this put up, we select the position we created (emr_tip_role).
To make use of the identical position, add the next assertion to the belief coverage of the service position:

{
  "Model": "2008-10-17",
  "Assertion": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "elasticmapreduce.amazonaws.com",
 "AWS": "arn:aws:iam::xxxxxx:role/emr_tip_role"
      },
      "Action": [
              "sts:AssumeRole",
              "sts:SetContext"
              ]
    }
  ]
}

Choose Allow trusted id propagation to mean you can management and log person entry throughout related purposes.

For Select who can entry your utility, choose All customers and teams.

Later, we limit entry to assets utilizing Lake Formation. Nonetheless, there’s an choice right here to limit entry to solely assigned customers and teams.

Within the Networking and safety part, you may present non-compulsory particulars in your VPC, subnets, and safety group settings.
Select Create Studio.

On the Studios web page of the Amazon EMR console, find your Studio enabled with IAM Id Heart.
Copy the hyperlink for Studio Entry URL.

Enter the URL into an online browser and log in utilizing Okta credentials.

It is best to have the ability to efficiently sign up to the EMR Studio console.

Create an AWS Id Heart enabled safety configuration for EMR clusters

EMR safety configurations mean you can configure knowledge encryption, Kerberos authentication, and Amazon S3 authorization for the EMR File System (EMRFS) on the clusters. The safety configuration is out there to make use of and reuse while you create clusters.

To combine Amazon EMR with IAM Id Heart, you could first create an IAM position that authenticates with IAM Id Heart from the EMR cluster. Amazon EMR makes use of IAM credentials to relay the IAM Id Heart id to downstream providers similar to Lake Formation. The IAM position must also have the respective permissions to invoke the downstream providers.

Create a task (for this put up, referred to as emr-idc-application) with the next belief and permission coverage. The position referenced within the belief coverage is the InstanceProfile position for EMR clusters. This permits the EC2 occasion profile to imagine this position and act as an id dealer on behalf of the federated customers.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxxxxn:role/service-role/AmazonEMR-InstanceProfile-20240127T102444"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetContext"
            ]
        }
    ]
}

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "IdCPermissions",
            "Effect": "Allow",
            "Action": [
                "sso-oauth:*"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "GlueandLakePermissions",
            "Impact": "Enable",
            "Motion": [
                "glue:*",
                "lakeformation:GetDataAccess"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "S3Permissions",
            "Impact": "Enable",
            "Motion": [
                "s3:GetDataAccess",
                "s3:GetAccessGrantsInstanceForPrefix"
            ],
            "Useful resource": "*"
        }
    ]
}

Subsequent, you create certificates for encrypting knowledge in transit with Amazon EMR.

For this put up, we use OpenSSL to generate a self-signed X.509 certificates with a 2048-bit RSA non-public key.

The important thing permits entry to the issuer’s EMR cluster situations within the AWS Area getting used. For a whole information on creating and offering a certificates, consult with Offering certificates for encrypting knowledge in transit with Amazon EMR encryption.

Add my-certs.zip to an S3 location that will likely be used to create the safety configuration.

The EMR service position ought to have entry to the S3 location. The important thing permits entry to the issuer’s EMR cluster situations within the us-west-2 Area as specified by the *.us-west-2.compute.inner area title because the widespread title. You may change this to the Area your cluster is in.

$ openssl req -x509 -newkey rsa:2048 -keyout privateKey.pem -out certificateChain.pem -days 365 -nodes -subj '/CN=*.us-west-2.compute.inner'
$ cp certificateChain.pem trustedCertificates.pem
$ zip -r -X my-certs.zip certificateChain.pem privateKey.pem trustedCertificates.pem

Create an EMR safety configuration with IAM Id Heart enabled from the AWS Command Line Interface (AWS CLI) with the next code:

aws emr create-security-configuration --name "IdentityCenterConfiguration-with-lf-tip" --region "us-west-2" --endpoint-url https://elasticmapreduce.us-west-2.amazonaws.com --security-configuration '{
    "AuthenticationConfiguration":{
        "IdentityCenterConfiguration":{
            "EnableIdentityCenter":true,
            "IdentityCenterApplicationAssigmentRequired":false,
            "IdentityCenterInstanceARN": "arn:aws:sso:::occasion/ssoins-7907b0d7d77e3e0d",
            "IAMRoleForEMRIdentityCenterApplicationARN": "arn:aws:iam::1xxxxxxxxx0:position/emr-idc-application"
        }
    },
    "AuthorizationConfiguration": {
        "LakeFormationConfiguration": {
            "EnableLakeFormation": true
        }
    },
    "EncryptionConfiguration": {
        "EnableInTransitEncryption": true,
        "EnableAtRestEncryption": false,
        "InTransitEncryptionConfiguration": {
            "TLSCertificateConfiguration": {
                "CertificateProviderType": "PEM",
                "S3Object": "s3://<<Bucket Identify>>/emr-transit-encry-certs/my-certs.zip"
            }
        }
    }
}'

You may view the safety configuration on the Amazon EMR console.

Create a Service Catalog product template to create EMR clusters

EMR Studio with trusted id propagation enabled can solely work with clusters created from a template. Full the next steps to create a product template in Service Catalog:

On the Service Catalog console, select Portfolios beneath Administration within the navigation pane.
Select Create portfolio.

Enter a reputation in your portfolio (for this put up, EMR Clusters Template) and an non-compulsory description.
Select Create.

On the Portfolios web page, select the portfolio you simply created to view its particulars.

On the Merchandise tab, select Create product.

For Product kind, choose CloudFormation.
For Product title, enter a reputation (for this put up, EMR-7.0.0).
Use the safety configuration IdentityCenterConfiguration-with-lf-tip you created in earlier steps with the suitable Amazon EMR service roles.
Select Create product.

The next is an instance CloudFormation template. Replace the account-specific values for SecurityConfiguration, JobFlowRole, ServiceRole, LogUri, Ec2KeyName, and Ec2SubnetId. We offer a pattern Amazon EMR service position and belief coverage in Appendix A on the finish of this put up.

'Parameters':
  'ClusterName':
    'Kind': 'String'
    'Default': 'EMR_TIP_Cluster'
  'EmrRelease':
    'Kind': 'String'
    'Default': 'emr-7.0.0'
    'AllowedValues':
    - 'emr-7.0.0'
  'ClusterInstanceType':
    'Kind': 'String'
    'Default': 'm5.xlarge'
    'AllowedValues':
    - 'm5.xlarge'
    - 'm5.2xlarge'
'Sources':
  'EmrCluster':
    'Kind': 'AWS::EMR::Cluster'
    'Properties':
      'Functions':
      - 'Identify': 'Spark'
      - 'Identify': 'Livy'
      - 'Identify': 'Hadoop'
      - 'Identify': 'JupyterEnterpriseGateway'       
      'SecurityConfiguration': 'IdentityCenterConfiguration-with-lf-tip'
      'EbsRootVolumeSize': '20'
      'Identify':
        'Ref': 'ClusterName'
      'JobFlowRole': <Occasion Profile Function>
      'ServiceRole': <EMR Service Function>
      'ReleaseLabel':
        'Ref': 'EmrRelease'
      'VisibleToAllUsers': !!bool 'true'
      'LogUri':
        'Fn::Sub': <S3 LOG Path>
      'Situations':
        "Ec2KeyName" : <Key Pair Identify>
        'TerminationProtected': !!bool 'false'
        'Ec2SubnetId': <subnet-id>
        'MasterInstanceGroup':
          'InstanceCount': !!int '1'
          'InstanceType':
            'Ref': 'ClusterInstanceType'
        'CoreInstanceGroup':
          'InstanceCount': !!int '2'
          'InstanceType':
            'Ref': 'ClusterInstanceType'
          'Market': 'ON_DEMAND'
          'Identify': 'Core'
'Outputs':
  'ClusterId':
    'Worth':
      'Ref': 'EmrCluster'
    'Description': 'The ID of the  EMR cluster'
'Metadata':
  'AWS::CloudFormation::Designer': {}
'Guidelines': {}

Trusted id propagation is supported from Amazon EMR 6.15 onwards. For Amazon EMR 6.15, add the next bootstrap motion to the CloudFormation script:

'BootstrapActions':
- 'Identify': 'spark-config'
'ScriptBootstrapAction':
'Path': 's3://emr-data-access-control-<aws-region>/customer-bootstrap-actions/idc-fix/replace-puppet.sh'

The portfolio now ought to have the EMR cluster creation product added.

Grant the EMR Studio position emr_tip_role entry to the portfolio.

Grant Lake Formation permissions to customers to entry knowledge

On this step, we allow Lake Formation integration with IAM Id Heart and grant permissions to the Id Heart person analyst1. If Lake Formation shouldn’t be already enabled, consult with Getting began with Lake Formation.

To make use of Lake Formation with Amazon EMR, create a customized position to register S3 places. It’s essential to create a brand new customized position with Amazon S3 entry and never use the default position AWSServiceRoleForLakeFormationDataAccess. Moreover, allow exterior knowledge filtering in Lake Formation. For extra particulars, consult with Allow Lake Formation with Amazon EMR.

Full the next steps to handle entry permissions in Lake Formation:

On the Lake Formation console, select IAM Id Heart integration beneath Administration within the navigation pane.

Lake Formation will robotically specify the right IAM Id Heart occasion.

Now you can view the IAM Id Heart integration particulars.

For this put up, we’ve got a Advertising database and a buyer desk on which we grant entry to our enterprise person analyst1. You should utilize an current database and desk in your account or create a brand new one. For extra examples, consult with Tutorials.

The next screenshot exhibits the main points of our buyer desk.

Full the next steps to grant analyst1 permissions. For extra info, consult with Granting desk permissions utilizing the named useful resource methodology.

On the Lake Formation console, select Knowledge lake permissions beneath Permissions within the navigation pane.
Select Grant.

Choose Named Knowledge Catalog assets.
For Databases, select your database (advertising and marketing).
For Tables, select your desk (buyer).

For Desk permissions, choose Choose and Describe.
For Knowledge permissions, choose All knowledge entry.
Select Grant.

The next screenshot exhibits a abstract of permissions that person analyst1 has. They’ve Choose entry on the desk and Describe permissions on the databases.

Check the answer

To check the answer, we log in to EMR Studio as enterprise person analyst1, create a brand new Workspace, create an EMR cluster utilizing a template, and use that cluster to carry out an evaluation. You possibly can additionally use the Workspace that was created through the Studio setup. On this demonstration, we create a brand new Workspace.

You want extra permissions within the EMR Studio position to create and listing Workspaces, use a template, and create EMR clusters. For extra particulars, consult with Configure EMR Studio person permissions for Amazon EC2 or Amazon EKS. Appendix B on the finish of this put up comprises a pattern coverage.

When the cluster is out there, we connect the cluster to the Workspace and run queries on the buyer desk, which the person has entry to.

Person analyst1 is now in a position to run queries for enterprise use instances utilizing their company id. To open a PySpark pocket book, we select PySpark beneath Pocket book.

When the pocket book is open, we run a Spark SQL question to listing the databases:

On this case, we question the buyer desk within the advertising and marketing database. We should always have the ability to entry the info.

%%sql
choose * from advertising and marketing.buyer

Audit knowledge entry

Lake Formation API actions are logged by CloudTrail. The GetDataAccess motion is logged every time a principal or built-in AWS service requests non permanent credentials to entry knowledge in a knowledge lake location that’s registered with Lake Formation. With trusted id propagation, CloudTrail additionally logs the IAM Id Heart person ID of the company id who requested entry to the info.

The next screenshot exhibits the main points for the analyst1 person.

Select View occasion to view the occasion logs.

The next is an instance of the GetDataAccess occasion log. We will hint that person analyst1, Id Heart person ID c8c11390-00a1-706e-0c7a-bbcc5a1c9a7f, has accessed the buyer desk.

{
    "eventVersion": "1.09",
    
….
        "onBehalfOf": {
            "userId": "c8c11390-00a1-706e-0c7a-bbcc5a1c9a7f",
            "identityStoreArn": "arn:aws:identitystore::xxxxxxxxx:identitystore/d-XXXXXXXX"
        }
    },
    "eventTime": "2024-01-28T17:56:25Z",
    "eventSource": "lakeformation.amazonaws.com",
    "eventName": "GetDataAccess",
    "awsRegion": "us-west-2",
….
        "requestParameters": {
        "tableArn": "arn:aws:glue:us-west-2:xxxxxxxxxx:desk/advertising and marketing/buyer",
        "supportedPermissionTypes": [
            "TABLE_PERMISSION"
        ]
    },
    …..
    }
}

Right here is an finish to finish demonstration video of steps to comply with for enabling trusted id propagation to your analytics circulation in Amazon EMR

Clear up

Clear up the next assets while you’re carried out utilizing this answer:

Conclusion

On this put up, we demonstrated methods to arrange and use trusted id propagation utilizing IAM Id Heart, EMR Studio, and Lake Formation for analytics. With trusted id propagation, a person’s company id is seamlessly propagated as they entry knowledge utilizing single sign-on throughout AWS analytics providers to construct analytics purposes. Knowledge directors can present fine-grained knowledge entry on to company customers and teams and audit utilization. To study extra, see Combine Amazon EMR with AWS IAM Id Heart.

Concerning the Authors

Pradeep Misra is a Principal Analytics Options Architect at AWS. He works throughout Amazon to architect and design fashionable distributed analytics and AI/ML platform options. He’s obsessed with fixing buyer challenges utilizing knowledge, analytics, and AI/ML. Outdoors of labor, Pradeep likes exploring new locations, attempting new cuisines, and taking part in board video games along with his household. He additionally likes doing science experiments along with his daughters.

Deepmala Agarwal works as an AWS Knowledge Specialist Options Architect. She is obsessed with serving to prospects construct out scalable, distributed, and data-driven options on AWS. When not at work, Deepmala likes spending time with household, strolling, listening to music, watching films, and cooking!

Abhilash Nagilla is a Senior Specialist Options Architect at Amazon Net Companies (AWS), serving to public sector prospects on their cloud journey with a give attention to AWS analytics providers. Outdoors of labor, Abhilash enjoys studying new applied sciences, watching films, and visiting new locations.

Appendix A

Pattern Amazon EMR service position and belief coverage:

Word: It is a pattern service position. High quality grained entry management is finished utilizing Lake Formation. Modify the permissions as per your enterprise steering and to conform along with your safety crew.

Belief coverage:

{
    "Model": "2008-10-17",
    "Assertion": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "elasticmapreduce.amazonaws.com",
   "AWS": "arn:aws:iam::xxxxxx:role/emr_tip_role"

            },
            "Action": [
                "sts:AssumeRole",
                "sts:SetContext"
            ]
        }
    ]
}

Permission Coverage:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "ResourcesToLaunchEC2",
            "Effect": "Allow",
            "Action": [
                "ec2:RunInstances",
                "ec2:CreateFleet",
                "ec2:CreateLaunchTemplate",
                "ec2:CreateLaunchTemplateVersion"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*::image/ami-*",
                "arn:aws:ec2:*:*:key-pair/*",
                "arn:aws:ec2:*:*:capacity-reservation/*",
                "arn:aws:ec2:*:*:placement-group/pg-*",
                "arn:aws:ec2:*:*:fleet/*",
                "arn:aws:ec2:*:*:dedicated-host/*",
                "arn:aws:resource-groups:*:*:group/*"
            ]
        },
        {
            "Sid": "TagOnCreateTaggedEMRResources",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:volume/*",
                "arn:aws:ec2:*:*:launch-template/*"
            ],
            "Situation": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "RunInstances",
                        "CreateFleet",
                        "CreateLaunchTemplate",
                        "CreateNetworkInterface"
                    ]
                }
            }
        },
        {
            "Sid": "ListActionsForEC2Resources",
            "Impact": "Enable",
            "Motion": [
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeCapacityReservations",
                "ec2:DescribeDhcpOptions",
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeLaunchTemplates",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribePlacementGroups",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcs"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AutoScaling",
            "Impact": "Enable",
            "Motion": [
                "application-autoscaling:DeleteScalingPolicy",
                "application-autoscaling:DeregisterScalableTarget",
                "application-autoscaling:DescribeScalableTargets",
                "application-autoscaling:DescribeScalingPolicies",
                "application-autoscaling:PutScalingPolicy",
                "application-autoscaling:RegisterScalableTarget"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AutoScalingCloudWatch",
            "Impact": "Enable",
            "Motion": [
                "cloudwatch:PutMetricAlarm",
                "cloudwatch:DeleteAlarms",
                "cloudwatch:DescribeAlarms"
            ],
            "Useful resource": "arn:aws:cloudwatch:*:*:alarm:*_EMR_Auto_Scaling"
        },
        {
            "Sid": "PassRoleForAutoScaling",
            "Impact": "Enable",
            "Motion": "iam:PassRole",
            "Useful resource": "arn:aws:iam::*:position/EMR_AutoScaling_DefaultRole",
            "Situation": {
                "StringLike": {
                    "iam:PassedToService": "application-autoscaling.amazonaws.com*"
                }
            }
        },
        {
            "Sid": "PassRoleForEC2",
            "Impact": "Enable",
            "Motion": "iam:PassRole",
            "Useful resource": "arn:aws:iam::xxxxxxxxxxx:position/service-role/<Occasion-Profile-Function>",
            "Situation": {
                "StringLike": {
                    "iam:PassedToService": "ec2.amazonaws.com*"
                }
            }
        },
        {
            "Impact": "Enable",
            "Motion": [
                "s3:*",
                "s3-object-lambda:*"
            ],
            "Useful resource": [
                "arn:aws:s3:::<bucket>/*",
                "arn:aws:s3:::*logs*/*"
            ]
        },
        {
            "Impact": "Enable",
            "Useful resource": "*",
            "Motion": [
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateFleet",
                "ec2:CreateLaunchTemplate",
                "ec2:CreateNetworkInterface",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DeleteLaunchTemplate",
                "ec2:DeleteNetworkInterface",
                "ec2:DeleteSecurityGroup",
                "ec2:DeleteTags",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeDhcpOptions",
                "ec2:DescribeImages",
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeLaunchTemplates",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribePrefixLists",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSpotInstanceRequests",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSubnets",
                "ec2:DescribeTags",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcEndpointServices",
                "ec2:DescribeVpcs",
                "ec2:DetachNetworkInterface",
                "ec2:ModifyImageAttribute",
                "ec2:ModifyInstanceAttribute",
                "ec2:RequestSpotInstances",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RunInstances",
                "ec2:TerminateInstances",
                "ec2:DeleteVolume",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DetachVolume",
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:ListInstanceProfiles",
                "iam:ListRolePolicies",
                "cloudwatch:PutMetricAlarm",
                "cloudwatch:DescribeAlarms",
                "cloudwatch:DeleteAlarms",
                "application-autoscaling:RegisterScalableTarget",
                "application-autoscaling:DeregisterScalableTarget",
                "application-autoscaling:PutScalingPolicy",
                "application-autoscaling:DeleteScalingPolicy",
                "application-autoscaling:Describe*"
            ]
        }
    ]
}

Appendix B

Pattern EMR Studio position coverage:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "AllowEMRReadOnlyActions",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:ListInstances",
                "elasticmapreduce:DescribeCluster",
                "elasticmapreduce:ListSteps"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AllowEC2ENIActionsWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateNetworkInterfacePermission",
                "ec2:DeleteNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowEC2ENIAttributeAction",
            "Impact": "Enable",
            "Motion": [
                "ec2:ModifyNetworkInterfaceAttribute"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*:*:security-group/*"
            ]
        },
        {
            "Sid": "AllowEC2SecurityGroupActionsWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:DeleteNetworkInterfacePermission"
            ],
            "Useful resource": "*",
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowDefaultEC2SecurityGroupsCreationWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateSecurityGroup"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowDefaultEC2SecurityGroupsCreationInVPCWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateSecurityGroup"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:vpc/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowAddingEMRTagsDuringDefaultSecurityGroupCreation",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": "arn:aws:ec2:*:*:security-group/*",
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true",
                    "ec2:CreateAction": "CreateSecurityGroup"
                }
            }
        },
        {
            "Sid": "AllowEC2ENICreationWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:network-interface/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:RequestTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowEC2ENICreationInSubnetAndSecurityGroupWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateNetworkInterface"
            ],
            "Useful resource": [
                "arn:aws:ec2:*:*:subnet/*",
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowAddingTagsDuringEC2ENICreation",
            "Impact": "Enable",
            "Motion": [
                "ec2:CreateTags"
            ],
            "Useful resource": "arn:aws:ec2:*:*:network-interface/*",
            "Situation": {
                "StringEquals": {
                    "ec2:CreateAction": "CreateNetworkInterface"
                }
            }
        },
        {
            "Sid": "AllowEC2ReadOnlyActions",
            "Impact": "Enable",
            "Motion": [
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribeTags",
                "ec2:DescribeInstances",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcs"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "AllowSecretsManagerReadOnlyActionsWithEMRTags",
            "Impact": "Enable",
            "Motion": [
                "secretsmanager:GetSecretValue"
            ],
            "Useful resource": "arn:aws:secretsmanager:*:*:secret:*",
            "Situation": {
                "StringEquals": {
                    "aws:ResourceTag/for-use-with-amazon-emr-managed-policies": "true"
                }
            }
        },
        {
            "Sid": "AllowWorkspaceCollaboration",
            "Impact": "Enable",
            "Motion": [
                "iam:GetUser",
                "iam:GetRole",
                "iam:ListUsers",
                "iam:ListRoles",
                "sso:GetManagedApplicationInstance",
                "sso-directory:SearchUsers"
            ],
            "Useful resource": "*"
        },
        {
            "Sid": "S3Access",
            "Impact": "Enable",
            "Motion": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetEncryptionConfiguration",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Useful resource": [
                "arn:aws:s3:::<bucket>",
                "arn:aws:s3:::<bucket>/*"
            ]
        },
        {
            "Sid": "EMRStudioWorkspaceAccess",
            "Impact": "Enable",
            "Motion": [
                "elasticmapreduce:CreateEditor",
                "elasticmapreduce:DescribeEditor",
                "elasticmapreduce:ListEditors",
                "elasticmapreduce:DeleteEditor",
                "elasticmapreduce:UpdateEditor",
                "elasticmapreduce:PutWorkspaceAccess",
                "elasticmapreduce:DeleteWorkspaceAccess",
                "elasticmapreduce:ListWorkspaceAccessIdentities",
                "elasticmapreduce:StartEditor",
                "elasticmapreduce:StopEditor",
                "elasticmapreduce:OpenEditorInConsole",
                "elasticmapreduce:AttachEditor",
                "elasticmapreduce:DetachEditor",
                "elasticmapreduce:ListInstanceGroups",
                "elasticmapreduce:ListBootstrapActions",
                "servicecatalog:SearchProducts",
                "servicecatalog:DescribeProduct",
                "servicecatalog:DescribeProductView",
                "servicecatalog:DescribeProvisioningParameters",
                "servicecatalog:ProvisionProduct",
                "servicecatalog:UpdateProvisionedProduct",
                "servicecatalog:ListProvisioningArtifacts",
                "servicecatalog:DescribeRecord",
                "servicecatalog:ListLaunchPaths",
                "elasticmapreduce:RunJobFlow",      
                "elasticmapreduce:ListClusters",
                "elasticmapreduce:DescribeCluster",
                "codewhisperer:GenerateRecommendations",
                "athena:StartQueryExecution",
                "athena:StopQueryExecution",
                "athena:GetQueryExecution",
                "athena:GetQueryRuntimeStatistics",
                "athena:GetQueryResults",
                "athena:ListQueryExecutions",
                "athena:BatchGetQueryExecution",
                "athena:GetNamedQuery",
                "athena:ListNamedQueries",
                "athena:BatchGetNamedQuery",
                "athena:UpdateNamedQuery",
                "athena:DeleteNamedQuery",
                "athena:ListDataCatalogs",
                "athena:GetDataCatalog",
                "athena:ListDatabases",
                "athena:GetDatabase",
                "athena:ListTableMetadata",
                "athena:GetTableMetadata",
                "athena:ListWorkGroups",
                "athena:GetWorkGroup",
                "athena:CreateNamedQuery",
                "athena:GetPreparedStatement",
                "glue:CreateDatabase",
                "glue:DeleteDatabase",
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:UpdateDatabase",
                "glue:CreateTable",
                "glue:DeleteTable",
                "glue:BatchDeleteTable",
                "glue:UpdateTable",
                "glue:GetTable",
                "glue:GetTables",
                "glue:BatchCreatePartition",
                "glue:CreatePartition",
                "glue:DeletePartition",
                "glue:BatchDeletePartition",
                "glue:UpdatePartition",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:BatchGetPartition",
                "kms:ListAliases",
                "kms:ListKeys",
                "kms:DescribeKey",
                "lakeformation:GetDataAccess",
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts",
                "s3:AbortMultipartUpload",
                "s3:PutObject",
                "s3:PutBucketPublicAccessBlock",
                "s3:ListAllMyBuckets",
                "elasticmapreduce:ListStudios",
                "elasticmapreduce:DescribeStudio",
                "cloudformation:GetTemplate",
                "cloudformation:CreateStack",
                "cloudformation:CreateStackSet",
                "cloudformation:DeleteStack",
                "cloudformation:GetTemplateSummary",
                "cloudformation:ValidateTemplate",
                "cloudformation:ListStacks",
                "cloudformation:ListStackSets",
                "elasticmapreduce:AddTags",
                "ec2:CreateNetworkInterface",
                "elasticmapreduce:GetClusterSessionCredentials",
                "elasticmapreduce:GetOnClusterAppUIPresignedURL",
                "cloudformation:DescribeStackResources"
            ],
            "Useful resource": [
                "*"
            ]
        },
        {
            "Sid": "AllowPassingServiceRoleForWorkspaceCreation",
            "Motion": "iam:PassRole",
            "Useful resource": [
                "arn:aws:iam::*:role/<Studio Role>",
                "arn:aws:iam::*:role/<EMR Service Role>",
                "arn:aws:iam::*:role/<EMR Instance Profile Role>"
            ],
            "Impact": "Enable"
        },
{
			"Sid": "Statement1",
			"Impact": "Enable",
			"Motion": [
				"iam:PassRole"
			],
			"Useful resource": [
				"arn:aws:iam::*:role/<EMR Instance Profile Role>"
			]
		}
    ]
}