From siloed knowledge to unified insights: Cross-account Athena Entry for Amazon Fast

Amazon Fast is an AI-powered unified intelligence service that brings collectively a company’s knowledge, structured knowledge and unstructured enterprise content material like paperwork, emails, and data bases right into a single service the place anybody can discover, analyze, and take motion. With over 40 utility integrations, Fast bridges the last-mile hole between insights and motion so customers can perceive their knowledge and act on it instantly.

Amazon Fast Sight, the enterprise intelligence (BI) functionality of Amazon Fast, is a unified BI service. It offers fashionable interactive dashboards, pure language querying, pixel-perfect reviews, machine studying (ML) insights, and embedded analytics at scale. Amazon Fast brings collectively AI brokers for enterprise insights, analysis, and automation in a single built-in expertise, serving to you’re employed smarter and sooner whereas sustaining safety and entry insurance policies.

Amazon Athena is a serverless, interactive question service that’s used to research knowledge instantly in Amazon Easy Storage Service (Amazon S3) utilizing customary SQL, with no infrastructure to handle and no knowledge to load. You level Athena at your knowledge saved in Amazon S3, outline the schema utilizing the AWS Glue Information Catalog, and begin querying.

Many enterprises centralize their Amazon Fast deployment in a single AWS account whereas their knowledge resides throughout a number of enterprise unit accounts. A monetary companies firm may run Fast in a central AWS account, whereas retail banking knowledge lives in Account A, funding banking in Account B, and danger administration in Account C. Till now, querying Amazon Athena knowledge throughout these accounts meant both managing a number of Fast subscriptions or absorbing all question prices within the central account.

At the moment, we’re saying cross-account Athena entry for Amazon Fast. With this function, prospects can question Athena knowledge in different AWS accounts utilizing AWS Identification and Entry Administration (IAM) position chaining, with question prices billed to the account the place the info resides. Within the context of cross-account Athena entry, position chaining allows Amazon Fast in a writer account to imagine a task within the buyer’s client account, which in flip has permissions to question knowledge in Athena and the AWS Glue Information Catalog with out sharing long-term credentials throughout account boundaries. On this publish, we stroll via the end-to-end setup: creating the IAM roles, configuring belief insurance policies, creating the cross-account knowledge supply in Fast, and constructing datasets from it.

Time period definitions

• Central Fast Account (Supply Account): The AWS account the place Amazon Fast is deployed
• Shopper Account: An AWS account the place Athena knowledge property (databases, tables, S3 knowledge) reside, accessed from the central Fast account
• RunAsRole (Position A): An IAM position within the central Fast account that Fast assumes first; holds no knowledge permissions, solely permission to chain into client account roles
• Shopper Account Position (Position B): An IAM position in every client account that grants Athena, AWS Glue, and S3 entry; trusts Position A
• Position Chaining: A two-step credential course of the place Fast assumes the RunAsRole, then makes use of these credentials to imagine the patron account position
• ExternalId: A safety situation (set to the DataSource ARN) utilized in belief insurance policies to stop confused deputy assaults throughout position assumption
• Scope-Down Coverage: An inline IAM coverage hooked up at runtime to limit chained credentials to solely assuming the particular client account position
• Athena Workgroup: The Athena execution surroundings within the client account below which queries run and prices are tracked

Answer overview

The answer makes use of a two-step position chaining mechanism involving two IAM roles:

1. Position A (RunAsRole) – lives within the central Fast account. Fast assumes this position first.
2. Position B (Shopper Account Position) – lives within the client account the place Athena knowledge resides. Position A chains into Position B to execute queries.

When a Fast person runs a question, the service assumes Position A, then makes use of these credentials to imagine Position B within the client account. Athena executes the question utilizing Position B’s credentials, so compute prices are billed to the patron account.

Stipulations

Earlier than you start, be certain the next are in place:

• Amazon Fast Enterprise Version lively within the central account
• IAM administrative entry in each accounts. You’ll create and configure roles in every
• AWS Command Line Interface (AWS CLI) put in and configured with credentials for each accounts, or entry to the AWS Administration Console
• Familiarity with IAM ideas, particularly belief insurance policies, permission insurance policies, and position assumption (sts:AssumeRole)
• Athena workgroup configured within the client account (the default main workgroup works for getting began)
• S3 bucket within the client account for Athena question outcomes (usually prefixed aws-athena-query-results-*)

Observe: To attach a number of client accounts, repeat the position setup steps for every account. Plan your IAM position naming conference prematurely to streamline administration at scale.

Technical structure

As organizations undertake lakehouse architectures and distribute knowledge throughout enterprise models, AWS Areas, and AWS accounts, they want a approach to question that knowledge centrally with out transferring it. Cross-account Athena entry for Amazon Fast addresses this requirement. Utilizing IAM position chaining, a central Fast deployment can attain into distributed knowledge shops throughout account boundaries with out knowledge replication, shared long-lived credentials, or a number of Fast subscriptions. The structure scales from a two-account proof of idea to an enterprise-wide deployment. On this part, we describe three patterns, every constructing on the earlier one.

Sample 1: Fundamental two account setup

Essentially the most simple deployment connects one central Fast account to at least one client account. That is the minimal viable configuration and maps on to the step-by-step walkthrough on this publish. The position chain described within the Answer Overview (Fast assumes Position A, Position A chains into Position B utilizing sts:AssumeRole, and Athena executes the question below Position B’s credentials) applies instantly right here. This sample is properly fitted to preliminary validation or for connecting a single enterprise unit’s knowledge to a shared (central) BI AWS account.

Sample 2: Hub and Spoke

Most enterprises centralize their Fast deployment in a single account (the hub) whereas knowledge is distributed throughout a number of enterprise unit accounts (the spokes). The hub-and-spoke mannequin extends the fundamental setup: Position A’s permission coverage lists a number of client position ARNs, which might reside in the identical account or throughout totally different accounts, and a separate knowledge supply is created in Fast for every. The important thing benefit of this sample is independence between spokes. Including a brand new enterprise unit requires creating a brand new Position B in that unit’s account and registering a brand new knowledge supply in Fast, with no modifications to present spokes. Every spoke controls its personal Position B permissions, figuring out which tables and S3 prefixes are uncovered to the central BI AWS Account. Value attribution follows naturally, every spoke’s Athena queries are billed to its personal account. As a result of a single Fast dashboard can reference knowledge sources from a number of client accounts, BI authors can construct cross-business-unit analytics with out leaving the unified Fast expertise. That is the really helpful sample for many enterprises.

As you scale past a handful of spokes, take into account templatizing the consumer-side setup. An AWS CloudFormation or CDK template for Position B, the Athena workgroup, and the required belief and permission insurance policies permits enterprise unit groups to self-service their onboarding, or the central BI staff to provision new spokes with a single stack deployment.

Sample 3: Information Mesh

In an information mesh, producers and shoppers are distinct accounts. A producer account owns and manages its uncooked knowledge, provisioning it right into a Shopper Account. For instance, utilizing AWS Lake Formation with AWS Useful resource Entry Supervisor (AWS RAM) to share tables throughout account boundaries. The particular provisioning mechanism is the area staff’s alternative and out of doors the scope. The Shopper Account, containing Position B, AWS Glue Information Catalog, and Athena workgroup, is what Amazon Fast connects to via position chain. The account publishes ruled knowledge merchandise to different Shopper Accounts utilizing AWS Glue Information Catalog useful resource insurance policies, making its knowledge accessible for cross-domain analytics. This peer-to-peer sharing between Shopper Accounts, every performing as each producer and client of information merchandise, is what defines the mesh.A BI writer in Fast can question throughout a number of Shopper Accounts in a single dashboard, with question prices attributed per Shopper Account. At enterprise scale, a single Amazon Fast deployment can hook up with tons of of Shopper Accounts. How area accounts provision uncooked knowledge into Shopper Accounts is exterior of scope.

Selecting a sample

The suitable sample depends upon your knowledge technique. The fundamental two-account setup validates the position chain finish to finish. Most enterprises will undertake hub-and-spoke as they onboard further enterprise models as a result of it retains spokes impartial, price attribution clear, and belief insurance policies simple. Organizations with robust knowledge possession boundaries or devoted area groups will naturally evolve into the info mesh sample, the place Shopper Accounts are distinct from the producer accounts that provide the info, and Amazon Fast serves because the unified analytics layer throughout all of them. We advocate beginning small and increasing as necessities develop.

Wanting forward

As agentic AI capabilities mature and AI brokers start to autonomously question, rework, and act on knowledge throughout organizational boundaries, the power to entry knowledge the place it lives, ruled, cost-attributed, and auditable, turns into foundational. Cross-account Athena entry is a constructing block for that future. At the moment, it connects Fast dashboards to distributed knowledge lakes. As agentic patterns evolve, the identical IAM position chaining mechanism can prolong to AI brokers that question Athena on behalf of enterprise customers, apply governance guidelines in actual time, and route compute prices to the info proprietor with out centralizing knowledge right into a single account.

Answer

Cross-account Athena entry for Amazon Fast makes use of IAM position chaining to bridge your central Fast account with a number of client accounts the place your knowledge lives. Quite than consolidating knowledge right into a single account or managing separate Fast subscriptions per enterprise unit, you configure two IAM roles that work in tandem (one in every account) to route queries and attribute prices accurately.

Create Position A (RunAsRole) within the Central Fast account

Position A lives within the central Fast account. Amazon Fast assumes this position when a person initiates a question. Position A holds no knowledge permissions of its personal; its sole function is to chain into the patron account. It wants two issues:

1. A belief coverage permitting the Fast service to imagine it.
2. A permission coverage permitting it to imagine Position B within the client account.

Create the belief coverage and identify it as role-a-trust-policy.json. This permits the Fast service principal to imagine Position A, scoped to knowledge supply operations in your account. Pattern coverage:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "quicksight.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringLike": {
                "aws:SourceAccount": "",
                "aws:SourceArn": "arn:aws:quicksight:*::datasource/*"
                }
            }
        }
    ]
}

Create the position utilizing the AWS CLI

aws iam create-role 
--role-name qs-athena-cross-account-role-a 
--assume-role-policy-document file://role-a-trust-policy.json" 
--description "Fast Athena Cross Account - RunAsRole"

Create the permissions coverage and identify it role-a-permission-policy.json.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
        "Effect": "Allow",
        "Action": "sts:AssumeRole",
        "Resource": "arn:aws:iam:::role/",
        "Condition": {
            "StringLike": {
            "sts:ExternalId": "arn:aws:quicksight:*::datasource/*"
                }
            }
        }
    ]
}

Connect the permission coverage as an inline coverage utilizing AWS CLI. This permits Position A to imagine Position B within the client account. The ExternalId situation ensures solely Fast knowledge sources can set off the position chain:

aws iam put-role-policy 
--role-name qs-athena-cross-account-role-a 
--policy-name AssumeConsumerRolePolicy 
--policy-document file://role-a-permission-policy.json 

Moreover, the IAM principal creating the Fast knowledge supply wants iam:PassRole permission on Position A.

Create Position B within the Shopper Account

Position B lives within the client account the place your Athena tables, AWS Glue Information Catalog, and S3 knowledge reside. Position A assumes Position B to execute the question.

Create the belief coverage and identify it role-b-trust-policy.json. This permits the Fast account to imagine Position B, with an ExternalId situation tied to the info supply ARN. Pattern coverage:

{
"Model": "2012-10-17",
"Assertion": [
        {
        "Effect": "Allow",
        "Principal": {
        "AWS": "arn:aws:iam:::root"
            },
        "Action": "sts:AssumeRole",
        "Condition": {
            "StringLike": {
            "sts:ExternalId": "arn:aws:quicksight:*::datasource/*"
                }
            }
        }
    ]
}

Create the position utilizing the AWS CLI:

aws iam create-role 
--role-name qs-athena-consumer-role 
--assume-role-policy-document file://role-b-trust-policy.json 
--description "Fast Athena Cross Account - Shopper Account Position"

Create the Athena, AWS Glue, and S3 permissions coverage and identify it role-b-permission-policy.json. Scope these to the particular databases, tables, and S3 places related to your knowledge.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": [
                "athena:BatchGetQueryExecution",
                "athena:CancelQueryExecution",
                "athena:GetCatalogs",
                "athena:GetExecutionEngine",
                "athena:GetExecutionEngines",
                "athena:GetNamespace",
                "athena:GetNamespaces",
                "athena:GetQueryExecution",
                "athena:GetQueryExecutions",
                "athena:GetQueryResults",
                "athena:GetQueryResultsStream",
                "athena:GetTable",
                "athena:GetTables",
                "athena:ListQueryExecutions",
                "athena:RunQuery",
                "athena:StartQueryExecution",
                "athena:StopQueryExecution",
                "athena:ListWorkGroups",
                "athena:ListEngineVersions",
                "athena:GetWorkGroup",
                "athena:GetDataCatalog",
                "athena:GetDatabase",
                "athena:GetTableMetadata",
                "athena:ListDataCatalogs",
                "athena:ListDatabases",
                "athena:ListTableMetadata"
            ],
            "Useful resource": [
        "arn:aws:athena:::workgroup/",
        "arn:aws:athena:::datacatalog/"
      ]
        },
        {
            "Impact": "Permit",
            "Motion": [
                "glue:GetCatalog",
                "glue:GetCatalogs",
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:GetTable",
                "glue:GetTables",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:BatchGetPartition"
            ],
            "Useful resource": [ "arn:aws:glue:::catalog",
                        "arn:aws:glue:::database/",
                        "arn:aws:glue:::table//*"
                    ]
        },
        {
            "Impact": "Permit",
            "Motion": [
                "s3:GetBucketLocation",
                "s3:GetObject",
				"s3:PutObject",
				"s3:AbortMultipartUpload",		 
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts"
            ],
            "Useful resource": [
                "arn:aws:s3:::",
                "arn:aws:s3:::/*",
                "arn:aws:s3:::aws-athena-query-results-*",
                "arn:aws:s3:::aws-athena-query-results-*/*"
            ]
        }
    ]
}

Connect the permission coverage as an inline coverage utilizing AWS CLI

aws iam put-role-policy 
--role-name qs-athena-consumer-role 
--policy-name AthenaGlueS3Access 
--policy-document file://role-b-permission-policy.json

Create the Cross-Account Athena Information Supply in Fast (central or supply account)

Utilizing the AWS CLI, enter the next:

aws quicksight create-data-source 
--aws-account-id  
--data-source-id "athena-cross-account" 
--name "Athena Cross Account - Shopper Information" 
--type ATHENA 
--data-source-parameters '{
"AthenaParameters": {
"WorkGroup": "main",
"RoleArn": "arn:aws:iam:::position/qs-athena-cross-account-role-a",
"ConsumerAccountRoleArn": "arn:aws:iam:::position/qs-athena-consumer-role"
}
}' 
--permissions '[{
"Principal": "arn:aws:quicksight:::user/default/",
"Actions": [
"quicksight:DescribeDataSource",
"quicksight:DescribeDataSourcePermissions",
"quicksight:PassDataSource",
"quicksight:UpdateDataSource",
"quicksight:DeleteDataSource",
"quicksight:UpdateDataSourcePermissions"
]
}]' 
--region 

Or utilizing Python (Boto3):

import boto3

consumer = boto3.consumer('quicksight', region_name="us-east-1")

response = consumer.create_data_source(
AwsAccountId='',
DataSourceId='athena-cross-account',
Title="Athena Cross Account - Shopper Information",
Kind="ATHENA",
DataSourceParameters={
'AthenaParameters': {
'WorkGroup': 'main',
'RoleArn': 'arn:aws:iam:::position/qs-athena-cross-account-role-a',
'ConsumerAccountRoleArn': 'arn:aws:iam:::position/qs-athena-consumer-role'
}
}
)
print(response['CreationStatus'])

Confirm the info supply was created efficiently:

aws quicksight describe-data-source 
--aws-account-id  
--data-source-id "athena-cross-account" 
--region  
--query 'DataSource.{Title:Title,Standing:Standing,Kind:Kind}'

Anticipated output:

{
"Title": "Athena Cross Account - Shopper Information",
"Standing": "CREATION_SUCCESSFUL",
"Kind": "ATHENA"
}

Share the Information Supply and Create Datasets

After the info supply is lively, share it with Fast authors utilizing the next pattern code:

aws quicksight update-data-source-permissions 
--aws-account-id  
--data-source-id "athena-cross-account" 
--grant-permissions '{
"Principal": "arn:aws:quicksight:::person/default/",
"Actions": [
"quicksight:DescribeDataSource",
"quicksight:DescribeDataSourcePermissions",
"quicksight:PassDataSource"
]
}' 
--region 

Authors can now open Fast, navigate to Datasets, create a brand new dataset, choose the Athena Cross Account knowledge supply, and browse the AWS Glue databases and tables from the patron account. They’ll construct datasets, apply transformations, and create dashboards all from the central Fast account, with question prices billed to the patron account.

Connecting a number of client accounts

To attach further client accounts, repeat above resolution for every position/account

• Replace Position A’s permission coverage to incorporate the brand new client account position ARN (or use a wildcard sample in case your client roles observe a naming conference).
• Create Position B within the new client account with the suitable belief and knowledge entry insurance policies.
• Create a brand new knowledge supply in Fast with the brand new ConsumerAccountRoleArn.
• Share the datasource.
• Every knowledge supply maps to at least one client account. Authors choose the suitable knowledge supply primarily based on which enterprise unit’s knowledge they want.

Safety issues

The safety mannequin for cross-account Athena entry is designed to allow distributed knowledge entry at scale whereas making certain that each question is permitted, scoped, and auditable. As described within the walkthrough, the ExternalId situation on Position B’s belief coverage prevents confused deputy assaults. Solely Fast knowledge sources from the approved (central) account can full the position chain. As well as, Fast applies an inline scope-down coverage every time it assumes Position A, proscribing every session to a single client position even when Position A’s standing permissions cowl a number of accounts. Collectively, these mechanisms be certain that every question chain is each authenticated and narrowly scoped. The buyer account maintains its personal least privilege boundary via Position B’s permissions. Every enterprise unit determines independently which tables, databases, and S3 prefixes are seen to the central BI AWS account. We advocate scoping AWS Glue and S3 permissions to particular assets somewhat than utilizing wildcards, proscribing Position B to a delegated Athena workgroup with question outcome encryption and price limits enabled, and utilizing separate Position B situations when the patron account accommodates knowledge at totally different classification ranges.

The total position chain is captured in AWS CloudTrail throughout each accounts from the preliminary AssumeRole within the central account via the chain into the patron account and the Athena queries that observe. This offers a whole audit path from question initiation to knowledge entry and helps alerting on anomalous patterns akin to sudden supply accounts or spikes in knowledge scanned.

As famous in Step 1 of the walkthrough, the IAM principal creating the info supply requires iam:PassRole on Position A, which prevents directors from associating arbitrary roles with knowledge sources. These mechanisms (ExternalId, scope-down, least privilege, CloudTrail, and PassRole) type a protection in depth mannequin that’s solely policy-driven and programmatic, making it properly fitted to automated governance as tooling on this area matures.

Value attribution

As a result of Athena queries execute below Position B’s credentials within the client account, all related AWS API calls happen in that account. Normal AWS billing handles price separation mechanically. The buyer account sees Athena question expenses, S3 entry prices, and AWS Glue catalog calls by itself invoice. The central Fast account incurs solely Fast session prices and SPICE storage. Earlier than this function, organizations both absorbed all question prices centrally (dropping per-business-unit visibility) or operated separate Fast subscriptions per enterprise unit, fragmenting the BI expertise. Cross-account Athena entry removes that trade-off: a unified Fast deployment with per-business-unit price visibility that requires no customized chargeback logic.

Clear up

To keep away from incurring ongoing expenses, delete the AWS assets (IAM roles, Fast Sight knowledge sources, insurance policies) that you simply created as a part of experimentation. For directions, see the service documentation.

Conclusion

Cross-account Athena entry for Amazon Fast allows enterprises to keep up a centralized BI AWS account whereas respecting multi-account knowledge governance and price boundaries. The position chaining method offers correct price attribution, maintains knowledge sovereignty per enterprise unit, and integrates with present IAM safety controls.

To get began, create the IAM roles in your Fast and client accounts as described on this publish, then create the info supply utilizing the ConsumerAccountRoleArn parameter. For extra particulars, see the Amazon Fast Person Information.

Concerning the authors