Which solution will minimize overhead and costs while meeting the required access patterns?

A company wishes to query data that resides in multiple AWS accounts from a central data lake. Each account has its own Amazon S3 bucket that stores data unique to its business function. Users from different accounts must be granted access to the data lake based on their roles.

Which solution will minimize overhead and costs while meeting the required access patterns?
A . Use AWS Kinesis Firehose to consolidate data from multiple accounts into a single account.
B . Use AWS Central Tower to centrally manage each account’s S3 buckets.
C . Create a scheduled Lambda function for transferring data from multiple accounts to the S3 buckets of a central account
D . Use AWS Lake Formation to consolidate data from multiple accounts into a single account.

View Answer

Answer: D

Explanation:

AWS Lake Formation is a service that makes it easy to set up a secure data lake in days. A data lake is a centralized, curated, and secured repository that stores all your data, both in its original form and prepared for analysis. A data lake enables you to break down data silos and combine different types of analytics to gain insights and guide better business decisions.

Amazon S3 forms the storage layer for Lake Formation. If you already use S3, you typically begin by registering existing S3 buckets that contain your data. Lake Formation creates new buckets for the data lake and import data into them. AWS always stores this data in your account, and only you have direct access to it.

AWS Lake Formation is integrated with AWS Glue which you can use to create a data catalog that describes available datasets and their appropriate business applications. Lake Formation lets you define policies and control data access with simple “grant and revoke permissions to data” sets at granular levels. You can assign permissions to IAM users, roles, groups, and Active Directory users using federation. You specify permissions on catalog objects (like tables and columns) rather than on buckets and objects.

Thus, the correct answer is: Use AWS Lake Formation to consolidate data from multiple accounts into a single account.

The option that says: Use AWS Kinesis Firehose to consolidate data from multiple accounts into a single account is incorrect. Setting up a Kinesis Firehose in each and every account to move data into a single

location is costly and impractical. A better approach is to set up cross-account sharing which is free with AWS Lake Formation.

The option that says: Create a scheduled Lambda function for transferring data from multiple accounts to the S3 buckets of a central account is incorrect. This could be done by utilizing the AWS SDK, but implementation would be difficult and quite challenging to manage. Remember that the scenario explicitly mentioned that the solution must minimize management overhead.

The option that says: Use AWS Central Tower to centrally manage each account’s S3 buckets is

incorrect because the AWS Central Tower service is primarily used to manage and govern multiple AWS

accounts and not just S3 buckets. Using the AWS Lake Formation service is a more suitable choice.

References:

https://aws.amazon.com/blogs/big-data/building-securing-and-managing-data-lakes-with-aws-lake-forma

tion/

https://docs.aws.amazon.com/lake-formation/latest/dg/how-it-works.html