Which technologies should you use?

HOTSPOT

You are designing a new Lambda architecture on Microsoft Azure.

The real-time processing layer must meet the following requirements:

Ingestion:

– Receive millions of events per second

– Act as a fully managed Platform-as-a-Service (PaaS) solution

– Integrate with Azure Functions

Stream processing:

– Process on a per-job basis

– Provide seamless connectivity with Azure services

– Use a SQL-based query language

Analytical data store:

– Act as a managed service

– Use a document store

– Provide data encryption at rest

You need to identify the correct technologies to build the Lambda architecture using minimal effort.

Which technologies should you use? To answer, select the appropriate options in the answer area.

NOTE: Each correct selection is worth one point.

View Answer

Answer:

Explanation:

Box 1: Azure Event Hubs

This portion of a streaming architecture is often referred to as stream buffering. Options include Azure Event Hubs, Azure IoT Hub, and Kafka.

Incorrect Answers: Not HDInsight Kafka

Azure Functions need a trigger defined in order to run. There is a limited set of supported trigger types, and Kafka is not one of them.

Box 2: Azure Stream Analytics

Azure Stream Analytics provides a managed stream processing service based on perpetually running SQL queries that operate on unbounded streams.

You can also use open source Apache streaming technologies like Storm and Spark Streaming in an HDInsight cluster.

Box 3: Azure SQL Data Warehouse

Azure SQL Data Warehouse provides a managed service for large-scale, cloud-based data warehousing. HDInsight supports Interactive Hive, HBase, and Spark SQL, which can also be used to serve data for analysis.

References: https://docs.microsoft.com/en-us/azure/architecture/data-guide/big-data/