Databricks Certified Associate Developer for Apache Spark 3.0 認定 Associate-Developer-Apache-Spark 試験問題:

1. Which of the following describes Spark actions?

A) Writing data to disk is the primary purpose of actions.
B) The driver receives data upon request by actions.
C) Actions are Spark's way of modifying RDDs.
D) Actions are Spark's way of exchanging data between executors.
E) Stage boundaries are commonly established by actions.

2. Which of the following code blocks reads in the parquet file stored at location filePath, given that all columns in the parquet file contain only whole numbers and are stored in the most appropriate format for this kind of data?

A) 1.spark.read.schema(
2. StructType([
3. StructField("transactionId", IntegerType(), True),
4. StructField("predError", IntegerType(), True)]
5. )).format("parquet").load(filePath)
B) 1.spark.read.schema(
2. StructType([
3. StructField("transactionId", StringType(), True),
4. StructField("predError", IntegerType(), True)]
5. )).parquet(filePath)
C) 1.spark.read.schema([
2. StructField("transactionId", NumberType(), True),
3. StructField("predError", IntegerType(), True)
4. ]).load(filePath)
D) 1.spark.read.schema([
2. StructField("transactionId", IntegerType(), True),
3. StructField("predError", IntegerType(), True)
4. ]).load(filePath, format="parquet")
E) 1.spark.read.schema(
2. StructType(
3. StructField("transactionId", IntegerType(), True),
4. StructField("predError", IntegerType(), True)
5. )).load(filePath)

3. Which of the following statements about Spark's DataFrames is incorrect?

A) Spark's DataFrames are equal to Python's DataFrames.
B) The data in DataFrames may be split into multiple chunks.
C) Data in DataFrames is organized into named columns.
D) RDDs are at the core of DataFrames.
E) Spark's DataFrames are immutable.

4. Which of the elements in the labeled panels represent the operation performed for broadcast variables?
Larger image

A) 1, 2
B) 2, 5
C) 1, 3, 4
D) 3
E) 2, 3

5. The code block displayed below contains multiple errors. The code block should remove column transactionDate from DataFrame transactionsDf and add a column transactionTimestamp in which dates that are expressed as strings in column transactionDate of DataFrame transactionsDf are converted into unix timestamps. Find the errors.
Sample of DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+----------------+
2.|transactionId|predError|value|storeId|productId| f| transactionDate|
3.+-------------+---------+-----+-------+---------+----+----------------+
4.| 1| 3| 4| 25| 1|null|2020-04-26 15:35|
5.| 2| 6| 7| 2| 2|null|2020-04-13 22:01|
6.| 3| 3| null| 25| 3|null|2020-04-02 10:53|
7.+-------------+---------+-----+-------+---------+----+----------------+ Code block:
1.transactionsDf = transactionsDf.drop("transactionDate")
2.transactionsDf["transactionTimestamp"] = unix_timestamp("transactionDate", "yyyy-MM-dd")

A) Column transactionDate should be dropped after transactionTimestamp has been written. The string indicating the date format should be adjusted. The withColumn operator should be used instead of the existing column assignment.
B) Column transactionDate should be dropped after transactionTimestamp has been written. The string indicating the date format should be adjusted. The withColumn operator should be used instead of the existing column assignment. Operator to_unixtime() should be used instead of unix_timestamp().
C) Column transactionDate should be wrapped in a col() operator.
D) The string indicating the date format should be adjusted. The withColumnReplaced operator should be used instead of the drop and assign pattern in the code block to replace column transactionDate with the new column transactionTimestamp.
E) Column transactionDate should be dropped after transactionTimestamp has been written. The withColumn operator should be used instead of the existing column assignment. Column transactionDate should be wrapped in a col() operator.

質問と回答：