A data engineer is using Structured Streaming to read in transaction data from a bronze Delta table. It was discovered that the data has quality issues where sometimes the transaction value is negative, and when that occurs, the rows need to be routed to a separate quarantine table. They have low latency requirements for the good data since it is used by downstream systems, but the bad data will only be analyzed periodically and has no production dependencies. The quarantine job needs to be implemented so that it cannot affect the production processes that depend on the good data, and the cost of the job needs to be minimized. How should the quarantine process be implemented in order to satisfy these requirements?

• A. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. Inside a foreachBatch function, the dataframe should be filtered so that records with a transaction value greater than or equal to 0 are written to the good data table and records with a transaction value less than 0 are written to a quarantine table. Try/Catch can be added around the writes in the foreachBatch function so that the stream can't fail.
• B. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing. Both should run as separate streams on the same cluster to minimize cost.
• C. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing, and should not share compute with other processes. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing, and should be implemented on a separate small cluster and only run once a day to minimize cost.
• D. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. A new boolean column called "quarantine" should be added to the dataframe, and its value should be set to true if the transaction value is less than 0 and false if the transaction value is greater than or equal to 0. Processing and storing all the data together will save costs.

Explanation: Only visible for PremiumVCEDump members. You can sign-up / login (it's free).

An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with the following code:
df = spark.read.format("parquet").load(f"/mnt/source/(date)")
Which code block should be used to create the date Python variable used in the above code block?

• A. date = dbutils.notebooks.getParam("date")
• B. date = spark.conf.get("date")
• C. dbutils.widgets.text("date", "null")
  date = dbutils.widgets.get("date")
• D. import sys
  date = sys.argv[1]
• E. input_dict = input()
  date= input_dict["date"]

Explanation: Only visible for PremiumVCEDump members. You can sign-up / login (it's free).

Which statement describes Delta Lake Auto Compaction?

• A. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.
• B. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.
• C. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.
• D. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.
• E. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.

Explanation: Only visible for PremiumVCEDump members. You can sign-up / login (it's free).

When scheduling Structured Streaming jobs for production, which configuration automatically recovers from query failures and keeps costs low?

• A. Cluster: New Job Cluster;
  Retries: None;
  Maximum Concurrent Runs: 1
• B. Cluster: New Job Cluster;
  Retries: Unlimited;
  Maximum Concurrent Runs: Unlimited
• C. Cluster: Existing All-Purpose Cluster;
  Retries: Unlimited;
  Maximum Concurrent Runs: 1
• D. Cluster: Existing All-Purpose Cluster;
  Retries: Unlimited;
  Maximum Concurrent Runs: 1
• E. Cluster: Existing All-Purpose Cluster;
  Retries: None;
  Maximum Concurrent Runs: 1

Explanation: Only visible for PremiumVCEDump members. You can sign-up / login (it's free).

The data architect has decided that once data has been ingested from external sources into the Databricks Lakehouse, table access controls will be leveraged to manage permissions for all production tables and views.
The following logic was executed to grant privileges for interactive queries on a production database to the core engineering group.
GRANT USAGE ON DATABASE prod TO eng;
GRANT SELECT ON DATABASE prod TO eng;
Assuming these are the only privileges that have been granted to the eng group and that these users are not workspace administrators, which statement describes their privileges?

• A. Group members are able to create, query, and modify all tables and views in the prod database, but cannot define custom functions.
• B. Group members have full permissions on the prod database and can also assign permissions to other users or groups.
• C. Group members are able to query all tables and views in the prod database, but cannot create or edit anything in the database.
• D. Group members are able to query and modify all tables and views in the prod database, but cannot create new tables or views.
• E. Group members are able to list all tables in the prod database but are not able to see the results of any queries on those tables.

Explanation: Only visible for PremiumVCEDump members. You can sign-up / login (it's free).