• ホーム
  • ブログ
  • Databricks-Certified-Professional-Data-Engineer問題集と練習テスト（125問題） [Q66-Q84]

Databricks-Certified-Professional-Data-Engineer問題集と練習テスト（125問題） [Q66-Q84]

Share

（2024)Databricks-Certified-Professional-Data-Engineer問題集と練習テスト（125問題)

ガイド（2024年最新)リアルなDatabricks Databricks-Certified-Professional-Data-Engineer試験問題


Databricks Certified Professional Data Engineerは、Databricksを使用してビッグデータを操作する知識とスキルを測定する認定試験です。Databricksは、データエンジニアがビッグデータ処理パイプラインを構築、展開、および管理することができるクラウドベースのデータ処理プラットフォームです。認定試験は、Databricksで作業するデータエンジニアの専門知識を検証するために設計されています。


この試験は、Databricksを使用してデータパイプラインを構築および維持する能力と専門性を証明するために設計された、選択式問題と実技演習で構成される試験です。試験に合格した候補者には、「Databricks Certified Professional Data Engineer」認証が授与されます。この認証は、Databricksの使用における候補者の専門性と能力を世界中の雇用主に認めさせることができるため、ビッグデータエンジニアリングおよび分析分野でキャリアを進めたい人にとって貴重な資格です。

 

質問 # 66
A member of the data engineering team has submitted a short notebook that they wish to schedule as part of a larger data pipeline. Assume that the commands provided below produce the logically correct results when run as presented.

Which command should be removed from the notebook before scheduling it as a job?

  • A. Cmd 5
  • B. Cmd 2
  • C. Cmd 6
  • D. Cmd 4
  • E. Cmd 3

正解：C

解説：
Explanation
Cmd 6 is the command that should be removed from the notebook before scheduling it as a job. This command is selecting all the columns from the finalDF dataframe and displaying them in the notebook. This is not necessary for the job, as the finalDF dataframe is already written to a table in Cmd 7. Displaying the dataframe in the notebook will only consume resources and time, and it will not affect the output of the job.
Therefore, Cmd 6 is redundant and should be removed.
The other commands are essential for the job, as they perform the following tasks:
Cmd 1: Reads the raw_data table into a Spark dataframe called rawDF.
Cmd 2: Prints the schema of the rawDF dataframe, which is useful for debugging and understanding the data structure.
Cmd 3: Selects all the columns from the rawDF dataframe, as well as the nested columns from the values struct column, and creates a new dataframe called flattenedDF.
Cmd 4: Drops the values column from the flattenedDF dataframe, as it is no longer needed after flattening, and creates a new dataframe called finalDF.
Cmd 5: Explains the physical plan of the finalDF dataframe, which is useful for optimizing and tuning the performance of the job.
Cmd 7: Writes the finalDF dataframe to a table called flat_data, using the append mode to add new data to the existing table.


質問 # 67
A Delta Lake table was created with the below query:
Consider the following query:
DROP TABLE prod.sales_by_store -
If this statement is executed by a workspace admin, which result will occur?

  • A. The table will be removed from the catalog and the data will be deleted.
  • B. Data will be marked as deleted but still recoverable with Time Travel.
  • C. Nothing will occur until a COMMIT command is executed.
  • D. An error will occur because Delta Lake prevents the deletion of production data.
  • E. The table will be removed from the catalog but the data will remain in storage.

正解：A

解説：
When a table is dropped in Delta Lake, the table is removed from the catalog and the data is deleted. This is because Delta Lake is a transactional storage layer that provides ACID guarantees. When a table is dropped, the transaction log is updated to reflect the deletion of the table and the data is deleted from the underlying storage. References:
* https://docs.databricks.com/delta/quick-start.html#drop-a-table
* https://docs.databricks.com/delta/delta-batch.html#drop-table


質問 # 68
A data architect has designed a system in which two Structured Streaming jobs will concurrently write to a single bronze Delta table. Each job is subscribing to a different topic from an Apache Kafka source, but they will write data with the same schema. To keep the directory structure simple, a data engineer has decided to nest a checkpoint directory to be shared by both streams.
The proposed directory structure is displayed below:

Which statement describes whether this checkpoint directory structure is valid for the given scenario and why?

  • A. No; Delta Lake manages streaming checkpoints in the transaction log.
  • B. Yes; both of the streams can share a single checkpoint directory.
  • C. No; only one stream can write to a Delta Lake table.
  • D. No; each of the streams needs to have its own checkpoint directory.
  • E. Yes; Delta Lake supports infinite concurrent writers.

正解：D

解説：
This is the correct answer because checkpointing is a critical feature of Structured Streaming that provides fault tolerance and recovery in case of failures. Checkpointing stores the current state and progress of a streaming query in a reliable storage system, such as DBFS or S3. Each streaming query must have its own checkpoint directory that is unique and exclusive to that query. If two streaming queries share the same checkpoint directory, they will interfere with each other and cause unexpected errors or data loss. Verified References: [Databricks Certified Data Engineer Professional], under "Structured Streaming" section; Databricks Documentation, under "Checkpointing" section.


質問 # 69
A team member is leaving the team and he/she is currently the owner of the few tables, instead of transfering the ownership to a user you have decided to transfer the ownership to a group so in the future anyone in the group can manage the permissions rather than a single individual, which of the following commands help you accomplish this?

  • A. ALTER OWNER ON table_name to 'group'
  • B. TRANSFER OWNER table_name to 'group'
  • C. ALTER TABLE table_name OWNER to 'group'
  • D. GRANT OWNER On table_name to 'group'
  • E. GRANT OWNER table_name to 'group'*

正解：C

解説：
Explanation
The answer is ALTER TABLE table_name OWNER to 'group'
Assign owner to object


質問 # 70
How VACCUM and OPTIMIZE commands can be used to manage the DELTA lake?

  • A. VACCUM command can be used to delete empty/blank parquet files in a delta table, OPTIMIZE command can be used to cache frequently delta tables for better perfor-mance.
  • B. OPTIMIZE command can be used to compact small parquet files, and the VAC-CUM command can be used to delete parquet files that are marked for deletion/unused.
    (Correct)
  • C. VACCUM command can be used to delete empty/blank parquet files in a delta table. OPTIMIZE command can be used to update stale statistics on a delta table.
  • D. VACCUM command can be used to compact small parquet files, and the OP-TIMZE command can be used to delete parquet files that are marked for dele-tion/unused.
  • E. VACCUM command can be used to compress the parquet files to reduce the size of the table, OPTIMIZE command can be used to cache frequently delta tables for better performance.

正解：B

解説：
Explanation
VACCUM:
You can remove files no longer referenced by a Delta table and are older than the retention thresh-old by running the vacuum command on the table. vacuum is not triggered automatically. The de-fault retention threshold for the files is 7 days. To change this behavior, see Configure data reten-tion for time travel.
OPTIMIZE:
Using OPTIMIZE you can compact data files on Delta Lake, this can improve the speed of read queries on the table. Too many small files can significantly degrade the performance of the query.


質問 # 71
Which of the following technologies can be used to identify key areas of text when parsing Spark Driver log4j output?

  • A. Scala Datasets
  • B. C++
  • C. Regex
  • D. Julia
  • E. pyspsark.ml.feature

正解：C

解説：
Regex, or regular expressions, are a powerful way of matching patterns in text. They can be used to identify key areas of text when parsing Spark Driver log4j output, such as the log level, the timestamp, the thread name, the class name, the method name, and the message. Regex can be applied in various languages and frameworks, such as Scala, Python, Java, Spark SQL, and Databricks notebooks. References:
* https://docs.databricks.com/notebooks/notebooks-use.html#use-regular-expressions
* https://docs.databricks.com/spark/latest/spark-sql/udf-scala.html#using-regular-expressions-in-udfs
* https://docs.databricks.com/spark/latest/sparkr/functions/regexp_extract.html
* https://docs.databricks.com/spark/latest/sparkr/functions/regexp_replace.html


質問 # 72
identifies if its needs to be converted to Fahrenheit or Celcius with a one-word letter F or C?
select udf_convert(60,'C') will result in 15.5
select udf_convert(10,'F') will result in 50

  • A. 1.CREATE UDF FUNCTION udf_convert(temp DOUBLE, measure STRING)
    2. RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
    3. ELSE (temp - 33 ) * 5/9
    4. END
  • B. 1. CREATE UDF FUNCTION udf_convert(temp DOUBLE, measure STRING)
    2. RETURNS DOUBLE
    3. RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
    4. ELSE (temp - 33 ) * 5/9
    5. END
  • C. 1.CREATE USER FUNCTION udf_convert(temp DOUBLE, measure STRING)
    2.RETURNS DOUBLE
    3.RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
    4. ELSE (temp - 33 ) * 5/9
    5.END
  • D. 1.CREATE FUNCTION udf_convert(temp DOUBLE, measure STRING)
    2.RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
    3. ELSE (temp - 33 ) * 5/9
    4. END
  • E. 1.CREATE FUNCTION udf_convert(temp DOUBLE, measure STRING)
    2.RETURNS DOUBLE
    3.RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
    4. ELSE (temp - 33 ) * 5/9
    5. END
    (Correct)

正解：E

解説：
Explanation
The answer is
1.CREATE FUNCTION udf_convert(temp DOUBLE, measure STRING)
2.RETURNS DOUBLE
3.RETURN CASE WHEN measure == 'F' then (temp * 9/5) + 32
4. ELSE (temp - 33 ) * 5/9
5. END


質問 # 73
While investigating a performance issue, you realized that you have too many small files for a given table, which command are you going to run to fix this issue

  • A. SHRINK table_name
  • B. COMPACT table_name
  • C. OPTIMIZE table_name
  • D. VACUUM table_name
  • E. MERGE table_name

正解：C

解説：
Explanation
The answer is OPTIMIZE table_name,
Optimize compacts small parquet files into a bigger file, by default the size of the files are determined based on the table size at the time of OPTIMIZE, the file size can also be set manually or adjusted based on the workload.
https://docs.databricks.com/delta/optimizations/file-mgmt.html
Tune file size based on Table size
To minimize the need for manual tuning, Databricks automatically tunes the file size of Delta tables based on the size of the table. Databricks will use smaller file sizes for smaller tables and larger file sizes for larger tables so that the number of files in the table does not grow too large.
Table Description automatically generated

Bottom of Form
Top of Form


質問 # 74
Newly joined data analyst requested read-only access to tables, assuming you are owner/admin which section of Databricks platform is going to facilitate granting select access to the user

  • A. Azure RBAC
  • B. Admin console
  • C. Azure Databricks control pane IAM
  • D. User settings
  • E. Data explorer

正解：E

解説：
Explanation
Anser is Data Explorer
https://docs.databricks.com/sql/user/data/index.html
Data explorer lets you easily explore and manage permissions on databases and tables. Users can view schema details, preview sample data, and see table details and properties. Administrators can view and change owners, and admins and data object owners can grant and revoke permissions.
To open data explorer, click Data in the sidebar.


質問 # 75
You have noticed that Databricks SQL queries are running slow, you are asked to look reason why queries are running slow and identify steps to improve the performance, when you looked at the issue you noticed all the queries are running in parallel and using a SQL endpoint(SQL Warehouse) with a single cluster. Which of the following steps can be taken to improve the performance/response times of the queries?
*Please note Databricks recently renamed SQL endpoint to SQL warehouse.

  • A. They can turn on the Serverless feature for the SQL endpoint(SQL warehouse).
  • B. They can increase the warehouse size from 2X-Smal to 4XLarge of the SQL end-point(SQL warehouse).
  • C. They can increase the maximum bound of the SQL endpoint(SQL warehouse)'s scaling range
  • D. They can turn on the Serverless feature for the SQL endpoint(SQL warehouse) and change the Spot Instance Policy to "Reliability Optimized."
  • E. They can turn on the Auto Stop feature for the SQL endpoint(SQL warehouse).

正解：C

解説：
Explanation
The answer is, They can increase the maximum bound of the SQL endpoint's scaling range when you increase the max scaling range more clusters are added so queries instead of waiting in the queue can start running using available clusters, see below for more explanation.
The question is looking to test your ability to know how to scale a SQL Endpoint(SQL Warehouse) and you have to look for cue words or need to understand if the queries are running sequentially or concurrently. if the queries are running sequentially then scale up(Size of the cluster from 2X-Small to 4X-Large) if the queries are running concurrently or with more users then scale out(add more clusters).
SQL Endpoint(SQL Warehouse) Overview: (Please read all of the below points and the below diagram to understand )
1.A SQL Warehouse should have at least one cluster
2.A cluster comprises one driver node and one or many worker nodes
3.No of worker nodes in a cluster is determined by the size of the cluster (2X -Small ->1 worker, X-Small ->2 workers.... up to 4X-Large -> 128 workers) this is called Scale up
4.A single cluster irrespective of cluster size(2X-Smal.. to ...4XLarge) can only run 10 queries at any given time if a user submits 20 queries all at once to a warehouse with 3X-Large cluster size and cluster scaling (min
1, max1) while 10 queries will start running the remaining 10 queries wait in a queue for these 10 to finish.
5.Increasing the Warehouse cluster size can improve the performance of a query, for example, if a query runs for 1 minute in a 2X-Small warehouse size it may run in 30 Seconds if we change the warehouse size to X-Small. this is due to 2X-Small having 1 worker node and X-Small having 2 worker nodes so the query has more tasks and runs faster (note: this is an ideal case example, the scalability of a query performance depends on many factors, it can not always be linear)
6.A warehouse can have more than one cluster this is called Scale out. If a warehouse is con-figured with X-Small cluster size with cluster scaling(Min1, Max 2) Databricks spins up an additional cluster if it detects queries are waiting in the queue, If a warehouse is configured to run 2 clusters(Min1, Max 2), and let's say a user submits 20 queries, 10 queriers will start running and holds the remaining in the queue and databricks will automatically start the second cluster and starts redirecting the 10 queries waiting in the queue to the second cluster.
7.A single query will not span more than one cluster, once a query is submitted to a cluster it will remain in that cluster until the query execution finishes irrespective of how many clusters are available to scale.
Please review the below diagram to understand the above concepts:

SQL endpoint(SQL Warehouse) scales horizontally(scale-out) and vertical (scale-up), you have to understand when to use what.
Scale-out -> to add more clusters for a SQL endpoint, change max number of clusters If you are trying to improve the throughput, being able to run as many queries as possible then having an additional cluster(s) will improve the performance.
Databricks SQL automatically scales as soon as it detects queries are in queuing state, in this example scaling is set for min 1 and max 3 which means the warehouse can add three clusters if it detects queries are waiting.

During the warehouse creation or after you have the ability to change the warehouse size (2X-Small....to
...4XLarge) to improve query performance and the maximize scaling range to add more clusters on a SQL Endpoint(SQL Warehouse) scale-out, if you are changing an existing warehouse you may have to restart the warehouse to make the changes effective.

How do you know how many clusters you need(How to set Max cluster size)?
When you click on an existing warehouse and select the monitoring tab, you can see warehouse utilization information(see below), there are two graphs that provide important information on how the warehouse is being utilized, if you see queries are being queued that means your warehouse can benefit from additional clusters. Please review the additional DBU cost associated with adding clusters so you can take a well balanced decision between cost and performance.


質問 # 76
A data ingestion task requires a one-TB JSON dataset to be written out to Parquet with a target part-file size of
512 MB. Because Parquet is being used instead of Delta Lake, built-in file-sizing features such as Auto-Optimize & Auto-Compaction cannot be used.
Which strategy will yield the best performance without shuffling data?

  • A. Set spark.sql.adaptive.advisoryPartitionSizeInBytes to 512 MB bytes, ingest the data, execute the narrow transformations, coalesce to 2,048 partitions (1TB*1024*1024/512), and then write to parquet.
  • B. Set spark.sql.shuffle.partitions to 512, ingest the data, execute the narrow transformations, and then write to parquet.
  • C. Ingest the data, execute the narrow transformations, repartition to 2,048 partitions (1TB*
    1024*1024/512), and then write to parquet.
  • D. Set spark.sql.files.maxPartitionBytes to 512 MB, ingest the data, execute the narrow transformations, and then write to parquet.
  • E. Set spark.sql.shuffle.partitions to 2,048 partitions (1TB*1024*1024/512), ingest the data, execute the narrow transformations, optimize the data by sorting it (which automatically repartitions the data), and then write to parquet.

正解：E

解説：
The key to efficiently converting a large JSON dataset to Parquet files of a specific size without shuffling data lies in controlling the size of the output files directly.
* Setting spark.sql.files.maxPartitionBytes to 512 MB configures Spark to process data in chunks of
512 MB. This setting directly influences the size of the part-files in the output, aligning with the target file size.
* Narrow transformations (which do not involve shuffling data across partitions) can then be applied to this data.
* Writing the data out to Parquet will result in files that are approximately the size specified by spark.sql.files.maxPartitionBytes, in this case, 512 MB.
* The other options involve unnecessary shuffles or repartitions (B, C, D) or an incorrect setting for this specific requirement (E).
References:
* Apache Spark Documentation: Configuration - spark.sql.files.maxPartitionBytes
* Databricks Documentation on Data Sources: Databricks Data Sources Guide


質問 # 77
To reduce storage and compute costs, the data engineering team has been tasked with curating a series of aggregate tables leveraged by business intelligence dashboards, customer-facing applications, production machine learning models, and ad hoc analytical queries.
The data engineering team has been made aware of new requirements from a customer-facing application, which is the only downstream workload they manage entirely. As a result, an aggregate table used by numerous teams across the organization will need to have a number of fields renamed, and additional fields will also be added.
Which of the solutions addresses the situation while minimally interrupting other teams in the organization without increasing the number of tables that need to be managed?

  • A. Send all users notice that the schema for the table will be changing; include in the communication the logic necessary to revert the new table schema to match historic queries.
  • B. Create a new table with the required schema and new fields and use Delta Lake's deep clone functionality to sync up changes committed to one table to the corresponding table.
  • C. Add a table comment warning all users that the table schema and field names will be changing on a given date; overwrite the table in place to the specifications of the customer-facing application.
  • D. Configure a new table with all the requisite fields and new names and use this as the source for the customer-facing application; create a view that maintains the original data schema and table name by aliasing select fields from the new table.
  • E. Replace the current table definition with a logical view defined with the query logic currently writing the aggregate table; create a new table to power the customer-facing application.

正解：D

解説：
This is the correct answer because it addresses the situation while minimally interrupting other teams in the organization without increasing the number of tables that need to be managed. The situation is that an aggregate table used by numerous teams across the organization will need to have a number of fields renamed, and additional fields will also be added, due to new requirements from a customer-facing application. By configuring a new table with all the requisite fields and new names and using this as the source for the customer-facing application, the data engineering team can meet the new requirements without affecting other teams that rely on the existing table schema and name. By creating a view that maintains the original data schema and table name by aliasing select fields from the new table, the data engineering team can also avoid duplicating data or creating additional tables that need to be managed. Verified References: [Databricks Certified Data Engineer Professional], under "Lakehouse" section; Databricks Documentation, under
"CREATE VIEW" section.


質問 # 78
A Delta Lake table representing metadata about content posts from users has the following schema:
user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE This table is partitioned by the date column. A query is run with the following filter:
longitude < 20 & longitude > -20
Which statement describes how data will be filtered?

  • A. The Delta Engine will scan the parquet file footers to identify each row that meets the filter criteria.
  • B. Statistics in the Delta Log will be used to identify partitions that might Include files in the filtered range.
  • C. No file skipping will occur because the optimizer does not know the relationship between the partition column and the longitude.
  • D. The Delta Engine will use row-level statistics in the transaction log to identify the flies that meet the filter criteria.
  • E. Statistics in the Delta Log will be used to identify data files that might include records in the filtered range.

正解：E

解説：
Explanation
This is the correct answer because it describes how data will be filtered when a query is run with the following filter: longitude < 20 & longitude > -20. The query is run on a Delta Lake table that has the following schema:
user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE. This table is partitioned by the date column. When a query is run on a partitioned Delta Lake table, Delta Lake uses statistics in the Delta Log to identify data files that might include records in the filtered range. The statistics include information such as min and max values for each column in each data file. By using these statistics, Delta Lake can skip reading data files that do not match the filter condition, which can improve query performance and reduce I/O costs. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Data skipping" section.


質問 # 79
A junior developer complains that the code in their notebook isn't producing the correct results in the development environment. A shared screenshot reveals that while they're using a notebook versioned with Databricks Repos, they're using a personal branch that contains old logic. The desired branch nameddev-2.3.9is not available from the branch selection dropdown.
Which approach will allow this developer to review the current logic for this notebook?

  • A. Use Repos to merge the current branch and the dev-2.3.9 branch, then make a pull request to sync with the remote repository
  • B. Use Repos to pull changes from the remote Git repository and select the dev-2.3.9 branch.
  • C. Use Repos to make a pull request use the Databricks REST API to update the current branch to dev-2.3.9
  • D. Use Repos to checkout the dev-2.3.9 branch and auto-resolve conflicts with the current branch
  • E. Merge all changes back to the main branch in the remote Git repository and clone the repo again

正解：B

解説：
This is the correct answer because it will allow the developer to update their local repository with the latest changes from the remote repository and switch to the desired branch. Pulling changes will not affect the current branch or create any conflicts, as it will only fetch the changes and not merge them. Selecting the dev-2.3.9 branch from the dropdown will checkout that branch and display its contents in the notebook.
Verified References: [Databricks Certified Data Engineer Professional], under "Databricks Tooling" section; Databricks Documentation, under "Pull changes from a remote repository" section.


質問 # 80
A dataset has been defined using Delta Live Tables and includes an expectations clause: CON-STRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') What is the expected behavior when a batch of data containing data that violates these constraints is processed?

  • A. Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
  • B. Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
  • C. Records that violate the expectation are added to the target dataset and flagged as in-valid in a field added to the target dataset.
  • D. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
  • E. Records that violate the expectation cause the job to fail.

正解：A

解説：
Explanation
The answer is, Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
Delta live tables support three types of expectations to fix bad data in DLT pipelines Review below example code to examine these expectations, Diagram Description automatically generated with medium confidence


質問 # 81
Which of the following statement is true about Databricks repos?

  • A. A workspace can only have one instance of git integration
  • B. You can approve the pull request if you are the owner of Databricks repos
  • C. Databricks Repos and Notebook versioning are the same features
  • D. Databricks repos allow you to comment and commit code changes and push them to a remote branch
  • E. You cannot create a new branch in Databricks repos

正解：D

解説：
Explanation
See below diagram to understand the role Databricks Repos and Git provider plays when building a CI/CD workdlow.
All the steps highlighted in yellow can be done Databricks Repo, all the steps highlighted in Gray are done in a git provider like Github or Azure Devops Diagram Description automatically generated


質問 # 82
All records from an Apache Kafka producer are being ingested into a single Delta Lake table with the following schema:
key BINARY, value BINARY, topic STRING, partition LONG, offset LONG, timestamp LONG There are 5 unique topics being ingested. Only the "registration" topic contains Personal Identifiable Information (PII). The company wishes to restrict access to PII. The company also wishes to only retain records containing PII in this table for 14 days after initial ingestion. However, for non-PII information, it would like to retain these records indefinitely.
Which of the following solutions meets the requirements?

  • A. Data should be partitioned by the topic field, allowing ACLs and delete statements to leverage partition boundaries.
  • B. Data should be partitioned by the registration field, allowing ACLs and delete statements to be set for the PII directory.
  • C. All data should be deleted biweekly; Delta Lake's time travel functionality should be leveraged to maintain a history of non-PII information.
  • D. Because the value field is stored as binary data, this information is not considered PII and no special precautions should be taken.
  • E. Separate object storage containers should be specified based on the partition field, allowing isolation at the storage level.

正解：B

解説：
Partitioning the data by the topic field allows the company to apply different access control policies and retention policies for different topics. For example, the company can use the Table Access Control feature to grant or revoke permissions to the registration topic based on user roles or groups. The company can also use the DELETE command to remove records from the registration topic that are older than 14 days, while keeping the records from other topics indefinitely. Partitioning by the topic field also improves the performance of queries that filter by the topic field, as they can skip reading irrelevant partitions. References:
* Table Access Control: https://docs.databricks.com/security/access-control/table-acls/index.html
* DELETE: https://docs.databricks.com/delta/delta-update.html#delete-from-a-table


質問 # 83
A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.
Which approach would allow using DLT expectations to validate all expected records are present in this table?

  • A. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null
  • B. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
  • C. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table
  • D. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table

正解：C

解説：
To validate that all records from the source are included in the derived table, creating a view that performs a left outer join between the validation_copy table and the report table is effective. The view can highlight any discrepancies, such as null values in the report table's key columns, indicating missing records. This view can then be referenced in DLT (Delta Live Tables) expectations for the report table to ensure data integrity. This approach allows for a comprehensive comparison between the source and the derived table.
References:
* Databricks Documentation on Delta Live Tables and Expectations: Delta Live Tables Expectations


質問 # 84
......


Databricks Certified Professional Data Engineer は、Databricks を使用してビッグデータパイプラインを構築および管理する専門家の能力を証明するために設計された試験です。Databricks は、大規模なデータを処理するための協調環境を提供する統一された分析プラットフォームです。Databricks Certified Professional Data Engineer 試験は、Databricks を使用して大規模なデータ処理ソリューションを設計、構築、展開する候補者の能力を検証します。

 

Databricks-Certified-Professional-Data-Engineer試験問題集パスできる2024年最新の認証された試験問題：https://www.passtest.jp/Databricks/Databricks-Certified-Professional-Data-Engineer-shiken.html

Databricks-Certified-Professional-Data-Engineer試験問題リアルな最新問題PDF：https://drive.google.com/open?id=14SZNYjwmGP2LZ66gW10fFXJod3zd1vQC

関するブログ

もっと
Databricks-Certified-Professional-Data-Engineer 無料問題集