Databricks-Certified-Professional-Data-Engineer練習テスト問題は更新された125問題あります [Q66-Q85]

Databricks-Certified-Professional-Data-Engineer練習テスト問題は更新された125問題あります

Databricks Databricks-Certified-Professional-Data-Engineer問題集で一発合格できる問題を試そう！

Databricks認定プロフェッショナルデータエンジニア認定試験は、DataBricksプラットフォームで複雑なデータソリューションの構築と管理のスキルを実証したいデータ専門家にとって貴重な認定です。この試験は、共同環境でデータエンジニアリングワークフローを設計、実装、および管理するために必要な知識とスキルをテストするように設計されています。試験に合格した候補者は、Databricksテクノロジーを深く理解し、スケーラブルで信頼できるデータソリューションを構築および管理できるようになります。

 

質問 # 66
A Structured Streaming job deployed to production has been experiencing delays during peak hours of the day.
At present, during normal execution, each microbatch of data is processed in less than 3 seconds. During peak hours of the day, execution time for each microbatch becomes very inconsistent, sometimes exceeding 30 seconds. The streaming write is currently configured with a trigger interval of 10 seconds.
Holding all other variables constant and assuming records need to be processed in less than 10 seconds, which adjustment will meet the requirement?

• A. Decrease the trigger interval to 5 seconds; triggering batches more frequently allows idle executors to begin processing the next batch while longer running tasks from previous batches finish.
• B. Decrease the trigger interval to 5 seconds; triggering batches more frequently may prevent records from backing up and large batches from causing spill.
• C. The trigger interval cannot be modified without modifying the checkpoint directory; to maintain the current stream state, increase the number of shuffle partitions to maximize parallelism.
• D. Use the trigger once option and configure a Databricks job to execute the query every 10 seconds; this ensures all backlogged records are processed with each batch.
• E. Increase the trigger interval to 30 seconds; setting the trigger interval near the maximum execution time observed for each batch is always best practice to ensure no records are dropped.

正解：B

解説：
Explanation
The adjustment that will meet the requirement of processing records in less than 10 seconds is to decrease the trigger interval to 5 seconds. This is because triggering batches more frequently may prevent records from backing up and large batches from causing spill. Spill is a phenomenon where the data in memory exceeds the available capacity and has to be written to disk, which can slow down the processing and increase the execution time1. By reducing the trigger interval, the streaming query can process smaller batches of data more quickly and avoid spill. This can also improve the latency and throughput of the streaming job2.
The other options are not correct, because:
Option A is incorrect because triggering batches more frequently does not allow idle executors to begin processing the next batch while longer running tasks from previous batches finish. In fact, the opposite is true. Triggering batches more frequently may cause concurrent batches to compete for the same resources and cause contention and backpressure2. This can degrade the performance and stability of the streaming job.
Option B is incorrect because increasing the trigger interval to 30 seconds is not a good practice to ensure no records are dropped. Increasing the trigger interval means that the streaming query will process larger batches of data less frequently, which can increase the risk of spill, memory pressure, and timeouts12. This can also increase the latency and reduce the throughput of the streaming job.
Option C is incorrect because the trigger interval can be modified without modifying the checkpoint directory. The checkpoint directory stores the metadata and state of the streaming query, such as the offsets, schema, and configuration3. Changing the trigger interval does not affect the state of the streaming query, and does not require a new checkpoint directory. However, changing the number of shuffle partitions may affect the state of the streaming query, and may require a new checkpoint directory4.
Option D is incorrect because using the trigger once option and configuring a Databricks job to execute the query every 10 seconds does not ensure that all backlogged records are processed with each batch. The trigger once option means that the streaming query will process all the available data in the source and then stop5. However, this does not guarantee that the query will finish processing within 10 seconds, especially if there area lot of records in the source. Moreover, configuring a Databricks job to execute the query every 10 seconds may cause overlapping or missed batches, depending on the execution time of the query.
References: Memory Management Overview, Structured Streaming Performance Tuning Guide, Checkpointing, Recovery Semantics after Changes in a Streaming Query, Triggers

質問 # 67
A junior data engineer has manually configured a series of jobs using the Databricks Jobs UI. Upon reviewing their work, the engineer realizes that they are listed as the "Owner" for each job. They attempt to transfer
"Owner" privileges to the "DevOps" group, but cannot successfully accomplish this task.
Which statement explains what is preventing this privilege transfer?

• A. A user can only transfer job ownership to a group if they are also a member of that group.
• B. The creator of a Databricks job will always have "Owner" privileges; this configuration cannot be changed.
• C. Other than the default "admins" group, only individual users can be granted privileges on jobs.
• D. Databricks jobs must have exactly one owner; "Owner" privileges cannot be assigned to a group.
• E. Only workspace administrators can grant "Owner" privileges to a group.

正解：D

解説：
Explanation
The reason why the junior data engineer cannot transfer "Owner" privileges to the "DevOps" group is that Databricks jobs must have exactly one owner, and the owner must be an individual user, not a group. A job cannot have more than one owner, and a job cannot have a group as an owner. The owner of a job is the user who created the job, or the user who was assigned the ownership by another user. The owner of a job has the highest level of permission on the job, and can grant or revoke permissions to other users or groups. However, the owner cannot transfer the ownership to a group, only to another user. Therefore, the junior data engineer's attempt to transfer "Owner" privileges to the "DevOps" group is not possible. References:
Jobs access control: https://docs.databricks.com/security/access-control/table-acls/index.html Job permissions:
https://docs.databricks.com/security/access-control/table-acls/privileges.html#job-permissions

質問 # 68
The data governance team is reviewing code used for deleting records for compliance with GDPR. They note the following logic is used to delete records from the Delta Lake table namedusers.

Assuming thatuser_idis a unique identifying key and that contains all users that have requested deletion, which statement describes whether successfully executing the above logic guarantees that the records to be deleted are no longer accessible and why?

• A. Yes; Delta Lake ACID guarantees provide assurance that the delete command succeeded fully and permanently purged these records.
• B. No; the Delta cache may return records from previous versions of the table until the cluster is restarted.
• C. No; files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files.
• D. No; the Delta Lake delete command only provides ACID guarantees when combined with the mergeinto command.
• E. Yes; the Delta cache immediately updates to reflect the latest data files recorded to disk.

正解：C

解説：
Explanation
The code uses the DELETE FROM command to delete records from the users table that match a condition based on a join with another table called delete_requests, which contains all users that have requested deletion.
The DELETE FROM command deletes records from a Delta Lake table by creating a new version of the table that does not contain the deleted records. However, this does not guarantee that the records to be deleted are no longer accessible, because Delta Lake supports time travel, which allows querying previous versions of the table using a timestamp or version number. Therefore, files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files from physical storage.
Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Delete from a table" section; Databricks Documentation, under "Remove files no longer referenced by a Delta table" section.

質問 # 69
You are asked to write a python function that can read data from a delta table and return the Data-Frame, which of the following is correct?

• A. Python function will result in out of memory error due to data volume
• B. Python function can return a DataFrame
• C. Write SQL UDF to return a DataFrame
• D. Python function cannot return a DataFrame
• E. Write SQL UDF that can return tabular data

正解：A

解説：
Explanation
The answer is Python function can return a DataFrame
The function would something like this,
1.get_source_dataframe(tablename):
2. df = spark.read.table(tablename)
3.return df
df = get_source_dataframe('test_table')
since there is no action spark returns a Dataframe and assigns to df python variable

質問 # 70
The downstream consumers of a Delta Lake table have been complaining about data quality issues impacting performance in their applications. Specifically, they have complained that invalid latitude and longitude values in the activity_details table have been breaking their ability to use other geolocation processes.
A junior engineer has written the following code to add CHECK constraints to the Delta Lake table:

A senior engineer has confirmed the above logic is correct and the valid ranges for latitude and longitude are provided, but the code fails when executed.
Which statement explains the cause of this failure?

• A. The activity details table already contains records that violate the constraints; all existing data must pass CHECK constraints in order to add them to an existing table.
• B. The activity details table already exists; CHECK constraints can only be added during initial table creation.
• C. The current table schema does not contain the field valid coordinates; schema evolution will need to be enabled before altering the table to add a constraint.
• D. The activity details table already contains records; CHECK constraints can only be added prior to inserting values into a table.
• E. Because another team uses this table to support a frequently running application, two-phase locking is preventing the operation from committing.

正解：A

解説：
The failure is that the code to add CHECK constraints to the Delta Lake table fails when executed. The code uses ALTER TABLE ADD CONSTRAINT commands to add two CHECK constraints to a table named activity_details. The first constraint checks if the latitude value is between -90 and 90, and the second constraint checks if the longitude value is between -180 and 180. The cause of this failure is that the activity_details table already contains records that violate these constraints, meaning that they have invalid latitude or longitude values outside of these ranges. When adding CHECK constraints to an existing table, Delta Lake verifies that all existing data satisfies the constraints before adding them to the table. If any record violates the constraints, Delta Lake throws an exception and aborts the operation. Verified References:
[Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Add a CHECK constraint to an existing table" section.
https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-alter-table.html#add-constraint

質問 # 71
What are the advantages of the Hashing Features?

• A. Easily reverse engineer vectors to determine which original feature mapped to a vector location
• B. Less pass through the training data
• C. Requires the less memory

正解：B、C

解説：
Explanation
SGD-based classifiers avoid the need to predetermine vector size by simply picking a reasonable size and
shoehorning the training data into vectors of that size. This approach is known as feature hashing. The
shoehorning is done by picking one or more locations by using a hash of the name of the variable for
continuous variables or a hash of the variable name and the category name or word for categorical, text*like, or
word-like data.
This hashed feature approach has the distinct advantage of requiring less memory and one less pass through
the training data, but it can make it much harder to reverse engineer vectors to determine which original
feature mapped to a vector location. This is because multiple features may hash to the same location. With
large vectors or with multiple locations per feature, this isn't a problem for accuracy but it can make it hard to
understand what a classifier is doing.
An additional benefit of feature hashing is that the unknown and unbounded vocabularies typical of word-like
variables aren't a problem.

質問 # 72
You are working on a process to query the table based on batch date, and batch date is an input parameter and expected to change every time the program runs, what is the best way to we can parameterize the query to run without manually changing the batch date?

• A. Create a notebook parameter for batch date and assign the value to a python variable and use a spark data frame to filter the data based on the python variable
• B. There is no way we can combine python variable and spark code
• C. Store the batch date in the spark configuration and use a spark data frame to filter the data based on the spark configuration.
• D. Create a dynamic view that can calculate the batch date automatically and use the view to query the data
• E. Manually edit code every time to change the batch date

正解：A

解説：
Explanation
The answer is, Create a notebook parameter for batch date and assign the value to a python variable and use a spark data frame to filter the data based on the python variable

質問 # 73
The Delta Live Tables Pipeline is configured to run in Development mode using the Triggered Pipeline Mode.
what is the expected outcome after clicking Start to update the pipeline?

• A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped
• B. All datasets will be updated continuously and the pipeline will not shut down. The compute resources will persist with the pipeline
• C. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional development and testing
• D. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated
• E. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist after the pipeline is stopped to allow for additional development and testing

正解：B

解説：
Explanation
The answer is All datasets will be updated once and the pipeline will shut down. The compute re-sources will persist to allow for additional testing.
DLT pipeline supports two modes Development and Production, you can switch between the two based on the stage of your development and deployment lifecycle.
Development and production modes
When you run your pipeline in development mode, the Delta Live Tables system:
*Reuses a cluster to avoid the overhead of restarts.
*Disables pipeline retries so you can immediately detect and fix errors.
In production mode, the Delta Live Tables system:
*Restarts the cluster for specific recoverable errors, including memory leaks and stale credentials.
*Retries execution in the event of specific errors, for example, a failure to start a cluster.
Use the buttons in the Pipelines UI to switch between develop-ment and production modes. By default, pipelines run in development mode.
Switching between development and production modes only controls cluster and pipeline execution behavior.
Storage locations must be configured as part of pipeline settings and are not affected when switching between modes.
Please review additional DLT concepts using below link
https://docs.databricks.com/data-engineering/delta-live-tables/delta-live-tables-concepts.html#delta-live-tables-c

質問 # 74
Data engineering team has provided 10 queries and asked Data Analyst team to build a dashboard and refresh the data every day at 8 AM, identify the best approach to set up data refresh for this dashaboard?

• A. The entire dashboard with 10 queries can be refreshed at once, single schedule needs to be set up to refresh at 8 AM.
• B. A dashboard can only refresh one query at a time, 10 schedules to set up the refresh.
• C. Setup JOB with linear dependency to all load all 10 queries into a table so the dashboard can be refreshed at once.
• D. Each query requires a separate task and setup 10 tasks under a single job to run at 8 AM to refresh the dashboard
• E. Use Incremental refresh to run at 8 AM every day.

正解：A

解説：
Explanation
The answer is,
The entire dashboard with 10 queries can be refreshed at once, single schedule needs to be set up to refresh at
8 AM.
Automatically refresh a dashboard
A dashboard's owner and users with the Can Edit permission can configure a dashboard to auto-matically refresh on a schedule. To automatically refresh a dashboard:
* Click the Schedule button at the top right of the dashboard. The scheduling dialog appears.
* Graphical user interface, text, application, email, Teams Description automatically generated
* 2.In the Refresh every drop-down, select a period.
* 3.In the SQL Warehouse drop-down, optionally select a SQL warehouse to use for all the queries.
If you don't select a warehouse, the queries execute on the last used SQL ware-house.
* 4.Next to Subscribers, optionally enter a list of email addresses to notify when the dashboard is automatically updated.
* Each email address you enter must be associated with a Azure Databricks account or con-figured as an alert destination.
* 5.Click Save. The Schedule button label changes to Scheduled.

質問 # 75
The data architect has mandated that all tables in the Lakehouse should be configured as external (also known as "unmanaged") Delta Lake tables.
Which approach will ensure that this requirement is met?

• A. When the workspace is being configured, make sure that external cloud object storage has been mounted.
• B. When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.
• C. When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.
• D. When data is saved to a table, make sure that a full file path is specified alongside the Delta format.
• E. When a database is being created, make sure that the LOCATION keyword is used.

正解：B

解説：
Explanation
To create an external or unmanaged Delta Lake table, you need to use the EXTERNAL keyword in the CREATE TABLE statement. This indicates that the table is not managed by the catalog and the data files are not deleted when the table is dropped. You also need to provide a LOCATION clause to specify the path where the data files are stored. For example:
CREATE EXTERNAL TABLE events ( date DATE, eventId STRING, eventType STRING, data STRING) USING DELTA LOCATION '/mnt/delta/events'; This creates an external Delta Lake table named events that references the data files in the '/mnt/delta/events' path. If you drop this table, the data files will remain intact and you can recreate the table with the same statement.
References:
https://docs.databricks.com/delta/delta-batch.html#create-a-table
https://docs.databricks.com/delta/delta-batch.html#drop-a-table

質問 # 76
The Databricks workspace administrator has configured interactive clusters for each of the data engineering groups. To control costs, clusters are set to terminate after 30 minutes of inactivity. Each user should be able to execute workloads against their assigned clusters at any time of the day.
Assuming users have been added to a workspace but not granted any permissions, which of the following describes the minimal permissions a user would need to start and attach to an already configured cluster.

• A. Workspace Admin privileges, cluster creation allowed. "Can Attach To" privileges on the required cluster
• B. Cluster creation allowed. "Can Attach To" privileges on the required cluster
• C. "Can Restart" privileges on the required cluster
• D. "Can Manage" privileges on the required cluster
• E. Cluster creation allowed. "Can Restart" privileges on the required cluster

正解：C

解説：
Explanation
https://learn.microsoft.com/en-us/azure/databricks/security/auth-authz/access-control/cluster-acl
https://docs.databricks.com/en/security/auth-authz/access-control/cluster-acl.html

質問 # 77
A user wants to use DLT expectations to validate that a derived table report contains all records from the source, included in the table validation_copy.
The user attempts and fails to accomplish this by adding an expectation to the report table definition.
Which approach would allow using DLT expectations to validate all expected records are present in this table?

• A. Define a temporary table that perform a left outer join on validation_copy and report, and define an expectation that no report key values are null
• B. Define a SQL UDF that performs a left outer join on two tables, and check if this returns null values for report key values in a DLT expectation for the report table.
• C. Define a view that performs a left outer join on validation_copy and report, and reference this view in DLT expectations for the report table
• D. Define a function that performs a left outer join on validation_copy and report and report, and check against the result in a DLT expectation for the report table

正解：C

解説：
To validate that all records from the source are included in the derived table, creating a view that performs a left outer join between the validation_copy table and the report table is effective. The view can highlight any discrepancies, such as null values in the report table's key columns, indicating missing records. This view can then be referenced in DLT (Delta Live Tables) expectations for the report table to ensure data integrity. This approach allows for a comprehensive comparison between the source and the derived table.
References:
* Databricks Documentation on Delta Live Tables and Expectations: Delta Live Tables Expectations

質問 # 78
The DevOps team has configured a production workload as a collection of notebooks scheduled to run daily using the Jobs UI. A new data engineering hire is onboarding to the team and has requested access to one of these notebooks to review the production logic.
What are the maximum notebook permissions that can be granted to the user without allowing accidental changes to production code or data?

• A. Can Read
• B. Can Manage
• C. No permissions
• D. Can Edit
• E. Can Run

正解：C

解説：
This is the correct answer because it is the maximum notebook permissions that can be granted to the user without allowing accidental changes to production code or data. Notebook permissions are used to control access to notebooks in Databricks workspaces. There are four types of notebook permissions: Can Manage, Can Edit, Can Run, and Can Read. Can Manage allows full control over the notebook, including editing, running, deleting, exporting, and changing permissions. Can Edit allows modifying and running the notebook, but not changing permissions or deleting it. Can Run allows executing commands in an existing cluster attached to the notebook, but not modifying or exporting it. Can Read allows viewing the notebook content, but not running or modifying it. In this case, granting Can Read permission to the user will allow them to review the production logic in the notebook without allowing them to make any changes to it or run any commands that may affect production data. Verified References: [Databricks Certified Data Engineer Professional], under "Databricks Workspace" section; Databricks Documentation, under "Notebook permissions" section.

質問 # 79
You have configured AUTO LOADER to process incoming IOT data from cloud object storage every 15 mins, recently a change was made to the notebook code to update the processing logic but the team later realized that the notebook was failing for the last 24 hours, what steps team needs to take to reprocess the data that was not loaded after the notebook was corrected?

• A. Move the files that were not processed to another location and manually copy the files into the ingestion path to reprocess them
• B. Delete the checkpoint folder and run the autoloader again
• C. Enable back_fill = TRUE to reprocess the data
• D. Manually re-load the data
• E. Autoloader automatically re-processes data that was not loaded

正解：E

解説：
Explanation
The answer is,
Autoloader automatically re-processes data that was not loaded using the checkpoint.

質問 # 80
A Delta Lake table representing metadata about content from user has the following schema:
Based on the above schema, which column is a good candidate for partitioning the Delta Table?

• A. Date
• B. User_id
• C. Post_time
• D. Post_id

正解：A

解説：
Partitioning a Delta Lake table improves query performance by organizing data into partitions based on the values of a column. In the given schema, the date column is a good candidate for partitioning for several reasons:
* Time-Based Queries: If queries frequently filter or group by date, partitioning by the date column can significantly improve performance by limiting the amount of data scanned.
* Granularity: The date column likely has a granularity that leads to a reasonable number of partitions (not too many and not too few). This balance is important for optimizing both read and write performance.
* Data Skew: Other columns like post_id or user_id might lead to uneven partition sizes (data skew), which can negatively impact performance.
Partitioning by post_time could also be considered, but typically date is preferred due to its more manageable granularity.
References:
* Delta Lake Documentation on Table Partitioning: Optimizing Layout with Partitioning

質問 # 81
A user new to Databricks is trying to troubleshoot long execution times for some pipeline logic they are working on. Presently, the user is executing code cell-by-cell, using calls to confirm code is producing the logically correct results as new transformations are added to an operation. To get a measure of average time to execute, the user is running each cell multiple times interactively.
Which of the following adjustments will get a more accurate measure of how code is likely to perform in production?

• A. The Jobs Ul should be leveraged to occasionally run the notebook as a job and track execution time during incremental code development because Photon can only be enabled on clusters launched for scheduled jobs.
• B. Calling display () forces a job to trigger, while many transformations will only add to the logical query plan; because of caching, repeated execution of the same logic does not provide meaningful results.
• C. Production code development should only be done using an IDE; executing code against a local build of open source Spark and Delta Lake will provide the most accurate benchmarks for how code will perform in production.
• D. Scala is the only language that can be accurately tested using interactive notebooks; because the best performance is achieved by using Scala code compiled to JARs. all PySpark and Spark SQL logic should be refactored.
• E. The only way to meaningfully troubleshoot code execution times in development notebooks Is to use production-sized data and production-sized clusters with Run All execution.

正解：B

解説：
Explanation
This is the correct answer because it explains which of the following adjustments will get a more accurate measure of how code is likely to perform in production. The adjustment is that calling display() forces a job to trigger, while many transformations will only add to the logical query plan; because of caching, repeated execution of the same logic does not provide meaningful results. When developing code in Databricks notebooks, one should be aware of how Spark handles transformations and actions. Transformations are operations that create a new DataFrame or Dataset from an existing one, such as filter, select, or join. Actions are operations that trigger a computation on a DataFrame or Dataset and return a result to the driver program or write it to storage, such as count, show, or save. Calling display() on a DataFrame or Dataset is also an action that triggers a computation and displays the result in a notebook cell. Spark uses lazy evaluation for transformations, which means that they are not executed until an action is called. Spark also uses caching to store intermediate results in memory or disk for faster access in subsequent actions. Therefore, calling display() forces a job to trigger, while many transformations will only add to the logical query plan; because of caching, repeated execution of the same logic does not provide meaningful results. To get a more accurate measure of how code is likely to perform in production, one should avoid calling display() too often or clear the cache before running each cell. Verified References: [Databricks Certified Data Engineer Professional], under "Spark Core" section; Databricks Documentation, under "Lazy evaluation" section; Databricks Documentation, under "Caching" section.

質問 # 82
An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the fieldpk_id.
For auditing purposes, the data governance team wishes to maintain a full record of all values that have ever been valid in the source system. For analytical purposes, only the most recent value for each record needs to be recorded. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour.
Which solution meets these requirements?

• A. Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
• B. Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.
• C. Use merge into to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.
• D. Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
• E. Ingest all log information into a bronze table; use merge into to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.

正解：C

解説：
This is the correct answer because it meets the requirements of maintaining a full record of all values that have ever been valid in the source system and recreating the current table state with only the most recent value for each record. The code ingests all log information into a bronze table, which preserves the raw CDC data as it is. Then, it uses merge into to perform an upsert operation on a silver table, which means it will insert new records or update or delete existing records based on the change type and the pk_id columns. This way, the silver table will always reflect the current state of the source table, while the bronze table will keep the history of all changes. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Upsert into a table using merge" section.

質問 # 83
A data engineer wants to create a relational object by pulling data from two tables. The relational object must
be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to
avoid copying and storing physical data.
Which of the following relational objects should the data engineer create?

• A. Database
• B. View
• C. Spark SQL Table
• D. Temporary view
• E. Delta Table

正解：B

質問 # 84
A production workload incrementally applies updates from an external Change Data Capture feed to a Delta Lake table as an always-on Structured Stream job. When data was initially migrated for this table, OPTIMIZE was executed and most data files were resized to 1 GB. Auto Optimize and Auto Compaction were both turned on for the streaming production job. Recent review of data files shows that most data files are under 64 MB, although each partition in the table contains at least 1 GB of data and the total table size is over 10 TB.
Which of the following likely explains these smaller file sizes?

• A. Z-order indices calculated on the table are preventing file compaction C Bloom filler indices calculated on the table are preventing file compaction
• B. Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations
• C. Databricks has autotuned to a smaller target file size based on the overall size of data in the table
• D. Databricks has autotuned to a smaller target file size based on the amount of data in each partition

正解：B

解説：
This is the correct answer because Databricks has a feature called Auto Optimize, which automatically optimizes the layout of Delta Lake tables by coalescing small files into larger ones and sorting data within each file by a specified column. However, Auto Optimize also considers the trade-off between file size and merge performance, and may choose a smaller target file size to reduce the duration of merge operations, especially for streaming workloads that frequently update existing records. Therefore, it is possible that Auto Optimize has autotuned to a smaller target file size based on the characteristics of the streaming production job. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Auto Optimize" section.
https://docs.databricks.com/en/delta/tune-file-size.html#autotune-table 'Autotune file size based on workload'

質問 # 85
......

Databricks認定のプロフェッショナルDATA-Engineer認定試験は、DataBricksを使用するデータエンジニアにとって貴重な資格です。この認定は、Databricksテクノロジーとデータエンジニアリングの概念に関する候補者の専門知識を示しています。この認定は、候補者の専門能力開発と継続的な学習に対するコミットメントも示しています。

Databricks認定プロフェッショナルデータエンジニア試験は、候補者がデータエンジニアリングの概念を深く理解し、Databricksプラットフォームの強力な知識を持つ必要がある挑戦的で厳密な試験です。ただし、適切な準備により、候補者は試験に合格し、この貴重な認証を達成できます。 Databricksは、オンラインコース、練習試験、学習ガイドなど、候補者が試験の準備を支援するために、さまざまなリソースとトレーニングプログラムを提供しています。

 

Databricks Databricks-Certified-Professional-Data-Engineer試験問題集で[2024年最新] 練習有効な試験問題集解答：https://www.passtest.jp/Databricks/Databricks-Certified-Professional-Data-Engineer-shiken.html

Databricks-Certified-Professional-Data-Engineer問題集を掴み取れ！[最新2024]Databricks試験が合格できます：https://drive.google.com/open?id=1IsOzvjaWuj49l5UyUz-6w5e4UjqjWnWZ