無料提供されるDatabricks-Certified-Data-Engineer-Professional問題集とDatabricks-Certified-Data-Engineer-Professionalリアル試験問題 [Q118-Q137]

無料提供されるDatabricks-Certified-Data-Engineer-Professional問題集とDatabricks-Certified-Data-Engineer-Professionalリアル試験問題

Databricks Databricks-Certified-Data-Engineer-Professional実際の問題とブレーン問題集

質問 # 118
A platform engineer needs to report the resource consumption, categorized by SKU tier, across all workspaces. The engineer decides to use the system.billing.usage system table to create a query. Which SQL query will accurately return the daily usage by product?

• A.
• B.
• C.
• D.

正解：B

解説：
This query correctly aggregates usage at a daily granularity by truncating the usage start timestamp to the day and summing the usage quantity, which represents DBUs. Grouping by both the derived daily value and the SKU name ensures usage is accurately categorized by product tier across all workspaces.

質問 # 119
When a new Databricks project starts, the central IP team provisions the required infrastructure using Terraform and a Service Principal. This includes creating a Databricks workspace, a Unity Catalog linked to an External Location, and a Databricks group containing all project team members. Project teams must store all assets - e.g., tables and volumes, as Managed assets in Unity Catalog. This model hides infrastructure complexity while giving teams autonomy within their catalog. They can create and manage schemas, tables, volumes, and related objects but cannot rename, delete, or change catalog permissions, those remain under IT's control. Which rights should the project group be granted to enable this model?

• A. The group needs to have ALL PRIVILEGES on the catalog.
• B. The group should be made OWNER of the catalog.
• C. The group needs to have USE CATALOG and USE SCHEMA on the catalog.
• D. The group needs to have ALL PRIVILEGES and the MANAGE on the catalog.

正解：C

解説：
Granting the project group USE CATALOG (and USE SCHEMA on existing schemas) allows team members to work within the catalog without giving them ownership or administrative control.
This enables teams to create and manage schemas, tables, and volumes inside the catalog while preventing them from renaming, deleting, or modifying catalog-level permissions, which remain under central IT governance.

質問 # 120
A data engineer is using Structured Streaming to read in transaction data from a bronze Delta table. It was discovered that the data has quality issues where sometimes the transaction value is negative, and when that occurs, the rows need to be routed to a separate quarantine table. They have low latency requirements for the good data since it is used by downstream systems, but the bad data will only be analyzed periodically and has no production dependencies. The quarantine job needs to be implemented so that it cannot affect the production processes that depend on the good data, and the cost of the job needs to be minimized. How should the quarantine process be implemented in order to satisfy these requirements?

• A. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing, and should not share compute with other processes. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing, and should be implemented on a separate small cluster and only run once a day to minimize cost.
• B. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing. Both should run as separate streams on the same cluster to minimize cost.
• C. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. Inside a foreachBatch function, the dataframe should be filtered so that records with a transaction value greater than or equal to 0 are written to the good data table and records with a transaction value less than 0 are written to a quarantine table. Try/Catch can be added around the writes in the foreachBatch function so that the stream can't fail.
• D. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. A new boolean column called "quarantine" should be added to the dataframe, and its value should be set to true if the transaction value is less than 0 and false if the transaction value is greater than or equal to 0. Processing and storing all the data together will save costs.

正解：A

解説：
Separating the quarantine processing from the low-latency production stream ensures the good- data pipeline cannot be slowed down or destabilized by quarantine writes or failures. Filtering out negative values in the production stream preserves latency and reliability for downstream systems. Running the quarantine stream on a small, independent cluster on a daily schedule minimizes cost while still capturing the bad records for periodic analysis, and it fully isolates quarantine operations from production dependencies.

質問 # 121
A query is taking too long to run. After investigating the Spark UI, the data engineer discovered a significant amount of disk spill. The compute instance being used has a core-to-memory ratio of
1:2. What are the two steps the data engineer should take to minimize spillage? (Choose two.)

• A. Reduce spark.sql.files.maxPartitionBytes.
• B. Choose a compute instance with a higher core-to-memory ratio.
• C. Choose a compute instance with more network bandwidth.
• D. Choose a compute instance with more disk space.
• E. Increase spark.sql.files.maxPartitionBytes.

正解：A、B

解説：
Databricks recommends addressing disk spilling--which occurs when Spark tasks run out of memory--by increasing memory per core and controlling partition size. Selecting an instance type with a higher memory-to-core ratio (A) provides each task with more available RAM, directly reducing the chance of spilling to disk. Additionally, reducing spark.sql.files.maxPartitionBytes (D) creates smaller partitions, preventing any single task from holding too much data in memory.
Increasing partition size (C) or disk capacity (B) does not solve memory bottlenecks, and bandwidth (E) affects network I/O, not spill behavior. Therefore, the correct actions are A and D.

質問 # 122
The data engineering team has configured a Databricks SQL query and alert to monitor the values in a Delta Lake table. The recent_sensor_recordings table contains an identifying sensor_id alongside the timestamp and temperature for the most recent 5 minutes of recordings.
The below query is used to create the alert:

The query is set to refresh each minute and always completes in less than 10 seconds. The alert is set to trigger when mean (temperature) > 120. Notifications are triggered to be sent at most every 1 minute.
If this alert raises notifications for 3 consecutive minutes and then stops, which statement must be true?

• A. The maximum temperature recording for at least one sensor exceeded 120 on three consecutive executions of the query
• B. The total average temperature across all sensors exceeded 120 on three consecutive executions of the query
• C. The recent_sensor_recordingstable was unresponsive for three consecutive runs of the query
• D. The average temperature recordings for at least one sensor exceeded 120 on three consecutive executions of the query
• E. The source query failed to update properly for three consecutive minutes and then restarted

正解：D

解説：
This is the correct answer because the query is using a GROUP BY clause on the sensor_id column, which means it will calculate the mean temperature for each sensor separately. The alert will trigger when the mean temperature for any sensor is greater than 120, which means at least one sensor had an average temperature above 120 for three consecutive minutes. The alert will stop when the mean temperature for all sensors drops below 120.

質問 # 123
A data engineer needs to implement column masking for a sensitive column in a Unity Catalog- managed table. The masking logic must dynamically check if users belong to specific groups defined in a separate table (group_access) that maps groups to allowed departments. Which approach should the engineer use to efficiently enforce this requirement?

• A. Create a view without selecting the sensitive column.
• B. Create a UDF that hardcodes allowed groups and apply it as a column mask.
• C. Apply a column mask that references the group_access mapping table in its UDF.
• D. Use a row filter to restrict access based on the user's group.

正解：C

解説：
Databricks Unity Catalog supports dynamic column masking, where masking logic can be implemented using SQL functions or UDFs that reference external mapping tables or metadata for context-aware access control.
By referencing the group_access table inside the masking function, the mask dynamically evaluates whether a requesting user belongs to an authorized group. If permitted, the original column value is returned; otherwise, a masked value (such as NULL or asterisks) is shown.
This method enables fine-grained, data-driven masking policies while maintaining a single authoritative access mapping source.
Hardcoding values (A) reduces flexibility, and row filters (D) apply to entire rows rather than specific columns. Therefore, C correctly aligns with Databricks best practices for dynamic masking.

質問 # 124
A data engineer wants to ingest a large collection of image files (JPEG and PNG) from cloud object storage into a Unity Catalog-managed table for analysis and visualization. Which two configurations and practices are recommended to incrementally ingest these images into the table? (Choose two.)

• A. Move files to a volume and read with SQL editor.
• B. Use Auto Loader and set cloudFiles.format to "BINARYFILE".
• C. Use Auto Loader and set cloudFiles.format to "TEXT".
• D. Use the pathGlobFilter option to select only image files (e.g., "*.jpg,*.png").
• E. Use Auto Loader and set cloudFiles.format to "IMAGE".

正解：B、D

解説：
Databricks Auto Loader supports ingestion of binary file formats using the cloudFiles.format option. For ingesting JPEG or PNG image files, the correct setting is "BINARYFILE", which loads the raw binary content and file metadata into a DataFrame. Additionally, when processing files from object storage, it is best practice to apply pathGlobFilter to limit ingestion to specific file types and reduce unnecessary scanning of non-image files. Options like "IMAGE" or "TEXT" are invalid, and using volumes with SQL editors does not provide incremental ingestion. Therefore, combining Auto Loader with cloudFiles.format="BINARYFILE" and pathGlobFilter ensures scalable, incremental ingestion of image data into Unity Catalog tables.

質問 # 125
The business reporting tem requires that data for their dashboards be updated every hour. The total processing time for the pipeline that extracts transforms and load the data for their pipeline runs in 10 minutes.
Assuming normal operating conditions, which configuration will meet their service-level agreement requirements with the lowest cost?

• A. Schedule a job to execute the pipeline once an hour on a dedicated interactive cluster.
• B. Schedule a Structured Streaming job with a trigger interval of 60 minutes.
• C. Configure a job that executes every time new data lands in a given directory.
• D. Schedule a job to execute the pipeline once an hour on a new job cluster.

正解：D

解説：
Scheduling a job to execute the data processing pipeline once an hour on a new job cluster is the most cost-effective solution given the scenario. Job clusters are ephemeral in nature; they are spun up just before the job execution and terminated upon completion, which means you only incur costs for the time the cluster is active. Since the total processing time is only 10 minutes, a new job cluster created for each hourly execution minimizes the running time and thus the cost, while also fulfilling the requirement for hourly data updates for the business reporting team's dashboards.

質問 # 126
The security team is exploring whether or not the Databricks secrets module can be leveraged for connecting to an external database.
After testing the code with all Python variables being defined with strings, they upload the password to the secrets module and configure the correct permissions for the currently active Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from user. They then modify their code to the following (leaving all other variables unchanged).

Which statement describes what will happen when the above code is executed?

• A. The connection to the external table will succeed; the string "redacted" will be printed.
• B. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the password will be printed in plain text.
• C. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the encoded password will be saved to DBFS.
• D. The connection to the external table will succeed; the string value of password will be printed in plain text.
• E. The connection to the external table will fail; the string "redacted" will be printed.

正解：A

解説：
This is the correct answer because the code is using the dbutils.secrets.get method to retrieve the password from the secrets module and store it in a variable. The secrets module allows users to securely store and access sensitive information such as passwords, tokens, or API keys. The connection to the external table will succeed because the password variable will contain the actual password value. However, when printing the password variable, the string "redacted" will be displayed instead of the plain text password, as a security measure to prevent exposing sensitive information in notebooks.

質問 # 127
The data architect has mandated that all tables in the Lakehouse should be configured as external Delta Lake tables.
Which approach will ensure that this requirement is met?

• A. Whenever a database is being created, make sure that the location keyword is used
• B. When tables are created, make sure that the external keyword is used in the create table statement.
• C. When configuring an external data warehouse for all table storage. leverage Databricks for all ELT.
• D. When the workspace is being configured, make sure that external cloud object storage has been mounted.
• E. Whenever a table is being created, make sure that the location keyword is used.

正解：E

解説：
This is the correct answer because it ensures that this requirement is met. The requirement is that all tables in the Lakehouse should be configured as external Delta Lake tables. An external table is a table that is stored outside of the default warehouse directory and whose metadata is not managed by Databricks. An external table can be created by using the location keyword to specify the path to an existing directory in a cloud storage system, such as DBFS or S3. By creating external tables, the data engineering team can avoid losing data if they drop or overwrite the table, as well as leverage existing data without moving or copying it.

質問 # 128
A data engineer wants to refactor the following DLT code, which includes multiple table definitions with very similar code.

In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.

The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for these tables.
How can the data engineer fix this?

• A. Wrap the loop inside another table definition, using generalized names and properties to replace with those from the inner table
• B. Convert the list of configuration values to a dictionary of table settings, using table names as keys.
• C. Load the configuration values for these tables from a separate file, located at a path provided by a pipeline parameter.
• D. Convert the list of configuration values to a dictionary of table settings, using different input the for loop.

正解：B

解説：
The issue with the refactored code is that it tries to use string interpolation to dynamically create table names within the dlc.table decorator, which will not correctly interpret the table names.
Instead, by using a dictionary with table names as keys and their configurations as values, the data engineer can iterate over the dictionary items and use the keys (table names) to properly configure the table settings. This way, the decorator can correctly recognize each table name, and the corresponding configuration settings can be applied appropriately.

質問 # 129
An upstream system is emitting change data capture (CDC) logs that are being written to a cloud object storage directory. Each record in the log indicates the change type (insert, update, or delete) and the values for each field after the change. The source table has a primary key identified by the field pk_id.
For analytical purposes, only the most recent value for each record needs to be recorded in the target Delta Lake table in the Lakehouse. The Databricks job to ingest these records occurs once per hour, but each individual record may have changed multiple times over the course of an hour.
Which solution meets these requirements?

• A. Deduplicate records in each batch by pk_id and overwrite the target table.
• B. Use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a table, then propagate all changes throughout the system.
• C. Iterate through an ordered set of changes to the table, applying each in turn to create the current state of the table, (insert, update, delete), timestamp of change, and the values.
• D. Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.

正解：D

質問 # 130
The data science team has created and logged a production model using MLflow. The following code correctly imports and applies the production model to output the predictions as a new DataFrame named preds with the schema "customer_id LONG, predictions DOUBLE, date DATE".

The data science team would like predictions saved to a Delta Lake table with the ability to compare all predictions across time. Churn predictions will be made at most once per day.
Which code block accomplishes this task while minimizing potential compute costs?

• A.
• B. preds.write.mode("append").saveAsTable("churn_preds")
• C.
• D.
• E. preds.write.format("delta").save("/preds/churn_preds")

正解：B

質問 # 131
The DevOps team has configured a production workload as a collection of notebooks scheduled to run daily using the Jobs Ul. A new data engineering hire is onboarding to the team and has requested access to one of these notebooks to review the production logic. What are the maximum notebook permissions that can be granted to the user without allowing accidental changes to production code or data?

• A. Can manage
• B. Can run
• C. Can edit
• D. Can Read

正解：D

解説：
Granting a user 'Can Read' permissions on a notebook within Databricks allows them to view the notebook's content without the ability to execute or edit it. This level of permission ensures that the new team member can review the production logic for learning or auditing purposes without the risk of altering the notebook's code or affecting production data and workflows. This approach aligns with best practices for maintaining security and integrity in production environments, where strict access controls are essential to prevent unintended modifications.

質問 # 132
A Delta table of weather records is partitioned by date and has the below schema:
date DATE, device_id INT, temp FLOAT, latitude FLOAT, longitude FLOAT
To find all the records from within the Arctic Circle, you execute a query with the below filter:
latitude > 66.3
Which statement describes how the Delta engine identifies which files to load?

• A. The Hive metastore is scanned for min and max statistics for the latitude column
• B. All records are cached to attached storage and then the filter is applied
• C. The Delta log is scanned for min and max statistics for the latitude column
• D. The Parquet file footers are scanned for min and max statistics for the latitude column
• E. All records are cached to an operational database and then the filter is applied

正解：C

解説：
This is the correct answer because Delta Lake uses a transaction log to store metadata about each table, including min and max statistics for each column in each data file. The Delta engine can use this information to quickly identify which files to load based on a filter condition, without scanning the entire table or the file footers. This is called data skipping and it can improve query performance significantly. Verified Reference: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; [Databricks Documentation], under "Optimizations - Data Skipping" section.
In the Transaction log, Delta Lake captures statistics for each data file of the table. These statistics indicate per file:
- Total number of records
- Minimum value in each column of the first 32 columns of the table
- Maximum value in each column of the first 32 columns of the table
- Null value counts for in each column of the first 32 columns of the table When a query with a selective filter is executed against the table, the query optimizer uses these statistics to generate the query result. it leverages them to identify data files that may contain records matching the conditional filter.
For the SELECT query in the question, The transaction log is scanned for min and max statistics for the price column.

質問 # 133
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.
Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?

• A. Total VMs: 2
  200 GB per Executor
  80 Cores / Executor
• B. Total VMs: 8
  50 GB per Executor
  20 Cores / Executor
• C. Total VMs: 4
  100 GB per Executor
  40 Cores/Executor
• D. Total VMs: 1
  400 GB per Executor
  160 Cores / Executor

正解：D

解説：
https://docs.databricks.com/en/clusters/cluster-config-best-practices.html

質問 # 134
A distributed team of data analysts share computing resources on an interactive cluster with autoscaling configured. In order to better manage costs and query throughput, the workspace administrator is hoping to evaluate whether cluster upscaling is caused by many concurrent users or resource-intensive queries.
In which location can one review the timeline for cluster resizing events?

• A. Workspace audit logs
• B. Executor's log file
• C. Driver's log file
• D. Cluster Event Log
• E. Ganglia

正解：D

解説：
The Cluster Event Log in Databricks will show the timeline for cluster resizing events, including details about when and why a cluster was resized (scaled up or down). This log would help the workspace administrator determine the causes of cluster scaling, whether due to many concurrent users submitting jobs or a few users running resource-intensive queries.

質問 # 135
A Delta table of weather records is partitioned by date and has the below schema:
date DATE, device_id INT, temp FLOAT, latitude FLOAT, longitude FLOAT
To find all the records from within the Arctic Circle, you execute a query with the below filter:
latitude > 66.3
Which statement describes how the Delta engine identifies which files to load?

• A. The Hive metastore is scanned for min and max statistics for the latitude column
• B. All records are cached to attached storage and then the filter is applied Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from
• C. The Delta log is scanned for min and max statistics for the latitude column
• D. The Parquet file footers are scanned for min and max statistics for the latitude column
• E. All records are cached to an operational database and then the filter is applied

正解：C

解説：
This is the correct answer because Delta Lake uses a transaction log to store metadata about each table, including min and max statistics for each column in each data file. The Delta engine can use this information to quickly identify which files to load based on a filter condition, without scanning the entire table or the file footers. This is called data skipping and it can improve query performance significantly. Verified Reference: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; [Databricks Documentation], under "Optimizations - Data Skipping" section.
In the Transaction log, Delta Lake captures statistics for each data file of the table. These statistics indicate per file:
- Total number of records
- Minimum value in each column of the first 32 columns of the table
- Maximum value in each column of the first 32 columns of the table
- Null value counts for in each column of the first 32 columns of the table When a query with a selective filter is executed against the table, the query optimizer uses these statistics to generate the query result. it leverages them to identify data files that may contain records matching the conditional filter.
For the SELECT query in the question, The transaction log is scanned for min and max statistics for the price column.

質問 # 136
A data engineer is masking a column containing email addresses. The goal is to produce output strings of identical length for all rows, while generating different outputs for different email values.
Which SQL function should be used to achieve this?

• A. sha1(email)
• B. sha2(email, 0)
• C. mask(email, '?')
• D. hash(email)

正解：D

解説：
The hash() function in Databricks SQL returns a deterministic fixed-length integer (or hexadecimal string) derived from the input. When applied to sensitive identifiers like email addresses, it produces a unique value for each distinct input while ensuring uniform output size, making it suitable for anonymization where referential consistency is required.
Functions like mask() perform pattern-based substitutions that change string lengths, and sha1() or sha2() produce long hexadecimal strings of varying lengths (depending on hash size), which may not match requirements for fixed-length masking.
Therefore, the correct choice for fixed-length, deterministic pseudonymization of email addresses is hash(email), as it maintains analytical usability while anonymizing sensitive data.

質問 # 137
......

Databricks-Certified-Data-Engineer-Professional合格させる問題集でDatabricks24時間で試験合格できます：https://www.passtest.jp/Databricks/Databricks-Certified-Data-Engineer-Professional-shiken.html

最新問題を使おうDatabricks-Certified-Data-Engineer-Professional試験問題と解答PDFで一年間無料更新：https://drive.google.com/open?id=1VE3tMeOK_RCfCXALtRFop7lt6C3VSy-x