[2024年更新]Databricks-Certified-Professional-Data-Engineerリアルな試験問題集でDatabricks-Certified-Professional-Data-Engineer練習テスト
Databricks-Certified-Professional-Data-Engineer問題集でDatabricks Certification高確率練習問題集
DCPDE試験の準備をするには、候補者は、データモデリング、データ統合、データ変換、データ品質などのデータエンジニアリングの概念を確実に理解する必要があります。また、Apache Spark、Apache Kafka、Apache Hadoopなどのビッグデータテクノロジーの操作経験もあります。
質問 # 16
Which of the following locations in Databricks product architecture hosts jobs/pipelines and queries?
- A. Databricks Filesystem
- B. Data plane
- C. JDBC data source
- D. Control plane
- E. Databricks web application
正解：D
解説：
Explanation
The answer is Control Plane,
Databricks operates most of its services out of a control plane and a data plane, please note serverless features like SQL Endpoint and DLT compute use shared compute in Control pane.
Control Plane: Stored in Databricks Cloud Account
*The control plane includes the backend services that Databricks manages in its own Azure account. Notebook commands and many other workspace configurations are stored in the control plane and encrypted at rest.
Data Plane: Stored in Customer Cloud Account
*The data plane is managed by your Azure account and is where your data resides. This is also where data is processed. You can use Azure Databricks connectors so that your clusters can connect to external data sources outside of your Azure account to ingest data or for storage.
Here is the product architecture diagram highlighted where
質問 # 17
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then
perform a streaming write into a new table. The code block used by the data engineer is below:
1. (spark.table("sales")
2. .withColumn("avg_price", col("sales") / col("units"))
3. .writeStream
4. .option("checkpointLocation", checkpointPath)
5. .outputMode("complete")
6. ._____
7. .table("new_sales")
8.)
If the data engineer only wants the query to execute a single micro-batch to process all of the available data,
which of the following lines of code should the data engineer use to fill in the blank?
- A. .trigger(continuous="once")
- B. .processingTime("once")
- C. .processingTime(1)
- D. .trigger(once=True)
- E. .trigger(processingTime="once")
正解：D
質問 # 18
A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Events are recorded once per minute per device.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:
Choose the response that correctly fills in the blank within the code block to complete this task.
- A. to_interval("event_time", "5 minutes").alias("time")
- B. window("event_time", "10 minutes").alias("time")
- C. "event_time"
- D. lag("event_time", "10 minutes").alias("time")
- E. window("event_time", "5 minutes").alias("time")
正解：E
解説：
This is the correct answer because the window function is used to group streaming data by time intervals. The window function takes two arguments: a time column and a window duration. The window duration specifies how long each window is, and must be a multiple of 1 second. In this case, the window duration is "5 minutes", which means each window will cover a non-overlapping five-minute interval. The window function also returns a struct column with two fields: start and end, which represent the start and end time of each window. The alias function is used to rename the struct column as "time". Verified References: [Databricks Certified Data Engineer Professional], under "Structured Streaming" section; Databricks Documentation, under "WINDOW" section.
https://www.databricks.com/blog/2017/05/08/event-time-aggregation-watermarking-apache-sparks-structured-str
質問 # 19
The data architect has mandated that all tables in the Lakehouse should be configured as external (also known as "unmanaged") Delta Lake tables.
Which approach will ensure that this requirement is met?
- A. When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.
- B. When a database is being created, make sure that the LOCATION keyword is used.
- C. When the workspace is being configured, make sure that external cloud object storage has been mounted.
- D. When data is saved to a table, make sure that a full file path is specified alongside the Delta format.
- E. When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.
正解：A
解説：
To create an external or unmanaged Delta Lake table, you need to use the EXTERNAL keyword in the CREATE TABLE statement. This indicates that the table is not managed by the catalog and the data files are not deleted when the table is dropped. You also need to provide a LOCATION clause to specify the path where the data files are stored. For example:
CREATE EXTERNAL TABLE events ( date DATE, eventId STRING, eventType STRING, data STRING) USING DELTA LOCATION '/mnt/delta/events'; This creates an external Delta Lake table named events that references the data files in the '/mnt/delta/events' path. If you drop this table, the data files will remain intact and you can recreate the table with the same statement.
References:
* https://docs.databricks.com/delta/delta-batch.html#create-a-table
* https://docs.databricks.com/delta/delta-batch.html#drop-a-table
質問 # 20
The data science team has created and logged a production model using MLflow. The following code correctly imports and applies the production model to output the predictions as a new DataFrame named preds with the schema "customer_id LONG, predictions DOUBLE, date DATE".
The data science team would like predictions saved to a Delta Lake table with the ability to compare all predictions across time. Churn predictions will be made at most once per day.
Which code block accomplishes this task while minimizing potential compute costs?
- A.
- B.
- C. preds.write.mode("append").saveAsTable("churn_preds")
- D. preds.write.format("delta").save("/preds/churn_preds")
- E.
正解：C
質問 # 21
You currently working with the marketing team to setup a dashboard for ad campaign analysis, since the team is not sure how often the dashboard should be refreshed they have decided to do a manual refresh on an as needed basis. Which of the following steps can be taken to reduce the overall cost of the compute when the team is not using the compute?
*Please note that Databricks recently change the name of SQL Endpoint to SQL Warehouses.
- A. They can turn on the Auto Stop feature for the SQL endpoint(SQL Warehouse).
- B. They can decrease the maximum bound of the SQL endpoint(SQL Warehouse) scaling range.
- C. They can decrease the cluster size of the SQL endpoint(SQL Warehouse).
- D. They can turn on the Serverless feature for the SQL endpoint(SQL Warehouse).
- E. They can turn on the Serverless feature for the SQL endpoint(SQL Warehouse) and change the Spot Instance Policy from "Reliability Optimized" to "Cost optimized"
正解：A
解説：
Explanation
The answer is, They can turn on the Auto Stop feature for the SQL endpoint(SQL Warehouse).
Use auto stop to automatically terminate the cluster when you are not using it.
質問 # 22
You are working on a process to query the table based on batch date, and batch date is an input parameter and expected to change every time the program runs, what is the best way to we can parameterize the query to run without manually changing the batch date?
- A. Create a dynamic view that can calculate the batch date automatically and use the view to query the data
- B. Store the batch date in the spark configuration and use a spark data frame to filter the data based on the spark configuration.
- C. There is no way we can combine python variable and spark code
- D. Create a notebook parameter for batch date and assign the value to a python variable and use a spark data frame to filter the data based on the python variable
- E. Manually edit code every time to change the batch date
正解：D
解説：
Explanation
The answer is, Create a notebook parameter for batch date and assign the value to a python variable and use a spark data frame to filter the data based on the python variable
質問 # 23
A data engineer is configuring a pipeline that will potentially see late-arriving, duplicate records.
In addition to de-duplicating records within the batch, which of the following approaches allows the data engineer to deduplicate data against previously processed records as it is inserted into a Delta table?
- A. Rely on Delta Lake schema enforcement to prevent duplicate records.
- B. VACUUM the Delta table after each batch completes.
- C. Perform an insert-only merge with a matching condition on a unique key.
- D. Perform a full outer join on a unique key and overwrite existing data.
- E. Set the configuration delta.deduplicate = true.
正解：C
解説：
To deduplicate data against previously processed records as it is inserted into a Delta table, you can use the merge operation with an insert-only clause. This allows you to insert new records that do not match any existing records based on a unique key, while ignoring duplicate records that match existing records. For example, you can use the following syntax:
MERGE INTO target_table USING source_table ON target_table.unique_key = source_table.unique_key WHEN NOT MATCHED THEN INSERT * This will insert only the records from the source table that have a unique key that is not present in the target table, and skip the records that have a matching key. This way, you can avoid inserting duplicate records into the Delta table.
References:
* https://docs.databricks.com/delta/delta-update.html#upsert-into-a-table-using-merge
* https://docs.databricks.com/delta/delta-update.html#insert-only-merge
質問 # 24
The data engineering team is using a bunch of SQL queries to review data quality and monitor the ETL job every day, which of the following approaches can be used to set up a schedule and auto-mate this process?
- A. They can schedule the query to refresh every 1 day from the query's page in Databricks SQL.
- B. They can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL
- C. They can schedule the query to run every 1 day from the Jobs UI
- D. They can schedule the query to refresh every 1 day from the SQL endpoint's page in Databricks SQL.
- E. They can schedule the query to run every 12 hours from the Jobs UI.
正解：A
解説：
Explanation
Explanation
Individual queries can be refreshed on a schedule basis,
To set the schedule:
1. Click the query info tab.
Graphical user interface, text, application, email Description automatically generated
* Click the link to the right of Refresh Schedule to open a picker with schedule intervals.
Graphical user interface, application Description automatically generated
* Set the schedule.
The picker scrolls and allows you to choose:
* An interval: 1-30 minutes, 1-12 hours, 1 or 30 days, 1 or 2 weeks
* A time. The time selector displays in the picker only when the interval is greater than 1 day and the day selection is greater than 1 week. When you schedule a specific time, Databricks SQL takes input in your computer's timezone and converts it to UTC. If you want a query to run at a certain time in UTC, you must adjust the picker by your local offset. For example, if you want a query to execute at 00:00 UTC each day, but your current timezone is PDT (UTC-7), you should select 17:00 in the picker:
Graphical user interface Description automatically generated
* Click OK.
Your query will run automatically.
If you experience a scheduled query not executing according to its schedule, you should manually trigger the query to make sure it doesn't fail. However, you should be aware of the following:
* If you schedule an interval-for example, "every 15 minutes"-the interval is calculated from the last successful execution. If you manually execute a query, the scheduled query will not be executed until the interval has passed.
* If you schedule a time, Databricks SQL waits for the results to be "outdated". For example, if you have a query set to refresh every Thursday and you manually execute it on Wednesday, by Thursday the results will still be considered "valid", so the query wouldn't be scheduled for a new execution. Thus, for example, when setting a weekly schedule, check the last query execution time and expect the scheduled query to be executed on the selected day after that execution is a week old. Make sure not to manually execute the query during this time.
If a query execution fails, Databricks SQL retries with a back-off algorithm. The more failures the further away the next retry will be (and it might be beyond the refresh interval).
Refer documentation for additional info,
https://docs.microsoft.com/en-us/azure/databricks/sql/user/queries/schedule-query
質問 # 25
A data engineering manager has noticed that each of the queries in a Databricks SQL dashboard takes a few
minutes to update when they manually click the "Refresh" button. They are curious why this might be
occurring, so a team member provides a variety of reasons on why the delay might be occurring.
Which of the following reasons fails to explain why the dashboard might be taking a few minutes to update?
- A. The queries attached to the dashboard might all be connected to their own, unstarted Databricks clusters
- B. The queries attached to the dashboard might take a few minutes to run under normal circumstances
- C. The Job associated with updating the dashboard might be using a non-pooled endpoint
- D. The queries attached to the dashboard might first be checking to determine if new data is available
- E. The SQL endpoint being used by each of the queries might need a few minutes to start up
正解：C
質問 # 26
How do you create a delta live tables pipeline and deploy using DLT UI?
- A. Within the Workspace UI, click on SQL Endpoint, select Delta Live tables and create pipelinea and select the notebook with DLT code.
- B. Under Cluster UI, select SPARK UI and select Structured Streaming and click create pipeline and select the notebook with DLT code.
- C. Use VS Code and download DBX plugin, once the plugin is loaded you can build DLT pipelines and select the notebook with DLT code.
- D. There is no UI, you can only setup DELTA LIVE TABLES using Python and SQL API and select the notebook with DLT code.
- E. Within the Workspace UI, click on Workflows, select Delta Live tables and create a pipeline and select the notebook with DLT code.
正解：E
解説：
Explanation
The answer is, Within the Workspace UI, click on Workflows, select Delta Live tables and create a pipeline and select the notebook with DLT code.
https://docs.databricks.com/data-engineering/delta-live-tables/delta-live-tables-quickstart.html
質問 # 27
The research team has put together a funnel analysis query to monitor the customer traffic on the e-commerce platform, the query takes about 30 mins to run on a small SQL endpoint cluster with max scaling set to 1 cluster. What steps can be taken to improve the performance of the query?
- A. They can turn on the Serverless feature for the SQL endpoint.
- B. They can turn off the Auto Stop feature for the SQL endpoint to more than 30 mins.
- C. They can turn on the Serverless feature for the SQL endpoint and change the Spot In-stance Policy from
"Cost optimized" to "Reliability Optimized." - D. They can increase the cluster size anywhere from X small to 3XL to review the per-formance and select the size that meets the required SLA.
- E. They can increase the maximum bound of the SQL endpoint's scaling range anywhere from between 1 to 100 to review the performance and select the size that meets the re-quired SLA.
正解：D
解説：
Explanation
The answer is, They can increase the cluster size anywhere from 2X-Small to 4XL(Scale Up) to review the performance and select the size that meets your SLA. If you are trying to improve the performance of a single query at a time having additional memory, additional worker nodes mean that more tasks can run in a cluster which will improve the performance of that query.
The question is looking to test your ability to know how to scale a SQL Endpoint(SQL Warehouse) and you have to look for cue words or need to understand if the queries are running sequentially or concurrently. if the queries are running sequentially then scale up(Size of the cluster from 2X-Small to 4X-Large) if the queries are running concurrently or with more users then scale out(add more clusters).
SQL Endpoint(SQL Warehouse) Overview: (Please read all of the below points and the below diagram to understand )
1.A SQL Warehouse should have at least one cluster
2.A cluster comprises one driver node and one or many worker nodes
3.No of worker nodes in a cluster is determined by the size of the cluster (2X -Small ->1 worker, X-Small ->2 workers.... up to 4X-Large -> 128 workers) this is called Scale Up
4.A single cluster irrespective of cluster size(2X-Smal.. to ...4XLarge) can only run 10 queries at any given time if a user submits 20 queries all at once to a warehouse with 3X-Large cluster size and cluster scaling (min
1, max1) while 10 queries will start running the remaining 10 queries wait in a queue for these 10 to finish.
5.Increasing the Warehouse cluster size can improve the performance of a query, example if a query runs for 1 minute in a 2X-Small warehouse size, it may run in 30 Seconds if we change the warehouse size to X-Small.
this is due to 2X-Small has 1 worker node and X-Small has 2 worker nodes so the query has more tasks and runs faster (note: this is an ideal case example, the scalability of a query performance depends on many factors, it can not always be linear)
6.A warehouse can have more than one cluster this is called Scale Out. If a warehouse is configured with X-Small cluster size with cluster scaling(Min1, Max 2) Databricks spins up an additional cluster if it detects queries are waiting in the queue, If a warehouse is configured to run 2 clusters(Min1, Max 2), and let's say a user submits 20 queries, 10 queriers will start running and holds the remaining in the queue and databricks will automatically start the second cluster and starts redirecting the 10 queries waiting in the queue to the second cluster.
7.A single query will not span more than one cluster, once a query is submitted to a cluster it will remain in that cluster until the query execution finishes irrespective of how many clusters are available to scale.
Please review the below diagram to understand the above concepts:
Scale-up-> Increase the size of the SQL endpoint, change cluster size from 2X-Small to up to 4X-Large If you are trying to improve the performance of a single query having additional memory, additional worker nodes and cores will result in more tasks running in the cluster will ultimately improve the performance.
During the warehouse creation or after, you have the ability to change the warehouse size (2X-Small....to
...4XLarge) to improve query performance and the maximize scaling range to add more clusters on a SQL Endpoint(SQL Warehouse) scale-out if you are changing an existing warehouse you may have to restart the warehouse to make the changes effective.
質問 # 28
At the end of the inventory process a file gets uploaded to the cloud object storage, you are asked to build a process to ingest data which of the following method can be used to ingest the data incrementally, the schema of the file is expected to change overtime ingestion process should be able to handle these changes automatically. Below is the auto loader command to load the data, fill in the blanks for successful execution of the below code.
1.spark.readStream
2..format("cloudfiles")
3..option("cloudfiles.format","csv)
4..option("_______", 'dbfs:/location/checkpoint/')
5..load(data_source)
6..writeStream
7..option("_______",' dbfs:/location/checkpoint/')
8..option("mergeSchema", "true")
9..table(table_name))
- A. cloudfiles.schemalocation, checkpointlocation
- B. cloudfiles.schemalocation, cloudfiles.checkpointlocation
- C. schemalocation, checkpointlocation
- D. checkpointlocation, schemalocation
- E. checkpointlocation, cloudfiles.schemalocation
正解：A
解説：
Explanation
The answer is cloudfiles.schemalocation, checkpointlocation
When reading the data cloudfiles.schemalocation is used to store the inferred schema of the incoming data.
When writing a stream to recover from failures checkpointlocation is used to store the offset of the byte that was most recently processed.
質問 # 29
The business intelligence team has a dashboard configured to track various summary metrics for retail stories.
This includes total sales for the previous day alongside totals and averages for a variety of time periods. The fields required to populate this dashboard have the following schema:
For Demand forecasting, the Lakehouse contains a validated table of all itemized sales updated incrementally in near real-time. This table named products_per_order, includes the following fields:
Because reporting on long-term sales trends is less volatile, analysts using the new dashboard only require data to be refreshed once daily. Because the dashboard will be queried interactively by many users throughout a normal business day, it should return results quickly and reduce total compute associated with each materialization.
Which solution meets the expectations of the end users while controlling and limiting possible costs?
- A. Populate the dashboard by configuring a nightly batch job to save the required to quickly update the dashboard with each query.
- B. Use Structure Streaming to configure a live dashboard against the products_per_order table within a Databricks notebook.
- C. Define a view against the products_per_order table and define the dashboard against this view.
- D. Use the Delta Cache to persists the products_per_order table in memory to quickly the dashboard with each query.
正解：A
解説：
Given the requirement for daily refresh of data and the need to ensure quick response times for interactive queries while controlling costs, a nightly batch job to pre-compute and save the required summary metrics is the most suitable approach.
* By pre-aggregating data during off-peak hours, the dashboard can serve queries quickly without requiring on-the-fly computation, which can be resource-intensive and slow, especially with many users.
* This approach also limits the cost by avoiding continuous computation throughout the day and instead leverages a batch process that efficiently computes and stores the necessary data.
* The other options (A, C, D) either do not address the cost and performance requirements effectively or are not suitable for the use case of less frequent data refresh and high interactivity.
References:
* Databricks Documentation on Batch Processing: Databricks Batch Processing
* Data Lakehouse Patterns: Data Lakehouse Best Practices
質問 # 30
You are working on a process to load external CSV files into a delta table by leveraging the COPY INTO command, but after running the command for the second time no data was loaded into the table name, why is that?
1.COPY INTO table_name
2.FROM 'dbfs:/mnt/raw/*.csv'
3.FILEFORMAT = CSV
- A. COPY INTO does not support incremental load, use AUTO LOADER
- B. COPY INTO only works one time data load
- C. Run REFRESH TABLE sales before running COPY INTO
- D. Use incremental = TRUE option to load new files
- E. COPY INTO did not detect new files after the last load
正解：E
解説：
Explanation
The answer is COPY INTO did not detect new files after the last load,
COPY INTO keeps track of files that were successfully loaded into the table, the next time when the COPY INTO runs it skips them.
FYI, you can change this behavior by using COPY_OPTIONS 'force'= 'true', when this option is enabled all files in the path/pattern are loaded.
1.COPY INTO table_identifier
2. FROM [ file_location | (SELECT identifier_list FROM file_location) ]
3. FILEFORMAT = data_source
4. [FILES = [file_name, ... | PATTERN = 'regex_pattern']
5. [FORMAT_OPTIONS ('data_source_reader_option' = 'value', ...)]
6. [COPY_OPTIONS 'force' = ('false'|'true')]
質問 # 31
What is the underlying technology that makes the Auto Loader work?
- A. DataFrames
- B. Structured Streaming
- C. Loader
- D. Live DataFames
- E. Delta Live Tables
正解：B
質問 # 32
If you create a database sample_db with the statement CREATE DATABASE sample_db what will be the default location of the database in DBFS?
- A. Statement fails "Unable to create database without location"
- B. Default location, /user/db/
- C. Default Storage account
- D. Default Location, dbfs:/user/hive/warehouse
- E. Default location, DBFS:/user/
正解：D
解説：
Explanation
The Answer is dbfs:/user/hive/warehouse this is the default location where spark stores user data-bases, the default can be changed using spark.sql.warehouse.dir a parameter. You can also provide a custom location using the LOCATION keyword.
Here is how this works,
Graphical user interface, text, application, email Description automatically generated
Default location
FYI, This can be changed used using cluster spark config or session config.
Modify spark.sql.warehouse.dir location to change the default location
Graphical user interface, text, application Description automatically generated
質問 # 33
The data engineering team maintains a table of aggregate statistics through batch nightly updates. This includes total sales for the previous day alongside totals and averages for a variety of time periods including the 7 previous days, year-to-date, and quarter-to-date. This table is namedstore_saies_summaryand the schema is as follows:
The tabledaily_store_salescontains all the information needed to updatestore_sales_summary. The schema for this table is:
store_id INT, sales_date DATE, total_sales FLOAT
Ifdaily_store_salesis implemented as a Type 1 table and thetotal_salescolumn might be adjusted after manual data auditing, which approach is the safest to generate accurate reports in thestore_sales_summarytable?
- A. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and append new rows nightly to the store_sales_summary table.
- B. Use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update.
- C. Implement the appropriate aggregate logic as a Structured Streaming read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
- D. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and overwrite the store_sales_summary table with each Update.
- E. Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
正解：B
解説：
The daily_store_sales table contains all the information needed to update store_sales_summary. The schema of the table is:
store_id INT, sales_date DATE, total_sales FLOAT
The daily_store_sales table is implemented as a Type 1 table, which means that old values are overwritten by new values and no history is maintained. The total_sales column might be adjusted after manual data auditing, which means that the data in the table may change over time.
The safest approach to generate accurate reports in the store_sales_summary table is to use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update. Structured Streaming is a scalable and fault-tolerant stream processing engine built on Spark SQL. Structured Streaming allows processing data streams as if they were tables or DataFrames, using familiar operations such as select, filter, groupBy, or join. Structured Streaming also supports output modes that specify how to write the results of a streaming query to a sink, such as append, update, or complete. Structured Streaming can handle both streaming and batch data sources in a unified manner.
The change data feed is a feature of Delta Lake that provides structured streaming sources that can subscribe to changes made to a Delta Lake table. The change data feed captures both data changes and schema changes as ordered events that can be processed by downstream applications or services. The change data feed can be configured with different options, such as starting from a specific version or timestamp, filtering by operation type or partition values, or excluding no-op changes.
By using Structured Streaming to subscribe to the change data feed for daily_store_sales, one can capture and process any changes made to the total_sales column due to manual data auditing. By applying these changes to the aggregates in the store_sales_summary table with each update, one can ensure that the reports are always consistent and accurate with the latest data. Verified References: [Databricks Certified Data Engineer Professional], under "Spark Core" section; Databricks Documentation, under "Structured Streaming" section; Databricks Documentation, under "Delta Change Data Feed" section.
質問 # 34
Drop the customers database and associated tables and data, all of the tables inside the database are managed tables. Which of the following SQL commands will help you accomplish this?
- A. DROP DELTA DATABSE customers
- B. All the tables must be dropped first before dropping database
- C. DROP DATABASE customers CASCADE
- D. DROP DATABASE customers INCLUDE
- E. DROP DATABASE customers FORCE
正解：D
解説：
Explanation
The answer is DROP DATABASE customers CASCADE
Drop database with cascade option drops all the tables, since all of the tables inside the database are managed tables we do not need to perform any additional steps to clean the data in the storage.
質問 # 35
A Data engineer wants to run unit's tests using common Python testing frameworks on python functions defined across several Databricks notebooks currently used in production.
How can the data engineer run unit tests against function that work with data in production?
- A. Define and unit test functions using Files in Repos
- B. Define units test and functions within the same notebook
- C. Define and import unit test functions from a separate Databricks notebook
- D. Run unit tests against non-production data that closely mirrors production
正解：D
解説：
The best practice for running unit tests on functions that interact with data is to use a dataset that closely mirrors the production data. This approach allows data engineers to validate the logic of their functions without the risk of affecting the actual production data. It's important to have a representative sample of production data to catch edge cases and ensure the functions will work correctly when used in a production environment.
References:
* Databricks Documentation on Testing: Testing and Validation of Data and Notebooks
質問 # 36
......
Databricks-Certified-Professional-Data-Engineerリアルな問題と知能問題集：https://www.passtest.jp/Databricks/Databricks-Certified-Professional-Data-Engineer-shiken.html
合格できるDatabricks-Certified-Professional-Data-Engineer試験と最新Databricks-Certified-Professional-Data-Engineer試験問題集PDF2024：https://drive.google.com/open?id=19X8PrRX-pt6Mz6U1YPMAzmT-8HZFzFsy