試験高合格率保証2023年12月22日 Databricks-Certified-Professional-Data-Engineer試験問題と正確な回答！
テストエンジン練習問題Databricks-Certified-Professional-Data-Engineer有効最新の問題集
Databricks認定プロフェッショナルデータエンジニア試験は、データエンジニアリングに関連する幅広いトピックをカバーする包括的な試験です。これらのトピックには、データモデリング、データ取り込み、データ統合、データ変換、データストレージ、およびデータ分析が含まれます。候補者は、一連のタスクと演習を完了することによって、これらの領域での知識とスキルを証明する必要があります。
質問 # 13
Which distribution does Databricks support for installing custom Python code packages?
- A. nom
- B. CRAN
- C. Wheels
- D. CRAM
- E. jars
- F. sbt
正解：A
質問 # 14
You are designing an analytical to store structured data from your e-commerce platform and un-structured data from website traffic and app store, how would you approach where you store this data?
- A. Data lakehouse can store structured and unstructured data and can enforce schema
- B. Traditional data warehouses are good for storing structured data and enforcing schema
- C. Data lakehouse can only store unstructured data but cannot enforce a schema
- D. Use traditional data warehouse for structured data and use data lakehouse for un-structured data.
正解：A
解説：
Explanation
The answer is, Data lakehouse can store structured and unstructured data and can enforce schema What Is a Lakehouse? - The Databricks Blog Graphical user interface, text, application Description automatically generated
質問 # 15
Which of the following command can be used to drop a managed delta table and the underlying files in the storage?
- A. DROP TABLE table and run VACUUM command
- B. DROP TABLE table_name CASCADE
- C. DROP TABLE table_name
- D. Use DROP TABLE table_name command and manually delete files using com-mand dbutils.fs.rm("/path",True)
- E. DROP TABLE table_name INCLUDE_FILES
正解：C
解説：
Explanation
The answer is DROP TABLE table_name,
When a managed table is dropped, the table definition is dropped from metastore and everything including data, metadata, and history are also dropped from storage.
質問 # 16
You were asked to create or overwrite an existing delta table to store the below transaction data.
- A. 1.CREATE IF EXSITS REPLACE TABLE transactions (
2.transactionId int,
3.transactionDate timestamp,
4.unitsSold int) - B. 1.CREATE OR REPLACE TABLE transactions (
2.transactionId int,
3.transactionDate timestamp,
4.unitsSold int) - C. 1.CREATE OR REPLACE TABLE IF EXISTS transactions (
2.transactionId int,
3.transactionDate timestamp,
4.unitsSold int)
5.FORMAT DELTA - D. 1.CREATE OR REPLACE DELTA TABLE transactions (
2.transactionId int,
3.transactionDate timestamp,
4.unitsSold int)
正解：B
解説：
Explanation
The answer is
1.CREATE OR REPLACE TABLE transactions (
2.transactionId int,
3.transactionDate timestamp,
4.unitsSold int)
When creating a table in Databricks by default the table is stored in DELTA format.
質問 # 17
Your team has hundreds of jobs running but it is difficult to track cost of each job run, you are asked to provide a recommendation on how to monitor and track cost across various workloads
- A. Use Tags, during job creation so cost can be easily tracked
- B. Use workspace admin reporting
- C. Use a single cluster for all the jobs, so cost can be easily tracked
- D. Use job logs to monitor and track the costs
- E. Create jobs in different workspaces, so we can track the cost easily
正解：A
解説：
Explanation
The answer is Use Tags, during job creation so cost can be easily tracked Review below link for more details
https://docs.databricks.com/administration-guide/account-settings/usage-detail-tags-aws.html Here is a view how tags get propagated from pools to clusters and clusters without pools, Diagram Description automatically generated
質問 # 18
You noticed that a team member started using an all-purpose cluster to develop a notebook and used the same all-purpose cluster to set up a job that can run every 30 mins so they can update un-derlying tables which are used in a dashboard. What would you recommend for reducing the overall cost of this approach?
- A. Change the cluster all-purpose to job cluster when scheduling the job
- B. Change the cluster mode from all-purpose to single-mode
- C. Enable auto termination after 30 mins
- D. Reduce the number of nodes and enable auto scale
- E. Reduce the size of the cluster
正解：A
解説：
Explanation
While using an all-purpose cluster is ok during development but anytime you don't need to interact with a notebook, especially for a scheduled job it is less expensive to use a job cluster. Using an all-purpose cluster can be twice as expensive as a job cluster.
Please note: The compute cost you pay the cloud provider for the same cluster type and size be-tween an all-purpose cluster and job cluster is the same the only difference is the DBU cost.
The total cost of cluster = Total cost of VM compute(Azure or AWS or GCP) + Cost per DBU The per DBU cost varies between all-purpose and Job Cluster Here is the recent cost estimate from AWS between Jobs Cluster and all-purpose Cluster, for jobs compute its
$0.15 cents per DBU v$0.55 cents per DBU for all-purpose
Graphical user interface Description automatically generated
How do I check how much the DBU cost for my cluster?
When you click on an exister cluster or when you look at the cluster details you will see this in the top right corner Graphical user interface, text, application, email Description automatically generated
質問 # 19
How VACCUM and OPTIMIZE commands can be used to manage the DELTA lake?
- A. VACCUM command can be used to delete empty/blank parquet files in a delta table. OPTIMIZE command can be used to update stale statistics on a delta table.
- B. VACCUM command can be used to compact small parquet files, and the OP-TIMZE command can be used to delete parquet files that are marked for dele-tion/unused.
- C. VACCUM command can be used to delete empty/blank parquet files in a delta table, OPTIMIZE command can be used to cache frequently delta tables for better perfor-mance.
- D. VACCUM command can be used to compress the parquet files to reduce the size of the table, OPTIMIZE command can be used to cache frequently delta tables for better performance.
- E. OPTIMIZE command can be used to compact small parquet files, and the VAC-CUM command can be used to delete parquet files that are marked for deletion/unused.
(Correct)
正解：E
解説：
Explanation
VACCUM:
You can remove files no longer referenced by a Delta table and are older than the retention thresh-old by running the vacuum command on the table. vacuum is not triggered automatically. The de-fault retention threshold for the files is 7 days. To change this behavior, see Configure data reten-tion for time travel.
OPTIMIZE:
Using OPTIMIZE you can compact data files on Delta Lake, this can improve the speed of read queries on the table. Too many small files can significantly degrade the performance of the query.
質問 # 20
You are asked to setup an AUTO LOADER to process the incoming data, this data arrives in JSON format and get dropped into cloud object storage and you are required to process the data as soon as it arrives in cloud storage, which of the following statements is correct
- A. AUTO LOADER needs to be converted to a Structured stream process
- B. AUTO LOADER is native to DELTA lake it cannot support external cloud object storage
- C. AUTO LOADER has to be triggered from an external process when the file arrives in the cloud storage
- D. AUTO LOADER can only process continuous data when stored in DELTA lake
- E. AUTO LOADER can support file notification method so it can process data as it arrives
正解：E
解説：
Explanation
Auto Loader supports two modes when ingesting new files from cloud object storage Directory listing: Auto Loader identifies new files by listing the input directory, and uses a directory polling approach.
File notification: Auto Loader can automatically set up a notification service and queue service that subscribe to file events from the input directory.
Diagram Description automatically generated
File notification is more efficient and can be used to process the data in real-time as data arrives in cloud object storage.
Choosing between file notification and directory listing modes | Databricks on AWS
質問 # 21
Which configuration parameter directly affects the size of a spark-partition upon ingestion of data into Spark?
- A. spark.sql.files.maxPartitionBytes
- B. spark.sql.autoBroadcastJoinThreshold
- C. spark.sql.adaptive.coalescePartitions.minPartitionNum
- D. spark.sql.adaptive.advisoryPartitionSizeInBytes
- E. spark.sql.files.openCostInBytes
正解：A
解説：
Explanation
This is the correct answer because spark.sql.files.maxPartitionBytes is a configuration parameter that directly affects the size of a spark-partition upon ingestion of data into Spark. This parameter configures the maximum number of bytes to pack into a single partition when reading files from file-based sources such as Parquet, JSON and ORC. The default value is 128 MB, which means each partition will be roughly 128 MB in size, unless there are too many small files or only one large file. Verified References: [Databricks Certified Data Engineer Professional], under "Spark Configuration" section; Databricks Documentation, under "Available Properties - spark.sql.files.maxPartitionBytes" section.
質問 # 22
A new data engineer has started at a company. The data engineer has recently been added to the company's
Databricks workspace as [email protected]. The data engineer needs to be able to query the table
sales in the database retail. The new data engineer already has been granted USAGE on the database retail.
Which of the following commands can be used to grant the appropriate permissions to the new data engineer?
- A. GRANT CREATE ON TABLE sales TO [email protected];
- B. GRANT USAGE ON TABLE [email protected] TO sales;
- C. GRANT USAGE ON TABLE sales TO [email protected];
- D. GRANT SELECT ON TABLE sales TO [email protected];
- E. GRANT SELECT ON TABLE [email protected] TO sales;
正解：D
質問 # 23
What is the purpose of the bronze layer in a Multi-hop architecture?
- A. Perform data quality checks, corrupt data quarantined
- B. Used as a data source for Machine learning applications.
- C. Provides efficient storage and querying of full unprocessed history of data
- D. Can be used to eliminate duplicate records
- E. Contains aggregated data that is to be consumed into Silver
正解：C
解説：
Explanation
The answer is Provides efficient storage and querying of full unprocessed history of data Medallion Architecture - Databricks Bronze Layer:
1.Raw copy of ingested data
2.Replaces traditional data lake
3.Provides efficient storage and querying of full, unprocessed history of data
4.No schema is applied at this layer
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
質問 # 24
You are working on a marketing team request to identify customers with the same information between two tables CUSTOMERS_2021 and CUSTOMERS_2020 each table contains 25 columns with the same schema, You are looking to identify rows that match between two tables across all columns, which of the following can be used to perform in SQL
- A. 1.SELECT * FROM CUSTOMERS_2021
2.EXCEPT
3.SELECT * FROM CUSTOMERS_2020 - B. 1.SELECT * FROM CUSTOMERS_2021
2. UNION
3.SELECT * FROM CUSTOMERS_2020 - C. 1.SELECT * FROM CUSTOMERS_2021
2. UNION ALL
3.SELECT * FROM CUSTOMERS_2020 - D. 1.SELECT * FROM CUSTOMERS_2021
2. INTERSECT
3.SELECT * FROM CUSTOMERS_2020 - E. 1.SELECT * FROM CUSTOMERS_2021 C1
2.INNER JOIN CUSTOMERS_2020 C2
3.ON C1.CUSTOMER_ID = C2.CUSTOMER_ID
正解：D
解説：
Explanation
Answer is,
1.SELECT * FROM CUSTOMERS_2021
2. INTERSECT
3.SELECT * FROM CUSTOMERS_2020
To compare all the rows between both the tables across all the columns using intersect will help us achieve that, an inner join is only going to check if the same column value exists across both the tables on a single column.
INTERSECT [ALL | DISTINCT]
*Returns the set of rows which are in both subqueries.
If ALL is specified a row that appears multiple times in the subquery1 as well as in subquery will be returned multiple times.
If DISTINCT is specified the result does not contain duplicate rows. This is the default.
質問 # 25
Delete records from the transactions Delta table where transactionDate is greater than current timestamp?
- A. DELETE FROM transactions if transctionDate > current_timestamp()
- B. DELETE FROM transactions where transactionDate > current_timestamp()
- C. DELET FROM transactions where transactionDate GE current_timestamp()
- D. DELETE FROM transactions FORMAT DELTA where transactionDate > cur-renct_timestmap()
- E. DELETE FROM transactions where transactionDate > current_timestamp() KEEP_HISTORY
正解：B
質問 # 26
An upstream source writes Parquet data as hourly batches to directories named with the current date. A nightly batch job runs the following code to ingest all data from the previous day as indicated by thedatevariable:
Assume that the fieldscustomer_idandorder_idserve as a composite key to uniquely identify each order.
If the upstream system is known to occasionally produce duplicate entries for a single order hours apart, which statement is correct?
- A. Each write to the orders table willrun deduplication over the union of new and existing records, ensuringno duplicate records are present.
- B. Each write to the orders table willonly contain unique records, but newly written records may have duplicates already present in the target table.
- C. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, the operation will tail.
- D. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, these records will be overwritten.
- E. Each write to the orders table willonly contain unique records, and only those records without duplicates in the target table will be written.
正解：B
解説：
Explanation
This is the correct answer because the code uses the dropDuplicates method to remove any duplicate records within each batch of data before writing to the orders table. However, this method does not check for duplicates across different batches or in the target table, so it is possible that newly written records may have duplicates already present in the target table. To avoid this, a better approach would be to use Delta Lake and perform an upsert operation usingmergeInto. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "DROP DUPLICATES" section.
質問 # 27
A data engineer has created a Delta table as part of a data pipeline. Downstream data analysts now need
SELECT permission on the Delta table.
Assuming the data engineer is the Delta table owner, which part of the Databricks Lakehouse Plat-form can
the data engineer use to grant the data analysts the appropriate access?
- A. Repos
- B. Jobs
B Dashboards - C. Databricks Filesystem
- D. Data Explorer
正解：A
質問 # 28
Your colleague was walking you through how a job was setup, but you noticed a warning message that said,
"Jobs running on all-purpose cluster are considered all purpose compute", the colleague was not sure why he was getting the warning message, how do you best explain this warning mes-sage?
- A. All-purpose cluster provide interactive messages that can not be viewed in a job
- B. All-purpose clusters take longer to start the cluster vs a job cluster
- C. All-purpose clusters are more expensive than the job clusters
- D. All-purpose clusters cannot be used for Job clusters, due to performance issues.
- E. All-purpose clusters are less expensive than the job clusters
正解：C
解説：
Explanation
Warning message:
Graphical user interface, text, application, email Description automatically generated
Pricing for All-purpose clusters are more expensive than the job clusters AWS pricing(Aug 15th 2022)Graphical user interface Description automatically generated
Bottom of Form
Top of Form
質問 # 29
You are working on IOT data where each device has 5 reading in an array collected in Celsius, you were asked to covert each individual reading from Celsius to Fahrenheit, fill in the blank with an appropriate function that can be used in this scenario.
Schema: deviceId INT, deviceTemp ARRAY<double>
SELECT deviceId, __(deviceTempC,i-> (i * 9/5) + 32) as deviceTempF
FROM sensors
- A. MULTIPLY
- B. FORALL
- C. ARRAYEXPR
- D. APPLY
- E. TRANSFORM
正解：E
解説：
Explanation
TRANSFORM -> Transforms elements in an array in expr using the function func.
1.transform(expr, func)
質問 # 30
A team member is leaving the team and he/she is currently the owner of the few tables, instead of transfering the ownership to a user you have decided to transfer the ownership to a group so in the future anyone in the group can manage the permissions rather than a single individual, which of the following commands help you accomplish this?
- A. TRANSFER OWNER table_name to 'group'
- B. GRANT OWNER On table_name to 'group'
- C. GRANT OWNER table_name to 'group'*
- D. ALTER TABLE table_name OWNER to 'group'
- E. ALTER OWNER ON table_name to 'group'
正解：D
解説：
Explanation
The answer is ALTER TABLE table_name OWNER to 'group'
Assign owner to object
質問 # 31
You are working on a dashboard that takes a long time to load in the browser, due to the fact that each visualization contains a lot of data to populate, which of the following approaches can be taken to address this issue?
- A. Increase size of the SQL endpoint cluster
- B. Use Delta cache to store the intermediate results
- C. Increase the scale of maximum range of SQL endpoint cluster
- D. Remove data from Delta Lake
- E. Use Databricks SQL Query filter to limit the amount of data in each visualization
正解：E
解説：
Explanation
Note*: The question may sound misleading but these are types of questions the exam tries to ask.
A query filter lets you interactively reduce the amount of data shown in a visualization, similar to query parameter but with a few key differences. A query filter limits data after it has been loaded into your browser.
This makes filters ideal for smaller datasets and environments where query executions are time-consuming, rate-limited, or costly.
This query filter is different from than filter that needs to be applied at the data level, this filter is at the visualization level so you can toggle how much data you want to see.
1.SELECT action AS `action::filter`, COUNT(0) AS "actions count"
2.FROM events
3.GROUP BY action
When queries have filters you can also apply filters at the dashboard level. Select the Use Dash-board Level Filters checkbox to apply the filter to all queries.
Dashboard filters
Query filters | Databricks on AWS
質問 # 32
A data engineer has ingested a JSON file into a table raw_table with the following schema:
1.transaction_id STRING,
2.payload ARRAY<customer_id:STRING, date:TIMESTAMP, store_id:STRING>
The data engineer wants to efficiently extract the date of each transaction into a table with the fol-lowing
schema:
1.transaction_id STRING,
2.date TIMESTAMP
Which of the following commands should the data engineer run to complete this task?
- A. 1.SELECT transaction_id, date from payload
2.FROM raw_table; - B. 1.SELECT transaction_id, date
2.FROM raw_table; - C. 1.SELECT transaction_id, payload[date]
2.FROM raw_table; - D. 1.SELECT transaction_id, explode(payload)
2.FROM raw_table; - E. 1.SELECT transaction_id, payload.date
2.FROM raw_table;
正解：E
質問 # 33
A data engineer is overwriting data in a table by deleting the table and recreating the table. Another data
engineer suggests that this is inefficient and the table should simply be overwritten instead.
Which of the following reasons to overwrite the table instead of deleting and recreating the table is incorrect?
- A. Overwriting a table results in a clean table history for logging and audit purposes
- B. Overwriting a table maintains the old version of the table for Time Travel
- C. Overwriting a table is an atomic operation and will not leave the table in an unfinished state
- D. Overwriting a table allows for concurrent queries to be completed while in progress
- E. Overwriting a table is efficient because no files need to be deleted
正解：A
質問 # 34
The security team is exploring whether or not the Databricks secrets module can be leveraged for connecting to an external database.
After testing the code with all Python variables being defined with strings, they upload the password to the secrets module and configure the correct permissions for the currently active user. They then modify their code to the following (leaving all other variables unchanged).
Which statement describes what will happen when the above code is executed?
- A. The connection to the external table will fail; the string "redacted" will be printed.
- B. The connection to the external table will succeed; the string "redacted" will be printed.
- C. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the encoded password will be saved to DBFS.
- D. The connection to the external table will succeed; the string value of password will be printed in plain text.
- E. An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the password will be printed in plain text.
正解：B
解説：
Explanation
This is the correct answer because the code is using the dbutils.secrets.get method to retrieve the password from the secrets module and store it in a variable. The secrets module allows users to securely store and access sensitive information such as passwords, tokens, or API keys. The connection to the external table will succeed because the password variable will contain the actual password value. However, when printing the password variable, the string "redacted" will be displayed instead of the plain text password, as a security measure to prevent exposing sensitive information in notebooks. Verified References: [Databricks Certified Data Engineer Professional], under "Security & Governance" section; Databricks Documentation, under
"Secrets" section.
質問 # 35
The marketing team is launching a new campaign to monitor the performance of the new campaign for the first two weeks, they would like to set up a dashboard with a refresh schedule to run every 5 minutes, which of the below steps can be taken to reduce of the cost of this refresh over time?
- A. Change the spot instance policy from reliability optimized to cost optimized
- B. Reduce the size of the SQL Cluster size
- C. Always use X-small cluster
- D. Setup the dashboard refresh schedule to end in two weeks
- E. Reduce the max size of auto scaling from 10 to 5
正解：D
解説：
Explanation
The answer is Setup the dashboard refresh schedule to end in two weeks
質問 # 36
Which of the following statements can successfully read the notebook widget and pass the python variable to a SQL statement in a Python notebook cell?
- A. 1.order_date = dbutils.widgets.get("widget_order_date")
2.
3.spark.sql("SELECT * FROM sales WHERE orderDate = order_date") - B. 1.order_date = dbutils.widgets.get("widget_order_date")
2.
3.spark.sql(f"SELECT * FROM sales WHERE orderDate = 'order_date' ") - C. 1.order_date = dbutils.widgets.get("widget_order_date")
2.
3.spark.sql(f"SELECT * FROM sales WHERE orderDate = 'f{order_date }'") - D. 1.order_date = dbutils.widgets.get("widget_order_date")
2.
3.spark.sql(f"SELECT * FROM sales WHERE orderDate = '${order_date }' ") - E. 1.order_date = dbutils.widgets.get("widget_order_date")
2.
3.spark.sql(f"SELECT * FROM sales WHERE orderDate = '{order_date}' ")
(Correct)
正解：E
質問 # 37
You noticed that colleague is manually copying the notebook with _bkp to store the previous ver-sions, which of the following feature would you recommend instead.
- A. Databricks notebooks should be copied to a local machine and setup source control locally to version the notebooks
- B. Databricks notebook can be exported as HTML and imported at a later time
- C. Databricks notebooks support change tracking and versioning
- D. Databricks notebooks can be exported into dbc archive files and stored in data lake
正解：C
解説：
Explanation
Answer is Databricks notebooks support automatic change tracking and versioning.
When you are editing the notebook on the right side check version history to view all the changes, every change you are making is captured and saved.
質問 # 38
......
試験解答Databricks-Certified-Professional-Data-Engineer最新版とテストエンジン：https://www.passtest.jp/Databricks/Databricks-Certified-Professional-Data-Engineer-shiken.html
合格させるDatabricks-Certified-Professional-Data-Engineer試験最新のDatabricks-Certified-Professional-Data-Engineer試験問題集PDF：https://drive.google.com/open?id=1IsOzvjaWuj49l5UyUz-6w5e4UjqjWnWZ