Image source 1
Celebrating
Excited to share that I’m now a Databricks Certified Data Engineer Associate.
Starting with almost ZERO experience with Databricks, I learned quite a bit. The exam covered five main topics and here are a few of the things I learned.
Topic 1: Databricks Lakehouse Platform
- How to use the Data Plane and the Control Plane to balance performance and cost
- How easy it is to collaborate with Databricks Notebooks
Topic 2: ELT with Spark SQL and Python
- How easy it is to switch between languages within the same notebook
- How much I love SQL. I felt at home learning Spark SQL since I have many years of experience using SQL.
Topic 3: Incremental Data Processing
- How to leverage ACID transactions and history
Topic 4: Production Pipelines
- How to troubleshoot quickly with the outstanding visuals for task execution history
- How to use native support for alerting
Topic 5: Data Governance
- How to capture user-level audit logs and data lineage with Unity Catalog
-
Image created by Heather Woods ↩