End The Data Engineering Nightmare with MetricsIt’s a nightmare when everyone, from analysts to data engineers, must remember that “sales” means transaction types 1, 4, and 5, minus…Jun 9Jun 9
Unity Catalog to Azure Key Vault: No more dbutils.secrets()In this article, I’ll walk you through granting Azure Databricks secure access to Azure Key Vault with an Azure Databricks Access…Jun 5A response icon3Jun 5A response icon3
MERGE in Python, which one has the nicest syntax and is the fastest?If you are not yet a member of Medium, you can access for free via a friend’s link.May 31A response icon1May 31A response icon1
MERGE in databricks — a few new improvements about which you need to knowRecently, we made a few improvements to the MERGE command, which you can not miss as it is one of the most popular commands for every data…May 29A response icon2May 29A response icon2
Stored Procedures in databricksDatabricks Runtime 17.0 introduced support for SQL stored procedures. For example, the procedure below moves orders to demonstrate that…May 27May 27
Databricks Asset Bundles with UI in the workspaceDatabricks Asset Bundles became a best practice for CI/CD and are now available in the workspace UI.May 21A response icon1May 21A response icon1
Collation Modifiers in DatabricksMake case-insensitive or accent-insensitive queries easier and improve performance.May 17A response icon1May 17A response icon1
Clean Landing Zone — autoloader cleanSourceOne of the most annoying things for me is data duplication. In particular, bronze and landing zones keep the same data in the medallion…May 14A response icon2May 14A response icon2
Writing Compound SQL Code with BEGIN ... END in DatabricksBEGIN … END will make migration from legacy SQL easier, but I also found it excellent for handling errors, including custom errors…May 7A response icon2May 7A response icon2
SQL Scripting in databricks: CASE statementDatabricks Runtime 16.3 introduced SQL scripting. I decided to examine all the new statements one by one—first, CASE. I tested the code…May 5May 5
