Imagining Agentic Architecture in an EnterpriseIn today’s fast-paced business environment, companies must constantly adapt and evolve. Digital transformation is not just a buzzword; it’s…Feb 26Feb 26
From Technology to Business Value: Rethinking Data Platforms with the Concept of Data ProductsIn the fast-paced world of technology, companies are increasingly adopting advanced data platforms to leverage the power of data. Whether…Feb 24Feb 24
Using Databricks Genie to Understand User Behavior and Drive AI InnovationIn the world of AI, businesses often jump on the Generative AI (Gen AI) bandwagon, sometimes without clear use cases, driven by the fear of…Feb 215Feb 215
Connecting Databricks Delta to Snowflake as Iceberg: Two Approaches — UniForm vs Delta DirectIntroductionFeb 20Feb 20
Google’s lakehouse offering, BigLake: A deep dive into various BigLake tablesIntroductionFeb 111Feb 111
Iceberg Catalog vs Delta Transaction Log: Understanding the DifferencesIn the evolving world of data engineering, choosing the right table format is crucial. Two widely discussed formats in the ecosystem are…Feb 9Feb 9
How to Achieve Network and Storage Isolation in Azure Databricks for Sensitive DataIn the world of data engineering, security is not just an option — it’s a necessity. As organizations increasingly move to the cloud for…Feb 91Feb 91
Agentic Systems and Databricks Genie: The Future of Autonomous Data ProcessingIntroduction: What Are Agentic Systems?Feb 94Feb 94
How Much Lock-In Is Too Much? The Real Truth About Avoiding Vendor Lock-In in Data PlatformsEveryone talks about avoiding lock-in, but let’s be real — there’s no such thing as zero lock-in.Feb 6Feb 6
Evolutionary Architecture for Data Platforms — A Guide to Future-Proofing Your Data SystemsIntroductionFeb 6Feb 6
Can Hive Metastore ride the wave of table formats and data modernization?Hive Metastore has remained an important aspect of the Hadoop ecosystem. Since the beginning, Hive Metastore held the promise of an…Sep 4, 2024Sep 4, 2024
Why Iceberg requires a Catalog for ACID while Delta doesn’tBoth Apache Iceberg and Delta Lake offer ACID guarantees for Parquet files, each using different mechanisms to achieve this. Data lakes on…Aug 18, 20241Aug 18, 20241
Moving Away from Age-Old ‘Partitioning’: A New Era of Data ManagementIn the world of big data, traditional partitioning has long been the cornerstone of efficient query performance. However, as data volumes…Aug 13, 2024Aug 13, 2024
Polaris vs. Unity Catalog: Clearing Up the ConfusionThe month of June 2024 has been particularly confusing for data and AI professionals, especially those trying to understand the evolving…Jun 30, 2024Jun 30, 2024
Published inAnalytics VidhyaPipeline JunglesThis word came to my mind after I had numerous workloads running in production and I couldn't guarantee the SLA of my machine learning…Apr 26, 2021Apr 26, 2021
