The data cloud industry is facing a pivotal moment as it grapples with the challenges of standardizing data governance across various compute engines.
With the rise of open table formats, such as Apache Iceberg, companies are striving to balance technical metadata management with strong security measures. Snowflake Inc. and Databricks Inc. are at the forefront of this battle, each developing unique strategies to enhance data governance and interoperability.
“Snowflake actually provides this Polaris Catalog, which is a technical metadata catalog,” said Sanjeev Mohan (pictured, right), principal at SanjMo. “If you want to do [role-based access control], row-level security, column-level security … you need Horizon. If you don’t have a Horizon, then every single compute engine, like if it’s Spark or Dremio or Trino or Starburst, they have to figure out how to apply data access governance onto Iceberg. That’s why Horizon is so important.”
Mohan, along with George Gilbert (left), senior analyst at theCUBE Research, spoke with …