I want to highlight some reporting in Runtime. Open storage formats have disrupted the rivalry between Snowflake and Databricks, creating a new battleground in the data industry. Enterprises are adopting open-source table formats, allowing data to be stored in low-cost services like AWS S3 and processed by various tools. Snowflake and Databricks have converged on open formats like Delta Lake and Iceberg, with Iceberg emerging as the leading choice. Microsoft and Google are betting on services that work with any format, potentially calming a standards war. The decoupling of storage from computing engines, driven by the rise of open formats, is reshaping the landscape. Security and governance remain key challenges in the adoption of open formats.
I’ve talked about data management as a service on the show, and this leads to the obvious statement. When selecting cloud services, consider those that offer support for multiple storage formats. This ensures future-proofing and provides the flexibility to adapt as industry standards evolve. For those not familiar, this is a good set of information to start with.