Like many, I'm leaving Reddit after 15+ years there. I'm new to Lemmy and have no idea what I'm doing yet!
I'll miss the DE community that had started to gain steam on Reddit, but let's see if we can't recreate that success here.
Come say hey, and lets start talking about data! :)
This was announced by Databricks at their Data & AI summit.
It's interesting to see these older data warehouses starting to implement features that newer projects have proven over the past few years.
This method of incremental materialisation for streaming ingestion is the default in ClickHouse, and there's all sorts of so-called "streaming databases" that have popped up that exclusively do this e.g. Materialize, Rising Wave, etc.
I don't know what Databricks' implementation is like, I'm not a customer of theirs, so I'm interesting to see how successful it is. I have been using ClickHouse for some time now and it's a super powerful feature to get away from batch & schedules.