199 private links
All the most-impactful projects in the data engineering world are now written in Rust. More than the classics fast, safe and high-level language and C interop. The author provides highlight on:
The TL;DR is that most organizations don't even have that much [big] data, a few hundreds GB to single-digit TB for the 98%
One server is enough. [The industry] is instead focusing on simpler solutions.
Rust provides reusability! "For example, Arroyo, Ballista, delta-rs, InfluxDB and many more all use the Apache DataFusion query engine."
This is a massive piece of engineering that they don't have to re-invent themselves and can instead benefit from a software package built and continuously improved by many organizations that have a vested and shared interest in making it great.