Перейти к содержимому

Apache Doris as a Unified Data Platform Engine

VeloDB

0:00 / 0:00

Apache Doris as a Unified Data Platform Engine

15 просмотров · 15 ч назад
VeloDB
569 подписчиков
15 просмотров · 15 ч назад
This session was part of Apache Doris Summit 2026. Abstract: Real-time Pricing Engine — we migrated a legacy system of 20+ SQL Server stored procedures (with linked servers and custom change tracking) through a failed 160K-line Spark attempt, to a clean solution: 31 Kafka CDC sink connectors + cascading. Materialized views that compute final prices for 10K SKUs × 100+ stores in ~5 minutes. 200 lines of SQL replaced 160,000 lines of Scala. Streaming CDC Sink — 33 Kafka Connect sink connectors streaming real-time operational data from PostgreSQL, SQL Server, and MongoDB into Doris with exactly-once delivery and automatic schema evolution. Query Analytics Platform — Doris reads our Iceberg lakehouse via external catalog, rebuilds pre-computed aggregations, and powers executive dashboards showing query costs, cluster utilization, and platform health. High-Concurrency Point Queries — we used Doris as a cache layer with row store and PreparedStatement short-circuit optimization. It worked — but I'll honestly share why I eventually moved this to PostgreSQL, and what I learned. Metadata Catalog with Search — we're building a DataHub replacement entirely on Doris: NGram BloomFilter indexes for substring search, inverted indexes for keyword search, and IVF vector indexes for multilingual semantic search. No Elasticsearch, no Neo4j — just Doris. #apachedoris