Apache Doris as a Unified Data Platform Engine

About this session

Real-time Pricing Engine — we migrated a legacy system of 20+ SQL Server stored procedures (with linked servers and custom change tracking) through a failed 160K-line Spark attempt, to a clean solution: 31 Kafka CDC sink connectors + cascading. Materialized views that compute final prices for 10K SKUs × 100+ stores in ~5 minutes. 200 lines of SQL replaced 160,000 lines of Scala. Streaming CDC Sink — 33 Kafka Connect sink connectors streaming real-time operational data from PostgreSQL, SQL Server, and MongoDB into Doris with exactly-once delivery and automatic schema evolution. Query Analytics Platform — Doris reads our Iceberg lakehouse via external catalog, rebuilds pre-computed aggregations, and powers executive dashboards showing query costs, cluster utilization, and platform health. High-Concurrency Point Queries — we used Doris as a cache layer with row store and PreparedStatement short-circuit optimization. It worked — but I'll honestly share why I eventually moved this to PostgreSQL, and what I learned. Metadata Catalog with Search — we're building a DataHub replacement entirely on Doris: NGram BloomFilter indexes for substring search, inverted indexes for keyword search, and IVF vector indexes for multilingual semantic search. No Elasticsearch, no Neo4j — just Doris.

Speaker

Vitalii Kalmykov

Data Platform Lead, Silpo · Fozzy Group

Vitalii Kalmykov is a data architect with 25 years in data, leading the data platform team at Fozzy Group — one of Ukraine's largest retail groups. Over the past four years he architected the group's unified data platform and personally built most of it, hands-on, alongside a team of five engineers — Flink, Spark, Kafka, Iceberg, Trino, Temporal, Airflow, and Apache Doris, plus every integration between them. Today that platform runs 10,000+ Iceberg tables and powers roughly 10,000 dbt pipelines built by 70+ analysts — and Doris has grown from an OLAP engine into a multi-role workhorse for analytics, real-time pricing, streaming CDC, and catalog search.

LinkedIn ↗ · Vitalii Kalmykov