Apache Software Foundation

Assembly Line

TsFile: A Standard Format for IoT Time Series Data

📅 Date:

✍️ Author: Susan Hall

🔖 Topics: Data Architecture, IIoT, Open Source

🏢 Organizations: Apache Software Foundation, Tsinghua University

TsFile is a columnar storage file format designed for time series data, featuring advanced compression to minimize storage, high throughput of read and write, and deep integration with processing and analysis tools such as Apache projects Spark and Flink. TsFile is designed to support a “high ingestion rate up to tens of million data points per second and rare updates only for the correction of low-quality data; compact data packaging and deep compression for long-live historical data; traditional sequential and conditional query, complex exploratory query, signal processing, data mining and machine learning.”

TsFile is the underlying storage file format for the Apache IoTDB time-series database. IoTDB represents more than a decade of work at China’s Tsinghua University School of Software. It became a top-level project with the Apache Software Foundation in 2020.

Read more at The New Stack