FiLark: a streaming-first software framework for end-to-end exploration, annotation, and algorithm integration in distributed acoustic sensing

Problem
This paper addresses the limitations of conventional batch-oriented analysis frameworks in the context of Distributed Acoustic Sensing (DAS) systems, which generate continuous, ultra-high-channel-count data streams. The authors highlight the inadequacy of existing workflows for tasks such as interactive exploration, scalable event annotation, and real-time monitoring, particularly when these tasks rely on manually selected data segments and offline processing. This work is presented as a preprint and has not yet undergone peer review.

Method
The core technical contribution is the FiLark framework, which adopts a “streaming-first” approach across all components, including data access, signal processing, visualization, and monitoring. FiLark treats DAS sources, including continuous multi-file recordings, as a unified stream. Key features include:

• An OpenGL-based ring-buffer renderer that allows for interactive browsing and visualization of long-duration recordings while maintaining constant memory usage.
• An integrated annotation interface that enables event labeling directly within continuous data streams, facilitating the creation of machine-learning-ready labeled datasets without the need for offline preprocessing.
• A signal processing library that includes various operators (temporal, spatial, spectral, and decomposition-based) with both CPU and GPU-accelerated implementations via PyTorch. This library supports stateful chunked execution, ensuring continuity and semantic integrity across segment boundaries.
• A standardized monitor interface that integrates streaming detectors and learning-based models into the visualization workflow.

By leveraging a common streaming abstraction, FiLark allows for seamless transfer of interactive processing configurations to scalable production pipelines.

Results
The paper does not provide quantitative results or comparisons against specific baselines or benchmarks, focusing instead on the architectural and functional capabilities of the FiLark framework. The authors emphasize the framework’s ability to handle long-duration recordings and facilitate real-time processing, but specific performance metrics or effect sizes are not disclosed.

Limitations
The authors acknowledge that the framework is still in development and may require further optimization for specific use cases. They do not address potential scalability issues when dealing with extremely high data rates or the need for extensive computational resources in real-time applications. Additionally, the lack of empirical performance evaluations against existing frameworks limits the assessment of FiLark’s effectiveness.

Why it matters
FiLark has significant implications for the field of DAS and real-time data processing. By providing a unified streaming framework, it enables more efficient workflows for data exploration, annotation, and algorithm integration, which are critical for advancing machine learning applications in acoustic sensing. The ability to create reproducible datasets directly from continuous streams can enhance the development of robust models and facilitate real-time monitoring applications. This framework could serve as a foundation for future research and development in distributed sensing technologies and their applications.

Authors: Jintao Li, Weichang Li, Kai Tong, Xaingyu Guo
Source: arXiv:2605.20132v1
URL: https://arxiv.org/abs/2605.20132v1