Exact ReLU realization of tensor-product refinement iterates

Problem
This paper addresses a gap in the literature regarding the exact realization of dyadic refinement operators in two dimensions, specifically for continuous piecewise linear functions. Prior work primarily focused on one-dimensional cases, leaving a lack of understanding in the two-dimensional context. The authors present a preprint that extends the exact realization theory for refinement cascades to a genuinely two-dimensional setting, which is crucial for applications in image processing and multiscale analysis.

Method
The core technical contribution involves the development of a framework for scalar dyadic refinement operators defined on ( \mathbb{R}^2 ) of the form ( (Vf)(x,y) = \sum_{(j,k) \in \mathbb{Z}^2} c_{j,k} f(2x-j, 2y-k) ), where only finitely many coefficients ( c_{j,k} ) are nonzero. The authors prove that for any compactly supported continuous piecewise linear seed function ( g: \mathbb{R}^2 \to \mathbb{R} ), the iterates ( V^n g ) can be realized exactly using ReLU networks of fixed width and depth ( O(n) ). The proof employs a one-dimensional exact loop-controller framework, extending it to two dimensions by transporting tensor-product residual dynamics across the product of two polygonal loops. The remaining seam ambiguity is resolved through a final readout and selector step, while the matrix cascade is managed by a fixed-depth recursive block. The authors also demonstrate how general compactly supported continuous piecewise linear seeds can be decomposed into a finite number of components, facilitating exact clamped gluing on the support window.

Results
The authors do not provide specific numerical results or comparisons against named baselines in the abstract. However, they assert that their method achieves exact realizations of dyadic refinement iterates, which is a significant theoretical advancement. The implications of this work suggest that the proposed framework can be effectively utilized in practical applications, although quantitative performance metrics are not disclosed.

Limitations
The authors acknowledge that their work is limited to compactly supported continuous piecewise linear functions, which may restrict the applicability of their results to more general function classes. Additionally, the complexity of the proposed ReLU networks, while fixed in depth, may still lead to high computational costs for large ( n ). The paper does not address potential scalability issues or the performance of the method on non-linear or higher-dimensional functions.

Why it matters
This work has significant implications for the field of multiscale analysis and image processing, as it provides a foundational framework for exact realizations of dyadic refinement operators in two dimensions. The extension of the loop-controller method to tensor-product cases opens avenues for further research in multivariate function approximation and refinement techniques. By establishing a theoretical basis for exact ReLU realizations, this paper paves the way for more efficient neural network architectures that can leverage dyadic refinement in practical applications.

Authors: Tsogtgerel Gantumur
Source: arXiv:2605.03917
URL: https://arxiv.org/abs/2605.03917v1