A Convex Quasilinearization Method for Solving Nonlinear PDEs with Physics-Informed Neural Networks

Problem
This work addresses the limitations of standard physics-informed neural networks (PINNs) in solving nonlinear partial differential equations (PDEs), particularly the nonconvex nature of gradient-based training that can hinder convergence and efficiency. The authors propose a novel method, LiL-Q, which leverages Bellman-Kalaba quasilinearization to transform the nonlinear problem into a series of linear subproblems. This approach is particularly relevant as it provides a solution framework that is not only efficient but also avoids the pitfalls of traditional optimization methods. The paper is a preprint and has not undergone peer review.

Method
The core technical contribution is the introduction of the Linear-in-Learnables (LiL) trial space, which consists of representations with trainable parameters that enter linearly. This includes random-feature extreme learning machines, spectral polynomial bases, and trigonometric expansions, all implemented as physics-informed neural networks. The method employs a direct linear least-squares QR factorization to solve the linear subproblems generated by the quasilinearization process. The authors establish local Newton-Kantorovich convergence for the outer iteration, which is governed by the best-approximation residual of the trial space rather than an optimization tolerance, thus ensuring robust convergence properties.

Results
LiL-Q is evaluated on seven benchmark problems, including scalar nonlinear PDEs (Bratu, viscous Burgers, Buckley-Leverett), coupled systems (plane-strain elasticity, incompressible Navier-Stokes equations in 2D and 3D), and steady-state Darcy flow with heterogeneous permeability. The method demonstrates rapid convergence, typically achieving results in single-digit outer iterations, even with coarse basis sizes and irrespective of the number of parameters. Notably, when the exact solution is within the span of the trial space, LiL-Q recovers it to machine precision in a single solve. On the Navier-Stokes benchmarks, LiL-Q matches or surpasses existing PINN solvers while utilizing up to two orders of magnitude fewer trainable parameters, showcasing significant efficiency improvements.

Limitations
The authors acknowledge that the method’s performance is contingent on the trial space’s ability to approximate the exact solution. If the exact solution lies outside the span of the trial space, convergence may not be guaranteed. Additionally, while the method shows promise across various benchmarks, its applicability to more complex or higher-dimensional PDEs remains to be fully explored. The authors do not discuss potential computational overhead associated with the QR factorization in high-dimensional settings.

Why it matters
The LiL-Q method represents a significant advancement in the numerical solution of nonlinear PDEs, particularly in the context of physics-informed neural networks. By circumventing the challenges of nonconvex optimization, this approach opens new avenues for efficient and accurate modeling of complex physical systems. The implications of this work extend to various fields, including fluid dynamics and material science, where accurate PDE solutions are critical. This research contributes to the growing body of literature on enhancing the capabilities of neural networks in scientific computing, as published in arXiv.