A novel approach to solve hyperbolic Buckley-Leverett equation by using a transformer based physics informed neural network

Solving hyperbolic partial differential equation is a challenging task due to the non-linear feature that requires to capture the shock wave. Numerical solution relies on discretization of both spatial and temporal domain, and iterative approach like Newton's method is involved and time step size is crucial for stability and convergence in the presence of non-linearity. Physics-informed neural networks (PINNs) offer a new and versatile approach for solving partial different equations by minimizing the residual of governing equations and approaching to the initial and boundary conditions. Currently, most PINNs are built based on a simple fully connected neural network which exhibits some limitations to model complex non-linear partial differential equations. In this paper, a novel method is developed to combine Transformer model and PINNs approach (Tr-PINN) to solving a hyperbolic partial differential equation directly without any prior knowledge. Tr-PINN method is based on a series of Transformer blocks where self-attention mechanism is used to capture the non-linearity features of the solution. Unlike most PINNs models generate inputs with spatial and temporal vectors only, Tr-PINN introduces the non-linearity term mobility ratio as additional input vector. The method is tested on a classical hyperbolic problem, called Buckley-Leverett equation with non-convex flux function. We found that the Tr-PINN method can capture the water shock front effectively and provide a general solution for Buckley-Leverett equation under various mobility ratio conditions.