Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism

Physics-Informed Neural Networks (PINNs) have emerged recently as a promising application of deep neural networks to the numerical solution of nonlinear partial differential equations (PDEs). However, it has been observed that the original PINN algorithm can produce inaccuracies around sharp transitions in the solution, as well as display instability during training. This has prompted recent efforts in developing adaptive algorithms for PINNs. This paper introduces self-adaptive PINNs, a novel algorithm based on a simple soft attention mechanism that requires no extra hyperparameters. Self-adaptive PINNs are based on trainable weights that can automatically force the neural network to focus on difficult regions of the solution. We demonstrate the performance of the proposed self-adaptive PINN algorithm in the solution of the Allen-Cahn PDE, which displays sharp space and time transitions.