The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

This work is devoted to a review of our recent studies in the modeling and computation of nano optical devices. Motivated by technological advances at nano scale, to quantitatively understand the mechanism and improve the designing, we make an effort to model nano optical systems involving multiple physical processes across different time and space scales, and develop multiscale and adaptive numerical methods for simulation. Challenges on rigorous analysis of the models and algorithms are also discussed.

In [23], we, together with our collaborator, proposed a family of $H$(curl$^2$)- conforming elements on both triangular and rectangular meshes. The elements provide a brand new method to solve the quad-curl problem in 2 dimensions. In this paper, we turn our focus to 3 dimensions and construct $H$(curl$^2$)-conforming finite elements on tetrahedral meshes. The newly proposed elements have been proved to have the optimal interpolation error estimate. Having the tetrahedral elements, we can solve the quad-curl problem in any Lipschitz domain by the conforming finite element method. We also provide several numerical examples of using our elements to solve the quad-curl problem. The results of the numerical experiments show the correctness of our elements.

We present a new numerical method for solving the elliptic homogenization problem. The main idea is that the missing effective matrix is reconstructed by solving the local least-squares in an offline stage, which shall be served as the input data for the online computation. The accuracy of the proposed method is analyzed with the aid of the refined estimates of the reconstruction operator. Two dimensional and three dimensional numerical tests confirm the efficiency of the proposed method, and illustrate that this online-offline strategy may significantly reduce the cost without loss of the accuracy.

Variational source conditions are known to be a versatile tool for establishing error bounds, and these recently attract much attention. We establish sufficient conditions for general spectral regularization methods which yield convergence rates under variational source conditions. Specifically, we revisit the asymptotical regularization, Runge-Kutta integrators, and verify that these methods satisfy the proposed conditions. Numerical examples confirm the theoretical findings.

Recently, there are numerous works on developing surrogate models under the idea of deep learning. Many existing approaches use high fidelity input and solution labels for training. However, it is usually difficult to acquire sufficient high fidelity data in practice. In this work, we propose a network which can utilize computational cheap low-fidelity data together with limited high-fidelity data to train surrogate models, where the multi-fidelity data are generated from multiple underlying models. The network takes a context set as input (physical observation points, low fidelity solution at observed points) and output (high fidelity solution at observed points) pairs. It uses the neural process to learn a distribution over functions conditioned on context sets and provide the mean and standard deviation at target sets. Moreover, the proposed framework also takes into account the available physical laws that govern the data and imposes them as constraints in the loss function. The multi-fidelity physics-constrained network (MFPC-Net) (1) takes datasets obtained from multiple models at the same time in the training, (2) takes advantage of the available physical information, (3) learns a stochastic process which can encode prior beliefs about the correlation between two fidelity with a few observations, and (4) produces predictions with uncertainty. The ability of representing a class of functions is ensured by the property of neural process and is achieved by the global latent variables in the neural network. Physical constraints are added to the loss using Lagrange multipliers. An algorithm to optimize the loss function is proposed to effectively train the parameters in the network on an ad hoc basis. Once trained, one can obtain fast evaluations of the entire domain of interest given a few observation points from a new low- and high-fidelity model pair. Particularly, one can further identify the unknown parameters such as permeability fields in elliptic PDEs with a simple modification of the network. Several numerical examples for both forward and inverse problems are presented to demonstrate the performance of the proposed method.

We are concerned with the inverse problem of recovering a conductive medium body. The conductive medium body arises in several applications of practical importance, including the modelling of an electromagnetic object coated with a thin layer of a highly conducting material and the magnetotellurics in geophysics. We consider the determination of the material parameters inside the body as well as on the conductive interface by the associated electromagnetic far-field measurement. Under the transverse-magnetic polarisation, we derive two novel unique identifiability results in determining a 2D piecewise conductive medium body associated with a polygonal-nest or a polygonal-cell geometry by a single active or passive far-field measurement.

Optimization on a unit sphere finds crucial applications in science and engineering. However, derivatives of the objective function may be difficult to compute or corrupted by noises, or even not available in many applications. Hence, we propose a Derivative-Free Geometric Algorithm (DFGA) which, to the best of our knowledge, is the first derivative-free algorithm that takes trust region framework and explores the spherical geometry to solve the optimization problem with a spherical constraint. Nice geometry of the spherical surface allows us to pursue the optimization at each iteration in a local tangent space of the sphere. Particularly, by applying Householder and Cayley transformations, DFGA builds a quadratic trust region model on the local tangent space such that the local optimization can essentially be treated as an unconstrained optimization. Under mild assumptions, we show that there exists a subsequence of the iterates generated by DFGA converging to a stationary point of this spherical optimization. Furthermore, under the Łojasiewicz property, we show that all the iterates generated by DFGA will converge with at least a linear or sublinear convergence rate. Our numerical experiments on solving the spherical location problems, subspace clustering and image segmentation problems resulted from hypergraph partitioning, indicate DFGA is very robust and efficient for solving optimization on a sphere without using derivatives.