Approximation of functions

$$ \newcommand{\half}{\frac{1}{2}} \newcommand{\halfi}{{1/2}} \newcommand{\tp}{\thinspace .} \newcommand{\x}{\boldsymbol{x}} \newcommand{\X}{\boldsymbol{X}} \renewcommand{\u}{\boldsymbol{u}} \renewcommand{\v}{\boldsymbol{v}} \newcommand{\w}{\boldsymbol{w}} \newcommand{\e}{\boldsymbol{e}} \newcommand{\f}{\boldsymbol{f}} \newcommand{\Ix}{\mathcal{I}_x} \newcommand{\Iy}{\mathcal{I}_y} \newcommand{\Iz}{\mathcal{I}_z} \newcommand{\If}{\mathcal{I}_s} % for FEM \newcommand{\Ifd}{{I_d}} % for FEM \newcommand{\sequencei}[1]{\left\{ {#1}_i \right\}_{i\in\If}} \newcommand{\sequencej}[1]{\left\{ {#1}_j \right\}_{j\in\If}} \newcommand{\basphi}{\varphi} \newcommand{\baspsi}{\psi} \newcommand{\refphi}{\tilde\basphi} \newcommand{\psib}{\boldsymbol{\psi}} \newcommand{\xno}[1]{x_{#1}} \newcommand{\Xno}[1]{X_{(#1)}} \newcommand{\xdno}[1]{\boldsymbol{x}_{#1}} \newcommand{\dX}{\, \mathrm{d}X} \newcommand{\dx}{\, \mathrm{d}x} \newcommand{\Real}{\mathbb{R}} \newcommand{\Integer}{\mathbb{Z}} $$

Many successful numerical solution methods for differential equations, including the finite element method, aim at approximating the unknown function by a sum $$ \begin{equation} u(x) \approx \sum_{i=0}^N c_i\baspsi_i(x), \tag{1} \end{equation} $$ where $ \baspsi_i(x) $ are prescribed functions and $ c_0,\ldots,c_N $ are unknown coefficients to be determined. Solution methods for differential equations utilizing (1) must have a principle for constructing $ N+1 $ equations to determine $ c_0,\ldots,c_N $. Then there is a machinery regarding the actual constructions of the equations for $ c_0,\ldots,c_N $, in a particular problem. Finally, there is a solve phase for computing the solution $ c_0,\ldots,c_N $ of the $ N+1 $ equations.

Especially in the finite element method, the machinery for constructing the discrete equations to be implemented on a computer is quite comprehensive, with many mathematical and implementational details entering the scene at the same time. From an ease-of-learning perspective it can therefore be wise to follow an idea of Larson and Bengzon [1] and introduce the computational machinery for a trivial equation: $ u=f $. Solving this equation with $ f $ given and $ u $ on the form (1), means that we seek an approximation $ u $ to $ f $. This approximation problem has the advantage of introducing most of the finite element toolbox, but without involving demanding topics related to differential equations (e.g., integration by parts, boundary conditions, and coordinate mappings). This is the reason why we shall first become familiar with finite element approximation before addressing finite element methods for differential equations.

First, we refresh some linear algebra concepts about approximating vectors in vector spaces. Second, we extend these concepts to approximating functions in function spaces, using the same principles and the same notation. We present examples on approximating functions by global basis functions with support throughout the entire domain. That is, the functions are in general nonzero on the entire domain. Third, we introduce the finite element type of basis functions with local support, meaning that each function is nonzero except in a small part of the domain. We explain all details of the computational algorithms involving such functions. Four types of approximation principles are covered: 1) the least squares method, 2) the $ L_2 $ projection or Galerkin method, 3) interpolation or collocation, and 4) the regression method.