Uncertainty and Error in CFD Simulations

This page provides a classification of uncertainties and errors that cause CFD simulation results to differ from their true or exact values. This discussion not only applies to the CFD code, but other computer programs used in the analysis process such as CAD packages, grid generators, and flow visualizers.

Defining Uncertainty and Error

Uncertainty and Error are commonly used interchangeably in everyday language. Here we follow the definitions of the AIAA Guidlines:

Uncertainty is defined as:

"A potential deficiency in any phase or activity of the modeling process that is due to the lack of knowledge." (AIAA G-077-1998)

Error is defined as:

A recoqnizable deficiency in any phase or activity of modeling and simulation that is not due to lack of knowledge. (AIAA G-077-1998)

The key phrase differentiating the definitions of uncertainty and error is lack of knowledge.

The key word in the definition of uncertainty is potential, which indicates that deficiencies may or may not exist. Lack of knowledge has primarily to do with lack of knowledge about the physical processes that go into building the model. Sensitivity and uncertainty analyses can be used to better determine uncertainty. Uncertainty applies to describing deficiencies in turbulence modeling. There is a lot about turbulence modeling that is not understood. One approach for determining the level of uncertainty and it effect on one's analysis is to run a number of simulations with a variety of turbulence models and see how the modeling affects the results.

The definition for error implies that the deficiency is identifiable upon examination. Errors can also be classified as acknowledged or unacknowledged:

Acknowledged errors (examples include round-off error and discretization error) have procedures for identifying them and possibly removing them. Otherwise they can remain in the code with their error estimated and listed.

Unacknowledged errors (examples include computer programming errors or usage errors) have no set procedures for finding them and may continue within the code or simulation.

One can differentiate between local and global errors. Local errors refer to errors at a grid point or cell, whereas global errors refer to errors over the entire flow domain. We are interested here in the global error of the solution that accounts for the local error at each grid point but is more than just the sum of the local errors. Local errors are transported, advected, and diffused throughout the grid.

The definition of error presented here is different than that an experimentalist may use, which is "the difference between the measured value and the exact value". Experimentalist usually define uncertainty as "the estimate of error". These definitions are inadequate for computational simulations because the exact value is typically not known. Further these definitions link error with uncertainty. The defintions provided in the above paragraphs are more definite because they differentiate error and uncertainty according to what is known.

Classification of Errors

Here we provide a classification or taxonomy of error.

Acknowledged Error

1. Physical approximation error
• Physical modeling error
• Geometry modeling error
2. Computer round-off error
3. Iterative convergence error
4. Discretization error
• Spatial discretization error
• Temporal discretization error

Unacknowledged Error

1. Computer programming error
2. Usage error

Each of these types of errors are discussed below.

Physical Approximation Error

Physical modeling errors are those due to uncertainty in the formulation of the model and deliberate simplifications of the model. These errors deal with the continuum model only. Converting the model to discrete form for the code is discussed as part of discretization errors. Errors in the modeling of the fluids or solids problem are concerned with the choice of the governing equations which are solved and models for the fluid or solid properties. Further, the issue of providing a well-posed problem can contribute to modeling errors. Often modeling is required for turbulence quantities, transistion, and boundary conditions (bleed, time-varying flow, surface roughness). Mehta lists sources of uncertainty in physical models as 1) the phenomenon is not thoroughly understood; 2) parameters used in the model are known but with some degree of uncertainty; 3) appropriate models are simplified, thus introducing uncertainty; and 4) an experimental confirmation of the models is not possible or is incomplete. Even when a physical process is known to a high level of accuracy, a simplified model may be used within the CFD code for the convenience of a more efficient computation. Physical modeling errors are examined by performing validation studies that focus on certain models (i.e. inviscid flow, turbulent boundary layers, real-gas flows, etc...).

Computer Round-Off Error

Computer round-off errors develop with the representation of floating point numbers on the computer and the accuracy at which numbers are stored. With advanced computer resources, numbers are typically stored with 16, 32, or 64 bits. Round-off errors are not considered significant when compared with other errors. If computer round-off errors are suspected of being significant, one test is to run the code at a higher precision or on a computer known to store floating point numbers at a higher precision. One can attempt to iterate a coarse grid solution to a residual of machine zero; however, this may not be possible for more complex algorithms.

Iterative Convergence Error

The iterative convergence error exists because the iterative methods used in the simulation must have a stopping point eventually. The error scales to the variation in the solution at the completion of the simulations.

Discretization Errors

Discretization errors are those errors that occur from the representation of the governing flow equations and other physical models as algebraic expressions in a discrete domain of space (finite-difference, finite-volume, finite-element) and time. The discrete spatial domain is known as the grid or mesh. The temporal discreteness is manifested through the time step taken. Discretization error is also known as numerical error. A consistent numerical method will approach the continuum representation of the equations and zero discretization error as the number of grid points increases and the size of the grid spacing tends to zero. As the mesh is refined, the solution should become less sensitive to the grid spacing and approach the continuum solution. This is grid convergence. Such thinking also applies to the time step. The grid convergence study is a useful procedure for determining the level of discretization error existing in a CFD solution. "Ordered" discretization errors are those dependent on the grid size and vanish as the grid size approaches zero. These are the errors that are addressed by a grid convergence study. Further details can be found on the pages entitled Examining Spatial (Grid) Convergence and Examining Temporal Convergence.

The discretization error is of most concern to a CFD code user during an application. Discretization errors are of major concern because they are dependent on the quality of the grid; however, it is often difficult to precisely indicate the relationship between a quality grid and an accurate solution prior to beginning the simulation. The level of discretization error is dependent on grid quality. The grid should be generated with consideration of such things as resolution, density, aspect ratio, stretching, orthogonality, grid singularities, and zonal boundary interfaces.

The level of discretization error is dependent on the features of the flow as resolved by the grid. Errors may develop due to representation of discontinuities (shocks, slip surfaces, interfaces, ...) on a grid. Interpolation errors come about at zonal interfaces where the solution of one zone is approximated on the boundary of the other zone.

The truncation error is the difference between the partial differential equation (PDE) and the finite equation. The truncation error is a function of the grid quality and flow gradients. Dispersive error terms causes oscillations in the solution. One fix to this is adding artificial dissipation to decrease the size of the dispersive errors. Dissipation error terms cause a smoothing of gradients. However, a level of dissipation comparable to the actual physical viscosity may contaminate the solution. Boundary layers may thicken. The truncation error terms are those of the expansion which are not used in the discretized equation. If the order of the leading term of the truncation error is of second-order, it is known as a numerical viscosity (dimensions of length2 / time), which is dimensions of kinematic viscosity. A positive viscous term will indicate that errors will be damped whereas a negative viscous term will indicate that errors will grow (unstable).

Included in the discretization error are errors due to not properly converging the solution with respect to the iterations to the steady-state solution or within a time step. This is reffered to as iterative convergence.

Computer Programming Errors

Programming errors are "bugs" and mistakes made in programming or writing the code. They are the responsibility of the programmers. These type of errors are discovered by systematically performing verification studies of subprograms of the code and the entire code, reviewing the lines of code, and performing validation studies of the code. The programming errors should be removed from the code prior to release.

Usage Errors

Usage errors are due to the application of the code in a less-than-accurate or improper manner. Usage errors may actually show up as modeling and discretization errors. The user sets the models, grid, algorithm, and inputs used in a simulation, which then establishes the accuracy of the simulation. There may be blatant errors, such as attempting to compute a known turbulent flow with an assumption of inviscid flow. A converged solution may be obtained; however, the conclusions drawn from the simulation may be incorrect. The errors may not be as evident, such as proper choice of turbulence model parameters for separated flows with shocks. The potential for usage errors increases with an increased level of options available in a CFD code. Usage errors are minimized through proper training and the accumulation of experience.

The user may intentionally introduce modeling and discretization error as an attempt to expedite the simulation at the expense of accuracy. This may be proper in the conceptual stage of a design study where more general information is needed at less accuracy. Even in the later stages, there may not be proper computational resources to simulate at the proper grid density. One has to understand the level of accuracy accompanying the results.

Usage errors should be controlable through proper training and analysis.

Usage errors can exist in the CAD, grid generation, and post-processing software, in addition to the CFD code.