Deriving Equations for Atmospheric Refraction

On this page I explain how terrestrial atmospheric refraction is calculated from basic physical laws and how the equations and constants used in refraction equations are derived. The same principle can be applied to derive astronomical refraction, although this is not the topic of this page.

Atmospheric Refraction is the bending of light due to a gradient of the refractive index caused by a density gradient of the air. This light bending causes observed objects in the distance to be displaced (mainly up) or distorted und inverted (mirage). In survey it is important to know how much light gets bent to be able to correct the observed height of a distant object for Refraction effects.

Definition of the Refraction Coefficient

The density gradient along a light ray is difficult to measure in practice. Consequently, the path of a light ray can not be kown exactly. But as long as the air is not too turbulent, we can use an average gradient. In this case a light ray can be approximated by an arc with the radius r.

Refraction can be expressed by the Refraction Coefficient k, which is the ratio of the radius of the earth to the radius of a bent light ray:


per Definition

' =' 'Refraction Coefficient
' =' '6371 km = radius of the earth
' =' 'radius of the light ray curve

Note: The phenomenon of refraction is not dependent on the shape of the earth. We could define a refraction factor that is not related to the radius of the earth. In fact we could simply use the curvature κ = 1 / r of the light ray. We can always get this curvature from k by dividing all equations for k through R: κ = k / R.

If we know the pressure, temperature and temperature gradient at the observer and assume the same conditions along the line of sight, we can use the following simple equation to calculate the Refraction Coefficient [1]:

' =' 'Refraction Coefficient
' =' 'Air pressure at the observer in mBar or hPa or 1/100 Pa, Standard = 1013.25 mBar
' =' 'Temperature at the observer in Kelvin, Standard = 288.15 K = 15°C
' =' 'Temperature Gradient at the observer in K/m or °C/m, Standard = −0.0065°C/m

How this equation is derived from the refractive index gradient of the air will be shown on this page.

Correcting for Refraction

If we know the Refraction Coefficient, height measurements using a theodolite can be corrected with the following equations:

' =' 'Refraction Angle in radian. Multiply with 180°/π to get degrees
' =' 'Refraction Coefficient
' =' 'target distance in m
' =' '6,371,000 m = radius of the earth

A positive Refraction Angle means the light ray is bent down and the object appears too high. In a zenith angle measurement we have to add the Refraction Angle to the zenith angle to get the correct angle as measured without refraction.

From the Refraction Angle we can calculate the magnitude l of how much an object at distance d appears to be raised due to Refraction:

' =' 'apparent lift of object due to refraction
' =' 'Refraction Angle in radian, see (3)
' =' 'Refraction Coefficient
' =' 'target distance in m
' =' '6,371,000 m = radius of the earth

As Refraction Angles in radian are numbers much less than 1, this equations are very good approximations. The distance d can be the line of sight distance or the distance along the light ray or the distance along the surface of the earth. They are all essentially the same as long as d is much smaller than R.

How does Refraction work?

Before we can derive equations for atmospheric refraction we first have to investigate how refraction works. We know that the speed of light in a medium is slower than in a vacuum. This is expressed by the refractive index:


per Definition

' =' 'Refractive Index or Index of Refraction, a dimensionless number 1; for vacuum n = 1
' =' '299,792,458 m/s = speed of light in vaccum [2]
' =' 'speed of light in a medium

The refractive index depends on the density and composition of the medium and the wavelength of the light. In a homogeneous solid like glas the refractive index is constant. As soon as light, or any electromagnetic wave, encounters a boundary with a sudden change in the refractive index, the light gets bent by an angle that can be calculated by Snell's law if we know the refractive index of both sides of the boundary. Light gets always bent towards the denser medium, that is the medium with a greater refractive index.

A typical value for the refractive index of air at sea level under standard atmospheric conditions is no = 1.000278. Since it is so close to 1, the difference between the refractive index of air and that in a vacuum (n = 1) is of most interest. Since n−1 is very small, it is useful to define Refractivity, which is equal to:


per Definition

' =' 'Refractivity
' =' 'Refractive Index or Index of Refraction

So Refractivity of air at sea level is typically N = 278 N-units.

Refraction in the Atmosphere

ZoomFig. 1: Ray bundle traversing the atmosphere

Refraction in the atmosphere is more complicated than in a solid with constant refractive index. The air density decreases continuously with increasing altitude. This is called a density gradient. Consequently the refractive index also decreases continuously with increasing altitude until it reaches 1 in the vaccum of space. How is light affected by a refractive index gradient? We can't apply Snell's law here because we have no boundaries.

But we can use calculus to derive an equation that relates curvature of a light ray to any refractive index gradient.

Consider a bundle of light rays propagating through the air with a refractive index gradient as shown in Fig. 1. The radius of curvature of the light ray is denoted as r. All rays in the bundle must traverse the air in the same time dt, if they are part of the same bundle. Lets consider two rays in that bundle.

Path 2: The wave on this path has a radius of r and a velocity v and traverses the path in a time dt. The length of the path that is traversed in time dt is r·dθ. Because length is velocity times time, we can write this as:


Path 1: The refractive index n is different since the path is higher in the atmosphere. We denote the refractinve index here as n + dn where dn is the difference in the indices. The radius of this path is r + dr. The associated velocity is v + dv and the time interval dt is the same for both rays. Similar to (7) we can write for this ray:


We can get rid of dθ and dt by dividing equations (7) and (8):


Subtracting 1 from both sides we optain:


The definition of the refractive index is n = c/v, where c is the speed of light in vacuum and v the speed of light in the medium. Differentiating with respect to v gives:


We have now a relation between the refractive index and the speed of light in the medium. But we need a relation between the refractive index and the radius of curvature of the light ray. We can get this by replacing the green term in (11) with the blue term of (10):


The inverse of the radius of curvature r is called curvature. Hence, the relation (12) describes the curvature of the ray as a function of the refractive index n and the refractive index gradient dn/dr.

We can make some simplifications: if we assume that the ray propagates close to parallel to the ground, so that the angle β in the figure is small, then dr ≈ dh. The refractive index n of air is very close to 1, so we can neglegt the term 1/n.

So the curvature of the light ray can be expressed as:

' =' 'radius of curvature of the bent light ray
' =' 'refractive index gradient

Note: As the refractive index decreases with increasing altitude, the refractive index gradient is negative, but due to the minus sign the curvature is positive. From our derivation follows, that a positive curvature means that light gets bent down toward the surface. This causes objects in the distance to appear higher than they are.

This is true even if the light ray starts initialliy horizontal. This is due to the gradient of the refractive index, not the curvature of the atmosphere. So even on a flat earth with a vertical gradient, horizontal light gets bent down towards the denser part of the atmosphere.

Note: If the refractive index gradient is 0, the curvature 1/r of the light ray is zero, which means the light ray is straight.

Now we can calculate the curvature 1/r of a light ray at any point from the refractive index gradient dn/dh at that point. And as the Refraction Coefficient k is a measure of this curvature, using the relation (1), the Refraction Coefficient can be calculated from this gradient:

' =' 'Refraction Coefficient
' =' '6371 km = Radius of the earth
' =' 'radius of curvature of the light ray
' =' 'refractivity gradient in 1/km, standard is −22.4/km for k = 0.143
' =' 'refractive index gradient in 1/km
' =' '(n1) · 106 = refractivity
' =' 'refractive index
' =' 'altitude

I found this derivation on the following PDF:

Note: The end result in the linked document is expressed as 1/K, where K is called the K-value. I call this value the Refraction Factor a. The connection between the Refraction Coefficient k and K or a is:

' =' 'K-value or Refraction Factor
' =' 'Refraction Coefficient

Calculations like for hidden height assume a straight light ray. But there is a trick to keep the same equations for bent light, by increasing the radius of the earth in the equations by the Refraction Factor a and making the light ray straight. The geometry of the two situations is the same.

A value of a = K = 7/6 corresponds to Standard Refraction k = 0.143.

Calculating Refractivity of Air

Before we can derive the equation for the Rrefraction Coefficient k as a function of atmospheric pressure, temperature and temperature gradient, we have to calculate the Refractivity of the air N dependent on some atmospheric parameters.

The refractivity of the atmosphere at a certain point is a function of many things, including the wavelength of light, the air pressure P, the absolute temperate T, and the humidity e at that point. A commonly used expression is [3] [4]:

' =' 'Refractivity
' =' 'Air pressure in mBar or hPa or 1/100 Pa
' =' 'Absolute Temperature in Kelvin
' =' 'Humidity expressed as the partial pressure of water vapour, in mBar
' =' '79.0 K/mBar for light (λ = 550 nm)
' =' '4810 K

For an accuracy of less than 0.5% some simplifying assumptions may be made if we limit the range of certain variables [4]:

  • Temperature: −50°C ≤ T+40°C
  • Pressure: 200 mBar ≤ P1100 mBar
  • Humidity and CO2 content can be neglegted

So equation (16) can be simplified to:

' =' 'Refractivity
' =' 'Air pressure in mBar or hPa or 1/100 Pa
' =' 'Absolute Temperature in Kelvin
' =' '79.0 K/mBar for light (λ = 550 nm), see (19)

The constant K1 is calculated as follows: The refractivity is proportional to the density of the air. So we can write:

gas law
' =' 'refractivity at altitude h
' =' 'refractivity at sea level at International Standard Atmosphere, see below
' =' 'proportionality factor
' =' '1.225 kg/m3 = density of air at sea level and standard atmosphere
' =' 'density of air at altitude h
' =' 'air pressure at altitude h in Pa
' =' 'air pressure at altitude h in mBar
' =' 'absolute temperature at altitude h
' =' '287.058 J/(kg·K) = Specific Gas Constant of dry air

Note: The factor 100 is present so we can use the pressure P in mBar instead of Pa (Pascal).

The blue term is our K1. Now we only have to find the refractivity No for light of wavelength λ = 550 nm, standard pressure at sea level Po = 1013.25 mBar and standard absolute temperature at sea level To = 288.15 K = 15°C. Humidity is set to 0 and CO2 content is set to standard 450 ppm, which is irrelevant for an accuracy of less than 0.5% as can be explored with the Calculator for Refractivity based on Ciddor Equation.

We can use the Calculator for Refractivity based on Ciddor Equation (or even the older Edlén Equation) to find No = 278. The Ciddor equation was derived empirically by accurate measurements of the refractive index of air under many different conditions [5].

Now we can calculate K1:


Deriving the Equation for the Refraction Coefficient

As we know how the Refraction Coefficient k can be calculated from the refractive index gradient dn/dh or the refractivity gradient dN/dh (14), we can derive the equation for the Refraction Coefficient from atmospheric parameters like pressure P, temperature T and temperature gradient dT/dh as shown in (2).

To be able to calculate the refractivity gradient dN/dh, we need equations to calculate pressure P and temperature T as functions of altitude h. This equations are provided for the model of the International Standard Atmosphere. For the troposphere with a linear temperature gradient α:

' =' 'Air pressure at altitude h in mBar or hPa or 1/100 Pa
' =' 'Air pressure at altitude href
' =' 'Absolute temperatur at href in K
' =' 'Reference altitude
' =' 'Temperature Gradient in K/m
' =' 'Altitude above href
' =' '9.80665 m/s2 gravitational acceleration
' =' 'Specific Gas Constant; dry air = 287.058 J/(kg·K)
' =' 'Absolute Temperature at altitude h in Kelvin
' =' 'Absolute Temperatur at href
' =' 'Temperature Gradient
' =' 'Altitude above href
' =' 'Reference altitude

We can insert this equations in (17) and get refractivity as a function of altitude:

' =' 'hhref

We need the refractivity gradient, so we have to calculate the derivative with respect to h:


We can split the blue term into 2 terms:


Applying this split and simplifying we get:


Resubstituting the expressions for P and T:




α is simply the temperature gradient α = dT/dh, so we finally get:


We can insert some values: g/RS = 0.0343 K/m, K1 = 79.0 K/mBar and R = 6.37 × 106 m:


to get finally:

' =' 'Refraction Coefficient
' =' 'Air pressure at the observer in mBar or hPa or 1/100 Pa, Standard = 1013.25 mBar
' =' 'Temperature at the observer in Kelvin, Standard = 288.15 K = 15°C
' =' 'Temperature Gradient at the observer in K/m or °C/m, Standard = −0.0065°C/m

Note: I used a value g = 9.83 m/s2 to get the published constant 0.0343. This is slightly more than average gravitational acceleration. Gravitational acceleration on earth at sea level is in the range of g = 9.7639...9.8337 m/s2 [6].


Terrestrial refraction; Wikipedia
Speed of Light; Wikipedia
Atmospheric Refraction; Radio and Microwave Wireless Systems; Prof. Sean Victor Hum
Very good derivation of atmospheric refraction due to refractive index gradients
The Constants in the Equation for Atmospheric Refractive Index at Radio Frequencies; Ernest K. Smith, Jr., and Stanley Weintraub
Refractivity of Air; An Introduction to Green Flashes; Andrew T. Young
Chronoligy of early measurements of the refractive index of air
Gravity of Earth; Wikipedia
An Accurate Method for Computing Atmospheric Refraction
Stone, R. C.; Journal: Publications of the Astronomical Society of the Pacific, v.108, p.1051-1058; Bibliographic Code: 1996PASP..108.1051S
International GNSS Service
The IGS collects, archives, and distributes GPS observation data sets of sufficient accuracy to satisfy the objectives of a wide range of applications and experimentation. These data sets are used by the IGS to generate the data products which are made available to interested users through this website.
Concepts and Solutions to Overcome the Refraction Problem in Terrestrial Precision Measurement; Prof. Dr. Hilmar Ingensand
Keywords: Refraction, dispersometry, scintillometry, temperature gradient
Weitere Infos zur Seite
Erzeugt Friday, June 21, 2019
von wabis
Zum Seitenanfang
Geändert Tuesday, August 20, 2019
von wabis