# Deriving Equations for Atmospheric Refraction

On this page I explain how terrestrial atmospheric refraction is calculated from basic physical laws and how the equations and constants used in refraction equations are derived. The same principle can be applied to derive astronomical refraction, although this is not the topic of this page.

Atmospheric Refraction is the bending of light due to a gradient of the refractive index caused by a density gradient of the air. This light bending causes observed objects in the distance to be displaced (mainly up) or distorted and inverted (mirage). In survey it is important to know how much light gets bent to be able to correct the observed height of a distant object for Refraction effects.

## Definition of the Refraction Coefficient

The density gradient along a light ray is difficult to measure in practice. Consequently, the path of a light ray can not be known exactly. But as long as the air is not too turbulent, we can use an average gradient. In this case a light ray can be approximated by an arc with the radius r.

Refraction can be expressed by the Refraction Coefficient k, which is the ratio of the radius of the earth to the radius of a bent light ray:

(1)

per Definition

where'
 $k$ ' =' 'Refraction Coefficient $\kappa$ ' =' '1 / r = curvature of light ray $R$ ' =' '6371 km = radius of the earth $r$ ' =' 'radius of curvature of the light ray

Note: The phenomenon of refraction is not dependent on the shape of the earth. We could define a refraction factor that is not related to the radius of the earth. In fact we could simply use the curvature κ = 1 / r of the light ray. We can always get this curvature from k by dividing all equations for k through R: κ = k / R.

If we know the pressure, temperature and temperature gradient at the observer and assume the same conditions along the line of sight, we can use the following simple equation to calculate the Refraction Coefficient [1]:

(2)
where'
 $k$ ' =' 'Refraction Coefficient $P$ ' =' 'Air pressure at the observer in mbar or hPa or 1/100 Pa, Standard = 1013.25 mbar $T$ ' =' 'Temperature at the observer in Kelvin, Standard = 288.15 K = 15°C $\mathrm{d} T/\mathrm{d} h$ ' =' 'Temperature Gradient at the observer in K/m or °C/m, Standard = −0.0065°C/m

Or if you wish to calculate the ray curvature without assuming the radius of the earth R:

(3)
where'
 $\kappa$ ' =' 'Curvature of light ray $r$ ' =' 'radius of curvature of the light ray $P$ ' =' 'Air pressure at the observer in mbar or hPa or 1/100 Pa, Standard = 1013.25 mbar $T$ ' =' 'Temperature at the observer in Kelvin, Standard = 288.15 K = 15°C $\mathrm{d} T/\mathrm{d} h$ ' =' 'Temperature Gradient at the observer in K/m or °C/m, Standard = −0.0065°C/m

How this equation is derived from the refractive index gradient of the air will be shown on this page.

## Correcting for Refraction

If we know the Refraction Coefficient, height measurements using a theodolite can be corrected with the following approximation equations, as long as the distance d is much less than the radius of curvature of the light ray:

(4)
where'
 $\varphi$ ' =' 'Refraction Angle in radian. Multiply with 180°/π to get degrees $\kappa$ ' =' 'Curvature of light ray, see (3) $r$ ' =' 'radius of curvature of the light ray, see (3) $k$ ' =' 'Refraction Coefficient, see (2) $d$ ' =' 'target distance in m $R$ ' =' '6,371,000 m = radius of the earth

A positive Refraction Angle means the light ray is bent down and the object appears too high. In a zenith angle measurement we have to add the Refraction Angle to the zenith angle to get the correct angle as measured without refraction.

From the Refraction Angle we can calculate the magnitude l of how much an object at distance d appears to be raised due to Refraction:

(5)
where'
 $l$ ' =' 'apparent lift of object due to refraction $\varphi$ ' =' 'Refraction Angle in radian, see (4) $\kappa$ ' =' 'Curvature of light ray, see (3) $r$ ' =' 'radius of curvature of the light ray, see (3) $k$ ' =' 'Refraction Coefficient, see (2) $d$ ' =' 'target distance in m $R$ ' =' '6,371,000 m = radius of the earth

As Refraction Angles in radian are numbers much less than 1, this equations are very good approximations. The distance d can be the line of sight distance or the distance along the light ray or the distance along the surface of the earth. They are all essentially the same as long as d is much smaller than R or r respectively.

## How does Refraction work?

Before we can derive equations for atmospheric refraction we first have to investigate how refraction works. We know that the speed of light in a medium is slower than in a vacuum. This is expressed by the refractive index:

(6)

per Definition

where'
 $n$ ' =' 'Refractive Index or Index of Refraction, a dimensionless number ≥ 1; for vacuum n = 1 $c$ ' =' '299,792,458 m/s = speed of light in vaccum [2] $v$ ' =' 'speed of light in a medium

The refractive index depends on the density and composition of the medium and the wavelength of the light. In a homogeneous solid like glas the refractive index is constant. As soon as light, or any electromagnetic wave, encounters a boundary with a sudden change in the refractive index, the light gets bent by an angle that can be calculated by Snell's law if we know the refractive index of both sides of the boundary. Light gets always bent towards the denser medium, that is the medium with a greater refractive index.

Explanation of how Refraction works; Wikipedia
https://en.wikipedia.org/wiki/Refraction

Rather dark Anatidae and the basics of refraction
Excellent derivation of Snell's law and the refractive index by AB science

A typical value for the refractive index of air at sea level under standard atmospheric conditions is no = 1.000278. Since it is so close to 1, the difference between the refractive index of air and that in a vacuum (n = 1) is of most interest. Since n−1 is very small, it is useful to define Refractivity, which is equal to:

(7)

per Definition

where'
 $N$ ' =' 'Refractivity $n$ ' =' 'Refractive Index or Index of Refraction

So Refractivity of air at sea level is typically N = 278 N-units.

## Refraction in the Atmosphere

Fig. 1: Ray bundle traversing the atmosphere

Refraction in the atmosphere is more complicated than in a solid with constant refractive index. The air density decreases continuously with increasing altitude. This is called a density gradient. Consequently the refractive index also decreases continuously with increasing altitude until it reaches 1 in the vaccum of space. How is light affected by a refractive index gradient? We can't apply Snell's law here because we have no boundaries.

But we can use calculus to derive an equation that relates curvature of a light ray to any refractive index gradient.

Consider a bundle of light rays propagating through the air with a refractive index gradient as shown in Fig. 1. The radius of curvature of the light ray is denoted as r. All rays in the bundle must traverse the air in the same time dt, if they are part of the same bundle. Lets consider two rays in that bundle.

Path 2: The wave on this path has a radius of r and a velocity v and traverses the path in a time dt. The length of the path that is traversed in time dt is r·dθ. Because length is velocity times time, we can write this as:

 (8)

Path 1: The refractive index n is different since the path is higher in the atmosphere. We denote the refractive index here as n + dn where dn is the difference in the indices. The radius of this path is r + dr. The associated velocity is v + dv and the time interval dt is the same for both rays. Similar to (8) we can write for this ray:

 (9)

We can get rid of dθ and dt by dividing equations (8) and (9):

 (10)

Subtracting 1 from both sides we obtain:

 (11)

The definition of the refractive index is n = c/v, where c is the speed of light in vacuum and v the speed of light in the medium. Differentiating with respect to v gives:

 (12)

We have now a relation between the refractive index and the speed of light in the medium. But we need a relation between the refractive index and the radius of curvature of the light ray. We can get this by replacing the green term in (12) with the blue term of (11):

 (13)

The inverse of the radius of curvature r is called curvature. Hence, the relation (13) describes the curvature of the ray as a function of the refractive index n and the refractive index gradient dn/dh.

We can make some simplifications: if we assume that the ray propagates close to parallel to the ground, so that the angle β in the figure is small, then cos(β) ≈ 1. The refractive index n of air is very close to 1, so we can neglegt the term 1/n.

So the curvature of the light ray can be expressed as:

(14)
where'
 $\kappa$ ' =' 'curvature of the bent light ray (0 = straight) $r$ ' =' 'radius of curvature of the bent light ray $\beta$ ' =' 'inclination angle with respect to horizontal $n$ ' =' 'refractive index $\mathrm{d} n/\mathrm{d} h$ ' =' 'refractive index gradient

Note: As the refractive index decreases with increasing altitude, the refractive index gradient is negative, but due to the minus sign the curvature is positive. From our derivation follows, that a positive curvature means that light gets bent down toward the surface. This causes objects in the distance to appear higher than they are.

This is true even if the light ray starts initialliy horizontal. This is due to the gradient of the refractive index, not the curvature of the atmosphere. So even on a flat earth with a vertical gradient, horizontal light gets bent down towards the denser part of the atmosphere.

Note: If the refractive index gradient is 0, the curvature 1/r of the light ray is zero, which means the light ray is straight.

Now we can calculate the curvature 1/r of a light ray at any point from the refractive index gradient dn/dh at that point. And as the Refraction Coefficient k is a measure of this curvature, using the relation (1), the Refraction Coefficient can be calculated from this gradient:

(15)
where'
 $k$ ' =' 'Refraction Coefficient $R$ ' =' '6371 km = Radius of the earth $r$ ' =' 'radius of curvature of the light ray $\mathrm{d} N/\mathrm{d} h$ ' =' 'refractivity gradient in 1/km, standard is −22.4/km for k = 0.143 $\mathrm{d} n/\mathrm{d} h$ ' =' 'refractive index gradient in 1/km $N$ ' =' '(n − 1) · 106 = refractivity $n$ ' =' 'refractive index $h$ ' =' 'altitude

I found this derivation on the following PDF:

Note: The end result in the linked document is expressed as 1/K, where K is called the K-value. I call this value the Refraction Factor a. The connection between the Refraction Coefficient k and K or a is:

(16)
where'
 $K = a$ ' =' 'K-value or Refraction Factor $k$ ' =' 'Refraction Coefficient

Calculations like for hidden height assume a straight light ray. But there is a trick to keep the same equations for bent light, by increasing the radius of the earth in the equations by the Refraction Factor a and making the light ray straight. The geometry of the two situations is the same.

A value of a = K = 7/6 corresponds to Standard Refraction k = 0.143.

## Calculating Refractivity of Air

Before we can derive the equation for the Rrefraction Coefficient k as a function of atmospheric pressure, temperature and temperature gradient, we have to calculate the Refractivity of the air N dependent on some atmospheric parameters.

The refractivity of the atmosphere at a certain point is a function of many things, including the wavelength of light, the air pressure P, the absolute temperature T, and the humidity e at that point. A commonly used expression is [3] [4]:

(17)
where'
 $N$ ' =' 'Refractivity $P$ ' =' 'Air pressure in mbar or hPa or 1/100 Pa $T$ ' =' 'Absolute Temperature in Kelvin $e$ ' =' 'Humidity expressed as the partial pressure of water vapour, in mbar $K_1$ ' =' '79.0 K/mbar for light (λ = 550 nm) $K_2$ ' =' '4810 K

For an accuracy of less than 0.5% some simplifying assumptions may be made if we limit the range of certain variables [4]:

• Temperature: −50°C ≤ T+40°C
• Pressure: 200 mbar ≤ P1100 mbar
• Humidity and CO2 content can be neglegted

So equation (17) can be simplified to:

(18)
where'
 $N$ ' =' 'Refractivity $P$ ' =' 'Air pressure in mbar or hPa or 1/100 Pa $T$ ' =' 'Absolute Temperature in Kelvin $K_1$ ' =' '79.0 K/mbar for light (λ = 550 nm), see (20)

The constant K1 is calculated as follows: The refractivity is proportional to the density of the air. So we can write:

(19)
gas law
where'
 $N(h)$ ' =' 'refractivity at altitude h $N_o$ ' =' 'refractivity at sea level at International Standard Atmosphere, see below $N_o/\rho_o$ ' =' 'proportionality factor $\rho_o$ ' =' '1.225 kg/m3 = density of air at sea level and standard atmosphere $\rho(h)$ ' =' 'density of air at altitude h $P(h)$ ' =' 'air pressure at altitude h in Pa $P'(h)$ ' =' 'air pressure at altitude h in mbar $T(h)$ ' =' 'absolute temperature at altitude h $R_\mathrm{S}$ ' =' '287.058 J/(kg·K) = Specific Gas Constant of dry air

Note: The factor 100 is present so we can use the pressure P in mbar instead of Pa (Pascal).

The blue term is our K1. Now we only have to find the refractivity No for light of wavelength λ = 550 nm, standard pressure at sea level Po = 1013.25 mbar and standard absolute temperature at sea level To = 288.15 K = 15°C. Humidity is set to 0 and CO2 content is set to standard 450 ppm, which is irrelevant for an accuracy of less than 0.5% as can be explored with the Calculator for Refractivity based on Ciddor Equation.

We can use the Calculator for Refractivity based on Ciddor Equation (or even the older Edlén Equation) to find No = 278. The Ciddor equation was derived empirically by accurate measurements of the refractive index of air under many different conditions [5].

Now we can calculate K1:

(20)

## Deriving the Equation for the Refraction Coefficient

As we know how the Refraction Coefficient k can be calculated from the refractive index gradient dn/dh or the refractivity gradient dN/dh (15), we can derive the equation for the Refraction Coefficient from atmospheric parameters like pressure P, temperature T and temperature gradient dT/dh as shown in (2).

To be able to calculate the refractivity gradient dN/dh, we need equations to calculate pressure P and temperature T as functions of altitude h. This equations are provided for the model of the International Standard Atmosphere. For the troposphere with a linear temperature gradient α:

(21)
with
where'
 $P(h)$ ' =' 'Air pressure at altitude h in mbar or hPa or 1/100 Pa $P_{\mathrm{ref}}$ ' =' 'Air pressure at altitude href $T_{\mathrm{ref}}$ ' =' 'Absolute temperatur at href in K $h_{\mathrm{ref}}$ ' =' 'Reference altitude $\alpha$ ' =' 'Temperature Gradient in K/m $h$ ' =' 'Altitude above href $g$ ' =' '9.80665 m/s2 gravitational acceleration $R_\mathrm{S}$ ' =' 'Specific Gas Constant; dry air = 287.058 J/(kg·K)
(22)
where'
 $T(h)$ ' =' 'Absolute Temperature at altitude h in Kelvin $T_\mathrm{ref}$ ' =' 'Absolute Temperatur at href $\alpha$ ' =' 'Temperature Gradient $h$ ' =' 'Altitude above href $h_\mathrm{ref}$ ' =' 'Reference altitude

We can insert this equations in (18) and get refractivity as a function of altitude:

(23)
where'
 $\Delta h$ ' =' 'h − href

We need the refractivity gradient, so we have to calculate the derivative with respect to h:

 (24)

We can split the blue term into 2 terms:

 (25)

Applying this split and simplifying we get:

 (26)

Resubstituting the expressions for P and T:

 (27)

gives:

 (28)

α is simply the temperature gradient α = dT/dh, so we finally get:

(29)

We can insert some values: g/RS = 0.0343 K/m, K1 = 79.0 K/mbar and R = 6.37 × 106 m:

 (30)

to get finally:

(31)
where'
 $k$ ' =' 'Refraction Coefficient $P$ ' =' 'Air pressure at the observer in mbar or hPa or 1/100 Pa, Standard = 1013.25 mbar $T$ ' =' 'Temperature at the observer in Kelvin, Standard = 288.15 K = 15°C $\mathrm{d} T/\mathrm{d} h$ ' =' 'Temperature Gradient at the observer in K/m or °C/m, Standard = −0.0065°C/m

Note: I used a value g = 9.83 m/s2 to get the published constant 0.0343. This is slightly more than average gravitational acceleration. Gravitational acceleration on earth at sea level is in the range of g = 9.7639...9.8337 m/s2 [6].

## Influence of Water Vapor

The absolute value of the refractive index is irrelevant for the bending of light, only the gradient, the amount of change per height, is relevant for how much light gets bent.

(32)
where'
 $\kappa$ ' =' 'curvature of light ray $r$ ' =' 'radius of curvature of light ray $\mathrm{d} n / \mathrm{d} h$ ' =' 'refractive index gradient, i.e. the change per altitude

If the refractive index gradient is zero, i.e. no change in refractive index with increasing altitude, then the curvature of the light ray is zero, no matter what the absolute value of the refractive index is.

The refractive index is proportional to the density of the air. So if the density is constant, then the refractive index is constant and we have no light bending in a constant density. Because the density decreases with increasing altitude, the refractive index decreases proportionally and light gets bent down towards the denser part of the atmosphere.

The density of moist air can be calculated as follows: [7]

(33)
where'
 $\rho_\mathrm{ma}$ ' =' 'density of moist air $\rho_\mathrm{da}$ ' =' 'density of dry air $K_\mathrm{w}(x)$ ' =' 'correction factor depending on humidity ratio $x$ ' =' 'humidity ratio $R_\mathrm{a}$ ' =' '287.1 J/kg/K = specific gas constant of dry air $R_\mathrm{w}$ ' =' '461.5 J/kg/K = specific gas constant of water vapor

The maximal humidity ratio for air at 15°C is x = 0.01062 [8]. The corresponding correction factor Kw due to maximum saturation humidity at 15°C is then:

(34)

So saturated moist air density is 0.9936 times the density of dry air. Moist air is maximal 0.64% less dense than dry air at 15°C. The general influence on the density gradient is neglegtable.

As the equations for the refractive index show (see Calculator for Refractivity based on Ciddor Equation), the influence of water vapor and CO2 is marginal compared to pressure, temperature and the wavelength of light. So humidity and CO2 are ignored or a mean value is choosen, which does change refraction calculations only very marginal.

The graphic shows how water vapor can change the density or refractive index respectively from the red curve (dry air) to the blue curve (moist to dry air).

The air cannot have less humidity than 0. This limits the influence of water vapor on the refractive index as follows:

• Left: If humidity stays constant over great height ranges, then it does reduce the magnitude of density very slightly evenly, but does not change to density gradient. So there is no change in the bending of light between dry and moist air.
• Center: If humidity reduces slowly over great height ranges, then there is a very, very slight change in the density gradient, the slope of the curve. The corresponding change in light bending is there but can be neglegted.
• Right: To have any noticable influence on refraction at all, humidity has to change very quick. It has to go from maximum humidity to dry air in a very small layer directly above the water to get a strong enough gradient that has any effect on refraction at all. How fast moist air dries is limited by physics.

Generally the density gradient depends mostly on the temperature gradient, then on pressure and temperature and only very little on the humidity gradient. A constant humidity has no influence on refraction at all, a decreasing humidity with altitude makes refraction a tiny bit less, an increasing humidity a tiny bit stronger, very marginal, neglegtable in practice.

The surface layer is the only place where refraction can vary considerably due to steep temperature gradients due to heat exchange between ground and air. Above the surface layer density always decreases slowly according to the equations of the International Standard Atmosphere.

## References

Terrestrial refraction; Wikipedia
https://en.wikipedia.org/wiki/Atmospheric_refraction#Terrestrial_refraction
Speed of Light; Wikipedia
https://en.wikipedia.org/wiki/Speed_of_light
Atmospheric Refraction; Radio and Microwave Wireless Systems; Prof. Sean Victor Hum
Very good derivation of atmospheric refraction due to refractive index gradients
http://www.waves.utoronto.ca/prof/svhum/ece422/notes/20a-atmospheric-refr.pdf
[4]
The Constants in the Equation for Atmospheric Refractive Index at Radio Frequencies; Ernest K. Smith, Jr., and Stanley Weintraub
http://web.archive.org/web/20180721151000/https://nvlpubs.nist.gov/nistpubs/jres/50/jresv50n1p39_A1b.pdf
Refractivity of Air; An Introduction to Green Flashes; Andrew T. Young
Chronoligy of early measurements of the refractive index of air
https://aty.sdsu.edu/explain/atmos_refr/air_refr.html
Gravity of Earth; Wikipedia
https://en.wikipedia.org/wiki/Gravity%5Fof%5FEarth
Density of Moist Humid Air; The Engineering ToolBox
https://www.engineeringtoolbox.com/density-air-d_680.html
Humidity Ratio of Air; The Engineering ToolBox
https://www.engineeringtoolbox.com/humidity-ratio-air-d_686.html
[9]
An Accurate Method for Computing Atmospheric Refraction
Stone, R. C.; Journal: Publications of the Astronomical Society of the Pacific, v.108, p.1051-1058; Bibliographic Code: 1996PASP..108.1051S
[10]
International GNSS Service
The IGS collects, archives, and distributes GPS observation data sets of sufficient accuracy to satisfy the objectives of a wide range of applications and experimentation. These data sets are used by the IGS to generate the data products which are made available to interested users through this website.
http://www.igs.org/
[11]
Concepts and Solutions to Overcome the Refraction Problem in Terrestrial Precision Measurement; Prof. Dr. Hilmar Ingensand
Keywords: Refraction, dispersometry, scintillometry, temperature gradient
https://www.fig.net/resources/proceedings/fig_proceedings/fig_2002/Js28/JS28_ingensand.pdf

### Blog-Functions

 About Walter Bislin (wabis) Rights Public Domain
 More Page Infos / Sitemap Created Friday, June 21, 2019
 Scroll to Top of Page Changed Tuesday, June 23, 2020