On this page I explain how terrestrial atmospheric refraction is calculated from basic physical laws and how the equations and constants used in refraction equations are derived. The same principle can be applied to derive astronomical refraction, although this is not the topic of this page.

Atmospheric Refraction is the bending of light due to a gradient of the refractive index caused by a density gradient of the air. This light bending causes observed objects in the distance to be displaced (mainly up) or distorted and inverted (mirage). In a layer above the surface refraction can vary considerably. Above this layer refraction varies only little and we can derive a mean standard value, see Refraction Coefficient as a Function of Altitude.

In survey it is important to know how much light gets bent to be able to correct the observed height of a distant object for Refraction effects.

Definition of the Refraction Coefficient

The density gradient along a light ray is difficult to measure in practice. Consequently, the path of a light ray can not be known exactly. But as long as the air is not too turbulent, we can use an average gradient. In this case a light ray can be approximated by an arc with the radius r.

Refraction can be expressed by the Refraction Coefficient k, which is the ratio of the radius of the earth to the radius of a bent light ray:

Note: The phenomenon of refraction is not dependent on the shape of the earth. We could define a refraction factor that is not related to the radius of the earth. In fact we could simply use the curvature κ = 1 / r of the light ray. We can always get this curvature from k by dividing all equations for k through R: κ = k / R.

If we know the pressure, temperature and temperature gradient at the observer and assume the same conditions along the line of sight, we can use the following simple equation to calculate the Refraction Coefficient [1]:

Or if you wish to calculate the ray curvature without assuming the radius of the earth R:

How this equation is derived from the refractive index gradient of the air will be shown on this page.

Correcting for Refraction

If we know the Refraction Coefficient, height measurements using a theodolite can be corrected with the following approximation equations, as long as the distance d is much less than the radius of curvature of the light ray:

A positive Refraction Angle means the light ray is bent down and the object appears too high. In a zenith angle measurement we have to add the Refraction Angle to the zenith angle to get the correct angle as measured without refraction.

From the Refraction Angle we can calculate the magnitude l of how much an object at distance d appears to be raised due to Refraction:

As Refraction Angles in radian are numbers much less than 1, this equations are very good approximations. The distance d can be the line of sight distance or the distance along the light ray or the distance along the surface of the earth. They are all essentially the same as long as d is much smaller than R or r respectively.

How does Refraction work?

Before we can derive equations for atmospheric refraction we first have to investigate how refraction works. We know that the speed of light in a medium is slower than in a vacuum. This is expressed by the refractive index:

The refractive index depends on the density and composition of the medium and the wavelength of the light. In a homogeneous solid like glas the refractive index is constant. As soon as light, or any electromagnetic wave, encounters a boundary with a sudden change in the refractive index, the light gets bent by an angle that can be calculated by Snell's law if we know the refractive index of both sides of the boundary. Light gets always bent towards the denser medium, that is the medium with a greater refractive index.

A typical value for the refractive index of air at sea level under standard atmospheric conditions is n_o = 1.000278. Since it is so close to 1, the difference between the refractive index of air and that in a vacuum (n = 1) is of most interest. Since n−1 is very small, it is useful to define Refractivity, which is equal to:

So Refractivity of air at sea level is typically N = 278 N-units.

Refraction in the Atmosphere

Refraction in the atmosphere is more complicated than in a solid with constant refractive index. The air density decreases continuously with increasing altitude. This is called a density gradient. Consequently the refractive index also decreases continuously with increasing altitude until it reaches 1 in the vaccum of space. How is light affected by a refractive index gradient? We can't apply Snell's law here because we have no boundaries.

But we can use calculus to derive an equation that relates curvature of a light ray to any refractive index gradient.

Consider a bundle of light rays propagating through the air with a refractive index gradient as shown in Fig. 1. The radius of curvature of the light ray is denoted as r. All rays in the bundle must traverse the air in the same time dt, if they are part of the same bundle. Lets consider two rays in that bundle.

Path 2: The wave on this path has a radius of r and a velocity v and traverses the path in a time dt. The length of the path that is traversed in time dt is r·dθ. Because length is velocity times time, we can write this as:

Path 1: The refractive index n is different since the path is higher in the atmosphere. We denote the refractive index here as n + dn where dn is the difference in the indices. The radius of this path is r + dr. The associated velocity is v + dv and the time interval dt is the same for both rays. Similar to (8) we can write for this ray:

We can get rid of dθ and dt by dividing equations (8) and (9):

Subtracting 1 from both sides we obtain:

The definition of the refractive index is n = c/v, where c is the speed of light in vacuum and v the speed of light in the medium. Differentiating with respect to v gives:

We have now a relation between the refractive index and the speed of light in the medium. But we need a relation between the refractive index and the radius of curvature of the light ray. We can get this by replacing the green term in (12) with the blue term of (11):

The inverse of the radius of curvature r is called curvature. Hence, the relation (13) describes the curvature of the ray as a function of the refractive index n and the refractive index gradient dn/dh.

We can make some simplifications: if we assume that the ray propagates close to parallel to the ground, so that the angle β in the figure is small, then cos(β) ≈ 1. The refractive index n of air is very close to 1, so we can neglegt the term 1/n.

So the curvature of the light ray can be expressed as:

Note: As the refractive index decreases with increasing altitude, the refractive index gradient is negative, but due to the minus sign the curvature is positive. From our derivation follows, that a positive curvature means that light gets bent down toward the surface. This causes objects in the distance to appear higher than they are.

This is true even if the light ray starts initialliy horizontal. This is due to the gradient of the refractive index, not the curvature of the atmosphere. So even on a flat earth with a vertical gradient, horizontal light gets bent down towards the denser part of the atmosphere.

Note: If the refractive index gradient is 0, the curvature 1/r of the light ray is zero, which means the light ray is straight.

Now we can calculate the curvature 1/r of a light ray at any point from the refractive index gradient dn/dh at that point. And as the Refraction Coefficient k is a measure of this curvature, using the relation (1), the Refraction Coefficient can be calculated from this gradient:

I found this derivation on the following PDF:

Note: The end result in the linked document is expressed as 1/K, where K is called the K-value. I call this value the Refraction Factor a. The connection between the Refraction Coefficient k and K or a is:

Calculations like for hidden height assume a straight light ray. But there is a trick to keep the same equations for bent light, by increasing the radius of the earth in the equations by the Refraction Factor a and making the light ray straight. The geometry of the two situations is the same.

A value of a = K = 7/6 corresponds to Standard Refraction k = 0.143.

Calculating Refractivity of Air

Before we can derive the equation for the Rrefraction Coefficient k as a function of atmospheric pressure, temperature and temperature gradient, we have to calculate the Refractivity of the air N dependent on some atmospheric parameters.

The refractivity of the atmosphere at a certain point is a function of many things, including the wavelength of light, the air pressure P, the absolute temperature T, and the humidity e at that point. A commonly used expression is [3] [4]:

For an accuracy of less than 0.5% some simplifying assumptions may be made if we limit the range of certain variables [4]:

• Temperature: −50°C ≤ T ≤ +40°C
• Pressure: 200 mbar ≤ P ≤ 1100 mbar
• Humidity and CO₂ content can be neglegted

So equation (17) can be simplified to:

The constant K₁ is calculated as follows: The refractivity is proportional to the density of the air. So we can write:

Note: The factor 100 is present so we can use the pressure P in mbar instead of Pa (Pascal).

The blue term is our K₁. Now we only have to find the refractivity N_o for light of wavelength λ = 550 nm, standard pressure at sea level P_o = 1013.25 mbar and standard absolute temperature at sea level T_o = 288.15 K = 15°C. Humidity is set to 0 and CO₂ content is set to standard 450 ppm, which is irrelevant for an accuracy of less than 0.5% as can be explored with the Calculator for Refractivity based on Ciddor Equation.

We can use the Calculator for Refractivity based on Ciddor Equation (or even the older Edlén Equation) to find N_o = 278. The Ciddor equation was derived empirically by accurate measurements of the refractive index of air under many different conditions [5].

Now we can calculate K₁:

Deriving the Equation for the Refraction Coefficient

As we know how the Refraction Coefficient k can be calculated from the refractive index gradient dn/dh or the refractivity gradient dN/dh (15), we can derive the equation for the Refraction Coefficient from atmospheric parameters like pressure P, temperature T and temperature gradient dT/dh as shown in (2).

To be able to calculate the refractivity gradient dN/dh, we need equations to calculate pressure P and temperature T as functions of altitude h. This equations are provided for the model of the International Standard Atmosphere. For the troposphere with a linear temperature gradient α:

We can insert this equations in (18) and get refractivity as a function of altitude:

We need the refractivity gradient, so we have to calculate the derivative with respect to h:

We can split the blue term into 2 terms:

Applying this split and simplifying we get:

Resubstituting the expressions for P and T:

gives:

α is simply the temperature gradient α = dT/dh, so we finally get:

We can insert some values: g/R_S = 0.0343 K/m, K₁ = 79.0 K/mbar and R = 6.37 × 10⁶ m:

to get finally:

Note: I used a value g = 9.83 m/s² to get the published constant 0.0343. This is slightly more than average gravitational acceleration. Gravitational acceleration on earth at sea level is in the range of g = 9.7639...9.8337 m/s² [6].

Influence of Water Vapor

The absolute value of the refractive index is irrelevant for the bending of light, only the gradient, the amount of change per height, is relevant for how much light gets bent.

If the refractive index gradient is zero, i.e. no change in refractive index with increasing altitude, then the curvature of the light ray is zero, no matter what the absolute value of the refractive index is.

The refractive index is proportional to the density of the air. So if the density is constant, then the refractive index is constant and we have no light bending in a constant density. Because the density decreases with increasing altitude, the refractive index decreases proportionally and light gets bent down towards the denser part of the atmosphere.

The density of moist air can be calculated as follows: [7]

The maximal humidity ratio for air at 15°C is x = 0.01062 [8]. The corresponding correction factor K_w due to maximum saturation humidity at 15°C is then:

So saturated moist air density is 0.9936 times the density of dry air. Moist air is maximal 0.64% less dense than dry air at 15°C. The general influence on the density gradient is neglegtable.

As the equations for the refractive index show (see Calculator for Refractivity based on Ciddor Equation), the influence of water vapor and CO2 is marginal compared to pressure, temperature and the wavelength of light. So humidity and CO2 are ignored or a mean value is choosen, which does change refraction calculations only very marginal.

The graphic shows how water vapor can change the density or refractive index respectively from the red curve (dry air) to the blue curve (moist to dry air).

The air cannot have less humidity than 0. This limits the influence of water vapor on the refractive index as follows:

• Left: If humidity stays constant over great height ranges, then it does reduce the magnitude of density very slightly evenly, but does not change to density gradient. So there is no change in the bending of light between dry and moist air.
• Center: If humidity reduces slowly over great height ranges, then there is a very, very slight change in the density gradient, the slope of the curve. The corresponding change in light bending is there but can be neglegted.
• Right: To have any noticable influence on refraction at all, humidity has to change very quick. It has to go from maximum humidity to dry air in a very small layer directly above the water to get a strong enough gradient that has any effect on refraction at all. How fast moist air dries is limited by physics.

Generally the density gradient depends mostly on the temperature gradient, then on pressure and temperature and only very little on the humidity gradient. A constant humidity has no influence on refraction at all, a decreasing humidity with altitude makes refraction a tiny bit less, an increasing humidity a tiny bit stronger, very marginal, neglegtable in practice.

The surface layer is the only place where refraction can vary considerably due to steep temperature gradients due to heat exchange between ground and air. Above the surface layer density always decreases slowly according to the equations of the International Standard Atmosphere. See also Refraction Coefficient as a Function of Altitude.

References