The aim of here is to produce accurate formulae for determining refraction as a function of local atmospheric conditions at the 4.1km altitude of JCMT.
It was found during this study that the approximation of the type given by Allen, for instance, of the form
R = A * tan(z) + B * tan**3(z) (1)
is most suitable, and sufficiently accurate to large zenith distances. The challenge was then to express A and B in terms of local temperature, T, pressure, p, and humidity, h, so that real-time calculations of refraction can be made, and thus keep the JCMT pointing accuracy to acceptable levels.
The approach is to
- develop a model of the atmosphere for the calculation of refraction under varying conditions
- test its sensitivity to various profiles of T,p and h
- compare the results with previous determinations of refraction, and
- proceed to determine the values of A and B in (1) above so as to reduce the calculation from the lengthy integrations through the atmosphere required from the model to a single formula with real-time application.
The method of Allen(AQ3,1973,p124) is used to calculate refractive index. The Earth's atmosphere is considered to be radially symmetric. The basic variations of temperature, density and pressure with altitude are taken to be the American Standard Atmosphere (ASA) as given by Allen(p121). The local (Hawaiian) atmosphere has been studied by Takahasi (U.of H., Hilo). His data reveals the LHA to be 14 degs warmer than the ASA at all altitudes up to 15 km. Seasonal variations of 1-2 degrees are seen on top of this. The model features the ASA temperature profile with the facility for adding a constant at all altitudes, and the ASA pressure profile with the facility of applying a constant multiplicative factor at all altitudes. From Takahashi's results T=4degC is adopted as the reference temperature for the summit of Mauna Kea in the formulations below. Local experience prompted the adoption of 20% as the reference humidity level.
Takahashi's data were also examined for data on the local humidity structure. A feature of his data is that the air is essentially quite dry above 11km. Otherwise many profiles were seen, but it was found during the analysis that refraction in the radially symmetric case, as in the plane case, depends essentially only upon the refractive indices of the top and bottom layers, and thus happily lends itself to formulation based on local conditions only.
The mathematical model for calculating the refraction maintains a radially symmetric atmosphere and resolves the atmosphere according to the formula :
height-resolution = [ height * cos (Z) ] / [10 + Z**2/300 ]
where Z is the zenith distance in degrees
The height-resolution increases with height (altitude) and decreases with increasing zenith distance. Resolutions at low altitudes are much in excess of the 1km resolution of the ASA, but this was felt necessary in order to achieve the highest possible accuracy in the derived refraction ( 0.1" ).
Fig.1 shows one interface between two levels in the atmosphere at which refraction is assumed to take place. The refraction is calculated in the usual way once the refractive indices of the two layers have been determined, and the process repeated at the next level. The total refraction is then the sum of these components.
It is clear at this stage that except at zenith distances close to 90o refraction depends essentially on the humidity at the observer's locale and not upon the atmospheric profile of the humidity.
A check on the programming was made by calculating the optical refractions as given by AQ3,p125. Prevailing conditions are quoted as 760mmHg pressure,10C temperature and 4mmHg water vapour pressure, which is equivalent to a relative humidity of 88%. A model using a uniform 90% humidity to 11km was used. The results were essentially as given by Allen, with differences of less than 1 arc second for elevations above 20 degrees, increasing to 2" at 3 degrees. A comparison with the AQ3 results is given in Table 1.
( Note: The formulation differs from that quoted by Allen for the contribution due to the water : the term quoted, supposedly dependent upon the water vapour pressure, f, is quoted without an f-dependence. The term quoted by Allen was multiplied by f before being used in these calculations.)
A comparison of the millimeter refractions with those of Ulich(1981) is shown in Table 2.
It is seen that except under extreme atmospheric conditions the agreement with Ulich is very good.
With confidence that the model is yielding accurate refractions we proceed to search for a simple formulation that will reproduce the results of the model integrations.
Results in the optical (0.55micron wavelength) showed that refraction could indeed be expressed as in (1). The value of A in (1) is essentially the refraction at 45deg elevation, and its variation with each of the parameters T,p, and h was determined with the other two parameters fixed at nominal values. It is found that for a wavelength of 0.55 microns and an observatory height of 4.1 km : then :
a) allowing T alone to vary ( i.e. p = 0 %, h = 20% ) gives
A = 35.893 - 0.138(T-4) + 0.000432(T-4)**2
B = -0.0359 + 0.000127(T-4) + C(Z)
where T is the temperature in deg C
Z is the Zenith distance
and C(Z) = 0 for Z < 80deg
= 0.0002(Z-80)(Z-79) for 80deg < Z < 85deg
b) allowing p alone to vary (i.e. T = 4degC , h = 20%) gives
A = 35.893 + 0.359p
B = -0.0357 - 0.00034p + C(Z)
where p is the percentage variation in pressure from the ASA
and C(Z) is as above
c) allowing h alone to vary (i.e. T = 4degC , p = 0 ) gives
A = 35.893 - 0.000667(h-20)
B = -0.0360 + C(Z)
where h is the % relative humidity at the observers altitude.
These results suggest that, to a first approximation, in (1),
A = 35.893 - (h-20)/1500 + 0.359p -0.135(T-4) + 0.000432(T-4)**2
and B = -0.0359 + 0.000127(T-4) - 0.00034p + C(Z).
The constant terms are the averages of their values derived above, and
are accurate to about 0.002 and 0.0002 respectively.
It is seen that humidity is relatively unimportant in 'optical' refraction.
By examining the residuals ( formula minus model) at zenith distances of 45 degrees, it is possible to resolve ther non-linear terms arising from the interactions of variations of T,p,h from their nominal values. The only non- linear term of significance may be approximated by the following addition to the term A in (1) :
- 0.0013p(T-4).
Note that this is independent of humidity.
The formula was then compared with the model calculations at a comprehensive number of combinations of T,p,h and Z. It is found that, even at the most extreme conditions considered, the errors are less than 1/3" for Z<75, and <1/2" for Z<80. At atmospheric conditions closer to nominal the errors are essentially neglible. Experience at Mauna Kea suggests that temperatures outside the range -15 to 15C and pressure variations from nominal of greater than 5% are extremely rare.
The error contours in the T-p plane for the case of Z=85 are shown in Fig.2 for the humidity values of 20%, 50% and 80%. It can be seen that only in extreme weather conditions will refraction errors be larger than 1".
Refraction in the radio window is described by all sources (e.g. AQ3) as being wavelength independent. The formulation of Allen is used, and a check on the results is made against those for Kitt Peak at 2km altitude given by Ulich (J. mm & IR ast. nn, nn. 198n), though Ulich's quoted barometric pressure at Kitt Peak of 614 mbar is maintained in the present calculations at the ASA value of nnn mbar.
In a similar fashion to the optical case the form of (1) is adopted and
a) allowing T alone to vary (i.e. p = 0 %, h = 20% ) gives
A = 36.798 - 0.0294(T-4) + 0.00329(T-4)**2
+ 0.000042(T-4)**3
B = -0.0356 + 0.0001(T-4) + C(Z)
where T is the temperature in deg C
Z is the Zenith distance
and C(Z) = 0 for Z < 80deg
= 0.0002(Z-80)(Z-79) for 80deg < Z < 85deg
b) allowing p alone to vary (i.e. T = 4degC , h = 20%) gives
A = 36.801 + 0.3527p
B = -0.0355 - 0.00030p + C(Z)
where p is the percentage variation in pressure from the ASA
and C(Z) is as above
c) allowing h alone to vary (i.e. T = 4degC , p = 0 ) gives
A = 36.800 + 0.0768(h-20)
B = -0.0357 - 0.00001(h-20) + C(Z)
where h is the % relative humidity .
This result is independent of the precise humidity profile thereabove.
The results in section 6 suggest that, to a first approximation, in (1),
A = 36.800 + 0.0768(h-20) + 0.3527p -0.0294(T-4) + 0.00329(T-4)**2 + 0.000042(T-4)**3
and B = -0.0356 + 0.00010(T-4) - 0.00030p - 0.00001(h-20) + C(Z).
The constant terms are the averages of their values derived in section 3, and are accurate to about 0.002 and 0.0002 respectively. It is seen that humidity is much more important in 'radio' refraction, and that the formulae for A and B are similar in form and size in both cases.
The formula above was tested at an elevation of 45 degrees and revealed second-order ('cross') terms in T and h and a small term in T and p. These were evaluated as additional terms for A :
+ 0.00133(h-20)
+ 0.00490(h-20)(T-4) + 0.000140(h-20)(T-4)**2
+ 0.00000222(h-20)(T-4)**3 - 0.00125p(T-4)
(Note added 02 Jun 1995 : should not the first of these terms, and its
transcription below, be 0.00133(h-20)p ? )
For Z<80 errors of greater than 1" occur only when the temperature drops to -20C. We show in Fig.3 the error contours in T-p space for zenith distance 85 degrees, and for humidity values of 20%, 50% and 80%. Again, it is only under the most extreme of conditions that the new formula diverges from an integrated solution by more than 1".
The following formulae are suggested for determining the refraction at the summit of Mauna Kea :
Z = Zenith Distance (o)
T = temperature (oC)
p = pressure variation (%)
h = relative humidity (%)
C(Z) = 0 for Z < 80deg
= 0.0002(Z-80)(Z-79) for 80deg < Z < 85deg
a. optical refraction
A = 35.893 - (h-20)/1500 + 0.359p -0.135(T-4) + 0.000432(T-4)**2
- 0.0013p(T-4).
B = -0.0359 + 0.000127(T-4) - 0.00034p + C(Z).
b. (sub)millimetre refraction
A = 36.800 + 0.0768(h-20) + 0.3527p - 0.0294(T-4) + 0.00329(T-4)**2
+ 0.000042(T-4)**3 + 0.00133(h-20)
+ 0.00490(h-20)(T-4) + 0.000140(h-20)(T-4)**2
+ 0.00000222(h-20)(T-4)**3 - 0.00125p(T-4)
B = -0.0356 + 0.00010(T-4) - 0.00030p - 0.00001(h-20) + C(Z).
The refraction is then given by
A tan(Z) + B tan3(Z)
These formulae are, of course, limited in application, and will not account for local cloud structures or non-laminar air flows which destroy the radial symmetry requirements of the model.
If it is required that neither optical nor millimeter refrcation errors exceed 1" when the elevation is greater than 10 degrees then the errors of measurement of temperature, pressure and relative humidity should not exceed 2C, 0.5% and 2% respectively.
Iain Coulson
Original Version : Sep 1987
Latest Update : Aug 2001