Announcement 2008-04-16

Error in MST Radar altitudes (and ranges) for the period 6th February 2007 - 8th April 2008
Owing to a bug in the new MST radar control and data acquisition system, there have been errors in the reported ranges and altitudes of all MST radar data products for the period from 13:44:03 UT on 6th February 2007 until 14:37:00 UT on 8th April 2008, inclusive.

A compensation for this problem was introduced into the signal processing software starting with the cycle/dwell beginning at 12:48:15 UT on 7th February 2008. Click here for more details. However, it turns out that this over-corrected the problem. Moreover, this meant that the data at the lowest three range gates were unreliable.

The problem has now been fully understood (for the ST-mode) and solved. Appropriate changes were introduced to both the data acquisition software and the signal processing software for cycles/dwells starting at, or after, 14:50:03 UT on 8th April 2008.

Cartesian and radial files for the period 6th February 2007 - 8th April 2008 (inclusive) will eventually be recreated with the corrections incorporated, However, until then, data users should apply the compensations shown below. If you have any doubt as to whether or not your files are affected, or as to how to carry out the compensation, please contact the NERC MST Radar Facility Project Scientist directly.

This page covers the following topics:

• How to tell if your files are affected
• How to compensate for this error in affected ST-mode data
• How to compensate for this error in affected M-mode data
• Evidence for the problem

How to tell if your files are affected
For the time being, all files containing data for the period 6th February 2007 - 8th April 2008 should be considered to be affected. However, when the data are eventually reprocessed, the compensated Cartesian and radial data files will be identifiable from the fact that their creation dates, contained in the netCDF global attribute history:

"File created YYYY-MM-DD HH:MM:SS +00:00 on machine claudius"

are after 9th April 2008 (2008-04-09). Files are typically created a few hours after the 24 hour period which they cover. It is likely that the spectral files will be left as they are and users will need to apply the corrections themselves.


How to compensate for this error in affected ST-mode data
Dates and times below are given in the format YYYY-MM-DD HH:MM:SS. The bottom and top range gate numbers are recorded in the Cartesian and radial files by the global attributes data_bottom_range_gate_number and data_top_range_gate_number, respectively. Spectral files can store ST-mode and M-mode data simultaneously and so have variables (rather than global attributes) named bottom_range_gate_number_for_ST_mode, top_range_gate_number_for_ST_mode, bottom_range_gate_number_for_M_mode, top_range_gate_number_for_M_mode.

• for Cartesian cycles starting between 2007-02-06 13:44:03 UT and 2008-02-07 11:50:27 UT
  
  • actual range gate number = recorded range gate number + 4
  • actual altitude (of range gates above mean sea level) = recorded altitude + 596.7 m
  
  
• for Cartesian cycles starting between 2008-02-07 12:48:15 UT and 2008-04-08 14:31:56 UT
  • actual range gate number = recorded range gate number - 3
  • actual altitude (of range gates above mean sea level) = recorded altitude - 447.5 m
  • Data from the lowest three range gates should be ignored as they are unreliable.
  
  
• For Radial dwells starting between 2007-02-06 13:44:03 UT and 2008-02-07 11:54:57 UT
  • actual range gate number = recorded range gate number + 4
  • actual range (of range gates from the radar) = recorded range + 600.0 m
  
  
• For Radial dwells starting between 2008-02-07 12:48:15 UT and 2008-04-08 14:37:00 UT
  • actual range gate number = recorded range gate number - 3
  • actual range (of range gates from the radar) = recorded range - 450.0 m
  • Data from the lowest three range gates should be ignored as they are unreliable.
  
  
• For Spectral dwells starting between 2007-02-06 13:44:03 UT and 2008-02-07 11:54:57 UT
  • actual range gate number = recorded range gate number + 4
    
  • actual range (of range gates from the radar) = recorded range + 600.0 m
  
  
• For Spectral dwells starting between 2008-02-07 12:48:15 UT and 2008-04-08 14:37:00 UT
  • actual range gate number = recorded range gate number + 4
    
  • actual range (of range gates from the radar) = recorded range + 600.0 m
  • Note that the first two compensations are the same as those for the period 2007-02-06 13:44:03 UT - 2008-02-07 11:54:57 UT. This is not the case for the Cartesian and radial data.
  • Data from the lowest three range gates should be ignored as they are unreliable.

How to compensate for this error in affected M-mode data
If you are making use of M-mode data, even for periods unaffected by the range problem, please contact the NERC MST Radar Facility Project Scientist.

The legacy documentation is incomplete, and sometimes ambiguous, in regard to the M-mode range gates. Moreover, there is reason to believe that there are (slight) bugs in the original data acquisition software. Consequently a solution to the ranging problem has been based on second-guessing what was intended rather than what was implemented in the original software. It is anticipated that any errors in range are less than 1 km. Unfortunately there is no independent method of verifying the altitudes of M-mode range gates.

It is known that a hardware pre-processing unit prevents data from the middle range gates, nominally numbers 192 - 385 (i.e. a gap of 192 gates), from being passed to the data acquisition computer (since no useful radar returns are expected from this altitude region). It is also known that the number of the lowest recorded ST-mode range gate is determined by half the transmitted pulse length, i.e. range gate number 4 for standard 8 us observations, and is not 0 (as was expected). It seems likely that the same offset would apply to the M-mode so that the two altitude ranges are still separated by 192 range gates. The hardware range gate number is therefore related to apparent range gate number through the relationship:

2 hardware_range_gate_number = apparent_range_gate_number - m_mode_range_gate_offset

For dwells/cycles starting before 2007-02-06 13:44:03 UT (i.e. using the original data acquisition system), the following was assumed:
m_mode_range_gate_offset = 190

For dwells/cycles starting between 2007-02-06 13:44:03 UT and 2008-02-07 11:54:57 UT, the following was assumed:
m_mode_range_gate_offset = 194

For dwells/cycles starting after 2008-02-07 14:37:00 UT, the following has been assumed:
m_mode_range_gate_offset = 192 + int(length_of_transmitted_pulse_length_in_us/2)

i.e. m_mode_range_gate_offset = 196 for standard observations with a pulse length of 8 us. The range gate translation is carried out by a PROM, which will need to be investigated before any progress can be made on this. Owing to the lack of correlative data, there is no easy way to verify this any more at the present time.


Evidence for the problem
The fact that there was a problem became apparent almost as soon as the new radar control and data acquisition system began operations in February 2007. The monthly comparison statistics of wind-profile data against the Met Office model fields showed:

• there was an increased bias in the differences between radar and model wind (northward and eastward) components
• the magnitude of this bias had a maximum value around the tropopause level, i.e. where there is typically a sharp reduction in wind speed with increasing altitude
• the wind direction was largely unaffected
• there was a decrease in radar altitude coverage

Nevertheless, the cause for this was not immediately apparent, since the internal consistency of the radar observations remained high. However, once it was decided that the problems lay in the range gating, comparisons against Aberporth radiosonde wind-profile data demonstrated that shifting the range gates upwards lead to a better agreement. The initial offset of +7 range gates (used between 2007-02-06 13:44:03 UT and 2008-02-07 11:54:57 UT) was based on the presumption that the error lay in the decoding routine. This implied an offset determined by the length of the transmitted pulse in us (typically 8 us) less 1. However, after an exhaustive comparison of the old and new data acquisition codes, justification could only be found for an offset of +4 range gates. This arises owing to the fact that the number of the lowest ST-mode range gate for which data are recorded is not 0. It is determined by half the transmitted pulse length, i.e. range gate number 4 for standard 8 us observations. This fact is not recorded anywhere in the legacy documentation but is inferred from the original data acquisition code. The first of the plots below shows the correlation coefficients between radiosonde-derived and radar-derived wind speeds for various radar range gate offsets. Thus uses 147 radiosondes launched from Aberporth, approximately 45 km to the SW of the MST radar site, between June and December 2007. The radiosonde sample closest to each radar range gate was used. Radar winds are averages over 1 hour starting from the time of the radiosonde launch. A radar range gate offset of 0 implies that no attempt has been made to compensate for the altitude error. The peak correlation coefficient would apparently be associated with an offset of between +4 and +5 range gates. This provides a high degree of confidence in the value of +4 chosen.
The second plot shows the relationship between radiosonde-derived and radar-derived wind speeds assuming an offset of +4 range gates.