A quick guide – for all details see the Cookbook .
JCMT continuum data obtained with ACSIS are written to disk and available at the JCMT or via CADC.
For information on downloading and installing Starlink suite of data reduction and analysis software click here.
Raw JCMT heterodyne data obtained with ACSIS data files follow a fixed naming convention. For instance, within the file name “a20070420_00014_01_0001.sdf“, the sub-sections have the following meaning:
- “a”: indicates data has been obtained by ACSIS.
- “20070420”: Indicates the UT date on which the observation was take.
- “00014″: An index that uniquely identifies the observation within the night in question. So for instance, this file holds data for the 14th observation taken on the 20th April 2007.
- “01”: instruments with two sidebands and/or observations using multiple sub-systems this number identifies the sideband and/or sub-system.
- “0001“: The sub scan index i.e. if the are too much data to fit into one raw file.
- “.sdf”: Indicates the file holds data in the STARLINK NDF format.
To invoke the ORAC-DR software:
This sets up ORAC-DR to run in your current working directory. You will also need to specify where your raw data is stored. For tcsh users we do:
setenv ORAC_DATA_IN folder/
or for bash:
For help with ORAC-DR simply do:
To reduce your raw Heterodyne data using the ORAC-DR software run:
oracdr -loop file -file mylist.lis
where mylist.lis is a list of raw files (either for a specific wavelength, single/multiple observations/sub arrays) that should be located in the “folder” directory as specified by ORAC_DATA_IN. the raw data will be automatically reduced using a per-defined reduction recipe.
oracdr -loop file -file mylist.lis -nodisplay -log sf
The above is another way of running the ORAC-DR pipeline. In the above example the output will be written to screen and logged in a .oracdr_* file (as specified by -log sf s = screen f = file) and not displayed in an xwindow (as specified -nodisplay).
To really understand what is happening to your data it is advised to also (for the first few times/reductions) run with -verbose. This will print messages from the Starlink engines (rather than just ORAC-DR messages):
oracdr -loop file -file mylist.lis -nodisplay -log sf -verbose
If you have HARP data and one or two receptors are bad it is possible to exclude them (i.e H02 and H08 in this example) from a reduction by using a bad receptors mask by running:
oracdr -loop file -file mylist.lis -nodisplay -log sf -verbose -calib bad_receptors=\"H02:H08\"
Please note sometimes this will mean you are required to update the default Quality Assurance parameters as fewer pixels will remain in your map.
Data that have been reduced by the pipeline have Tsys and RMS for all receptors calculated, the pipeline also find and fits baselines, locates regions of emission using Clumpfind, creates moments maps, velocity maps and integrated intensity images. The output of the pipeline includes the following:
- .oracdr_*.log – The ORAC-DR log file
- log.group – The files contributing to each group
- a20140103_00043_01_cube001.sdf – baselined cube
- a20140103_00043_01_integ.sdf – Integrated intensity image
- a20140103_00043_01_rimg*.sdf – Representative image (same as integ file), used to form rimg PNG
- a20140103_00043_01_sp001.sdf – Spectrum taken from position of peak intensity in the integ file
- a20140103_00043_01_rsp*.sdf – Representative spectrum (same as sp001), used to form rsp PNG
- a20140103_00043_01_iwc.sdf – Intensity weighted co-ordinate image
- a20140103_00043_01_noise.sdf – Noise map
- a20140103_00043_01_reduced001.sdf – Final trimmed, baselined cube of the 1st (of n) file.
- a20140103_00043_01_rmslo.sdf – Low-frequency noise
- a20140103_00043_01_rmshi.sdf – High-frequency noise
- log.qa – Quality assurance reports
- log.noisestats – Noise statistics for each observation and group
Files beginning with prefix a are individual reductions of raw observations, the a simply identifies the data as being an ACSIS obervation. Files beginning with the prefix ga are coadded reductions of a number of observation (as determined by the files included in mylist.lis). The group files produced are as outlined above with the difference of the ga prefix and the omission of leading zero’s.
- ga20140103_43_1_reduced001.sdf – Combined baselined cube
- ga20140103_43_1_integ.sdf – Combined integrated intensity image
For larger cubes the reduced file is split into several subcubes with the numbers 001, 002, …, which can be combined using the KAPPA command PASTE.
paste g*.sdf out=fullmap
or then they can be combined into a single co-added map with the PICARD recipe MOSAIC_JCMT_IMAGES:
picard -log sf MOSAIC_JCMT_IMAGES *files.sdf
For greater versatility, the STARLINK KAPPA package contains many commands for visualising images in many different ways, including displaying multiple pictures in a grid, overlaying masks and contours, scatter plots, histograms, etc, etc.
Data can also be viewed in SPLAT (single spectra):
ORAC-DR is a pipeline that calls several STARLINK commands in order to reduce your data. the precise behavior of the pipeline is governed by the recipe file. You can find out which recipe is set in the data header via the FITS header RECIPE keyword in any of your raw files. For example both of these options will return the same result:
>> kappa >> fitsval ga20140103_43_1_reduced001.sdf RECIPE
>> fitslist ga20140103_43_1_reduced001.sdf | grep RECIPE
It is possible to run with a different recipe (i.e. REDUCE_SCIENCE_GRADIENT). Links to specific reductions can be found in Appendix B – Main Recipes and are executed by entering the following:
>> oracdr -loop file -files myfiles.list REDUCE_SCIENCE_GRADIENT
You can tailor the recipe parameters by supplying a .ini file (called myparams.ini in the following example). This file contains the recipe name (which must match the one assigned to your data, whether from the header or any different one specified on the command line) followed by the options you wish to specify in the following format:
MOMENTS_LOWER_VELOCITY = -30.0
MOMENTS_UPPER_VELOCITY = 155.0
PIXEL_SCALE = 6.0,10.0
SPREAD_METHOD = gauss
SPREAD_WIDTH = 9
SPREAD_FWHM_OR_ZERO = 6
REBIN = 2
For more information about running ORAC-DR click here.
Polarization-separated reduction (Nāmakanui specific)
The Nāmakanui instruments produce data from both sidebands and polarizations. It is strongly recommended by the observatory that users check the data for each polarisation separately before combining them to use the total intensity spectra. To do this, you can make use of the bad receptors mask, running: (instructions are for a bash-type shell)
# Invoke the ORAC-DR software oracdr_acsis # reduce only P0 data mkdir p0 ORAC_DATA_IN=$(pwd) ORAC_DATA_OUT=$(pwd)/p0 # define P1 as "bad receptor" # in the line below, NU** is for Uu data, NW** is for Aweoweo, and NA** is for Alaihi; echo "NW1L NW1U NU1L NU1U NA1L NA1U" >> $ORAC_DATA_OUT/bad_receptors.lis # call the reduction pipeline oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE
where the filelist.lis is a text file containing raw data files to be reduced (see Section “Quick Reduction” above for details on this and the other parameters). Similarly, to reduce P1:
# reduce only P1 mkdir p1 export ORAC_DATA_IN=$(pwd) export ORAC_DATA_OUT=$(pwd)/p1 # define P0 as "bad receptor" # in the line below, NU** is for Uu data, NW** is for Aweoweo, and NA** is for Alaihi; echo "NW0L NW0U NU0L NU0L NA0L NA0U" >> $ORAC_DATA_OUT/bad_receptors.lis # call the reduction pipeline oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE
and you will find your spectra in P0-only and P1-only under the folders p0/ and p1/, respectively. Note, for the receptors, N stands for Nāmakanui, the next letter is either A for Alaihi, U for ʻŪʻū or W for ʻĀweoweo. The number corresponds to polarization 0 or 1, and the final letter is L or U for Lower or Upper sideband.
Important note: These instructions to reduce each polarisation separately will not work for standard calibrator sources. ORAC-DR assigns the reduction recipe “REDUCE_STANDARD” to any observation with a ‘standard’ flag in it (i.e., standard calibrators). The REDUCE_STANDARD recipe, by default, ignores the “bad_recptors” file supplied to ORAC-DR. The way to fix this is to specify the recipe as “REDUCE_SCIENCE_NARROWLINE” when running the reduction as follows:
oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE REDUCE_SCIENCE_NARROWLINE
If you want to produce the polarization-combined spectra simply remove the parameter “-calib bad_receptors=FILE”:
# reduce both pols combined mkdir combined export ORAC_DATA_IN=$(pwd) export ORAC_DATA_OUT=$(pwd)/combined oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose
and you will obtain the spectra with both polarizations combined inside the combined/ folder.
If you have reduced several Heterodyne files that you wish to combine into a single co-added map, this can be done with the PICARD recipe MOSAIC_JCMT_IMAGES:
picard -recpars mypar.lis MOSAIC_JCMT_IMAGES *files.sdf
where the “parameter file” mypar.lis looks like the following, with a mosaic method specified (in this example makemos):
[MOSAIC_JCMT_IMAGES] MOSAIC_TASK = makemos MAKEMOS_METHOD = mean
Other useful commands
Header information can be checked with
kappa fitslist a20070420_00014_01_0001.sdf
Fitslist is a powerful tool and with it you can check particular attributes: Dates of observations
fitslist file | grep DATE-OBS fitslist file | grep DATE-END
fitslist file | grep TAU225ST TAU225EN fitslist file | grep WVMTAUST WVMTAUEN
smurf gettsys -statistics myfile #This command works for raw data only. hdstrace a20090718_00024_01_0001.more.tsys nlines=all fitslist file | grep MEDTSYS
Tracking receptor (HARP only):
fitslist file | grep INSTAP
fitslist file | grep SAM_MODE fitslist file | grep SW_MODE fitslist file | grep CHOP_FRQ CHOP_PA CHOP_THR
Bandwidth and resolution
fitslist file | grep BWMODE fitslist file | grep SUBBANDS fitslist file | grep NCHNSUBS
fitslist file | grep SKYREFX fitslist file | grep SKYREFY
fitslist file | grep SW_MODE fitslist file | grep CHOP_FRQ CHOP_PA CHOP_THR
fitslist file | grep INT_TIME
fitslist file | grep BACKEND
The positional information can be found with (add fullframe for even more info) ndftrace which displays the attributes of the data structure. This will tell you the units of the data, pixel bounds, dimensions and axis assignations.
kappa ndftrace a20070420_00014_01_0001.sdf ndftrace file fullframe fullwcs
other information in fits extensions can be accessed with
or in more detail for e.g. Trx with
hdstrace a20070420_00014_01_0001.more.acsis.trx nlines=all
Files contributing to a coadd can be examined via:
Nāmakanui Receptors – fitsheaders
To look at what receptors were working at the time – a particualrly useful trick for `Ū`ū or ʻĀweoweo, one can use fitsval or fitslist to look at the specific fitsheader “RECPTORS” (not a typo) e.g. All examples given below are for ʻŪʻū – Note that for ʻĀweoweo, NU*L and NU*U -> NW*L and NW*U (N stands for Nāmakanui, the next letter is either U for ʻŪʻū or W for ʻĀweoweo. The number corresponds to polarization 0 or 1, and the final letter is L or U for Lower or Upper sideband.)
fitslist a20201203_00124_01_0001.sdf | grep RECPTORS
RECPTORS= 'NU0L NU0U NU1L NU1U ' / Active FE receptor IDs for this observation
or by using fitsval e.g.
fitsval a20201203_00124_01_0001.sdf RECPTORS
NU0L NU0U NU1L NU1U
and also look at fitsheaders OBS_SB and TRACK_SB e.g.
OBS_SB = 'USB ' / The observed sideband (this subsystem) TRACK_SB= 'USB ' / The tracking sideband (primary subsystem)
after a reduction has been made one can look for the fitsheader RECPUSED to find out what receptors are in the reduced file:
RECPUSED= 'NU1U NU0U' / Used receptor IDs
Conversion to CLASS format
It is possible to convert the spectra to CLASS format, or the cubes to GDF files, readable by the IRAM GILDAS software package, see here.