{"id":8051,"date":"2018-01-25T17:59:05","date_gmt":"2018-01-26T03:59:05","guid":{"rendered":"http:\/\/www.eaobservatory.org\/jcmt\/?page_id=8051"},"modified":"2020-11-24T15:58:58","modified_gmt":"2020-11-25T01:58:58","slug":"starlink-analysis","status":"publish","type":"page","link":"https:\/\/www.eaobservatory.org\/jcmt\/science\/reductionanalysis-tutorials\/starlink-analysis\/","title":{"rendered":"Tutorial: Analysis with Starlink"},"content":{"rendered":"<div id=\"toc_container\" class=\"no_bullets\"><p class=\"toc_title\">Contents<\/p><ul class=\"toc_list\"><li><a href=\"#Get_the_data\"><span class=\"toc_number toc_depth_1\">1<\/span> Get the data:<\/a><\/li><li><a href=\"#Start_Starlink\"><span class=\"toc_number toc_depth_1\">2<\/span> Start Starlink<\/a><\/li><li><a href=\"#Getting_started_with_your_data\"><span class=\"toc_number toc_depth_1\">3<\/span> Getting started with your data<\/a><ul><li><a href=\"#Task_Convert_FITS_cube_to_Radio_Velocity_SDF\"><span class=\"toc_number toc_depth_2\">3.1<\/span> Task: Convert FITS cube to Radio Velocity SDF<\/a><\/li><li><a href=\"#Task_Examining_what_is_in_your_file\"><span class=\"toc_number toc_depth_2\">3.2<\/span> Task: Examining what is in your file<\/a><\/li><li><a href=\"#Task_plot_the_scan_pattern_of_a_SCUBA-2_map\"><span class=\"toc_number toc_depth_2\">3.3<\/span> Task: plot the scan pattern of a SCUBA-2 map<\/a><\/li><\/ul><\/li><li><a href=\"#Measuring_Noise_in_the_Map\"><span class=\"toc_number toc_depth_1\">4<\/span> Measuring Noise in the Map<\/a><ul><li><a href=\"#Task_Comparing_noise_estimates\"><span class=\"toc_number toc_depth_2\">4.1<\/span> Task: Comparing noise estimates<\/a><\/li><li><a href=\"#Task_Regrid_map_and_check_the_noise\"><span class=\"toc_number toc_depth_2\">4.2<\/span> Task: Regrid map and check the noise<\/a><\/li><li><a href=\"#Task_measure_the_noise_in_a_map\"><span class=\"toc_number toc_depth_2\">4.3<\/span> Task: measure the noise in a map<\/a><\/li><\/ul><\/li><li><a href=\"#Masking_Extracting_and_Trimming\"><span class=\"toc_number toc_depth_1\">5<\/span> Masking, Extracting and Trimming<\/a><ul><li><a href=\"#Task_Identifying_the_NDF_pixel_coordinates_of_a_region_with_Gaia\"><span class=\"toc_number toc_depth_2\">5.1<\/span> Task: Identifying the NDF pixel coordinates of a region with Gaia<\/a><\/li><li><a href=\"#Task_extract_a_region_using_WCS_coordinates\"><span class=\"toc_number toc_depth_2\">5.2<\/span> Task: extract a region using WCS coordinates.<\/a><\/li><li><a href=\"#Task_CROP_a_SCUBA-2_image_with_PICARD\"><span class=\"toc_number toc_depth_2\">5.3<\/span> Task: CROP a SCUBA-2 image with PICARD<\/a><\/li><li><a href=\"#Task_ARD_Masking\"><span class=\"toc_number toc_depth_2\">5.4<\/span> Task: ARD Masking<\/a><\/li><li><a href=\"#Task_masking_based_on_SNR\"><span class=\"toc_number toc_depth_2\">5.5<\/span> Task: masking based on SNR<\/a><\/li><\/ul><\/li><li><a href=\"#Extracting_Values\"><span class=\"toc_number toc_depth_1\">6<\/span> Extracting Values<\/a><ul><li><a href=\"#Task_Using_pluck_to_extract_spectra\"><span class=\"toc_number toc_depth_2\">6.1<\/span> Task: Using pluck to extract spectra<\/a><\/li><\/ul><\/li><li><a href=\"#Coordinate_conversion\"><span class=\"toc_number toc_depth_1\">7<\/span> Coordinate conversion<\/a><ul><li><a href=\"#Task_Convert_a_map_to_galactic_coordinates\"><span class=\"toc_number toc_depth_2\">7.1<\/span> Task: Convert a map to galactic coordinates<\/a><\/li><li><a href=\"#Task_Align_one_map_with_another\"><span class=\"toc_number toc_depth_2\">7.2<\/span> Task: Align one map with another.<\/a><\/li><\/ul><\/li><li><a href=\"#Fitting_Spectra\"><span class=\"toc_number toc_depth_1\">8<\/span> Fitting Spectra<\/a><ul><li><a href=\"#Task_use_fit1d_to_fit_your_entire_cube\"><span class=\"toc_number toc_depth_2\">8.1<\/span> Task: use fit1d to fit your entire cube<\/a><\/li><\/ul><\/li><li><a href=\"#Clumpfinding\"><span class=\"toc_number toc_depth_1\">9<\/span> Clumpfinding<\/a><ul><li><a href=\"#Task_Use_fellwalker_to_find_clumps\"><span class=\"toc_number toc_depth_2\">9.1<\/span> Task: Use fellwalker to find clumps.<\/a><\/li><\/ul><\/li><\/ul><\/div>\n\n<p><strong>Note:<\/strong> <em>This tutorial assumes that the computer to be used already has a functioning installation of the latest version of the Starlink software suite (<a href=\"http:\/\/starlink.eao.hawaii.edu\/starlink\/Releases\">the latest release is available for download here<\/a>).<\/em><\/p>\n<p>This tutorial aims to go through some of the many analysis tools within Starlink.<\/p>\n<h2><span id=\"Get_the_data\">Get the data:<\/span><\/h2>\n<p>Download the tarball from <a href=\"http:\/\/ftp.eao.hawaii.edu\/jcmt\/usersmeetings\/JCMT_tutorial_2018_Starlink_Analysis.tar.gz\">HERE<\/a>. This contains<\/p>\n<ol>\n<li>\u00a0An archival JCMT CO 3-2 HARP reduced map:<br \/>\n<tt>jcmth20061220_00044_01_reduced001_nit_000.fits<\/tt><\/li>\n<li>\u00a0A reduced 850um SCUBA-2 map of G34.3, as produced in the Basic SCUBA-2 DR tutorial.<br \/>\n<tt>scuba2_850.sdf<\/tt><\/li>\n<li>A TST catalogue file showing the JCMT state information from a SCUBA-2 raw obsrvation:<br \/>\n<tt>jcmtstate_s20120501_00068.TST<\/tt><\/li>\n<\/ol>\n<h2><span id=\"Start_Starlink\">Start Starlink<\/span><\/h2>\n<p>If using BASH\/ZSH:<\/p>\n<pre>export STARLINK_DIR=\/path\/to\/starlink\/directory\/star-2017A\r\nsource $STARLINK_DIR\/etc\/profile<\/pre>\n<p>If using TCSH\/CSH:<\/p>\n<pre>setenv STARLINK_DIR \/path\/to\/starlink\/directory\/star-2017A\r\nsource $STARLINK_DIR\/etc\/login\r\nsource $STARLINK_DIR\/etc\/profile<\/pre>\n<h2><span id=\"Getting_started_with_your_data\">Getting started with your data<\/span><\/h2>\n<h3><span id=\"Task_Convert_FITS_cube_to_Radio_Velocity_SDF\">Task: Convert FITS cube to Radio Velocity SDF<\/span><\/h3>\n<p>Take a HARP cube downloaded from the JCMT Science Archive in FITS format, convert it back to a <tt>.sdf<\/tt> NDF file, and then change the barycentric frequency axis into a radio velocity axis with the standard of rest set to LSRK.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d8ce9\"  tabindex=\"0\" title=\"Click to show worked solution\"    >Click to show worked solution<\/span><div id=\"target-id69db70e1d8ce9\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>The first file we are going to look at is an archival HARP map <tt>jcmth20061220_00044_01_reduced001_nit_000.fits<\/tt><\/p>\n<p>This is a 2006 reduced cube containing a CO 3-2 HARP observation of Ros 6.<\/p>\n<p>The first thing we will do is convert it from FITS format into NDF &#8212; the Starlink data format. To do this we need to inialise the convert package:<\/p>\n<pre>$convert\r\nCONVERT commands are now available -- (Version 1.8)\r\n \r\n Defaults for automatic NDF conversion are set.\r\n \r\n Type conhelp for help on CONVERT commands.\r\n Type \"showme sun55\" to browse the hypertext documentation.<\/pre>\n<p>We then run the command fits2ndf:<\/p>\n<pre><tt>$fits2ndf jcmth20061220_00044_01_reduced001_nit_000.fits co32_cube.sdf<\/tt><\/pre>\n<p>JCMT archive data uses Barycentric Frequency as its frequency coordinate system; use <tt>ndftrace<\/tt> to examine this. First of all you will need to initialise the package by typing <tt>kappa<\/tt>. Then, we will use <tt>ndftrace<\/tt> to see what is in this file.<\/p>\n<p>(Throughout this tutorial we often will show you the output of the commands. To make it clear what is output vs what is input this tutorial puts a <tt>$<\/tt> at the start of the command you need to type, indicating the shell prompt.)<\/p>\n<pre>$ ndftrace co32_cube.sdf\r\n\r\n   NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/co32_cube:\r\n      Title:  Ros6\r\n      Label:  TA*   corrected antenna temperature\r\n      Units:  K\r\n\r\n   Shape:\r\n      No. of dimensions:  3\r\n      Dimension size(s):  119 x 118 x 1880\r\n      Pixel bounds     :  -57:61, -58:59, -940:939\r\n      Total pixels     :  26398960\r\n\r\n   Data Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   Variance Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   World Co-ordinate Systems:\r\n      Number of co-ordinate Frames: 5\r\n\r\n      Current co-ordinate Frame (Frame 5):\r\n\r\n        Frame title         : \"3-d compound coordinate system\"\r\n        Domain              : SKY-DSBSPECTRUM\r\n        First pixel centre  : 6:35:01.2, 4:06:50, 346.2167\r\n\r\n           Axis 1:\r\n              Label              : Right ascension\r\n              Units              : hh:mm:ss.s\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 2:\r\n              Label              : Declination\r\n              Units              : ddd:mm:ss\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 3:\r\n              Label              : Frequency (USB)\r\n              Units              : GHz\r\n              Nominal Pixel scale: 0.0004882367 GHz\r\n\r\n\r\n   Extensions:\r\n      FITS             &lt;_CHAR*80&gt;\r\n      SMURF            \r\n      PROVENANCE       \r\n\r\n\r\n   History Component:\r\n      Created    :  2014 Nov 20 09:22:05\r\n      No. records:  20\r\n      Last update:  2014 Nov 20 09:23:59 (FITSMOD         (KAPPA 2.2))\r\n      Update mode:  NORMAL\r\n\r\n<\/pre>\n<p>This shows you the label, units, number and shape of the dimensions, some of the coordinate values, and also shows you the extensions in the NDF file.<\/p>\n<p>You can see that the third axis is Frequency, in units of GHz. We can check what standard of rest it is using the <tt>wcsattrib<\/tt> command:<\/p>\n<pre>$ wcsattrib co32_cube.sdf mode=get name=StdofRest\r\nBarycentric\r\n<\/pre>\n<p>We are going to change the third axis to be in Radio Velocity, and we are also going to change to the LSRK standard of rest, again using the <tt>wcsattrib<\/tt> but this time in <tt>mode=set<\/tt>. To see the list of WCS information you can see\/alter with <tt>wcsattrib<\/tt>, refer to the KAPPA user note (SUN\/95) at <a href=\"http:\/\/www.starlink.ac.uk\/cgi-bin\/htxserver\/sun95.htx\/sun95.html?xref_ap_frmatt\">www.starlink.ac.uk\/cgi-bin\/htxserver\/sun95.htx\/sun95.html?xref_ap_frmatt<\/a><\/p>\n<pre># Change to radio velocity (vrad)  and the LSRK std of rest.\r\nwcsattrib ndf=co32_cube.sdf mode=set name=system newval=vrad\r\nwcsattrib ndf=co32_cube.sdf mode=set name=StdofRest newval=LSRK\r\n<\/pre>\n<p>If we look at again at our file with ndftrace, you can see the changes to the 3rd axis:<\/p>\n<pre>$ ndftrace co32_cube.sdf\r\n\r\n   NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/co\r\n32cube:\r\n      Title:  Ros6\r\n      Label:  TA*   corrected antenna temperature\r\n      Units:  K\r\n\r\n   Shape:\r\n      No. of dimensions:  3\r\n      Dimension size(s):  119 x 118 x 1880\r\n      Pixel bounds     :  -57:61, -58:59, -940:939\r\n      Total pixels     :  26398960\r\n\r\n   Data Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   Variance Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   World Co-ordinate Systems:\r\n      Number of co-ordinate Frames: 5\r\n\r\n      Current co-ordinate Frame (Frame 5):\r\n\r\n        Frame title         : \"3-d compound coordinate system\"\r\n        Domain              : SKY-DSBSPECTRUM\r\n        First pixel centre  : 6:35:01.2, 4:06:50, -381.1352\r\n\r\n           Axis 1:\r\n              Label              : Right ascension\r\n              Units              : hh:mm:ss.s\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 2:\r\n              Label              : Declination\r\n              Units              : ddd:mm:ss\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 3:\r\n              Label              : Radio velocity (USB)\r\n              Units              : km\/s\r\n              Nominal Pixel scale: 0.4233065 km\/s\r\n\r\n\r\n   Extensions:\r\n      FITS             &lt;_CHAR*80&gt;\r\n      SMURF            \r\n      PROVENANCE       \r\n\r\n\r\n   History Component:\r\n      Created    :  2014 Nov 20 09:22:05\r\n      No. records:  23\r\n      Last update:  2018 Jan 25 20:03:17 (NDFCOPY         (KAPPA 2.5-5))\r\n      Update mode:  NORMAL\r\n<\/pre>\n<p>You can see the 3rd axis in now in units of km\/s, with a pixel scale 0.423 km\/s per channel.<br \/>\n<\/div>\n<h3><span id=\"Task_Examining_what_is_in_your_file\">Task: Examining what is in your file<\/span><\/h3>\n<p>Looking at the cube file you produced above, identify the molecule and transition, the original date of the observation and what. Find out if the autogrid option was used in the original makecube command that gridded the data, and when that command was run. Find out all the original observations and files that were included in the file. Finally, see what extensions are present in your data file.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d8d4c\"  tabindex=\"0\" title=\"Click to show worked solution\"    >Click to show worked solution<\/span><div id=\"target-id69db70e1d8d4c\" class=\"collapseomatic_content \">\n<strong>Worked Example<\/strong><\/p>\n<p>We&#8217;ve already used KAPPA&#8217;s <tt>wcsattrib<\/tt> and <tt>ndftrace<\/tt> above to look at what is in your file. There are several other commands that can be helpful.<\/p>\n<ol>\n<li>\u00a0<tt>fitslist<\/tt> This command shows you the contents of the FITS extension of the NDF. This is where we store information about the observation the file was produced by (including integration time, scan pattern, tracking system) , the telescope information, and the environmental information.\n<pre>$ fitslist ..\/Starlink_Analysis\/co32_cube.sdf<\/pre>\n<\/li>\n<li>\n<pre>SIMPLE  =                    T \/ file does conform to FITS standard\r\nBITPIX  =                  -32 \/ number of bits per data pixel\r\nNAXIS   =                    3 \/ number of data axes\r\nNAXIS1  =                  119 \/ length of data axis 1\r\nNAXIS2  =                  118 \/ length of data axis 2\r\nNAXIS3  =                 1880 \/ length of data axis 3\r\nEXTEND  =                    T \/ FITS dataset may contain extensions\r\nCOMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy\r\nCOMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&amp;A...376..359H\r\nLBOUND1 =                  -57 \/ Pixel origin along axis 1\r\nLBOUND2 =                  -58 \/ Pixel origin along axis 2\r\nLBOUND3 =                 -940 \/ Pixel origin along axis 3\r\nLABEL   = 'T%+%+A%^50+%NUMTILES)\r\nINSTREAM= 'JCMT    '           \/ Source of input stream\r\nDPDATE  = '2014-11-20T09:23:54' \/ Data processing date\r\nDPRCINST= 'jac-92883'          \/ Data processing recipe instance ID\r\n\r\n\r\n                               \/ Provenance:\r\nPRVCNT  =                    6 \/ Number of parents\r\nPRV1    = 'a20061220_00044_01_0001' \/ Name of the 1st parent\r\nPRV2    = 'a20061220_00044_01_0002' \/ Name of the 2nd parent\r\nPRV3    = 'a20061220_00044_01_0003' \/ Name of the 3rd parent\r\nPRV4    = 'oractempuMu_kY'     \/ Name of the 4th parent\r\nPRV5    = 'oractempEjA5Sw'     \/ Name of the 5th parent\r\nPRV6    = 'oractempSgkQoo'     \/ Name of the 6th parent\r\nOBSCNT  =                    1 \/ Number of root-ancestor headers\r\nOBS1    = 'acsis_44_20061220T095454_1' \/ Name of the 1st root ancestor\r\nFILEID  = 'jcmth20061220_00044_01_reduced001_nit_000' \/ Filename minus extension\r\n\r\nCHECKSUM= '0a7F3T6D0Z6D0Z6D'   \/ HDU checksum updated 2014-11-20T09:24:00\r\nDATASUM = '1474200105'         \/ data unit checksum updated 2014-11-20T09:24:00\r\nEND\r\n<\/pre>\n<p>You can also get the information from a specific header with &#8216;fitsval&#8217;. E.g. to find the value of the instrume keyword<\/p>\n<pre>$ fitsval co32_cube.sdf keyword=instrume\r\nHARP\r\n<\/pre>\n<\/li>\n<li><tt>hislist<\/tt> will show you the Starlink commands that were carried out on the file. This is very useful for checking that you have done what you thought you did to a file, or when you come back to work again on files after a long pause.\n<pre>$ hislist co32_cube.sdf\r\n\r\n   History listing for NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/co32_cube:\r\n\r\n   History structure created 2014 Nov 20 09:22:05.443\r\n\r\n1: 2014 Nov 20 09:22:05.538 - MAKECUBE        (SMURF V1.6.0)\r\n\r\n   Parameters: ALIGNSYS=FALSE AUTOGRID=TRUE BADMASK='and' CROTA=0\r\n      DETECTORS=! FBL=[1.7235973391403,0.071799010054101]\r\n      FBR=[1.7201560002942,0.071799006183432]\r\n      FLBND=[1.7201409775556,0.07178446001926,-381.34682629178]\r\n      FUBND=[1.7236123539655,0.07521704716548,414.46932659916]\r\n      FTL=[1.7235977665515,0.075202393861675]\r\n      FTR=[1.7201555650017,0.075202389806852] GENVAR='tsys'\r\n      IN=@^oractempXdGhiP INWEIGHT=TRUE JSATILES=FALSE LBND=[-57,-58]\r\n      LBOUND=[-57,-58,-940] NPOLBIN=1 NTILE=1 OUT=@ga20061220_44_1_cube\r\n      OUTCAT=! PIXREF=[60.461,60.045] PIXSIZE=6 POSERRFATAL=FALSE REF=!\r\n      REFLAT='4:12:43.9' REFLON='6:34:37.72' REFPIX1=60.461 REFPIX2=60.045\r\n      SPARSE=FALSE SPECBOUNDS='-381.347,414.469' SPECUNION=TRUE\r\n      SPREAD='NEAREST' SYSTEM='tracking' TILEBORDER=0 TILEDIMS=-256\r\n      TRIM=FALSE TRIMTILES=TRUE UBND=[61,59] UBOUND=[61,59,939]\r\n      USEDETPOS=FALSE WEIGHTS=FALSE MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/smurf\/smurf_mon\r\n   Group: IN=\"a20061220_00044_00_bl001, a20061220_00044_00_bl002,\r\n      a20061220_00044_00_bl003\"\r\n\r\n2: 2014 Nov 20 09:22:22.324 - FITSMOD         (KAPPA 2.2)\r\n\r\n   Parameters: COMMENT='$' EDIT='update' EXISTS=TRUE KEYWORD='DATE-OBS'\r\n      MODE='Interface' NDF=@ga20061220_44_1_cube001 POSITION=!\r\n      READONLY=FALSE STRING=TRUE VALUE='2006-12-20T09:54:54' MSG_FILTER=!\r\n      QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n3: 2014 Nov 20 09:22:22.374 - FITSMOD         (KAPPA 2.2)\r\n\r\n   Parameters: COMMENT='$' EDIT='update' EXISTS=TRUE KEYWORD='DATE-END'\r\n      MODE='Interface' NDF=@ga20061220_44_1_cube001 POSITION=!\r\n      READONLY=FALSE STRING=TRUE VALUE='2006-12-20T10:33:33' MSG_FILTER=!\r\n      QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n4: 2014 Nov 20 09:22:22.884 - ORAC-DR - _CREATE_CUBE_GROUP_ (29bb4c)\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_cube001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n5: 2014 Nov 20 09:22:34.018 - SUB             (KAPPA 2.2)\r\n\r\n   Parameters: IN1=@ga20061220_44_1_cube001 IN2=@oractempS28Rf4\r\n      OUT=@ga20061220_44_1_bl001 TITLE=! MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/kappa_mon\r\n\r\n6: 2014 Nov 20 09:22:34.759 - SETVAR          (KAPPA 2.2)\r\n\r\n   Parameters: COMP='VARIANCE' FROM=@ga20061220_44_1_cube001\r\n      NDF=@ga20061220_44_1_bl001 MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n7: 2014 Nov 20 09:22:35.894 - ORAC-DR - _REMOVE_BASELINE_THROUGH_SMOOTHING_\r\n...\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_bl001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n8: 2014 Nov 20 09:22:36.027 - SETBB           (KAPPA 2.2)\r\n\r\n   Parameters: AND=FALSE BB='255' NDF=@ga20061220_44_1_bl001 OR=FALSE\r\n      MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n9: 2014 Nov 20 09:22:36.328 - NDFCOPY         (KAPPA 2.2)\r\n\r\n   Parameters: COMP='DATA' EXTEN=FALSE IN=@ga20061220_44_1_bl001 LIKE=!\r\n      OUT=@ga20061220_44_1_reduced001 TITLE=! TRIM=FALSE TRIMBAD=FALSE\r\n      TRIMWCS=TRUE MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n10: 2014 Nov 20 09:22:38.077 - ORAC-DR - _TAG_AS_REDUCED_PRODUCT_ (29bb4c)\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_reduced001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n11: 2014 Nov 20 09:22:38.215 - SETBB           (KAPPA 2.2)\r\n\r\n   Parameters: AND=FALSE BB='0' NDF=@ga20061220_44_1_reduced001 OR=FALSE\r\n      MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n12: 2014 Nov 20 09:22:41.082 - SETBB           (KAPPA 2.2)\r\n\r\n   Parameters: AND=FALSE BB='255' NDF=@ga20061220_44_1_reduced001 OR=FALSE\r\n      MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n13: 2014 Nov 20 09:23:43.241 - ORAC-DR -\r\n_CREATE_MOMENTS_MAPS_THROUGH_SMOOTH...\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_reduced001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n14: 2014 Nov 20 09:23:44.445 - SETBB           (KAPPA 2.2)\r\n\r\n   Parameters: AND=FALSE BB='0' NDF=@ga20061220_44_1_reduced001 OR=FALSE\r\n      MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n15: 2014 Nov 20 09:23:44.847 - SETBB           (KAPPA 2.2)\r\n\r\n   Parameters: AND=FALSE BB='255' NDF=@ga20061220_44_1_reduced001 OR=FALSE\r\n      MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/ndfpack_mon\r\n\r\n16: 2014 Nov 20 09:23:48.025 - ORAC-DR -\r\n_CREATE_MOMENTS_MAPS_THROUGH_SMOOTH...\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_reduced001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n17: 2014 Nov 20 09:23:49.451 - ORAC-DR - _CREATE_NOISE_MAP_ (29bb4c)\r\n\r\n   Astro::FITS::Header::NDF - write FITS header to file\r\n      \/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014\r\n      -11 -20_08-49-03\/ga20061220_44_1_reduced001\r\n   Arguments: -nodisplay -log hs -verbose -recsuffix ADV,CADC\r\n      -batch -loop file -file \/tmp\/7hzdep6Ydp -recpars\r\n      recpars-M06BH23B.ini\r\n   Software:\r\n      \/net\/kamaka\/export\/data\/stardev-stable\/bin\/oracdr\/src\/b\r\n      in\/ oracdr\r\n\r\n18: 2014 Nov 20 09:23:59.727 - jsawrapdr\r\n\r\n   Arguments: --debugxfer\r\n      --outdir=\/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014-11-2\r\n      8-49-03\r\n      --inputs=\/net\/kamaka\/export\/data\/jsa_proc\/input\/000\/000092\/000092883\/\r\n      ut_files_job.lis --id=jac-92883 --mode=night --cleanup=cadc\r\n      --drparameters=-recpars recpars-M06BH23B.ini -persist\r\n      --transdir=\/net\/kamaka\/export\/data\/jsa_proc\/output\/000\/000092\/0000928\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/Perl\/bin\/jsawrapdr\r\n\r\n19: 2014 Nov 20 09:23:59.772 - WCSATTRIB       (KAPPA 2.2)\r\n\r\n   Parameters: MODE='MSet' NDF=@ga20061220_44_1_reduced001.sdf REMAP=TRUE\r\n      SETTING='System(1)=FK5,System(3)=FREQ,StdOfRest=BARY' MSG_FILTER=!\r\n      QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/wcsattrib\r\n\r\n20: 2014 Nov 20 09:23:59.835 - FITSMOD         (KAPPA 2.2)\r\n\r\n   Parameters: EXISTS=TRUE MODE='File' NDF=@ga20061220_44_1_reduced001.sdf\r\n      READONLY=FALSE TABLE=@\/tmp\/PjQSB6jkf9 MSG_FILTER=! QUIET=!\r\n   Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/kappa\/fitsmod\r\n\r\n21: 2018 Jan 25 22:26:19.487 - WCSATTRIB       (KAPPA 2.5-5)\r\n\r\n   Parameters: MODE='set' NAME='system' NDF=@co32_cube.sdf NEWVAL='vrad'\r\n      REMAP=TRUE STATE=FALSE VALUE='Compound' MSG_FILTER=! QUIET=!\r\n   Software: \/star\/bin\/kappa\/wcsattrib\r\n\r\n22: 2018 Jan 25 22:26:20.955 - WCSATTRIB       (KAPPA 2.5-5)\r\n\r\n   Parameters: MODE='set' NAME='StdofRest' NDF=@co32_cube.sdf NEWVAL='LSRK'\r\n      REMAP=TRUE STATE=TRUE VALUE='Barycentric' MSG_FILTER=! QUIET=!\r\n   Software: \/star\/bin\/kappa\/wcsattrib\r\n\r\n\r\n\r\n<\/pre>\n<p>Can you see the 2 wcsattrib commands you just ran on the file?<\/li>\n<li><tt>provshow<\/tt> lets you see the NDF files that were used to create the file you are exmaining. Its mostly commonly used to check which observations you included in a coadded file.\n<pre>$ provshow co32_cube.sdf \r\n\r\n0: \/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/co32_cube\r\n   Parents:  1,2,3,6,5,4\r\n   Date:  2014-11-20 09:23:32\r\n   Creator:  KAPPA:NDFCOPY\r\n\r\n1: a20061220_00044_01_0001.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n   More:  OBSIDSS=acsis_44_20061220T095454_1\r\n\r\n2: a20061220_00044_01_0002.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n   More:  OBSIDSS=acsis_44_20061220T095454_1\r\n\r\n3: a20061220_00044_01_0003.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n   More:  OBSIDSS=acsis_44_20061220T095454_1\r\n\r\n4: .\/oractempSgkQoo.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n\r\n5: .\/oractempEjA5Sw.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n\r\n6: .\/oractempuMu_kY.sdf\r\n   Parents:  \r\n   Date:  \r\n   Creator:  \r\n\r\n<\/pre>\n<p>Can you see which observation was included. How many raw files went into this cube?<\/li>\n<li><tt>hdstrace<\/tt> shows you the entire structure of the file. This lets you see any extensions included in the data.\n<pre>$ hdstrace co32_cube.sdf \r\n\r\nCO32_CUBE  \r\n\r\n   DATA_ARRAY              {structure}\r\n      ORIGIN(3)      &lt;_INTEGER&gt;      -57,-58,-940\r\n      DATA(119,118,1880)  &lt;_REAL&gt;    *,*,*,-3.457747,1.411959,-2.120131,0.4018285,-1.767746,-0.7077882,1.616902,0.9329623,-1.961738,0.8053908,1.151465,6.122407,2.263416,-3.084188,-0.8098075,0.4601...\r\n                                     ... 0.4417757,3.771597,-0.383148,1.626338,-0.6567043,-2.779671,2.554621,-1.354576,1.24353,-2.09212,-0.320193,1.203157,-2.560659,0.1859031,*,-0.1394619,-4.89816...\r\n\r\n   HISTORY               {structure}\r\n      CREATED        &lt;_CHAR*24&gt;      '2014-NOV-20 09:22:05.443'\r\n      UPDATE_MODE    &lt;_CHAR*8&gt;       'NORMAL'\r\n      CURRENT_RECORD  &lt;_INTEGER&gt;     22\r\n      RECORDS(25)          {array of structures}\r\n\r\n      Contents of RECORDS(1)\r\n         DATE           &lt;_CHAR*24&gt;      '2014-NOV-20 09:22:05.538'\r\n         COMMAND        &lt;_CHAR*30&gt;      'MAKECUBE        (SMURF V1.6.0)'\r\n         USER           &lt;_CHAR*7&gt;       'jsaproc'\r\n         HOST           &lt;_CHAR*6&gt;       'ikaika'\r\n         DATASET        &lt;_CHAR*70&gt;      '\/export\/data\/jsa_proc\/scratch\/000\/000092\/000092883\/2014-11-20_08-49...'\r\n         TEXT(18)       &lt;_CHAR*72&gt;      'Parameters: ALIGNSYS=FALSE AUTOGRID=TRUE BADMASK='and' CROTA=0','   DETECTORS=! FBL=[1.7235973391403,0.071799010054101]','   FBR=[1.7201560002942,0.0717990...\r\n                                        ... '   USEDETPOS=FALSE WE...','Software: \/net\/kamaka\/export\/data\/stardev-stable\/bin\/smurf\/smurf_mon','Group: IN=\"a20061220_00044_00_bl001, a20061220_00044_...\r\n\r\n   MORE                      {structure}\r\n      FITS(273)      &lt;_CHAR*80&gt;      'SIMPLE  =                    T \/ file does conform to FITS standard','BITPIX  =                  -32 \/ number of bits per data pixel','NAXIS   =\r\n                                     ... 'FILEID  = 'jcmth20061220...','','CHECKSUM= '0a7F3T6D0Z6D0Z6D'   \/ HDU checksum updated 2014-11-20T09:24:00','DATASUM = '1474200105'         \/ data unit ch...\r\n      SMURF               {structure}\r\n         EXP_TIME                  {structure}\r\n            DATA_ARRAY              {structure}\r\n               ORIGIN(2)      &lt;_INTEGER&gt;      -57,-58\r\n               DATA(119,118)  &lt;_REAL&gt;         *,*,*,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05...\r\n                                              ... 2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2.05,2...\r\n\r\n            TITLE          &lt;_CHAR*4&gt;       'Ros6'\r\n            LABEL          &lt;_CHAR*19&gt;      'Total exposure time'\r\n            UNITS          &lt;_CHAR*1&gt;       's'\r\n            WCS                       {structure}\r\n               DATA(178)      &lt;_CHAR*32&gt;      ' Begin FrameSet',' Nframe = 5',' Currnt = 5',' Lnk2 = 1',' Lnk3 = 1',' Lnk4 = 1',' Lnk5 = 1',' Frm1 =',' Begin Frame',' Naxes = 2',' Domain = \"GRID\"'...\r\n                                              ' Nout = 2',' Invert = 0',' IsA Mapping',' UntRd = 1',' PlrLg = 1.72189238902213',' End SphMap',' End CmpMap',' End CmpMap',' End CmpMap',' End CmpMap...\r\n\r\n         EFF_TIME                  {structure}\r\n            DATA_ARRAY              {structure}\r\n               ORIGIN(2)      &lt;_INTEGER&gt;      -57,-58\r\n               DATA(119,118)  &lt;_REAL&gt;         *,*,*,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.38...\r\n                                              ... 0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,0.3804878,*,0.3804878,0.76...\r\n\r\n            TITLE          &lt;_CHAR*4&gt;       'Ros6'\r\n            LABEL          &lt;_CHAR*26&gt;      'Effective integration time'\r\n            UNITS          &lt;_CHAR*1&gt;       's'\r\n            WCS                       {structure}\r\n               DATA(178)      &lt;_CHAR*32&gt;      ' Begin FrameSet',' Nframe = 5',' Currnt = 5',' Lnk2 = 1',' Lnk3 = 1',' Lnk4 = 1',' Lnk5 = 1',' Frm1 =',' Begin Frame',' Naxes = 2',' Domain = \"GRID\"'...\r\n                                              ' Nout = 2',' Invert = 0',' IsA Mapping',' UntRd = 1',' PlrLg = 1.72189238902213',' End SphMap',' End CmpMap',' End CmpMap',' End CmpMap',' End CmpMap...\r\n\r\n         TSYS                      {structure}\r\n            DATA_ARRAY              {structure}\r\n               ORIGIN(2)      &lt;_INTEGER&gt;      -57,-58\r\n               DATA(119,118)  &lt;_REAL&gt;         *,*,*,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,388.4387,...\r\n                                              ... 371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,371.6322,*,371.6322,...\r\n\r\n            TITLE          &lt;_CHAR*4&gt;       'Ros6'\r\n            LABEL          &lt;_CHAR*28&gt;      'Effective system temperature'\r\n            UNITS          &lt;_CHAR*1&gt;       'K'\r\n            WCS                       {structure}\r\n               DATA(178)      &lt;_CHAR*32&gt;      ' Begin FrameSet',' Nframe = 5',' Currnt = 5',' Lnk2 = 1',' Lnk3 = 1',' Lnk4 = 1',' Lnk5 = 1',' Frm1 =',' Begin Frame',' Naxes = 2',' Domain = \"GRID\"'...\r\n                                              ' Nout = 2',' Invert = 0',' IsA Mapping',' UntRd = 1',' PlrLg = 1.72189238902213',' End SphMap',' End CmpMap',' End CmpMap',' End CmpMap',' End CmpMap...\r\n\r\n      PROVENANCE         {structure}\r\n         DATA(345)      &lt;_INTEGER&gt;      2,7,1598506062,1280070990,825242112,825306420,540029485,842676528,842218035,1095434240,977358928,1128678478,5853263,1481880154,-1,6,1,2,3,6,5,4,808464993,84...\r\n                                        ... 1431199559,19532,-16777216,16777215,771751936,1634889519,1835365475,1968010608,777612127,6710387,1598506062,1280070990,1145962496,1431199559,19532,-1677...\r\n\r\n   TITLE          &lt;_CHAR*4&gt;       'Ros6'\r\n   LABEL          &lt;_CHAR*57&gt;      'T%+%+A%^50+%&lt;20+*%+   corrected antenna temperature'\r\n   UNITS          &lt;_CHAR*1&gt;       'K'\r\n   VARIANCE                {structure}\r\n      DATA(119,118,1880)  &lt;_REAL&gt;    *,*,*,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,4.915271,...\r\n                                     ... 4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,4.499031,*,4.499031,...\r\n      ORIGIN(3)      &lt;_INTEGER&gt;      -57,-58,-940\r\n\r\n   WCS                       {structure}\r\n      DATA(274)      &lt;_CHAR*32&gt;      ' Begin FrameSet',' Nframe = 5',' Currnt = 5',' Lnk2 = 1',' Lnk3 = 1',' Lnk4 = 1',' Lnk5 = 1',' Frm1 =',' Begin Frame',' Naxes = 3',' Domain = \"GRID\"',' Ax1 ='...\r\n                                     ... ' End Sp...',' MapB =',' Begin ZoomMap',' Nin = 1',' IsA Mapping',' Zoom = 0.001',' End ZoomMap',' End CmpMap',' End CmpMap',' End CmpMap',' End CmpMap',' ...\r\n\r\nEnd of Trace.\r\n<\/pre>\n<\/li>\n<\/ol>\n<\/div>\n<h3><span id=\"Task_plot_the_scan_pattern_of_a_SCUBA-2_map\">Task: plot the scan pattern of a SCUBA-2 map<\/span><\/h3>\n<p>Use SMURF&#8217;s <tt>jcmtstate2cat<\/tt> to JCMT information raw SCUBA-2 observation (if you don&#8217;t have any raw SCUBA-2 observations on disk, an example output catalogue is provided in the file jcmtstate_s20120501_00068.TST in this tutorial&#8217;s tarball.). Use TOPCAT to plot the scan pattern.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d8db0\"  tabindex=\"0\" title=\"Show worked example on click\"    >Show worked example on click<\/span><div id=\"target-id69db70e1d8db0\" class=\"collapseomatic_content \">\n<strong>Worked Solution<\/strong><\/p>\n<p>You can use all the same tools listed above to look at raw data as well. However, there is another tool that is useful for raw data: <tt>jcmtstate2cat<\/tt> in SMURF. This lets you see the JCMTSTATE extension in a raw file which contains timestreams of information about the position of the telescope, the instrumental data and the WVM. To avoid packaging up all the raw files, we&#8217;ve included the output catalogue here. If you needed to generate the file, here we show how to do it on from the s8a sub arrays of one of the observations in the SCUBA-2 basic DR tutorial.<\/p>\n<pre>smurf\r\njcmtstate2cat ..\/SCUBA2_tutorial\/raw\/s8a20120501_00068*.sdf &gt; jcmtstate_s20120501_00068.TST\r\n<\/pre>\n<p>This stores the catalogue in a file named <tt> jcmtstate_s20120501_00068.TST<\/tt>. You can open and plot this in Topcat.<\/p>\n<pre>topcat -f TST jcmtstate_s20120501_00068.TST&amp;\r\n<\/pre>\n<p>This opens the file in TOPCAT, and tells TOPCAT that the file is in TST format.<\/p>\n<p>Try creating a plot of the telescope scan position. You would normally do this by going to the Scatter Plot button, then selecting the DRA and DDec columns to plot. Try making other plots!<\/p>\n<div id=\"attachment_8044\" style=\"width: 237px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-topcat-jcmtstate2cat.png\"><img aria-describedby=\"caption-attachment-8044\" loading=\"lazy\" class=\"size-medium wp-image-8044\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-topcat-jcmtstate2cat-227x300.png\" alt=\"\" width=\"227\" height=\"300\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-topcat-jcmtstate2cat-227x300.png 227w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-topcat-jcmtstate2cat-113x150.png 113w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-topcat-jcmtstate2cat.png 516w\" sizes=\"(max-width: 227px) 100vw, 227px\" \/><\/a><p id=\"caption-attachment-8044\" class=\"wp-caption-text\">The scan pattern of a JCMT raw SCUBA-2 observation is being shown.<\/p><\/div>\n<\/div>\n<h2><span id=\"Measuring_Noise_in_the_Map\">Measuring Noise in the Map<\/span><\/h2>\n<h3><span id=\"Task_Comparing_noise_estimates\">Task: Comparing noise estimates<\/span><\/h3>\n<p>Compare the noise estimated in a blank region of the spectra with the noise found in the Variance array<br \/>\n<span class=\"collapseomatic \" id=\"id69db70e1d8e89\"  tabindex=\"0\" title=\"Show worked example on click\"    >Show worked example on click<\/span><div id=\"target-id69db70e1d8e89\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>Take a reduced spectral cube, measure the noise on a blank part of the spectrum. Compare this noise with the measured noise from the Variance array.<\/p>\n<p>First of all, we will open our cube in Gaia in order to identify a blank region of the spectra over which to measure the noise.<\/p>\n<div id=\"attachment_8045\" style=\"width: 310px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion.png\"><img aria-describedby=\"caption-attachment-8045\" loading=\"lazy\" class=\"size-medium wp-image-8045\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-300x275.png\" alt=\"\" width=\"300\" height=\"275\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-300x275.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-768x703.png 768w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-1024x938.png 1024w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-164x150.png 164w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion-150x137.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-blankspectralregion.png 1030w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-8045\" class=\"wp-caption-text\">Finding a blank spectral section in Gaia.<\/p><\/div>\n<p>The region from -265.0 to -78 km\/s looks like it is very blank &#8212; we will use this to measure the noise.<\/p>\n<p>You can either perform the collapse within Gaia by using the &#8216;collapse&#8217; button on the Spectral cube window, and then either save the file or use the imaging tool in Gaia to examine the statistics, or you can use the command line. The command line is more repeatable.<\/p>\n<p>First we will use KAPPA&#8217;s collapse command to collapse over the blank region. This command supports a wide range of estimators allowing you to produce integrated maps, RMS estiamtes and moment maps. Please see the documentation for full details. Here we will use the <tt>sigma<\/tt> estimator to get the standard deviation in that region.<\/p>\n<pre>collapse in=co32_cube.sdf axis=vrad low=-265.0 high=-78.0 estimator=sigma out=co32_noisemap.sdf\r\n<\/pre>\n<p>Note that we have to give the velocity range as floating points; if we give integers it will assume we mean pixel coordinates instead of km\/s.<\/p>\n<p>Now use the stats command to look at the values.<\/p>\n<pre>$ stats co32_noisemap.sdf \r\n\r\n   Pixel statistics for the NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/co\r\n32_noisemap\r\n\r\n      Title                     : Ros6\r\n      NDF array analysed        : DATA\r\n\r\n         Pixel sum              : 11429.2\r\n         Pixel mean             : 0.829648\r\n         Standard deviation     : 0.390908\r\n         Skewness               : 2.14229\r\n         Kurtosis               : 4.21748\r\n         Minimum pixel value    : 0.518394\r\n            At pixel            : (-12, -13)\r\n            Co-ordinate         : (6:34:43.1, 4:11:20)\r\n         Maximum pixel value    : 2.65666\r\n            At pixel            : (60, -30)\r\n            Co-ordinate         : (6:34:14.2, 4:09:38)\r\n         Total number of pixels : 14042\r\n         Number of pixels used  : 13776 (98.1%)\r\n         No. of pixels excluded : 266 (1.9%)\r\n<\/pre>\n<p>Across the whole map we have a noise of 0.83+\/-0.39 K<\/p>\n<p>You can also open this map in Gaia, and use the Image Analysis-&gt;Image regions toolbar to measure the noise only in the center of the map.<\/p>\n<pre>gaiadisp co32_noisemap.sdf\r\n<\/pre>\n<div id=\"attachment_8046\" style=\"width: 160px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise.png\"><img aria-describedby=\"caption-attachment-8046\" loading=\"lazy\" class=\"size-medium wp-image-8046\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise-150x300.png\" alt=\"\" width=\"150\" height=\"300\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise-150x300.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise-512x1024.png 512w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise-75x150.png 75w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-gaia-measurenoise.png 557w\" sizes=\"(max-width: 150px) 100vw, 150px\" \/><\/a><p id=\"caption-attachment-8046\" class=\"wp-caption-text\">Using the Image Regions command to measure the average noise across the centre of a noise map.<\/p><\/div>\n<\/div>\n<h3><span id=\"Task_Regrid_map_and_check_the_noise\">Task: Regrid map and check the noise<\/span><\/h3>\n<p>Regrid the map into 1 km\/s channels and see how the noise changes compared with before.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d8f84\"  tabindex=\"0\" title=\"Show Worked solution on click\"    >Show Worked solution on click<\/span><div id=\"target-id69db70e1d8f84\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>We will use <tt>sqorst<\/tt> (SQuash OR STretch) to rebin the data onto 1.0 km\/s channels. We will accept the default method of rebinning.<\/p>\n<pre>sqorst in=co32_cube.sdf axis=3 mode=pixelscale pixscale=1.0 out=co32_1kms.sdf method=auto\r\n<\/pre>\n<p>Now, we will repeat the previous <tt>collapse<\/tt> command on this file in order to measure the noise after rebinning. Because we set the collapse limits in terms of a velocity instead of using a pixel index, we don&#8217;t have to change them after rebinning.<\/p>\n<pre>collapse in=co32_1kms.sdf axis=vrad low=-265.15 high=-78.05 estimator=sigma out=co32_1kms_noisemap.sdf<\/pre>\n<p>As before, we see the results with <tt>stats<\/tt>:<\/p>\n<pre>$ stats co32_1kms_noisemap.sdf\r\n    Pixel statistics for the NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/co\r\n32_1kms_noisemap\r\n\r\n      Title                     : Ros6\r\n      NDF array analysed        : DATA\r\n\r\n         Pixel sum              : 8072.22\r\n         Pixel mean             : 0.585963\r\n         Standard deviation     : 0.277194\r\n         Skewness               : 2.13845\r\n         Kurtosis               : 4.24379\r\n         Minimum pixel value    : 0.341995\r\n            At pixel            : (17, -12)\r\n            Co-ordinate         : (6:34:31.5, 4:11:26)\r\n         Maximum pixel value    : 1.91068\r\n            At pixel            : (-19, -58)\r\n            Co-ordinate         : (6:34:45.9, 4:06:50)\r\n         Total number of pixels : 14042\r\n         Number of pixels used  : 13776 (98.1%)\r\n         No. of pixels excluded : 266 (1.9%)\r\n<\/pre>\n<p>Here you can see we now have a mean RMS noise in our regridded map of 0.59+\/0.28 K. if you wish, you can verify that this has approximately followed the radiometer equation.<\/p>\n<p>We can also compare the noise we have measured on the map with the noise in the VARIANCE component of the file. The easiest way is using the &#8216;comp&#8217; option in stats to set which component to analyse (DATA is the default), and selecting the ERROR component, defined as the square root of the variance component.<\/p>\n<pre>$stats co32_1kms.sdf comp=ERROR\r\n\r\n\r\n   Pixel statistics for the NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/co\r\n32_1kms\r\n\r\n      Title                     : Ros6\r\n      NDF array analysed        : ERROR\r\n\r\n         Pixel sum              : 6.0305e+06\r\n         Pixel mean             : 0.550634\r\n         Standard deviation     : 0.267865\r\n         Skewness               : 2.21899\r\n         Kurtosis               : -2.97147\r\n         Minimum pixel value    : 0.364528\r\n            At pixel            : (-37, 6, -397)\r\n            Co-ordinate         : (6:34:53.1, 4:13:14, -380.4388)\r\n         Maximum pixel value    : 1.63818\r\n            At pixel            : (57, 51, -397)\r\n            Co-ordinate         : (6:34:15.4, 4:17:44, -380.4388)\r\n         Total number of pixels : 11163390\r\n         Number of pixels used  : 10951920 (98.1%)\r\n         No. of pixels excluded : 211470 (1.9%)\r\n\r\n<\/pre>\n<p>We see a noise from the variance component of 0.55 +\/- 0.26 K, compared with 0.59 +\/- 0.28 K measured across one part of the empty spectra. The variance component was generated by SMURF&#8217;s gridding command &#8216;makecube&#8217;, and be default is generated from the Tsys values of the input spectra.<br \/>\n<\/div>\n<h3><span id=\"Task_measure_the_noise_in_a_map\">Task: measure the noise in a map<\/span><\/h3>\n<p>Measure the noise across a SCUBA-2 map. Use the scuba2_map.sdf file provided with this tutorial.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d8faf\"  tabindex=\"0\" title=\"Worked solution\"    >Worked solution<\/span><div id=\"target-id69db70e1d8faf\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>We will use the SCUBA-2 map provided in the sample data, &lt;code&gt;scuba2_map.sdf&lt;\/code&gt;. This is the same as the &lt;code&gt;gs20120501_68_850_reduced.sdf&lt;\/code&gt; file provided in the SCUBA-2 tutorial &lt;code&gt;example_reduced&lt;\/code&gt; data directory<\/p>\n<p>There are several ways you can do this. The simplest is to examine the ERROR component of the data.<\/p>\n<pre>$stats scuba2_map.sdf comp=ERROR\r\n\r\nPixel statistics for the NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/sc\r\nuba2_map\r\n\r\n      Title                     : G34.3\r\n      NDF array analysed        : ERROR\r\n\r\n         Pixel sum              : 1640087.30197531\r\n         Pixel mean             : 41.0555547705848\r\n         Standard deviation     : 50.659757537598\r\n         Skewness               : 3.90590482923681\r\n         Kurtosis               : 26.0299464638736\r\n         Minimum pixel value    : 10.6386591437756\r\n            At pixel            : (-3, 23, 1)\r\n            Co-ordinate         : (18:53:19.667, 1:16:26.30, 0.00085)\r\n         Maximum pixel value    : 1157.3041341558\r\n            At pixel            : (-13, 122, 1)\r\n            Co-ordinate         : (18:53:22.334, 1:23:02.30, 0.00085)\r\n         Total number of pixels : 58081\r\n         Number of pixels used  : 39948 (68.8%)\r\n         No. of pixels excluded : 18133 (31.2%)\r\n\r\n<\/pre>\n<p>You can also see this in Gaia by going to Image Analysis-&gt; Image Regions. Select the region you want to get statistics over and use the &#8216;Stats Selected&#8217; button to generate the statistics. you can also use this to compare the noise in a blank part of the sky with the noise in the variance component.<\/p>\n<div id=\"attachment_8055\" style=\"width: 310px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise.png\"><img aria-describedby=\"caption-attachment-8055\" loading=\"lazy\" class=\"size-medium wp-image-8055\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-300x234.png\" alt=\"\" width=\"300\" height=\"234\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-300x234.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-768x598.png 768w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-1024x798.png 1024w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-193x150.png 193w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise-150x117.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-Gaia-measureS2noise.png 1172w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-8055\" class=\"wp-caption-text\">Measuring the noise in a blank field of a SCUBA-2 map with the Gaia Image Regions toolbar.<\/p><\/div>\n<p>In the example shown in the screen shot above, the Std. Deviation of the data area is 24.4 (measuring the RMS noise in that region); by comparison the noise in the Variance array has a mean value in that region of 338, corresponding to an error of 18.4. Fairly close, but there is clearly some difference!<\/p>\n<p>Another method, designed to be used on reductions of a single observation, is to run the PICARD recipe SCUBA2_MAPSTATS on your file. This would like so:<\/p>\n<pre>$ picard -log sf -nodisplay SCUBA2_MAPSTATS scuba2_map.sdf \r\nPicard Says: No display will be used\r\nPicard Says: Pre-starting mandatory monoliths...Done\r\nChecking for next data file: \/export\/data\/sgraves\/sd\/tutorial-data\/JCMT_HETERODYNE_analysis_tutorial_2016\/scuba2_map.sdf\r\nStoring: scuba2_map\r\nPicard Says: Creating temporary bad observation rules file\r\nA new group 20120501#-1 has been created\r\nOverriding PICARD instrument class to PICARD_SCUBA2_850\r\nSorting Groups\r\nREDUCING: scuba2_map\r\nUsing recipe SCUBA2_MAPSTATS specified on command-line\r\nProcessing data for G34.3\r\n\r\nCalling _WRITE_MAPSTATS_LOGFILE_: write logfile with results from image analysis\r\nWriting results to log.mapstats...\r\nCropping map using: RA= 18:53:18.6, Dec= 01:14:58.26, RADIUS=120.\r\nRMS is 13.5415200736753\r\n\r\nRecipe took 2.018 seconds to evaluate and execute.\r\n\r\nPicard processing complete\r\nProcessed one recipe which completed successfully\r\nExiting...\r\n<\/pre>\n<p>In the output directory, you will now find a logfile anmed <tt>log.mapstats<\/tt>:<\/p>\n<pre>$cat log.mapstats\r\n# Map results: noise, NEFD, exp_time log file - created on Thu Jan 25 20:45:30 2018 UT\r\n#\r\n# (YYYYMMDD.frac) (YYYY-MM-DDThh:mm:ss) () () () (um) (deg)  () ()  () () (s)  (s)  ()  ()  ()  ()  (\")  (\")  ()  ()  ()  ()  ()\r\n# UT   HST  Obs  Source  Mode  Filter  El  Airmass  Trans  Tau225  Tau  t_elapsed  t_exp  rms  rms_units  nefd  nefd_units  RA  Dec  mapsize  pixscale  project  recipe  filename\r\n20120501.6984144  2012-05-01T06:45:43    68  G34.3  daisy        850  52  1.26  0.783  0.047  0.194  318  68.08  1.354E+01  mJy\/bm  1.078E+02  mJy.s^0.5   18:53:18.6   01:14:58.26  240.0  4.0  M12AEC05  REDUCE_SCAN  scuba2_map\r\n<\/pre>\n<p>This summarises information about the observation, as well as calculating the noise over central portion of the MAP, and the NEFD (Noise Equivalent Flux Density, usually used for tracking instrumental performance). If you run this command on a coadded map, it cannot accurately fill in most of the columns about the observation.<\/p>\n<\/div>\n<h2><span id=\"Masking_Extracting_and_Trimming\">Masking, Extracting and Trimming<\/span><\/h2>\n<h3><span id=\"Task_Identifying_the_NDF_pixel_coordinates_of_a_region_with_Gaia\">Task: Identifying the NDF pixel coordinates of a region with Gaia<\/span><\/h3>\n<span class=\"collapseomatic \" id=\"id69db70e1d9071\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d9071\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>By default, the X: and Y: boxes in Gaia will show you the pixels relative to the bottom left of the map. Normally, the NDF PIXEL coordinates (relative to the pixel origin shown in ndftrace) are more useful. You can see these values in Gaia in two ways: one, you can open Image Analysis-&gt;Change Coordinates-&gt; Show All Coordinates. This will bring up a seperate window that will show you the PIXEl (as well as GRID, AXIS&lt; FRACTION and SKY) coordinates of your cursor as you move it around in the Gaia window.<\/p>\n<p>Alternatively, you can start Gaia with the -pixel_indices=1 flag, and it will then show you the NDF PIXEL coordinates by default.<\/p>\n<pre>gaia -pixel_indices 1 scuba2_map.sdf\r\n<\/pre>\n<p>When you need to identify the pixel ranges, you can then use these coordinates. Test this out by finding the pixel values for a section of the map, then use <tt>ndfcopy<\/tt> and an NDF section to copy that part out. Open your new map in Gaia and verify you copied the part you were looking for. Remember that pixel indices have to be given in integers!<\/p>\n<pre>ndfcopy scuba2_map.sdf\\(10:65,-5:53\\) extracted.sdf\r\ngaia -pixel_indices 1 extracted.sdf &amp;\r\n<\/pre>\n<\/div>\n<h3><span id=\"Task_extract_a_region_using_WCS_coordinates\">Task: extract a region using WCS coordinates.<\/span><\/h3>\n<p>Try and extract the region from RA\/Dec diagonal corners:(18:53:16, 1:14:35) to (18:53:01, 1:18:27.63)<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d9093\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d9093\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>The main difficulity with this is formatting the strings correctly and using shell escapes. D There are several ways this could be done. One example is<\/p>\n<pre>ndfcopy scuba2_map.sdf\\(18h53m16.3s:18h53m01.4s,1d14m35s:1d18m27.6s\\) extracted2.sdf<\/pre>\n<p>In this example, don&#8217;t include any spaces next to the comma in the centre of the NDF section or it won&#8217;t be parsed correctly. If instead of using backslashes for your shell escapes you enclose the entire NDF and NDF section inside double and single quotes, then you can use spaces after the comman. E.g.:<\/p>\n<pre>ndftrace \"'scuba2_map.sdf(18h53m16.3s:18h53m01.4s, 1d14m35s:1d18m27.6s)'\"\r\n<\/pre>\n<p>NDF sections can be passed to commands anywhere an entire NDF could be passed. Once you get the hang of them they are a very useful feature of Starlink.<br \/>\n<\/div>\n<h3><span id=\"Task_CROP_a_SCUBA-2_image_with_PICARD\">Task: CROP a SCUBA-2 image with PICARD<\/span><\/h3>\n<p>Use PICARD to crop a SCUBA-2 image into the requested area of the original observation with PICARD. An advanced option: try varying the radius.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d90b2\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d90b2\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>You can do this using the PICARD task CROP_SCUBA2_IMAGES:<\/p>\n<pre>$ picard -log sf CROP_SCUBA2_IMAGES scuba2_map.sdf \r\nSetting up display infrastructure (display tools will not be started until necessary)...Done\r\nPicard Says: Pre-starting mandatory monoliths...Done\r\nChecking for next data file: \/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/scuba2_map.sdf\r\nStoring: scuba2_map\r\nA new group 20120501#-1 has been created\r\nOverriding PICARD instrument class to PICARD_SCUBA2_850\r\nSorting Groups\r\nREDUCING: scuba2_map\r\nUsing recipe CROP_SCUBA2_IMAGES specified on command-line\r\nProcessing data for G34.3\r\n\r\nCalling _CROP_SCUBA2_IMAGE_: trim image to specified map size\r\nTrimming image to specified map size\r\nTrimming scuba2_map...\r\nCropping map using: RA= 18:53:18.6, Dec= 01:14:58.26, RADIUS=120.\r\nRemoving temporary files...\r\nChecking scuba2_map_crop...\tKeeping extension\r\nRecipe took 1.308 seconds to evaluate and execute.\r\n\r\nPicard processing complete\r\nProcessed one recipe which completed successfully\r\nExiting...\r\n\r\nPicard Says: Goodbye\r\n<\/pre>\n<p>By default this will use the center position of the original observation, and the &#8216;radius&#8217; of the central area of the SCAN type, and produce a square cut out around the center.<\/p>\n<p>If you wish to crop out a circular area or a larger area, you can provide a recpars file to alter this. E.g. to crop out a CIRCLE of radius 250&#8243;, you would save in a text file:<\/p>\n<pre>[CROP_SCUBA2_IMAGES]\r\nCROP_METHOD=CIRCLE\r\nMAP_RADIUS=250\r\n<\/pre>\n<p>Save this file as e.g. recpars.txt and then run<\/p>\n<pre>$ picard -log sf CROP_SCUBA2_IMAGES -recpars=recpars.txt scuba2_map.sdf \r\nSetting up display infrastructure (display tools will not be started until necessary)...Done\r\nPicard Says: Pre-starting mandatory monoliths...Done\r\nChecking for next data file: \/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/scuba2_map.sdf\r\nStoring: scuba2_map\r\nA new group 20120501#-1 has been created\r\nOverriding PICARD instrument class to PICARD_SCUBA2_850\r\nSorting Groups\r\nREDUCING: scuba2_map\r\nUsing recipe CROP_SCUBA2_IMAGES specified on command-line\r\nProcessing data for G34.3\r\n\r\nCalling _CROP_SCUBA2_IMAGE_: trim image to specified map size\r\nTrimming image to specified map size\r\nOutput image will be a circle of radius 250 arcsec\r\nMasking the weights and exposure time images...\r\nRemoving temporary files...\r\nChecking scuba2_map_tmpmask...\tRemoving\r\nChecking scuba2_map_crop...\tKeeping extension\r\nRecipe took 2.668 seconds to evaluate and execute.\r\n\r\nPicard processing complete\r\nProcessed one recipe which completed successfully\r\nExiting...\r\n\r\nPicard Says: Goodbye\r\n<\/pre>\n<\/div>\n<h3><span id=\"Task_ARD_Masking\">Task: ARD Masking<\/span><\/h3>\n<p>Make an ARD file and use the ardmask command to extract out an annular region around a point you choose.<\/p>\n<p>ARD docs: http:\/\/starlink.eao.hawaii.edu\/docs\/sun183.htx\/sun183.html<br \/>\nKAPPA guide to using ARD:<br \/>\nhttp:\/\/www.starlink.ac.uk\/cgi-bin\/htxserver\/sun95.htx\/sun95.html\/?xref_se_ardwork<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d90f2\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d90f2\" class=\"collapseomatic_content \">\n<strong>Worked Example<\/strong><\/p>\n<p>ARD format files are fairly straight forward, so they can be made yourself. A simple file to extract out a ring of inner radius 70&#8243;, outer radius 100&#8243; and centered on RA=18:53:18.7, Dec=1:14:54.3 would be:<\/p>\n<pre>COFRAME(SKY, SYSTEM=FK5, equinox=2000)\r\nCIRCLE(18:53:18.7,1:14:54.3, :1:40.) .AND. .NOT. CIRCLE(18:53:18.7,1:14:54.3, :1:10.0)\r\n<\/pre>\n<p>Note that we define the radii in(third element of the CIRCLE command) in degrees:arcminutes:arcseconds. To create an annulus we have defined one larger circle, and then said to exclude the central region of the circle.<\/p>\n<p>We save this as e.g. file ARD_ring.ard, and then we can apply it with <tt>ardmask<\/tt>.<\/p>\n<pre>ardmask in=scuba2_map.sdf ardfile=ARD_ring.ard out=scuba2_map_2d.sdf inside=False\r\n<\/pre>\n<p>Note that we need to remove the redundant third axis of the data &#8212; here we do this with ndf copy.<\/p>\n<pre>ndfcopy in=scuba2_map.sdf out=scuba2_map2d.sdf trim=true trimwcs=true \r\n<\/pre>\n<p>The result should look like:<\/p>\n<div id=\"attachment_8059\" style=\"width: 237px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-annulus-ARD.png\"><img aria-describedby=\"caption-attachment-8059\" loading=\"lazy\" class=\"size-medium wp-image-8059\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-annulus-ARD-227x300.png\" alt=\"\" width=\"227\" height=\"300\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-annulus-ARD-227x300.png 227w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-annulus-ARD-114x150.png 114w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-annulus-ARD.png 443w\" sizes=\"(max-width: 227px) 100vw, 227px\" \/><\/a><p id=\"caption-attachment-8059\" class=\"wp-caption-text\">An annulus created with an ARD file.<\/p><\/div>\n<p>You could also try using the ARDGEN interactive utility to create an ARD file. First of all, you need to display the image in an Xwindow using the KAPPA&#8217;s display command:<\/p>\n<pre>display scuba2_map.sdf dev=xw mode=faint\r\n<\/pre>\n<p>(This displays the map in the xwindows graphics device, using mode=faint so that you are not prompted to give a low and high value to scale between.)<\/p>\n<p>You then run the command <tt>ardgen dev=xw<\/tt> and follow the prompts, selecting the shape you want to make and clicking in the xwindow to mark shapes. Remember you can use the ? at a command line prompt to see the help.<\/p>\n<div id=\"attachment_8060\" style=\"width: 310px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen.png\"><img aria-describedby=\"caption-attachment-8060\" loading=\"lazy\" class=\"size-medium wp-image-8060\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-300x200.png\" alt=\"\" width=\"300\" height=\"200\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-300x200.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-768x513.png 768w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-1024x684.png 1024w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-225x150.png 225w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen-150x100.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-ardgen.png 1328w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-8060\" class=\"wp-caption-text\">ARDGEN in use. you can see the circular shape defined by the user overlain on the image shown with display.<\/p><\/div>\n<\/div>\n<h3><span id=\"Task_masking_based_on_SNR\">Task: masking based on SNR<\/span><\/h3>\n<ol>\n<li>Mask out all pixels in a cube or map with a detection of less than 5 sigma (try <tt>errclip<\/tt>).<\/li>\n<li>Mask out all spectra in a cube where the <strong>peak<\/strong> of the spectra has a detection of less than 5 sigma. Try using makesnr, thresh, manic and copybad.<\/li>\n<\/ol>\n<span class=\"collapseomatic \" id=\"id69db70e1d91cb\"  tabindex=\"0\" title=\"Show worked solution for part 1\"    >Show worked solution for part 1<\/span><div id=\"target-id69db70e1d91cb\" class=\"collapseomatic_content \">\n<p><strong>Mask out all pixels in a cube or map with a detection of less than 5 sigma (try <tt>errclip<\/tt>)<\/strong><\/p>\n<p>The command <tt>errclip<\/tt> can be used to clip a map or cube based on SNR levels, using the variance component of the map to define the snr. You can use it like:<\/p>\n<pre>errclip in=co32_cube.sdf mode=SNR out=co32_clipped5sigma limit=5.0\r\n<\/pre>\n<p>Open the resulting map in Gaia and take a look at it: you&#8217;ll see that the baseline and linewing regions are marked as bad, and only the strong parts of the line centre in bright regions are remaining.<br \/>\n<\/div>\n<span class=\"collapseomatic \" id=\"id69db70e1d9204\"  tabindex=\"0\" title=\"Show worked solution for part 2\"    >Show worked solution for part 2<\/span><div id=\"target-id69db70e1d9204\" class=\"collapseomatic_content \">\n<strong>Worked example: part 2<\/strong><\/p>\n<p><strong>Mask out all spectra in a cube where the <strong>peak<\/strong> of the spectra has a detection of less than 5 sigma. Try using makesnr, thresh, manic and copybad. <\/strong><\/p>\n<p>Often, its useful not to clip out individual channels on the basis of the noise limit, but instead to clip out only whole spectra where the peak was not detected at greater than 5 sigma.<\/p>\n<p>First of all, we will need to collapse the cube down to a map of the peak values in each spectrum, using <tt>collapse<\/tt>. You can determine the velocity range to perform the collapse over by inspecting the cube in Gaia.<\/p>\n<pre>collapse in=co32_cube.sdf axis=vrad low=-6.5 high=25.67 estimator=max out=maxvalue\r\n<\/pre>\n<p>(If you would prefer to clip based on a detection in the integrated line, you could use the <tt>estimator=integ<\/tt> option instead.)<\/p>\n<p>We will now create an SNR map from this 2-D peak map with the <tt>makesnr<\/tt> command:<\/p>\n<pre>makesnr in=maxvalue.sdf out=snr.sdf minvar=0.0\r\n<\/pre>\n<p>Then we use the <tt>thresh<\/tt> command to set bad all pixels with an SNR less than 5. We also select a spurious high value to set bad, as we have to provide both thrlo and thrhi options; this won&#8217;t cause any problems.<\/p>\n<pre>thresh in=snr out=snr_thr5 thrlo=5.0 thrhi=5000 newlo=bad newhi=bad\r\n<\/pre>\n<p>You now need to grow this mask into a 3-D mask you can apply to your original map. To do this we first need to find out what the pixel indices of are input cube are;<\/p>\n<pre>$ ndftrace co32_cube.sdf \r\n\r\n   NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/co32_cube:\r\n      Title:  Ros6\r\n      Label:  TA*   corrected antenna temperature\r\n      Units:  K\r\n\r\n   Shape:\r\n      No. of dimensions:  3\r\n      Dimension size(s):  119 x 118 x 1880\r\n      Pixel bounds     :  -57:61, -58:59, -940:939\r\n      Total pixels     :  26398960\r\n\r\n   Data Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   Variance Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      Bad pixels may be present\r\n\r\n   World Co-ordinate Systems:\r\n      Number of co-ordinate Frames: 5\r\n\r\n      Current co-ordinate Frame (Frame 5):\r\n\r\n        Frame title         : \"3-d compound coordinate system\"\r\n        Domain              : SKY-DSBSPECTRUM\r\n        First pixel centre  : 6:35:01.2, 4:06:50, -381.1352\r\n\r\n           Axis 1:\r\n              Label              : Right ascension\r\n              Units              : hh:mm:ss.s\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 2:\r\n              Label              : Declination\r\n              Units              : ddd:mm:ss\r\n              Nominal Pixel scale: 5.99999 arc-sec\r\n\r\n           Axis 3:\r\n              Label              : Radio velocity (USB)\r\n              Units              : km\/s\r\n              Nominal Pixel scale: 0.4233065 km\/s\r\n\r\n\r\n   Extensions:\r\n      FITS             \r\n      SMURF            \r\n      PROVENANCE       \r\n\r\n\r\n   History Component:\r\n      Created    :  2014 Nov 20 09:22:05\r\n      No. records:  22\r\n      Last update:  2018 Jan 25 22:26:20 (WCSATTRIB       (KAPPA 2.5-5))\r\n      Update mode:  NORMAL\r\n<\/pre>\n<p>We can see here that our c32_cube goes from -940:939 in pixels in the 3rd, velocity, axis. We will now use the command <tt>manic<\/tt> to grow our 2D mask into a 3D mask we can apply to our original data; manic increases the dimensionality of the cube by (in this case) repeating the image plane pixels in every plane we add in the third dimension. The command we use is:<\/p>\n<pre>manic in=snr_thr5 out=cube_mask axes=\\[1,2,0\\] ubound=939 lbound=-940\r\n<\/pre>\n<p>Finally, we copy the bad pixel mask from this cube to our original CO 3-2 cube, and then open the masked map in Gaia to examine it.<\/p>\n<pre>copybad in=co32_cube.sdf ref=cube_mask.sdf  out=co32_masked.sdf\r\ngaiadisp co32_masked.sdf\r\n<\/pre>\n<h2><span id=\"Extracting_Values\">Extracting Values<\/span><\/h2>\n<p>You can always of course use the <tt>stats<\/tt> command to get the sum\/mean etc value for small regions, or within extracted NDF sections or ARD masked\/extracted areas, and you may find that sufficient for reading out some of your values. The pluck command can be used to easily get data values at a specific point on the map however, or to extract the whole spectra from that point.<br \/>\n<\/div>\n<h3><span id=\"Task_Using_pluck_to_extract_spectra\">Task: Using pluck to extract spectra<\/span><\/h3>\n<p><strong>Using the co32_cube.sdf provided with this tutorial, extract spectra from the following coordinates:<br \/>\n6:34:37.7,4:12:37.20, 6:34;32.77,4:13:09.77, 6:34:24.47, 4:08:56.91 . Examine the output spectra, either with Splat or by using linplot<\/strong><\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d9247\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d9247\" class=\"collapseomatic_content \">\n<strong>Worked Example<\/strong><\/p>\n<p>You can use pluck interactively, but here we will use it by giving it a text file. The format of the input catalogue is:<\/p>\n<pre>6:34:37.7 4:12:37.20 1\r\n6:34:32.77 4:13:09.77 1\r\n6:34:24.47 4:08:56.91 1\r\n<\/pre>\n<p>Note that you have to provide a value for axis 3, even though the output will include all values from that axis. We then run the command with:<\/p>\n<pre>$ pluck in=co32_cube.sdf axes=vrad mode=file coin=spectra_positions.txt container=True out=container_spectra.sdf method=linear\r\nCreating\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/container_spectra.PLUCK\r\n_1.\r\nCreating\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/container_spectra.PLUCK\r\n_2.\r\nCreating\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/container_spectra.PLUCK\r\n_3.\r\n<\/pre>\n<p>We used a linear resampling method to extract the pixels; others are available, but may require setting the <tt>params<\/tt> paramter to use. See the documentation for details. We now have an output file <tt>container_spectra.sdf<\/tt> which contains 3 input spectra as separate NDFs. We can copy them out to their own files with:<\/p>\n<pre>ndfcopy container_spectra.PLUCK_1 pluck_1\r\nndfcopy container_spectra.PLUCK_2 pluck_2\r\nndfcopy container_spectra.PLUCK_3 pluck_3\r\n<\/pre>\n<p>If you use ndftrace to look at one of the spectra, it includes all the information about the spatial position it was extracted from.<\/p>\n<pre>$ ndftrace pluck_*.sdf \r\n\r\n   NDF structure\r\n\/export\/data\/sgraves\/sd\/tutorial-data\/Starlink_Analysis\/pluck_1:\r\n      Title:  Ros6\r\n      Label:  TA*   corrected antenna temperature\r\n      Units:  K\r\n\r\n   Shape:\r\n      No. of dimensions:  3\r\n      Dimension size(s):  1 x 1 x 1880\r\n      Pixel bounds     :  1:1, 1:1, -940:939\r\n      Total pixels     :  1880\r\n\r\n   Data Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      There are no bad pixels present\r\n\r\n   Variance Component:\r\n      Type        :  _REAL\r\n      Storage form:  SIMPLE\r\n      There are no bad pixels present\r\n\r\n   World Co-ordinate Systems:\r\n      Number of co-ordinate Frames: 5\r\n\r\n      Current co-ordinate Frame (Frame 5):\r\n\r\n        Frame title         : \"3-d compound coordinate system\"\r\n        Domain              : SKY-DSBSPECTRUM\r\n        First pixel centre  : 6:34:37.7, 4:12:37, -381.1352\r\n\r\n           Axis 1:\r\n              Label              : Right ascension\r\n              Units              : hh:mm:ss.s\r\n              Nominal Pixel scale: 6.00000 arc-sec\r\n\r\n           Axis 2:\r\n              Label              : Declination\r\n              Units              : ddd:mm:ss\r\n              Nominal Pixel scale: 6.00000 arc-sec\r\n\r\n           Axis 3:\r\n              Label              : Radio velocity (USB)\r\n              Units              : km\/s\r\n              Nominal Pixel scale: 0.4233065 km\/s\r\n\r\n\r\n   Extensions:\r\n      FITS             \r\n      SMURF            \r\n      PROVENANCE       \r\n\r\n\r\n   History Component:\r\n      Created    :  2014 Nov 20 09:22:05\r\n      No. records:  24\r\n      Last update:  2018 Jan 26 02:41:53 (NDFCOPY         (KAPPA 2.5-5))\r\n      Update mode:  NORMAL\r\n<\/pre>\n<p>You can also view the spectra in splat with:<\/p>\n<pre>splat pluck_*.sdf&amp;\r\n<\/pre>\n<p>Or display them with KAPPA&#8217;s linplot command:<\/p>\n<pre>linplot pluck_1 dev=xw\r\n<\/pre>\n<\/div>\n<h2><span id=\"Coordinate_conversion\">Coordinate conversion<\/span><\/h2>\n<h3><span id=\"Task_Convert_a_map_to_galactic_coordinates\">Task: Convert a map to galactic coordinates<\/span><\/h3>\n<p><strong>Use <tt>wcsattrib<\/tt> to change the Coordinate system of your CO 3-2 Ros6 map into Galactic Coordinates. Check the output map has the new coordinates<\/strong><\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d9265\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d9265\" class=\"collapseomatic_content \">\n<p><strong>Worked Example<\/strong><\/p>\n<p>Setting a new coordinate system in a map is easy. However, please note that this will not change the pixels in anyway: if you want to <strong>regrid<\/strong> your map onto a pixel grid aligned with a new coordinate system (this can matter more for extremely large maps), then you should look into the <tt>regrid<\/tt> command. Starlink NDF files, unlike FITS files, generally have multiple WCS systems associated with them, with defined transformations between them. Therefore, changing the coordinates only involves running wcsattrib to tell it to set the current system to your desired one. In this case, you will do:<\/p>\n<pre>wcsattrib ndf=co32_cube.sdf mode=set name=System newval=Galactic\r\n<\/pre>\n<p>And thats it!<\/p>\n<p>You can verify it by opening and Gaia and seeing which coordinate system is the default, or by converting to a FITS file with:<\/p>\n<pre>convert\r\nndf2fits co32_cube.sdf co32_galactic.fits\r\n<\/pre>\n<p>and opening it with your favourite non Gaia program. For example, see what the Python plotting program APLpy made of the FITS version of this file:<\/p>\n<p><a href=\"http:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted.png\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-8068\" title=\"aplpy_galacticconverted\" src=\"http:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted-300x270.png\" width=\"300\" height=\"270\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted-300x270.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted-768x691.png 768w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted-167x150.png 167w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted-150x135.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/aplpy_galacticconverted.png 1000w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><br \/>\n<\/div>\n<h3><span id=\"Task_Align_one_map_with_another\">Task: Align one map with another.<\/span><\/h3>\n<p>This is already done as part of Tutorial 5: CO subtraction. Check there for the full details, or see the <tt>wcsalign<\/tt> documentation at: http:\/\/www.starlink.ac.uk\/cgi-bin\/htxserver\/sun95.htx\/sun95.html?xref_WCSALIGN<\/p>\n<h2><span id=\"Fitting_Spectra\">Fitting Spectra<\/span><\/h2>\n<h3><span id=\"Task_use_fit1d_to_fit_your_entire_cube\">Task: use fit1d to fit your entire cube<\/span><\/h3>\n<p>SMURF&#8217;s <tt>fit1d<\/tt> routine can be used to fit a 1D spectrum to every pixel in your cube. Try it out on the co32_cube.sdf or co32_masked.sdf.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d9283\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d9283\" class=\"collapseomatic_content \">\n<strong>Worked Example<\/strong><\/p>\n<p>I highly recommend reading the documentation for fit1D: it has a lot of options, but is very powerful. As a quick overview, here is one example of using it.<\/p>\n<p>First, create a configuration file. An example one can be found in $STARLINK_DIR\/bin\/smurf\/smurf_fit1d.def , including descriptions of various parameters. We are going to use the basic paramters, and only provide updated values for the range parameter. This determines the radio velocity range we will fit the profiles within, and we will use -12 to 88 km\/s.<\/p>\n<p>This mostly-default config will fit a single Gausshermite2 to each spectrum. We define our config file as:<\/p>\n<pre>^$SMURF_DIR\/smurf_fit1d.def\r\nrange=(-12.0,88.0)<\/pre>\n<p>We save it as myfit1d.def, and then run our command as:<\/p>\n<pre>$ fit1d in=co32_masked.sdf out=fittedprofiles.sdf rms=! config=^myfit1d.def\r\nCopying input file 1 to output for fitting...\r\nFitting data using 1 GAUSSHERMITE2(s) over pixel range [873,1109]\r\n...Will use 16 threads to fit 14042 profiles.\r\n(FitProfileThread 16) ...at profile 14000 - 845 of 887 (i=26318120)\r\n...Writing parameter cubes and finishing up.\r\n<\/pre>\n<p>(A Caret, <tt>^<\/tt>, indicates to read the contents of the text file that follows.)<\/p>\n<p>If you open the resulting cube up in Gaia, and also open the original map in Gaia next to it via the File-&gt;new Window option, you can use the Open Cube-&gt;Options-&gt;Slave to move about in one cube and see spectra from both cubes. Alternatively, you can use the <tt>sub<\/tt> command to view the residuals from the fit.<\/p>\n<pre>sub in1=co32_masked.sdf in2=fittedprofiles.sdf out=residuals.sdf\r\n<\/pre>\n<div id=\"attachment_8071\" style=\"width: 310px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d.png\"><img aria-describedby=\"caption-attachment-8071\" loading=\"lazy\" class=\"size-medium wp-image-8071\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-300x224.png\" alt=\"\" width=\"300\" height=\"224\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-300x224.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-768x573.png 768w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-1024x764.png 1024w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-201x150.png 201w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d-150x112.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-residuals-fit1d.png 1256w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-8071\" class=\"wp-caption-text\">Examining the residuals from a fit1d line fitting procedure with Gaia.<\/p><\/div>\n<\/div>\n<h2><span id=\"Clumpfinding\">Clumpfinding<\/span><\/h2>\n<h3><span id=\"Task_Use_fellwalker_to_find_clumps\">Task: Use fellwalker to find clumps.<\/span><\/h3>\n<p>Use the Fellwalker algorithm to find clumps of emission in the G34.3 SCUBA-2 map for this tutorial. Copy out some of the individual clumps from the CUPID extension and view them in Gaia.<\/p>\n<span class=\"collapseomatic \" id=\"id69db70e1d932c\"  tabindex=\"0\" title=\"Show worked solution\"    >Show worked solution<\/span><div id=\"target-id69db70e1d932c\" class=\"collapseomatic_content \">\n<strong>Worked Example<\/strong><\/p>\n<p>If you&#8217;re not familiar with Fellwalker, you should read the description of the algorithm in the <a href=\"http:\/\/www.starlink.ac.uk\/docs\/sun255.htx\/sun255.htm\">CUPID manual<\/a>, or <a href=\"http:\/\/www.sciencedirect.com\/science\/article\/pii\/S2213133714000675\"> David Berry&#8217;s 2015 A&amp;C paper<\/a>.<\/p>\n<p>We first of all need to select our configuration parameters. Here, we are going to look for all clumps of 5 sigma above the noise, and we will subdivide neighbouring clumps into separate clumps if they have gaps between their peaks of at least 3sigma. We will follow the clumps down to 3 sigma above the noise. The config file we need to define is then:<\/p>\n<pre>FellWalker.Noise=3*RMS\r\nFellWalker.MinHeight=5*RMS\r\nFellWalker.MinDip=3*RMS\r\n<\/pre>\n<p>We save that into a text file, here named <tt>config.lis<\/tt>, and then we can call <tt>findclumps<\/tt> (after initialising cupid) as:<\/p>\n<pre>findclumps scuba2_map.sdf out=scuba2_clumps.sdf \\\r\n      outcat=clumpcat.FITS method=fellwalker deconv=no accept\r\n<\/pre>\n<p>We have to provide the special keyword <tt>accept<\/tt> indicating that all other parameters that would be prompted for should use their default value. This is necessary if you don&#8217;t want to be prompted to accept the default RMS value (calculated from the VARIANCE array);<br \/>\n<tt>findclumps<\/tt> will otherwise wait for you to either accept the calculated RMS or give a different value.<\/p>\n<p>We now have an output catalog in <tt>clumpcat.FITS<\/tt> that we can open and view with Topcat, and an output clumpmask in <tt>scuba2_clumps.sdf<\/tt> that we can view in Gaia. Go ahead and open the mask in Gaia:<\/p>\n<pre>gaia scuba2_clumps.sdf&amp;\r\n<\/pre>\n<div id=\"attachment_8072\" style=\"width: 304px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpmask.png\"><img aria-describedby=\"caption-attachment-8072\" loading=\"lazy\" class=\"size-medium wp-image-8072\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpmask-294x300.png\" alt=\"\" width=\"294\" height=\"300\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpmask-294x300.png 294w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpmask-147x150.png 147w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpmask.png 520w\" sizes=\"(max-width: 294px) 100vw, 294px\" \/><\/a><p id=\"caption-attachment-8072\" class=\"wp-caption-text\">Example of a clump mask (the output from findclumps)<\/p><\/div>\n<p>You can see the output is a mask over the full area of each clump, where the value of that clump is the same as the clump index in the output catalogue. Pixels not assigned to a clump are set as BAD values.<\/p>\n<p>The output file also has a special CUPID extension; for <tt>fellwalker<\/tt>, this contains an array of all the identified clumps with the data values from the original data included. You can access these like you do any other NDF extension, so to copy out (for example) clump 3 to its own NDF file you would do:<\/p>\n<pre>ndfcopy scuba2_clumps.sdf.more.CUPID.CLUMPS\\(3\\) clump3.sdf\r\n<\/pre>\n<p>Or you can even open that clump directly in Gaia with:<\/p>\n<pre>gaiadisp scuba2_clumps.sdf.more.CUPID.CLUMPS\\(3\\)\r\n<\/pre>\n<div id=\"attachment_8073\" style=\"width: 310px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3.png\"><img aria-describedby=\"caption-attachment-8073\" loading=\"lazy\" class=\"size-medium wp-image-8073\" src=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3-300x285.png\" alt=\"\" width=\"300\" height=\"285\" srcset=\"https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3-300x285.png 300w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3-158x150.png 158w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3-150x143.png 150w, https:\/\/www.eaobservatory.org\/jcmt\/wp-content\/uploads\/sites\/2\/2018\/01\/screenshot-clumpextension-clump3.png 305w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-8073\" class=\"wp-caption-text\">The 3rd Clump found by Fellwalker, as saved in the CUPID.CLUMPS extension.<\/p><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Note: This tutorial assumes that the computer to be used already has a functioning installation of the latest version of the Starlink software suite (the latest release is available for download here). This tutorial aims to go through some of the many analysis tools within Starlink. Get the data: Download\u2026 <a class=\"continue-reading-link\" href=\"https:\/\/www.eaobservatory.org\/jcmt\/science\/reductionanalysis-tutorials\/starlink-analysis\/\">Continue reading<\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"parent":4510,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/pages\/8051"}],"collection":[{"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/comments?post=8051"}],"version-history":[{"count":27,"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/pages\/8051\/revisions"}],"predecessor-version":[{"id":11542,"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/pages\/8051\/revisions\/11542"}],"up":[{"embeddable":true,"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/pages\/4510"}],"wp:attachment":[{"href":"https:\/\/www.eaobservatory.org\/jcmt\/wp-json\/wp\/v2\/media?parent=8051"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}