Year: 2019

A new parameter called SMOOTH450 has been added to the pol2map script (used for processing POL2 data). It is only accessed when processing 450 um data and defaults to False, which results in no changes to the behaviour of pol2map. If SMOOTH450 is set to True, pol2map performs an additional smoothing that results in the resolution of the resulting I, Q and U coadds (and the vector catalogue) being degraded to the resolution expected for 850 um data. This allows 450 and 850 um results to be compared more easily. A side-effect of the smoothing is that the noise levels in the final 450 um maps and catalogues is lower.

The smoothing is applied to the I, Q and U maps made from each individual observation before using them to form the I, Q and U coadds. The same smoothing kernel is used for I, Q and U and is formed by deconvolving the expected 850 um beam shape using the expected 450 um beam shape as the PSF (Point Spread Function). The “expected” beam shapes are two-component Gaussians as described in “SCUBA-2: on-sky calibration using submillimetre standard sources” (Dempsey et al, 2013, MNRAS, 430, 2534).

The following plot shows the the mean radial total intensity profile of 3C279 determined from 25 POL2 observations. The black curve shows the 450 um profile produced using SMOOTH450=NO (the default), the red curve shows the 450 um profile produced using SMOOTH450=YES and the blue curve shows the 850 um profile. It can be seen that the smoothed 450 curve is very similar to the 850 curve.

The following vector maps shows 450 um vectors created from four observations of DR21. The blue vectors were created with SMOOTH450=NO and the red vectors with SMOOTH450=YES (for comparison, the right hand map shows the same red vectors without the blue vectors). Both red and blue used the same selection criterion (i>5*di && dp <0.5 && dang < 10). The main difference is that there are many more red vectors than blue vectors. This is because the smoothing introduced by SMOOTH450 has reduced the noise level, thus allowing more vectors to pass the above selection criterion.

A potentially serious bug has recently been found and fixed in the POL2MAP command.

Summary: If you run POL2MAP three times to produce a vector catalogue – once to create the auto-masked I map (step 1), once to create the externally-masked I map (step 2) and once to create the Q/U maps and vector catalogue (step 3) – and you set the BINSIZE parameter to a value larger than the PIXSIZE parameter (4 arc-seconds by default), then the I (total intensity) and P (polarisation) values in the final catalogue may be badly wrong (the angles in the vector catalogue are unaffected by this bug). But if you combine steps 2 and 3 into a single invocation of POL2MAP (as is done in the on-line POL2 tutorial), or if you do not set a value for the BINSIZE parameter, then all the values in the vector catalogue will be correct.

This bug has now been fixed and so it is safe to use the 3-step process with a large BINSIZE if you rsync starlink from EAO on 31st October or later. But existing vector catalogues created previously using  the 3-step process with a large BINSIZE should be treated with caution. As noted above, the I and P values in such catalogues could be badly wrong, but the ANG values will be correct. It may be advisable to re-create such catalogues using the new software and compare the old and new catalogues for differences. Note, since the I values may have changed, any vector selection criteria based on the I value or SNR may be affected.

Details: When the POL2MAP script needs to create an I, Q or U coadd map, it will re-use any previously created coadd if such a coadd exists with the correct name. This is what happens in the typical 3-step processing mode: step 3 re-uses the externally masked I coadd created at step 2. If the requested vector catalogue bin size (parameter BINSIZE) is set to a value larger than the map pixel size (parameter PIXSIZE), then the coadd maps are first binned up to the requested bin size before creating the catalogue. However, the bug described in this post resulted in this binning stage being skipped for any coadd that was created by a previous run of POL2MAP. So in a typical 3-step process (with BINSIZE set to, say, 12 arc-seconds) the Q and U coadds were correctly binned up to a pixel size of 12 arc-seconds at step 3 prior to creating the catalogue, but the pre-existing I coadd (created at step 2 with 4 arc-second pixels) was not binned up. This means that the I pixel value used to form each vector would come from the wrong point on the sky, resulting in both the I and the P (polarisation) value being wrong in the catalogue. Since the Q and U coadds are both created at the same time as the vector catalogue (i.e. in step 3) they would be binned correctly, and so the vector angles in the catalogue would be correct.

Summary:

New IP models are now available when reducing POL2 data. At 450 um the new model is a distinct improvement over the previous model. At 850 um the new and old models show little difference. To use the new models, it is necessary to add ipmodel=AUG2019 to the configuration when running pol2map:

% pol2map config='ipmodel=AUG2019'

Details:

Some POL2 users have seen evidence of significant inaccuracy in the model used to correct 450 um data for Instrumental Polarisation (IP). For instance, this can be seen in the following plots, which compare Q maps for DR21, created from observations taken in 2015 and 2017. The bottom left image shows the 450 um Q map from 2017 data, the bottom centre image shows the 450 um Q map from 2015 data and the  bottom right image shows the difference.  All these images use the same scaling. The difference appears to be a scaled version of the total intensity image (shown at top right), as confirmed by the scatter plot shown in the top centre.

The fact that the difference between the 2015 and 2017 Q maps looks like a scaled version of the total intensity map suggests strongly that the difference is caused by an inaccuracy in the IP correction (the IP correction subtracts a fraction of the total intensity map from the Q and U maps).

The current model used for IP correction is based on a set of observations of a bright unpolarised point source (Uranus). However, the high noise and low Q/U levels at 450 um in these Uranus observations makes it difficult to determine the model accurately. So a procedure has been developed that allows the IP model to be determined instead from a set of observations of any bright extended (possibly polarised) sources. This procedure has been applied to existing observations of OMC1/OrionA, DR21 and G034.257+0.155 to determine new IP models at both 850 um and 450 um. Observations of OrionB and Serpens main field2 were also used, but failed to produce any usable data at 450 um.

The new procedure is described in detail in the attached file.

Using the new 450 um IP model, there seems to be no significant difference between the Q/U maps made from the 2015 data and the 2017 data for DR21, as shown below:

and for completeness the corresponding U maps are shown below – again there is no significant difference between 2015 and 2017:

The map-making process used by the pol2map command uses two masks, each of which divides the field up into source and background regions:

• The ‘AST’ mask: this is used to define the background regions that are to be forced to zero after each iteration of the map-maker algorithm (except the last iteration). This form of masking helps prevent the growth of artificial large scale structures within the map. Any real astronomical signal present within the masked background regions will tend to be suppressed in the final map, so it is important that the AST mask correctly identifies regions of significant emission down to a low level.
• The ‘PCA’ mask: this is used to define the source regions that are to be excluded from the Principal Component Analysis. This analysis is used to remove the correlated backgrounds in the bolometer time stream data. The time-stream data for astronomical sources are not correlated across bolometers, and so tend to disrupt the PCA. For this reason source regions are excluded from the analysis.

Two separate masks are used because experience has shown that disruption of the PCA is caused mainly by the brighter central source regions. Consequently, the source regions within the PCA mask can  be smaller than the source regions within the AST mask.

A total intensity map of DR 21 showing the AST mask in green and the slightly smaller PCA mask in blue.

Default masks are created automatically by pol2map in a manner specified by the MASK parameter.

1. On ‘step 1’ of a typical POL2 data reduction, MASK is left at its default valuer of “Auto”, causing new masks to be generated automatically at the end of each iteration of the map-making algorithm. This ‘auto-masking’ process identifies an initial set of sources by thresholding the current map estimate at the SNR value specified by configuration parameter “xxx.ZERO_SNR”, where “xxx” is either “AST” or “PCA”, depending on which mask is being generated. Each of these initial source regions is then expanded to include adjoining pixels down to the SNR level specified by configuration parameter “xxx.ZERO_SNRLO”.
2. On ‘step 2’ and ‘step 3’ of a typical POL2 data reduction, MASK is set to the coadd of all the total intensity maps created at step 1. The pol2map script first creates a pair of AST and PCA masks from the supplied coadded map and then uses these masks on all iterations of the map-making algorithm. The findclumps command in the Starlink CUPID package is used by pol2map to create the masks. The process used by findclumps is the same as describe above for step 1 – initial sources are defined by a fixed SNR threshold within the supplied coadd and these are then extended down to a lower SNR threshold. The pol2map command sets these threshold values to the values of the four configuration parameters listed above – “AST.ZERO_SNR”, “AST.ZERO_SNRLO”, “PCA.ZERO_SNR” and  “PCA.ZERO_SNRLO”.

These configuration parameter all default to the following values specified in the pol2map script:

AST.ZERO_SNR = 3 
AST.ZERO_SNRLO = 2 
PCA.ZERO_SNR = 5 
PCA.ZERO_SNRLO = 3

If you wish to investigate the effects of changing these values, you should supply new values using the CONFIG parameter of the pol2map command. For instance:

% more conf
ast.zero_snr = 2.5
ast.zero_snrlo = 1.5
% pol2map config=^conf

Any values not specified will retain the default value listed above.

Note, prior to 10th July 2019 the above method could only be used at step 1 (the supplied settings were ignored if supplied at step 2 or 3). Later versions of pol2map do not suffer from this problem – the supplied values are honoured at all steps.

Footnote – for completeness it should be mentioned that the  COM and FLT models are also masked, in addition to the AST and PCA models. At step 1 (the auto-masking stage) the masking of COM and FLT is controlled by a similar set of configuration parameters to AST or PCA, except that “XXX” becomes “COM” or “FLT”. At steps 2 and 3 (the external-masking stages), the COM and FLT models are masked using the PCA mask generated by findclumps, and so COM and FLT masking cannot be controlled independently  of the PCA mask.

Vector maps created from POL2 data need to have the instrumental polarisation (IP) removed to be useful. Determining a good model for the IP has been a challenge, and not surprisingly the currently accepted model has somewhat uncertain parameter values.  These uncertainties in the IP model translate into uncertainties in the final vector maps in complicated ways that cannot easily be estimated analytically. However, a possibly approach to estimating the uncertainties in the vector map is to produce several vector maps using slightly different IP models and then look at the differences between the resulting maps.

The pol2map command provides two options for doing this:

1. The level of IP – as a percentage if the incoming total intensity – removed by the IP model can be raised or lowered by a specified amount from its default value. To do this, use configuration parameter “ipoffset“. For instance, to create a vector map using an IP level that is 0.3% greater than the default, do:% cat conf
ipoffset=0.3
% pol2map config=^conf ...
2. The IP removed by the IP model can be rotated by a specified angle from its default orientation. To do this, use configuration parameter “ipangoff“. For instance, to create a vector map using an IP at an angle of 0.5 degrees to its default angle, do:% cat conf
ipangoff=0.5
% pol2map config=^conf ...

The above changes should be applied when running pol2map to create the Q and U maps, together with the final vector catalogue (“step 3”).

Previous investigation of the IP model suggests that the above values for ipoffset and ipangoff (i.e. 0.3 % and 0.5 deg.) are reasonable estimates of the uncertainties in the IP model parameters. Rotating the IP by up to 0.5 degree should usually have little effect on the final map, so it is probably  reasonable to ignore ipangoff and concentrate on  the effects of changing ipoffset. A possibly strategy is to use pol2map (step 3) three times to generate three sets of Q and U maps, with corresponding vector catalogues, using ipoffset values of -0.3, 0 and +0.3. The visual differences between the vector maps should give a handle on the uncertainties caused by the IP model.

The script I described yesterday (see Checking for convergence in pol2map log files) had a few bugs. Please download it again if you are using it.

When processing POL2 data, it is important to know that the map-making algorithm converged correctly for all observations. This information is available in the pol2map log file, along with the rest of the makemap or skyloop output.  However, the log file can be very long and so finding the relevant information may  not be straight-forward, particularly if you are unfamiliar with the screen output usually created by makemap or skyloop.

To help with this, I have written a simple python script called pol2logcheck.py, which searches a specified pol2map log file for the relevant information and reports any observations that did not converge. Use it as in the following example:

% pol2logcheck.py omc1/pol2map.log.3

omc1/pol2map.log.3: 

 Looks like a step 1 log file
  The following observation(s) failed to converge:
    20190104 #14

If you wish to combine POL2 observations for multiple overlapping fields, the best way to proceed is probably as follows (note, for this to work correctly you will need a Starlink build from 9th June 2019 or later):

1. 1.  Run “step 1” independently for each field. In other words, use pol2map to create an auto-masked total intensity map for each field. The following assumes that pol2map is run within directories called “field1″, field2”, etc, to create the auto-masked maps and the I/Q/U time-stream data for each field.
   2. Co-add the auto-masked total intensity maps for all fields. First create a text file holding the paths to the separate auto-masked total intensity maps, and then run pol2map as follows:

      % more infiles
      field1/iauto
      field2/iauto
      % pol2map in=^infiles iout=iauto_mosaic \
                qout=! uout=! multiobject=yes
      
      

   3. Run “step 2” and “step 3” for each field, using the mosaic created above as the mask field for all fields. For instance, for the first field:

      % cd field1
      % pol2map in=qudata/\* iout=iext qout=! \
                uout=! mapdir=maps mapvar=yes \
                skyloop=yes mask=../iauto_mosaic 
      % pol2map in=qudata/\* iout=! qout=qext \
                uout=uext mapdir=maps mapvar=yes \
                skyloop=yes mask=../iauto_mosaic \
                ipref=iext cat=mycat debias=yes

      Then do the same for field2, field3, etc. If preferred, steps 2 and 3 can be combined into a single invocation of pol2map:

      % cd field1
      % pol2map in=qudata/\* iout=iext qout=qext \
                uout=uext mapdir=maps mapvar=yes \
                skyloop=yes mask=../iauto_mosaic \
                ipref=iext cat=mycat debias=yes

   4. Co-add the external-masked I/Q/U maps for all fields and create a vector catalogue from the co-added maps. First create a text file holding the paths to the external-masked I/Q/U maps for all fields, and then run pol2map as follows:

      % more infiles
      field1/iext
      field1/qext
      field1/uext
      field2/iext
      field2/qext
      field2/uext
      % pol2map in=^infiles iout=iext_mosaic \
                qout=qext_mosaic uout=uext_mosaic \
                multiobject=yes cat=mycat_mosaic \
                debias=yes ipcor=no

A bug has recently been fixed in the SMURF pol2map command  that could cause the maps created by pol2map to be blurred or show bad negative bowling around bright sources if the input observations do not all use the same reference point. So if you are using pol2map to combine data from POL2 observations with different reference positions, then you should ensure that you are using a version of starlink that includes the bug fix – i.e. was created on or after 10th January 2019. To check this, do:

% more $STARLINK_DIR/manifests/starlink.version

This will show details about your starlink build, including the date. To update it using rsync from Hilo, see these instructions.

The bug caused the pointing corrections determined at “step 1” to be in error by an amount roughly equal to the change in reference position between observations. These bad pointing corrections in turn lead to various problems (bad bowling or blurring) in the maps created at “step 2”. Consequently, it is not possible to correct an existing bad reduction – the only option is to do the whole reduction again from the start (having first ensured that your starlink installation is up to date).