User guide to the e-MERLIN data reduction pipeline -------------------------------------------------- This document is intended only as a guide to using the e-MERLIN data reduction pipeline, it is not meant to be a data reduction tutorial. For more detailed information on calibrating interferometers, see the AIPS cookbook or any of the other standard texts. The pipeline is written in Python and uses ParselTongue to interface with AIPS. To run it, you will therefore need working installations of AIPS, Python and ParselTongue. The scripts have been tested with AIPS 31DEC11 and Python 2.7. For the gory details on ParselTongue, including installation instructions, see the JIVE wiki: http://www.jive.nl/dokuwiki/doku.php?id=parseltongue:parseltongue ############################### # Before running the pipeline # ############################### By this point, your data should have been loaded, uvfix'd, msort'd, indxr'd (e.g. by one of the load scripts, or by hand if you're feeling masochistic), flagged in frequency (using your favourite methods), avspc, dbcon, msort, indxr (the dbcon script carries out these procedures), optional final ibled/edita/whatever (BY THE USER) and be ready for calibration. Note that things will go *much* faster if your data have been averaged in time and/or frequency. ############### # Quick start # ############### Once your data have been prepared and flagged, invoke the pipeline with the following incantation: argo@dop250> parseltongue eMERLIN_pipeline.py calib.inputs where calib.inputs is a plain text file giving a set of input parameters for the script (the file can be called anything you like). There is an example inputs file below. Only a few parameters are required, everything else is optional and will be set to sensible defaults if they are missing from the inputs file. Required inputs: userno = 667 # AIPS user number indisk = 1 # AIPS disk to use targets = M82 # comma-separated lists of target source(s) phsrefs = M82A # comma-separated list of phase cal(s) fluxcals = 1331+305 # Flux cal(s) pointcals = 1407+284 # Point cal bpasscals = 0555+398 # Bandpass cal pointflux = 2.4,2.4,2.4,2.4 # Point cal flux list for each IF (Jy) Things you need to know: ------------------------ * The file "eMERLIN_tasks.py" contains task definitions and is required by the pipeline. * If there is more than one file in the specified AIPS catalogue, the script will complain and quit. * If you have tables (e.g. from a previous calibration) that you want to keep, make a backup outside of AIPS before running the pipeline because it will delete old tables before starting. * Diagnostic plots will (if output directory is not specified) be dumped to the current working directory. * If you have more than one target/phscal pair, you can specify them as comma separated lists. This is all the information you need to get going. For the gory details, keep reading. ############### # The details # ############### The script will first check the specified AIPS catalogue for existing files from a previous run, exiting with a complaint if it finds more than one file. Assuming it finds just one file, it then removes stale calibration tables (any SN, BP, PL tables, and any CL>1, so make a backup if you want to keep your existing tables) before proceeding with calibration. The basic outline of the process is as follows: ----------------------------------------------- (optionally) load DBCON'd data file from disk (optionally) msort & indxr output some pre-calibration diagnostic plots initial fringe fit on everything but the target(s) clcal the solutions plot some diagnostics setjy on the pointcal calib (phase only on the phase cal) clcal the solutions calib (amp and phase) getjy clcal the solutions bandpass calibration (optionally) iterate selfcal on the phase calibrator map the phase cal and source There is no option for continuum/wide-field/line imaging since it is assumed that the data contain more than one channel, a bandpass calibration is carried out anyway and wide-field imaging is carried out. Spectral line imaging will be implemented later. Calibrator information: ----------------------- Note that the e-MERLIN system currently lists 3C286 and OQ208 by their IAU designations (1331+305 and 1407+284 respectively). The name of each source given in the inputs to the script must match those in the fits file's source table. Flux calibration info For channel 1 in IF 1 at 4128 MHz on the Mk2 - Pi baseline we (currently, Summer 2011) expect the following: IF1 3C286 = 7.705 Jy OQ208 = 2.49 Jy IF2 = 7.558 2.48 IF3 = 7.416 2.47 IF4 = 7.279 2.44 Running the script ------------------ Once your data have been prepared and flagged, invoke the pipeline with the following incantation: argo@dop250> parseltongue eMERLIN_pipeline.py calib.inputs where calib.inputs is a plain text file giving a set of input parameters for the script (the file can be called anything you like). There is an example inputs file below. Only a few parameters are required, everything else is optional and these will be set to sensible defaults if they are missing from the inputs file. Required inputs: userno = 667 # AIPS user number indisk = 1 # AIPS disk to use targets = M82 # comma-separated lists of target source(s) phsrefs = M82A # comma-separated list of phase cal(s) fluxcals = 1331+305 # Flux cal(s) pointcals = 1407+284 # Point cal bpasscals = 0555+398 # Bandpass cal pointflux = 2.4,2.4,2.4,2.4 # Point cal flux list for each IF (Jy) Optional inputs: refant = Mk2 # Reference antenna - this defaults to Mk2, but obviously this is not always possible. The pipeline will check if the antenna is present, if not it will select one for you. Choices are: Mk2, Pi, Da, Kn, De, Cm, Lo. solint = 10 # Specify the solution interval in minutes dosort = 1 # Do we need to do a sort? Runs MSORT and INDXR. Defaults to 1. If you are *sure* your data are already sorted, turn this off (dosort = 0) to speed up processing. msgkill = -5 # How chatty do you want AIPS to be? Defaults to -5. Messages are printed to the terminal and AIPS messages are written to eMERLIN_pipeline.log in the current working directory. doselfcal = 0 # Do selfcal on phase calibrator? 0=no, any other positive integer = this many rounds of selfcal. Default is 0. imsize = 1024 # Parameters for imaging. The imsize parameter only applies to your target field, not the calibrators. cellsize = 0.02 # Parameters for imaging. Note that the cellsize will be reset if you set it outside a sensible range for the frequency given in the FITS header. plotdir = /my/dir/ # Where to dump the plots if not in interactive mode, defaults to the current working directory if not specified. fitsfile = data.fits # If your data require loading, name of fitsfile on disk. fitsdir = /my/dir/ # If your data require loading, location of fitsfile on disk. Defaults to the current working directory. fring_snr = 9 # SNR threshold for fring, defaults to 9. ######################################################### # Example inputs for the e-MERLIN calibration pipeline. # ######################################################### ################### # Required inputs # - these must be specified before the script will run ################### # AIPS info userno = 667 indisk = 1 # Lists of sources to process. For multiple sources you can specify comma-separated lists of phsrefs and targets. targets = 1512+470 phsrefs = 1500+478 fluxcals = 1331+305 bpasscals = 0555+398 pointcals = 1407+284 # List of fluxes to use for your pointcal defined above pointflux = 2.4885, 2.4705, 2.4580, 2.4305 ################### # Optional inputs # - recommended, but will be set to sensible defaults if not set here ################### # Reference antennas. Choices are: Mk2, Pi, Da, Kn, De, Cm, Lo # The default refant is Mk2 (if in the array) refant = Mk2 solint = 10 # Do we need to do a sort? Runs MSORT and INDXR. (0 -> no, 1 -> yes, defaults to 1 if not set here) # If you are *sure* your data are already sorted, turn this off (dosort = 0) to speed up processing. Defaults to 1. dosort = 0 # How chatty do you want AIPS to be? Defaults to -5. msgkill = -5 # Do selfcal on phase calibrator? 0=no, any other positive integer = this many rounds of selfcal. Default is 0. doselfcal = 3 # Parameters for imaging. Note that the cellsize will be reset if you set it outside a sensible range for the frequency given in the FITS header. # The imsize parameter only applies to your target field. imsize = 2048 cellsize = 0.02 # Where to dump the plots if not in interactive mode, defaults to the current working directory #plotdir = /home/argo/e-MERLIN/data # Directory containing fitsfile to load if your data are not already loaded in AIPS. # If the file is in the current working directory, you only need to specify "fitsfile". #fitsfile = myfitsfile.fits #fitsdir=/home/argo/e-MERLIN/data/ # snr for fring, defaults to 9 fring_snr = 9 ############################### # End of calib.inputs example # ############################### Written by mkargo 2011 Contact: argo@astron.nl