EOVSA flare pipeline

The frequency of observing calibrator sources during EOVSA solar observations is much less than that one typically would like to observe to properly take into account the instrumental gain variations. Hence self-calibration is often needed to calibrate the data to a level of satisfaction. A self-calibration pipeline suitable for generating calibrated dataset and quicklook images have been developed. Here we explain the various inputs of that pipeline and discuss various factors that is needed to be considered before supplying the input values. We also provide the steps for running the code on the pipeline machine.

Format of the pipeline

The pipeline for now consists of an input file and two codes. The input file is named as inputs.py . The other two codes are gen_IDB_MS.py and IDB_selfcal_pipeline_version.py. The code gen_IDB_MS.py is used to generate the measurement set (MS) from the raw files and calibrate the data using the gaintables derived from the calibrator observations. The second script is where the self-calibration happens. The main reason for having two separate codes is that "gen_IDB_MS.py" uses SUNCASA which for now runs on CASA versions<=5.4 . However, IDB_selfcal_pipeline_version.py requires CASA >=5.6. Hence for now the user needs to run the codes using the appropriate CASA versions.

Description of the inputs

An example inputs file in given below.

## Task handlers ###
cal_disk = 0 ## apply calibration tables from full disc imaging
identify_data_gap=1  ### identify data gaps
doslfcal = 1  # main cycle of doing selfcalibration
doapply = 0  # apply the results

# ============ declaring the working directories ============
### remember / is necessary in all the folder names

workpath = '/data1/testing/20211101/'
slfcaldir = workpath+ 'slfcal_v3/'  # place to put all selfcalibration products
imagedir = slfcaldir + 'images/'  # place to put all selfcalibration images
caltbdir = slfcaldir+'caltables/'  # place to put calibration tables
slfdisktbdir = slfcaldir + 'slfdisktb/'

# ============= time to image =================
starttime='2017-08-20 19:20:00'   ### has strict formating rules
endtime='2017-08-20 19:48:00'


# ============ selfcal parameters ===============
selfcal_solint=720 ### in seconds; this is the time difference between successive selfcal cycles
selfcal_avgint=720 ### in seconds; this is the time which will be averaged during selfcal
do_selfcal_at_end=False ### if True enforces, that a selfcal at the both ends are tried.
selfcal_spw='0~'+str(maximum_spw)  ### blank means all
refantenna = '0'
calc_cell=True ### If set to False use the value in beam given below
cell=[10]  ### size needs to be same as the number of spw listed in selfcal_spw
calc_imsize=True   ### is False uses the value given below
imsize=42  ### in solar radius, the full image size at the first frequency. Other frequencies, the value will be scaled.
           ### The default value of 42 solar radius is for ~1 GHz

max_frac_freq_avg=0.5  ### I will average at most this much fractional bandwidth

maxiter=10  ### maximum selfcal iterations
uvlim=25
avg_spw_max=5
final_clean_int=60
final_clean_cadence=60
flag_antennas = '' ###anything except 13~15. Those antennas are always flagged. 
phasecenter=''

# ========== end of input parameters =================

The task handlers listed in the inputs.py controls what functions will be performed by the IDB_selfcal_pipeline_version.py. A value of 0 means that functionality will be run. Please give 1 as input if the said task is desired. The next group of inputs are the working directories. After that, the time duration of interest should be provided. Please note that the format used when providing the starttime and endtime should be followed exactly.