Practical Calibration Tutorial

This page is a practical tutorial for calibrating EOVSA data. Note currently the calibration processes can only be done on EOVSA site machines (such as pipeline), as the calibration SQL database is only available there.

Preparation

First, identify whether or not EOVSA had the observation for the time range of the event (e.g., a flare). There are many ways to do this, but a nice way is to use the RHESSI Browser and check "EOVSA Radio Data" on the upper left. Use the time you identified to find the corresponding IDB file(s) under /data1/eovsa/fits/IDB/yyyymmdd/. *Typically*, each IDB data file has a 10-minute duration. Then, on pipeline, it is advised to go to your working directory and copy the IDB data into your directory. Note, never, NEVER, work directly on the IDB data in the original data directory! Here I use /data1/eovsa/fits/IDB/20170821/IDB20170821202020 (2017 Aug 21 C flare) as an example.

Check Status of Calibration Products

Check the calibration status page for status of the calibration products. Navigate to the date of interest and check the numbers under "r" (reference phase), "p" (daily phase calibration), and "tp" (total-power and attenuation calibration). "1" means the respective calibration product already exists. "0" means otherwise.

Total-Power and Attenuation Calibration

Checking total-power calibration quality

As introduced in this page, the total-power and absolute flux calibration is done by comparing with RSTN daily radio flux density measurements. It is always a good idea to check the quality of the calibration. To do that, first, start IPython in your working directory. Here I am loading all pylab modules (I know this is not a good way, but simple).

ipython --pylab

Check the quality of the absolute flux (total power) calibration for the particular day

In [1]: from util import Time
In [2]: import daily_xsp
In [3]: daily_xsp.cal_qual(Time('2017-08-21'))

This command will apply attenuation calibration and feed rotation from the database to total-power gain calibration data, and display a figure on the right showing the calibrated total-power spectrum on all antennas and all polarizations. The first scan on the figure is done with the array dwelling on the Sun and stepping through different attenuator levels. The second and third scans are done by moving across the solar disk. A perfect calibration would achieve something similar to the RSTN flux for all antennas and all polarizations. This is not always the case, however. But it helps us to evaluate the quality of the absolution gain calibration.

If the calibration is deemed not satisfactory, one could go back and re-generate the attenuator gain calibration (perhaps it was not generated in the first place). This can be done by running the following command on pipeline (you need to change the date for your desired data)

python /common/python/current/calibration.py '2017-08-21 21:36'

Note the time used here (21:36). Typically, two calibrations are done on 17:30/18:30 and 21:30. Each calibration takes about 5 min. The time selection indicates that we use the second calibration, and make sure that we wait until the second calibration is done.

Apply All Calibrations to Visibility Data

Start CASA in your working directory

casa

• find Miriad files in the database.

import dump_tsys as dt
from util import Time
import numpy as np
import os

trange = Time(['2017-08-21 20:15:00', '2017-08-21 20:35:00'])
info = dt.rd_fdb(Time('2017-08-21'))
st_ts = info['ST_TS'].astype(np.float)
sidx = np.where(
    np.logical_and(info['SOURCEID'] == 'Sun', info['PROJECTID'] == 'NormalObserving') & \
    np.logical_and(st_ts >= trange[0].lv, st_ts <= trange[1].lv))
filelist = info['FILE'][sidx]

• Import Miriad file into CASA.

outpath = 'msdata/'
if not os.path.exists(outpath):
    os.makedirs(outpath)
inpath = '/data1/eovsa/fits/IDB/{}/'.format(trange[0].datetime.strftime("%Y%m%d"))
ncpu = 1
importeovsa(idbfiles=[inpath + ll for ll in filelist], ncpu=ncpu, timebin="0s", 
            width=1, visprefix=outpath,nocreatms=False, doconcat=False,
            modelms="", doscaling=False, keep_nsclms=False, udb_corr=True)

idbfiles could be a string of the IDB file name, or a list of IDB files. The output is automatically named "IDB20170821202020.ms".

• calibrate the CASA MS files.

msfiles = [outpath + ll + '.ms' for ll in filelist]
concatvis = os.path.basename(msfiles[0])[:11] + '_concat.ms'
vis = calibeovsa(msfiles, caltype=['refpha', 'phacal'], interp='nearest', doflag=True, flagant='13~15',
                 doimage=False, doconcat=True,
                 msoutdir='msdata', concatvis=concatvis, keep_orig_ms=True)

idbfiles could be a string of the IDB file name, or a list of IDB files. The output is automatically named "IDB20170821202020.ms".