Practical Calibration Tutorial: Difference between revisions
| Line 34: | Line 34: | ||
</pre> | </pre> | ||
* | * find Miriad files in the database. | ||
<pre style="background-color: #FCEBD9"> | <pre style="background-color: #FCEBD9"> | ||
import dump_tsys as dt | import dump_tsys as dt | ||
| Line 41: | Line 41: | ||
import os | import os | ||
trange = Time(['2017-08-21 20: | trange = Time(['2017-08-21 20:15:00', '2017-08-21 20:35:00']) | ||
info = dt.rd_fdb(Time('2017-08-21')) | info = dt.rd_fdb(Time('2017-08-21')) | ||
sidx = np.where( | sidx = np.where( | ||
| Line 49: | Line 49: | ||
</pre> | </pre> | ||
* Import Miriad file into CASA. | |||
<pre style="background-color: #FCEBD9"> | <pre style="background-color: #FCEBD9"> | ||
outpath = 'msdata/' | outpath = 'msdata/' | ||
| Line 59: | Line 61: | ||
modelms="", doscaling=False, keep_nsclms=False, udb_corr=True) | modelms="", doscaling=False, keep_nsclms=False, udb_corr=True) | ||
</pre> | </pre> | ||
idbfiles could be a string of the IDB file name, or a list of IDB files. The output is automatically named "IDB20170821202020.ms". | |||
* calibrate the CASA MS files. | |||
<pre style="background-color: #FCEBD9"> | <pre style="background-color: #FCEBD9"> | ||
msfiles = [outpath + ll + '.ms' for ll in filelist] | msfiles = [outpath + ll + '.ms' for ll in filelist] | ||
Revision as of 13:55, 26 July 2019
This page is a practical tutorial for calibrating EOVSA data. Note currently the calibration processes can only be done on EOVSA site machines (such as pipeline), as the calibration SQL database is only available there.
Preparation
First, identify whether or not EOVSA had the observation for the time range of the event (e.g., a flare). There are many ways to do this, but a nice way is to use the RHESSI Browser and check "EOVSA Radio Data" on the upper left. Use the time you identified to find the corresponding IDB file(s) under /data1/eovsa/fits/IDB/yyyymmdd/. *Typically*, each IDB data file has a 10-minute duration. Then, on pipeline, it is advised to go to your working directory and copy the IDB data into your directory. Note, never, NEVER, work directly on the IDB data in the original data directory! Here I use /data1/eovsa/fits/IDB/20170821/IDB20170821202020 (2017 Aug 21 C flare) as an example.
Check Status of Calibration Products
Check the calibration status page for status of the calibration products. Navigate to the date of interest and check the numbers under "r" (reference phase), "p" (daily phase calibration), and "tp" (total-power and attenuation calibration). "1" means the respective calibration product already exists. "0" means otherwise.
Reference Calibration and Phase Calibration
Total-Power and Attenuation Calibration
As introduced in this page, the total-power and absolute flux calibration is done by comparing with RSTN daily radio flux density measurements. It is always a good idea to check the quality of the calibration. To do that, first, start IPython in your working directory. Here I am loading all pylab modules (I know this is not a good way, but simple).
ipython --pylab
Check the quality of the absolute flux (total power) calibration for the particular day
In [1]: from util import Time
In [2]: import daily_xsp
In [3]: daily_xsp.cal_qual(Time('2017-08-21'))
This command will apply attenuation calibration and feed rotation from the database to total-power gain calibration data, and display a figure on the right showing the calibrated total-power spectrum on all antennas and all polarizations. The first scan on the figure is done with the array dwelling on the Sun and stepping through different attenuator levels. The second and third scans are done by moving across the solar disk. A perfect calibration would achieve something similar to the RSTN flux for all antennas and all polarizations. This is not always the case, however. But it helps us to evaluate the quality of the absolution gain calibration.
If the calibration is deemed not satisfactory, one could go back and re-generate the attenuator gain calibration (perhaps it was not generated in the first place). This can be done by running the following command on pipeline (you need to change the date for your desired data)
python /common/python/current/calibration.py '2017-08-21 21:36'
Note the time used here (21:36). Typically, two calibrations are done on 17:30/18:30 and 21:30. Each calibration takes about 5 min. The time selection indicates that we use the second calibration, and make sure that we wait until the second calibration is done.
Apply All Calibrations to Visibility Data
Start CASA in your working directory
casa
- find Miriad files in the database.
import dump_tsys as dt
from util import Time
import numpy as np
import os
trange = Time(['2017-08-21 20:15:00', '2017-08-21 20:35:00'])
info = dt.rd_fdb(Time('2017-08-21'))
sidx = np.where(
np.logical_and(info['SOURCEID'] == 'Sun', info['PROJECTID'] == 'NormalObserving') & \
np.logical_and(info['ST_TS'].astype(np.float) >= trange[0].lv, info['ST_TS'].astype(np.float) <= trange[1].lv))
filelist = info['FILE'][sidx]
- Import Miriad file into CASA.
outpath = 'msdata/'
if not os.path.exists(outpath):
os.makedirs(outpath)
inpath = '/data1/eovsa/fits/IDB/{}/'.format(trange[0].datetime.strftime("%Y%m%d"))
ncpu = 1
importeovsa(idbfiles=[inpath + ll for ll in filelist], ncpu=ncpu, timebin="0s", width=1, visprefix=outpath,
nocreatms=False, doconcat=False,
modelms="", doscaling=False, keep_nsclms=False, udb_corr=True)
idbfiles could be a string of the IDB file name, or a list of IDB files. The output is automatically named "IDB20170821202020.ms".
- calibrate the CASA MS files.
msfiles = [outpath + ll + '.ms' for ll in filelist]
concatvis = os.path.basename(msfiles[0])[:11] + '_concat.ms'
vis = calibeovsa(msfiles, caltype=['refpha', 'phacal'], interp='nearest', doflag=True, flagant='13~15',
doimage=False, doconcat=True,
msoutdir='msdata', concatvis=concatvis, keep_orig_ms=True)
idbfiles could be a string of the IDB file name, or a list of IDB files. The output is automatically named "IDB20170821202020.ms".