Reference Gain Calibration

From EOVSA Wiki
Revision as of 04:11, 22 December 2017 by Schhabra (talk | contribs)
Jump to navigation Jump to search

Reference complex gain calibration refers to determining the "system" amplitude and phase with high precision as a function of IF band (i.e. integrated over each 500 MHz IF band). A typical reference calibration would cover 34 bands, 2 polarizations, and 13 antennas. The routines (on pipeline machine) to retrieve and analyze the data are in the module refcal_anal.py. Start from a time range during which the reference calibration observation occurred (check the phasecal results to see which 1-hr scan had stable phases) 

import refcal_anal as ra
from util import Time
#example time range of the refcal scan that took place BEFORE the observation
trange=Time(['2017-04-08T14:00','2017-04-08T15:00'])


Now read the data from that timerange 

out=ra.rd_refcal(trange)

Note that there are optional arguments that can be given, such as projid='PHASECAL' (the default) and srcid='1229+060', but these are generally not needed.

To correct for differential feed rotation (for refcal data after 05:00 UT on 2017-Jul-01), you can use: 

out_corr = ra.unrot_refcal(out)

Take a look at the phases (averaged over each band): 

ra.graph(out_corr)

By default it displays only bands 5, 11, 17, 23 and polarization XX. To change the bands and polarization to display, use the "bandplt" and 'pol' parameters (better to check more bands). 

ra.graph(out_corr, bandplt=[5,13,19,25], pol=1)


Observations within this time range will be used for averaging to get the reference phases/amplitudes. Data will be flagged if the SNR is less than minsnr, which can be supplied. The default minsnr is set to 1.0. We can further determine a finer selection of time range within this scan that has clean and stable phases if needed, by setting the new timerange in "trange" below. 

refcal=ra.refcal_anal(out_corr,timerange=trange,scanidx=[0])

It will generate four plots. First and second are phases and amplitudes of the data similar as those produced by ra.graph(), but with the selected time range for averaging highlighted (one can use "bandplt" and "pol" to choose bands and polarization to display as well). The 3rd plot is the averaged phase vs. frequency and the 4th is the averaged amplitude vs. frequency. Those data will be our "analyzed" reference calibration data saved in the returned dictionary (named "refcal" here). The dictionary contains the actual complex values (refcal['refcal']), a flag array with the same shape (refcal['flag']), and some timestamps, e.g., refcal['t_mid'] is the middle of the time range used for averaging. Both refcal['refcal'] and refcal['flag'] have a shape of (15, 2, 34) -- which corresponds to # of antennas, # of polarizations, and # of bands.

If you determine some antenna, band, polarization need to be flagged/unflagged, they have to be manually changed in the returned refcal['flag'] array. In this example, Antenna 12 was not tracking, and the algorithm did not pick up all of them, so we have to do this manually -- setting refcal['flag'][11, :, :]=1 

refcal['flag'][11]=1

If satisfied with the results (by looking at the plots), we can go ahead and send them to the SQL database 

import cal_header as ch
ch.refcal2sql(refcal)

To retrieve the refcal data back from SQL database based on, e.g., a solar data one wish to calibrate. We can provide a time: 

t=Time('2017-04-07T19:40')
refcal=ra.sql2refcal(t)

The resulted refcal have three keys: 'amp' (15 x 2 x 34 array), 'pha' (15 x 2 x 34 array), and 'timestamp' (in jd). The refcal data found will be the closest one PRIOR to the supplied observation time.

Updating the Reference Calibrations Table

The table below lists the known good reference calibrations. To facilitate adding entries to this table, do the following, where the time is after the desired refcal, but before any later refcal: 

t=Time('2017-06-23T19:40')  # retrieves the refcal at 13:07 UT on that day
refcal=ra.sql2refcal(t)
ra.graph_results(refcal, savefigs=True)

This will create summary plots of the amplitude and phase, will write the two corresponding *.png files into the /common/webplots/refcal folder, and will print as a text string the lines needed to add an entry. Just cut from the terminal and paste into the wiki editor at the appropriate place to preserve time order.

List of Analyzed Reference Calibrations

Date Timestamp Source Obs Trange Scan Idx Avg Trange Bands Phase Plot Amp Plot Comments
2017/04/02 06:36:50 3c273 04:20~11:00 All 05:47~07:26 5, 7, 9, 11, 13, 15, 17, 19, 21, 23 Phase Amp Ant 12 was not tracking. Delay center change at 11:38:17.
2017/04/04 09:32:50 3c273 04:43~10:57 All 08:46~10:19 5, 7, 9, 11, 13, 15, 17, 19, 21, 23 Phase Amp Ant 12 was not tracking. Delay center change at 16:43:51.
2017/04/05 06:44:10 3c273 04:17~10:30 All 04:27~09:34 5, 7, 9, 11, 13, 15, 17, 19, 21, 23 Phase Amp Ant 12 was not tracking.
2017/04/06 09:54:50 3c273 04:14~10:49 All 05:39~08:11 5, 7, 9, 11, 13, 15, 17, 19, 21, 23 Phase Amp Ant 12 was not tracking.
2017/04/07 08:30:51 3c273 04:00~10:30 1, 3, 5, 7, 9, 11, 13, 15, 17 07:53~09:10 5~34 Phase Amp Ant 12 was not tracking.
2017/04/08 06:14:51 3c273 05:00~10:30 1, 3, 5, 7, 9, 11, 13 05:00~07:00 5~34 Phase Amp Ant 12 was not tracking. Ant 13 not working. Delay center change at 2017-04-08T03:12:26. Another refcal record is added at 2017-04-09T06:14:45.
2017/04/10 06:57:10 3c273 04:30~10:30 2, 6, 8 5~34 Phase Amp Ant 12 was not tracking. Delay center change at 2017-04-08 13:20:39.
2017/04/13 13:48:39 2136+006 0 5~23 Phase Amp
2017/04/16 06:36:48 3c273 03:00~11:00 05:55~07:20 5~34 Phase Amp Ant 12 was not tracking.
2017/04/17 06:41:51 3c273 03:29~10:07 All 05:55~07:29 5~34 Phase Amp Ant 12 was not tracking. Delay center change at 19:12:58.
2017/04/26 08:06:52 3c273 02:50~09:28 All 07:21~08:57 5~34 Phase Amp Ant 12 was not tracking. Delay center change at 04:37:28 and 11:51:26.
2017/06/11 12:56:22 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/14 02:33:19 1229+020 0 5~34 Phase Amp No calibration for: Ant 8 Ant 12
2017/06/14 13:01:19 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/15 13:05:48 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/16 13:05:48 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/20 13:06:26 0 5~34 Phase Amp No calibration for: Ant 1 Ant 12
2017/06/21 13:06:38 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/22 13:06:53 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/23 13:07:09 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/24 02:25:49 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/24 13:07:25 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/25 13:07:43 0 5~34 Phase Amp No calibration for: Ant 10 Ant 12
2017/06/26 13:15:47 0 5~34 Phase Amp No calibration for: Ant 10 Ant 12
2017/06/27 13:02:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/06/30 02:47:50 1229+020 0 5~34 Phase Amp No calibration for: Ant 12
2017/07/01 02:44:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 12
2017/07/02 03:01:20 1229+020 0 5~34 Phase Amp
2017/07/03 02:47:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 10
2017/07/04 02:47:49 1229+020 0 5~34 Phase Amp
2017/07/05 02:55:19 1229+020 0 5~34 Phase Amp
2017/07/06 02:47:49 1229+020 0 5~34 Phase Amp
2017/07/07 13:12:49 2253+161 0 5~34 Phase Amp
2017/07/09 02:35:49 1229+020 0 5~34 Phase Amp
2017/07/09 13:16:49 2253+161 0 5~34 Phase Amp
2017/07/10 13:14:17 0319+415 0 5~34 Phase Amp
2017/07/11 13:14:18 0319+415 0 5~34 Phase Amp
2017/07/12 13:14:24 0319+415 0 5~34 Phase Amp
2017/07/13 13:14:59 0319+415 0 5~34 Phase Amp
2017/07/14 13:15:35 0319+415 0 5~34 Phase Amp
2017/07/16 02:48:33 1229+020 0 5~34 Phase Amp
2017/07/16 13:16:48 0319+415 0 5~34 Phase Amp
2017/07/17 13:14:54 0319+415 0 5~34 Phase Amp
2017/07/18 13:15:31 0319+415 0 5~34 Phase Amp
2017/07/19 13:16:16 0319+415 0 5~34 Phase Amp
2017/07/20 13:16:49 0319+415 0 5~34 Phase Amp No calibration for: Ant 10
2017/07/21 13:20:57 0319+415 0 5~34 Phase Amp
2017/07/22 13:21:36 0319+415 0 5~34 Phase Amp
2017/07/23 13:34:22 0319+415 0 5~34 Phase Amp
2017/07/24 13:22:58 0319+415 0 5~34 Phase Amp
2017/07/25 13:23:38 0319+415 0 5~34 Phase Amp
2017/07/26 18:00:49 0319+415 0 5~34 Phase Amp
2017/07/28 02:42:36 1229+020 0 5~34 Phase Amp
2017/07/29 13:37:40 0319+415 0 5~34 Phase Amp
2017/07/31 13:27:05 0319+415 0 5~34 Phase Amp
2017/08/01 13:28:30 0319+415 0 5~34 Phase Amp
2017/08/02 13:29:13 0319+415 0 5~34 Phase Amp
2017/08/03 13:29:59 0319+415 0 5~34 Phase Amp
2017/08/05 02:35:58 1229+020 0 5~34 Phase Amp
2017/08/05 13:31:22 0319+415 0 5~34 Phase Amp
2017/08/06 13:32:06 0319+415 0 5~34 Phase Amp
2017/08/07 13:32:49 0319+415 0 5~34 Phase Amp
2017/08/08 13:33:33 0319+415 0 5~34 Phase Amp
2017/08/11 12:47:49 0319+415 0 5~34 Phase Amp
2017/08/12 12:43:50 0319+415 0 5~34 Phase Amp No calibration for: Ant 9
2017/08/13 12:39:49 0319+415 0 5~34 Phase Amp
2017/08/14 12:34:19 0319+415 0 5~34 Phase Amp
2017/08/15 12:31:50 0319+415 0 5~34 Phase Amp
2017/08/16 12:27:50 0319+415 0 5~34 Phase Amp
2017/08/17 12:23:49 0319+415 0 5~34 Phase Amp
2017/08/18 12:19:49 0319+415 0 5~34 Phase Amp
2017/08/19 12:15:49 0319+415 0 5~34 Phase Amp
2017/08/21 12:07:50 0319+415 0 5~34 Phase Amp
2017/08/22 12:03:49 0319+415 0 5~34 Phase Amp
2017/08/23 12:14:51 0319+415 0 5~34 Phase Amp
2017/08/24 11:55:49 0319+415 0 5~34 Phase Amp
2017/08/25 14:01:50 0319+415 0 5~34 Phase Amp
2017/08/26 13:57:49 0319+415 0 5~34 Phase Amp
2017/08/28 01:03:49 1229+020 0 5~34 Phase Amp
2017/08/29 00:59:49 1229+020 0 5~34 Phase Amp
2017/08/29 13:49:49 0319+415 0 5~34 Phase Amp
2017/08/30 13:49:49 0319+415 0 5~34 Phase Amp
2017/08/31 13:50:49 0319+415 0 5~34 Phase Amp
2017/09/01 13:51:49 0319+415 0 5~34 Phase Amp
2017/09/02 13:52:49 0319+415 0 5~34 Phase Amp
2017/09/03 13:53:49 0319+415 0 5~34 Phase Amp
2017/09/04 13:54:49 0319+415 0 5~34 Phase Amp
2017/09/05 13:54:49 0319+415 0 5~34 Phase Amp
2017/09/06 13:55:58 0319+415 0 5~34 Phase Amp
2017/09/07 13:56:49 0319+415 0 5~34 Phase Amp
2017/09/08 13:56:49 0319+415 0 5~34 Phase Amp
2017/09/10 00:01:49 1229+020 0 5~34 Phase Amp
2017/09/10 13:57:50 0319+415 0 5~34 Phase Amp
2017/09/11 13:58:49 0319+415 0 5~34 Phase Amp
2017/09/12 14:00:11 0319+415 0 5~34 Phase Amp
2017/09/14 04:15:49 2253+161 0 5~34 Phase Amp Refcal at around 14 UT is also good.
2017/09/14 14:01:48 0319+415 0 5~34 Phase Amp
2017/09/15 14:39:27 0319+415 0 5~34 Phase Amp
2017/09/16 13:46:56 0319+415 0 5~34 Phase Amp
2017/09/17 13:47:46 0319+415 0 5~34 Phase Amp
2017/09/18 14:05:49 0319+415 0 5~34 Phase Amp
2017/09/19 14:06:50 0319+415 0 5~34 Phase Amp
2017/09/22 14:08:49 0319+415 0 5~34 Phase Amp
2017/09/23 14:09:50 0319+415 0 5~34 Phase Amp
2017/09/24 14:10:49 0319+415 0 5~34 Phase Amp
2017/09/25 14:17:33 0319+415 0 5~34 Phase Amp
2017/09/26 14:10:49 0319+415 0 5~34 Phase Amp
2017/09/27 14:06:49 0319+415 0 5~34 Phase Amp
2017/09/28 14:02:49 0319+415 0 5~34 Phase Amp
2017/09/29 14:01:20 0319+415 0 5~34 Phase Amp
2017/09/30 13:54:50 0319+415 0 5~34 Phase Amp
2017/10/01 13:50:49 0319+415 0 5~34 Phase Amp
2017/10/02 13:47:28 0319+415 0 5~34 Phase Amp
2017/10/03 13:43:25 0319+415 0 5~34 Phase Amp
2017/10/04 13:40:52 0319+415 0 5~34 Phase Amp No calibration for: Ant 10
2017/10/06 02:42:39 2253+161 0 5~34 Phase Amp
2017/10/06 13:32:53 0319+415 0 5~34 Phase Amp
2017/10/07 13:30:22 0319+415 0 5~34 Phase Amp
2017/10/08 13:26:22 0319+415 0 5~34 Phase Amp
2017/10/10 02:31:49 2253+161 0 5~34 Phase Amp
2017/10/11 02:23:50 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/12 02:19:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/13 02:15:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/13 12:54:23 0319+415 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/14 12:50:22 0319+415 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/16 02:07:50 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/18 01:59:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/19 12:30:22 0319+415 0 5~34 Phase Amp No calibration for: Ant 12
2017/10/20 12:22:22 0319+415 0 5~34 Phase Amp
2017/10/21 12:22:22 0319+415 0 5~34 Phase Amp
2017/10/22 12:18:22 0319+415 0 5~34 Phase Amp
2017/10/24 12:10:22 0319+415 0 5~34 Phase Amp Few frequencies have unusually high amplitudes.
2017/10/24 01:35:50 2253+161 0 5~34 Phase Amp
2017/10/25 01:31:50 2253+161 0 5~34 Phase Amp
2017/10/26 12:02:22 0319+415 0 5~34 Phase Amp
2017/10/27 11:58:22 0319+415 0 5~34 Phase Amp
2017/10/28 11:54:22 0319+415 0 5~34 Phase Amp
2017/10/29 11:50:22 0319+415 0 5~34 Phase Amp Ant 8 was down.
2017/11/02 14:50:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/03 14:52:20 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/04 14:53:51 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/05 14:54:50 1229+020 0 5~34 Phase Amp Ant 3 was down. No calibration for: Ant 3
2017/11/07 00:39:49 2253+161 0 5~34 Phase Amp
2017/11/07 14:56:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/08 14:57:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/09 14:58:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/10 15:00:50 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/12 00:22:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/12 15:02:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/15 00:12:50 2253+161 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/15 15:07:00 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/16 15:08:14 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/17 15:09:29 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/19 00:13:26 2253+161 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/19 15:11:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/21 00:11:50 2253+161 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/22 00:10:49 2253+161 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/22 15:15:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/23 15:16:50 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/24 15:18:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/25 15:19:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/11/26 15:20:49 1229+020 0 5~34 Phase Amp No calibration for: Ant 3
2017/12/03 00:05:49 2253+161 0 5~34 Phase Amp
2017/12/04 15:29:13 1229+020 0 5~34 Phase Amp No calibration on Ant 3 and 6
2017/12/06 00:05:14 2253+161 0 5~34 Phase Amp First refcal on 2017-12-05 is bad. No Ant3 calibration.
2017/12/07 00:05:06 2253+161 0 5~34 Phase Amp First refcal on 2017-12-06 had Windscram. No Ant3 calibration.
2017/12/07 15:32:07 1229+020 0 5~34 Phase Amp No Ant 3 calibration.
2017/12/08 15:33:05 1229+020 0 5~34 Phase Amp No Ant3 calibration.
2017/12/13 15:37:19 1229+020 0 5~34 Phase Amp
2017/12/18 00:06:26 2253+161 0 5~34 Phase Amp No calibration for: Ant 4
2017/12/18 15:40:50 1229+020 0 5~34 Phase Amp No calibration for: Ant 4 Ant 6
2017/12/19 23:57:49 2253+161 0 5~34 Phase Amp
Retrieved from "http://ovsa.njit.edu//wiki/index.php?title=Reference_Gain_Calibration&oldid=2141"