2017 April

Back to Observing Log

Apr 01

23:50 UT We had some nice solar activity (M4 flare) around 21:50 UT, which was completely covered by RHESSI. I am taking some calibration data (3C84, pcal.fsq) starting around 00:20 UT, and hope that we can do some imaging of this event. The wind has been high for two straight days, but has finally subsided for the first time, to enable the calibration.

Apr 02

11:04 UT The system took data on 3C273 and 3C286 all night, using pcal.fsq and 30-min scans. The data look very nice, but I am not happy with the oscillations I saw in yesterday's solar data, which I suspect may be due to an unsynchronized correlator clock. Therefore, I rebooted the ROACHes (Kjell connected the 10 MHz reference to the signal generator on Friday), in the hopes that it will perform better in the long-term. I am now taking data on 3C286 with 3bands.fsq, with which to determine and correct delay changes due to the reboot. There was another M flare overnight, so more activity may be present today.

Apr 03

02:07 UT We got no less than three (!) M-class flares today (yesterday in UT day). The M2 flare observed after 1800UT was covered by RHESSI in its rise phase. The observations went well, and I also did a gain/attenuation calibration observation, to attempt to remove the effect of attenuation changes on total power and correlation data. Unfortunately, the wind came up this afternoon and we did not get a successful 3C84 calibration. We did get the morning and noon one, however.

Apr 04

Fig. 1: Demonstration of $\Delta B_{z}$ plot for 12 baselines, plotting $\Delta B_{z}$ (cm) vs. frequency using just the phase difference $\Delta \phi =\phi _{3C273}-\phi _{3C286}$, then again using $\Delta \phi -2\pi$ and a third time using $\Delta \phi -4\pi$. The set of points show a curve when an inappropriate phase offset is used, but a straight line at the correct $B_{z}$ value when the correct one is used.

04:20 UT I am running a new schedule called two_src_pa-track.scd, which is tracking the Ant14 position angle, alternating between 3C273 and 3C286 every 30 minutes, and using pcal.fsq. The wind is going to be a problem, it looks like... I corrected Ant 13's $B_{x}$ and $B_{y}$. I also have good $B_{z}$ corrections, but I am not sure of the sign, so I am afraid to set them. I discovered a nice way to plot the data (Figure 1), to find the $B_{z}$ correction without any $2\pi$ ambiguities.

04:42 UT After about 20 min, I noticed a problem with the schedule--it was not listing the type of observation as PHASECAL (called it PACAL instead). I updated the schedule to fix that, and while I was at it, I bit the bullet and set the $B_{z}$ values after all. We'll just see how it goes, and whether I have the wrong sign or not. I elected to just follow the same approach as for $B_{x}$ and $B_{y}$.

Fig. 2: A new measurement of $\Delta B_{z}$ (cm) after applying the corrections indicated in Figure 1. Now all of the errors are zero! Note the change of scale.

05:54 UT The data look great! The $B_{z}$ corrections are definitely an improvement, see Figure 2. Also, the PA rotation is working perfectly, so that the XY and YX phases on ants 1-8 are just noise, and the other polarizations are strong and flat. The wind is cooperating, so I suppose this will continue all evening and provide an excellent basis for checking all of the baseline corrections.

Fig. 3: Phase of 3C273 (blue) and 3C286 (orange) vs. hour angle based on observations on 2017-04-04, demonstrating near-perfect baseline corrections.

Fig. 4: Phase of 3C273 (blue) and 3C286 (orange) vs. frequency based on observations on 2017-04-04, demonstrating near-perfect baseline corrections. The blue points can almost not be seen, being behind and covered by the orange ones. Only ants 8 and 13 still need a little tweaking of $B_{z}$ position.

12:03 UT The calibration demonstrates that the baselines are nailed. Figure 3 shows the phases vs. HA on the lowest frequency. Figure 4 shows multi-band delays (phase vs. frequency) after integrating over the entire timerange. In Figure 3, the phases are identical, and essentially flat, for the two sources 3C273 (blue) and 3C286 (orange). In Figure 4, the phases are identical (blue points are almost invisible) over the entire frequency range, except there is perhaps a slight $B_{z}$ error on Ant 8 and, especially, Ant 13.

16:50 UT For some reason, this morning's calibrator observation did not run. I am running it now, although since it is already in solar daytime, I cannot run it for an hour. I will limit it to 30 minutes and hope no large flares occur. Also, I updated the delays (only very small changes were needed).

Apr 05

21:56 UT Just a note to say that I found the tunable LO not switching (must have missed the FSEQ-ON command somehow), and it was that way for about 20 minutes (since 21:35 UT). I issued FSEQ-ON, and it seems to be running now.

Apr 07

13:00 UT I did a calibration observation on 3C273 that alternated between the Lo and Hi frequency receivers all night, using two new sequences that covered every frequency band. The sequences are pcal_lo-all.fsq and pcal_hi-all.fsq. I did 10-min scans on the Lo receiver, and then 30-min scans on the Hi receiver. Although a few scans did not run due to wind, we had several nice ones, and the data quality is very good. Having every band revealed some 1-ns delay errors. We expect to run this sequence of observations as the standard one in the future. The schedule file was 3c273_lohi.scd, and uses two macro commands LOSELECT.ctl and HISELECT.ctl to switch receivers.

20:00 UT We had a nice C-class solar flare at about 19:50 UT.

Apr 08

04:35 UT I am repeating the calibration as for last night, after adjusting delays. The first scan went okay, but now the wind has come up and will spoil the next one. I notice that Ant13 front end is not returning data. I tried rebooting, but it did not help. So there will be no data from that antenna tonight...

13:21 UT The observations went well except for a couple of scans. Ant13 did come back toward the end of the night, but was not pointing well. The delays were all excellent, except, strangely, for Ant 8, which was off by 1 ns. I have updated that delay this morning. I am now about to do an experiment. I will do the PA-SWEEP exercise on bands 13, 15 and 17, in succession, on source 2136+006. I am hoping to learn something more about feed rotation.

15:12 UT I have analyzed the data, and it is interesting, but a bit too noisy. This would be worthwhile to repeat on 3C273 tonight. Actually, I could interrupt the solar schedule and do it on 3C84 when it becomes available around 19:00 UT. The problem is that it's parallactic angle is so large.

Apr 09

18:22 UT I have been working all morning on the gain levels in the system. I am very frustrated that there continues to be some sort of bug in the DCM_cal routine in roachcal.py. It seems to work when I fix the attenuation of the DCMs prior to the measurement (by issuing the dcmauto-off command). However, if I try to use dcmauto-on, which reads the current DCM table from the database, it seems to mix up the bands somehow. While working on this, I discovered that Ant 2 had the wrong FEM coefficients! It had the ones from Ant 1, which are very different. After I got this corrected, the unusually low Ant 2 DCM levels are gone! This has been a problem for 6 months or more... I also got Ant 14 DCM attenuations working, so it should be okay now to use dcmauto-on for all antennas.

What got me started on this is that I find the auto-correlations are way different from the power levels, meaning that the equalizer coefficients are not correct. Before fixing them, I wanted to make sure that the ADCs were in a good range, which they are now.

Apr 10

04:45 UT I have been doing more tests. At the end of the solar day, I did three PA-SWEEP scans on 3C84, each one a single frequency, with the idea of getting better SNR than I had earlier on 2136+006. That worked great, so I will be able to investigate these data in some detail and perhaps get a better idea of the effects of feed rotation. I also worked for awhile longer on changing the EQ coefficients, and it is very clear that setting them around 4 is much better than the current value of 32. I recall that I did that before and then did not like the results, but there is really no question that it is the right thing to do. I am now observing 3C273 with the broad frequency range (both receivers), to check that the results are okay, and make sure the phase does not change when the EQ coefficients are reset.

05:18 UT The results are in, and the delays are unchanged. It is not possible to tell if the data are in any way better with the more appropriate EQ coefficients. It will be more obvious with the solar data, which should have more structure (more dynamic range), but may also show more blemishes as a result. But linearity is important, and we have to have good EQ coefficients. Since Ant 14 can now do DCM attenuation vs. band, I just now edited PHASECAL.ctl to allow DCMAUTO-ON for that antenna. It will take effect on the next scan change, starting at 05:27 UT.

21:46 UT We had a very small, nearly invisible solar flare (B5), but when I looked at the amplitudes and phases on various baselines using the UDB file, it showed up very well! So I think my adjustments did help with the dynamic range, and the ability to see faint events.

Apr 11

03:02 UT Tonight (and possibly into tomorrow), I am doing interferometric pointing. It will be interesting to see if the results are clearer than before, now that we have better phases, delays, etc.

18:53 UT I have analyzed about 14 hours of pointing measurements, and updated the Ant 14 pointing parameters. The predicted improvement after applying the solution (observed - fit) is not really that good. The residuals are 0.018 deg after correction, 0.046 before correction). I have updated the pointing coefficients at 18:30 UT, and am now continuing with pointing measurements to see if they are an improvement.

The original pointing coefficients were as follows:

  P1    P2    P3    P4    P5    P6    P7    P8
-1875  1442  1470  -162   862  -127  9878   468

After this update, the new values were:

  P1    P2    P3    P4    P5    P6    P7    P8
-1510  1477  2457   667  1196  -163  9859   674

Apr 12

02:49 UT Well, clearly the pointing is NOT better, but I cannot find any error in what I am doing. The pointing is in terms of RAO and DECO, but the parameters are in HA and DEC, so I change the sign of the RAO to convert to HAO. Still, the error appears to be mainly in RA/HA, since the source is found in the RA direction, but not in the DEC direction. Now the wind has come up, and who knows how long that will last. I will just let it run overnight and analyze in the morning.

15:41 UT This morning the results are clearly worse than the original. I noted that the HA offsets are now positive, yet when I calculate new parameters I get a greater P1 coefficient. Therefore, I tried recalculating the offsets after changing the sign of the HA offsets. If I do that with yesterday's original measurements, I get new coefficients

  P1    P2    P3    P4    P5    P6    P7    P8
-3065  1694    51 -1807  1175  -234  9879   665

so as a test I have updated Ant 14 pointing coefficients to these values. I also tried calculating the predicted coefficients using the new measurements, and I get:

  P1    P2    P3    P4    P5    P6    P7    P8
-3898  3342   498 -2507  1805  -528  9867   642

In a perfect world, these two should agree, and I am concerned that they do not, especially P2. Still, I will let it take more data and see if things improve.

17:19 UT The pointing is still worse than it originally was. Looking into the documentation, it appears that the corrections are done in HA, not HA/cos(dec). I now realize that the equations have a cos(dec) term in them already, so in fact I should be providing offsets in HA, not angle on the sky (HA*cos(dec)). This must be the problem! I'll try it.

18:32 UT I discovered that I have not been converting to angle on the sky after all, so there is nothing to change. I am wasting time, so I switched back to the original coefficients and will go back to observing the Sun. I will keep going through the steps and try to find the problem, but I can use the existing data for this purpose, without additional observations. I realize now (too late, because the wind has come up) that it should be possible to test the pointing coefficients in the controller by pointing the antenna at a source, and then turning the pointing corrections on and off. The resulting virtual axis will change by an amount that I can calculate for myself. If the amounts do not agree, I will know that my calculation differs from what is in the controller (not a small concern, since I did discover a discrepancy in the documentation).

Apr 14

20:47 UT This morning I did a complete test of the 27-m pointing model, by setting each parameter to unity in turn, with all others zero, and observing the change in pointing by turning the pointing model on and off (there is a button for that purpose in the antenna controller software). I found that there were several problems that affected P3, P5 and P6, but they all boil down to the use of tan(HA) instead of the correct tan(Dec) in each of these three terms! This affects the HA axis only. I also found that I had the sign of P8 backwards, although my sign matches the pointing model that Mark Godwin claims he is using, so it is really his sign error.

22:03 UT I wrote a version of mountcal (anta_broken_mountcal) that uses the actual, broken pointing model in Ant 14 to fit the CALPNT offsets that I originally had. The updated pointing parameters are

  P1    P2    P3    P4    P5    P6    P7    P8
-4905  2372  2154 -2527  1356   -54  9162  -307 

so I entered those and will take another series of pointing measurements (starting 22:09 UT) to see if the pointing is any better. From the residuals of the fit I should expect somewhat better than a factor of 2 improvement in pointing RMS. I should know in 5-6 hours, I suppose.

Apr 15

12:30 UT I continued to take pointing data all night, and a new analysis gives very different numbers. I'll try one more time to update, but if this does not work I will give up until the controller pointing model is fixed. Here are the new numbers:

   P1    P2    P3    P4    P5    P6    P7    P8
-10201  4630  3924 -6559  2428    98  8172  -163 

15:27 UT The corrections are clearly the wrong sign in HA, and so I checked the code and realized that at some point as a test I removed the change of sign from RA to HA. I reinstated that in anta_broken_mountcal(), and the new parameters are:

  P1    P2    P3    P4    P5    P6    P7    P8
  448    23   459  1520   152   -41 10192   338 

Note how small most of the corrections are now. I have high hopes that this will finally improve the results. The next scan using these values will start at 15:40 UT.

19:25 UT Yes! Finally, the pointing is quite good over all of the sources I have observed since 15:40 UT. I will continue on other sources for awhile, and do one more update, then we'll observe some sources this evening. Yikes, I see that the wind is creeping up, and may cause wind scrams shortly.

23:22 UT I got a few more pointing measurements, but then the wind came up. I am worried that I do not have enough diversity in sky locations, so I think I will stick with the current parameters for tonight's calibration. In any case, the pointing is much improved.

Apr 16

03:25 UT Started an 8-h run with two_src.scd, using the all-band pcal_hi-all.fsq, and not switching receivers. This spends 30 min on each source, alternating between 3C273 and 3C286. This is mainly to verify that the amplitude of both sources is constant over the night, since the pointing should be much better. If this works, I can use 3C286 to determine the spectrum of 3C273, and we can also proceed with our calibrator survey.

14:08 UT The data are really nice, and clearly show a big improvement in pointing. It is also a nice dataset to update baseline errors, to the mm level! I already have the Bz errors, and now I will work on Bx and By. I have not implemented the errors yet, because I want to do the entire update at once, so today's observations of the Sun, which will commence soon, will not have the update. I should mention that I changed the early- and late-day calibrations to use pcal_hi-all.fsq, since the SNR should be sufficient. I also changed the mid-day calibration to use pcal.fsq, since we really need more frequencies.

Apr 17

02:27 UT I have set up the schedule to repeat last night's observations, and restarted the schedule so that it reads the new baseline information. I will be interested to see if the baselines are now even better determined, so that the phases are flatter and better behaved that before.

15:23 UT I am disappointed that the Bz error seems even higher than before, but not in an obvious way that would imply an error in sign or something. The error is mainly an Ant 14 error of about 0.02 ns, so that this error appears on all baselines. This suggests that there is some systematic error that is dominating the measurement, but I am not sure what it would be. This is a baseline error of about 6 mm.

Apr 18

19:02 UT I tried yet another long calibration last night, but the fast correlator started losing packets after the first scan, and data were not recorded successfully. This morning I rebooted the correlator and set the delays again, then observed the Sun for a very short time, but again the correlator started losing packets. I gave up and reloaded the old correlator, set its delays, and am now back to normal observing. We will continue this way to catch any activity that might come from the current region on the Sun.

Apr 19

13:04 UT I noticed that yesterday's 3C84 observation showed a bad delay for Ant 4 -- I likely got the wrong sign in my earlier analysis. It was 12 ns off. I updated the delays to fix this, for today's solar observations. It was windy overnight, so there were no 3C273/3C286 calibrations.

Apr 20

13:04 UT Hmm, same time as yesterday, to the minute! Just noting that we are doing a calibrator survey (70-min/source), which has been going since 02:24 UT. So far the observations look good.

Apr 22-23

No observations (no ADC clock)

Apr 24

18:10 UT On Saturday there was a power outage, and it turns out the digitizer clock was not on a UPS, so it was not possible to restart the system. Today, I restarted as soon as possible, but there are very high winds so I cannot take calibrator data to enable delay measurements. Therefore, although we are on the Sun today, the data will be no good (some bad delays, plus no calibration). We will get the calibration and delays as soon as we can after the wind dies down.

Apr 25

11:52 UT The wind is still high, probably for all day long. There is no opportunity to set delays, so probably no good data today.

Apr 26

04:10 UT The wind was indeed high all day. This evening it has abated a bit, so I grabbed some calibration on 3C273, but the results are horrible. VPol is not only bad, but zero for many frequencies. I attempted to fix that by setting the ADC attenuation from the normal 15 to only 5, to somewhat compensate for the low RF power on Ant 14 VPol. However, all of the baselines except Ant 8 had really bad data, so I decided to up the EQ coefficients from 4 to 8, and reboot the ROACHes and LO1A. Any of those things could have been at fault, I suppose. I am now taking more data, to see what we have, but it will take awhile. I really should go to fewer bands.

04:27 UT Well! The good data are back! I am not sure if it was rebooting the ROACHes, or what, but all looks okay even with pcal_all-hi.fsq.

04:38 UT I updated the delays, and now will take more data with pcal.fsq, alternating between 3C273 and 3C286. Note that with my change of attenuation for Ant14 VPol, I do get reasonable phase coherence at lower frequencies, but it dies badly at higher frequencies. But at least I am not getting zeros.

11:55 UT The observations went quite well last night. I tweaked the delays on a couple of antennas as a result. Wind is low so far today, so the observations should be relatively normal on calibrators and the Sun.

Apr 27

12:48 UT The wind was high all night (and the last calibrator scan of yesterday did not succeed). I looked at some recent 2-m pointing (SOLPNT) results, and it is clear that there are systematic, time-dependent offsets that need to be improved. Apparently (not a surprise) my initial pointing calibrations using optical observations of stars is not permanent. However, now that Ant 14 is working, there is no reason we cannot do interferometric radio pointing on calibrators. To that end, this morning I created the necessary files and infrastructure for that (ProjectID CALPNT2M, and also .scd, .ctl, and .trj files with the same name). I tested it with testpntcal2m.scd, and it seems to work. I cannot analyze and test the actual data, however, since Ant 14 is not tracking due to WINDSCRAM. I will fully test it at the first opportunity.