GSFIT Data Format: Difference between revisions
| (8 intermediate revisions by the same user not shown) | |||
| Line 36: | Line 36: | ||
</pre> | </pre> | ||
'''NOTE:''' The name of the variable stored in a data file may be arbitrarily, since GSFIT would accept as a valid input data the first restored ''array of map structure'' that, in addition to a standard SSW IDL map structure, has at least the following required tags: ''' | '''NOTE:''' The name of the variable stored in a data file may be arbitrarily, since GSFIT would accept as a valid input data the first restored ''array of map structure'' that, in addition to a standard SSW IDL map structure, has at least the following required tags: '''FREQ, FREQUNIT, STOKES, RMS''' | ||
=GSFIT Parameter Maps Structure= | =GSFIT Parameter Maps Structure= | ||
| Line 108: | Line 108: | ||
GSFIT and GSFITCP applications save the list of spectral fit solutions in binary "*.log" files that are formatted as a header section followed by identical records logged in the order in which they are computed at runtime. | GSFIT and GSFITCP applications save the list of spectral fit solutions in binary "*.log" files that are formatted as a header section followed by identical records logged in the order in which they are computed at runtime. | ||
Although such log files may appear to be traditional IDL ''*.sav'' files, attempting to restore them via an IDL command line using the native '''''RESTORE''''' procedure would only recover the first record logged in the file. To recover the full information stored in such files, one may use one of the two support routines that are part of the GSFIT package | Although such log files may appear to be traditional IDL ''*.sav'' files, attempting to restore them via an IDL command line using the native '''''RESTORE''''' procedure would only recover the first record logged in the file. To recover the full information stored in such files, one may use one of the two support routines that are part of the GSFIT package, | ||
==Restoring a GSFIT log file as a pixel-based fit list== | ==Restoring a GSFIT log file as a pixel-based fit list== | ||
The '''''gsfit_log2list''''' function may be used to restore a fit list from a GSFIT ''*.log'' file in its raw format using the following calling sequence: | |||
<pre> | <pre> | ||
IDL> fitlist=gsfit_log2list('gsfitcp.log') | IDL> fitlist=gsfit_log2list('gsfitcp.log') | ||
| Line 127: | Line 127: | ||
FITLIST STRUCT = -> <Anonymous> Array[409600] | FITLIST STRUCT = -> <Anonymous> Array[409600] | ||
</pre> | </pre> | ||
GSFITCP and GSFIVIEW applications use internally the '''''gsfit_log2list.pro''''' to import a previously saved fit list, to add additional fit solutions to it in another session, or to convert the list to a parameter map structure for visualization and further analysis. However, one may use this function from the IDL console, or programmatically, to restore and concatenate two or more fit lists stored in separate GSFIT ''*.log' files. | |||
==Restoring a GSFIT log file as a parameter maps structure== | ==Restoring a GSFIT log file as a parameter maps structure== | ||
The '''''gsfit_log2map''''' function may be used to restore a fit list from a GSFIT *.log file and convert it to a parameter maps structure, using the following calling sequence: | |||
<pre> | <pre> | ||
IDL> maps=gsfit_log2map(file) | IDL> maps=gsfit_log2map(file) | ||
| Line 140: | Line 142: | ||
% RESTORE: Restored variable: XDR. | % RESTORE: Restored variable: XDR. | ||
% GSFIT_LOG2LIST: Restoring 409599 additional records | % GSFIT_LOG2LIST: Restoring 409599 additional records | ||
</pre> | </pre> | ||
However, the '''''gsfit_log2map''''' function, instead of a file path, may also accept as argument an already restore pixel-based fit list solution restored by the '''''gsfit_log2fit''''' function. | |||
Latest revision as of 23:37, 11 May 2019
This section describes the input and output data formats used by the GSFIT, GSFITCP, and GSFITVIEW applications, which are part of the GSFIT SSW IDL package developed for fitting the spatially-resolved solar flare radio spectra obtained with the EOVSA instrument. A tutorial about reduction of EOVSA data may be found here.
GSFIT Input Data Format
GSFIT expects input files to contain a multidimensional array of traditional SSW IDL map structures that must have a least two first mandatory dimensions, the number of frequencies and number of time frames, and an optional third dimension, the number of polarizations, if more than one are provided. The STOKES I demo EOVSA map cube shipped with the GSFIT package is described below.
IDL> help, EOMAPS EOMAPS STRUCT = -> <Anonymous> Array[30, 10] IDL> help,eomaps,/str ** Structure <214d9c00>, 26 tags, length=91472, data length=91449, refs=1: DATA FLOAT Array[151, 151] XC DOUBLE 249.22902 YC DOUBLE -224.62914 DX DOUBLE 2.0000000 DY DOUBLE 2.0000000 TIME STRING ' 4-Sep-2017 20:46:36.500' ID STRING 'EOVSA 3.42 GHz' DUR FLOAT 4.00000 XUNITS STRING 'arcsec' YUNITS STRING 'arcsec' ROLL_ANGLE DOUBLE 0.00000000 ROLL_CENTER DOUBLE Array[2] SOHO BYTE 0 L0 DOUBLE 0.00000000 B0 DOUBLE 7.2364274 RSUN DOUBLE 951.09152 RMS FLOAT 3.17954e+006 SNR FLOAT 97.6565 SCALEFAC FLOAT 1.18274 RMSXRAN FLOAT Array[2] RMSYRAN FLOAT Array[2] DATAUNITS STRING 'K' RMSUNITS STRING 'K' FREQ FLOAT 3.42000 FREQUNIT STRING 'GHz' STOKES STRING 'XX'
NOTE: The name of the variable stored in a data file may be arbitrarily, since GSFIT would accept as a valid input data the first restored array of map structure that, in addition to a standard SSW IDL map structure, has at least the following required tags: FREQ, FREQUNIT, STOKES, RMS
GSFIT Parameter Maps Structure
This is an IDL structure that stores a final product of the GSFIT package. This output data structure is described below.
IDL> help, maps ** Structure <1e0e3430>, 28 tags, length=189883336, data length=189666533, refs=1: FITMAPS STRUCT -> <Anonymous> Array[30, 100] ;[nfreq x ntimes] GSFIT solution maps DATAMAPS STRUCT -> <Anonymous> Array[30, 100] ;[nfreq x ntimes] input data maps use to compute the GSFIT solutions ERRMAPS STRUCT -> <Anonymous> Array[30, 100] ;[nfreq x ntimes] input error maps used to weight the input data N_NTH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated nonthermal electron densities ERRN_NTH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated nonthermal electron densities B STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated absolute magnetic field strength ERRB STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated THETA STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated LOS angle of the magnetic field vector ERRTHETA STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated LOS angle of the magnetic field vector N_TH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated thermal electron densities ERRN_TH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated thermal electron densities DELTA STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated distribution over energy power law slope ERRDELTA STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated distribution over energy power law slope E_MAX STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated maximum energy cutoff of nonthermal electrons ERRE_MAX STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated maximum energy cutoff of nonthermal electrons T_E STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated plasma temperature ERRT_E STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated plasma temperature CHISQR STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of fit CHISQR RESIDUAL STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of fit Residuals PEAKFLUX STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated peak flux of the fit solution ERRPEAKFLUX STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated peak flux of the fit solution PEAKFREQ STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of estimated peak frequency of the fit solution ERRPEAKFREQ STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated peak frequency of the fit solution WB STRUCT -> <Anonymous> Array[100] ;[ntimes] maps estimated magnetic field energy ERRWB STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated magnetic field energy WNTH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps estimated nonthermal electron energy ERRWNTH STRUCT -> <Anonymous> Array[100] ;[ntimes] maps of 1-sigma fit errors of the estimated nonthermal electron energy HEADER STRUCT -> <Anonymous> Array[1] ;structure containing information about the GSFIT solutions from which these maps were created
GSFIT FitList Format
This is the internal raw data format used by GSFIT and GSFITCP to store in memory or log on the disk the pixel-based fit solutions in the sequential order they ae produced at run-time. Each individual pixel-based fit solution structure described below contains the position and temporal information that, together to a header fit task structure, are used to build the fit parameter maps structure described above..
IDL> help,fitlist,/str ** Structure <21fed180>, 24 tags, length=504, data length=504, refs=2: X LONG 67 ;pixel X index Y LONG 42 ;pixel Y index T LONG 0 ;pixel time frame index FMIN INT 0 FMAX INT 23 DATA FLOAT Array[30] ERRDATA FLOAT Array[30] SPECFIT FLOAT Array[30] N_NTH DOUBLE 0.00000000 ERRN_NTH DOUBLE 0.00000000 B DOUBLE 0.00000000 ERRB DOUBLE 0.00000000 THETA DOUBLE 0.00000000 ERRTHETA DOUBLE 0.00000000 N_TH DOUBLE 0.00000000 ERRN_TH DOUBLE 0.00000000 DELTA DOUBLE 0.00000000 ERRDELTA DOUBLE 0.00000000 E_MAX DOUBLE 0.00000000 ERRE_MAX DOUBLE 0.00000000 T_E DOUBLE 0.00000000 ERRT_E DOUBLE 0.00000000 CHISQR DOUBLE 0.00000000 RESIDUAL DOUBLE 0.00000000
GSFIT Log File Format
GSFIT and GSFITCP applications save the list of spectral fit solutions in binary "*.log" files that are formatted as a header section followed by identical records logged in the order in which they are computed at runtime.
Although such log files may appear to be traditional IDL *.sav files, attempting to restore them via an IDL command line using the native RESTORE procedure would only recover the first record logged in the file. To recover the full information stored in such files, one may use one of the two support routines that are part of the GSFIT package,
Restoring a GSFIT log file as a pixel-based fit list
The gsfit_log2list function may be used to restore a fit list from a GSFIT *.log file in its raw format using the following calling sequence:
IDL> fitlist=gsfit_log2list('gsfitcp.log')
% Compiled module: MULTI_RESTORE.
% RESTORE: Portable (XDR) SAVE/RESTORE file.
% RESTORE: Save file written by gnita@baozi.hpcnet.campus.njit.edu, Fri Mar 1 18:15:09 2019.
% RESTORE: IDL version 8.5 (linux, x86_64).
% RESTORE: Restored variable: HEADER.
% RESTORE: Restored variable: TEMPLATE.
% RESTORE: Restored variable: HINT.
% RESTORE: Restored variable: KEY.
% RESTORE: Restored variable: XDR.
% GSFIT_LOG2LIST: Restoring 409599 additional records
IDL> help, fitlist
FITLIST STRUCT = -> <Anonymous> Array[409600]
GSFITCP and GSFIVIEW applications use internally the gsfit_log2list.pro to import a previously saved fit list, to add additional fit solutions to it in another session, or to convert the list to a parameter map structure for visualization and further analysis. However, one may use this function from the IDL console, or programmatically, to restore and concatenate two or more fit lists stored in separate GSFIT *.log' files.
Restoring a GSFIT log file as a parameter maps structure
The gsfit_log2map function may be used to restore a fit list from a GSFIT *.log file and convert it to a parameter maps structure, using the following calling sequence:
IDL> maps=gsfit_log2map(file) % RESTORE: Portable (XDR) SAVE/RESTORE file. % RESTORE: Save file written by gnita@baozi.hpcnet.campus.njit.edu, Fri Mar 1 18:15:09 2019. % RESTORE: IDL version 8.5 (linux, x86_64). % RESTORE: Restored variable: HEADER. % RESTORE: Restored variable: TEMPLATE. % RESTORE: Restored variable: HINT. % RESTORE: Restored variable: KEY. % RESTORE: Restored variable: XDR. % GSFIT_LOG2LIST: Restoring 409599 additional records
However, the gsfit_log2map function, instead of a file path, may also accept as argument an already restore pixel-based fit list solution restored by the gsfit_log2fit function.