Gentlemen and Ladies
Below a description of the EWS (Expert Work Station:
pronounced "Ooze") software for coherent estimation
of MIDI visibilities. The package consists of C-routines
that do most of the work and IDL front-ends for selecting
data and viewing results.
I. Summary of C-Language programs II. Summary of IDL routines calling C routines(probably what you want to know!) III. Summary of IDL Calibration Database Routines IV. Summary of other IDL routines V. More details about (I) VI. More details about (II) VII. More details about (III) VIII. More details about (IV) IX. Description of file selection GUI (Gorgonzola) X. Standard file names and contents used by scripts
1. oir1dCompressData "inFiles" maskFile outFile 2. oirFormFringes infile outfile [-smooth nSmooth] [-averageOrder order] 3. oirRotateInsOpd infile outfile [waveRefFile] 4. oirGroupDelay infile outfile [-smooth gSmooth] [-weight weightFile] [-first firstFile ] [-maxopd maxOpd] [-search] 5. oirFGroupDelay infile outfile [-smooth gSmooth] [-medsmooth medsmooth] [-maxopd maxOpd]
[-first firstFile] [-ngrad nGrad] [-search] [-sigout sigOut] [-exinterval exInterval] 6. oirPowerDelay infile outfile [-smooth gSmooth] [-ampsmooth ampSmooth][-delaysmooth delaySmooth ] [-medsmooth medSmooth] 7. oirRotateGroupDelay infile delayFile outfile [-phase phaseFile]
[-smooth smooth] [-ngrad nGrad] [-frames frames] 8. oirAutoFlag infile delayFile flagFile [-maxopd maxOpd] [-minopd minOpd] [-jitteropd jitteropd] [-SN SN] 9. oirAverageVis infile flagFile outFile 10. oirChopPhotoImages -in "inFiles" -out outFile [-nDrop nDrop] [-dSky dSky] [-order order] [-ref curveFile] [-skymask skymaskfile] [-sky0] 11. oirMakePhotoSpectra [-A Afile] [-B Bfile] [-AB ABfile] -mask maskFile] -out outFile [-shift maskShift] [-autoshift] 12. oirSci2HiPhot -AB ABfile -outbase outFileBase -ref refCoordFile -cross CrossCoeffFile 13. oirRedCal sourceTag 14. oirCalibrateVis targTag calTags [-calspec calSpecFile] [-calflux calFlux10] [-cald calDiameter] [-nophot] 15 oirRescaleSpectra templateFile scaleFile outFile 16. oirCrossCoeff baseName -cross outputFile -mask maskFile [-autoshift] [-sky] [-dSky dSkyValue ] 17 oirCurveCal -A Afile -B Bfile -out outFile [-fudge fudge] 18. oirMeanRMS infile outfile [-float]
1. midiPipe, tag, filelist [,mask=maskfile]
[,smooth=smooth][,gsmooth=gsmooth][,/noAve],
[,mindopd=minopd][,maxopd=maxopd],
[/twopass][,msmooth=msmooth][,ngrad=ngrad][,dSky=dSky]
2. midiVisPipe, tag, filelist [,mask=maskfile]
[,smooth=smooth][,gsmooth=gsmooth][,/noAve]
[,minopd=minopd][,maxopd=maxopd],[/twopass],
[,msmooth=msmooth][,ngrad=ngrad]
3. midiPhotoPipe, tag, filelist [,mask=maskfile] [,dSky=dSky] [,curve=curve]
4. midiSearch, tag, fileList [,mask=mask][,smooth=smooth][,gsmooth=gsmooth]
5. midiCalibrate, sourceTag, calTag, [,calDatabase=cdb] [,calFlux10=calFlux10], [,diam=diam] [,/noPhot] [,/print]
6. midiSPipe, tag, files, cross=cross [,photoFile=photoFile]
[,curve=curve] [,mask=maskfile] [,smooth=smooth] [,gsmooth=gsmooth] [,dSky=dSky] [,/dAve]
7. midiMakeMask, tag, filelist [,initialMask=initialMask] [,shiftOnly=shiftOnly]
[,smooth=smooth][,gsmooth=gsmooth][,/noAve]
[,factor=factor][,/noDelete]
8. midiCrossCoeff, tag, fileList [,curveFile = curveFile]
9. midiProcess2Vis, targetFiles, calFiles, targTag, calTag, [db=db],[mask=mask],[parmfile=parmfile]
10. midiFringeImage, tag, file, [/nodelete],[smooth=smooth],[noAve=noAve]
11. simData=midiDecorrelate( targetdata, caldata, targtag, caltag, ratios,[parmfile=parmfile],...)
1. cdb = midiCalDatabase(fitsfile) 2. cdb = vboekelbase() 3. nSource = cdb->sourceByName (name, sourceData, diamData, photData, specData) 4. nSource = cdb->sourceByAlias (alias, sourceData, diamData, photData, specData) 5. radii = cdb->sourcesNearPosition (ra, dec, sources, nmax=nmax, rmax=rmax) 6. sourceData = cdb->sourceFromFile(fitsFile, diamData=diamData, photData=photData, specData=specData, rad=rad, /silent) 7. sourceFlux = cdb->spectrumFromFile(fitsFile, diamData=diamData, specData=specData, rad=rad, /silent)
1. data=oirGetData, (file [,rows][,col=col]) 2. oirGetVis, (file,[wave=wave]) 3. midiDelayPlot, tag, title 4. corr = midiGetCorr (tag[,wave=wave]) 5. phase = midiGetPhase (tag[,wave=wave]) 6. phot = midiGetPhot (tag) 7. opd = midiGetOpd(tag) 8. delay = midiGetDelay(tag) 9. choppedData = midiChopImage(inputFiles) 10. keyValue = midiGetKeyword(keyword, inputFiles) 11. complexVisData = midiCVis(oirVisibilityStruct) 12. p = midiPhase (cArray) 13. c = pseudoComplex (pcArray) 14. c = midiGetComplex (tag,qualifier)Table-of-Contents C-routines IDL-Routines
Purpose:
Compress MIDI raw (PRISM/GRISM) files by multiplying each MIDI
window by a floating point mask (which has zeros in rows
not containing useful data) and summing in the y-direction
(perpendicular to the PRISM/GRISM dispersion). The result
is a 1-dimensional spectrum for each window region.
Parameters:
"inFiles" a list of raw MIDI (PRISM/GRISM) input files representing
one exposure. There may be more than one file because they
were broken up into 100MB files by the on-line system.
If there is more than one file they should be separated
by spaces and included between double quotes: "file1 file2 file3"
maskFile: a MIDI style FITS binary table whose imaging_data section
contains the masks described above. If this is not
specified it defaults to an environmental variable specified
externally (typically in the vltisetup script):
for PRISM/HIGH_SENS $prismhmask
for PRISM/SCI_PHOT $prismsmask
for GRISM/HIGH_SENS $grismhmask
for GRISM/SCI_PHOT $grismsmask
outFile: the name of the output file. This will be a MIDI style
FITS binary table in exactly the same format as "inFiles" and
maskFile except that the y-dimension of each DATA array is 1.
Purpose:
Take the output of oir1dCompressData and form spectrally dispersed
fringes by subtraction of the two interferometic channels and
supression of background. The latter is accomplished by
the subtraction itself, by a high-pass filter (in
the time domain) applied to the individual spectral channels,
and finally, optionally, by removal of the a polynomial fit to the flux
(over wavelength) from the individual spectral channels.
This removes instrumental and sky backgrounds that vary
more slowly than the fringe. The fringe should vary quickly
because of the OPD modulation imposed by the MIDI piezos.
Parameters:
inFile: name of the output from oir1dCompressData
outFile: output file name. The format is a MIDI style FITS table
imaging_data file with only one DATA region (
the combined I1-I2 channels).
nSmooth: (optional) width of boxcar used in highpass filtering.
Default = 50
Data is smoothed in time with boxcar, and this smoothed version
is subtracted from input data, yielding a highpass filter.
order: If specified, a polynomial fit of the specified order is
made to the pixels in each frame between 7 and 13.8 microns,
and this fit is subtracted from this individual pixels.
The sky background often generates a slowly varying offset of the
spectrum, which is not entirely removed by the preceding filtering
actions. This offset generates an artefact at zero OPD that can confuse
oirGroupDelay or oirFGroupDelay if the source flux is very weak.
Specifying -averageOrder >=0 removes this artefact, but it also
removes your source if it crosses zero during a OPD scan.
Default: no removal of average
Comments:
values of order >3 are not accepted and values <0 are the same as 0
values >0 (e.g. 1 or 2) are possible but probably not advisable
specify -averageOrder 0 if you observed with offset OPD tracking
(then there are no OPD zero crossings) and if your source is very weak
and the delay solutions returned by oirGroupDelay show a zigzag
pattern typical of zero OPD artefacts.
if you use this option be sure to use it on your calibrators too.
Purpose:
Remove the known Instrumental OPD components from the fringe spectra
that oirFormFringes produced. Each data point at frequency
k (in spatial units: 2 pi/lambda) is multiplied by exp(-i*k* OPD)
where OPD is the sum of the OPD from the MIDI piezos and the
VLTI delay lines.
Special Notes:
1. The result is a FITS imaging_data file in
"pseudo-complex" format. I.e. it is in REAL format
but each row is twice as long as the original data:
each value is a (real, imaginary) pair.
2. The VLTI Delay values may contain large fictitious
offsets of up to 1cm if the system was not correctly
zeroed at setup. To avoid applying these large
fictitious offsets, the program determines the
MODE of the tracking positions during the run and
subtracts this before applying the OPD shift. This
modal value is printed by the program and also
stored in the output header as "OPD0" in meters.
Parameters:
inFile: name of the output from oirFormFringes
outFile: output file name. The format is a MIDI style FITS table
imaging_data file in pseudo-complex format.
Purpose:
Estimate the group delay for each measured spectrum. The
group delay is the peak of the Fourier Transform of the
spectrum (thus in the delay domain, not the frequency
domain). Each spectrum is fourier transformed. Note
that because the MIDI beam-combiner has only two output
phases it is essentially a cosine correlator and not
a complex correlator. This means that the group-delay
has TWO peaks: at positive and negative delays.
Because the actual OPD was modulated by the MIDI piezos
and this modulation was deRotated in oirRotateInsOpd,
one of the two peaks is nearly stationary from frame
to frame (only atmospheric OPD movement remains) while
the other peak moves around with TWICE the instrumental
modulation. Thus if we average a few frames together
the wrong side band is strongly suppressed. In this
program a gaussian smoothing of sigma = gSmooth frames
(default 4) used in this averaging.
Parameters:
infile: output of oirRotateInsOpd
outFile: output file containing TWO interesting tables:
1. A pseudo-complex imaging_data table containing
the raw FFTs of the input data
2. A DELAY table containing the value of the peak
delay (after smoothing) of the above table at
each frame. This DELAY table has three columns:
TIME (same as raw data), TELESCOPE (always 1,2)
and DELAY (peak of group delay, with OPD0 added),
in seconds (IAU standard). Thus you have to
multiply by the speed of light to get meters.
gSmooth (optional; Default: 4 frames)
If specified can change the smoothing in the frame domain
before looking for a delay peak. Default is gaussian sigma = 4
frames. Choose larger value, like 10-15 for weak sources in good weather,
shorter for strong sources in bad weather.
maxOpd (optional: Default: 200 microns) Do not look for
delay peaks more than maxOpd away from either
a. OPD of delayline tracking system (default)
b. OPD calculated by a first pass through data
and specified by firstFile option
firstFile (optional; Default: not used)
If specified it contains a the filename of the output from a
previous run of oirGroupDelay or oirPowerDelay
The delay table from this file is used as a first guess before searching for the
group delay. If maxopd is fairly small, this limits the search
range of oirGroupDelay and it is thus less likely to misidentify
noise peaks as delay peaks
weightFile (optional; Default: uniform spectral weighting)
If specified this is the name of a MIDI type file containing an
IMAGING_DATA table with at least one record containing
a 1 x nwavelength array. This will be used as relative weights
for the input spectra before looking for the group delay. This
can improve the S/N for sources with a non-flat spectrum.
-search (optional; Default: tracking mode)
If specified oirGroupDelay assumes that this is SEARCH mode
data covering a large OPD range, rather than TRACK mode data.
In SEARCH mode, the program slides its relative OPD zero point
along with each scan to avoid running out of range during the FFT.
Data of this type can be analyzed with the IDL program midiSearch.
Purpose:
A more modern version of oirGroupDelay
with special features to produce more sensitive and reliable estimates
of the delay for faint sources. The basic procedure is the same
as in oirGroupDelay, but there are many new features. Among these:
a. Smoothing intervals etc. are specified as times in seconds rather than
frame counts.
b. The delay table is not calulated at every frame by only at a fraction of
the smoothing interval specified by sigout
c. If requested the program will fit, in each averaging period specified by smooth,
an OPD that changes linearly with time, rather than being constant. This
allows the integration interval used to search for the delay peak to be
lengthened. The range of OPD rates to be searched is specified by
the parameter ngrad
d. The final delay track can be median smoothed over a time interval
medSmooth before being written to output
e. The search for the delay peak can be restricted to maxopd microns from
either the telescope delay tracking positions or a first guess estimate
of the delay positions typically provided by oirPowerDelay.
This reduces the chance of finding a noise peak.
f. When finding the value of the delay in a time interval around a specific
time, you can exclude a small amount of data at the center of the interval.
This has the effect of removing a bias when the delay value is used
to rotate the phases of the data at the center. The bias is introduced
because the noise in the data influences the correction to the phase of the
same data, tending to cause all phases to approach zero. The size
of the exclusion interval is set by exInterval
Parameters:
infile: output of oirRotateInsOpd
outFile: output file containing TWO interesting tables:
1. A pseudo-complex imaging_data table containing
the raw FFTs of the input data
2. A DELAY table containing the value of the peak
delay (after smoothing) of the above table at
each output sampling time. This DELAY table has three columns:
TIME (same as raw data), TELESCOPE (always 1,2)
and DELAY (peak of group delay, with OPD0 added),
in seconds (IAU standard). Thus you have to
multiply by the speed of light to get meters.
The delay table contains output sampled at
a fraction (sigOut) of the specified gSmooth
interval.
gSmooth (optional; Default: 0.2 sec)
If specified can change the smoothing in the frame domain
before looking for a delay peak. Default is gaussian sigma = 0.2 sec
frames. Choose larger value, like 0.6-0.8 for weak sources in good weather,
shorter for strong sources in bad weather.
maxOpd (optional: Default: 200 microns) Do not look for
delay peaks more than maxOpd away from either
a. OPD of delayline tracking system (default)
b. OPD calculated by a first pass through data
and specified by firstFile option
medSmooth (optional: Default: no median smoothing) Before writing
the output delay table, smooth the values with a median filter
of width medSmooth (seconds). For low signal sources a
fairly large value e.g. 1 sec, is advisable.
nGrad (optional, default = 1) Check for linear OPD variations
within smooth time interval specified by gSmooth. Program
looks for maximum coherence over phase gradients in a range
of +/-nGrad * 1 radian/gSmooth interval.
exInterval (optional: 1.2) Exclude data within this interval
when estimating delays, to avoid noise bias. The excluded
interval = exInterval * 0.25 * gSmooth
firstFile (optional; Default: not used)
If specified it contains a the filename of the output from a
previous run of oirGroupDelay or oirPowerDelay. The delay table
from this file is used as a first guess before searching for the
group delay. If maxopd is fairly small, this limits the search
range of oirGroupDelay and it is thus less likely to misidentify
noise peaks as delay peaks
-search (optional; Default: tracking mode)
If specified oirGroupDelay assumes that this is SEARCH mode
data covering a large OPD range, rather than TRACK mode data.
In SEARCH mode, the program slides its relative OPD zero point
along with each scan to avoid running out of range during the FFT.
Data of this type can be analyzed with the IDL program faintSearch.
Purpose:
Make a crude estimate of the group delay for very low S/N data
as a function of time by first smoothing the delay data coherently
as specified by gSmooth and then smoothing the the delay data
incoherently over a much longer period of timeampSmooth, and possibly
smoothing in the delay direction by delaySmooth pixels.
Then search for the peak in the delay direction and write the
delay value at the peak into the output delay table.
This incoherent track can be used for a second pass of oirFGroupDelay.
Parameters:
infile: output of oirFGroupDelay. or oirFGroupDelay.
outFile: output file containing TWO interesting tables:
1. A real imaging_data table containing
the FFTs of the input data, first smooth coherently
by gSmooth seconds, then converted to
ABS(f)^2, then smoothed by ampSmooth seconds.
2. A DELAY table containing the value of the peak
delay (after smoothing) of the above table at
each averaging interval. This DELAY table has three columns:
TIME (same as raw data), TELESCOPE (always 1,2)
and DELAY (peak of group delay, with OPD0 added),
in seconds (IAU standard). Thus you have to
multiply by the speed of light to get meters.
gSmooth (optional; Default: 0.4 s)
Coherent averaging interval.
ampSmooth: (optional; Default 16 s) after coherent averaging
by gSmooth, take the absolute square of the
delay function (FT of the spectra) and average in
the time direction with a gaussian with a standard
deviation of ampSmooth seconds.
delaySmooth: (optional; Default 10 pixels) Smooth the
delay function by a guassian with this sigma (pixels)
in the delay direction. This lowers the resolution
in delay, but improves the S/N.
medSmooth: (optional: Default 15 sec) Median smooth the positions
of the delay peaks over this interval before writing
the output delay table.
maxOpd (optional: Default: 400 microns) Do not look for
delay peaks more than maxOpd away from the OPD of delayline
tracking system (default)
Purpose:
Remove the group delay measured by oirGroupDelay as
well as the instrumental OPD from fringe data
(the output of oirFormFringes). Same algorithm
as specified in oirRotateInsOpd.
Additionally estimate an offset PHASE for each
frame and remove it. This phase is primarily
the result of water vapor dispersion (i.e.
non-constant index of refraction). Several
spectra are averaged together to remove the
beats from the off-side-band and improve the signal/noise,
then the phase offset is determined and removed.
Parameters:
infile: output of oirFormFringes
delayFile: output of oirGroupDelay
outfile: output, in format identical with
output of oirRotateInsOpd
smooth: 1-sigma size of gaussian smoothing function in estimating water vapor phases
default: 0.2 sec
frames: if set then specify smooth in frames instead of seconds
ngrad: as in oirFGroupDelay., try to find best
linear gradient of water vapor phase within smooth. Default: no gradients
phasefile: instead of finding w.v. phases myself, use the output of a
previous run of this program in specified file.
nophase, nogroup, noopd: suppress respectively: water vapor estimation, removal
of group delay estimates, removal of instrumental opd, for diagnostic purposes.
Purpose:
Try to choose (automatically) which frames to include in the output
visibility estimation: This is currently NOT done on the basis of
estimated amplitude since this biases the result. The current
criteria are:
1. OPD tracking distance from 0-point. Experience shows significant
attenuation of fringe (a few procent) for PRISM if this is larger
than about 150 microns. So if ABS(instrumental OPD-group OPD) >
maxOpd the frame is flagged.
When the OPD tracking point crosses zero MIDI can get confused with
sky background, especially if oirFormFringes is run
with the -removeAverage or -averageOrder options. So if
ABS(instrumental OPD-group OPD) < minOpd the frame is flagged.
2. Jumps in OPD. There are some instrumental OPD jumps (due to
telescope refocusing) where the fringe is presumably attenuated.
If the 2nd difference in the group OPD is > jumpOpd the frame is
flagged.
3. Low signal, poor tracking: If the OPD jitter is larger than the
specified jitterOpd, this is considered poor tracking and the frames
are flagged. The OPD estimates from oirFGroupDelay
are read. We take the 2nd difference to remove trends, then take the absoute value
and then median smooth over 50 frames before comparing to jitterOpd.
default flagging limit: 1.5 microns.
Output is written into a FLAG Fits table that specifies the flagged
time intervals.
Parameters:
inFile: output of oirFormFringes (only used to read instrumental
OPD)
delayFile: output of oirGroupDelay
flagFile: name of file to contain FLAG tables
maxOpd: max acceptable difference between tracking and true OPD (microns)
minOpd: min acceptable difference between tracking and true OPD (microns)
jumpOpd: OPD jumps larger than this (microns) are flagged
Purpose:
Average all unflagged but phase rotated frames together to arrive
at a single (complex) visibility.
Output is in IAU/ESO OI_VIS format FITS tables which is different
from imaging_data. These can be accessed with my IDL oirGetVis
routine: data_array = oirGetVis(file, wave=wave) where
wave contains the channel wavelengths in microns. data_array
is an array of structures containing (among other things)
visamp and visphi (amplitude and phase).
Parameters:
infile: output of oirRotateGroupDelay
flagFile: output of oirAutoFlag (or manual flagging)
outFile: name of OI_VIS file
Purpose:
The first step of the operations for reducing chopping photometry.
Sky and Target frames are averaged, target-sky computed, and an on-slit estimate of residual
sky is made and removed. The result is output as a set of detector images for one, possibly
multiple, input file. For HIGH_SENS photometry this must be run sequentially for the AOPEN
and BOPEN photometry input. For SCI_PHOT photometry it must be run once on the same
file used for interferometry. The output contains one set of images for the average of all
good input frames, and then a series of such sets made with (5) independent subsets of the
input frames so that the rms variations in calculated parameters can be estimated.
Parameters:
"inFiles" a list of raw MIDI (PRISM/GRISM) input files representing
one exposure. There may be more than one file because they
were broken up into 100MB files by the on-line system.
If there is more than one file they should be separated
by spaces and included between double quotes: "file1 file2 file3"
outFile output file name
nDrop exclude this many frames after a transition from Sky to Target
if chop mirror has not settled (default = 2)
dSky Normally after Target-Sky subtraction, an additional sky correction is
estimated at the top and bottom of the slit. This is necessary for
faint sources (under 1Jy). The program normally estimates the sky from
rows 7-11 and 23-27 (PRISM) or 4-8 and 27-31 (GRISM). If specified,
dSky indicates that the user wishes to move these rows dSky pixels apart
(dSky positive) or together (dSky negative). If dSky is large
enough so that the sky region moves out of the measured window, no
on-slit sky subtraction is performed (this is indicated in the
text output of the program).
maskfile sky maskfile containing image in standard MIDI format designating area to
fit sky background along spectrum. If specified dSky and refFile are ignored
refFile file containing accurate description of curvature of spectra on detector in order
to more accurately position sky subtraction zones. If not present spectra
are assumed to be straight.
polyOrder order of polynomial fit to background along slit. Default=1 for UTs 0 for ATs
sky0 if set in SCIPHOT mode, make a special attempt to debias sky images
Purpose:
Apply a mask to the output(s) of oirChopPhotoImages
and compute all kinds of geometric and arithmetic averages of the AOPEN and BOPEN channels.
They these are summed in the y-direction both with and without Masking
(as described in oir1dCompressData) to produce spectra.
The mask should be IDENTICAL with the one specified there.
Input are two separate (sets of) files (AFile and Bfile) containing raw chopping
data for AOPEN and BOPEN shutter positions, or one file in SCIPHOT mode (ABFile).
Output is a imaging_data FITS table file with 12 rows, each row
containing a single DATA1 array:
1. Total Flux (ADU/s/channel) for AOPEN shutter position
2. Total Flux (ADU/s/channel) for BOPEN shutter position
3. Total Flux GeometricMean of A&B (i.e. of rows 1 and 2).
4. Masked Flux (ADU/s/channel) for AOPEN shutter position
5. Masked Flux (ADU/s/channel) for BOPEN shutter position
6. Masked Flux GeometricMean of A&B (i.e. of rows 4 and 5).
7-12 are the (poorly) estimated rms of the above quantities.
Parameters:
"AFiles" AOPEN HISENS chopped photometry files from oirChopPhotoImages
"BFiles" BOPEN HISENS chopped photometry files fromoirChopPhotoImages
"ABFiles" ABOPEN SCIPHOT chopped photometry files from oirChopPhotoImages
outFile FITS imaging_data file containing output
maskFile mask file. Should be same as the one for interferometric data
-shift move mask up/down shift pixels before masking
-autoshift figure out shift myself by crosscorrelating mask with images
If specified, overrides shift.
-outMaskFile If specified, put result mask of -shift or -autoshift in this file
for later usage (e.g. in interferometric processing)
Purpose:
Convert chopped interferometric data in SCI_PHOT format into something that looks like what
you would have gotten if you had run a standard photometric sequence (AOPEN/BOPEN) in SCI_PHOT
mode and which can be digested by oirMakePhotoSpectra.
If you have chopped raw tracking data in SCI_PHOT mode, first run it through
oirChopPhotoImages to subtract sky from target
frames and average everything together. The output is then the input (ABfile)
to the current program. The photometric channels (channels 1 and 4) are bent according
to the information in refCoordFile and rescaled according to the coefficients in
crossCoeffFile to what you would have measured in channels 2 and 3 if you had observed with
only one shutter open. There are then two output files: outFileBase.Aphotometry.fits and
outFileBase.Bphotometry.fits that simulate the result you would have gotten with shutters
AOPEN and BOPEN. These can now be fed into oirMakePhotoSpectra
to produce photometric spectra.
Parameters:
ABfile: output of oirChopPhotoImages for an ABOPEN file containing
chopped data.
outFileBase: prefix for the two output files. These will be named outFileBase.Aphotometry.fits and
outFileBase.Bphotometry.fits
refCoordFile: a "data" file containing an IMAGING_DETECTOR table that describes the curvature of the
spectra on the detector. This information will be used to recurve the photometric channels to
look like the interferometric channels (so that the same masks can be applied). This will
typically be created by a run of oirCurveCal.
crossCoeffFile: a "data" file containing the cross coupling coefficients between the interferometric
photometric channels. This will typically be created by a run of oirCrossCoeff.
Purpose:
This program is somewhat different in purpose than the previous. It
takes the output files from several of the previous steps, as
applied to a calibrator and computes the instrumental visibility
and spectrophotometric calibration.
In this and the programs described below I use "tags" to identify
all the files belonging to one observation set. The full file
names are tag.type.fits where type describes the file type
(see complete listing below). Thus oirRedCal expects all
input files to have this type of name.
The output of oirRedCal is an OI_VIS table with 3 rows:
1. Instrumental visibility (amplitude and phase)
2. Spectrophotometric calibration (ADU/s/Jy)
3. "Strehl Ratio": not really, but the ratio of masked
counts/total counts per spectral channel.
Parameters:
sourceTag: the initial part of input file names, may also include a
directory. e.g. hd10380 or /strw11/jaffe/ngc4151
The type flags actually used are photometry (output of
oirChopPhotometry), and correlated flux (output of oirAverageVis),
thus the real file names would be hd10380.photometry.fits
and hd10380.corr.fits.
output: is not specified. The output is written into a file named
sourceTag.redcal.fits and contains the abovemention OI_VIS table.
Purpose:
Combine reductions of a target and calibrator observations to produce
calibrated visibilities and plots. The output is three FITS table files.
1. An OI_VIS table file: targetTag.calvis.fits containing one
row per input row in the file targetTag.corr.fits
This contains calibrated visamp and visphi.
2. An OI_VIS table file: targetTag.calcorr.fits containing one
row per input row in the file targetTag.corr.fits
This table contains calibrated correlated fluxes and phases
3. An imaging_data file: targetTag.calphot.fits containing two
rows:
1. calibrated photometry (Jy) total
2. calibrated photometry (Jy) under mask
Parameters:
targetTag: descriptor for science target. The types actually
used are .photometry and .corr
calTag: descriptor for calibrator. The type actually used
is .redcal
output is as described above: targetTag.calvis and .calphot
-calSpecFile: a FITS file containing a MIDI IMAGING_DATA table
with one row containing the calibrator flux (Jy) at the
same wavelengths as the target being calibrated. If specified
this data overrides specification of calflux.
-calflux calFlux10: specify the flux of the calibrator used
for flux calibration in Jansky at 10 microns. A Rayleigh-
Jeans spectrum is assumed. If not specified 1 Jy is used.
-cald calDiameter: visibility calibrator diameter in milliarcsec.
If not specified 0.0 mas is used.
-nophot: Assume that source photometry is not very accurate.
The effect is that correlated fluxes are computed as:
Jy(calibrator) * raw_correlated_flux(target)/raw_correlated_flux(calibrator)
instead of Jy(calibrator) * visibility(target) raw_total_flux(target)/raw_total_flux(calibrator)
Purpose:
Take a reduced photometric data file (e.g. template.photometry.fits) produced
by dispPhot or midiPhotoPipe from AOPEN/BOPEN
chopped photometry data, and rescale it using photometry data on the same target taken in
SCI_PHOT mode (produced by dispSciPhot or oirSci2HiPhot)
so that output = (A + BX) * template, where A and B are
chosen to give the best least squares fit of output to scale.
Parameters:
templateFile: a nice reduced standard output of chopped photometry data taken
in standard AOPEN/BOPEN mode with the SCI_PHOT beam combiner.
scaleFile: a nice reduced standard output of photometry data taken from
the outer PA+PB channels of interferometry data taken in SCI_PHOT mode.
output: name of output file
Purpose: calculate cross-coupling coefficients (kappa coefficients)
for SCI_PHOT mode data
Parameters:
Afile output of oirChopPhotoImages representing data
taken in chopping photometry mode with the shutter in AOPEN position
Bfile output of oirChopPhotoImages representing data
taken in chopping photometry mode with the shutter in BOPEN position
coordFile: FITS file containing "imaging_detector" table specifying
distortions and wavelength information for this SCI_PHOT detector
mode.
outFile: output file name containing the kappa coefficients in the form of fits
IMAGING_DATA tables.
Purpose:
Read reduced chop imaging files (from oirChopPhotoImages).
Fit gaussians to the spectra at each x-position, then fit power laws to the centers
of these gaussians to represent the curvature of the spectra with x. When you
are done, write the coefficients of these power laws into the dmc/dmp parameters
in the IMAGING_DETECTOR tables in the output file. While you are at it, since
you've done most of the work already, write out an IMAGING_DATA table with
gaussians as specified by the power laws because these make a plausible mask for
extracting photometry and correlated flux from normal measurements.
Parameters:
AFile: output of oirChopPhotoImages for data taken on
a bright star with the shutter in AOPEN position.
BFile: output of oirChopPhotoImages for data taken on
a bright star with the shutter in BOPEN position.
output: name of FITS file to contain output tables
fudge: optional; The half widths of the exponential used to create the masks are increased
by this factor. If omitted, 1.0 is used.
Purpose: Computes the temporal mean and RMS of the DATA sections
of a MIDI data file.
Inputs:
input: input data file, compressed or otherwise
output: output data file. IMAGING_DATA table contains only 2 rows, the mean and rms.
float: must be specified if the input data is notraw, i.e. not integer format
Purpose: run midiVisPipe, midiPhotoPipe, and oirRedCal on 3 files specifying
interferometric and photometric raw data. Effectively a
complete but uncalibrated reduction of this set of data.
Plots tracking versus estimated group delay and estimated
instrumental visibility. After running midiPipe on a science
target and a calibrator you should run midiCalibrate,tag, caltag
to calibrate the science data.
This procedure produces many files, of which the most important
for routine reduction are:
tag.corr.fits
containing the estimated correlated flux (instrumental units)
[AB]tag.photometry.fits
containing the estimated photometric fluxes (instrumental units)
tag.redcal.fits
containing the estimated instrumental visibility
For experts:
tag.groupdelay.fits
contains group delay tracking information.
Parameters:
tag: output tag to identify files to dispVisPhot
files: a string array of 3 input file names, each of
which may be a blank-separated list of files. The files must
specify one interferometric observation and two (A- and B-
shutter) photometry observations.
You can choose the files with the GORGONZOLA GUI.
mask (optional): mask file name. If not specified a default
depending on the instrumental setup. For old prism high_sens
files (before Dec 2003) use:
mask = getenv('drsRoot')+'/maskfiles/prismmask_before_2003dec01.fits'
EWS blackbelts can make their own masks with midiMakeMask
smooth:Delay high-pass smoothing width in seconds to suppress background
fluctuations. Default is 1 sec. This is too long for weak sources; you might try 0.2s
gsmooth: group delay estimation averageing half-width (seconds). Default
is 0.1 but use larger value (~0.6 sec) for weak sources during stable atmosphere,
smaller for strong sources during rapid OPD changes. For weak
sources during rapid OPD changes, give up.
msmooth: median smoothing of group delay estimates (seconds). Default =1.0
Use a large value (1-2 s) for weak sources under stable atmosphere.
twopass: set on for very weak sources. oirFGroupDelay makes two passes to determine group delay.
First a crude estimate with large value of maxopd. These estimates are then smoothed heavily
and a second pass of oirFGroupDelay is run with the maxopd constraint
set that only group delay values near the smoothed first pass are allowed. This
suppresses finding of noise peaks at random delays when the signal is weak.
maxopd: maximum allowed OPD (microns) with respect to physical delay line OPD (1st pass). Default =100
Ave: order polynomial used to fit and remove spectral sky background in oirFormFringes. For weak sources this improves background removal and improves trackering. Default = 0 (constant) for non-zero OPD offset tracking, -1 (no background removal) for zero OPD tracking.
dSky: move on-slit regions for sky estimation (c.f. oirMakePhotoSpectra ).
A large positive number means no on-slit sky estimation.
Run oir1dCompressData
oirFormFringes
oirRotateInsOpd
oirFGroupDelay
oirRotateGroupDelay
oirAutoFlag and
oirAutoFlag
which reduces the interferometric scans specified by fileList and produces
Tag.corr.fits, and
Tag.groupdelay.fits files.
This is same as midiPipe but the photometric reduction is skipped. See midiPipe for details.
Run oirChopPhotoImages and oirMakePhotoSpectra
which reduce the photometric scans specified by fileList and produce Tag.[AB]photometry.fits files.
Inputs: Tag, fileList, and mask are described in midiPipe
skymask: a blackbelt option. Instead of dSky or the default positions to measure the
on-slit sky background, use the specified mask file in oirChopPhotoImages
This can be created by midiMakeMask from calibrator data
Similar to midiVisPipe but the fileList should point to data
taken in fringe Search mode instead of fringe Tracking mode.
The output contains the usual fits files and also a plot of fringe amplitude versus OPD
during the search phase.
run oirCalibrateVis with specified parameters.
If the calDatabase parameter is specified as an existing database
(c.f. Calibration Database ) then the database is searched
for a calibrator within 1' of the position in your calibration files, and if found,
the calibrator spectral and diameter data are taken from there.
In addition a whole bunch of plots is produced in a file named sourceTag.ps. It also produces the
FITS files: sourceTag.calphot.fits containing calibrated photometry,
sourceTag.calvis.fits" containing the calibrated visibility.
If /print is specified, a dozen or so sourceTag.xxxx.dat files are produced containing the information
that was plotted as ASCII tables.
sourceTag.calcorr.fits" containing the calibrated correlated flux
(analogous to uncalibrated sourceTag.corr.fits"
Purpose: run dispVis and dispSciPhot or dispSciPhotPhot
to reduce a (chopped/interferometric/SCI_PHOT) input file, and optionally some chopped/photometry/SCI_PHOT data.
Parameters: as in midiPipe with the addition of:
file(s): an array of either 1 or 3 character strings specifying (blank separated) sets input
files. If there is only one string, this contains data taken in interferometric mode
with shutter in ABOPEN position. The interferometric signal is extracted with
dispVis. If photoFile is NOT specified, the photometric
signal is extracted from the same input files with dispSciPhot.
If photoFile IS specified, dispSciPhotPhot is
run. This extracts the photometric signal in the same way, but uses it to rescale
previously reduced real photometric observations using oirRescaleSpectra.
If 3 input arrays are given, the reduction proceeds with rescaling as just described, but
the raw photometric observations in AOPEN/BOPEN mode are specified in the
2nd and 3rd strings.
coord: file containing distortion/wavelength information
for individual MIDI channels, not yet included automatically
with data from Paranal. Defaults to either of :
$drsRoot/software/minrts/config/mioSetup_FIELD_PRISM_SCI_PHOT_SLIT.tmp
$drsRoot/software/minrts/config/mioSetup_SPECTRAL_GRISM_SCI_PHOT_SLIT.tmp
cross: file containing kappa/cross-coupling coefficients. Should have been computed
from a previous run of midiCrossCoeff. It is the User's
responsibility to do this.
photoFile: contains the reduced output of standard AOPEN/BOPEN photometric observations of
the same target. Optional. If provided, and file[s] only contains one, interferometric
data set, it will be used as a template for rescaling the photometric data, as described
in oirRescaleSpectra.
Purpose: Create a new mask file by constructing an image of the correlated flux as a function of
wavelength and slit position on the detector and fitting a set of smoothed gaussians to this image.
This mask is then (near) optimized for S/N in looking for fringes. In this form the procedure is
usually applied to a strong calibrator. The process is to track the calibrator fringe using an initial
mask, e.g. the default mask, with the procedure faintVisPipe.
The OPD information from this run is then used to construct a coherent image of
the entire detector plane. This image is smoothed somewhat and then fit with a set
of gaussians whose position and width are allowed to vary slowly with wavelength. This set,
normalized to a peak of unity at each wavelength, forms the new mask. For low S/N sources,
i.e. your targets, it is probably a bad idea to perform this procedure. Alternatively you
can ask midiMakeMask to consider only shifting an initial mask (the one generated from a calibrator)
to get the best overlap with the fringe image.
Outputs: two fits image data files:
tag.srcmask.fits: a mask to use in midi(Vis)Pipe to specify region-of-interest
tag.skymask.fits: a mask to use in midi(Photo)Pipe to specify sky regions
Inputs:
tag: The usual string to keep track of the output files
fileList: As in faintVisPipe, the file containing raw MIDI interferometry data
initialMask: (optional; default: system reference masks) a mask to use while looking for fringes during
the first pass through the data. If you are using the shiftOnly option, this should be
the output of a previous run of midiMakeMask on a strong source.
shiftOnly: (optional: default: full-fit) If specified, consider only shifting the y-position of
the initial mask, but do not change its shape.
smooth,gsmooth,noAve: Parameters for faintVisPipe.
factor: (optional: default=1.0) For a full fit to the coherent image, increase the width of the determined
best fit gaussians by this factor
ave: spectral background removal polynomial parameter. See midiPipe
noDelete: (optional: default=delete) Normally all the (large) scratch files used to calculate
the new mask are deleted by the procedure. If you specify noDelete they are
left intact for possible diagnostics.
Calculate the Cross Coupling (Kappa) coefficients between the interferometric and photometric channels for data taken with the
SCI_PHOT beam combiner. The input raw data should consist of two file (sets) containing AOPEN and BOPEN
data with this combiner for a bright star. This routine runs dispCrossCoeff after digesting the input arguments
and filling in defaults. This dechops the input images, straightens out the curved photometry channel images,
sums the photometry in the y-direction, and divides the interferometry images by the photometry images to
yield the kappa coefficients in the output file.
Inputs:
tag: output tag to identify files to dispCrossCoeff.
fileList: an array of two strings, containing the AOPEN and BOPEN files. As usual
if these are multiple files, these should be concatenated and separated by blanks.
curve: a file containing descriptions of the curvature of the spectra. You can generate this yourself
using oirCurveCal. If not specified this midiCrossCoeff
uses the environmental variable prismscurve or grismscurve which are usually
to be found in $drsRoot/software/minrts/minrts/config with file names like:
minrtsCurve_FIELD_PRISM_SCI_PHOT.fits or minrtsCurve_SPECTRAL_GRISM_SCI_PHOT.fits
Completely process target and calibrator correlated flux data to yield calibrated correlated fluxes. Does not handle photometry data. Runs midiVisPipe
on both input data sets, and midiCalibrate to execute calibration.
Default parameters for midiVisPipe are used unless you specify a parmfile.
Inputs:
targetdata: raw interferometry data for target. If multiple files these
should be blank separated and surrounded by "" signs.
caldata: same as above for calibrator
targtag: character string to mark all output files for target
caltag: character string to mark all output files for target
db: previously created database containing calibrator spectra etc.
default: van Boekel database.
mask: file containing mask used for both data sets.
Default: new mask will be created from calibrator data using midiMakeMask
parmfile:A text file containing valid IDL commands to set (a few) parameters for midiVisPipe.
These will be executed using the IDL execute command before running midiVisPipe. Parameters that can be set are smooth,gsmooth,msmooth
Outputs: all the normal outputs from midiVisPipe for both target and calibrator, plus outputs of midiCalibrate.
Compute a 2-dimension (wavelength,y-position) complex image of correlated flux from a source. This in contrast to a normal reduction that collapses the
raw spectrum to 1-dimension before further processing. The principel
use of this routine is to create images of calibrators that can be
used for constructing masks.
Inputs:
tag: character string to mark output files
file: raw interferometric input data
smooth: input high-pass filter interval for oirFormFringes (seconds)
noAve: if specified, do not remove spectral average in oirFormFringes
noDelete: This routine generates large temporary files that are
deleted at the end. If nodelete is specified (for diagnostic purposes)
this is suppressed.
Outputs:
A pseudo-complex image file tag.fringeimages.fits that contains
the complex output separately for each interferometric input channel.
Produce a set of simulations to indicate how much flux MIDI/EWS loses as a source becomes
weaker and the group delay tracking becomes inaccurate. The procedure assumes that you have
already run midiVisPipe or a similar procedure on both your target data and on a strong calibrator.
midiDecorrelate will then:
Extract the reduced correlated fluxes from both data sets.
Rescale the--assumed noiseless--calibrator data so that it will produce a spectrum identical to the target.
Extract a compressed noise data set from the target data displaced along the slit.
In a loop: attenuate the rescaled calibrator data by a given factor, add it to the noise data,
and reduce it with midiVisPipe as if it were a normal observation.
Inputs:
targetdata: raw interferometry data for target.
caldata: raw data for calibrator
targtag: reference name for reduced data files for target
caltag: reference name for reduced data files for calibrator
tag: reference name for data files produced by this program (default="weak")
ratios: vector of values by which to attenuate rescaled calibrator
...: typical midiVisPipe parameters may be specified: mask,gsmooth,smooth,msmooth,twopass,ave
parmfile: A text file containing IDL commands to be executed before starting the midiVisPipe procedure.
This is an alternative way to specify values to be used for gsmooth etc. i.e. the file may contain statements like:
gsmooth=.6
msmooth=2.
Outputs:
An array of IDL oirVis structures, i.e. what you would get with oirGetVis(tag.corr.fits) on each of the simulations.
You can run midiCvis on each element of the array.
The first structure in the array corresponds to the unmodified calibrator.
The second element corresponds to your unmodified target.
The succeeding elements correspond to the rescaled calibrator multiplied by the elements of ratios and added to noise data.
Thus if you specified ratios=[1.,.1] on a moderately strong target, the 3rd output element should look almost identical to the 2nd, differing by a small addition of noise.
If the target is very strong the 4th element will look like the 2nd, multiplied by 0.1,
but if the target is not so strong, the tracking errors at this low flux will reduce the output
flux by even more. Running this procedure with a variety of ratios should allow you
to estimate the importance of this loss of flux for your target.
These routines allow access to databases containing position, flux, diameter, and spectral data for calibrators. Here I describe only the routines for creating and IDL version of a database from a Fits representation, and for retrieving data.
cdb = midiCalDatabase(fitsDatabaseFile) Purpose: Create an IDL internal version of a cal database from a fits file. There is currently only one such Fits database:vBoekelDatabase.fits which is included in the EWS distribution. This database was provided in 2010 by Roy van Boekel, with no guarantee on quality, and no obligation to maintain it. It contains Cohen model representations of spectrophotometric data for about 800 stars. This function returns an IDL object:cdb (you can choose any name you like for this variable), which can be used in the access routines described below. Parameters: fitsDatabaseFile: The name of a fits file containing calibrator data in "Jaffe Calibrator Table Format". I can provide documentation on this format to anyone who wants it. The file "vBoekelDatabase.fits" is contained in the directory "root"/idl/calibrators/databases in the distribution .tar file for MIA+EWS
cdb = vboekelbase () Purpose: a short cut to midiCalDatabase that assumes that you want the v.Boekel database. Parameters: None
nFound = cdb->sourceByName (name, sourceData, diamData, photData, specData) Purpose: Find a source in an existing database given its name and return the stored data. Note: In this routine the name must agree with the standard name stored in the database. These are typically HD names (uppercase, no blanks or underscores). In most cases it is better to call sourceByAlias Parameters: name: source name, character string Returns directly: a vector of 4 integers describing the number of data sets with this name describing: 1. =1 if some sources are found; =0 if none found 2. Source Names 3. Photometry 4. SpectroPhotometry Returns in calling argument list: SourceData:a structure containing mostly positional data about source, e.g. name, ra, dec (equinox/epoch) and velocities, proper motions and parallaxes (if known) diamData:a structure containing diameter information (if known), e.g. name, wavelength band where measured, diameter (and error), date... photData: photometry data: name, band, flux(Jy) (U,Q,V if known), quality flag specData: spectrophotometry data: name, band, date, quality,wavelength array, flux array(Jy).
nFound = cdb->sourceByAlias(name, sourceData, diamData, photData, specData) Purpose: Same as sourceByName; actually this program calls the other program. First however the name is converted to "standard" format (uppercase, no blanks). Then an alias table is searched for equivalent names. For the moment this table is largely empty but the intent is to fill it with other names (e.g. HR, HIP, IRAS, 2mass, constellation names). However if you use this routine with HD names you don't have to worry about blank spaces and upper/lower case problems. Parameters: see sourceByName
opd = cdb->sourcesNearPosition(ra, dec, sources, nmax=nmax, rmax=rmax) Purpose: get information on calibrator sources near a specified position Parameters: ra: Right Ascension. Can be specified as a scalar=position in degrees; an array of three numbers= [hours, minutes, seconds], or a blank-separated character string:"hh mm ss.ss" dec: declination. Same choices as ra for formats but all in degrees. nmax: (optional; default 30) maximum number of sources to return rmax: (optional; default 1 degree) largest acceptable distance from specified position Returns: A vector with the distance (degrees) from the calibrators found to the specified position. Returns in argument list: sources: source structures for the calibrators found you can use sources.name as inputs to sourceByName to get additional data about these sources.
sourceData = cdb->sourceFromFile(fitsFile, diamData=diamData, photData=photData, specData=specData, rad=rad, /silent) Purpose: Find the calibrator nearest to a position specified in a MIDI data file, either raw data or reduced data. The information returned is similar to that in sourceByName. Parameters: fitsFile: reference data file. The RA and DEC specified in the FITS header are used as reference positions. rad: (optional: default = 1 arc minute) maximum acceptable distance (degrees) from reference position silent: (optional: default=noisy) Suppress some of the message generated while looking for targets Returns: A source structure as defined in source ByName Returns in argument list: diameter, photometry and spectrophotometry data
spectrum = cdb->spectrumFromFile(fitsfile, diamdata, wavelength, rad=rad,/silent) Purpose: Find calibrator near position specified in a reference file header and return its spectrum (if found) interpolated to the wavelengths specified in the MIDI file. Parameters: see sourceFromFile Returns: Spectrum of the calibrator (Jy) found near the position specified in file header, interpolated to the wavelengths specified in the wavelength tables contained in the file.
Purpose: read data from a MIDI FITS ImageData table file into an internal IDL structure.
Parameters:
datafile: name of input file. e.g. CenA.compressed.fits If the input data consists
of multiple, to-be-concatenated files, specify files as a single character string
with a space between the successive files
rows: optional. If specified this is an IDL array containing the rows numbers that you want.
These start at 1 not 0 (FITS standard). This option cannot be used for multiple files.
Default: all rows in file(s).
col optional. A character string array specifying which columns of the data table to read.
E.g. col=['DATA1','TIME'] Specifying this does not save time, but it can save space inside IDL.
Purpose: read data from a MIDI FITS OIR Vis table file into an internal IDL structure. Parameters: visFile: name of input file. e.g. CenA.corr.fits wave optional. Returns wavelength information for vis data
Purpose: Make a plot of tracked OPD and estimated Group Delay as a function of time for data which has already been reduced using midiPipe or midiVisPipe. The plot is similar to the one shown by these functions, but by setting !x.range or !y.range before calling the routine, or setting other IDL plot variables, you can influence the appearance of the plot, or direct it to a file, etc. Parameters: tag: as in midiPipe, sets a directory and prefix that determines which data to use. title: a character string to be used at the top of the plot as a title.
corr = midiGetCorr (tag[,wave=wave]) Purpose: return a vector containing the (uncalibrated) Correlated Flux computed by midiPipe, for the data specified by tag. I.e. this is the visamp vector stored in the file tag.corr.fits Parameters: tag: as used everywhere. wave = variable_name: returns into the specified variable a vector of wavelengths (microns) corresponding to the data arrays described below. Returns: a vector of values specifying the uncalibrated correlated flux amplitude (ADU/s) for the data specified by tag
phase = midiGetPhase (tag[,wave=wave]) Purpose: return a vector containing the (uncalibrated) phase of the correlated signal, as computed by midiPipe, for the data specified by tag. I.e. this is the visphi vector stored in the file tag.corr.fits Parameters: tag: as used everywhere. wave = variable_name: returns into the specified variable a vector of wavelengths (microns) corresponding to the data arrays described below. Returns: a vector of values specifying the uncalibrated correlated phase (degrees) the data specified by tag
phot = midiGetPhot(tag) Purpose: return a structure containing the (uncalibrated) photometry for the data specified by tag. Parameters: tag: as used everywhere. Returns: an IDL structure named whatever is on the left-hand side of the = sign containing the uncalibrated photometric data as computed by midiPipe or by midiPhotoPipe. This is the data contained in the file tag.photometry.fits. For example phot[0].data1 contains a vector of the estimated total flux (ADU/s) from telescope A.
opd = midiGetOpd(tag) Purpose: get OPD tracking positions for the data specified by tag. Parameters: tag: as used everywhere. Returns: an vector sign containing the OPD tracking position (microns).
mydelay = midiGetDelay(tag) Purpose: get estimated Group Delay positions for the data specified by tag. Parameters: tag: as used everywhere. Returns: an IDL structure with the group delay information returned by oirGroupDelay The more interesting elements in this structure are: mydelay.delay: the estimated group delay (microns) for each frame mydelay.time: time (MJD days) of each frame mydelay.amplitude: amplitude of fringe signal found at this delay (ADUs)
choppedData = midiChopImage(inputFiles) Purpose: Average all "target" frames in a set of chopped data (e.g. target acquisition or photometry frames) and subtract the average of all "sky" frames. Parameters: inputFiles: name of files, if multiple files this should be a single, blank-separated list: e.g. "file1.fits file2.fits" Returns: an IDL structure with elements like "choppedData.DATA1" "choppedData.DATA1" that represent the signals in each of the MIDI windows for this data. The structure has two rows: choppedData[0] contains Target-Sky while choppedData[1] contains Sky.
keyValue = midiGetKeyword(keyword, inputFiles) Purpose: Get keyword values from primary headers of one or more FITS files. Parameters: keyword: a character string specifying keyword. For ESO-style HIERARCH keywords specify at least two of the keywords and enough to be unique, e.g. "FILT NAME". inputFiles: a single character string or array of character strings specifying a set of files to search. If an array is specified, an array of values will be returned. If the keyword is missing in the header, the program returns 0. If it is present in some files and missing in others the program will probably crash. Returns: an IDL array containing the values of the specified keywords.
complexVisibilitySpectrum = midiCVis(oirVisibilityStruct) Purpose: Convert visibility structure into IDL complex visibility vector Parameters: oirVisibilityStruct: The IDL structure representation of a visibility spectrum. This is what is returned by oirGetVis when reading the output of oirAverageVis. So you can combine these: spectrum=midiCVis(oirGetVis(visFile,wave=wave)) Returns: an IDL complex array containing the visibility spectrum.
p = midiPhase (cArray) Purpose: compute the phase of an array of complex numbers. Parameters: cArray: an IDL complex array Returns: An array of the phases of each element of cArray, i.e. arctan(Im(cArray)/Re(cArray)), in the range -pi <-> pi. Note: p = midiPhased(cArray) returns values in degrees (-180->180)
c = pseudoComplex (pcArray) Purpose: convert a 2-dimensional array of pseudoComplex numbers into IDL complex numbers and transpose the array. Several of the c-programs, such as oirGroupDelay produce output data arrays that are in fact complex. These are stored in the FITS files as pseudoComplex, i.e. as pairs of real numbers. Inside IDL it is easier to deal with true complex numbers, so this routine converts the FITS pairs into IDL complex numbers. In addition, most of the FITS files are stored with channel number as the X-axis and frame number, or time as the Y-axis. For visualization it us usually more convenient the other way around, so pseudoComplextransposes the array. Parameters: pcArray: a 2-d array of real numbers, each two values representing one complex number, arranged in channel-time order. Returns: An array of the complex numbers arranged in time-channel order.
c = midiGetComplex (tag,qualifier) Purpose: read a MIDI image file stored in pseudoComplex format. Parameters: tag: 1st component of a MIDI FITS file name qualifier: 2nd component of MIDI FITS file name. e.g. The output of oirFGroupDelay might be CenA.groupdelay.fits. The data1 element of the file contents can be read into an IDL complex array with: carray = midiGetComplex('CenA','groupdelay') Returns: An array of the complex numbers arranged in time-channel order.
EWS includes a data selection GUI, somewhat like GASGANO. To browse your raw data, and select some of it type:
FileList = midiGui()There are alternate versions described below. There is a faster version that caches the contents of the file headers on disk, but it only works if you have write permission on the data disks:
FileList = midiGuis()Note that IDL is indifferent to upper/lower case letters in commands and variable names (but not indifferent to case in FileNames)
It brings up a GUI and returns a list of files selected.
The gui is pretty spartan. Up at the top there are 10 buttons that let you select files, modify the display a bit, and quit.
Below this there is a scrollable area that shows information on each of the available files and on the left a column of ASTERISKs (*) showing which files are currently selected, or blanks if they are not. Initially files are brought up all un-selected.
In the next column you see the file name. NOTE: Due to ESO limitations on file lengths, MIDI files cannot be longer than 100 MB or so, while a single observations may generate Gigabytes of data. The on-line data system breaks the data up into 100 MB files. Gorgonzola recognizes this process, and the displayed file name is only the FIRST file of all the files that constitute an observation. When Gorgonzola returns, all the sub-files for an observations are concatenated into a single character string with spaces between the file names. Most of the EWS programs can recognize such a string and process the files sequentially. NOTE further, however, that if you use such a string to call a C-program directly (e.g. oir1dCompressData), you have to enclose the concatenated string in double quotes "...".
The other columns show keyword values for each observations, gleaned from the file primary FITS headers.
Two things:
We start with the easy ones:
QUIT causes the GUI to leave without making a selection
SELECT causes the GUI to leave and returns the selected files
UP/DOWN IDL and X-windows crash if thousands of widgets are put
on the screen, so I have limited the display to 100 rows.
If the rows you want to see are outside this range you
can push UP or DOWN to scroll. NOTE that the scroll bar
on the right only allows scrolling within the 100 displayed
rows.
HIDE Remove all currently unselected rows from the display. This
can considerably simplify things if you've already de-selected
a lot of files.
SHOW Reverse of HIDE: show all files, selected or not. This gives
you the opportunity to selectively re-select unselected files.
Sometimes the first few non-selected files are not displayed
after hitting the SHOW button. Try clicking the UP button.
To the right of this row the current HIDE/SHOW mode is displayed
Then the real meat: the BOOLEAN buttons.
These are not strictly boolean but sort of.
When you click one of them, the "caught" value
is compared to all other values in the same column
and the currently list of selected files is modified:
In the second row down on the GUI are six comparison operators:
= Equal != Not Equal > Greater than < Less than <= Greater than or equal > Less than or equalTo the right of this row the currently selected comparison is shown.
In the third row two combination operators:
AND The results of the comparison are
logically ANDed with the current selection
OR The results of the comparison are
logicall ORed with the current selection
Confused? It actually works fairly intuitively.
Suppose you have made a hundred or so exposures during a day, and suppose you are looking for a group of exposures with the PRISM in HIGH_SENS mode. The fifth column of selction buttons is labelled "INSGRIS" (shortend from INS GRIS NAME). Look down this column, with the slider if necessary until you find a button with PRISM displayed. Click on this and the words INGRIS and PRISM should appear below the AND/OR buttons to indicate your selection. Now click on the OR button. All observations with GRIS=PRISM are ORed with the initial selection (i.e. nothing), and asterisks should appear in the left hand column of all PRISM observations.
Now click on HIDE and all non-PRISM observations should disappear.
Now go down the INSOPT1 column and find an observations with OPT1=HIGH_SENS. Click on this, and then on AND. This will AND the set of HIGH_SENS observations with the current PRISM selection and any SCI_PHOT or OPEN observations should disappear. You can continue by selecting on OBSTARG (target name as given by the observer) or NRTSMODE (what the online program thought it was doing), or FILTER or SHUTTER position.
Using the other relational buttons, you can also select by time, because the file names are ordered by time. For example you can click on the file name of first observation you want, then click on the >= button and then on AND. All the earlier files will be deselected. Then you can click on the last observation, the <= button and AND, and the later observations will disappear.
When you have made the selection you want, click on SELECT.
At the top right the total number of files in the list, and the total number currently selected, are displayed.
If you click on a data item and then click HIDE, not much will happen (except that previously de-selected rows may disappear). You first have to press on of the boolean buttons to make something change.
files = midiGuiS()
Look for a file in the search directory called midiGui.SAV, and if you find it, assume that it contains the keyword values for all the MIDI files in this directory. This speeds things up a lot. If you don't find it, parse all the headers and store the summary in the abovenamed file upon exit. This will crash if you don't have write privliges.
If you can added/deleted files in the directory, so that the cached version is not up to date, just delete it and try midiGuiS again.
files = midiGui(dir=datadir) or midiGuiS(dir=datadir)
gorgonzola assumes that idl is running in the directory where the data is. If this is not the case you can use the above forms to specify a different data directory. You can also use the IDL utility pushd:
pushd, datadir files = midigui() popd
The following standard file types are produced by the specified programs.
Some notes indicate how to access the contents of the programs:
Some of the lower level programs to access this data are described below
but almost all are listed here:
oirGetData(dataFileName [,rows] [,col=col],[ierr=ierr])
Get raw or semi-reduced detector data
oirGetVis(visFileName [,wave=wave])
Get reduced visibility data and wavelength information
oirGetDelay(groupDelayFileName)
Get estimated group delay for each reduced frame
oirGetOpd(fileName)
From raw or partially reduced data get piezo+VLTI delay
line OPD position
oirGetMeanRMS(inputfile)
Take pixel by pixel mean and rms of all frames in an observation
oirGetDetector(fileName)
Get data from IMAGING_DETECTOR table, specifying detector layout
oirGetWaveNo(detectorFile, region)
Get wavenumbers (here defined as 2*!pi/lambda(micron)) for
each pixel in specified detector region (1-relative)
midChopImage (datafile)
Return 2-row structure with mean of Target- mean of Sky
detector images in first row, and mean of Sky in 2nd row.
File types:
.compressed
Produced by oir1dCompressData, contains a header, an
imaging_detector table and a floating point imaging_data table.
The last has a 1-dimensional DATAn array for each input frame
for each detector window.
This can be accessed with oirGetData(dataFileName).
dataArray = oirGetData(dataFileName)
oirGetData works on raw and partially reduced detector data.
It returns an array of structures. Each structure corresponds to
one data frame, and corresponds to ESO FITS interferometry
data tables. Each row of the table contains information on
the time of exposure (MJD days), exposure time (sec), OPD positions,
chopping mirror positions, and finally the data itself.
dataArray[15].time contains the MJD time of the 16th frame
(IDL is 0-relative).
dataArray.data1 is in general a 3-dimensional data cube, where
each plane represents the detector data from the 1st specified
detector region, and the 3rd dimension is frame number.
The data arrays are INT for raw data and FLOAT for partially
reduced data. Note that the raw data (16-bit integers) is
offset by -32768 from the true detector zero values.
.fringes
Produced by oirFormFringes, contains a header, an imaging_detector table
and a imaging_data table with a single DATA1 array for each input frame.
This can be accessed with
table = oirGetData(dataFileName)
data = table.data1
.insopd
Produced by oirRotateInsOpd. Contains a header and imaging_detector table,
and a pseudo-complex imaging_data table with a single DATA1 array per row.
Header contains keyword OPD0 describing offset subtracted from all
OPDs before rotation (=mode of tracking OPDs).
This can be accessed with
table = oirGetData(dataFileName).
data = pseudocomplex(table.data1)
but more directly with:
data = midiGetComplex(tag, 'insopd')
data is them COMPLEX(nFrames, nFreq)
.groupdelay
Produced by oirGroupDelay or oirFGroupDelay.
Contains a header and imaging_detector table,
and a pseudo-complex imaging_data table with the FFT of .insopd. The
header contains some special keywords:
OPD0: as in .insopd
OPD1,OPD2: coordinates of delay direction in output FFT. The delays
= OPD1 + OPD2*(xpix-1).
This table can be accessed with gdc=midiGetComplex(tag,'groupdelay')
gdc is COMPLEX(nFrames, 512), with the y-axis representing delay.
Additionally there is a 3rd table DELAY containing the
time (MJD days), telescope numbers and estimated OPD (seconds) and fringe amplitude. This
can be accessed with oirGetDelay(fileName) or midiGetDelay(tag).
.ungroupdelay
Produced by oirRotateGroupDelay . Contains the usual header and
detector tables and a imaging_data table in pseudo-complex format.
Can be accessed with ugdc=midiGetComplex(tag,'ungroupdelay')which is COMPLEX(nFrame, nFreq)
Additionally there is a 3rd table DISPERSION containing (per frame)
the value of the average phase subtracted from each row. The
table has three columns "TIME" "TELESCOPE" and "DISPERSION", which
for the purpose at hand, is the subtracted mean phase in degrees.
This is accessed by the generic fitstable routines:
t = obj_new('fitstable', filename, extname='DISPERSION')
dispData = t->readrows()
obj_destroy,t
.flag
Produced by oirAutoFlag. Contains a header and a FLAG table with the
time intervals that have been flagged. There is no oirGetFlag routine.
You can use:
t = obj_new('fitstable', filename, extname='FLAG')
flagData = t->readrows()
obj_destroy,t
.corr
Produced by oirAverageVis. Contains a header, an OI_WAVELENGTH table
specifying the wavelength for each channel, and an OI_VIS table
containing averaged correlated fluxes.
Can be accessed with
visTable = oirGetVis(filename, wave=wave) with wave optional
visTable.visamp is an array of (nWave) correlated fluxes (Jy)
visTable.visphi is an array of (nWave) differential phases (degree)
visTable.visamperr contains the estimated rms of visamp
visTable.visphierr contains the estimated rms of visphi
wave contains nWave values of the wavelength (microns)
These visibilities can be converted to a complex array with vis=midiCVis(visTable)
.[AB]photometry
Produced by oirChopPhotoImages .
Header, detector tables, and imaging_data table in float format
Normally contains 7 rows of two-dimensional "sky-subtracted" detector images. The first
is the average of all target frames - average of all sky frames.
The next 5 are similar images for 5 subsets of the data (for noise estimation).
The last is the average of all sky frames.
Can be accessed with photimages = oirGetData('tag.Aphotometry.fits')
.photometry
Produced by oirMakePhotoSpectra .
Header, detector tables, and imaging_data table in float format
Can be accessed with photData = oirGetData('tag.photometry.fits')
Normally contains 12 rows(see oirMakePhotoSpectra above)
photData[0].data1 contains the 1-dimensional photometry spectra for telescope A
photData[1].data1 contains the 1-dimensional photometry spectra for telescope B
...etc
.redcal
Produced by oirRedCal. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with vis=oirGetVis(filename, wave=wave)
.calphot
Produced by oirCalibrateVis. Header, detector, imaging_data table
Can be accessed with photdata=oirGetData(filename)
.calvis
Produced by oirCalibrateVis. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with calibratedVisibilities=oirGetVis(filename, wave=wave)
.calcorr
Produced by oirCalibrateVis. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with calibratedFluxes=oirGetVis(filename, wave=wave)
Table-of-Contents C-routines IDL-Routines