The funcs module contains classes and functions of general utility of astronomical purpose. Most of these functions are used all over the place throughout mascara, but can also be used as stand-alone functions.
The mascara.funcs module is composed of three submodules to make the organisation and loading simpler. The submodules are mostly independant from each other and relate to different topics:
mascara.funcs.timing contains various timing routine which are commonly used in astronomy. The calendar conversion functions concentrates mostly on converting calendar dates into Julian dates back and forth using mascara.funcs.timing.calendar2jd() and mascara.funcs.timing.jd2calendar() For instance, for converting a calendar date to Julian Date:
>>> from mascara.funcs.timing import calendar2jd
>>> calendar2jd(2015, 02, 23, 11, 23, 43)
>>> 2457076.974803241
It is also possible convert a time difference in Julian date:
>>> from datetime import timedelta
>>> calendar2jd(timedelta(seconds=20))
0.00023148
Moreover, it is possible to check for the time continuity inside a sequence. mascara.funcs.timing.checkdiscontinuity() verifies that given the observing time and exposure time of two consecutive images, these are taken one after the other.
>>> from mascara.funcs.timing import checkdiscontinuity
>>> checkdiscontinuity((2457076.9748, 2457076.9948), 10.)
True
mascara.funcs.mio contains a large number of I/O functions, going from writing files to disks, finding files or creating directories. Some basics examples are below:
For saving an image and header into a compressed fits file with mascara.funcs.mio.saveCompFits() :
>>> from mascara.funcs.mio import saveCompFits, savepng
>>> data = numpy.random.rand(1000,1000)
>>> header = pyfits.Header()
>>> saveCompFits('mydirectory', 'myfitsname', data, header)
For saving an image in .png format and binning it beforehand:
>>> image = numpy.random.rand(1000, 1000)
>>> savepng(image, 'mydirectory', 'myimagename', binfactor=4)
For finding all the files ending with ‘LPS’ in mydirectory with a given format use mascara.funcs.mio.find_files() :
>>> from mascara.funcs.mio import find_files, list_directories, pick_directories
>>> fn = find_files('mydirectory', '*LPS', format='.fits')
It works also for finding files starting with LPS in another format:
>>> fn = find_files('mydirectory', 'LPS*', format='.h5')
Similarly, to list all the subdirectories inside a given directory which have a given common root with mascara.funcs.mio.list_directories()
>>> directories = list_directories('mydirectory', 'LP')
>>> print directories
['mydirectory/201410LPS',
'mydirectory/201410LPW',
'mydirectory/201522LPS']
And if inside that list, an additional selection along another root is required with mascara.funcs.pick_directories()
>>> pick_directories(directories, 'S')
['mydirectory/201410LPS',
'mydirectory/201522LPS']
mascara.funcs.mio.check_free_space() calculates the available disk space on a give hard drive. The function is valid both on Linux and Windows platforms. The output is in MB.
>>> from mascara.funcs.mio import check_free_space, make_disk_space, moveData
>>> check_free_space(folder='C:\\')
450273L
It is possible to then free some disk space by deleting files inside a folder. mascara.funcs.mio.make_disk_space() will by default remove the oldest files / subdirectory inside the given folder
>>> make_disk_space(path='E:\\', file = 'thisone.txt')
To copy a directory from one location to another:
>>> moveData(frompath, topath)
mascara.funcs.utils includes some general functions for statistics:
- mascara.funcs.utils.find_outliers(), to find N-sigma outliers inside a distribution
- mascara.funcs.utils.robuststd(), to calculate a somewhat more robust standard deviation of a distribution
- mascara.funcs.utils.rebin(), to rebin a array to new dimensions
- mascara.funcs.utils.statbin(), to rebin and obtain the standard deviation inside each bin
- mascara.funcs.utils.percentile(), to calculate the Nth percentile of a distribution
Let’s demonstrate how they work:
>>> distr = numpy.random.rand(1000) >>> distr = numpy.hstack((distr, np.ones(10)+1.5)) >>> print distr.std(), mascara.funcs.utils.robuststd(distr) 0.348824171156 0.288635778078 >>> mascara.funcs.utils.percentile(distr, 0.75) 0.75398454483993305But also for reading in and working with stallar catalogues:
- mascara.funcs.utils.loadStellarCatalog(), to read in a stellar catalog
- mascara.funcs.utils.cleanupStellarCatalog(), to flag and remove the closest stars in order to avoid blending issues
- mascara.funcs.utils.skyQuad(), creates a complicated grid on the image which can be used to build low order transmission maps...
>>> bla = mascara.funcs.utils.loadStellarCatalog(filename='StarCat.fits', Vmag=(4., 7.)) >>> ra, dec, vmag, bmag, sID, sptype = bla
The Stacker class is for easy stacking of images taken from the same camera at close observing time. The images are ‘rotated’, practically shifted, from their observing time to a reference time. In order to work best, it is recommended that the observing time and reference time do not differ by more than 10min. The Stacker class needs as inputs a mascara.observer.Site and mascara.observer.Camera instance.
>>> from mascara.funcs.utils import Stacker
>>> stacker = Stacker(site, camera)
>>> stacker.image_rotate(image, obstime, reftime, exposuretime=6.4)
>>> binnedimage, nimages = stacker.binImage(datetime.now())
Python 3.2 and higher up versions enable easy logging with multiprocessing. The two following classes are used to reproduce this feature on Python 2.7:
Those are loading for logging in the following way:
>>> import mascara
>>> import multiprocessing
>>> ### Start a queue which allows communication between the processes
>>> qlog = multiprocessing.Queue()
>>> ### Convert the queue to a handling function
>>> qh = mascara.funcs.utils.QueueHandler(qlog)
>>> ### Get the logger
>>> logger = logging.getLogger()
>>> ### Give the logger the handler
>>> logger.addHandler(qh)
>>> ### And now create the function which will read from the queue
>>> ql = Mmascara.funcs.utils.QueueListener(qlog, handler)
>>> ql.start()
Stacker creates a stacker instance which is used to stack images on top of each other in order to bin them temporally. The stacker first align the images before stacking them. In order to do so, it requires valid site and camera instances as inputs.
Initialise the Stacker Class.
Bin the actual image
“rotate” image to new reference frame. The image is actually not rotated, but shifted in pieces to the new location for the given TimeReference.
- image, the image to ‘rotate’
- exposureTime, default false, else precise the exposure time
of the image
This class is new in python 3.2 and later but has been copied here for python 2.7. It sends a logging handler to a Queue. It would be used together with a multiprocessing queue to centralise logging to file in a multi-processing configuration and to avoid write contention between processes.
Queue listener works in pair with QueueHandler for writing/printing log files using logging in the case of multi-processes. This class implements an internal thread listener which watches for LogRecords being added to Queue, removes them and passes them a list of handler for processing.
Monitors the queue for records and ask the handler to deal with them. This method run on a separate internal thread. The thread will terminate once it sees the sentinel object in the queue.
Handle a record. This just loops through the handlers offering them the record to handle.
timing functions used for MASCARA
Given an exposure time -texp- and an array containing at least the last 2 dates of observation, checktimescale verifies that the latest exposure is made in the continuity of the previous one. This is relevant in cases of a serie of observations during one night. If the 2 exposures diverge by more than twice the exposure time, than a discontinuity is found, and the function returns True.
Convert a Julian day date (JD) into a calendar date, using a datetime format.
Calendar to Jd converts a datetime object into a Julian date object
Conversion of a given Julian date to an epoch expressed in decimal years.
Keyword: Julian. If not set, the reference epoch is 1900
Output: epoch. The date converted in decimal years.
I/O functions used for MASCARA
List of the files in the given format in the given path directory. It works very similarly to : ls mydirectory/test*.fits
can feed it a root and it’ll list all directories having that root in their names in the given path.
- Example:
>>> dires = list_directories('D:\LaPalma\', '20141114')
picks the right directories for a list using the root as reference
- dirlist, a list of directories
- root, a reference to search for in the list
CheckFreeSpace looks how much free space is in the given folder... preferably the hard disk. To be used to for overwriting/deleting old raw data at the beginning of the night, to avoid getting into memory troubles.
Input: folder, a string giving the path of the folder to check
Output: the amount of free space in MB
Removes the oldest files in the path
Copy the Data from the directory frompath to the directory topath. and delete afterwards the frompath.
savepng saves the input image at the given path. The image can be rebinned using the binfactor keyword. The scaling is defined via ...
to save an image in a compressed fits.
- path, the path for saving the images
- name, the name of the image
- data, the data to be saved
- header, the corresponding header
The data is saved in a CompImageHDU, using a RICE compression type. If the data is not in UINT16 format, it will be converted before the compression.
Utilities functions used for MASCARA
find_outliers searches for outliers in Y as a function of X.
Calculates a robust standard deviation for the distribution X, by removing gradually the outliers. Usefull when a few outliers are skewing the calculation of the standard deviation.
Percentile computes the qth percentile of the data. It is based on the numpy version percentile, however without any interpolation of the numbers. And is faster than the scipy.scoreatpercentile() function.
Returns a gaussian function centered in xo, with a width w and an amplitude A.
rebins an image to a new format. Condition: newformat is smaller than the original image format.
statbin returns the mean values and standard deviation of the values inside a bin.
loadStellarCatalog load in memory a binary fits table catalog of stellar names and coordinates for further use (for instance predict the visible stars from an observing site).
Astrostars is a short routine for selecting stars within a catalog that are suited for astrometry. They are bright (limiting magnitude defined via mag1), and have no close neighbours (define close via dist) of magnitude down to mag2.
skyQuad does nothing with the image itself. Creates tiles for further sky quality analysis.
Convert any type of spherical coordinates in a matrix of cartesian rectangular coordinates to be used for coordinates conversion.
Example: To convert (ra-dec)(J2000) to (ra-dec)(to date):
>>> X = pol2rect(ra, dec)
>>> Xnew = N*P*X # where N and P are the nutation and precession
matrices.
>>> ranew, decnew = rec2pol(Xnew)
Conversion from rectangular coordinates to equatorial. Special care has to be taken around the celestial poles
Rotmat generates a 3x3 rotation matrix for a given angle around the requested axis (x, y, or z)
Makevortex uses a Delauney griding method to create from the input coordinates a triangular grid. Each star is matched with its closest neighboor to form a triangle. The method is used in the astrometric solution for identifying the stars together.
match_vortex matches together observed vortices made of the measured stellar positions to the vortices made from the calculated stellar positions.