.. _mascfunctapis:

********************************
funcs -- general usage functions
********************************

Introduction
============

The :mod:`funcs` module contains classes and functions of general utility of astronomical purpose.
Most of these functions are used all over the place throughout `mascara`, but can also be used
as stand-alone functions.

The :mod:`mascara.funcs` module is composed of three submodules to make the organisation and 
loading simpler. The submodules are mostly independant from each other and relate to different 
topics:

.. _timing-routines:

timing -- Calendar conversion and timing functions
==================================================

:mod:`mascara.funcs.timing` contains various timing routine which are commonly used in astronomy.
The calendar conversion functions concentrates mostly on converting calendar dates into Julian dates back and forth using :func:`mascara.funcs.timing.calendar2jd` and :func:`mascara.funcs.timing.jd2calendar`
For instance, for converting a calendar date to Julian Date:

>>> from mascara.funcs.timing import calendar2jd
>>> calendar2jd(2015, 02, 23, 11, 23, 43)
>>> 2457076.974803241

It is also possible convert a time difference in Julian date:

>>> from datetime import timedelta
>>> calendar2jd(timedelta(seconds=20))
    0.00023148
	
Moreover, it is possible to check for the time continuity inside a sequence. :func:`mascara.funcs.timing.checkdiscontinuity`   verifies that given the observing time and exposure time of two consecutive images, these are taken one after the other. 

>>> from mascara.funcs.timing import checkdiscontinuity
>>> checkdiscontinuity((2457076.9748, 2457076.9948), 10.)
    True
	
.. _I/O-routines:

mio -- I/O functions
====================

:mod:`mascara.funcs.mio` contains a large number of I/O functions, going from writing files to disks, finding files or creating directories. Some basics examples are below:

Saving functions
----------------

For saving an image and header into a compressed fits file with :func:`mascara.funcs.mio.saveCompFits` :

>>> from mascara.funcs.mio import saveCompFits,  savepng
>>> data = numpy.random.rand(1000,1000)
>>> header = pyfits.Header()
>>> saveCompFits('mydirectory', 'myfitsname', data, header)

For saving an image in .png format and binning it beforehand:

>>> image = numpy.random.rand(1000, 1000)
>>> savepng(image, 'mydirectory', 'myimagename', binfactor=4)


Find files or directory
-----------------------

For finding all the files ending with 'LPS' in mydirectory with a given format use :func:`mascara.funcs.mio.find_files` :

>>> from mascara.funcs.mio import find_files, list_directories, pick_directories
>>> fn = find_files('mydirectory', '*LPS', format='.fits')

It works also for finding files starting with LPS in another format:

>>> fn = find_files('mydirectory', 'LPS*', format='.h5')

Similarly, to list all the subdirectories inside a given directory which have a given common root with :func:`mascara.funcs.mio.list_directories`  

>>> directories = list_directories('mydirectory', 'LP')
>>> print directories
    ['mydirectory/201410LPS', 
	 'mydirectory/201410LPW', 
	 'mydirectory/201522LPS']
	
And if inside that list, an additional selection along another root is required with :func:`mascara.funcs.pick_directories`

>>> pick_directories(directories, 'S')
    ['mydirectory/201410LPS', 
	 'mydirectory/201522LPS']


Disk operations
---------------

:func:`mascara.funcs.mio.check_free_space` calculates the available disk space on a give hard drive. The function is valid both on Linux and Windows platforms. The output is in MB.

>>> from mascara.funcs.mio import check_free_space, make_disk_space, moveData
>>> check_free_space(folder='C:\\')
    450273L
	
It is possible to then free some disk space by deleting files inside a folder. :func:`mascara.funcs.mio.make_disk_space` will by default remove the oldest files / subdirectory inside the given folder

>>> make_disk_space(path='E:\\', file = 'thisone.txt')

To copy a directory from one location to another:

>>> moveData(frompath, topath)

.. _general:

utils -- General usage functions
================================

:mod:`mascara.funcs.utils` includes some general functions for statistics: 

 * :func:`mascara.funcs.utils.find_outliers`, to find N-sigma outliers inside a distribution
 * :func:`mascara.funcs.utils.robuststd`, to calculate a somewhat more robust standard deviation of a distribution
 * :func:`mascara.funcs.utils.rebin`, to rebin a array to new dimensions
 * :func:`mascara.funcs.utils.statbin`, to rebin and obtain the standard deviation inside each bin
 * :func:`mascara.funcs.utils.percentile`, to calculate the Nth percentile of a distribution
 
 Let's demonstrate how they work:
 
 >>> distr = numpy.random.rand(1000)
 >>> distr = numpy.hstack((distr, np.ones(10)+1.5))
 >>> print distr.std(), mascara.funcs.utils.robuststd(distr)
     0.348824171156 0.288635778078
 >>> mascara.funcs.utils.percentile(distr, 0.75)
     0.75398454483993305
	
 But also for reading in and working with stallar catalogues:
 
  * :func:`mascara.funcs.utils.loadStellarCatalog`, to read in a stellar catalog
  * :func:`mascara.funcs.utils.cleanupStellarCatalog`, to flag and remove the closest stars in order to avoid blending issues
  * :func:`mascara.funcs.utils.skyQuad`, creates a complicated grid on the image which can be used to build low order transmission maps...
  
 >>> bla = mascara.funcs.utils.loadStellarCatalog(filename='StarCat.fits', Vmag=(4., 7.))
 >>> ra, dec, vmag, bmag, sID, sptype = bla

	
.. _stacker:

The Stacker class
-----------------

The Stacker class is for easy stacking of images taken from the same camera at close observing time. 
The images are 'rotated', practically shifted, from their observing time to a reference time. In order to 
work best, it is recommended that the observing time and reference time do not differ by more than 10min. 
The Stacker class needs as inputs a :class:`mascara.observer.Site` and :class:`mascara.observer.Camera` instance.

>>> from mascara.funcs.utils import Stacker
>>> stacker = Stacker(site, camera)
>>> stacker.image_rotate(image, obstime, reftime, exposuretime=6.4)
>>> binnedimage, nimages = stacker.binImage(datetime.now())

.. _logger:

The multiprocessing logging classes
-----------------------------------

Python 3.2 and higher up versions enable easy logging with multiprocessing. The two following classes are used to reproduce this feature on Python 2.7:

 * :class:`mascara.funcs.utils.QueueHandler`
 * :class:`mascara.funcs.utils.QueueListener`
 
Those are loading for logging in the following way:

    >>> import mascara
    >>> import multiprocessing
    >>> ### Start a queue which allows communication between the processes
    >>> qlog = multiprocessing.Queue()  
    >>> ### Convert the queue to a handling function
    >>> qh = mascara.funcs.utils.QueueHandler(qlog)
    >>> ### Get the logger
    >>> logger = logging.getLogger()
    >>> ### Give the logger the handler
    >>> logger.addHandler(qh)
    >>> ### And now create the function which will read from the queue
    >>> ql = Mmascara.funcs.utils.QueueListener(qlog, handler)
    >>> ql.start()
 
 

	
References/APIs
===============	

Stacking of images
------------------

.. autoclass:: mascara.funcs.utils.Stacker
   :members: __init__, makeGrid, image_rotate, binImage
	
Logging with multiprocessing on Python 2.7:
-------------------------------------------
.. autoclass:: mascara.funcs.utils.QueueHandler
   :members: __init__, enqueue, prepare, emit
.. autoclass:: mascara.funcs.utils.QueueListener
   :members: __init__, dequeue, start, prepare, handle, _monitor, stop   
   
timing -- Calendar conversion and timing functions:
---------------------------------------------------
.. automodule:: mascara.funcs.timing
.. autofunction:: mascara.funcs.timing.checkdiscontinuity
.. autofunction:: mascara.funcs.timing.jd2calendar
.. autofunction:: mascara.funcs.timing.calendar2jd
.. autofunction:: mascara.funcs.timing.jd2epoch
.. autofunction:: mascara.funcs.timing.epoch2jd

mio -- I/O functions:
---------------------

.. automodule:: mascara.funcs.mio
.. autofunction:: mascara.funcs.mio.find_files
.. autofunction:: mascara.funcs.mio.list_directories
.. autofunction:: mascara.funcs.mio.pick_directories
.. autofunction:: mascara.funcs.mio.check_free_space
.. autofunction:: mascara.funcs.mio.make_disk_space
.. autofunction:: mascara.funcs.mio.moveData
.. autofunction:: mascara.funcs.mio.savepng
.. autofunction:: mascara.funcs.mio.saveCompFits


General usage functions:
------------------------
.. automodule:: mascara.funcs.utils
.. autofunction:: mascara.funcs.utils.find_outliers
.. autofunction:: mascara.funcs.utils.robuststd
.. autofunction:: mascara.funcs.utils.percentile
.. autofunction:: mascara.funcs.utils.gaussian
.. autofunction:: mascara.funcs.utils.rebin
.. autofunction:: mascara.funcs.utils.statbin

.. autofunction:: mascara.funcs.utils.loadStellarCatalog
.. autofunction:: mascara.funcs.utils.cleanupStellarCatalog
.. autofunction:: mascara.funcs.utils.skyQuad
.. autofunction:: mascara.funcs.utils.pol2rect
.. autofunction:: mascara.funcs.utils.rect2pol
.. autofunction:: mascara.funcs.utils.rotmat

.. autofunction:: mascara.funcs.utils.makevortex
.. autofunction:: mascara.funcs.utils.match_vortex