diff --git a/.gitignore b/.gitignore index cc914eaebbb175060e01407d2d60c987651a3051..0a15bc0bada254eef4f82108d5680100303dfaec 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,8 @@ /src/Makefile.in /src/fits/Makefile.in /src/wcs/Makefile.in +/src/levmar/Makefile.in +/tests/Makefile.in # configure generated files /Makefile @@ -30,20 +32,26 @@ /man/Makefile /sextractor.spec /src/.deps/ +/src/levmar/.deps/ /src/Makefile /src/fits/.deps/ /src/fits/Makefile /src/wcs/.deps/ /src/wcs/Makefile +/src/levmar/Makefile +/tests/Makefile /stamp-h1 /man/sex.1 # build objects and targets *.o /src/sex +/src/ldactoasc /src/f /src/fits/libfits.a /src/wcs/libwcs_c.a +/src/levmar/liblevmar.a +/doc/build/ # generated by icc with -prof-gen=srcpos option *.dyn diff --git a/doc/src/Using.rst b/doc/src/Using.rst index 524bf0d577f685a3a043339a4f6228ada6f2b2fb..47142defa3e9febae791be3ed7b96c8eba87a930 100644 --- a/doc/src/Using.rst +++ b/doc/src/Using.rst @@ -1,5 +1,7 @@ .. File Using.rst +.. _using-sextractor: + Using SExtractor ================ diff --git a/doc/src/Weighting.rst b/doc/src/Weighting.rst new file mode 100644 index 0000000000000000000000000000000000000000..17cfb4961cdcc2ffb445616264f5b5029f1d639f --- /dev/null +++ b/doc/src/Weighting.rst @@ -0,0 +1,136 @@ +.. File Weighting.rst + +Weighting +========= + +The noise level in astronomical images is often fairly constant, that is, constant values for the gain, +the background noise and the detection thresholds can be used over the whole frame. +Unfortunately in some cases, like strongly vignetted or composited images, this approximation +is no longer good enough. +This leads to detecting clusters of detected noise peaks in the noisiest parts of the image, +or missing obvious objects in the most sensitive ones. +|SExtractor| is able to handle images with variable noise. +It does it through *weight maps*, which are frames having the same size as the images where objects are detected or +measured, and which describe the noise intensity at each pixel. +These maps are internally stored in units of *absolute variance* (in :math:`ADU^2`) +We employ the generic term *weight map* because these maps can also be interpreted as quality index maps: +infinite variance (:math:`\ge 10^{30}` by definition in |SExtractor|) means that the related pixel in the science frame is totally unreliable and should be ignored. +The variance format was adopted as it linearizes most of the operations done over weight maps (see below). + +This means that the noise covariances between pixels are ignored. +Although raw CCD images have essentially white noise, this is not the case for warped images, for which resampling may +induce a strong correlation between neighbouring pixels. +In theory, all non-zero covariances within the geometrical limits of the analysed patterns should be taken into account to derive +thresholds or error estimates. +Fortunately, the correlation length of the noise is often smaller than the patterns to be detected or measured, +and constant over the image. In that case one can apply a simple *fudge factor* to the estimated variance to account for +correlations on small scales. This proves to be a good approximation in general, although it certainly leads to +underestimations for the smallest patterns. + +.. _weigth-map-format: + +Weight-map formats +------------------ + +|SExtractor| accepts in input, and converts to its internal variance format, several types of weight-maps. +This is controlled through the ``WEIGHT_TYPE`` configuration keyword. +Those weight-maps can either be read from a FITS file, whose name is specified by the ``WEIGHT_IMAGE`` keyword, +or computed internally. +Valid ``WEIGHT_TYPE`` values are: + +* ``NONE`` : + No weighting is applied. The related ``WEIGHT_IMAGE`` and ``WEIGHT_THRESH`` (see below) parameters are ignored. + + +* ``BACKGROUND`` : + the science image itself is used to compute internally a variance map (the related ``WEIGHT_IMAGE`` + parameter is ignored). Robust (:math:`3\sigma`-clipped) variance estimates are first computed within the same + background meshes as those described in ?? [#f1]. + The resulting low-resolution variance map is then bicubic-spline-interpolated on the fly to produce + the actual full-size variance map. + A check-image with ``CHECKIMAGE_TYPE`` ``MINIBACK_RMS`` can be requested to examine the low-resolution variance map. + + +* ``MAP_RMS`` : + the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of + absolute standard deviations (in ADUs per pixel). + + +* ``MAP_VAR`` : + the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of + relative variance. A robust scaling to the appropriate absolute level is then performed by comparing this + variance map to an internal, low-resolution, absolute variance map built from the science image itself. + + +* ``MAP_WEIGHT`` : + the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of + relative weights. + The data are converted to variance units (by definition variance :math:`\propto\frac{1}{weight}`), + and scaled as for ``MAP_VAR``. + ``MAP_WEIGHT`` is the most commonly used type of weight-map: a flat-field, for example, is generally + a good approximation to a perfect weight-map. + +Effect of weighting +------------------- + +Weight-maps modify the working of |SExtractor| in the following respects: + +#. Bad pixels are discarded from the background statistics. + If more than :math:`50\%` of the pixels in a background mesh are bad, the local background value and + its standard deviation are replaced by interpolation of the nearest valid meshes. + +#. The detection threshold *t* above the local sky background is adjusted for each pixel *i* + with variance :math:`\sigma^2_i` : :math:`t_i=DETECT\_THRESH\times\sqrt{\sigma^2_i}`, where + ``DETECT_THRESH`` is expressed in units of standard deviations of the background noise. + Pixels with variance above the threshold set with the ``WEIGHT_THRESH`` parameter are therefore simply not detected. + This may result in splitting objects crossed by a group of bad pixels. + Interpolation (see ???) should be used to avoid this problem. + If convolution filtering is applied for detection, the variance map is convolved too. + This yields optimum scaling of the detection threshold in the case where noise is uncorrelated from pixel to pixel. + Non-linear filtering operations (like those offered by artificial retinae) are not affected. + +#. The cleaning process (??) takes into account the exact individual thresholds assigned to each pixel for + deciding about the fate of faint detections. + +#. Error estimates like ``FLUXISO_ERR`` , ``ERRA_IMAGE`` , etc., make use of individual variances too. + Local background-noise standard deviation is simply set to :math:`\sqrt{\sigma^2_i}`. + In addition, if the ``WEIGHT_GAIN`` parameter is set to ``Y`` (which is the default), it is assumed that the + local pixel gain (i.e., the conversion factor from photo-electrons to ADUs) is inversely proportional to + :math:`\sigma^2_i`, its median value over the image being set by the ``GAIN`` configuration parameter. + In other words, it is then supposed that the changes in noise intensities seen over the images are due to gain changes. + This is the most common case: correction for vignetting, or coverage depth. + When this is not the case, for instance when changes are purely dominated by those of the read-out noise, + ``WEIGHT_GAIN`` shall be set to ``N``. + +#. Finally, pixels with weights beyond ``WEIGHT_THRESH`` are treated just like pixels discarded by the masking process (??). + +Combining weight maps +--------------------- + +All the weighting options listed in :ref:`weigth-map-format` can be applied separately to detection and measurement images +(:ref:`using-sextractor`), even if some combinations may not always make sense. +For instance, the following set of configuration lines: + +``WEIGHT_IMAGE`` *rms.fits*, *weight.fits* + +``WEIGHT_TYPE`` ``MAP_RMS``, ``MAP_WEIGHT`` + +will load the FITS file *rms.fits* and use it as an RMS map for adjusting the detection threshold and cleaning, +while the *weight.fits* weight map will only be used for scaling the error estimates on measurements. +This can be done in single- as well as in dual-image mode (:ref:`using-sextractor`). +``WEIGHT_IMAGE`` can be ignored for ``BACKGROUND`` ``WEIGHT_TYPE``. +It is of course possible to use weight-maps for detection or for measurement only. +The following configuration: + +``WEIGHT_IMAGE`` *weight.fits* + +``WEIGHT_TYPE`` ``NONE``, ``MAP_WEIGHT`` + +will apply weighting only for measurements; detection and cleaning operations will remain unaffected. + +.. rubric:: Footnotes + +.. [#f1] The mesh-filtering procedures act on the variance map, too. + +.. include:: keys.rst + diff --git a/doc/src/conf.py b/doc/src/conf.py index f8769eb9d294bdc9dd3a59b6f0ac9517eb3ac709..f367940e3f5f10772205737d52e5d94df6acaf57 100644 --- a/doc/src/conf.py +++ b/doc/src/conf.py @@ -81,7 +81,7 @@ language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: # -today = 'Thu Oct 26 2017' +today = 'Mon Nov 06 2017' # # Else, today_fmt is used as the format for a strftime call. # diff --git a/doc/src/index.rst b/doc/src/index.rst index 1289111aecc9908fe02db460aa5d5ede99856f8a..8ff6dfab8da3648f9aecfa399fbb6cf94c9c1a34 100644 --- a/doc/src/index.rst +++ b/doc/src/index.rst @@ -20,6 +20,7 @@ Contents Using Config Param + Weighting Measurements references diff --git a/man/sex.1 b/man/sex.1 deleted file mode 100644 index 4f21cbe9253c46e8bcb88232f2926dbfc3d181b6..0000000000000000000000000000000000000000 --- a/man/sex.1 +++ /dev/null @@ -1,54 +0,0 @@ -.TH SEXTRACTOR "1" "October 2017" "SExtractor 2.24.2" "User Commands" -.SH NAME -sex \- extract a source catalogue from an astronomical FITS image -.SH SYNOPSIS -.B sex \fIimage\fR [\fI-c configuration-file\fR] -.RS -[\fI-parameter1 value1 -parameter2 value2 ...\fR] -.RE -.TP -.B sex \fIimage\_detect\fR \fIimage_measure\fR [\fI-c configuration-file\fR] -.RS -[\fI-parameter1 value1 -parameter2 value2 ...\fR] -.RE -.TP -.B sex \fI-d\fR -.SH DESCRIPTION -SExtractor is a program that builds a catalogue of objects from an astronomical -image. Although it is particularly oriented towards reduction of large scale -galaxy-survey data, it performs rather well on moderately crowded star fields. -.RE -See http://astromatic.net/software/sextractor for more details. -.SS "Operation modes:" -.TP -\fB\-h\fR, \fB\-\-help\fR -print this help, then exit -.TP -\fB\-V\fR, \fB\-\-version\fR -print version number, then exit -.TP -\fB\-d\fR, \fB\-\-dump\fR -dump a default configuration file -.SH MANUAL -The full documentation for -.B SExtractor -is maintained as a PDF manual available at -.B http://astromatic.net/software/sextractor -.SH "REPORTING BUGS" -Please report bugs to the SExtractor software forum at -http://astromatic.net/forum. -.SH "SEE ALSO" -.BR -eye(1), missfits(1), psfex(1), scamp(1), sky(1), stiff(1), stuff(1), swarp (1), -ww(1) -.SH AUTHORS -Written by Emmanuel Bertin (Institut d'Astrophysique de Paris) -.PP -.SH LICENSE -SExtractor is licensed under the terms of the General Public License (GPL), -version 3. -.SH COPYRIGHT -.PP -This is free software; see the source for copying conditions. There is NO -warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -