@@ -12,95 +14,68 @@ is not necessary to operate the software.
...
@@ -12,95 +14,68 @@ is not necessary to operate the software.
Memory requirements depend on the size of the images to be processed.
Memory requirements depend on the size of the images to be processed.
Processing a single image should typically require about 100MB of memory.
Processing a single image should typically require about 100MB of memory.
For large images (hundreds of Mpixels or more), or in double-image / weighted
For large images (hundreds of Mpixels or more), or in double-image / weighted mode, |SExtractor|'s memory footprint should be around 500MB, and up to 2GB in the worst cases.
mode, |SExtractor|'s memory footprint should be around 500MB, and up to 2GB in
Swap-space can be put to contribution, although a strong performance hit is to be expected.
the worst cases. Swap-space can be put to contribution, although a strong
performance hit is to be expected.
Obtaining |SExtractor|
Obtaining |SExtractor|
----------------------
----------------------
For Linux users, the simplest way to have |SExtractor| up and running is to
For Linux users, the simplest way to have |SExtractor| up and running is to install the standard binary package the comes with your Linux distribution.
install the standard binary package the comes with your Linux distribution.
Run, e.g., ``apt-get sextractor`` (on Debian) or ``dnf sextractor`` (Fedora) as root and |SExtractor|, as well as all its dependencies, will automatically be installed.
Run, e.g., ``apt-get sextractor`` (on Debian) or ``dnf sextractor`` (Fedora) as
If you decided to install the package this way you may skip the following and move straight to the :ref:`next section <Using Sextractor>`.
root and |SExtractor|, as well as all its dependencies, will automatically be
installed. If you decided to install the package this way you may skip the
However if |SExtractor| is not available in your distribution, or to obtain the most recent version, the |SExtractor| source package can be downloaded from `the official GitHub repository <https://github.com/astromatic/sextractor>`_ .
following and move straight to the :ref:`next section <Using Sextractor>`.
One may choose `one of the stable releases <https://github.com/astromatic/sextractor/releases>`_, or for the fearless, `a copy of the current master development branch <https://github.com/astromatic/sextractor/archive/master.zip>`_.
However if |SExtractor| is not available in your distribution, or to obtain the
most recent version, the |SExtractor| source package can be downloaded from
`the official GitHub repository <https://github.com/astromatic/sextractor>`_ .
One may choose `one of the stable releases
<https://github.com/astromatic/sextractor/releases>`_, or for the fearless,
|SExtractor| has been developed on `GNU/Linux <http://en.wikipedia.org/wiki/Linux>`_ machines and should compile on any `POSIX <http://en.wikipedia.org/wiki/POSIX>`_-compliant system (this includes |OSX|_ and `Cygwin <http://www.cygwin.com>`_ on |Windows|_, at the price of some difficulties with the configuration), provided that the *development* packages of the following libraries have been installed:
`GNU/Linux <http://en.wikipedia.org/wiki/Linux>`_ machines and should compile on
any `POSIX <http://en.wikipedia.org/wiki/POSIX>`_-compliant system (this
includes |OSX|_ and `Cygwin <http://www.cygwin.com>`_ on |Windows|_, at the
price of some difficulties with the configuration), provided that the
*development* packages of the following libraries have been installed:
* |ATLAS|_ V3.6 and above [#atlas_install]_,
* |ATLAS|_ V3.6 and above [#atlas_install]_,
* |FFTw|_ V3.0 and above [#fftw_install]_,
* |FFTw|_ V3.0 and above [#fftw_install]_,
On Fedora/Redhat distributions for instance, the development packages above are
On Fedora/Redhat distributions for instance, the development packages above are available as ``atlas-devel`` and ``fftw-devel``.
available as ``atlas-devel`` and ``fftw-devel``. Note that |ATLAS| and |FFTw|
Note that |ATLAS| and |FFTw| are not necessary if |SExtractor| is linked with |Intel|'s |MKL|_ library.
are not necessary if |SExtractor| is linked with |Intel|'s |MKL|_ library.
Installation
Installation
------------
------------
To install from the |GitHub| source package, you must first uncompress the
To install from the |GitHub| source package, you must first uncompress the archive:
archive:
.. code-block:: console
.. code-block:: console
$ unzip sextractor-<version>.zip
$ unzip sextractor-<version>.zip
A new directory called :file:`sextractor-<version>` should now appear at the
A new directory called :file:`sextractor-<version>` should now appear at the current location on your disk.
current location on your disk. Enter the directory and generate the files
Enter the directory and generate the files required by the `autotools <http://en.wikipedia.org/wiki/GNU_Build_System>`_, which the package relies on:
required by the `autotools <http://en.wikipedia.org/wiki/GNU_Build_System>`_,
which the package relies on:
.. code-block:: console
.. code-block:: console
$ cd sextractor-<version>
$ cd sextractor-<version>
$ sh autogen.sh
$ sh autogen.sh
A :program:`configure` script is created. This script has many options, which
A :program:`configure` script is created.
may be listed with the ``--help`` option:
This script has many options, which may be listed with the ``--help`` option:
.. code-block:: console
.. code-block:: console
$ ./configure --help
$ ./configure --help
No options are required for compiling with the default GNU C compiler
No options are required for compiling with the default GNU C compiler (:program:`gcc`) if all the required libraries are installed at their default locations:
(:program:`gcc`) if all the required libraries are installed at their default
locations:
.. code-block:: console
.. code-block:: console
$ ./configure
$ ./configure
Compared to :program:`gcc` and the librairies above, the combination of the
Compared to :program:`gcc` and the librairies above, the combination of the |Intel| compiler (:program:`icc`) and the |MKL|_ libraries can give the |SExtractor| executable a strong boost in performance, thanks to better vectorized code.
|Intel| compiler (:program:`icc`) and the |MKL|_ libraries can give the
If :program:`icc` and the |MKL| are installed on your system [#geticc]_ , you can take advantage of them using
|SExtractor| executable a strong boost in performance, thanks to better
vectorized code. If :program:`icc` and the |MKL| are installed on your system
[#geticc]_ , you can take advantage of them using
.. code-block:: console
.. code-block:: console
$ ./configure --enable-mkl
$ ./configure --enable-mkl
Additionally, if the |SExtractor| binary is to be run on a different machine
Additionally, if the |SExtractor| binary is to be run on a different machine that does not have :program:`icc` and the |MKL| installed (e.g., a cluster computing node), you must configure a partially statically linked executable using
that does not have :program:`icc` and the |MKL| installed (e.g., a cluster
computing node), you must configure a partially statically linked executable
using
.. code-block:: console
.. code-block:: console
...
@@ -112,34 +87,22 @@ In all cases, |SExtractor| can now be compiled with
...
@@ -112,34 +87,22 @@ In all cases, |SExtractor| can now be compiled with
$ make -j
$ make -j
An :file:`src/sex` executable is created. For system-wide installation, run
An :file:`src/sex` executable is created. For system-wide installation, run the usual
the usual
.. code-block:: console
.. code-block:: console
$ sudo make install
$ sudo make install
You may now check that the software is properly installed by simply
You may now check that the software is properly installed by simply typing in your shell:
typing in your shell:
.. code-block:: console
.. code-block:: console
$ sex
$ sex
which will return the version number and other basic information (note that
which will return the version number and other basic information (note that some shells require the :program:`rehash` command to be run before making a freshly installed executable accessible in the execution path).
some shells require the :program:`rehash` command to be run before making a
freshly installed executable accessible in the execution path).
.. [#mac_install] Mac OS X |.dmg|_ packages should be available soon.
.. [#mac_install] Mac OS X |.dmg|_ packages should be available soon.
.. [#atlas_install] Use the ``--with-atlas`` and/or
.. [#atlas_install] Use the ``--with-atlas`` and/or ``--with-atlas-incdir`` options of the |SExtractor| :command:`configure` script to specify the |ATLAS| library and include paths if |ATLAS| files are installed at unusual locations.
``--with-atlas-incdir`` options of the |SExtractor| :command:`configure`
.. [#fftw_install] Make sure that |FFTW| has been compiled with :command:`configure` options ``--enable-threads --enable-float``.
script to specify the |ATLAS| library and include paths if |ATLAS| files are
.. [#geticc] The Linux versions of the |Intel| compiler and |MKL| are `available for free to academic researchers, students, educators and open source contributors <http://software.intel.com/qualify-for-free-software>`_.
installed at unusual locations.
.. [#fftw_install] Make sure that |FFTW| has been compiled with
|SExtractor|_ (Source-Extractor) is a program that builds a catalog
|SExtractor|_ (Source-Extractor) is a program that builds a catalog of objects from an astronomical image. It is particularly oriented towards the reduction of large scale galaxy-survey data, but it also performs well on moderately crowded star fields. Its main features are:
of objects from an astronomical image. It is particularly oriented
towards the reduction of large scale galaxy-survey data, but it also performs
well on moderately crowded star fields. Its main features are:
* Support for multi-extension FITS (|MEF|_)
* Support for multi-extension FITS (|MEF|_)
* Speed: up to about 50 Mpixel/s or 10,000 sources/s with a 3 GHz processor
* Speed: up to about 50 Mpixel/s or 10,000 sources/s with a 3 GHz processor
...
@@ -22,8 +21,5 @@ well on moderately crowded star fields. Its main features are:
...
@@ -22,8 +21,5 @@ well on moderately crowded star fields. Its main features are:
* Built-in catalog cross-identification.
* Built-in catalog cross-identification.
* Special mode for photographic scans.
* Special mode for photographic scans.
* |XML|_ |VOTable|_-compliant catalog output.
* |XML|_ |VOTable|_-compliant catalog output.
* |XSLT|_ filter sheet provided for convenient access to metadata from a
* |XSLT|_ filter sheet provided for convenient access to metadata from a regular web browser.
|SExtractor|_ *is free software: you can redistribute it and/or modify it under
|SExtractor|_ *is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.*
the terms of the GNU General Public License as published by the Free
|SExtractor| *is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Software Foundation, either version 3 of the License, or (at your
See the GNU General Public License for more details.
option) any later version.* |SExtractor| *is distributed in the hope that it will
You should have received a copy of the GNU General Public License along with* |SExtractor|. *If not, see* `www.gnu.org/licenses/ <http://www.gnu.org/licences/>`_.
be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details. You should have received a copy
of the GNU General Public License along with* |SExtractor|. *If not, see*
Once sources have been detected and deblended, they enter the measurement phase. |SExtractor| performs three categories of measurements: isophotal, full, and model-fitting.
.. include:: global.rst
Once sources have been detected and deblended, they enter the measurement phase.
|SExtractor| performs three categories of measurements: isophotal, full, and model-fitting.
.. _isophotal_measurements:
.. _isophotal_measurements:
Isophotal
Isophotal
Measurements are made on the isophotal object footprints, which are defined on the filtered detection image. Only pixels with values above the threshold set with ``ANALYSIS_THRESH`` are considered [#thresh]_, which makes the analysis extremely fast, but obviously strongly dependent on the threshold itself. This is an issue particularly when the amplitude of the bakground noise varies over the image. Many of the isophotal measurements (e.g., ``X_IMAGE``, ``Y_IMAGE``, ``FLUX_ISO``) are necessary for the internal operations of |SExtractor| and are therefore executed even if they are not requested.
Measurements are made on the isophotal object footprints, which are defined on the filtered detection image. Only pixels with values above the threshold set with ``ANALYSIS_THRESH`` are considered [#thresh]_, which makes the analysis extremely fast, but obviously strongly dependent on the threshold itself. This is an issue particularly when the amplitude of the bakground noise varies over the image. Many of the isophotal measurements (e.g., :param:`X_IMAGE`, :param:`Y_IMAGE`, :param:`FLUX_ISO`) are necessary for the internal operations of |SExtractor| and are therefore executed even if they are not requested.
Full
Full
Measurements have access to all pixels of the image. These measurements are generally more sophisticated, less affected by variable biases induced by the detection threshold, and still reasonably fast. They are done at a later stage of the processing, after CLEANing and MASKing.
Measurements have access to all pixels of the image. These measurements are generally more sophisticated, less affected by variable biases induced by the detection threshold, and still reasonably fast. They are done at a later stage of the processing, after CLEANing and MASKing.
...
@@ -23,5 +26,4 @@ Model-fitting
...
@@ -23,5 +26,4 @@ Model-fitting
.. [#thresh] Pixel values also have to exceed the local detection threshold set with ``DETECT_THRESH``.
.. [#thresh] Pixel values also have to exceed the local detection threshold set with ``DETECT_THRESH``.
.. [#psf_models] PSF models be computed using the |PSFEx|_ package.
.. [#psf_models] PSF models be computed using the |PSFEx|_ package.
The following parameters are derived from the spatial distribution
The following parameters are derived from the spatial distribution :math:`\cal S` of pixels detected above the analysis threshold (see :ref:`description<isophotal_measurements>`).
:math:`\cal S` of pixels detected above the analysis threshold (see :ref:`description<isophotal_measurements>`).
.. important::
.. important::
Unless otherwise noted, pixel values :math:`I_i` are taken from the (filtered) detection image.
Unless otherwise noted, pixel values :math:`I_i` are taken from the (filtered) detection image.
.. note::
.. note::
Unless otherwise noted, all parameter names given below are only prefixes. They must be followed by _IMAGE if the results shall be expressed in pixel coordinates or _WORLD, _SKY, _J2000 or _B1950 for |WCS|_ coordinates (see :ref:`coord_suffix`).
Unless otherwise noted, all parameter names given below are only prefixes. They must be followed by _IMAGE if the results shall be expressed in pixel coordinates or :param:`_WORLD`, :param:`_SKY, :param:`_J2000` or :param:`_B1950` for |WCS|_ coordinates (see :ref:`coord_suffix`).
These coordinates define two corners of a rectangle which encloses the
These coordinates define two corners of a rectangle which encloses the detected object:
detected object:
.. math::
.. math::
:label: xminymax
:label: xminymax
...
@@ -30,16 +30,12 @@ detected object:
...
@@ -30,16 +30,12 @@ detected object:
{\tt YMAX} & = & \max_{i \in {\cal S}} y_i,
{\tt YMAX} & = & \max_{i \in {\cal S}} y_i,
\end{aligned}
\end{aligned}
where :math:`x_i` and :math:`y_i` are respectively the x-coordinate and
where :math:`x_i` and :math:`y_i` are respectively the x-coordinate and y-coordinate of pixel :math:`i`.
y-coordinate of pixel :math:`i`.
Barycenter: X, Y
Barycenter: :param:`X`, :param:`Y`
----------------
----------------------------------
Barycenter coordinates generally define the position of the “center” of
Barycenter coordinates generally define the position of the “center” of a source, although this definition can be inadequate or inaccurate if its spatial profile shows a strong skewness or very large wings. X and Y are simply computed as the first order moments of the profile:
a source, although this definition can be inadequate or inaccurate if
its spatial profile shows a strong skewness or very large wings. X and Y
are simply computed as the first order moments of the profile:
.. math::
.. math::
:label: xy
:label: xy
...
@@ -51,28 +47,17 @@ are simply computed as the first order moments of the profile:
...
@@ -51,28 +47,17 @@ are simply computed as the first order moments of the profile:
It is sometimes useful to have the position XPEAK,YPEAK of the pixel
It is sometimes useful to have the position :param:`XPEAK`, :param:`YPEAK` of the pixel with maximum intensity in a detected object, for instance when working with likelihood maps, or when searching for artifacts. For better robustness, PEAK coordinates are computed on *filtered* profiles if available. On symmetrical profiles, PEAK positions and barycenters coincide within a fraction of pixel (:param:`XPEAK` and :param:`YPEAK` coordinates are quantized by steps of 1 pixel, hence :param:`XPEAK_IMAGE` and :param:`YPEAK_IMAGE` are integers). This is no longer true for skewed profiles, therefore a simple comparison between PEAK and barycenter coordinates can be used to identify asymmetrical objects on well-sampled images.
with maximum intensity in a detected object, for instance when working
with likelihood maps, or when searching for artifacts. For better
robustness, PEAK coordinates are computed on *filtered* profiles if
available. On symmetrical profiles, PEAK positions and barycenters
coincide within a fraction of pixel (XPEAK and YPEAK coordinates are
quantized by steps of 1 pixel, thus XPEAK_IMAGE and YPEAK_IMAGE are
integers). This is no longer true for skewed profiles, therefore a
simple comparison between PEAK and barycenter coordinates can be used to
identify asymmetrical objects on well-sampled images.
2nd order moments: X2, Y2, XY
2nd order moments: :param:`X2`, :param:`Y2`, :param:`XY`
These parameters are intended to describe the detected object as an
These parameters are intended to describe the detected object as an elliptical shape. :param:`A` and :param:`B` are the lengths of the semi-major and semi-minor axes, respectively.
elliptical shape. A and B are its semi-major and semi-minor axis
More precisely, they represent the maximum and minimum spatial dispersion of the object profile along any direction.
lengths, respectively. More precisely, they represent the maximum and
:param:`THETA` is the position-angle of the :param:`A` axis relative to the first image axis.
minimum spatial dispersion of the object profile along any direction.
It is counted positive in the direction of the second axis.
THETA is the position-angle of the A axis relative to the first image
axis. It is counted positive in the direction of the second axis. Here
is how they are computed:
2nd-order moments can easily be expressed in a referential rotated from
Here is how shape parameters are computed. 2nd-order moments can easily be expressed in a referential rotated from the :math:`x,y` image coordinate system by an angle +\ :math:`\theta`:
the :math:`x,y` image coordinate system by an angle +\ :math:`\theta`:
.. math::
.. math::
:label: varproj
:label: varproj
...
@@ -122,8 +101,7 @@ the :math:`x,y` image coordinate system by an angle +\ :math:`\theta`:
...
@@ -122,8 +101,7 @@ the :math:`x,y` image coordinate system by an angle +\ :math:`\theta`:
\sin^2\theta)\:\overline{xy}.
\sin^2\theta)\:\overline{xy}.
\end{array}
\end{array}
One can find interesting angles :math:`\theta_0` for which the variance
One can find the angle(s) :math:`\theta_0` for which the variance is minimized (or maximized) along :math:`x_{\theta}`:
is minimized (or maximized) along :math:`x_{\theta}`:
.. math::
.. math::
:label: theta0
:label: theta0
...
@@ -145,24 +123,22 @@ If :math:`\overline{y^2} \neq \overline{x^2}`, this implies:
...
@@ -145,24 +123,22 @@ If :math:`\overline{y^2} \neq \overline{x^2}`, this implies:
A and B can be computed directly from the 2nd-order moments, using the
:param:`A` and :param:`B` can be computed directly from the 2nd-order moments, using the following equations derived from :eq:`varproj` after some algebra:
following equations derived from :eq:`varproj` after some algebra:
.. math::
.. math::
:label: aimage_2
:label: aimage_2
...
@@ -171,26 +147,21 @@ following equations derived from :eq:`varproj` after some algebra:
...
@@ -171,26 +147,21 @@ following equations derived from :eq:`varproj` after some algebra:
Note that A and B are exactly halves the :math:`a` and :math:`b`
Note that :param:`A` and :param:`B` are exactly halves the :math:`a` and :math:`b` parameters computed by the COSMOS image analyser :cite:`1980SPIE_264_208S`.
parameters computed by the COSMOS image analyser :cite:`1980SPIE_264_208S`.
Actually, :math:`a` and :math:`b` are defined in :cite:`1980SPIE_264_208S` as the semi-major and semi-minor axes of an elliptical shape with constant surface brightness, which would have the same 2nd-order moments as the analyzed object.
Actually, :math:`a` and :math:`b` are defined in :cite:`1980SPIE_264_208S`
as the semi-major and semi-minor axes of an elliptical shape with constant
surface brightness, which would have the same 2nd-order moments as the
A, B and THETA are not very convenient to use when, for instance, one
:param:`A`, :param:`B` and :param:`THETA` are not very convenient to use when, for instance, one wants to know if a particular |SExtractor| detection extends over some
wants to know if a particular |SExtractor| detection extends over some
position.
position. For this kind of application, three other ellipse parameters
For this kind of application, three other ellipse parameters are provided; :param:`CXX`, :param:`CYY`, and :param:`CXY`.
are provided; CXX, CYY and CXY. They do nothing more than describing the
They do nothing more than describing the same ellipse, but in a different way: the elliptical shape associated to a detection is now parameterized as
same ellipse, but in a different way: the elliptical shape associated to
a detection is now parameterized as
.. math::
.. math::
:label: ellipse
:label: ellipse
...
@@ -198,8 +169,8 @@ a detection is now parameterized as
...
@@ -198,8 +169,8 @@ a detection is now parameterized as
Uncertainties on the position of the barycenter can be estimated using
Uncertainties on the position of the barycenter can be estimated using
photon statistics. In practice, such estimates are a lower-value of the full
photon statistics.
uncertainties because they do not include, for instance, the contribution of
In practice, such estimates are a lower-value of the full uncertainties because they do not include, for instance, the contribution of detection biases or contamination by neighbors.
detection biases or contamination by neighbors. Furthermore, |SExtractor| does
Furthermore, |SExtractor| does not currently take into account possible correlations of the noise between adjacent pixels. Hence variances simply write:
not currently take into account possible correlations of the noise between adjacent
where :math:`{\sigma_B}_i` is the local background noise and
where :math:`{\sigma_B}_i` is the local background noise and :math:`g_i` the local gain — conversion factor — for pixel :math:`i` (see :ref:`effect_of_weighting` for more details). Semi-major axis :param:`ERRA`, semi-minor axis :param:`ERRB`, and position angle :param:`ERRTHETA` of the :math:`1\sigma` position error ellipse are computed from the covariance matrix exactly like for :ref:`basic shape parameters<shape_iso_def>`:
:math:`g_i` the local gain — conversion factor — for pixel :math:`i`
(see :ref:`effect_of_weighting` for more details). Semi-major axis ERRA, semi-minor
axis ERRB, and position angle ERRTHETA of the :math:`1\sigma` position
error ellipse are computed from the covariance matrix exactly like in
[chap:abtheta] for shape parameters:
.. math::
.. math::
:label: errabtheta
:label: errabtheta
...
@@ -285,7 +252,8 @@ error ellipse are computed from the covariance matrix exactly like in
...
@@ -285,7 +252,8 @@ error ellipse are computed from the covariance matrix exactly like in
Apart from the mathematical singularities that can be found in some of
Apart from the mathematical singularities that can be found in some of the above equations describing shape parameters (and which |SExtractor| is able to handle, of course), some detections with very specific shapes may yield unphysical parameters, namely null values for :param:`B`, :param:`ERRB`, or even :param:`A` and :param:`ERRA`.
the above equations describing shape parameters (and which |SExtractor|
Such detections include single-pixel objects and horizontal, vertical or diagonal lines which are 1-pixel wide. They will generally originate from glitches; but strongly undersampled and/or low S/N genuine sources may also produce such shapes.
handles, of course), some detections with very specific shapes may yield
quite unphysical parameters, namely null values for B, ERRB, or even A
and ERRA. Such detections include single-pixel objects and horizontal,
vertical or diagonal lines which are 1-pixel wide. They will generally
originate from glitches; but very undersampled and/or low S/N genuine
sources may also produce such shapes.
For basic shape parameters, the following convention was adopted: if the
For basic shape parameters, the following convention was adopted: if the
light distribution of the object falls on one single pixel, or lies on a
light distribution of the object falls on one single pixel, or lies on a
...
@@ -328,16 +291,10 @@ sufficiently thin line of pixels, which we translate mathematically by
...
@@ -328,16 +291,10 @@ sufficiently thin line of pixels, which we translate mathematically by
then :math:`\overline{x^2}` and :math:`\overline{y^2}` are incremented
then :math:`\overline{x^2}` and :math:`\overline{y^2}` are incremented by :math:`\rho`. |SExtractor| sets :math:`\rho=1/12`, which is the variance of a 1-dimensional top-hat distribution with unit width.
by :math:`\rho`. |SExtractor| sets :math:`\rho=1/12`, which is the
Therefore :math:`1/\sqrt{12}` represents the typical minor-axis values assigned (in pixels units) to undersampled sources in |SExtractor|.
variance of a 1-dimensional top-hat distribution with unit width.
Therefore :math:`1/\sqrt{12}` represents the typical minor-axis values
assigned (in pixels units) to undersampled sources in |SExtractor|.
Positional errors are more difficult to handle, as objects with very
Positional errors are more difficult to handle, as objects with very high signal-to-noise can yield extremely small position uncertainties, just like singular profiles do. Therefore |SExtractor| first checks that :eq:`singutest` is true. If this is the case, a new test is conducted:
high signal-to-noise can yield extremely small position uncertainties,
just like singular profiles do. Therefore |SExtractor| first checks that
:eq:`singutest` is true. If this is the case, a new test is conducted:
.. math::
.. math::
:label: singutest2
:label: singutest2
...
@@ -345,12 +302,8 @@ just like singular profiles do. Therefore |SExtractor| first checks that
...
@@ -345,12 +302,8 @@ just like singular profiles do. Therefore |SExtractor| first checks that
Measurements performed through a *window* function (an *envelope*) do not have many of the drawbacks of isophotal measurements. |SExtractor| implements “windowed” versions for most
Measurements performed through a *window* function (an *envelope*) do not have many of the drawbacks of isophotal measurements. |SExtractor| implements “windowed” versions for most
of the measurements described in the :ref:`previous section<position_iso>`:
of the measurements described in the :ref:`previous section<position_iso_def>`:
.. note::
.. note::
Unless otherwise noted, all parameter names given below are only prefixes. They must be followed by _IMAGE if the results shall be expressed in pixel coordinates or _WORLD, _SKY, _J2000 or _B1950 for |WCS|_ coordinates (see :ref:`coord_suffix`).
Unless otherwise noted, all parameter names given below are only prefixes. They must be followed by _IMAGE if the results shall be expressed in pixel coordinates or :param:`_WORLD`, :param:`_SKY`, :param:`_J2000` or :param:`_B1950` for |WCS|_ coordinates (see :ref:`coord_suffix`).
The computations involved are roughly the same except that the pixel
The computations involved are roughly the same except that the pixel values are integrated within a circular Gaussian window as opposed to
values are integrated within a circular Gaussian window as opposed to
the object’s isophotal footprint.
the object’s isophotal footprint. The Gaussian window is scaled to each
The Gaussian window is scaled to each object; the Gaussian FWHM is the diameter of the disk that contains half of the object flux (:math:`d_{50}`).
object; its FWHM is the diameter of the disk that contains half of the
Note that in double-image mode (§[chap:using]) the window is scaled based on the *measurement* image.
object flux (:math:`d_{50}`). Note that in double-image mode
(§[chap:using]) the window is scaled based on the *measurement* image.
.. _xywin:
.. _xywin:
Windowed centroid: XWIN, YWIN
Windowed centroid: :param:`XWIN`, :param:`YWIN`
-----------------------------
-----------------------------------------------
This is an iterative process. The computation starts by initializing the
This is an iterative process.
windowed centroid coordinates :math:`\overline{x_{\tt WIN}}^{(0)}` and
Computation starts by initializing the windowed centroid coordinates :math:`\overline{x_{\tt WIN}}^{(0)}` and :math:`\overline{y_{\tt WIN}}^{(0)}` to their basic :math:`\overline{x}` and :math:`\overline{y}` isophotal equivalents, respectively.
:math:`\overline{y_{\tt WIN}}^{(0)}` to their basic :math:`\overline{x}`
Then at each iteration :math:`t`, :math:`\overline{x_{\tt WIN}}` and :math:`\overline{y_{\tt WIN}}` are refined using:
and :math:`\overline{y}` isophotal equivalents, respectively. Then at
each iteration :math:`t`, :math:`\overline{x_{\tt WIN}}` and
:math:`\overline{y_{\tt WIN}}` are refined using:
.. math::
.. math::
:label: xywin
:label: xywin
...
@@ -54,7 +51,8 @@ each iteration :math:`t`, :math:`\overline{x_{\tt WIN}}` and
...
@@ -54,7 +51,8 @@ each iteration :math:`t`, :math:`\overline{x_{\tt WIN}}` and
and :math:`s_{\tt WIN} = d_{50} / \sqrt{8 \ln 2}`. The process stops
and :math:`s_{\tt WIN} = d_{50} / \sqrt{8 \ln 2}`.
when the change in position between two iterations is less than
Process stops when the change in position between two iterations is less than :math:`2\times10^{-4}` pixel, a condition which is achieved in about 3 to 5 iterations in practice.
:math:`2\times10^{-4}` pixel, a condition which is generally achieved in
about 3 to 5 iterations.
Although they are slower, it is recommended to use whenever possible windowed position parameters instead of their isophotal equivalents; the measurements they provide are generally much more accurate (:numref:`fig_xwinprec`).
The centroiding accuracy of :param:`XWIN_IMAGE` and :param:`YWIN_IMAGE` is actually very close to that of PSF-fitting on focused and properly sampled star images. Windowed measurements can also be applied to galaxies.
Although the iterative nature of the processing slows down the
It has been verified that for isolated objects with Gaussian-like profiles, their accuracy is close to the theoretical limit set by image noise [#win_accuracy]_.
processing , it is recommended to use whenever possible windowed
parameters instead of their isophotal equivalents, since the
measurements they provide are much more precise (:numref:`fig_xwinprec`).
The precision in centroiding offered by XWIN_IMAGE and YWIN_IMAGE is
actually very close to that of PSF-fitting on focused and properly
sampled star images, and can also be applied to galaxies. It has been
verified that for isolated, Gaussian-like PSFs, its accuracy is close to
the theoretical limit set by image noise [#win_accuracy]_.
.. _fig_xwinprec:
.. _fig_xwinprec:
...
@@ -92,15 +82,12 @@ the theoretical limit set by image noise [#win_accuracy]_.
...
@@ -92,15 +82,12 @@ the theoretical limit set by image noise [#win_accuracy]_.
:figwidth: 100%
:figwidth: 100%
:align: center
:align: center
Comparison between isophotal and windowed centroid measurement residuals on
Comparison between isophotal and windowed centroid measurement residuals on simulated, background noise-limited images.
simulated, background noise-limited images.
*Left*: histogram of the difference between :param:`X_IMAGE` and the true centroid in x.
*Left*: histogram of the difference between X_IMAGE and the true centroid
*Right*: histogram of the difference between :param:`XWIN_IMAGE` and the true centroid in x.
in x.
*Right*: histogram of the difference between XWIN_IMAGE and the true centroid
in x.
Windowed 2nd order moments: X2, Y2, XY
Windowed 2nd order moments: :param:`X2`, :param:`Y2`, :param:`XY`
They are computed from the windowed 2nd order moments exactly the same
Ellipse parameters are computed from the windowed 2nd order moments exactly the same way as in :eq:`ellipse_2` describing the :ref:`isophotal ellipse parameters<ellipse_iso_def>`:
way as in :eq:`ellipse_2` describing the :ref:`isophotal ellipse parameters<ellipse_iso>`:
.. math::
.. math::
:label: ellipsewin_2
:label: ellipsewin_2
...
@@ -158,15 +143,14 @@ way as in :eq:`ellipse_2` describing the :ref:`isophotal ellipse parameters<elli
...
@@ -158,15 +143,14 @@ way as in :eq:`ellipse_2` describing the :ref:`isophotal ellipse parameters<elli
The semi-major axis ERRAWIN, semi-minor axis ERRBWIN, and position angle
Semi-major axis :param:`ERRAWIN`, semi-minor axis :param:`ERRBWIN`, and position angle :param:`ERRTHETAWIN` of the :math:`1\sigma` position error ellipse are computed from the covariance matrix elements :math:`{\rm var}(\overline{x_{\tt WIN}})`, :math:`{\rm var}(\overline{y_{\tt WIN}})`, :math:`{\rm cov}(\overline{x_{\tt WIN}},\overline{y_{\tt WIN}})`, similarly to the :ref:`isophotal error ellipse <poserr>`:
ERRTHETAWIN of the :math:`1\sigma` position error ellipse are computed
The noise level in astronomical images is often fairly constant, that is, constant values for the gain,
The noise level in astronomical images is often fairly constant, that is, constant values for the gain, the background noise and the detection thresholds can be used over the whole frame.
the background noise and the detection thresholds can be used over the whole frame.
Unfortunately in some cases, like strongly vignetted or composited images, this approximation is no longer good enough.
Unfortunately in some cases, like strongly vignetted or composited images, this approximation
This leads to detecting clusters of detected noise peaks in the noisiest parts of the image, or missing obvious objects in the most sensitive ones.
is no longer good enough.
This leads to detecting clusters of detected noise peaks in the noisiest parts of the image,
or missing obvious objects in the most sensitive ones.
|SExtractor| is able to handle images with variable noise.
|SExtractor| is able to handle images with variable noise.
It does it through *weight maps*, which are frames having the same size as the images where objects are detected or
It does it through *weight maps*, which are frames having the same size as the images where objects are detected or measured, and which describe the noise intensity at each pixel.
measured, and which describe the noise intensity at each pixel.
These maps are internally stored in units of *absolute variance* (in :math:`ADU^2`)
These maps are internally stored in units of *absolute variance* (in :math:`ADU^2`)
We employ the generic term *weight map* because these maps can also be interpreted as quality index maps:
We employ the generic term *weight map* because these maps can also be interpreted as quality index maps: infinite variance (:math:`\ge 10^{30}` by definition in |SExtractor|) means that the related pixel in the science frame is totally unreliable and should be ignored.
infinite variance (:math:`\ge 10^{30}` by definition in |SExtractor|) means that the related pixel in the science frame is totally unreliable and should be ignored.
The variance format was adopted as it linearizes most of the operations done over weight maps (see below).
The variance format was adopted as it linearizes most of the operations done over weight maps (see below).
This means that the noise covariances between pixels are ignored.
This means that the noise covariances between pixels are ignored. Although raw CCD images have essentially white noise, this is not the case for warped images, for which resampling may induce a strong correlation between neighboring pixels.
Although raw CCD images have essentially white noise, this is not the case for warped images, for which resampling may
In theory, all non-zero covariances within the geometrical limits of the analysed patterns should be taken into account to derive thresholds or error estimates.
induce a strong correlation between neighbouring pixels.
Fortunately, the correlation length of the noise is often smaller than the patterns to be detected or measured, and constant over the image.
In theory, all non-zero covariances within the geometrical limits of the analysed patterns should be taken into account to derive
In that case one can apply a simple *fudge factor* to the estimated variance to account for correlations on small scales.
thresholds or error estimates.
This proves to be a good approximation in general, although it certainly leads to underestimations for the smallest patterns.
Fortunately, the correlation length of the noise is often smaller than the patterns to be detected or measured,
and constant over the image. In that case one can apply a simple *fudge factor* to the estimated variance to account for
correlations on small scales. This proves to be a good approximation in general, although it certainly leads to
underestimations for the smallest patterns.
.. _weigth-map-format:
.. _weigth-map-format:
...
@@ -34,41 +27,34 @@ Weight-map formats
...
@@ -34,41 +27,34 @@ Weight-map formats
|SExtractor| accepts in input, and converts to its internal variance format, several types of weight-maps.
|SExtractor| accepts in input, and converts to its internal variance format, several types of weight-maps.
This is controlled through the ``WEIGHT_TYPE`` configuration keyword.
This is controlled through the ``WEIGHT_TYPE`` configuration keyword.
Those weight-maps can either be read from a FITS file, whose name is specified by the ``WEIGHT_IMAGE`` keyword,
Those weight-maps can either be read from a FITS file, whose name is specified by the ``WEIGHT_IMAGE`` keyword, or computed internally.
or computed internally.
Valid ``WEIGHT_TYPE`` values are:
Valid ``WEIGHT_TYPE`` values are:
* ``NONE`` :
* ``NONE`` :
No weighting is applied. The related ``WEIGHT_IMAGE`` and ``WEIGHT_THRESH`` (see below) parameters are ignored.
No weighting is applied.
The related ``WEIGHT_IMAGE`` and ``WEIGHT_THRESH`` (see below) parameters are ignored.
* ``BACKGROUND`` :
* ``BACKGROUND`` :
the science image itself is used to compute internally a variance map (the related ``WEIGHT_IMAGE``
The science image itself is used to compute internally a variance map (the related ``WEIGHT_IMAGE`` parameter is ignored).
parameter is ignored). Robust (:math:`3\sigma`-clipped) variance estimates are first computed within the same
Robust (:math:`3\sigma`-clipped) variance estimates are first computed within the same background meshes as those described in ?? [#f1]_.
background meshes as those described in ?? [#f1].
The resulting low-resolution variance map is then bicubic-spline-interpolated on the fly to produce the actual full-size variance map.
The resulting low-resolution variance map is then bicubic-spline-interpolated on the fly to produce
the actual full-size variance map.
A check-image with ``CHECKIMAGE_TYPE`` ``MINIBACK_RMS`` can be requested to examine the low-resolution variance map.
A check-image with ``CHECKIMAGE_TYPE`` ``MINIBACK_RMS`` can be requested to examine the low-resolution variance map.
* ``MAP_RMS`` :
* ``MAP_RMS`` :
the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of
The FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of absolute standard deviations (in ADUs per pixel).
absolute standard deviations (in ADUs per pixel).
* ``MAP_VAR`` :
* ``MAP_VAR`` :
the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of
The FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of relative variance.
relative variance. A robust scaling to the appropriate absolute level is then performed by comparing this
A robust scaling to the appropriate absolute level is then performed by comparing this variance map to an internal, low-resolution, absolute variance map built from the science image itself.
variance map to an internal, low-resolution, absolute variance map built from the science image itself.
* ``MAP_WEIGHT`` :
* ``MAP_WEIGHT`` :
the FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of
The FITS image specified by the ``WEIGHT_IMAGE`` file name must contain a weight-map in units of relative weights.
relative weights.
The data are converted to variance units (by definition variance :math:`\propto\frac{1}{weight}`), and scaled as for ``MAP_VAR``.
The data are converted to variance units (by definition variance :math:`\propto\frac{1}{weight}`),
``MAP_WEIGHT`` is the most commonly used type of weight-map: a flat-field, for example, is generally a good approximation to a perfect weight-map.
and scaled as for ``MAP_VAR``.
``MAP_WEIGHT`` is the most commonly used type of weight-map: a flat-field, for example, is generally
a good approximation to a perfect weight-map.
.. _effect_of_weighting:
.. _effect_of_weighting:
...
@@ -78,12 +64,9 @@ Effect of weighting
...
@@ -78,12 +64,9 @@ Effect of weighting
Weight-maps modify the working of |SExtractor| in the following respects:
Weight-maps modify the working of |SExtractor| in the following respects:
#. Bad pixels are discarded from the background statistics.
#. Bad pixels are discarded from the background statistics.
If more than :math:`50\%` of the pixels in a background mesh are bad, the local background value and
If more than :math:`50\%` of the pixels in a background mesh are bad, the local background value and the background standard deviation are replaced by interpolation of the nearest valid meshes.
its standard deviation are replaced by interpolation of the nearest valid meshes.
#. The detection threshold *t* above the local sky background is adjusted for each pixel *i*
#. The detection threshold *t* above the local sky background is adjusted for each pixel *i* with variance :math:`\sigma^2_i` : :math:`t_i=DETECT\_THRESH\times\sqrt{\sigma^2_i}`, where ``DETECT_THRESH`` is expressed in units of standard deviations of the background noise.
with variance :math:`\sigma^2_i` : :math:`t_i=DETECT\_THRESH\times\sqrt{\sigma^2_i}`, where
``DETECT_THRESH`` is expressed in units of standard deviations of the background noise.
Pixels with variance above the threshold set with the ``WEIGHT_THRESH`` parameter are therefore simply not detected.
Pixels with variance above the threshold set with the ``WEIGHT_THRESH`` parameter are therefore simply not detected.
This may result in splitting objects crossed by a group of bad pixels.
This may result in splitting objects crossed by a group of bad pixels.
Interpolation (see ???) should be used to avoid this problem.
Interpolation (see ???) should be used to avoid this problem.
...
@@ -91,34 +74,28 @@ Weight-maps modify the working of |SExtractor| in the following respects:
...
@@ -91,34 +74,28 @@ Weight-maps modify the working of |SExtractor| in the following respects:
This yields optimum scaling of the detection threshold in the case where noise is uncorrelated from pixel to pixel.
This yields optimum scaling of the detection threshold in the case where noise is uncorrelated from pixel to pixel.
Non-linear filtering operations (like those offered by artificial retinae) are not affected.
Non-linear filtering operations (like those offered by artificial retinae) are not affected.
#. The cleaning process (??) takes into account the exact individual thresholds assigned to each pixel for
#. The cleaning process (??) takes into account the exact individual thresholds assigned to each pixel for deciding about the fate of faint detections.
deciding about the fate of faint detections.
#. Error estimates like ``FLUXISO_ERR`` , ``ERRA_IMAGE`` , etc., make use of individual variances too.
#. Error estimates like ``FLUXISO_ERR`` , ``ERRA_IMAGE`` , etc. also make use of individual variances.
Local background-noise standard deviation is simply set to :math:`\sqrt{\sigma^2_i}`.
Local background-noise standard deviation is simply set to :math:`\sqrt{\sigma^2_i}`.
In addition, if the ``WEIGHT_GAIN`` parameter is set to ``Y`` (which is the default), it is assumed that the
In addition, if the ``WEIGHT_GAIN`` parameter is set to ``Y`` (which is the default), it is assumed that the local pixel gain (i.e., the conversion factor from photo-electrons to ADUs) is inversely proportional to :math:`\sigma^2_i`, the median value over the image being set by the ``GAIN`` configuration parameter.
local pixel gain (i.e., the conversion factor from photo-electrons to ADUs) is inversely proportional to
:math:`\sigma^2_i`, its median value over the image being set by the ``GAIN`` configuration parameter.
In other words, it is then supposed that the changes in noise intensities seen over the images are due to gain changes.
In other words, it is then supposed that the changes in noise intensities seen over the images are due to gain changes.
This is the most common case: correction for vignetting, or coverage depth.
This is the most common case: correction for vignetting, or coverage depth.
When this is not the case, for instance when changes are purely dominated by those of the read-out noise,
When this is not the case, for instance when changes are purely dominated by those of the read-out noise, ``WEIGHT_GAIN`` shall be set to ``N``.
``WEIGHT_GAIN`` shall be set to ``N``.
#. Finally, pixels with weights beyond ``WEIGHT_THRESH`` are treated just like pixels discarded by the masking process (??).
#. Finally, pixels with weights beyond ``WEIGHT_THRESH`` are treated just like pixels discarded by the masking process (??).
Combining weight maps
Combining weight maps
---------------------
---------------------
All the weighting options listed in :ref:`weigth-map-format` can be applied separately to detection and measurement images
All the weighting options listed in :ref:`weigth-map-format` can be applied separately to detection and measurement images (:ref:`using-sextractor`), even if some combinations may not always make sense.
(:ref:`using-sextractor`), even if some combinations may not always make sense.
For instance, the following set of configuration lines:
For instance, the following set of configuration lines:
``WEIGHT_IMAGE`` *rms.fits*, *weight.fits*
``WEIGHT_IMAGE`` *rms.fits*, *weight.fits*
``WEIGHT_TYPE`` ``MAP_RMS``, ``MAP_WEIGHT``
``WEIGHT_TYPE`` ``MAP_RMS``, ``MAP_WEIGHT``
will load the FITS file *rms.fits* and use it as an RMS map for adjusting the detection threshold and cleaning,
will load the FITS file *rms.fits* and use it as an RMS map for adjusting the detection threshold and cleaning, while the *weight.fits* weight map will only be used for scaling the error estimates on measurements.
while the *weight.fits* weight map will only be used for scaling the error estimates on measurements.
This can be done in single- as well as in dual-image mode (:ref:`using-sextractor`).
This can be done in single- as well as in dual-image mode (:ref:`using-sextractor`).
``WEIGHT_IMAGE`` can be ignored for ``BACKGROUND`` ``WEIGHT_TYPE``.
``WEIGHT_IMAGE`` can be ignored for ``BACKGROUND`` ``WEIGHT_TYPE``.
It is of course possible to use weight-maps for detection or for measurement only.
It is of course possible to use weight-maps for detection or for measurement only.
...
@@ -134,5 +111,4 @@ will apply weighting only for measurements; detection and cleaning operations wi
...
@@ -134,5 +111,4 @@ will apply weighting only for measurements; detection and cleaning operations wi
.. [#f1] The mesh-filtering procedures act on the variance map, too.
.. [#f1] The mesh-filtering procedures act on the variance map, too.
