Commit 3fa4ed00 authored by Emmanuel Bertin's avatar Emmanuel Bertin
Browse files

Doc: some refactoring.

parent 9d813987
Loading
Loading
Loading
Loading
+18 −26
Original line number Original line Diff line number Diff line
.. File Config.rst
.. File Config.rst


.. include:: global.rst

The configuration file
The configuration file
======================
======================


Each time it is run, |SExtractor| looks for a configuration file. If no
Each time it is run, |SExtractor| looks for a configuration file.
configuration file is specified in the command-line, it is assumed to be
If no configuration file is specified in the command-line, it is assumed to be called :file:`default.sex` and to reside in the current directory.
called :file:`default.sex` and to reside in the current directory. If no
If no configuration file is found, |SExtractor| will use its own internal default configuration.
configuration file is found, |SExtractor| will use its own internal
default configuration.


Creating a configuration file
Creating a configuration file
-----------------------------
-----------------------------


SExtractor can generate an ASCII dump of its internal default
SExtractor can generate an ASCII dump of its internal default configuration, using the ``-d`` option.
configuration, using the ``-d`` option. By redirecting the standard output
By redirecting the standard output of SExtractor to a file, one creates a configuration file that can easily be modified afterwards:
of SExtractor to a file, one creates a configuration file that can
easily be modified afterwards:


.. code-block:: console
.. code-block:: console


  $ sex -d > default.sex
  $ sex -d > default.sex


A more extensive dump with less commonly used parameters can be
A more extensive dump with less commonly used parameters can be generated by using the ``-dd`` option.
generated by using the ``-dd`` option.


Format of the configuration file
Format of the configuration file
--------------------------------
--------------------------------


The format is ASCII. There must be only one parameter set per line,
The format is ASCII. There must be only one parameter set per line, following the form::
following the form::


 Config-parameter      Value(s)
 Config-parameter      Value(s)


Extra spaces or linefeeds are ignored. Comments must begin with a ``#``
Extra spaces or linefeeds are ignored.
and end with a linefeed. Values can be of different types: strings (can
Comments must begin with a ``#`` and end with a linefeed.
be enclosed between double quotes), floats, integers, keywords or
Values can be of different types: strings (can be enclosed between double quotes), floats, integers, keywords or Boolean (`Y`/`y` or `N`/`n`).
Boolean (`Y`/`y` or `N`/`n`). Some parameters accept zero or several values,
Some parameters accept zero or several values, which must then be separated by commas.
which must then be separated by commas. Integers can be given as
Integers can be given as decimals, in octal form (preceded by digit O), or in hexadecimal (preceded by `0x`).
decimals, in octal form (preceded by digit O), or in hexadecimal
The hexadecimal format is particularly convenient for writing multiplexed bit values such as binary masks.
(preceded by `0x`). The hexadecimal format is particularly convenient for
Environment variables, written as ``$HOME`` or ``${HOME}`` are expanded.
writing multiplexed bit values such as binary masks. Environment
variables, written as ``$HOME`` or ``${HOME}`` are expanded.


.. _param_list:
.. _param_list:


Configuration parameter list
Configuration parameter list
----------------------------
----------------------------


Here is a complete list of all the configuration parameters known to
Here is a complete list of all the configuration parameters known to |SExtractor|.
|SExtractor|. Please refer to the next sections for a detailed description
Please refer to the next sections for a detailed description of their meaning.
of their meaning.


.. include:: keys.rst
+28 −65
Original line number Original line Diff line number Diff line
.. File Installing.rst
.. File Installing.rst


.. include:: global.rst

***********************
***********************
Installing the software
Installing the software
***********************
***********************
@@ -12,95 +14,68 @@ is not necessary to operate the software.


Memory requirements depend on the size of the images to be processed.
Memory requirements depend on the size of the images to be processed.
Processing a single image should typically require about 100MB of memory.
Processing a single image should typically require about 100MB of memory.
For large images (hundreds of Mpixels or more), or in double-image / weighted
For large images (hundreds of Mpixels or more), or in double-image / weighted mode, |SExtractor|'s  memory footprint should be around 500MB, and up to 2GB in the worst cases.
mode, |SExtractor|'s  memory footprint should be around 500MB, and up to 2GB in
Swap-space can be put to contribution, although a strong performance hit is to be expected.
the worst cases. Swap-space can be put to contribution, although a strong
performance hit is to be expected.


Obtaining |SExtractor|
Obtaining |SExtractor|
----------------------
----------------------


For Linux users, the simplest way to have |SExtractor| up and running is to
For Linux users, the simplest way to have |SExtractor| up and running is to install the standard binary package the comes with your Linux distribution.
install the standard binary package the comes with your Linux distribution.
Run, e.g., ``apt-get sextractor`` (on Debian) or ``dnf sextractor`` (Fedora) as root and |SExtractor|, as well as all its dependencies, will automatically be installed.
Run, e.g., ``apt-get sextractor`` (on Debian) or ``dnf sextractor`` (Fedora) as
If you decided to install the package this way you may skip the following and move straight to the :ref:`next section <Using Sextractor>`.
root and |SExtractor|, as well as all its dependencies, will automatically be

installed. If you decided to install the package this way you may skip the
However if |SExtractor| is not available in your distribution, or to obtain the most recent version, the |SExtractor| source package can be downloaded from `the official GitHub repository <https://github.com/astromatic/sextractor>`_ .
following and move straight to the :ref:`next section <Using Sextractor>`.
One may choose `one of the stable releases <https://github.com/astromatic/sextractor/releases>`_, or for the fearless, `a copy of the current master development branch <https://github.com/astromatic/sextractor/archive/master.zip>`_.

However if |SExtractor| is not available in your distribution, or to obtain the
most recent version, the |SExtractor| source package can be downloaded from
`the official GitHub repository <https://github.com/astromatic/sextractor>`_ .
One may choose `one of the stable releases
<https://github.com/astromatic/sextractor/releases>`_, or for the fearless,
`a copy of the current master development branch
<https://github.com/astromatic/sextractor/archive/master.zip>`_.


Software requirements
Software requirements
---------------------
---------------------


|SExtractor| has been developed on
|SExtractor| has been developed on `GNU/Linux <http://en.wikipedia.org/wiki/Linux>`_ machines and should compile on any `POSIX <http://en.wikipedia.org/wiki/POSIX>`_-compliant system (this includes |OSX|_ and `Cygwin <http://www.cygwin.com>`_ on |Windows|_, at the price of some difficulties with the configuration), provided that the *development* packages of the following libraries have been installed:
`GNU/Linux <http://en.wikipedia.org/wiki/Linux>`_ machines and should compile on
any `POSIX <http://en.wikipedia.org/wiki/POSIX>`_-compliant system (this
includes |OSX|_ and `Cygwin <http://www.cygwin.com>`_ on |Windows|_, at the
price of some difficulties with the configuration), provided that the
*development* packages of the following libraries have been installed:


* |ATLAS|_ V3.6 and above [#atlas_install]_,
* |ATLAS|_ V3.6 and above [#atlas_install]_,
* |FFTw|_ V3.0 and above [#fftw_install]_, 
* |FFTw|_ V3.0 and above [#fftw_install]_, 


On Fedora/Redhat distributions for instance, the development packages above are
On Fedora/Redhat distributions for instance, the development packages above are available as ``atlas-devel`` and ``fftw-devel``.
available as ``atlas-devel`` and ``fftw-devel``. Note that |ATLAS| and |FFTw|
Note that |ATLAS| and |FFTw| are not necessary if |SExtractor| is linked with |Intel|'s |MKL|_ library.
are not necessary if |SExtractor| is linked with |Intel|'s |MKL|_ library.


Installation
Installation
------------
------------


To install from the |GitHub| source package, you must first uncompress the
To install from the |GitHub| source package, you must first uncompress the archive:
archive:


.. code-block:: console
.. code-block:: console


  $ unzip sextractor-<version>.zip
  $ unzip sextractor-<version>.zip


A new directory called :file:`sextractor-<version>` should now appear at the
A new directory called :file:`sextractor-<version>` should now appear at the current location on your disk.
current location on your disk. Enter the directory and generate the files
Enter the directory and generate the files required by the `autotools <http://en.wikipedia.org/wiki/GNU_Build_System>`_, which the package relies on:
required by the `autotools <http://en.wikipedia.org/wiki/GNU_Build_System>`_,
which the package relies on:


.. code-block:: console
.. code-block:: console


  $ cd sextractor-<version>
  $ cd sextractor-<version>
  $ sh autogen.sh
  $ sh autogen.sh


A :program:`configure` script is created. This script has many options, which
A :program:`configure` script is created.
may be listed with the ``--help`` option:
This script has many options, which may be listed with the ``--help`` option:


.. code-block:: console
.. code-block:: console


  $ ./configure --help
  $ ./configure --help


No options are required for compiling with the default GNU C compiler
No options are required for compiling with the default GNU C compiler (:program:`gcc`) if all the required libraries are installed at their default locations:
(:program:`gcc`) if all the required libraries are installed at their default
locations:


.. code-block:: console
.. code-block:: console


  $ ./configure
  $ ./configure


Compared to :program:`gcc` and the librairies above, the combination of the
Compared to :program:`gcc` and the librairies above, the combination of the |Intel| compiler (:program:`icc`) and the |MKL|_ libraries can give the |SExtractor| executable a strong boost in performance, thanks to better vectorized code.
|Intel| compiler (:program:`icc`) and the |MKL|_ libraries can give the
If :program:`icc` and the |MKL| are installed on your system [#geticc]_ , you can take advantage of them using
|SExtractor| executable a strong boost in performance, thanks to better
vectorized code. If :program:`icc` and the |MKL| are installed on your system
[#geticc]_ , you can take advantage of them using


.. code-block:: console
.. code-block:: console


  $ ./configure --enable-mkl
  $ ./configure --enable-mkl


Additionally, if the |SExtractor| binary is to be run on a different machine
Additionally, if the |SExtractor| binary is to be run on a different machine that does not have :program:`icc` and the |MKL| installed (e.g., a cluster computing node), you must configure a partially statically linked executable using
that does not have :program:`icc` and the |MKL| installed (e.g., a cluster
computing node), you must configure a partially statically linked executable
using


.. code-block:: console
.. code-block:: console


@@ -112,34 +87,22 @@ In all cases, |SExtractor| can now be compiled with


  $ make -j
  $ make -j


An :file:`src/sex` executable is created. For system-wide installation, run
An :file:`src/sex` executable is created. For system-wide installation, run the usual
the usual


.. code-block:: console
.. code-block:: console


  $ sudo make install
  $ sudo make install


You may now check that the software is properly installed by simply
You may now check that the software is properly installed by simply typing in your shell:
typing in your shell:


.. code-block:: console
.. code-block:: console


  $ sex
  $ sex


which will return the version number and other basic information (note that
which will return the version number and other basic information (note that some shells require the :program:`rehash` command to be run before making a freshly installed executable accessible in the execution path).
some shells require the :program:`rehash` command to be run before making a
freshly installed executable accessible in the execution path).


.. [#mac_install] Mac OS X |.dmg|_ packages should be available soon.
.. [#mac_install] Mac OS X |.dmg|_ packages should be available soon.
.. [#atlas_install] Use the ``--with-atlas`` and/or
.. [#atlas_install] Use the ``--with-atlas`` and/or ``--with-atlas-incdir`` options of the |SExtractor| :command:`configure` script to specify the |ATLAS| library and include paths if |ATLAS| files are  installed at unusual locations.
   ``--with-atlas-incdir`` options of the |SExtractor| :command:`configure`
.. [#fftw_install] Make sure that |FFTW| has been compiled with :command:`configure` options ``--enable-threads --enable-float``.
   script to specify the |ATLAS| library and include paths if |ATLAS| files are 
.. [#geticc] The Linux versions of the |Intel| compiler and |MKL| are `available for free to academic researchers, students, educators and open source contributors <http://software.intel.com/qualify-for-free-software>`_.
   installed at unusual locations.
.. [#fftw_install] Make sure that |FFTW| has been compiled with
   :command:`configure` options ``--enable-threads --enable-float``.
.. [#geticc] The Linux versions of the |Intel| compiler and |MKL| are
   `available for free to academic researchers, students, educators and open
   source contributors <http://software.intel.com/qualify-for-free-software>`_.

.. include:: keys.rst
+4 −8
Original line number Original line Diff line number Diff line
.. File Introduction.rst
.. File Introduction.rst


.. include:: global.rst

Introduction
Introduction
============
============


|SExtractor|_ (Source-Extractor) is a program that builds a catalog
|SExtractor|_ (Source-Extractor) is a program that builds a catalog of objects from an astronomical image. It is particularly oriented towards the reduction of large scale galaxy-survey data, but it also performs well on moderately crowded star fields. Its main features are:
of objects from an astronomical image. It is particularly oriented
towards the reduction of large scale galaxy-survey data, but it also performs
well on moderately crowded star fields. Its main features are:


* Support for multi-extension FITS (|MEF|_)
* Support for multi-extension FITS (|MEF|_)
* Speed: up to about 50 Mpixel/s or 10,000 sources/s with a 3 GHz processor
* Speed: up to about 50 Mpixel/s or 10,000 sources/s with a 3 GHz processor
@@ -22,8 +21,5 @@ well on moderately crowded star fields. Its main features are:
* Built-in catalog cross-identification.
* Built-in catalog cross-identification.
* Special mode for photographic scans.
* Special mode for photographic scans.
* |XML|_ |VOTable|_-compliant catalog output.
* |XML|_ |VOTable|_-compliant catalog output.
* |XSLT|_ filter sheet provided for convenient access to metadata from a
* |XSLT|_ filter sheet provided for convenient access to metadata from a regular web browser.
  regular web browser.

.. include:: keys.rst
+6 −10
Original line number Original line Diff line number Diff line
.. File License.rst
.. File License.rst


.. include:: global.rst

License
License
=======
=======


|SExtractor|_ *is free software: you can redistribute it and/or modify it under
|SExtractor|_ *is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.*
the terms of the GNU General Public License as published by the Free
|SExtractor| *is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Software Foundation, either version 3 of the License, or (at your
See the GNU General Public License for more details.
option) any later version.* |SExtractor| *is distributed in the hope that it will
You should have received a copy of the GNU General Public License along with* |SExtractor|. *If not, see* `www.gnu.org/licenses/ <http://www.gnu.org/licences/>`_.
be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details. You should have received a copy
of the GNU General Public License along with* |SExtractor|. *If not, see*
`www.gnu.org/licenses/ <http://www.gnu.org/licences/>`_.


.. include:: keys.rst
+5 −3
Original line number Original line Diff line number Diff line
Measurements
Measurements
============
============


Once sources have been detected and deblended, they enter the measurement phase. |SExtractor| performs three categories of measurements: isophotal, full, and model-fitting.
.. include:: global.rst

Once sources have been detected and deblended, they enter the measurement phase.
|SExtractor| performs three categories of measurements: isophotal, full, and model-fitting.


.. _isophotal_measurements:
.. _isophotal_measurements:


Isophotal
Isophotal
  Measurements are made on the isophotal object footprints, which are defined on the filtered detection image. Only pixels with values above the threshold set with ``ANALYSIS_THRESH`` are considered [#thresh]_, which makes the analysis extremely fast, but obviously strongly dependent on the threshold itself. This is an issue particularly when the amplitude of the bakground noise varies over the image. Many of the isophotal measurements (e.g., ``X_IMAGE``, ``Y_IMAGE``, ``FLUX_ISO``) are necessary for the internal operations of |SExtractor| and are therefore executed even if they are not requested.
  Measurements are made on the isophotal object footprints, which are defined on the filtered detection image. Only pixels with values above the threshold set with ``ANALYSIS_THRESH`` are considered [#thresh]_, which makes the analysis extremely fast, but obviously strongly dependent on the threshold itself. This is an issue particularly when the amplitude of the bakground noise varies over the image. Many of the isophotal measurements (e.g., :param:`X_IMAGE`, :param:`Y_IMAGE`, :param:`FLUX_ISO`) are necessary for the internal operations of |SExtractor| and are therefore executed even if they are not requested.


Full
Full
  Measurements have access to all pixels of the image. These measurements are generally more sophisticated, less affected by variable biases induced by the detection threshold, and still reasonably fast. They are done at a later stage of the processing, after CLEANing and MASKing.
  Measurements have access to all pixels of the image. These measurements are generally more sophisticated, less affected by variable biases induced by the detection threshold, and still reasonably fast. They are done at a later stage of the processing, after CLEANing and MASKing.
@@ -23,5 +26,4 @@ Model-fitting
.. [#thresh] Pixel values also have to exceed the local detection threshold set with ``DETECT_THRESH``.
.. [#thresh] Pixel values also have to exceed the local detection threshold set with ``DETECT_THRESH``.
.. [#psf_models] PSF models be computed using the |PSFEx|_ package.
.. [#psf_models] PSF models be computed using the |PSFEx|_ package.


.. include:: keys.rst
Loading