updated doc

make latexpdf LATEXMKOPTS="-silent"

cd build/latex || exit

latexmk -pdf -dvi- -ps- -silent 'guidesforcsstdasdevelopers.tex'

cd ../..

open ./build/latex/guidesforcsstdasdevelopers.pdf

 No newline at end of file

Continuous Integration

======================

The Continuous Integration is based on Jenkins.

A push to git repository will trigger a build for the package on Jenkins platform.

Then an email with a build log and unit test coverage report will be delivered to

the relavant developer.

``build status`` explained

--------------------------

The ``build status`` is in the subject of the notification email from ``csst_das@nao.cas.cn``

Here are the explanations of them.

Successful

   Congratulations! Your code passes all tests.

Fixed

   Although last build fails, this build is successful.

Failure

   Current build fails in at least one test.

Still Failing

   Current build fails in at least one test. And last build fails.

``build log`` explained

-----------------------

The build log includes several sections on

1. a check on versions of requirements

   The status will be non-zero if any of your requirements is behind other developers.

2. a check on installation of requirements

   The status will be non-zero if ``pip install -r requirements.txt`` exit with non-zero status.

3. a check on installation of package

   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: bash

      rm -rf dist                                           # remove dist directory if exists

      python setup.py build_ext --inplace                   # build extension

      python setup.py sdist                                 # build source code

      pip install dist/*.tar.gz --force-reinstall --no-deps # install package without dependencies

4. a check on import of interfaces (top_level_interface)

   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: python

      from <your_package>.top_level_interface import *

5. unit tests and coverage

   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: bash

      coverage run -m pytest --import-mode=importlib

6. Numpydoc validation of interfaces

   .. code-block:: bash

      python -m numpydoc --validate {YOUR_PACKAGE}.top_level_interface.{YOUR_FUNCTION/CLASS}

An example of test summary is at the end of build log.

An example of test summary is at the end of build log (the attached ``build.zip``).

If all checks/tests/validations passed, you will see a section as below.

.. code-block::

    Numpydoc status: 0

Unit test coverage report

-------------------------

The ``build log`` also contains a section of unit test coverage report.

An example is below.

.. code-block::

       Name                                Stmts   Miss  Cover

    -------------------------------------------------------

    csst_proto/__init__.py                  3      0   100%

    csst_proto/demo.py                      8      3    62%

    csst_proto/flip_image.py               13      0   100%

    csst_proto/top_level_interface.py       3      0   100%

    tests/test_demos.py                     5      0   100%

    tests/test_flip_image.py                8      0   100%

    -------------------------------------------------------

    TOTAL                                  40      3    92%

This means the overall unit test coverage is 92%.

You can also find a detailed coverage report in ``html`` format in the attached ``htmlcov.zip``.

Caveats

-------

The Numpy style docstring check is only for the packages listed below. 

- csst_proto

- csst_cicd

- csst_ms_mbi_instrument

- csst_ms_mbi_distortion

- csst_ms_mbi_position

- csst_ms_mbi_flux

- csst_ms_mbi_photometry

We welcome other packages participate in.

 No newline at end of file

Code Structure

==============

An example code structure is shown below.

Files/directories with asterisks (*) marks are optional.

.. code-block::

    csst_proto                                # the repository name

    ├── csst_proto                            # the package name

    │   ├── data                              # package associated data directory

    │   ├── __init__.py                       # necessary file for a Python package

    │   ├── flip_image.py                     # Python modules

    │   ├── scratch.py

    │   ├── some_other_modules.py

    │   └── top_level_interface.py            # the top level interface module

    ├── doc                                   # *sphinx-based documentation directory

    │   ├── build

    │   ├── source

    │   ├── Makefile

    │   ├── apidoc.sh

    │   ├── contents.md

    │   ├── make.bat

    │   └── preview.sh

    ├── examples                              # *example scripts

    │   ├── how_this_code_will_be_used.py

    │   └── how_to_write_docstring.py

    ├── tests                                 # unit tests

    │   ├── test_flip_image.py

    │   └── test_other_functions.py

    ├── LICENSE                               # license file

    ├── README.md                             # README file (Markdown recommended)

    ├── install.sh                            # a single-line installation script

    ├── install_local.sh                      # local installation script

    ├── readthedocs.yml                       # *configuration file for readthedocs

    ├── requirements.txt                      # package dependencies

    └── setup.py                              # setup file

Code Style

==========

Python code should follow PEP 8

PEP 8 : The Base Python style

-----------------------------

Python code should follow **PEP 8**

- https://peps.python.org/pep-0008/

Whereas C/C++ code should follow PEP 7

Whereas C/C++ code should follow **PEP 7**

- https://peps.python.org/pep-0007/

Many IDEs (Pycharm, etc) and tools can be used to validate code style

We have a few requirements for our developers in addition to the original PEP8.

These are grabbed from the `Coding Guidelines for astropy-affiliated packages`_.

.. _Coding Guidelines for astropy-affiliated packages: https://docs.astropy.org/en/latest/development/codeguide.html#coding-style-conventions

- Only use 4-space indent

- ``import *`` is not allowed

- The use of automatic code formatters (e.g., Black) is strongly discouraged

Exceptions to PEP 8

-------------------

The following table summarizes all PEP 8 guidelines that are **not followed**

by the this Code Style Guide.

E501

   Allow lines to have > 80 characters, but <=120

E722

   Allow use of bare ``try ... except ...``

E121, E123, E126, E133, E226, E241, E242, E704, W503, W504 and W505

   They are ignored because they are not rules unanimously accepted,

   and PEP 8 does not enforce them.

.. tip::

    Click `here`_ for error code explanations and conventions

.. _here: https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes

- pycodestyle:

    - https://pycodestyle.pycqa.org/en/latest/

Naming conventions

------------------

In our case, we recommend the following method to check code style

All CSST Python source code is consistent with PEP 8 naming in the following ways:

- class names are ``CamelCase`` with leading uppercase

- function names should be ``lower_case_with_underscore``

- module variables used as module global constants are ``UPPERCASE_WITH_UNDERSCORES``

Code style check

----------------

Many tools can be used to check code style, such as

- PyLint: https://pylint.pycqa.org/en/latest/index.html

- Flake8: https://flake8.pycqa.org/en/latest/index.html

- pycodestyle: https://pycodestyle.pycqa.org/en/latest/index.html

All of them are from the PyCQA organization.

In our case, we recommend our developers to use the following command

(with pycodestyle) to check code style before committing code.

.. code-block:: bash

    pycodestyle ./**/*.py --ignore=E121,E123,E126,E226,E24,E704,W503,W504,E501,E722

Recommended IDEs

----------------

For writing Python packages, two powerful IDEs are worth to recommend:

- PyCharm: https://www.jetbrains.com/pycharm/

- VSCode: https://code.visualstudio.com/

Both of them have plugins or built-in tools to check code style.

Numpy-style docstring

=====================

---------------------

There are several popular docstring styles, namely Google, reStructuredText, and Numpy.

The CSST DAS adopts Numpydoc-style among the three.

We refer our developers to the Numpydoc official style guide for instructions on

    python -m numpydoc --validate csst_proto.top_level_interface.flip_image

An example with typical Numpy-style docstring is shown below

.. literalinclude:: ../../csst_proto/demo.py

   :linenos:

   :language: python

.. note::

    Numpy-style docstring uses a two-section summary for functions / classes,

    including ``a short summary`` (usually a single line) right after the quote

    and a following detailed ``extended summary``.

.. note::

    The ``See Also`` section is not required for our development.

.. note::

    Presently, Numpy style docstring check is only performed on the functions / classes

    listed in the ``__all__`` variable in ``top_level_interface`` module.

Markup languages

================

----------------

Markdown and reStructrured Text are two of popular markup languages.

reStructrured Text is used in Numpydoc docstrings as well as sphinx-based documentation.

Markdown is much easier and usually used to write `README` for a package.

author = 'CSST DAS Team'

# The full version, including alpha/beta/rc tags

release = '0.0.1'

release = '0.0.1alpha'

# -- General configuration ---------------------------------------------------

Using ``csst_common``

=====================

Some usages of the functions / classes in ``csst_common``. (To be updated)

``CsstMsDataManager``

---------------------

A class that helps developers to access simulation files.

.. code-block:: python

    # for full basic initialization

    dm_mbi = CsstMsDataManager(

        ver_sim="C5.2",                 # version of simulation, "C5.2" is the latest

        dir_l0="",                      # the L0/input directory

        dir_l1="",                      # the L1/output directory

        dir_pcref="",                   # position calibration reference (will be deprecated)

        path_aux="",                    # aux file paths (master bias, dark, flat)

        assert_all_detectors=False,     # if True, assert all detectors are available

        datatype="mbi",                 # "mbi" or "sls"

    )

    # for a quick start

    dm_mbi = CsstMsDataManager.quickstart(

        ver_sim="C5.2",

        datatype="mbi",

        dir_l1=".",

        exposure_id=100                 # the 100th exposure

    )

    # access L0 directory

    dm_mbi.dir_l0

    # access L1 directory

    dm_mbi.dir_l1

    # access dir_pcref

    dm_mbi.dir_pcref

    # access path_aux

    dm_mbi.path_aux

    # access ver_sim

    dm_mbi.ver_sim

    # access target detectors

    dm_mbi.target_detectors

    # access available detectors

    dm_mbi.available_detectors

    # define an L1 file (detector-specified)

    dm_mbi.l1_detector(detector=6)

    # define an L1 file (non-detector-specified)

    dm_mbi.l1_file("flipped_image.fits")

    # to get bias

    dm_mbi.get_bias()

    # to get dark

    dm_mbi.get_dark()

    # to get flat

    dm_mbi.get_flat()

``csst_common.logger.get_logger()``

-----------------------------------

Get the default configured ``logging.Logger``.

.. code-block:: python

    from csst_common.logger import get_logger

    logger = get_logger()

.. note::

    Developers should NOT use ``print`` function extensively in code.

    Generally ``print`` can be replaced with ``logger.debug("msg")``.

    For important information, use ``logger.info("msg")``.

    For warnings, use ``logger.warning("msg")``.

    For errors, use ``logger.error("msg")``.

    Conf https://docs.python.org/3/library/logging.html for more information

    about the usage of ``logging.Logger``.

``csst_common.status.CsstStatus``

---------------------------------

Developers should use ``csst_common.status.CsstStatus`` to return the

status of their interfaces.

.. code-block:: python

    from csst_common.status import CsstStatus

    # presently 3 kinds of status are available, they are

    CsstStatus.PERFECT

    CsstStatus.WARNING

    CsstStatus.ERROR

An example interface

--------------------

We recommend our developers to use the structure shown below in their code.

.. literalinclude:: ./example_interface.py

    :linenos:

    :language: python