Build and Test
==============

A push to git repository will trigger an automatic build and test on the Jenkins platform.
Then a notification email from ``csst_das@nao.cas.cn`` is composed,
attached with a ``build log`` and a ``coverage report``,
is sent to relavant developers and code reviewers.


``build status`` explained
--------------------------

The ``build status`` is in the subject of the notification email.
Here are the explanations of them.

Successful
   Congratulations! Your code passes all tests.
Fixed
   Although last build fails, this build is successful.
Failure
   Current build fails in at least one test.
Still Failing
   Current build fails in at least one test. And last build fails.


``build log`` explained
-----------------------

The build log includes 6 sections.

[1/6] Check on versions of requirements
   The status will be non-zero if any of your requirements is behind other developers.
[2/6] Check on installation of requirements
   The status will be non-zero if ``pip install -r requirements.txt`` exit with non-zero status.
[3/6] Check on installation of package
   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: bash

      rm -rf dist                                           # remove dist directory if exists
      python setup.py build_ext --inplace                   # build extension
      python setup.py sdist                                 # build source code
      pip install dist/*.tar.gz --force-reinstall --no-deps # install package without dependencies

[4/6] Check on import of interfaces (``top_level_interface``)
   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: python

      from <your_package>.top_level_interface import *

[5/6] Run unit tests and coverage
   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: bash

      coverage run -m pytest --import-mode=importlib

[6/6] ``Numpydoc`` validation of interfaces
   The status will be non-zero if the following code are unsuccessfully executed

   .. code-block:: bash

      python -m numpydoc --validate {YOUR_PACKAGE}.top_level_interface.{YOUR_FUNCTION/CLASS}


An example of test summary is at the end of build log (the attached ``build.zip``).
If all checks/tests/validations passed, you will see a section as below.

.. code-block:: none

    ===================== TEST SUMMARY =====================
    Test passed!
    Check requirements: 0
    Install requirements: 0
    Install package: 0
    Importability status: 0
    Unit test status: 0
    Numpydoc status: 0


Unit test coverage report
-------------------------

The ``build log`` also contains a section of unit test coverage report.
An example is below.

.. code-block:: none

    ---------- coverage: platform darwin, python 3.9.7-final-0 -----------
    Name                                Stmts   Miss  Cover   Missing
    -----------------------------------------------------------------
    csst_proto/__init__.py                  3      0   100%
    csst_proto/demo.py                      8      3    62%   56-57, 60
    csst_proto/flip_image.py               21      0   100%
    csst_proto/top_level_interface.py       3      0   100%
    -----------------------------------------------------------------
    TOTAL                                  35      3    91%


This means the overall unit test coverage is 91%.
The missing terms are also shown in the last column.
You can also find a detailed coverage report in ``html`` format in the attached ``htmlcov.zip``.


Caveats
-------
The Numpy style docstring check is only for the packages listed below. 

- csst_proto
- csst_cicd
- csst_ms_mbi_instrument
- csst_ms_mbi_distortion
- csst_ms_mbi_position
- csst_ms_mbi_flux
- csst_ms_mbi_photometry

We welcome other packages participate in.