updated doc

doc/latexpdf.sh

+make latexpdf LATEXMKOPTS="-silent"
+cd build/latex || exit
+latexmk -pdf -dvi- -ps- -silent 'guidesforcsstdasdevelopers.tex'
+cd ../..
+open ./build/latex/guidesforcsstdasdevelopers.pdf
\ No newline at end of file

doc/source/cicd.rst

 Continuous Integration
 ======================
+
 The Continuous Integration is based on Jenkins.
 A push to git repository will trigger a build for the package on Jenkins platform.
 Then an email with a build log and unit test coverage report will be delivered to
 the relavant developer.
 
+
+``build status`` explained
+--------------------------
+
+The ``build status`` is in the subject of the notification email from ``csst_das@nao.cas.cn``
+Here are the explanations of them.
+
+Successful
+   Congratulations! Your code passes all tests.
+Fixed
+   Although last build fails, this build is successful.
+Failure
+   Current build fails in at least one test.
+Still Failing
+   Current build fails in at least one test. And last build fails.
+
+
+``build log`` explained
+-----------------------
+
 The build log includes several sections on
 
 1. a check on versions of requirements
+   The status will be non-zero if any of your requirements is behind other developers.
 2. a check on installation of requirements
+   The status will be non-zero if ``pip install -r requirements.txt`` exit with non-zero status.
 3. a check on installation of package
+   The status will be non-zero if the following code are unsuccessfully executed
+
+   .. code-block:: bash
+
+      rm -rf dist                                           # remove dist directory if exists
+      python setup.py build_ext --inplace                   # build extension
+      python setup.py sdist                                 # build source code
+      pip install dist/*.tar.gz --force-reinstall --no-deps # install package without dependencies
+
 4. a check on import of interfaces (top_level_interface)
+   The status will be non-zero if the following code are unsuccessfully executed
+
+   .. code-block:: python
+
+      from <your_package>.top_level_interface import *
+
 5. unit tests and coverage
+   The status will be non-zero if the following code are unsuccessfully executed
+
+   .. code-block:: bash
+
+      coverage run -m pytest --import-mode=importlib
+
 6. Numpydoc validation of interfaces
 
+   .. code-block:: bash
+
+      python -m numpydoc --validate {YOUR_PACKAGE}.top_level_interface.{YOUR_FUNCTION/CLASS}
 
-An example of test summary is at the end of build log.
+
+An example of test summary is at the end of build log (the attached ``build.zip``).
 If all checks/tests/validations passed, you will see a section as below.
 
 .. code-block::
@@ -30,4 +78,39 @@ If all checks/tests/validations passed, you will see a section as below.
     Numpydoc status: 0
 
 
+Unit test coverage report
+-------------------------
+
+The ``build log`` also contains a section of unit test coverage report.
+An example is below.
+
+.. code-block::
+
+       Name                                Stmts   Miss  Cover
+    -------------------------------------------------------
+    csst_proto/__init__.py                  3      0   100%
+    csst_proto/demo.py                      8      3    62%
+    csst_proto/flip_image.py               13      0   100%
+    csst_proto/top_level_interface.py       3      0   100%
+    tests/test_demos.py                     5      0   100%
+    tests/test_flip_image.py                8      0   100%
+    -------------------------------------------------------
+    TOTAL                                  40      3    92%
+
+This means the overall unit test coverage is 92%.
+You can also find a detailed coverage report in ``html`` format in the attached ``htmlcov.zip``.
+
+
+Caveats
+-------
+The Numpy style docstring check is only for the packages listed below. 
+
+- csst_proto
+- csst_cicd
+- csst_ms_mbi_instrument
+- csst_ms_mbi_distortion
+- csst_ms_mbi_position
+- csst_ms_mbi_flux
+- csst_ms_mbi_photometry
 
+We welcome other packages participate in.
\ No newline at end of file

doc/source/codestyle.rst

-Code Structure
-==============
-An example code structure is shown below.
-Files/directories with asterisks (*) marks are optional.
-
-.. code-block::
-
-    csst_proto                                # the repository name
-    ├── csst_proto                            # the package name
-    │   ├── data                              # package associated data directory
-    │   ├── __init__.py                       # necessary file for a Python package
-    │   ├── flip_image.py                     # Python modules
-    │   ├── scratch.py
-    │   ├── some_other_modules.py
-    │   └── top_level_interface.py            # the top level interface module
-    ├── doc                                   # *sphinx-based documentation directory
-    │   ├── build
-    │   ├── source
-    │   ├── Makefile
-    │   ├── apidoc.sh
-    │   ├── contents.md
-    │   ├── make.bat
-    │   └── preview.sh
-    ├── examples                              # *example scripts
-    │   ├── how_this_code_will_be_used.py
-    │   └── how_to_write_docstring.py
-    ├── tests                                 # unit tests
-    │   ├── test_flip_image.py
-    │   └── test_other_functions.py
-    ├── LICENSE                               # license file
-    ├── README.md                             # README file (Markdown recommended)
-    ├── install.sh                            # a single-line installation script
-    ├── install_local.sh                      # local installation script
-    ├── readthedocs.yml                       # *configuration file for readthedocs
-    ├── requirements.txt                      # package dependencies
-    └── setup.py                              # setup file
-
-
 Code Style
 ==========
-Python code should follow PEP 8
+
+
+PEP 8 : The Base Python style
+-----------------------------
+Python code should follow **PEP 8**
 
 - https://peps.python.org/pep-0008/
 
-Whereas C/C++ code should follow PEP 7
+Whereas C/C++ code should follow **PEP 7**
 
 - https://peps.python.org/pep-0007/
 
-Many IDEs (Pycharm, etc) and tools can be used to validate code style
+We have a few requirements for our developers in addition to the original PEP8.
+These are grabbed from the `Coding Guidelines for astropy-affiliated packages`_.
+
+.. _Coding Guidelines for astropy-affiliated packages: https://docs.astropy.org/en/latest/development/codeguide.html#coding-style-conventions
+
+- Only use 4-space indent
+- ``import *`` is not allowed
+- The use of automatic code formatters (e.g., Black) is strongly discouraged
+
+
+Exceptions to PEP 8
+-------------------
+
+The following table summarizes all PEP 8 guidelines that are **not followed**
+by the this Code Style Guide.
+
+E501
+   Allow lines to have > 80 characters, but <=120
+E722
+   Allow use of bare ``try ... except ...``
+E121, E123, E126, E133, E226, E241, E242, E704, W503, W504 and W505
+   They are ignored because they are not rules unanimously accepted,
+   and PEP 8 does not enforce them.
+
+.. tip::
+    Click `here`_ for error code explanations and conventions
+
+.. _here: https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
+
 
-- pycodestyle:
-    - https://pycodestyle.pycqa.org/en/latest/
+Naming conventions
+------------------
 
-In our case, we recommend the following method to check code style
+All CSST Python source code is consistent with PEP 8 naming in the following ways:
+
+- class names are ``CamelCase`` with leading uppercase
+- function names should be ``lower_case_with_underscore``
+- module variables used as module global constants are ``UPPERCASE_WITH_UNDERSCORES``
+
+
+Code style check
+----------------
+Many tools can be used to check code style, such as
+
+- PyLint: https://pylint.pycqa.org/en/latest/index.html
+- Flake8: https://flake8.pycqa.org/en/latest/index.html
+- pycodestyle: https://pycodestyle.pycqa.org/en/latest/index.html
+
+All of them are from the PyCQA organization.
+In our case, we recommend our developers to use the following command
+(with pycodestyle) to check code style before committing code.
 
 .. code-block:: bash
 
     pycodestyle ./**/*.py --ignore=E121,E123,E126,E226,E24,E704,W503,W504,E501,E722
 
 
+Recommended IDEs
+----------------
+For writing Python packages, two powerful IDEs are worth to recommend:
+
+- PyCharm: https://www.jetbrains.com/pycharm/
+- VSCode: https://code.visualstudio.com/
+
+Both of them have plugins or built-in tools to check code style.
+
+
 Numpy-style docstring
-=====================
+---------------------
 There are several popular docstring styles, namely Google, reStructuredText, and Numpy.
 The CSST DAS adopts Numpydoc-style among the three.
 We refer our developers to the Numpydoc official style guide for instructions on
@@ -83,14 +104,31 @@ For example,
 
     python -m numpydoc --validate csst_proto.top_level_interface.flip_image
 
+An example with typical Numpy-style docstring is shown below
+
+.. literalinclude:: ../../csst_proto/demo.py
+   :linenos:
+   :language: python
+
+.. note::
+    Numpy-style docstring uses a two-section summary for functions / classes,
+    including ``a short summary`` (usually a single line) right after the quote
+    and a following detailed ``extended summary``.
+
+.. note::
+    The ``See Also`` section is not required for our development.
+
+.. note::
+    Presently, Numpy style docstring check is only performed on the functions / classes
+    listed in the ``__all__`` variable in ``top_level_interface`` module.
 
 Markup languages
-================
+----------------
 Markdown and reStructrured Text are two of popular markup languages.
 reStructrured Text is used in Numpydoc docstrings as well as sphinx-based documentation.
 Markdown is much easier and usually used to write `README` for a package.
 
 - Markdown
-    - https://www.markdownguide.org/
+  - https://www.markdownguide.org/
 - reStructured Text
-    - https://www.writethedocs.org/guide/writing/reStructuredText/
+  - https://www.writethedocs.org/guide/writing/reStructuredText/

doc/source/conf.py

@@ -22,7 +22,7 @@ copyright = '2022, CSST DAS Team'
 author = 'CSST DAS Team'
 
 # The full version, including alpha/beta/rc tags
-release = '0.0.1'
+release = '0.0.1alpha'
 
 
 # -- General configuration ---------------------------------------------------

doc/source/csst_common/csst_common.rst

+Using ``csst_common``
+=====================
+
+Some usages of the functions / classes in ``csst_common``. (To be updated)
+
+``CsstMsDataManager``
+---------------------
+
+A class that helps developers to access simulation files.
+
+.. code-block:: python
+
+    # for full basic initialization
+    dm_mbi = CsstMsDataManager(
+        ver_sim="C5.2",                 # version of simulation, "C5.2" is the latest
+        dir_l0="",                      # the L0/input directory
+        dir_l1="",                      # the L1/output directory
+        dir_pcref="",                   # position calibration reference (will be deprecated)
+        path_aux="",                    # aux file paths (master bias, dark, flat)
+        assert_all_detectors=False,     # if True, assert all detectors are available
+        datatype="mbi",                 # "mbi" or "sls"
+    )
+    # for a quick start
+    dm_mbi = CsstMsDataManager.quickstart(
+        ver_sim="C5.2",
+        datatype="mbi",
+        dir_l1=".",
+        exposure_id=100                 # the 100th exposure
+    )
+    # access L0 directory
+    dm_mbi.dir_l0
+    # access L1 directory
+    dm_mbi.dir_l1
+    # access dir_pcref
+    dm_mbi.dir_pcref
+    # access path_aux
+    dm_mbi.path_aux
+    # access ver_sim
+    dm_mbi.ver_sim
+    # access target detectors
+    dm_mbi.target_detectors
+    # access available detectors
+    dm_mbi.available_detectors
+    # define an L1 file (detector-specified)
+    dm_mbi.l1_detector(detector=6)
+    # define an L1 file (non-detector-specified)
+    dm_mbi.l1_file("flipped_image.fits")
+
+    # to get bias
+    dm_mbi.get_bias()
+    # to get dark
+    dm_mbi.get_dark()
+    # to get flat
+    dm_mbi.get_flat()
+
+
+``csst_common.logger.get_logger()``
+-----------------------------------
+
+Get the default configured ``logging.Logger``.
+
+
+.. code-block:: python
+
+    from csst_common.logger import get_logger
+    logger = get_logger()
+
+.. note::
+    Developers should NOT use ``print`` function extensively in code.
+    Generally ``print`` can be replaced with ``logger.debug("msg")``.
+    For important information, use ``logger.info("msg")``.
+    For warnings, use ``logger.warning("msg")``.
+    For errors, use ``logger.error("msg")``.
+    Conf https://docs.python.org/3/library/logging.html for more information
+    about the usage of ``logging.Logger``.
+
+
+``csst_common.status.CsstStatus``
+---------------------------------
+
+Developers should use ``csst_common.status.CsstStatus`` to return the
+status of their interfaces.
+
+.. code-block:: python
+
+    from csst_common.status import CsstStatus
+    # presently 3 kinds of status are available, they are
+    CsstStatus.PERFECT
+    CsstStatus.WARNING
+    CsstStatus.ERROR
+
+
+An example interface
+--------------------
+
+We recommend our developers to use the structure shown below in their code.
+
+.. literalinclude:: ./example_interface.py
+    :linenos:
+    :language: python

doc/source/csst_common/example_interface.py

+import logging
+import os
+from typing import Union
+
+import numpy as np
+from astropy.io import fits
+from csst_common.data_manager import CsstMsDataManager
+from csst_common.logger import get_logger
+from csst_common.status import CsstStatus
+
+
+def read_image(filename_input: str) -> np.ndarray:
+    """ Read image. """
+    return fits.getdata(filename_input)
+
+
+def process_data(data: np.ndarray) -> np.ndarray:
+    """ Process data. """
+    return np.fliplr(np.flipud(data))
+
+
+def check_results(dm: CsstMsDataManager, logger: logging.Logger) -> bool:
+    """ Check whether processed data are generated. """
+    existence = [
+        os.path.exists(dm.l1_detector(detector=detector, post="L1_processed.fits"))
+        for detector in dm.target_detectors
+    ]
+    if all(existence):
+        return True
+    else:
+        return False
+
+
+# process a single image
+def process_single_image(
+        filename_input: str,
+        filename_output: str,
+        logger: Union[None, logging.Logger] = None
+) -> CsstStatus:
+    """
+    Flip a single image.
+
+    Flip a single image with ``numpy.fliplr`` and ``numpy.flipud``.
+
+    Parameters
+    ----------
+    filename_input : str
+        The input filename.
+    filename_output : str
+        The output filename.
+    logger : logging.Logger
+        The logger.
+
+    Returns
+    -------
+    CsstStatus
+        The final status.
+
+    Examples
+    --------
+    >>> process_single_image(
+    >>>     filename_input="input_image.fits",
+    >>>     filename_output="output_image.fits",
+    >>>     logger=None
+    >>> )
+    """
+    # set default logger
+    if logger is None:
+        logger = get_logger()
+
+    # process data
+    try:
+        # this will NOT be written into the log file
+        logger.debug("Reading the image {}".format(filename_input))
+        # start processing
+        data = read_image(filename_input)
+        data_processed = process_data(data)
+        np.save(filename_output, data_processed)
+        # this will be written into the log file
+        logger.info("Processed image saved to {}".format(filename_output))
+        return CsstStatus.PERFECT
+    except DeprecationWarning:
+        # this will be written into the log file
+        logger.warning("Suffered DeprecationWarning!")
+        return CsstStatus.WARNING
+    except IOError:
+        # this will be written into the log file
+        logger.error("Suffered IOError!")
+        return CsstStatus.ERROR
+
+
+# process an exposure (MBI or SLS)
+def process_multiple_images(
+        dm: CsstMsDataManager,
+        logger: Union[None, logging.Logger] = None
+) -> CsstStatus:
+    """
+    Flip all images.
+
+    Flip all images in an exposure in a for-loop (not in parallel).
+
+    Parameters
+    ----------
+    dm : CsstMsDataManager
+        The data manager of the specified exposure.
+    logger : {None, logging.Logger}
+        The logger. If None, use the default logger.
+
+    Returns
+    -------
+    CsstStatus
+        The final status.
+
+    Examples
+    --------
+    >>> dm = CsstMsDataManager.quickstart(
+    >>>     ver_sim="C5.2", dir_l1=".", datatype="sls", exposure_id=100)
+    >>> logger = get_logger()
+    >>> process_multiple_images(dm, logger)
+    """
+
+    # set default logger
+    if logger is None:
+        logger = get_logger()
+
+    # process data
+    try:
+        # dm.target_detectors is a list of detector number that should be processed
+        # start processing
+        for detector in dm.target_detectors:
+            # this will NOT be written into the log file
+            logger.debug("Processing for detector {}".format(detector))
+            data = read_image(dm.l0_detector(detector=detector))
+            data_processed = process_data(data)
+            np.save(
+                dm.l1_detector(detector=detector, post="L1_processed.fits"),
+                data_processed
+            )
+        # check results
+        if check_results(dm=dm, logger=logger):
+            # this will be written into the log file
+            logger.info("Processed all images")
+            return CsstStatus.PERFECT
+        else:
+            # not all images are properly processed
+            return CsstStatus.WARNING
+    except DeprecationWarning:
+        # this will be written into the log file
+        logger.warning("Suffered DeprecationWarning!")
+        return CsstStatus.WARNING
+    except IOError:
+        # this will be written into the log file
+        logger.error("Suffered IOError!")
+        return CsstStatus.ERROR

doc/source/index.rst

@@ -3,7 +3,7 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-CSST pipeline prototype
+CSST Pipeline prototype
 =======================
 
 .. meta::
@@ -16,41 +16,35 @@ CSST pipeline prototype
 
 
 .. toctree::
-   :hidden:
-   :maxdepth: 1
+   :maxdepth: 2
    :caption: CONTENTS
 
-Hello!
-Welcome to `csst_proto`_ -- the prototype of CSST L1 pipeline.
 
-The aim of this website is to provide an example of how to write a functional
-module for CSST DAS pipline.
+Hi!
+This is an internal guide for CSST DAS developers.
+The associated package, `csst_proto`_, is the prototype of CSST L1 pipeline.
+The aim of this document is to provide an example of how to build a functional
+quality module for CSST DAS pipline.
 Here also host the basic standards of CSST DAS pipelines as well as many details.
 
 CSST DAS pipelines are mainly based on `Python`_.
+All CSST Python code should be consistent with the PEP 8 style guide.
 Developers should follow the standard `Python Packaging User Guide`_ to package
-their algorithms.
+their codes.
 
 .. _Python Packaging User Guide: https://packaging.python.org/en/latest/
 .. _Python: https://www.python.org/
 .. _csst_proto: https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto
 
-*Dress for the job you want, not the one you have.*
-A good code repository should
-- Use version control (e.g., git)
-- Use unit tests
-- Have good documentation
-- Use good structure
 
-Although it might be difficult in practice, it should always be the ultimate goal
-of an excellent programmer.
+This guide is also highly influenced by the two below.
+
+The guide for astropy-affiliated package development
+   https://docs.astropy.org/en/latest/development/docguide.html
 
-Last but not least, two very useful websites are listed below for reference.
+The guide for LSST developers
+   https://developer.lsst.io/
 
-   - The guide for astropy-affiliated package development
-      - https://docs.astropy.org/en/latest/development/docguide.html
-   - The guide for LSST developers
-      - https://developer.lsst.io/
 
 .. toctree::
    :hidden:
@@ -63,11 +57,14 @@ Last but not least, two very useful websites are listed below for reference.
 .. toctree::
    :hidden:
    :maxdepth: 2
-   :caption: CODE STYLE GUIDES
+   :caption: GUIDES FOR DEVELOPERS
 
-   git.rst
+   vcs.rst
+   packaging.rst
    codestyle.rst
-   sphinxdoc.rst
+   unittest.rst
+   simulation.rst
+   csst_common/csst_common.rst
 
 
 .. toctree::
@@ -76,7 +73,6 @@ Last but not least, two very useful websites are listed below for reference.
    :caption: CI / CD
 
    cicd.rst
-   unittest.rst
 
 
 .. toctree::

doc/source/packaging.rst

+Python Packaging
+================
+Python projects should be packaged in the standard way as shown in the
+official `Python Packaging User Guide`_, and more specifically `here`_.
+
+.. _here: https://packaging.python.org/en/latest/tutorials/packaging-projects/
+.. _Python Packaging User Guide: https://packaging.python.org/en/latest/
+
+
+Project Structure
+-----------------
+An example code structure is shown below.
+Files/directories with asterisks (``*``) marks are optional.
+
+.. code-block::
+
+    csst_proto                                # the repository name
+    ├── csst_proto                            # the package name
+    │   ├── data                              # package associated data directory
+    │   ├── __init__.py                       # necessary file for a Python package
+    │   ├── demo.py                           # Python modules
+    │   ├── flip_image.py
+    │   ├── scratch.py
+    │   └── top_level_interface.py            # the top level interface module
+    ├── doc                                   # *sphinx-based documentation directory
+    │   ├── build
+    │   ├── source
+    │   ├── Makefile
+    │   ├── apidoc.sh
+    │   ├── contents.md
+    │   ├── make.bat
+    │   └── preview.sh
+    ├── examples                              # *example scripts
+    │   ├── how_this_code_will_be_used.py
+    │   └── how_to_write_docstring.py
+    ├── tests                                 # unit tests
+    │   ├── test_flip_image.py
+    │   └── test_other_functions.py
+    ├── LICENSE                               # license file
+    ├── README.md                             # README file (Markdown recommended)
+    ├── install.sh                            # a single-line installation script
+    ├── install_local.sh                      # local installation script
+    ├── readthedocs.yml                       # *configuration file for readthedocs
+    ├── requirements.txt                      # package dependencies
+    └── setup.py                              # setup file
+
+.. note::
+    The ``__init__.py`` files is important, without which
+    the directory will not be seen as a ``Python package``.
+
+Python version
+--------------
+
+Developers should use ``Python 3.9.X`` to implement algorithms.
+The base docker image will be ``continuumio/anaconda3``, which uses
+``Python 3.9.12``.
+
+However, in this example, ``python_requires='>=3.8'`` is because
+``readthedocs.io`` only supports ``Python 3.8.X``.
+
+
+Relative import
+---------------
+When import a class / function within the same package, a ``relative import``
+should be used instead of ``absolute import``.
+An example is below (``top_level_interface.py``):
+
+.. literalinclude:: ../../csst_proto/top_level_interface.py
+   :linenos:
+   :language: python
+
+.. note::
+    Note the ``.`` means it's a relative import.
+
+README.md
+---------
+
+A Markdown format text file to describe your package.
+
+.. literalinclude:: ../../README.md
+
+LICENSE
+-------
+
+Among the many choices, we encourage our developers to use ``MIT`` LICENSE.
+An example is below.
+
+.. literalinclude:: ../../LICENSE
+
+
+``requirements.txt``
+--------------------
+``requirements.txt`` is a text file, within which a line represents a package and its version.
+
+.. literalinclude:: ../../requirements.txt
+   :linenos:
+
+.. note::
+    For each package, developers must specify a version.
+    DO NOT use ``>``, ``>=``, ``<``, ``<=`` or ``~=``, use ``==`` ONLY!
+
+
+``setup.py``
+------------
+We use a classic format of ``setup.py`` file as shown below.
+We refer readers to https://setuptools.pypa.io/en/latest/ for more information
+and usages about the package `setuptools`_.
+
+.. _setuptools: https://setuptools.pypa.io/en/latest/
+
+.. literalinclude:: ../../setup.py
+   :linenos:
+   :language: python
+
+.. note::
+
+   The ``install_requires`` keyword argument is commented because it takes no effect
+   when installed with ``pip install <your_package> --no-deps``.
+   Please DO NOT use it.
+
+Include data
+------------
+
+Data files can be packaged together with code.
+To include ``csst_proto/data/test_image.txt`` and ``csst_proto/data/table_data.vsb``,
+set the keyword ``package_data`` in ``setup.py`` as below:
+
+.. code-block:: python
+
+    ...
+    package_data={"": ["LICENSE", "README.md"],
+                  "csst_proto": ["data/test_image.txt",
+                                 "data/table_data.csv"
+                                 ]},
+    ...
+
+.. note::
+    The ``""`` key represents the data files located directly in the project directory.
+    For files associated with package ``csst_proto``, link the data files to ``csst_proto`` key.
+
+.. note::
+    DO NOT include any test data (e.g., CSST images) in the package.
+    We only encourage developers to pacakge configuration files inside.
+    Please keep your package *clean* and *tight*.
+
+To access the data, developers should use ``relative path``.
+In the example below, we show the function ``read_test_image``,
+an example of how to access data under ``csst_proto/data/``.
+In ``__init__.py``:
+
+.. literalinclude:: ../../csst_proto/__init__.py
+    :linenos:
+    :language: python
+
+and in ``flip_image.py``:
+
+.. literalinclude:: ../../csst_proto/flip_image.py
+    :linenos:
+    :language: python
+
+
+``top_level_interface`` module
+------------------------------
+This is a special requirement for all of CSST DAS packages.
+This module is used to store the interfaces (classes / functions) that
+will be called by users.
+
+.. tip::
+    Remember to set the ``__all__`` variable.
+    This should be a list containing all interfaces.
+
+.. note::
+    All the interfaces in this module should use a complete ``Numpy``-style docstring
+    -- They will go through docstring format validations in Jenkins!
+
+
+How to install package
+----------------------
+We recommend the following way to install your package.
+
+.. code-block:: bash
+
+    # install requirements
+    pip install -r requirements
+    # build extensions in place
+    python setup.py build_ext --inplace
+    # build source code
+    python setup.py sdist
+    # install from the source code
+    pip install dist/*.tar.gz --force-reinstall --no-deps
+
+
+Sphinx-based documentation
+--------------------------
+This is currently regarded as an optional section.
+But we still recommend developers to read a bit of the ``sphinx`` documentation.
+
+The sphinx homepage:
+
+- https://www.sphinx-doc.org/en/master/index.html
+
+A tutorial on how to write restructured text (.rst) files:
+
+- https://docutils.sourceforge.io/rst.html
+

doc/source/simulation.rst

+Using simulation data
+=====================
+
+Versions
+--------
+In pipeline system, we use ``ver_sim`` keyword in code (e.g., ``CsstMsDataManager``).
+The ``ver_sim`` uses a ``CX.Y`` format string, where
+``X`` denotes the Cycle during which the simulation is released,
+and ``Y`` denotes the serial number.
+For example, ``C5.2`` represents for the second released version of simulation in Cycle 5.
+s
+
+Server details
+--------------
+
+Server
+   ``dandelion``.
+   The ``IP``, ``Username`` and ``Password`` are available via request to ``csst_das@nao.cas.cn``.
+Simulation data path
+   - ``C5.2``: ``/nfsdata/share/csst_simulation_data/Cycle-5-SimuData``
+   - ``C3``: ``/nfsdata/share/csst_simulation_data/Cycle-3-SimuData``
+
+.. note::
+    Presently, the ``CsstMsDataManager`` only supports ``C5.2`` simulation.

doc/source/team.rst

@@ -23,12 +23,8 @@ To fit the schedule of CSST, a software suite is underway, including
 - simulation
 - data pipeline
 
-The timeline is splitted into cycles, among which each with 6-month length
+The timeline is splitted into cycles, and each cycle spans 6 months.
 
 - C6 (2022.7-2022.12)
     - integrate L1 pipelines
 - C5 (2022.1-2022.6)
-- C4
-- C3
-- C2
-- C1

doc/source/unittest.rst

@@ -5,6 +5,19 @@ while `doctest`_ is not recommended for limited usage.
 In the end, `pytest`_ is used to perform the unittests.
 And `coverage`_ is used to assess the unit test coverage.
 
+Introduction to ``unittest``
+----------------------------
+Here we show an example of an example unit test on ``csst_proto.top_level_interface.flip_image``
+
+.. literalinclude:: ../../tests/test_flip_image.py
+   :linenos:
+   :language: python
+
+
+
+Run Unit Tests
+--------------
+
 To run unit tests
 
 .. code-block:: bash
@@ -43,3 +56,57 @@ the conventions on how `pytest`_ discovers unit tests (e.g, `test_*.py` or `*_te
 For more information, click `here`_.
 
 .. _here: https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#test-discovery
+
+
+Data access
+-----------
+For the Main Survey, i.e., the multi-band imaging (MBI) and the slit-less spectra (SLS)
+developers might use two kinds of data to run their unit tests.
+
+Auxiliary data (data used for your unit test ONLY)
+   It should be put in ``/nfsdata/users/csstpipeline/L1Pipeline/unittests/<your_project_name>/``
+   You can obtain this path from the environment in your unit test scirpts,
+
+   .. code-block:: python
+
+      import os
+      os.getenv("AUX_DIR")
+
+
+Simulation data (officially released simulation data)
+   It should be accessed with the ``csst_common.data_manager.CsstMsDataManager``.
+   The ``CsstMsDataManager.quickstart`` provides an interface to initialize
+   an instance. See the next chapter for more details.
+
+   Here we show an example which initializes the ``CsstMsDataManager`` with the 100th exposure of ``C5.2``
+   simulation to perform a unit test for the project ``csst_ms_sls_instrument``.
+
+   .. code-block:: python
+
+        import os
+        import unittest
+        from csst_common.data_manager import CsstMsDataManager
+        from csst_ms_sls_instrument.top_level_interface import sls_instrument_pipeline
+
+
+        aux_dir = os.getenv("AUX_DIR")
+        out_dir = os.path.join(aux_dir, "csst_ms_sls_instrument")
+
+
+        class TestCsstMsSlsInstrument(unittest.TestCase):
+            def setUp(self) -> None:
+                self.dm = CsstMsDataManager.quickstart(ver_sim="C5.2", datatype="sls", dir_l1=out_dir, exposure_id=100)
+
+            def test_instrument(self):
+                test_detector = 1
+                # for l0 image path:
+                self.dm.l0_detector(test_detector)
+                # for l1 image path:
+                self.dm.l1_detector(detector=test_detector, post="flt.fits")
+                self.dm.l1_file(name="merged.fits", comment="This is a merged image.")
+                self.assertTrue(os.path.exists(self.dm.l0_detector(test_detector)))
+
+
+
+For developers on other instruments (CPIC, THZ, MCI and IFS),
+unit tests might be performed in their local environments.

doc/source/vcs.rst

+Version Control
+===============
+
+A functional quality software needs a version control system,
+which provides several advantages.
+
+    1. Record versions
+        so that you can rollback the software to any history version.
+    2. Collaboration between developers
+        so that multiple developers can work simultaneously to enhance efficiency.
+    3. Keep multiple braches
+        so that software can be developed in a separate branch
+        and it does not affect the stable (released) version.
+    4. Code safety
+        the approve-merge mechanism, etc.
+
+`git`
+-----
+`git` is probably the most widely used version control system presently.
+
+    - official website: https://git-scm.com/
+    - **Pro Git** (2nd Ed.): https://git-scm.com/book/en/v2
+
+
+Basic Usages
+------------
+To initialize a repository:
+
+.. code-block:: bash
+
+    git init
+
+To add a remote source
+
+.. code-block:: bash
+
+    git remote add origin https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto.git
+
+To show a remote source `origin`
+
+.. code-block:: bash
+
+    git remote show origin
+
+To clone a remote repository
+
+.. code-block:: bash
+
+    git clone https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto.git
+
+To pull remote code to local
+
+.. code-block:: bash
+
+    git pull origin main
+
+To view the git log
+
+.. code-block:: bash
+
+    git log
+
+To view the current status
+
+.. code-block:: bash
+
+    git status
+
+To add your revisions to version control
+
+.. code-block:: bash
+
+    git add ./test.py
+
+To make a commit
+
+.. code-block:: bash
+
+    git commit -m "make a test commit"
+
+To push to remote repository
+
+.. code-block:: bash
+
+    git push origin main
+
+To reset repository to a specific commit
+
+.. code-block:: bash
+
+    git reset 82cc809698b3b52327a0360eea10b88aacbadec6 --hard
+
+To configure your git
+
+.. code-block:: bash
+
+    git config --global user.name "Your Name"
+    git config --global user.email "your_email@address"
+
+You can also store your credentials on disk via
+
+.. code-block:: bash
+
+    git config --global credential.helper store
+
+
+.. note::
+
+    Please DO NOT use command ``git add .`` which makes the Version Control system
+    track all files in the repository -- It may include many useless files such as
+    ``*.py~`` (temporary files generated by text editor).
+
+
+Work Flow with Git
+------------------
+In CSST DAS pipeline development, each developer registers an account at the
+`CSST DAS GitLab`_, which is a self self-managed `GitLab`_ instance
+to host data processing pipelines at NAOC.
+More specifically, the L1 Pipelines are developed in the `csst-l1`_ group.
+
+Developers are required to upload their code to the repository.
+The repository should have three branches:
+
+- ``main`` branch: for a stable version
+- ``dev`` branch: for development
+- ``release`` branch: for code release
+
+After the pipeline is integrated (probably the end of C6),
+a code reviewer is required to merge approved code revisions into ``main`` branch.
+
+
+.. _CSST DAS GitLab: https://csst-tb.bao.ac.cn/code
+.. _GitLab: https://about.gitlab.com/
+.. _csst-l1: https://csst-tb.bao.ac.cn/code/csst-l1