diff --git a/doc/source/api/csst_proto.rst b/doc/source/api/csst_proto.rst index 7afaaf1d4e44fa75c1da6ed1b750fe988ea3033b..1e73d515718df0d7b566c82bbe8a814a7688f16c 100644 --- a/doc/source/api/csst_proto.rst +++ b/doc/source/api/csst_proto.rst @@ -1,26 +1,6 @@ API === - -csst\_proto.flip\_image ------------------------ - -.. automodule:: csst_proto.flip_image - :members: - :undoc-members: - :show-inheritance: - -csst\_proto.some\_other\_modules --------------------------------- - -.. automodule:: csst_proto.some_other_modules - :members: - :undoc-members: - :show-inheritance: - -csst\_proto.top\_level\_interface ---------------------------------- - .. automodule:: csst_proto.top_level_interface :members: :undoc-members: diff --git a/doc/source/cicd.rst b/doc/source/cicd.rst new file mode 100644 index 0000000000000000000000000000000000000000..4a378f6bda6335eaf227671741060d6e0505eb70 --- /dev/null +++ b/doc/source/cicd.rst @@ -0,0 +1,33 @@ +Continuous Integration +====================== +The Continuous Integration is based on Jenkins. +A push to git repository will trigger a build for the package on Jenkins platform. +Then an email with a build log and unit test coverage report will be delivered to +the relavant developer. + +The build log includes several sections on + +1. a check on versions of requirements +2. a check on installation of requirements +3. a check on installation of package +4. a check on import of interfaces (top_level_interface) +5. unit tests and coverage +6. Numpydoc validation of interfaces + + +An example of test summary is at the end of build log. +If all checks/tests/validations passed, you will see a section as below. + +.. code-block:: + + ===================== TEST SUMMARY ===================== + Test passed! + Check requirements: 0 + Install requirements: 0 + Install package: 0 + Importability status: 0 + Unit test status: 0 + Numpydoc status: 0 + + + diff --git a/doc/source/codestyle.rst b/doc/source/codestyle.rst index 83f02f330004ef484fbf1038a9176947489a3847..b57903f708fbfe76ca2957107472f646ab40ca42 100644 --- a/doc/source/codestyle.rst +++ b/doc/source/codestyle.rst @@ -1,39 +1,67 @@ -Numpydoc -======== +Code Structure +============== +An example code structure is shown below. +Files/directories with asterisks (*) marks are optional. -More specifically, +.. code-block:: -- PEP 7 – Style Guide for C Code: https://peps.python.org/pep-0007/ -- PEP 8 – Style Guide for Python Code: https://peps.python.org/pep-0008/ + csst_proto # the repository name + ├── csst_proto # the package name + │ ├── data # package associated data directory + │ ├── __init__.py # necessary file for a Python package + │ ├── flip_image.py # Python modules + │ ├── scratch.py + │ ├── some_other_modules.py + │ └── top_level_interface.py # the top level interface module + ├── doc # *sphinx-based documentation directory + │ ├── build + │ ├── source + │ ├── Makefile + │ ├── apidoc.sh + │ ├── contents.md + │ ├── make.bat + │ └── preview.sh + ├── examples # *example scripts + │ ├── how_this_code_will_be_used.py + │ └── how_to_write_docstring.py + ├── tests # unit tests + │ ├── test_flip_image.py + │ └── test_other_functions.py + ├── LICENSE # license file + ├── README.md # README file (Markdown recommended) + ├── install.sh # a single-line installation script + ├── install_local.sh # local installation script + ├── readthedocs.yml # *configuration file for readthedocs + ├── requirements.txt # package dependencies + └── setup.py # setup file +Code Style +========== +Python code should follow PEP 8 + - https://peps.python.org/pep-0008/ +Whereas C/C++ code should follow PEP 7 + - https://peps.python.org/pep-0007/ -Numpydoc style, conf https://numpydoc.readthedocs.io/en/latest/format.html + +Numpy-style docstring +===================== +There are several popular docstring styles, namely Google, reStructuredText, and Numpy. +The CSST DAS adopts Numpydoc-style among the three. +We refer our developers to the Numpydoc official style guide for instructions on +how to write docstrings. + - https://numpydoc.readthedocs.io/en/latest/format.html +In particular, a complete example is available here. + - https://numpydoc.readthedocs.io/en/latest/example.html Markup language =============== +Markdown and reStructrured Text are two of popular markup languages. +reStructrured Text is used in Numpydoc docstrings as well as sphinx-based documentation. +Markdown is much easier and usually used to write `README` for a package. -标记语言(Markup language) 可以分为三类 - - - 展示标记(Presentational markup) - what you see is what you get - - 步骤标记(Procedural markup) - e.g., Markdown, TeX - - 描述标记(Descriptive markup) - e.g., LaTex, HTML, XML, ... - -我们着重介绍两种在Python编程中被广泛运用的标记语言——Markdown和reStructured Text。 - - 1. Markdown - Markdown可能是最轻量级的可用于层次化展示的标记语言， - 用户只需要学习为数不多的规则即可开始使用， - 并且在github等各种环境中均有插件支持。 - 通常可以使用Markdown格式编写程序说明(`README.md`) - 2. reStructured Text - - - +- Markdown + - https://www.markdownguide.org/ - reStructured Text - https://www.writethedocs.org/guide/writing/reStructuredText/ -- \ No newline at end of file + - https://www.writethedocs.org/guide/writing/reStructuredText/ diff --git a/doc/source/conf.py b/doc/source/conf.py index daeb4f6d77c89dae5265f4394da631854c9c5898..ffc100de21da685f2891c0684ce47229d58bd284 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -63,4 +63,4 @@ latex_engine = 'xelatex' latex_use_xindy = False latex_elements = { 'preamble': '\\usepackage[UTF8]{ctex}\n', -} \ No newline at end of file +} diff --git a/doc/source/git.rst b/doc/source/git.rst index 2014ae63e178bb292a83988a4a68be32bbb49d25..ad9edef217c70f5d1efaa3479d45eed4f45f28fa 100644 --- a/doc/source/git.rst +++ b/doc/source/git.rst @@ -1,5 +1,5 @@ -Why version control? -==================== +Version Control +=============== A functional quality software needs a version control system, which provides several advantages. @@ -15,15 +15,18 @@ which provides several advantages. the approve-merge mechanism, etc. `git` -===== +----- `git` is probably the most widely used version control system presently. - official website: https://git-scm.com/ - **Pro Git** (2nd Edition)：https://git-scm.com/book/en/v2 - CSST DAS GitLab -=============== -CSST DAS uses a self-managed `gitlab` instance to host data processing pipelines at https://csst-tb.bao.ac.cn/code/. +--------------- +CSST DAS uses a self-managed `gitlab` instance to host data processing pipelines at + +- https://csst-tb.bao.ac.cn/code + +The Level 1 pipeline is host in `csst-l1` group: -The Level 1 pipeline is host in `csst-l1` group: https://csst-tb.bao.ac.cn/code/csst-l1. +- https://csst-tb.bao.ac.cn/code/csst-l1 diff --git a/doc/source/index.rst b/doc/source/index.rst index e745f1a0330df567a1a7b30e1972caf658ef8927..25503faef15f644e5c77e056157bc1c9bcbe6683 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -20,64 +20,52 @@ CSST pipeline prototype :maxdepth: 1 :caption: csst_proto -Hello! Welcome to `csst_proto` -- the CSST L1 pipeline prototype. +Hello! +Welcome to `csst_proto`_ -- the prototype of CSST L1 pipeline. -欢迎大家来到 `csst_proto` 的文档，这里是CSST一级流水线原型的文档。 +The aim of this website is to provide an example of how to write a functional +module for CSST DAS pipline. +Here also host the basic standards of CSST DAS pipelines as well as many details. -`csst_proto` 是一个基于 `Python` 的程序包，目的是为一级流水线功能模块开发提供打包/封装的参考格式。 -程序源码以及文档的源码都可以在CSST Gitlab中找到: https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto 。 +CSST DAS pipelines are mainly based on `Python`_. +Developers should follow the standard `Python Packaging User Guide`_ to package +their algorithms. -引用一句英语谚语："Dress for the job you want, not the one you have." -一个优秀的程序库（repository）应该有许多理想的特征，包含但不限于： +.. _Python Packaging User Guide: https://packaging.python.org/en/latest/ +.. _Python: https://www.python.org/ +.. _csst_proto: https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto -- Use version control （使用版本控制，比如 `git`） -- Use tests （测试/单元测试） -- Have good documentation （有好的文档/注释） -- Use good structure （有好的结构） +*Dress for the job you want, not the one you have.* +A good code repository should +- Use version control (e.g., git) +- Use unit tests +- Have good documentation +- Use good structure -尽管要在一个大的project中保持好的结构可能非常难，但这一定是优秀开发者的最求的目标。 +Although it might be difficult in practice, it should always be the ultimate goal +of an excellent programmer. +Last but not least, two very useful websites are listed below for reference. -这里的主要内容包括 - -- 版本控制 (version control) - - how to use `git` -- 代码风格 (Code style) - - PEP8 for Python - - PEP7 for C - - Numpydoc -- 单元测试 (unit test) - - `unittest` - - `doctest` - - gitlab CI/CD pipeline -- 文件结构 (file structures) - - CSST一级流水线功能模块命名 - - -- 开发时间线 (timeline for CSST developers) - - C6 - - C7 - - -最后放两个值得 CSST 团队学习的网站 - - astropy"旗下"的程序包开发指南：https://docs.astropy.org/en/latest/development/docguide.html - - LSST团队的内部开发指南：https://developer.lsst.io/ - -希望这个网站对大家的开发有帮助！ - + - The guide for astropy-affiliated package development + - https://docs.astropy.org/en/latest/development/docguide.html + - The guide for LSST developers + - https://developer.lsst.io/ .. toctree:: :hidden: :maxdepth: 2 - :caption: version control + :caption: TEAM - git.rst + team.rst .. toctree:: :hidden: :maxdepth: 2 - :caption: code style + :caption: CODE STYLE GUIDES + git.rst codestyle.rst sphinxdoc.rst @@ -85,8 +73,9 @@ Hello! Welcome to `csst_proto` -- the CSST L1 pipeline prototype. .. toctree:: :hidden: :maxdepth: 2 - :caption: unittest + :caption: CI / CD + cicd.rst unittest.rst @@ -95,4 +84,4 @@ Hello! Welcome to `csst_proto` -- the CSST L1 pipeline prototype. :maxdepth: 2 :caption: API - api/csst_proto.rst \ No newline at end of file + api/csst_proto.rst diff --git a/doc/source/sphinxdoc.rst b/doc/source/sphinxdoc.rst index 4a795a95ae417fb98d5b2404a4458d5e5d99ece1..a387c53f80169696d78098e3ca6504f52b156647 100644 --- a/doc/source/sphinxdoc.rst +++ b/doc/source/sphinxdoc.rst @@ -1,8 +1,8 @@ sphinx-based documentation ========================== -sphinx homepage: -https://www.sphinx-doc.org/en/master/index.html +The sphinx homepage: + - https://www.sphinx-doc.org/en/master/index.html -a tutorial on how to write restructured text (.rst) files: -https://docutils.sourceforge.io/rst.html +A tutorial on how to write restructured text (.rst) files: + - https://docutils.sourceforge.io/rst.html diff --git a/doc/source/team.rst b/doc/source/team.rst new file mode 100644 index 0000000000000000000000000000000000000000..ac75ae897fe2d94cccd346e80df5b907d08b700e --- /dev/null +++ b/doc/source/team.rst @@ -0,0 +1,34 @@ +CSST DAS +======== + +The `CSST`_ (Chinese Space Station Telescope) is a 2-meter space-based telescope +which is scheduled to be launched at December 2024. +The **CSST DAS** (Data Analysis System), led by Chao Liu, is the official team at +NAOC (National Astronomical Observatories, CAS) to build the data +reduction pipelines. + +.. _CSST: https://en.wikipedia.org/wiki/Xuntian + + +Facility resources +------------------ +Here should be a link to the CSST website + + +Timeline +-------- +To fit the schedule of CSST, a software suite is underway, including + +- survey schedule +- simulation +- data pipeline + +The timeline is splitted into cycles, among which each with 6-month length + +- C6 (2022.7-2022.12) + - integrate L1 pipelines +- C5 (2022.1-2022.6) +- C4 +- C3 +- C2 +- C1 diff --git a/doc/source/unittest.rst b/doc/source/unittest.rst index b54e57563d072c4cf080f085a363b6c3bdfe3e30..351adabe396d5f9201c50013c3d98f2ca5a54820 100644 --- a/doc/source/unittest.rst +++ b/doc/source/unittest.rst @@ -1,16 +1,45 @@ -Useful packages -=============== +Unit Tests +========== +Unit tests should be based on `unittest`_, +while `doctest`_ is not recommended for limited usage. +In the end, `pytest`_ is used to perform the unittests. +And `coverage`_ is used to assess the unit test coverage. -use +To run unit tests -- `unittest` (https://docs.python.org/3.8/library/unittest.html) -- `doctest` -- `pytest` (https://docs.pytest.org/en/stable/) -- `coverage` +.. code-block:: bash + pytest --import-mode=importlib -GitLab CI/CD pipeline -===================== +To run unit tests with coverage -自动化流水线。 +.. code-block:: bash + coverage run -m pytest --import-mode=importlib + +To display the coverage report + +.. code-block:: bash + + coverage report + +To generate the coverage report in html format + +.. code-block:: bash + + coverage html + +.. _unittest: https://docs.python.org/3.9/library/unittest.html +.. _doctest: https://docs.python.org/3.9/library/doctest.html +.. _pytest: https://docs.pytest.org/en/7.1.x +.. _coverage: https://coverage.readthedocs.io/en/6.5.0/ + + +Conventions +----------- +Before write his own unit test scripts, a developer should have a glance at +the conventions on how `pytest`_ discovers unit tests (e.g, `test_*.py` or `*_test.py` files). + +For more information, click `here`_. + +.. _here: https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#test-discovery