Commit 0777aaab authored by BO ZHANG's avatar BO ZHANG 🏀
Browse files

updated rtfd docs

parent 917028ef
Pipeline #179 passed with stages
in 14 seconds
API
===
csst\_proto.flip\_image
-----------------------
.. automodule:: csst_proto.flip_image
:members:
:undoc-members:
:show-inheritance:
csst\_proto.some\_other\_modules
--------------------------------
.. automodule:: csst_proto.some_other_modules
:members:
:undoc-members:
:show-inheritance:
csst\_proto.top\_level\_interface
---------------------------------
.. automodule:: csst_proto.top_level_interface
:members:
:undoc-members:
......
Continuous Integration
======================
The Continuous Integration is based on Jenkins.
A push to git repository will trigger a build for the package on Jenkins platform.
Then an email with a build log and unit test coverage report will be delivered to
the relavant developer.
The build log includes several sections on
1. a check on versions of requirements
2. a check on installation of requirements
3. a check on installation of package
4. a check on import of interfaces (top_level_interface)
5. unit tests and coverage
6. Numpydoc validation of interfaces
An example of test summary is at the end of build log.
If all checks/tests/validations passed, you will see a section as below.
.. code-block::
===================== TEST SUMMARY =====================
Test passed!
Check requirements: 0
Install requirements: 0
Install package: 0
Importability status: 0
Unit test status: 0
Numpydoc status: 0
Numpydoc
========
Code Structure
==============
An example code structure is shown below.
Files/directories with asterisks (*) marks are optional.
More specifically,
.. code-block::
- PEP 7 – Style Guide for C Code: https://peps.python.org/pep-0007/
- PEP 8 – Style Guide for Python Code: https://peps.python.org/pep-0008/
csst_proto # the repository name
├── csst_proto # the package name
│ ├── data # package associated data directory
│ ├── __init__.py # necessary file for a Python package
│ ├── flip_image.py # Python modules
│ ├── scratch.py
│ ├── some_other_modules.py
│ └── top_level_interface.py # the top level interface module
├── doc # *sphinx-based documentation directory
│ ├── build
│ ├── source
│ ├── Makefile
│ ├── apidoc.sh
│ ├── contents.md
│ ├── make.bat
│ └── preview.sh
├── examples # *example scripts
│ ├── how_this_code_will_be_used.py
│ └── how_to_write_docstring.py
├── tests # unit tests
│ ├── test_flip_image.py
│ └── test_other_functions.py
├── LICENSE # license file
├── README.md # README file (Markdown recommended)
├── install.sh # a single-line installation script
├── install_local.sh # local installation script
├── readthedocs.yml # *configuration file for readthedocs
├── requirements.txt # package dependencies
└── setup.py # setup file
Code Style
==========
Python code should follow PEP 8
- https://peps.python.org/pep-0008/
Whereas C/C++ code should follow PEP 7
- https://peps.python.org/pep-0007/
Numpydoc style, conf https://numpydoc.readthedocs.io/en/latest/format.html
Numpy-style docstring
=====================
There are several popular docstring styles, namely Google, reStructuredText, and Numpy.
The CSST DAS adopts Numpydoc-style among the three.
We refer our developers to the Numpydoc official style guide for instructions on
how to write docstrings.
- https://numpydoc.readthedocs.io/en/latest/format.html
In particular, a complete example is available here.
- https://numpydoc.readthedocs.io/en/latest/example.html
Markup language
===============
Markdown and reStructrured Text are two of popular markup languages.
reStructrured Text is used in Numpydoc docstrings as well as sphinx-based documentation.
Markdown is much easier and usually used to write `README` for a package.
标记语言(Markup language) 可以分为三类
- 展示标记(Presentational markup)
what you see is what you get
- 步骤标记(Procedural markup)
e.g., Markdown, TeX
- 描述标记(Descriptive markup)
e.g., LaTex, HTML, XML, ...
我们着重介绍两种在Python编程中被广泛运用的标记语言——Markdown和reStructured Text。
1. Markdown
Markdown可能是最轻量级的可用于层次化展示的标记语言,
用户只需要学习为数不多的规则即可开始使用,
并且在github等各种环境中均有插件支持。
通常可以使用Markdown格式编写程序说明(`README.md`)
2. reStructured Text
- Markdown
- https://www.markdownguide.org/
- reStructured Text
https://www.writethedocs.org/guide/writing/reStructuredText/
-
\ No newline at end of file
- https://www.writethedocs.org/guide/writing/reStructuredText/
Why version control?
====================
Version Control
===============
A functional quality software needs a version control system,
which provides several advantages.
......@@ -15,15 +15,18 @@ which provides several advantages.
the approve-merge mechanism, etc.
`git`
=====
-----
`git` is probably the most widely used version control system presently.
- official website: https://git-scm.com/
- **Pro Git** (2nd Edition):https://git-scm.com/book/en/v2
CSST DAS GitLab
===============
CSST DAS uses a self-managed `gitlab` instance to host data processing pipelines at https://csst-tb.bao.ac.cn/code/.
---------------
CSST DAS uses a self-managed `gitlab` instance to host data processing pipelines at
- https://csst-tb.bao.ac.cn/code
The Level 1 pipeline is host in `csst-l1` group:
The Level 1 pipeline is host in `csst-l1` group: https://csst-tb.bao.ac.cn/code/csst-l1.
- https://csst-tb.bao.ac.cn/code/csst-l1
......@@ -20,64 +20,52 @@ CSST pipeline prototype
:maxdepth: 1
:caption: csst_proto
Hello! Welcome to `csst_proto` -- the CSST L1 pipeline prototype.
Hello!
Welcome to `csst_proto`_ -- the prototype of CSST L1 pipeline.
欢迎大家来到 `csst_proto` 的文档,这里是CSST一级流水线原型的文档。
The aim of this website is to provide an example of how to write a functional
module for CSST DAS pipline.
Here also host the basic standards of CSST DAS pipelines as well as many details.
`csst_proto` 是一个基于 `Python` 的程序包,目的是为一级流水线功能模块开发提供打包/封装的参考格式。
程序源码以及文档的源码都可以在CSST Gitlab中找到: https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto 。
CSST DAS pipelines are mainly based on `Python`_.
Developers should follow the standard `Python Packaging User Guide`_ to package
their algorithms.
引用一句英语谚语:"Dress for the job you want, not the one you have."
一个优秀的程序库(repository)应该有许多理想的特征,包含但不限于:
.. _Python Packaging User Guide: https://packaging.python.org/en/latest/
.. _Python: https://www.python.org/
.. _csst_proto: https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto
- Use version control (使用版本控制,比如 `git`)
- Use tests (测试/单元测试)
- Have good documentation (有好的文档/注释)
- Use good structure (有好的结构)
*Dress for the job you want, not the one you have.*
A good code repository should
- Use version control (e.g., git)
- Use unit tests
- Have good documentation
- Use good structure
尽管要在一个大的project中保持好的结构可能非常难,但这一定是优秀开发者的最求的目标。
Although it might be difficult in practice, it should always be the ultimate goal
of an excellent programmer.
Last but not least, two very useful websites are listed below for reference.
这里的主要内容包括
- 版本控制 (version control)
- how to use `git`
- 代码风格 (Code style)
- PEP8 for Python
- PEP7 for C
- Numpydoc
- 单元测试 (unit test)
- `unittest`
- `doctest`
- gitlab CI/CD pipeline
- 文件结构 (file structures)
- CSST一级流水线功能模块命名
-
- 开发时间线 (timeline for CSST developers)
- C6
- C7
最后放两个值得 CSST 团队学习的网站
- astropy"旗下"的程序包开发指南:https://docs.astropy.org/en/latest/development/docguide.html
- LSST团队的内部开发指南:https://developer.lsst.io/
希望这个网站对大家的开发有帮助!
- The guide for astropy-affiliated package development
- https://docs.astropy.org/en/latest/development/docguide.html
- The guide for LSST developers
- https://developer.lsst.io/
.. toctree::
:hidden:
:maxdepth: 2
:caption: version control
:caption: TEAM
git.rst
team.rst
.. toctree::
:hidden:
:maxdepth: 2
:caption: code style
:caption: CODE STYLE GUIDES
git.rst
codestyle.rst
sphinxdoc.rst
......@@ -85,8 +73,9 @@ Hello! Welcome to `csst_proto` -- the CSST L1 pipeline prototype.
.. toctree::
:hidden:
:maxdepth: 2
:caption: unittest
:caption: CI / CD
cicd.rst
unittest.rst
......
sphinx-based documentation
==========================
sphinx homepage:
https://www.sphinx-doc.org/en/master/index.html
The sphinx homepage:
- https://www.sphinx-doc.org/en/master/index.html
a tutorial on how to write restructured text (.rst) files:
https://docutils.sourceforge.io/rst.html
A tutorial on how to write restructured text (.rst) files:
- https://docutils.sourceforge.io/rst.html
CSST DAS
========
The `CSST`_ (Chinese Space Station Telescope) is a 2-meter space-based telescope
which is scheduled to be launched at December 2024.
The **CSST DAS** (Data Analysis System), led by Chao Liu, is the official team at
NAOC (National Astronomical Observatories, CAS) to build the data
reduction pipelines.
.. _CSST: https://en.wikipedia.org/wiki/Xuntian
Facility resources
------------------
Here should be a link to the CSST website
Timeline
--------
To fit the schedule of CSST, a software suite is underway, including
- survey schedule
- simulation
- data pipeline
The timeline is splitted into cycles, among which each with 6-month length
- C6 (2022.7-2022.12)
- integrate L1 pipelines
- C5 (2022.1-2022.6)
- C4
- C3
- C2
- C1
Useful packages
===============
Unit Tests
==========
Unit tests should be based on `unittest`_,
while `doctest`_ is not recommended for limited usage.
In the end, `pytest`_ is used to perform the unittests.
And `coverage`_ is used to assess the unit test coverage.
use
To run unit tests
- `unittest` (https://docs.python.org/3.8/library/unittest.html)
- `doctest`
- `pytest` (https://docs.pytest.org/en/stable/)
- `coverage`
.. code-block:: bash
pytest --import-mode=importlib
GitLab CI/CD pipeline
=====================
To run unit tests with coverage
自动化流水线。
.. code-block:: bash
coverage run -m pytest --import-mode=importlib
To display the coverage report
.. code-block:: bash
coverage report
To generate the coverage report in html format
.. code-block:: bash
coverage html
.. _unittest: https://docs.python.org/3.9/library/unittest.html
.. _doctest: https://docs.python.org/3.9/library/doctest.html
.. _pytest: https://docs.pytest.org/en/7.1.x
.. _coverage: https://coverage.readthedocs.io/en/6.5.0/
Conventions
-----------
Before write his own unit test scripts, a developer should have a glance at
the conventions on how `pytest`_ discovers unit tests (e.g, `test_*.py` or `*_test.py` files).
For more information, click `here`_.
.. _here: https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#test-discovery
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment