csst_proto
CSST L1 pipeline prototype.
homepage
This code is available at CSST GitLab:
documentation
A sphinx-based documentation is available at
dependency
- numpy~=1.20.3
- joblib~=1.1.0
- astropy~=5.1
installation
csst_proto
can be installed with the following shell command
git clone https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto.git
cd csst_proto
pip install -r requirements.txt
python setup.py install
or a single-line command
sh -c "$(curl -fsSL https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto/-/raw/main/install.sh)"
instruction
A simple example on how to use this package:
from csst_proto.top_level_interface import flip_image
from astropy.io import fits
# read an L0 image
img = fits.getdata("CSST_MS_SCI_06_L0_img.fits")
# flip the image
img_flipped = flip_image(img=img)
algorithm description
The csst_proto
uses an awesome algorithm based on numpy.ndarray
indices to flip images.
git
: connecting collaborators
A very brief example of using git
:
# global setting for git (name and email address)
git config --global user.name "BO ZHANG"
git config --global user.email "bozhang@nao.cas.cn"
# initialize a repository with main branch
git init --initial-branch=main
# clone a remote repository
git clone https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto.git
# add some_module.py to version control system
git add ./some_module.py
# commit the changes
git commit -m "added some_module"
# add remote repository link
git remote add origin https://csst-tb.bao.ac.cn/code/csst-l1/csst_proto.git
# push changes to remote repository
git push origin main
# pull remote updates
git pull origin main
timeline
file / directory | description | urgent | C6 | C7 |
---|---|---|---|---|
csst_proto/ |
source code & data | √√ | √ | |
doc/ |
sphinx-based documentation | √ | ||
example/ |
code examples | - | - | - |
tests/ |
unit tests | √ | ||
.gitignore |
git file | √ | ||
.gitlab-ci.yml |
gitlab CI/CD pipeline | √ | ||
LICENSE |
code license | √ | ||
README.md |
readme file | √√ | √ | |
install.sh |
single-line installation script | √ | ||
install_local.sh |
local installation script | √ | ||
readthedocs.yml |
readthedocs.io configuration | √ | ||
requirements.txt |
code requirements | √ | √ | |
setup.py |
sphinx-based documentation | √√ | √ |
important stages
- stage 1:
- finished main code
- good code style
-
top_level_interface
available
- stage 2:
- finished
setup.py
- can be installed with
pip install <package_name/package_link>
from gitlab
- finished
- stage 3:
- complete docstrings for functions & classes
- finished instruction
README.md
- stage 4:
- finished unittest
- local unit test passed with
pytest
- stage 5:
- gitlab CI/CD pipeline available
- auto unit tests passed with
.gitlab-ci.yml
- stage 6:
- sphinx-based documentation available
doc/
readthedocs.yml
progress
module name | stage 1 (source code) |
stage 2 (packaging) |
stage 3 (instruction) |
stage 4 (unit tests) |
stage 5 (gitlab pipeline) |
stage 6 (sphinx doc) |
---|---|---|---|---|---|---|
csst_proto |
√ | √ | √ | √ | √ | √ |
csst_common |
√ | √ | √ | |||
csst_l1 |
√ | √ | √ | |||
csst_ms_mbi_instrument |
√ | √ | ||||
csst_ms_mbi_distortion |
√ | √ | ||||
csst_ms_mbi_position |
√ | √ | ||||
csst_ms_mbi_flux |
√ | √ | ||||
csst_ms_mbi_photometry |
√ | √ | ||||
csst_ms_sls_instrument |
√ | |||||
csst_ms_sls_mosaic |
√ | |||||
csst_ms_sls_position |
√ | |||||
csst_ms_sls_directimage |
√ ️ | |||||
csst_ms_sls_sky |
√ | |||||
csst_ms_sls_objextraction |
√ | |||||
csst_ms_sls_axe |
√ | |||||
csst_ms_sls_cde |
√ | |||||
csst_ms_qc0 |
√ | |||||
csst_cpic |
√ | |||||
csst_thz_dp1 |
√ | |||||
csst_mci_distortion |
√ | |||||
csst_mci_instrument |
√ | |||||
csst_mci_astrometry |
√ | |||||
csst_mci_flux |
√ | |||||
csst_mci_photometry |
√ | |||||
csst_ifs_rss |
√ | |||||
csst_ifs_cube |
√ | |||||
csst_ifs_wcs |
√ |
useful links
- python packaging user guide: https://packaging.python.org/en/latest/
- packaging binary extensions (C/C++): https://packaging.python.org/en/latest/guides/packaging-binary-extensions
-
setuptools
: https://packaging.python.org/en/latest/tutorials/installing-packages/ - recommended IDEs
- PyCharm: https://www.jetbrains.com/pycharm/
- VS Code: https://code.visualstudio.com/
-
git
: https://git-scm.com/- Pro Git: https://git-scm.com/book/en/v2
- Numpydoc: https://numpydoc.readthedocs.io/en/latest/format.html
- for C: https://peps.python.org/pep-0007/
- for Python: https://peps.python.org/pep-0008/
-
unittest
: https://docs.python.org/3.8/library/unittest.html - gitlab CI/CD pipeline: https://docs.gitlab.com/ee/ci/pipelines/
- gitlab runners: https://docs.gitlab.com/runner/
-
sphinx
: https://www.sphinx-doc.org/en/master/index.html - markdown: https://www.markdownguide.org/
- restructured text: https://docutils.sourceforge.io/rst.html
- readthedocs: https://readthedocs.io/ or https://rtfd.io/
data manager
for MS MBI
import os
from csst_common.data_manager import CsstMbiDataManager
from csst_common.params import CSST_PARAMS
# initialize data manager
dm = CsstMbiDataManager(
ver_sim="C5.1", dir_l0="/data/sim_data/MSC_0000100", dir_l1="/home/user/L1Pipeline/msc/work")
# process all CCDs
dm.set_detectors()
print("----- available detectors -----")
print(dm.available_detectors)
for detector in dm.target_detectors:
print("----- L0 images -----")
print(dm.l0_detector(detector=detector))
print(os.path.exists(dm.l0_ccd(detector=detector)))
print("----- L0 crs -----")
print(dm.l0_crs(detector=detector))
print(os.path.exists(dm.l0_ccd(detector=detector)))
print("----- L0 input cat -----")
print(dm.l0_cat(detector=detector))
print(os.path.exists(dm.l0_cat(detector=detector)))
print("----- L0 input log -----")
print(dm.l0_log(detector=detector))
print(os.path.exists(dm.l0_log(detector=detector)))
print("----- L1 images -----")
print(dm.l1_detector(detector, post="img.fits"))
print("----- L1 file -----")
print(dm.l1_file("flipped_image.fits"))