Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
csst-pipeline
csst_proto
Commits
d9f64d35
Commit
d9f64d35
authored
Oct 11, 2022
by
BO ZHANG
🏀
Browse files
added code preference
parent
9f37de02
Pipeline
#219
passed with stages
in 18 seconds
Changes
5
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
doc/source/csst_common/example_interface.py
View file @
d9f64d35
...
@@ -98,7 +98,7 @@ def process_single_image(
...
@@ -98,7 +98,7 @@ def process_single_image(
# process multiple images in an exposure (RECOMMENDED, at least for MBI or SLS)
# process multiple images in an exposure (RECOMMENDED, at least for MBI or SLS)
# define a single job
# define a single job
def
one_job
(
dm
,
detector
):
def
one_job
(
dm
:
CsstMsDataManager
,
detector
:
int
):
""" Process a single image, defined for parallel processing. """
""" Process a single image, defined for parallel processing. """
filepath_input
=
dm
.
l0_detector
(
detector
=
detector
)
filepath_input
=
dm
.
l0_detector
(
detector
=
detector
)
filepath_output
=
dm
.
l1_detector
(
detector
=
detector
,
post
=
"L1_processed.fits"
)
filepath_output
=
dm
.
l1_detector
(
detector
=
detector
,
post
=
"L1_processed.fits"
)
...
@@ -108,7 +108,7 @@ def one_job(dm, detector):
...
@@ -108,7 +108,7 @@ def one_job(dm, detector):
return
return
#
parallel
processin
g jobs
# process
in
serial / parallel
def
process_multiple_images
(
def
process_multiple_images
(
dm
:
CsstMsDataManager
,
dm
:
CsstMsDataManager
,
logger
:
Union
[
None
,
logging
.
Logger
]
=
None
,
logger
:
Union
[
None
,
logging
.
Logger
]
=
None
,
...
...
doc/source/index.rst
View file @
d9f64d35
...
@@ -62,7 +62,7 @@ The guide for LSST developers
...
@@ -62,7 +62,7 @@ The guide for LSST developers
vcs.rst
vcs.rst
packaging.rst
packaging.rst
codestyle.rst
codestyle.rst
p
ackages/packages
.rst
p
reference/preference
.rst
unittest.rst
unittest.rst
simulation.rst
simulation.rst
csst_common/csst_common.rst
csst_common/csst_common.rst
...
...
doc/source/p
ackages
/example_joblib.py
→
doc/source/p
reference
/example_joblib.py
View file @
d9f64d35
File moved
doc/source/p
ackages
/example_multiprocessing.py
→
doc/source/p
reference
/example_multiprocessing.py
View file @
d9f64d35
File moved
doc/source/p
ackages/packages
.rst
→
doc/source/p
reference/preference
.rst
View file @
d9f64d35
Packag
e
p
reference
Cod
e
P
reference
===============
===
===============
Initially we want our developers to following the
Initially we want our developers to following the
`coding guidelines for astropy-affiliated packages <https://docs.astropy.org/en/latest/development/codeguide.html>`_
`coding guidelines for astropy-affiliated packages <https://docs.astropy.org/en/latest/development/codeguide.html>`_
as much as possible.
as much as possible.
A few important conventions and special cases should be outlined here.
A few important conventions and special cases should be outlined here.
Basic
preference
Package
preference
----------------
----------------
--
Several packages are favored over others if they can be used to solve the problem under study.
Several packages are favored over others if they can be used to solve the problem under study.
Developers should use them as much as possible.
Developers should use them as much as possible.
...
@@ -66,3 +66,50 @@ The output is
...
@@ -66,3 +66,50 @@ The output is
For parallel computing with inter-communication or distributed computing,
For parallel computing with inter-communication or distributed computing,
we recommend developers to consider using ``mpi4py``: https://github.com/mpi4py/mpi4py.
we recommend developers to consider using ``mpi4py``: https://github.com/mpi4py/mpi4py.
Global variables
----------------
Usage of ``global`` should be prohibited.
In most cases, variables should be kept in their default scopes.
Using ``subprocess``
--------------------
``subprocess.run()`` is favored over ``subprocess.Popen()`` and ``os.system()`` as suggested in Python documentation:
- The ``subprocess`` module allows you to spawn new processes, connect to their input/output/error pipes,
and obtain their return codes. This module intends to replace several older modules and functions:
.. code-block:: python
os.system
os.spawn*
- The recommended approach to invoking ``subprocesses`` is to use the ``run()`` function for all use cases
it can handle. For more advanced use cases, the underlying ``Popen`` interface can be used directly.
Numpy multithreading
--------------------
Numpy sometime automatically uses multithreading. To see if you are actually using OpenBLAS or MKL, use
.. code-block:: python
numpy.__config__.show()
To set shut down this feature, use
.. code-block:: python
export MKL_NUM_THREADS=1
export NUMEXPR_NUM_THREADS=1
export OMP_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
.. note::
In most cases, this automatic multithreading does not enhance the performance in practice.
Therefore, the above setting will be used in CSST pipeline globally.
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment