Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
csst-cicd
csst-dag
Commits
dfa31851
Commit
dfa31851
authored
Jul 11, 2025
by
BO ZHANG
🏀
Browse files
remove parallel in Dispatcher.dispatch_obsgroup_detector
parent
b9d13950
Changes
1
Hide whitespace changes
Inline
Side-by-side
csst_dag/dag/_dispatcher.py
View file @
dfa31851
...
...
@@ -118,7 +118,7 @@ def extract_basis_table(dlist: list[dict], basis_keys: tuple) -> table.Table:
def
split_data_basis
(
data_basis
:
table
.
Table
,
n_split
:
int
=
1
)
->
list
[
table
.
Table
]:
"""Split data basis into n_split parts
.
"""
"""Split data basis into n_split parts
via obs_id
"""
assert
(
np
.
unique
(
data_basis
[
"dataset"
]).
size
==
1
),
"Only one dataset is allowed for splitting."
...
...
@@ -546,16 +546,8 @@ class Dispatcher:
def
dispatch_obsgroup_detector
(
plan_basis
:
table
.
Table
,
data_basis
:
table
.
Table
,
n_jobs
:
int
=
1
,
#
n_jobs: int = 1,
):
# parallel
if
n_jobs
!=
1
:
task_list
=
joblib
.
Parallel
(
n_jobs
=
n_jobs
)(
joblib
.
delayed
(
Dispatcher
.
dispatch_obsid
)(
plan_basis
,
_
)
for
_
in
split_data_basis
(
data_basis
,
n_split
=
n_jobs
)
)
return
sum
(
task_list
,
[])
# unique obsgroup basis (using group_by)
obsgroup_basis
=
plan_basis
.
group_by
(
keys
=
[
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment