Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
csst-cicd
csst-dag
Commits
eaead0e9
Commit
eaead0e9
authored
Jul 23, 2025
by
BO ZHANG
🏀
Browse files
distinguish level0/1 data basis keys
parent
72c9e035
Changes
1
Hide whitespace changes
Inline
Side-by-side
csst_dag/dag/_dispatcher.py
View file @
eaead0e9
...
@@ -68,9 +68,22 @@ PLAN_BASIS_KEYS = (
...
@@ -68,9 +68,22 @@ PLAN_BASIS_KEYS = (
)
)
# data basis keys
# data basis keys
DATA_BASIS_KEYS
=
(
LEVEL0_DATA_BASIS_KEYS
=
(
"instrument"
,
"dataset"
,
"dataset"
,
"obs_type"
,
"obs_group"
,
"obs_id"
,
"detector"
,
"file_name"
,
"_id"
,
"prc_status"
,
"qc_status"
,
)
LEVEL1_DATA_BASIS_KEYS
=
(
"instrument"
,
"instrument"
,
"dataset"
,
"obs_type"
,
"obs_type"
,
"obs_group"
,
"obs_group"
,
"obs_id"
,
"obs_id"
,
...
@@ -78,6 +91,9 @@ DATA_BASIS_KEYS = (
...
@@ -78,6 +91,9 @@ DATA_BASIS_KEYS = (
"file_name"
,
"file_name"
,
"_id"
,
"_id"
,
"prc_status"
,
"prc_status"
,
"qc_status"
,
"data_model"
,
"batch_id"
,
)
)
# join_type for data x plan
# join_type for data x plan
...
@@ -204,7 +220,7 @@ class Dispatcher:
...
@@ -204,7 +220,7 @@ class Dispatcher:
# data basis
# data basis
data_basis
=
extract_basis_table
(
data_basis
=
extract_basis_table
(
qr
.
data
,
qr
.
data
,
DATA_BASIS_KEYS
,
LEVEL0_
DATA_BASIS_KEYS
,
)
)
return
data_basis
return
data_basis
...
@@ -223,7 +239,7 @@ class Dispatcher:
...
@@ -223,7 +239,7 @@ class Dispatcher:
# data basis
# data basis
data_basis
=
extract_basis_table
(
data_basis
=
extract_basis_table
(
qr
.
data
,
qr
.
data
,
DATA_BASIS_KEYS
,
LEVEL1_
DATA_BASIS_KEYS
,
)
)
return
data_basis
return
data_basis
...
@@ -729,6 +745,6 @@ class Dispatcher:
...
@@ -729,6 +745,6 @@ class Dispatcher:
)
)
data_basis
=
extract_basis_table
(
data_basis
=
extract_basis_table
(
data_recs
.
data
,
data_recs
.
data
,
DATA_BASIS_KEYS
,
LEVEL0_
DATA_BASIS_KEYS
,
)
)
return
plan_basis
,
data_basis
return
plan_basis
,
data_basis
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment