Commit bb021490 authored by yuedong0607's avatar yuedong0607
Browse files

code refactoriztion

parent c6424dc8
Loading
Loading
Loading
Loading
+71 −32
Original line number Diff line number Diff line
# CSST instrument detector model
# CSST instrument model

This package is the authoritative interface between version-controlled
instrument data and the main simulator. Simulation steps must consume a
`ResolvedDetector` from `InstrumentRepository`; they must not parse YAML or
duplicate channel geometry.
instrument data and the main simulator. Simulation steps consume a
`ResolvedDetector` or resolve a reference through `InstrumentRepository`; they
do not parse YAML or depend on the package's storage layout.

## Catalog ownership
## Versioning model

Instrument data is split into three independently versioned layers:

| Catalog | Owns | Changes when |
| Layer | Location | Purpose |
|---|---|---|
| `detector_types.yaml` | Active size, pitch, detector technology, nominal electronics | A detector design changes |
| `readout_modes.yaml` | Channel boxes, transforms, per-channel scans, raw placement and output order | A controller/readout mode changes |
| `calibration_profiles.yaml` | Dated electronics, defects, exposure recipes and calibration references | A calibration release changes |
| `focal_plane.yaml` | Installed detector IDs, type/mode/profile references, placement, filter and survey role | The focal-plane build changes |
| Definition sets | `definition_sets/<name>/<version>/` | Detector types, readout geometry, calibrations, focal-plane assignments, and semantic reference assignments |
| Reference sets | `reference_sets/<category>/<version>/` | Curves, arrays, distortion models, optics, cosmic-ray data, and other files |
| Release compositions | `releases/<release>.yaml` | A small lock manifest that pins one definition set and one exact version of every reference category |

Every definition-set directory uses the fixed filenames `detector_types.yaml`,
`readout_modes.yaml`, `calibration_profiles.yaml`, `focal_plane.yaml`, and
`reference_assignments.yaml`; no additional definition-set manifest is required.

A release can therefore combine, for example, cosmic-ray references `1.0.0`
with filter transmissions `2.1.0`. Pins are always exact; values such as `2.x`
are not accepted as release policy.

The built-in compositions currently pin:

- `legacy-v1`: definition set `legacy/1.0.0`;
- `next-v1`: definition set `next/1.0.0` with 31 science slots (`00` through
  `30`) and the existing FGS definitions;
- reference set `1.0.0` for each category in both compositions.

Released versions should be treated as immutable. Publish changed definitions
or reference files under a new version, then add or update a composition
manifest to pin those exact versions. This avoids copying unchanged reference
files into every definition release.

Instrument releases are immutable directories under `releases/`. Add a new
directory for a new release; do not silently rewrite a release already used for
production.
`legacy-v1` preserves the values formerly stored in the simulator's legacy
`chip_definition.json` file.
## Catalog ownership

| Catalog | Owns |
|---|---|
| `detector_types.yaml` | Detector families and concrete detector types, including QE filenames, size, pitch, technology, and nominal electronics |
| `readout_modes.yaml` | Channel boxes, transforms, per-channel scan regions, raw placement, and output order |
| `calibration_profiles.yaml` | Electronics, defects, exposure recipes, and calibration reference IDs |
| `focal_plane.yaml` | Installed detector IDs, type/mode/calibration assignments, placement, filter, and survey role |
| `reference_assignments.yaml` | Semantic roles that map global, detector, filter, and slitless inputs to versioned reference files |

Each reference-set directory directly contains its files. Definitions refer to
those files by name, while release compositions select the category versions.
Repository loading validates schemas, cross-catalog relationships, pinned
versions, and files referenced by detector or calibration definitions.

## Runtime API

@@ -30,11 +60,21 @@ detector = instrument.detector("01")

for channel in detector.iter_channels():
    print(channel.id, channel.detector_bbox, channel.raw_shape)

with instrument.global_reference_path("telescope_efficiency") as path:
    print(path)

with instrument.filter_reference_path("NUV", "throughput") as path:
    print(path)

configs = instrument.materialized_detector_reference_paths(
    "01", "slitless_configurations"
)
```

`Chip` loads the built-in release by default. An overall simulation config can
select another checked-out release directory or override a detector's
mode/profile:
The simulation selects a built-in composition by default. A custom
`release_directory` points to an instrument-data root containing
`releases/`, `definition_sets/`, and `reference_sets/`:

```yaml
instrument_data:
@@ -46,25 +86,24 @@ instrument_data:
      calibration_profile: science_nominal_v1
```

## Versioned reference files

Each directory under `releases/` is a self-contained instrument release. Its
schema-validated YAML lives under `definitions/`; throughput curves, distortion
models, calibration arrays, and slitless-spectroscopy files live under
`reference_files/`. Consumers use a release-bound `InstrumentRepository` and
its `reference_path()` or `materialized_reference_path()` methods, so custom
release directories never fall back to files from a different built-in release.
Low-level `reference_path()` and `materialized_reference_path()` APIs remain
available for explicit user overrides. Main simulation code should use the
purpose-based accessors `global_reference_path()`, `detector_reference_path()`,
`materialized_detector_reference_paths()`, `filter_reference_path()`, and
`slitless_throughput_path()`. This keeps physical filenames and category choices
inside the versioned definition set rather than `observation_sim`.

## Validation and schemas

Unknown keys are rejected and cross-catalog validation checks references,
channel coverage/non-overlap, raw mosaic bounds, and calibration channel IDs.
Unknown keys are rejected. Cross-catalog validation checks detector families,
channel coverage/non-overlap, raw mosaic bounds, calibration channel IDs, and
all pinned references.

```bash
python -m csst_msc_instrument validate --release legacy-v1
python -m csst_msc_instrument emit-schema \
  csst_msc_instrument/schemas/1.0
python -m csst_msc_instrument validate --release next-v1
python -m csst_msc_instrument emit-schema csst_msc_instrument/schemas/1.0
```

The checked-in schemas are generated from the same Pydantic models. CI should
regenerate them and fail if the schema directory changes.
The checked-in JSON Schemas are generated from the same Pydantic models. CI
should regenerate them and fail if the schema directory changes.
+1 −0
Original line number Diff line number Diff line
"""Versioned CSST instrument definition sets."""
+7 −17
Original line number Diff line number Diff line
@@ -2,7 +2,7 @@ schema_version: "1.0"

calibration_profiles:
  science_nominal_v1:
    detector_type: science_ccd_v1
    detector_family: science_ccd
    valid_from: null
    valid_to: null
    readout_time_s: 40.0
@@ -10,23 +10,13 @@ calibration_profiles:
    n_psf_samples: 900
    exposures: {dark_s: 300.0, flat_s: 150.0}
    defects: {bad_pixel_fraction: 0.000019, bad_column_fraction: 0.0}
    references: {qe_curve: null, dark_histogram: hist_dark_default.npz, defect_map: null}
    references: {dark_histogram: hist_dark_default.npz, defect_map: null}
    channel_electronics: {}

  science_uv_v1:
    detector_type: science_ccd_v1
    valid_from: null
    valid_to: null
    readout_time_s: 40.0
    dark_current_e_per_pix_s: 0.02
    n_psf_samples: 900
    exposures: {dark_s: 300.0, flat_s: 150.0}
    defects: {bad_pixel_fraction: 0.0000032, bad_column_fraction: 0.0}
    references: {qe_curve: null, dark_histogram: hist_dark_default.npz, defect_map: null}
    channel_electronics: {}


  fgs_nominal_v1:
    detector_type: fgs_cmos_v1
    detector_family: fgs_cmos
    valid_from: null
    valid_to: null
    readout_time_s: 0.01
@@ -34,11 +24,11 @@ calibration_profiles:
    n_psf_samples: 900
    exposures: {dark_s: 300.0, flat_s: 150.0}
    defects: {bad_pixel_fraction: 0.00005, bad_column_fraction: 0.0}
    references: {qe_curve: null, dark_histogram: hist_dark_default.npz, defect_map: null}
    references: {dark_histogram: hist_dark_default.npz, defect_map: null}
    channel_electronics: {}

  fgs_coarse_psf_v1:
    detector_type: fgs_cmos_v1
    detector_family: fgs_cmos
    valid_from: null
    valid_to: null
    readout_time_s: 0.01
@@ -46,5 +36,5 @@ calibration_profiles:
    n_psf_samples: 100
    exposures: {dark_s: 300.0, flat_s: 150.0}
    defects: {bad_pixel_fraction: 0.00005, bad_column_fraction: 0.0}
    references: {qe_curve: null, dark_histogram: hist_dark_default.npz, defect_map: null}
    references: {dark_histogram: hist_dark_default.npz, defect_map: null}
    channel_electronics: {}
+66 −0
Original line number Diff line number Diff line
schema_version: "1.0"

detector_types:
  science_ccd_v1:
  science_ccd_visible_v1:
    detector_family: science_ccd
    qe_curve: Astro_MB.txt
    sensor_technology: CCD
    active_shape: {columns: 9216, rows: 9232}
    pixel_pitch: {x_um: 10.0, y_um: 10.0}
    pixel_scale_arcsec: 0.074
    default_readout_mode: science_16ch_legacy
    nominal_electronics:
      gain_e_per_adu: 1.4
      read_noise_e: 4.5
      bias_adu: 2000.0
      full_well_e: 90000.0
    simulation:
      diffusion_strength: 2.3

  science_ccd_uv_v1:
    detector_family: science_ccd
    qe_curve: UV0.txt
    sensor_technology: CCD
    active_shape: {columns: 9216, rows: 9232}
    pixel_pitch: {x_um: 10.0, y_um: 10.0}
    pixel_scale_arcsec: 0.074
    default_readout_mode: science_16ch_legacy
    nominal_electronics:
      gain_e_per_adu: 1.4
      read_noise_e: 4.5
      bias_adu: 2000.0
      full_well_e: 90000.0
    simulation:
      diffusion_strength: 2.3

  science_ccd_nir_v1:
    detector_family: science_ccd
    qe_curve: Basic_NIR.txt
    sensor_technology: CCD
    active_shape: {columns: 9216, rows: 9232}
    pixel_pitch: {x_um: 10.0, y_um: 10.0}
@@ -16,6 +50,8 @@ detector_types:
      diffusion_strength: 2.3

  fgs_cmos_v1:
    detector_family: fgs_cmos
    qe_curve: Astro_MB.txt
    sensor_technology: CMOS
    active_shape: {columns: 11264, rows: 7680}
    pixel_pitch: {x_um: 7.5, y_um: 7.5}
+50 −40
Original line number Diff line number Diff line
@@ -10,7 +10,7 @@ coordinate_system:
detectors:
- id: '01'
  name: GI-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -22,7 +22,7 @@ detectors:
    survey: spectroscopic
- id: '02'
  name: GV-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -34,9 +34,10 @@ detectors:
    survey: spectroscopic
- id: '03'
  name: GU-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: 251.5
    y_mm: 0.0
@@ -46,9 +47,10 @@ detectors:
    survey: spectroscopic
- id: '04'
  name: GU-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: 251.5
    y_mm: 101.3
@@ -58,7 +60,7 @@ detectors:
    survey: spectroscopic
- id: '05'
  name: GV-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -70,7 +72,7 @@ detectors:
    survey: spectroscopic
- id: '06'
  name: y-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -82,7 +84,7 @@ detectors:
    survey: photometric
- id: '07'
  name: i-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -94,7 +96,7 @@ detectors:
    survey: photometric
- id: 08
  name: g-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -106,7 +108,7 @@ detectors:
    survey: photometric
- id: 09
  name: r-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -118,7 +120,7 @@ detectors:
    survey: photometric
- id: '10'
  name: GI-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -130,7 +132,7 @@ detectors:
    survey: spectroscopic
- id: '11'
  name: z-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -142,9 +144,10 @@ detectors:
    survey: photometric
- id: '12'
  name: NUV-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: 48.75
    y_mm: -101.3
@@ -154,9 +157,10 @@ detectors:
    survey: photometric
- id: '13'
  name: NUV-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: 48.75
    y_mm: 0.0
@@ -166,9 +170,10 @@ detectors:
    survey: photometric
- id: '14'
  name: u-1
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: 48.75
    y_mm: 101.3
@@ -178,7 +183,7 @@ detectors:
    survey: photometric
- id: '15'
  name: y-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -190,7 +195,7 @@ detectors:
    survey: photometric
- id: '16'
  name: y-3
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -202,9 +207,10 @@ detectors:
    survey: photometric
- id: '17'
  name: u-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: -48.75
    y_mm: -101.3
@@ -214,9 +220,10 @@ detectors:
    survey: photometric
- id: '18'
  name: NUV-3
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: -48.75
    y_mm: 0.0
@@ -226,9 +233,10 @@ detectors:
    survey: photometric
- id: '19'
  name: NUV-4
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: -48.75
    y_mm: 101.3
@@ -238,7 +246,7 @@ detectors:
    survey: photometric
- id: '20'
  name: z-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -250,7 +258,7 @@ detectors:
    survey: photometric
- id: '21'
  name: GI-3
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -262,7 +270,7 @@ detectors:
    survey: spectroscopic
- id: '22'
  name: r-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -274,7 +282,7 @@ detectors:
    survey: photometric
- id: '23'
  name: g-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -286,7 +294,7 @@ detectors:
    survey: photometric
- id: '24'
  name: i-2
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -298,7 +306,7 @@ detectors:
    survey: photometric
- id: '25'
  name: y-4
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -310,7 +318,7 @@ detectors:
    survey: photometric
- id: '26'
  name: GV-3
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -322,9 +330,10 @@ detectors:
    survey: spectroscopic
- id: '27'
  name: GU-3
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: -251.5
    y_mm: -101.3
@@ -334,9 +343,10 @@ detectors:
    survey: spectroscopic
- id: '28'
  name: GU-4
  detector_type: science_ccd_v1
  detector_type: science_ccd_uv_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_uv_v1
  calibration_profile: science_nominal_v1
  calibration_overrides: {bad_pixel_fraction: 0.0000032}
  placement:
    x_mm: -251.5
    y_mm: 0.0
@@ -346,7 +356,7 @@ detectors:
    survey: spectroscopic
- id: '29'
  name: GV-4
  detector_type: science_ccd_v1
  detector_type: science_ccd_visible_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
@@ -358,7 +368,7 @@ detectors:
    survey: spectroscopic
- id: '30'
  name: GI-4
  detector_type: science_ccd_v1
  detector_type: science_ccd_nir_v1
  readout_mode: science_16ch_legacy
  calibration_profile: science_nominal_v1
  placement:
Loading