Commit 961677a7 authored by Wei Chengliang's avatar Wei Chengliang
Browse files

add demo1

parent a51fbaaa

demo1.ipynb

0 → 100755
+391 −0
Original line number Diff line number Diff line 
%% Cell type:code id:0a723089-b919-4aac-b5a9-350e5ac9d29f tags:



``` python

import numpy as np

import matplotlib.pyplot as plt

import astropy.io.fits as fitsio



from IPython.display import HTML, display



def setBG_incell(color):

    script = (

        "var cell = this.closest('.jp-CodeCell');"

        "var editor = cell.querySelector('.jp-Editor');"

        "editor.style.background='{}';"

        "this.parentNode.removeChild(this)"

    ).format(color)



    display(HTML('<img src onerror="{}" style="display:none">'.format(script)))



def setBG_outcell(color):

    script = (

        "var cell = this.closest('.jp-CodeCell');"

        "var editor = cell.querySelector('.jp-OutputArea');"

        "editor.style.background='{}';"

        "this.parentNode.removeChild(this)"

    ).format(color)



    display(HTML('<img src onerror="{}" style="display:none">'.format(script)))

```



%% Cell type:markdown id:87788753-5d20-4c2d-9633-f67d645b18d2 tags:



---



%% Cell type:markdown id:bfa294a2-fe05-4344-a715-aa45adb708c5 tags:



# CSST主巡天仿真

+ 仿真软件git仓库: [https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim)



  <img src="pics/csstsim_git.png" alt="csst_sim_git.png" width="200">



%% Cell type:markdown id:bbccb71f-1932-4f91-8c01-c3f367399ae6 tags:



## 1. 安装



%% Cell type:markdown id:33813cc5-8343-445e-8968-ae681185e5da tags:



## 1.1 [**CSST仿真v3.0.0 版本软件使用手册** ](https://www.kdocs.cn/l/cjyiU0SXGyn2)

  + 获取代码: **git clone** https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim.git

  + 配置环境: anaconda

    + **install.sh**, **requirements.txt**

    + python3.11

    + numpy, astropy, scipy, galsim, healpy, h5py, toml, lmfit, psutil, pyyaml, sep, numba, cython, cfitsio, mpi4py

  + 安装软件: **pip install -e .**

  + 测试数据: [**data_test2024**](https://pan.baidu.com/s/1WazLj3qCXZqgU7ngw0VKbw?pwd=kbsm)

    + PSF数据库

      + 多色成像: data_test2024/set1_dynamic

      + 无缝光谱: data_test2024/SLS_PSF_PCA_fp

    + 测试星表数据

      + 恒星星表: data_test2024/starcat_C9

      + 星系星表: data_test2024/galcat_C9/cat2CSSTSim_bundle-50sqDeg

      + 类星体星表: data_test2024/galcat_C9/qsosed

      + fits数据:  data_test2024/stampCatsIndex.hdf5 (stampCats)

      + 曝光指向数据: data_test2024/pointing50_C9





%% Cell type:markdown id:1a443dbb-64c3-41c0-818a-e9fa4ef464cf tags:



### 1.2 运行





+ <font color="grey"> [mpirun -np N] </font> **python3** LOCALPATH/csst_msc_sim/run_sim.py  **--catalog** C9_Catalog_fits **--config_file** config_overall_test2024.yaml  **-c** LOCALPATH/data_test2024/run_test2024/config



%% Cell type:code id:e0d31aa9-76ce-412f-8992-5588148a9b25 tags:



``` python

#配置conda环境

!conda info --envs

```



%% Cell type:code id:3fcceeca-8720-48aa-abdf-e3fdc90610ef tags:



``` python

#查看python版本

!python --version

```



%% Cell type:code id:343e430e-9c51-4f88-95bb-f614ea57dfe6 tags:



``` python

LOCALPATH="/public/home/chengliang/csstsimTest2024"

!ls --color $LOCALPATH

```



%% Cell type:code id:f3b06b32-4ac2-4b7d-bb29-9e2ae22d2402 tags:



``` python

#准备测试数据

!tree -C -L 2 $LOCALPATH/data_test2024/run_test2024

```



%% Cell type:code id:6210ffd7-389e-4c43-a3c0-b14a461d3c14 tags:



``` python

#下载CSST仿真软件

!tree -C -L 1 $LOCALPATH/csst_msc_sim

```



%% Cell type:code id:f31f465c-8bc8-4133-afaa-fede73b0af3a tags:



``` python

#进入CSST仿真软件目录并完成安装

%cd $LOCALPATH/csst_msc_sim

!pip install -e .

```



%% Cell type:code id:1d02e07d-8b2a-4f45-8e3d-cd4f79607c2f tags:



``` python

#检查CSST仿真软件模块

import observation_sim

observation_sim.__file__

```



%% Cell type:code id:97504b20-40b5-4f8e-a56b-69855ece6d2f tags:



``` python

```



%% Cell type:markdown id:7f8285c4-90a1-43e9-bb5c-d268906963e1 tags:



----



%% Cell type:markdown id:9bfd2585-f7bb-4cd3-8b6e-91ebe7728aa2 tags:



## 2.配置



%% Cell type:markdown id:9d39722c-d24a-48e0-b070-e0e11176f5e6 tags:



+ data_test2024/run_test2024/config

  + 全局配置: config_overall_test2024.yaml，主要是路径信息

  + 观测配置: obs_config_SCI_WIDE_phot.yaml，主要是成像chip设置和各类效应开关





<img src="pics/fp.png" alt="fp.png" width="400">





  + 为了便于检查，可以先按2*8格式输出图像: **format_output: NO**



<img src="pics/img_out2x8.png" alt="img_out2x8.png" width="400"> **$\rightarrow$** <img src="pics/img_out1x16.png" alt="img_out1x16.png" width="500">



%% Cell type:code id:95b16f37-7a35-4a14-8807-9c405c0ba41f tags:



``` python

%cd $LOCALPATH/data_test2024

```



%% Cell type:code id:042f05fe-4252-489c-9bb8-5682a2ef7934 tags:



``` python

!bat --theme="Monokai Extended Bright" --plain run_test2024/config/config_overall_test2024.yaml

setBG_outcell('black')

```



%% Cell type:code id:e7f6ec2a-abc5-4be1-84ad-e2c959cae26f tags:



``` python

!bat --theme="Monokai Extended Bright" --plain run_test2024/config/obs_config_SCI_WIDE_phot.yaml

setBG_outcell('black')

```



%% Cell type:code id:7b68c16a-2fd9-4930-bc68-f6df9d6e9584 tags:



``` python

```



%% Cell type:markdown id:ccab997f-9566-4f27-8c9f-5e2531e8b72d tags:



----



%% Cell type:markdown id:63e67d64-ef29-4f47-b9e2-7ae7fc2d0061 tags:



## 3.多色成像



%% Cell type:markdown id:47e51d31-5f88-4ffc-bcb0-695e4ebaa718 tags:



+ data_test2024/run_test2024/catalog/Catalog_example_test2024.py

  + **ra, dec, z**: [e.g., 27.87411627, -47.86911686, 0.]

  + **star**: [e.g., 0-galaxy, 1-star, 2-quasar, 3-stamp, 4-calib]

  + **mag_use_normal**: [e.g., 20.]

  + **sed**: [e.g., wave, flux]



<img src="pics/img_catalogExample.png" alt="img_catalogExample.png" width="600"> <img src="pics/img_star.png" alt="img_star.png" width="400">



%% Cell type:code id:74687bf0-3ca7-4f2d-9ad3-b9c304582b23 tags:



``` python

#catalog文件需要放到默认路径，即$LOCALPATH/csst_msc_sim/catalog/**

!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/Catalog_example_test2024.py

setBG_outcell('black')

```



%% Cell type:code id:cad37712-a130-4846-b6ae-ee5827f97c60 tags:



``` python

#修改配置文件后，运行仿真任务

setBG_incell('honeydew')



!srun -p debug -n 1 -c 4 python3 /public/home/chengliang/csstsimTest2024/csst_msc_sim/run_sim.py \

                                 --catalog Catalog_example_test2024 \

                                 --config_file config_overall_test2024.yaml \

                                 -c /public/home/chengliang/csstsimTest2024/data_test2024/run_test2024/config

```



%% Cell type:code id:dffd0a5b-597a-4dc0-9880-56292cbff80d tags:



``` python

#检查仿真结果

!tree $LOCALPATH/data_test2024/run_test2024/outputs/testRunX

```



%% Cell type:code id:a8583552-904c-427c-9bb0-4839b03ce83c tags:



``` python

#查看仿真星表

!cat $LOCALPATH/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01.cat

```



%% Cell type:code id:00697fbb-dbd2-46ee-b79e-58cbff53a3a6 tags:



``` python

#查看仿真图像(ds9)

filename = LOCALPATH + "/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01"

hdu = fitsio.open(filename+".fits")



fn = open(filename+".cat", 'r')

header1 = fn.readline()

ximg = []

yimg = []

for line in fn:

    line   = line.strip()

    columns= line.split()

    ximg.append(float(columns[3]))

    yimg.append(float(columns[4]))

ximg = np.array(ximg)

yimg = np.array(yimg)



plt.figure(figsize=(12,12))

plt.imshow(np.log10(hdu[1].data), origin='lower', vmin=np.log10(438), vmax=np.log10(550), cmap='grey')

plt.plot(ximg, yimg,c='#90EE90', marker="o", mfc="none", ms = 20)



plt.figure(figsize=(6,6))

plt.imshow(np.log10(hdu[1].data)[int(yimg[0])-32:int(yimg[0])+32, int(ximg[0])-32:int(ximg[0])+32], origin='lower',

           vmin=np.log10(438), vmax=np.log10(550), cmap='grey', extent=([int(ximg[0])-32, int(ximg[0])+32, int(yimg[0])-32, int(yimg[0])+32]))

```



%% Cell type:markdown id:d917f8e5-c634-4797-9b3e-dee241f3659f tags:



+ data_test2024/run_test2024/catalog/C9_Catalog.py

  + 将star/galaxy/quasar加入仿真列表: self.objs.append(obj)

  + 大星表推荐使用healpix天区索引

  + 更新SED的读取函数



%% Cell type:code id:317941f2-8960-4e2f-ab81-a78af40c6952 tags:



``` python

!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog.py

setBG_outcell('black')

```



%% Cell type:markdown id:6db5d367-424c-4637-b996-c02beadf2704 tags:



  + 打开所有效应的仿真图像

  <img src="pics/img_cti.png" alt="img_cti.png" width="1000">



%% Cell type:code id:6948ab17-3392-437b-9c24-24e573b2c38d tags:



``` python

```



%% Cell type:markdown id:d04845d2-af7d-4b7b-a97b-775b2f1ef365 tags:



---



%% Cell type:markdown id:143546c0-8dfb-4b9d-81e9-18e7af5e2f8a tags:



## 4. fits贴图



%% Cell type:markdown id:34107217-f7ba-4d58-aac2-7d4d1a67cd21 tags:



+ steps:

  1. 准备fits库(**图像**+**SED**): RA, DEC, z, mag_use_normal, image, pixScale

  2. 建立星表索引文件(*.hdf5): tools/index_fits_hdf5.py

  3. **修改catalog.py** (e.g. data_test2024/run_test2024/catalog/C9_Catalog_fits.py)

     + 将fits数据加入仿真列表:self.objs.append(obj)

     + 更新SED的读取函数

  5. 修改配置文件: config.yaml

  6. 提交仿真任务



%% Cell type:code id:ca23e1f3-b1bd-4e25-bdb3-ba01a6ebd5e6 tags:



``` python

!tree -L 1 $LOCALPATH/data_test2024/stampCats

```



%% Cell type:code id:44f8de98-a170-4c32-a889-c91fd3179b84 tags:



``` python

#使用index_fits_hdf5.py构建索引文件



setBG_incell("honeydew")

!cp $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py $LOCALPATH/data_test2024

#python3 index_fits_hdf5.py

```



%% Cell type:code id:8a2e5966-d6ca-41be-9c75-4e745fe0c46b tags:



``` python

!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog_fits.py

setBG_outcell('black')

```



%% Cell type:markdown id:193619be-9462-4be6-8201-9da571cb0055 tags:



  + 添加fits贴图的测试结果

  <img src="pics/img_fits.png" alt="img_fits.png" width="1000">



%% Cell type:code id:cd7c0489-0817-49da-9f97-db75c03b8b79 tags:



``` python

```



%% Cell type:markdown id:3d9d9d3d-4ba0-45f2-8ee8-bdcbaed30ad8 tags:



---



%% Cell type:markdown id:2360837e-4cd9-4d26-a61c-1e662db8a783 tags:



## 5.实用工具 (csst_msc_sim/tools)



%% Cell type:markdown id:614ecfdf-a9e0-43c7-9aeb-02959fa4202d tags:



### 5.1 [index_fits_hdf5.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/index_fits_hdf5.py)



%% Cell type:markdown id:5c18c3e1-6914-4d93-94b5-94595c56f0ed tags:



  + 目的：把fits图像库封装成仿真可读取的fits星表格式

  + 使用方法：

    + 准备fits图像数据库

    + 在index_fits_hdf5.py中，指定fits图像目录dir_cat

  + 在终端运行：

    + python index_fits_hdf5.py

  + 输出：代码完成后将生成fits图像索引文件，数据格式为hdf5

  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的索引文件和对应fits图像库能够用作仿真软件的输入星表。



%% Cell type:code id:b06f15cf-0399-4c02-9528-1e04a2efd951 tags:



``` python

!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py

setBG_outcell('black')

```



%% Cell type:markdown id:83c3c7ef-d53b-4e55-b233-15e838ce866b tags:



### 5.2 [get_PSF.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/get_PSF.py)



%% Cell type:markdown id:831dd90d-04b4-4030-9d87-3c19b2e9639e tags:



  + 目的：给定焦面位置，返回对应位置处的光学PSF图像

  + 使用方法：

      + 给定焦面采样位置x, y = imgPos[iobj, :]，在示例中通过读取仿真输出的.cat星表来设置imgPos

  + 在终端运行：

      + python get_PSF.py

  + 输出：代码完成后将生成对应位置的PSF图像

  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的PSF图像暂不包含探测器部分。



%% Cell type:code id:263c7f7e-e96d-4616-8da1-ced7501c71c5 tags:



``` python

!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/tools/get_PSF.py

setBG_outcell('black')

```



%% Cell type:markdown id:24effd04-2110-41ff-83dd-879b72aaccfd tags:



### 5.3 ...



%% Cell type:code id:5df4f734-d89a-4984-acae-4f01973a81ad tags:



``` python

```

pics/.DS_Store

0 → 100755
+6 KiB

File added.

No diff preview for this file type.

View file

pics/csstsim_git.png

0 → 100755
+45.1 KiB
Loading image diff...

pics/fp.png

0 → 100755
+458 KiB
Loading image diff...

pics/img_catalogExample.png

0 → 100755
+63.8 MiB
Loading image diff...