add demo1

961677a7 · Wei Chengliang · a51fbaaa · 961677a7 · 961677a7 · 961677a7
Commit 961677a7 authored Jul 23, 2024 by Wei Chengliang
--- a/demo1.ipynb
+++ b/demo1.ipynb
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0a723089-b919-4aac-b5a9-350e5ac9d29f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import astropy.io.fits as fitsio\n",
+    "\n",
+    "from IPython.display import HTML, display\n",
+    "\n",
+    "def setBG_incell(color):    \n",
+    "    script = (\n",
+    "        \"var cell = this.closest('.jp-CodeCell');\"\n",
+    "        \"var editor = cell.querySelector('.jp-Editor');\"\n",
+    "        \"editor.style.background='{}';\"\n",
+    "        \"this.parentNode.removeChild(this)\"\n",
+    "    ).format(color)\n",
+    "    \n",
+    "    display(HTML('<img src onerror=\"{}\" style=\"display:none\">'.format(script)))\n",
+    "\n",
+    "def setBG_outcell(color):    \n",
+    "    script = (\n",
+    "        \"var cell = this.closest('.jp-CodeCell');\"\n",
+    "        \"var editor = cell.querySelector('.jp-OutputArea');\"\n",
+    "        \"editor.style.background='{}';\"\n",
+    "        \"this.parentNode.removeChild(this)\"\n",
+    "    ).format(color)\n",
+    "    \n",
+    "    display(HTML('<img src onerror=\"{}\" style=\"display:none\">'.format(script)))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "87788753-5d20-4c2d-9633-f67d645b18d2",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bfa294a2-fe05-4344-a715-aa45adb708c5",
+   "metadata": {},
+   "source": [
+    "# CSST主巡天仿真\n",
+    "+ 仿真软件git仓库: [https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim)\n",
+    "\n",
+    "  <img src=\"pics/csstsim_git.png\" alt=\"csst_sim_git.png\" width=\"200\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bbccb71f-1932-4f91-8c01-c3f367399ae6",
+   "metadata": {},
+   "source": [
+    "## 1. 安装"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "33813cc5-8343-445e-8968-ae681185e5da",
+   "metadata": {},
+   "source": [
+    "## 1.1 [**CSST仿真v3.0.0 版本软件使用手册** ](https://www.kdocs.cn/l/cjyiU0SXGyn2)\n",
+    "  + 获取代码: **git clone** https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim.git\n",
+    "  + 配置环境: anaconda\n",
+    "    + **install.sh**, **requirements.txt**\n",
+    "    + python3.11\n",
+    "    + numpy, astropy, scipy, galsim, healpy, h5py, toml, lmfit, psutil, pyyaml, sep, numba, cython, cfitsio, mpi4py\n",
+    "  + 安装软件: **pip install -e .**\n",
+    "  + 测试数据: [**data_test2024**](https://pan.baidu.com/s/1WazLj3qCXZqgU7ngw0VKbw?pwd=kbsm)\n",
+    "    + PSF数据库\n",
+    "      + 多色成像: data_test2024/set1_dynamic\n",
+    "      + 无缝光谱: data_test2024/SLS_PSF_PCA_fp\n",
+    "    + 测试星表数据\n",
+    "      + 恒星星表: data_test2024/starcat_C9\n",
+    "      + 星系星表: data_test2024/galcat_C9/cat2CSSTSim_bundle-50sqDeg\n",
+    "      + 类星体星表: data_test2024/galcat_C9/qsosed\n",
+    "      + fits数据:  data_test2024/stampCatsIndex.hdf5 (stampCats)\n",
+    "      + 曝光指向数据: data_test2024/pointing50_C9\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1a443dbb-64c3-41c0-818a-e9fa4ef464cf",
+   "metadata": {},
+   "source": [
+    "### 1.2 运行\n",
+    "\n",
+    "\n",
+    "+ <font color=\"grey\"> [mpirun -np N] </font> **python3** LOCALPATH/csst_msc_sim/run_sim.py  **--catalog** C9_Catalog_fits **--config_file** config_overall_test2024.yaml  **-c** LOCALPATH/data_test2024/run_test2024/config"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e0d31aa9-76ce-412f-8992-5588148a9b25",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#配置conda环境\n",
+    "!conda info --envs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3fcceeca-8720-48aa-abdf-e3fdc90610ef",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#查看python版本\n",
+    "!python --version"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "343e430e-9c51-4f88-95bb-f614ea57dfe6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "LOCALPATH=\"/public/home/chengliang/csstsimTest2024\"\n",
+    "!ls --color $LOCALPATH"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f3b06b32-4ac2-4b7d-bb29-9e2ae22d2402",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#准备测试数据\n",
+    "!tree -C -L 2 $LOCALPATH/data_test2024/run_test2024"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6210ffd7-389e-4c43-a3c0-b14a461d3c14",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#下载CSST仿真软件\n",
+    "!tree -C -L 1 $LOCALPATH/csst_msc_sim"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f31f465c-8bc8-4133-afaa-fede73b0af3a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#进入CSST仿真软件目录并完成安装\n",
+    "%cd $LOCALPATH/csst_msc_sim\n",
+    "!pip install -e ."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1d02e07d-8b2a-4f45-8e3d-cd4f79607c2f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#检查CSST仿真软件模块\n",
+    "import observation_sim\n",
+    "observation_sim.__file__"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "97504b20-40b5-4f8e-a56b-69855ece6d2f",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f8285c4-90a1-43e9-bb5c-d268906963e1",
+   "metadata": {},
+   "source": [
+    "----"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9bfd2585-f7bb-4cd3-8b6e-91ebe7728aa2",
+   "metadata": {},
+   "source": [
+    "## 2.配置"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9d39722c-d24a-48e0-b070-e0e11176f5e6",
+   "metadata": {},
+   "source": [
+    "+ data_test2024/run_test2024/config\n",
+    "  + 全局配置: config_overall_test2024.yaml，主要是路径信息\n",
+    "  + 观测配置: obs_config_SCI_WIDE_phot.yaml，主要是成像chip设置和各类效应开关\n",
+    "\n",
+    "\n",
+    "<img src=\"pics/fp.png\" alt=\"fp.png\" width=\"400\">\n",
+    "\n",
+    "\n",
+    "  + 为了便于检查，可以先按2*8格式输出图像: **format_output: NO**\n",
+    "    \n",
+    "<img src=\"pics/img_out2x8.png\" alt=\"img_out2x8.png\" width=\"400\"> **$\\rightarrow$** <img src=\"pics/img_out1x16.png\" alt=\"img_out1x16.png\" width=\"500\">\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "95b16f37-7a35-4a14-8807-9c405c0ba41f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%cd $LOCALPATH/data_test2024"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "042f05fe-4252-489c-9bb8-5682a2ef7934",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain run_test2024/config/config_overall_test2024.yaml \n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7f6ec2a-abc5-4be1-84ad-e2c959cae26f",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain run_test2024/config/obs_config_SCI_WIDE_phot.yaml\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7b68c16a-2fd9-4930-bc68-f6df9d6e9584",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ccab997f-9566-4f27-8c9f-5e2531e8b72d",
+   "metadata": {},
+   "source": [
+    "----"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "63e67d64-ef29-4f47-b9e2-7ae7fc2d0061",
+   "metadata": {},
+   "source": [
+    "## 3.多色成像"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "47e51d31-5f88-4ffc-bcb0-695e4ebaa718",
+   "metadata": {},
+   "source": [
+    "+ data_test2024/run_test2024/catalog/Catalog_example_test2024.py\n",
+    "  + **ra, dec, z**: [e.g., 27.87411627, -47.86911686, 0.]\n",
+    "  + **star**: [e.g., 0-galaxy, 1-star, 2-quasar, 3-stamp, 4-calib]\n",
+    "  + **mag_use_normal**: [e.g., 20.]\n",
+    "  + **sed**: [e.g., wave, flux]\n",
+    "\n",
+    "<img src=\"pics/img_catalogExample.png\" alt=\"img_catalogExample.png\" width=\"600\"> <img src=\"pics/img_star.png\" alt=\"img_star.png\" width=\"400\">\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "74687bf0-3ca7-4f2d-9ad3-b9c304582b23",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#catalog文件需要放到默认路径，即$LOCALPATH/csst_msc_sim/catalog/**\n",
+    "!bat --theme=\"Monokai Extended Bright\" --plain $LOCALPATH/csst_msc_sim/catalog/Catalog_example_test2024.py\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cad37712-a130-4846-b6ae-ee5827f97c60",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#修改配置文件后，运行仿真任务\n",
+    "setBG_incell('honeydew')\n",
+    "\n",
+    "!srun -p debug -n 1 -c 4 python3 /public/home/chengliang/csstsimTest2024/csst_msc_sim/run_sim.py \\\n",
+    "                                 --catalog Catalog_example_test2024 \\\n",
+    "                                 --config_file config_overall_test2024.yaml \\\n",
+    "                                 -c /public/home/chengliang/csstsimTest2024/data_test2024/run_test2024/config"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dffd0a5b-597a-4dc0-9880-56292cbff80d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#检查仿真结果\n",
+    "!tree $LOCALPATH/data_test2024/run_test2024/outputs/testRunX"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a8583552-904c-427c-9bb0-4839b03ce83c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#查看仿真星表\n",
+    "!cat $LOCALPATH/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01.cat"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "00697fbb-dbd2-46ee-b79e-58cbff53a3a6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#查看仿真图像(ds9)\n",
+    "filename = LOCALPATH + \"/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01\"\n",
+    "hdu = fitsio.open(filename+\".fits\")\n",
+    "\n",
+    "fn = open(filename+\".cat\", 'r')\n",
+    "header1 = fn.readline()\n",
+    "ximg = []\n",
+    "yimg = []\n",
+    "for line in fn:\n",
+    "    line   = line.strip()\n",
+    "    columns= line.split()\n",
+    "    ximg.append(float(columns[3]))\n",
+    "    yimg.append(float(columns[4]))\n",
+    "ximg = np.array(ximg)\n",
+    "yimg = np.array(yimg)\n",
+    "\n",
+    "plt.figure(figsize=(12,12))\n",
+    "plt.imshow(np.log10(hdu[1].data), origin='lower', vmin=np.log10(438), vmax=np.log10(550), cmap='grey')\n",
+    "plt.plot(ximg, yimg,c='#90EE90', marker=\"o\", mfc=\"none\", ms = 20)\n",
+    "\n",
+    "plt.figure(figsize=(6,6))\n",
+    "plt.imshow(np.log10(hdu[1].data)[int(yimg[0])-32:int(yimg[0])+32, int(ximg[0])-32:int(ximg[0])+32], origin='lower',\n",
+    "           vmin=np.log10(438), vmax=np.log10(550), cmap='grey', extent=([int(ximg[0])-32, int(ximg[0])+32, int(yimg[0])-32, int(yimg[0])+32]))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d917f8e5-c634-4797-9b3e-dee241f3659f",
+   "metadata": {},
+   "source": [
+    "+ data_test2024/run_test2024/catalog/C9_Catalog.py\n",
+    "  + 将star/galaxy/quasar加入仿真列表: self.objs.append(obj)\n",
+    "  + 大星表推荐使用healpix天区索引\n",
+    "  + 更新SED的读取函数"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "317941f2-8960-4e2f-ab81-a78af40c6952",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog.py\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6db5d367-424c-4637-b996-c02beadf2704",
+   "metadata": {},
+   "source": [
+    "  + 打开所有效应的仿真图像\n",
+    "  <img src=\"pics/img_cti.png\" alt=\"img_cti.png\" width=\"1000\">\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6948ab17-3392-437b-9c24-24e573b2c38d",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d04845d2-af7d-4b7b-a97b-775b2f1ef365",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "143546c0-8dfb-4b9d-81e9-18e7af5e2f8a",
+   "metadata": {},
+   "source": [
+    "## 4. fits贴图"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "34107217-f7ba-4d58-aac2-7d4d1a67cd21",
+   "metadata": {},
+   "source": [
+    "+ steps:\n",
+    "  1. 准备fits库(**图像**+**SED**): RA, DEC, z, mag_use_normal, image, pixScale\n",
+    "  2. 建立星表索引文件(*.hdf5): tools/index_fits_hdf5.py\n",
+    "  3. **修改catalog.py** (e.g. data_test2024/run_test2024/catalog/C9_Catalog_fits.py)\n",
+    "     + 将fits数据加入仿真列表:self.objs.append(obj)\n",
+    "     + 更新SED的读取函数\n",
+    "  5. 修改配置文件: config.yaml\n",
+    "  6. 提交仿真任务"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ca23e1f3-b1bd-4e25-bdb3-ba01a6ebd5e6",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "!tree -L 1 $LOCALPATH/data_test2024/stampCats"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "44f8de98-a170-4c32-a889-c91fd3179b84",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#使用index_fits_hdf5.py构建索引文件\n",
+    "\n",
+    "setBG_incell(\"honeydew\")\n",
+    "!cp $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py $LOCALPATH/data_test2024\n",
+    "#python3 index_fits_hdf5.py"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8a2e5966-d6ca-41be-9c75-4e745fe0c46b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog_fits.py\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "193619be-9462-4be6-8201-9da571cb0055",
+   "metadata": {},
+   "source": [
+    "  + 添加fits贴图的测试结果\n",
+    "  <img src=\"pics/img_fits.png\" alt=\"img_fits.png\" width=\"1000\">"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cd7c0489-0817-49da-9f97-db75c03b8b79",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3d9d9d3d-4ba0-45f2-8ee8-bdcbaed30ad8",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2360837e-4cd9-4d26-a61c-1e662db8a783",
+   "metadata": {},
+   "source": [
+    "## 5.实用工具 (csst_msc_sim/tools)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "614ecfdf-a9e0-43c7-9aeb-02959fa4202d",
+   "metadata": {},
+   "source": [
+    "### 5.1 [index_fits_hdf5.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/index_fits_hdf5.py)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5c18c3e1-6914-4d93-94b5-94595c56f0ed",
+   "metadata": {},
+   "source": [
+    "  + 目的：把fits图像库封装成仿真可读取的fits星表格式\n",
+    "  + 使用方法：\n",
+    "    + 准备fits图像数据库\n",
+    "    + 在index_fits_hdf5.py中，指定fits图像目录dir_cat\n",
+    "  + 在终端运行：\n",
+    "    + python index_fits_hdf5.py \n",
+    "  + 输出：代码完成后将生成fits图像索引文件，数据格式为hdf5\n",
+    "  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的索引文件和对应fits图像库能够用作仿真软件的输入星表。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b06f15cf-0399-4c02-9528-1e04a2efd951",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "83c3c7ef-d53b-4e55-b233-15e838ce866b",
+   "metadata": {},
+   "source": [
+    "### 5.2 [get_PSF.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/get_PSF.py)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "831dd90d-04b4-4030-9d87-3c19b2e9639e",
+   "metadata": {},
+   "source": [
+    "  + 目的：给定焦面位置，返回对应位置处的光学PSF图像\n",
+    "  + 使用方法：\n",
+    "      + 给定焦面采样位置x, y = imgPos[iobj, :]，在示例中通过读取仿真输出的.cat星表来设置imgPos\n",
+    "  + 在终端运行：\n",
+    "      + python get_PSF.py \n",
+    "  + 输出：代码完成后将生成对应位置的PSF图像\n",
+    "  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的PSF图像暂不包含探测器部分。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "263c7f7e-e96d-4616-8da1-ced7501c71c5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!bat --theme=\"Monokai Extended Bright\" --plain $LOCALPATH/csst_msc_sim/tools/get_PSF.py\n",
+    "setBG_outcell('black')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "24effd04-2110-41ff-83dd-879b72aaccfd",
+   "metadata": {},
+   "source": [
+    "### 5.3 ..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5df4f734-d89a-4984-acae-4f01973a81ad",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
+%% Cell type:code id:0a723089-b919-4aac-b5a9-350e5ac9d29f tags:
+``` python
+import numpy as np
+import matplotlib.pyplot as plt
+import astropy.io.fits as fitsio
+from IPython.display import HTML, display
+def setBG_incell(color):
+    script = (
+        "var cell = this.closest('.jp-CodeCell');"
+        "var editor = cell.querySelector('.jp-Editor');"
+        "editor.style.background='{}';"
+        "this.parentNode.removeChild(this)"
+    ).format(color)
+    display(HTML('<img src onerror="{}" style="display:none">'.format(script)))
+def setBG_outcell(color):
+    script = (
+        "var cell = this.closest('.jp-CodeCell');"
+        "var editor = cell.querySelector('.jp-OutputArea');"
+        "editor.style.background='{}';"
+        "this.parentNode.removeChild(this)"
+    ).format(color)
+    display(HTML('<img src onerror="{}" style="display:none">'.format(script)))
+```
+%% Cell type:markdown id:87788753-5d20-4c2d-9633-f67d645b18d2 tags:
+---
+%% Cell type:markdown id:bfa294a2-fe05-4344-a715-aa45adb708c5 tags:
+# CSST主巡天仿真
+ 仿真软件git仓库: [https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim)
+  <img src="pics/csstsim_git.png" alt="csst_sim_git.png" width="200">
+%% Cell type:markdown id:bbccb71f-1932-4f91-8c01-c3f367399ae6 tags:
+## 1. 安装
+%% Cell type:markdown id:33813cc5-8343-445e-8968-ae681185e5da tags:
+## 1.1 [**CSST仿真v3.0.0 版本软件使用手册** ](https://www.kdocs.cn/l/cjyiU0SXGyn2)
+  + 获取代码: **git clone** https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim.git
+  + 配置环境: anaconda
+    + **install.sh**, **requirements.txt**
+    + python3.11
+    + numpy, astropy, scipy, galsim, healpy, h5py, toml, lmfit, psutil, pyyaml, sep, numba, cython, cfitsio, mpi4py
+  + 安装软件: **pip install -e .**
+  + 测试数据: [**data_test2024**](https://pan.baidu.com/s/1WazLj3qCXZqgU7ngw0VKbw?pwd=kbsm)
+    + PSF数据库
+      + 多色成像: data_test2024/set1_dynamic
+      + 无缝光谱: data_test2024/SLS_PSF_PCA_fp
+    + 测试星表数据
+      + 恒星星表: data_test2024/starcat_C9
+      + 星系星表: data_test2024/galcat_C9/cat2CSSTSim_bundle-50sqDeg
+      + 类星体星表: data_test2024/galcat_C9/qsosed
+      + fits数据:  data_test2024/stampCatsIndex.hdf5 (stampCats)
+      + 曝光指向数据: data_test2024/pointing50_C9
+%% Cell type:markdown id:1a443dbb-64c3-41c0-818a-e9fa4ef464cf tags:
+### 1.2 运行
+ <font color="grey"> [mpirun -np N] </font> **python3** LOCALPATH/csst_msc_sim/run_sim.py  **--catalog** C9_Catalog_fits **--config_file** config_overall_test2024.yaml  **-c** LOCALPATH/data_test2024/run_test2024/config
+%% Cell type:code id:e0d31aa9-76ce-412f-8992-5588148a9b25 tags:
+``` python
+#配置conda环境
+!conda info --envs
+```
+%% Cell type:code id:3fcceeca-8720-48aa-abdf-e3fdc90610ef tags:
+``` python
+#查看python版本
+!python --version
+```
+%% Cell type:code id:343e430e-9c51-4f88-95bb-f614ea57dfe6 tags:
+``` python
+LOCALPATH="/public/home/chengliang/csstsimTest2024"
+!ls --color $LOCALPATH
+```
+%% Cell type:code id:f3b06b32-4ac2-4b7d-bb29-9e2ae22d2402 tags:
+``` python
+#准备测试数据
+!tree -C -L 2 $LOCALPATH/data_test2024/run_test2024
+```
+%% Cell type:code id:6210ffd7-389e-4c43-a3c0-b14a461d3c14 tags:
+``` python
+#下载CSST仿真软件
+!tree -C -L 1 $LOCALPATH/csst_msc_sim
+```
+%% Cell type:code id:f31f465c-8bc8-4133-afaa-fede73b0af3a tags:
+``` python
+#进入CSST仿真软件目录并完成安装
+%cd $LOCALPATH/csst_msc_sim
+!pip install -e .
+```
+%% Cell type:code id:1d02e07d-8b2a-4f45-8e3d-cd4f79607c2f tags:
+``` python
+#检查CSST仿真软件模块
+import observation_sim
+observation_sim.__file__
+```
+%% Cell type:code id:97504b20-40b5-4f8e-a56b-69855ece6d2f tags:
+``` python
+```
+%% Cell type:markdown id:7f8285c4-90a1-43e9-bb5c-d268906963e1 tags:
+----
+%% Cell type:markdown id:9bfd2585-f7bb-4cd3-8b6e-91ebe7728aa2 tags:
+## 2.配置
+%% Cell type:markdown id:9d39722c-d24a-48e0-b070-e0e11176f5e6 tags:
+ data_test2024/run_test2024/config
+  + 全局配置: config_overall_test2024.yaml，主要是路径信息
+  + 观测配置: obs_config_SCI_WIDE_phot.yaml，主要是成像chip设置和各类效应开关
+<img src="pics/fp.png" alt="fp.png" width="400">
+  + 为了便于检查，可以先按2*8格式输出图像: **format_output: NO**
+<img src="pics/img_out2x8.png" alt="img_out2x8.png" width="400"> **$\rightarrow$** <img src="pics/img_out1x16.png" alt="img_out1x16.png" width="500">
+%% Cell type:code id:95b16f37-7a35-4a14-8807-9c405c0ba41f tags:
+``` python
+%cd $LOCALPATH/data_test2024
+```
+%% Cell type:code id:042f05fe-4252-489c-9bb8-5682a2ef7934 tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain run_test2024/config/config_overall_test2024.yaml
+setBG_outcell('black')
+```
+%% Cell type:code id:e7f6ec2a-abc5-4be1-84ad-e2c959cae26f tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain run_test2024/config/obs_config_SCI_WIDE_phot.yaml
+setBG_outcell('black')
+```
+%% Cell type:code id:7b68c16a-2fd9-4930-bc68-f6df9d6e9584 tags:
+``` python
+```
+%% Cell type:markdown id:ccab997f-9566-4f27-8c9f-5e2531e8b72d tags:
+----
+%% Cell type:markdown id:63e67d64-ef29-4f47-b9e2-7ae7fc2d0061 tags:
+## 3.多色成像
+%% Cell type:markdown id:47e51d31-5f88-4ffc-bcb0-695e4ebaa718 tags:
+ data_test2024/run_test2024/catalog/Catalog_example_test2024.py
+  + **ra, dec, z**: [e.g., 27.87411627, -47.86911686, 0.]
+  + **star**: [e.g., 0-galaxy, 1-star, 2-quasar, 3-stamp, 4-calib]
+  + **mag_use_normal**: [e.g., 20.]
+  + **sed**: [e.g., wave, flux]
+<img src="pics/img_catalogExample.png" alt="img_catalogExample.png" width="600"> <img src="pics/img_star.png" alt="img_star.png" width="400">
+%% Cell type:code id:74687bf0-3ca7-4f2d-9ad3-b9c304582b23 tags:
+``` python
+#catalog文件需要放到默认路径，即$LOCALPATH/csst_msc_sim/catalog/**
+!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/Catalog_example_test2024.py
+setBG_outcell('black')
+```
+%% Cell type:code id:cad37712-a130-4846-b6ae-ee5827f97c60 tags:
+``` python
+#修改配置文件后，运行仿真任务
+setBG_incell('honeydew')
+!srun -p debug -n 1 -c 4 python3 /public/home/chengliang/csstsimTest2024/csst_msc_sim/run_sim.py \
+                                 --catalog Catalog_example_test2024 \
+                                 --config_file config_overall_test2024.yaml \
+                                 -c /public/home/chengliang/csstsimTest2024/data_test2024/run_test2024/config
+```
+%% Cell type:code id:dffd0a5b-597a-4dc0-9880-56292cbff80d tags:
+``` python
+#检查仿真结果
+!tree $LOCALPATH/data_test2024/run_test2024/outputs/testRunX
+```
+%% Cell type:code id:a8583552-904c-427c-9bb0-4839b03ce83c tags:
+``` python
+#查看仿真星表
+!cat $LOCALPATH/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01.cat
+```
+%% Cell type:code id:00697fbb-dbd2-46ee-b79e-58cbff53a3a6 tags:
+``` python
+#查看仿真图像(ds9)
+filename = LOCALPATH + "/data_test2024/run_test2024/outputs/testRunX/10109100100413/CSST_MSC_MS_SCI_20231022050242_20231022050512_10109100100413_08_L0_V01"
+hdu = fitsio.open(filename+".fits")
+fn = open(filename+".cat", 'r')
+header1 = fn.readline()
+ximg = []
+yimg = []
+for line in fn:
+    line   = line.strip()
+    columns= line.split()
+    ximg.append(float(columns[3]))
+    yimg.append(float(columns[4]))
+ximg = np.array(ximg)
+yimg = np.array(yimg)
+plt.figure(figsize=(12,12))
+plt.imshow(np.log10(hdu[1].data), origin='lower', vmin=np.log10(438), vmax=np.log10(550), cmap='grey')
+plt.plot(ximg, yimg,c='#90EE90', marker="o", mfc="none", ms = 20)
+plt.figure(figsize=(6,6))
+plt.imshow(np.log10(hdu[1].data)[int(yimg[0])-32:int(yimg[0])+32, int(ximg[0])-32:int(ximg[0])+32], origin='lower',
+           vmin=np.log10(438), vmax=np.log10(550), cmap='grey', extent=([int(ximg[0])-32, int(ximg[0])+32, int(yimg[0])-32, int(yimg[0])+32]))
+```
+%% Cell type:markdown id:d917f8e5-c634-4797-9b3e-dee241f3659f tags:
+ data_test2024/run_test2024/catalog/C9_Catalog.py
+  + 将star/galaxy/quasar加入仿真列表: self.objs.append(obj)
+  + 大星表推荐使用healpix天区索引
+  + 更新SED的读取函数
+%% Cell type:code id:317941f2-8960-4e2f-ab81-a78af40c6952 tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog.py
+setBG_outcell('black')
+```
+%% Cell type:markdown id:6db5d367-424c-4637-b996-c02beadf2704 tags:
+  + 打开所有效应的仿真图像
+  <img src="pics/img_cti.png" alt="img_cti.png" width="1000">
+%% Cell type:code id:6948ab17-3392-437b-9c24-24e573b2c38d tags:
+``` python
+```
+%% Cell type:markdown id:d04845d2-af7d-4b7b-a97b-775b2f1ef365 tags:
+---
+%% Cell type:markdown id:143546c0-8dfb-4b9d-81e9-18e7af5e2f8a tags:
+## 4. fits贴图
+%% Cell type:markdown id:34107217-f7ba-4d58-aac2-7d4d1a67cd21 tags:
+ steps:
+  1. 准备fits库(**图像**+**SED**): RA, DEC, z, mag_use_normal, image, pixScale
+  2. 建立星表索引文件(*.hdf5): tools/index_fits_hdf5.py
+  3. **修改catalog.py** (e.g. data_test2024/run_test2024/catalog/C9_Catalog_fits.py)
+     + 将fits数据加入仿真列表:self.objs.append(obj)
+     + 更新SED的读取函数
+  5. 修改配置文件: config.yaml
+  6. 提交仿真任务
+%% Cell type:code id:ca23e1f3-b1bd-4e25-bdb3-ba01a6ebd5e6 tags:
+``` python
+!tree -L 1 $LOCALPATH/data_test2024/stampCats
+```
+%% Cell type:code id:44f8de98-a170-4c32-a889-c91fd3179b84 tags:
+``` python
+#使用index_fits_hdf5.py构建索引文件
+setBG_incell("honeydew")
+!cp $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py $LOCALPATH/data_test2024
+#python3 index_fits_hdf5.py
+```
+%% Cell type:code id:8a2e5966-d6ca-41be-9c75-4e745fe0c46b tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/catalog/C9_Catalog_fits.py
+setBG_outcell('black')
+```
+%% Cell type:markdown id:193619be-9462-4be6-8201-9da571cb0055 tags:
+  + 添加fits贴图的测试结果
+  <img src="pics/img_fits.png" alt="img_fits.png" width="1000">
+%% Cell type:code id:cd7c0489-0817-49da-9f97-db75c03b8b79 tags:
+``` python
+```
+%% Cell type:markdown id:3d9d9d3d-4ba0-45f2-8ee8-bdcbaed30ad8 tags:
+---
+%% Cell type:markdown id:2360837e-4cd9-4d26-a61c-1e662db8a783 tags:
+## 5.实用工具 (csst_msc_sim/tools)
+%% Cell type:markdown id:614ecfdf-a9e0-43c7-9aeb-02959fa4202d tags:
+### 5.1 [index_fits_hdf5.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/index_fits_hdf5.py)
+%% Cell type:markdown id:5c18c3e1-6914-4d93-94b5-94595c56f0ed tags:
+  + 目的：把fits图像库封装成仿真可读取的fits星表格式
+  + 使用方法：
+    + 准备fits图像数据库
+    + 在index_fits_hdf5.py中，指定fits图像目录dir_cat
+  + 在终端运行：
+    + python index_fits_hdf5.py
+  + 输出：代码完成后将生成fits图像索引文件，数据格式为hdf5
+  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的索引文件和对应fits图像库能够用作仿真软件的输入星表。
+%% Cell type:code id:b06f15cf-0399-4c02-9528-1e04a2efd951 tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/tools/index_fits_hdf5.py
+setBG_outcell('black')
+```
+%% Cell type:markdown id:83c3c7ef-d53b-4e55-b233-15e838ce866b tags:
+### 5.2 [get_PSF.py](https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim/-/blob/develop/tools/get_PSF.py)
+%% Cell type:markdown id:831dd90d-04b4-4030-9d87-3c19b2e9639e tags:
+  + 目的：给定焦面位置，返回对应位置处的光学PSF图像
+  + 使用方法：
+      + 给定焦面采样位置x, y = imgPos[iobj, :]，在示例中通过读取仿真输出的.cat星表来设置imgPos
+  + 在终端运行：
+      + python get_PSF.py
+  + 输出：代码完成后将生成对应位置的PSF图像
+  + 说明：该程序可独立于仿真代码单独运行，即不需要预先安装仿真代码。如已安装仿真代码，该程序使用方法不受影响。产生的PSF图像暂不包含探测器部分。
+%% Cell type:code id:263c7f7e-e96d-4616-8da1-ced7501c71c5 tags:
+``` python
+!bat --theme="Monokai Extended Bright" --plain $LOCALPATH/csst_msc_sim/tools/get_PSF.py
+setBG_outcell('black')
+```
+%% Cell type:markdown id:24effd04-2110-41ff-83dd-879b72aaccfd tags:
+### 5.3 ...
+%% Cell type:code id:5df4f734-d89a-4984-acae-4f01973a81ad tags:
+``` python
+```
--- a/pics/.DS_Store
+++ b/pics/.DS_Store
--- a/pics/csstsim_git.png
+++ b/pics/csstsim_git.png
--- a/pics/fp.png
+++ b/pics/fp.png
--- a/pics/img_catalogExample.png
+++ b/pics/img_catalogExample.png
--- a/pics/img_cti.png
+++ b/pics/img_cti.png
--- a/pics/img_fits.png
+++ b/pics/img_fits.png
--- a/pics/img_out1x16.png
+++ b/pics/img_out1x16.png
--- a/pics/img_out2x8.png
+++ b/pics/img_out2x8.png
--- a/pics/img_star.png
+++ b/pics/img_star.png