diff --git a/CHANGELOG.en.md b/CHANGELOG.en.md new file mode 100644 index 0000000000000000000000000000000000000000..f6ef6568e574a5f6d8328151ff7652155b82961c --- /dev/null +++ b/CHANGELOG.en.md @@ -0,0 +1,52 @@ +[English](CHANGELOG.en.md) | [中文](CHANGELOG.md) +* 2026.05.13: Updated to version v3.4. Changes include: + * Added ghost image simulation module + * Updated package dependencies + * Updated the dark current and hot pixel modules to support importing ground-test dark current data through the `input_dark` configuration parameter. +* 2025.07.10: Updated to version v3.3 for the 1000 square degree wide survey and 50 square degree deep survey simulations. Changes include: + * Updated several detector parameters based on the latest measurements + * Added slitless spectroscopy PSF and LED image generation tools + * Updated spectroscopy dispersion files + * Updated system throughput curves + * Updated FITS header keywords + * Added the `data_set` keyword in the config file + +* 2025.05.21: Updated to version v3.2. Changes include: + * Added bright-star PSF extrapolation module + * Added crosstalk simulation module + * Memory optimization for simulations + * Updated FITS header keywords + * Added charge diffusion in the slitless spectroscopy PSF model + * Updated PEP8 formatting conventions + * Bug fixes: truncated magnitude calculation, deep field calculation, slitless spectroscopy PSF, saturation overflow, and truncation issues in channel trailing caused by bad columns + +* 2024.08.03: Updated to version v3.1. Changes include: + * Added simulation of the guide-star stabilization process + * Added Milky Way extinction simulation + * Bug fixes: fixed issues such as inconsistent simulated magnitudes caused by filter light leakage + +* 2024.05.15: Updated to version v3.0. See the [Software Documentation](https://kdocs.cn/l/cjyiU0SXGyn2) for details of the new features. + +* 2023.07.29: Updated to version v2.1. Changes include: + * Added stray-light simulation module with config switch: `enable_straylight_model: True/False` + * Updated WCS definition: `x: E`, `y: -N` + * Updated FITS header keywords + * Adapted to the C6 50-square-degree simulation catalogs + * Bug fixes: fixed incorrect distortion directions caused by field distortion and related issues + +* 2023.05.01: Updated to version v2.0. New features include: + * C6 simulation support + * FITS image overlay functionality + * Individually configurable focal-plane detector properties + * One-degree rotation for slitless spectroscopy detectors + * Simulation of source distortion caused by field distortion + * Guide-star detector simulation + * Updated instrument parameters and bug fixes + +* 2022.08.23: Updated to the latest throughput curves and added version information to FITS image headers. Simulation software version numbering was reset; current version is v1.0.4. + +* 2022.06.20: Version v0.5 released on the `release_v0.5` branch (Note: version numbering has since been reset to v1.0; current branch is `release_v1.0.0`) + +* 2022.04.13: Fixed a calling error in the astrometry module + +* 2022.04.08: Fixed issues related to the catalog-only output option (`out_cat_only`) \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000000000000000000000000000000000000..ceb656628412d7f40abfdd1e5a65ea9653dffa8e --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,52 @@ +[English](CHANGELOG.en.md) | [中文](CHANGELOG.md) +* 2026.05.13: 更新至v3.4版本，内容包括： + * 增加鬼像仿真模块 + * 更新软件包依赖项 + * 更新暗电流和热像元模块，通过配置`input_dark`参数导入暗电流的地面测试数据 +* 2025.07.10: 针对1000平方度宽场及50平方度深场仿真，更新至v3.3版本，内容包括： + * 根据最新实测结果，更新部分探测器参数 + * 加入光谱PSF、LED图像获取工具 + * 更新光谱分光文件 + * 更新系统效率曲线 + * 更新头文件关键字 + * 在config中加入data_set关键字 + +* 2025.05.21: 更新至v3.2版本，内容包括： + * 加入亮星PSF外插模块 + * 加入串扰模块 + * 仿真内存优化 + * 头文件关键字更新 + * 在无缝光谱PSF模型中添加电子弥散 + * 更新PEP8格式规范 + * BUG修复：截断星等计算、深场计算、无缝光谱PSF、饱和溢出和坏像列在通道间的拖尾截断等 + +* 2024.08.03: 更新至v3.1版本，内容包括： + * 加入对导星稳像过程的仿真 + * 加入银河系消光的仿真 + * Bug修复：修复因滤光片漏光导致的仿真星等不吻合等bug + +* 2024.05.15: 更新至v3.0版本，新功能具体内容见[软件说明文档](https://kdocs.cn/l/cjyiU0SXGyn2)。 + +* 2023.07.29: 更新至v2.1版本，内容包括： + * 加入杂散光模块，config文件中添加开关：enable_straylight_model: True/False + * 调整WCS定义：x：E, y：-N + * 头文件关键字调整 + * 适配C6-50平方度仿真星表 + * Bug修复：修复像场畸变导致形变的方向错误等bug + +* 2023.05.01: 更新至v2.0版本，新功能包含： + * C6仿真实现 + * fits贴图功能 + * 焦面探测器属性单独定义 + * 无缝光谱探测器一度旋转 + * 像场畸导致目标形变的仿真 + * 导星探测器仿真 + * 仪器参数更新，bug修复等 + +* 2022.08.23: 更新至最新的透过率曲线，在图像头文件中加入版本号。仿真软件版本号进行重新标记，当前版本为1.0.4 + +* 2022.06.20: 版本v0.5上线，release_v0.5 branch(注：目前已重新设置为v1.0，release_v1.0.0) + +* 2022.04.13: 修复了天测模块中的调用错误 + +* 2022.04.08: 修复了仅输出星表选项("out_cat_only")存在的问题 \ No newline at end of file diff --git a/README.en.md b/README.en.md new file mode 100644 index 0000000000000000000000000000000000000000..20d79722f4007cb6e3995d68e4268bec2a20563b --- /dev/null +++ b/README.en.md @@ -0,0 +1,160 @@ +[English](README.en.md) | [中文](README.md) +# CSST Main Survey Simulation Software + + + + + + + + + +## Usage and Documentation + +* Installation and usage instructions are available in the [Software Documentation](https://kdocs.cn/l/cjyiU0SXGyn2) (continuously updated). + + + +* Please use GitLab Issues for bug reports and feedback. If you do not have a NAOC GitLab account, please use the [Feedback Form](https://f.kdocs.cn/w/XYcBkE63/). + + + +## Simulation Data Products + +* Cycle 9 simulation data products: [CSST Main Survey Cycle 9 Data Product Description](https://pan.cstcloud.cn/s/gHVB8p2TDA) + +* Previously released simulation data products: [CSST Main Survey Historical Simulation Data Products](https://pan.cstcloud.cn/s/zRF7mu3ET5c) + + + + + + +## Related Input Data + +* Data link: https://pan.baidu.com/s/1cVHovDs4cMsDWWNkoZDRDA?pwd=y63x + + * PSF data: + * Multiband imaging: + `CSSTSim_C9/PSFCube/set1_dynamic` + * Slitless spectroscopy: + `CSSTSim_C9/PSFCube/SLS_PSF_PCA_fp` + + * C9 catalog data: + * Stellar catalogs: + `CSSTSim_C9/starcat_C9` + * Galaxy catalogs: + `CSSTSim_C9/galaxycat_C9/cat2CSSTSim_bundle-50sqDeg` + * Quasar catalogs: + `CSSTSim_C9/galaxycat_C9/qsosed` + + + +## Citation + +> **Note:** The software and related data products in this project are intended **for scientific research and academic exchange only**. + +If you use this software or related datasets in your research, please cite the corresponding papers and acknowledge this project in your publication or report. + +| Category | Paper Title | DOI | +| :--- | :--- | :--- | +| **Simulation Software and Data** | Mock Observations for the CSST Mission: Main Surveys─An Overview of Framework and Simulation Suite | [10.1088/1674-4527/ae20fe](https://doi.org/10.1088/1674-4527/ae20fe) | +| **Simulation Software – Slitless Spectroscopy** | Mock Observations for the CSST Mission: Main Surveys—The Slitless Spectroscopy Simulation | [10.1088/1674-4527/ae20fa](https://doi.org/10.1088/1674-4527/ae20fa) | +| **Simulation Software – Galaxy Catalogs** | Mock Observations for the CSST Mission: Main Surveys─The Mock Catalog | [10.1088/1674-4527/ae20ff](https://doi.org/10.1088/1674-4527/ae20ff) | +| **Point Spread Function Data** | Mock Observations for the CSST Mission: End-to-end Performance Modeling of Optical System | [10.1088/1674-4527/ae20fb](https://doi.org/10.1088/1674-4527/ae20fb) | + +--- + +Thank you for your support and recognition of this project. + + + + +## Related Meetings and Workshops + +* May 8, 2023, Quanzhou, Fujian — CSST Scientific Data Processing System and Science Team Workshop: [Workshop Video](https://pan.baidu.com/s/1VH8rg6jC09ZsVlMmO2zhbQ), Video extraction code: `7hsu` + +* April 6, 2022, National Astronomical Observatories — Software Release and Mini Workshop Information: [20220406](https://www.kdocs.cn/l/cnovDlrgyDlu), [Workshop Video](https://pan.baidu.com/s/1eCtLTi3EZ8f6KQGvmqjp_Q?pwd=ow9q), Video extraction code: `ow9q` + + + +## Development Notes + +* Development workflow for this GitLab repository: [CSST Main Survey Simulation GitLab Workflow](https://kdocs.cn/l/cnvJY7RkzwW9) diff --git a/README.md b/README.md index f7d5fc271be1c03623d9593ae484f1fea37d8ed9..92ef2df89eda2bfaf6734970cb3feb8acb69c705 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ +[English](README.en.md) | [中文](README.md) # CSST主巡天仿真软件 -## 重要更新或问题修复： + ## 使用方法及相关说明 @@ -70,6 +71,22 @@ * 星系星表：CSSTSim_C9/galaxycat_C9/cat2CSSTSim_bundle-50sqDeg * 类星体星表：CSSTSim_C9/galaxycat_C9/qsosed +## 引用说明 +> **注意**：本项目的软件及相关数据产品**仅供科研与学术交流使用**。 + +如果您在研究工作中使用了本项目的软件或相关数据，请引用相关论文，并在论文或报告中注明本项目。 + +| | 论文标题 | DOI 链接 | +| :--- | :--- | :--- | +| **仿真软件及数据** | Mock Observations for the CSST Mission: Main Surveys─An Overview of Framework and Simulation Suite | [10.1088/1674-4527/ae20fe](https://doi.org/10.1088/1674-4527/ae20fe) | +| **仿真软件-无缝光谱** | Mock Observations for the CSST Mission: Main Surveys—The Slitless Spectroscopy Simulation | [10.1088/1674-4527/ae20fa](https://doi.org/10.1088/1674-4527/ae20fa) | +| **仿真软件-星系星表** | Mock Observations for the CSST Mission: Main Surveys─The Mock Catalog | [10.1088/1674-4527/ae20ff](https://doi.org/10.1088/1674-4527/ae20ff) | +| **点扩散函数数据** | Mock Observations for the CSST Mission: End-to-end Performance Modeling of Optical System | [10.1088/1674-4527/ae20fb](https://doi.org/10.1088/1674-4527/ae20fb) | + +--- +感谢对本项目工作的支持与认可。 + + ## 相关会议及workshop * 2023年5月8日，福建泉州，CSST科学数据处理系统与科学团队讨论会：[会议视频](https://pan.baidu.com/s/1VH8rg6jC09ZsVlMmO2zhbQ)，视频提取码：7hsu diff --git a/catalog/C10_Catalog.py b/catalog/C10_Catalog.py index 45f7a0d05e23f1a4b98dcf4d3b1789efc4b55aed..016cceb4cddb4a96833b877e2727cae22c6b32f1 100644 --- a/catalog/C10_Catalog.py +++ b/catalog/C10_Catalog.py @@ -1,26 +1,25 @@ import os import galsim -import random import copy import numpy as np import h5py as h5 import healpy as hp import astropy.constants as cons import traceback -from astropy.coordinates import spherical_to_cartesian from astropy.table import Table from scipy import interpolate from datetime import datetime from observation_sim.mock_objects import CatalogBase, Star, Galaxy, Quasar, ExtinctionMW -from observation_sim.mock_objects._util import tag_sed, getObservedSED, getABMAG, integrate_sed_bandpass, comoving_dist +from observation_sim.mock_objects._util import ( + getObservedSED, + getABMAG, + integrate_sed_bandpass, +) from observation_sim.astrometry.Astrometry_util import on_orbit_obs_position # (TEST) from astropy.cosmology import FlatLambdaCDM -from astropy import constants -from astropy import units as U -from astropy.coordinates import SkyCoord from astropy.io import fits import astropy.constants as atcons @@ -34,16 +33,28 @@ except ImportError: NSIDE = 128 -pointing_file_list = ['pointing_C10_1000deg_lat80.dat', 'pointing_C10_1000deg_latm80_.dat', 'pointing_C10_1000deg_p180_m30.dat', 'pointing_C10_1000deg_p320_m40.dat'] -star_file_list = ['CSST_C10_1000sqd_1.hdf5', 'CSST_C10_1000sqd_2_Fornax.hdf5', 'CSST_C10_1000sqd_3.hdf5', 'CSST_C10_1000sqd_4.hdf5'] +pointing_file_list = [ + "pointing_C10_1000deg_lat80.dat", + "pointing_C10_1000deg_latm80_.dat", + "pointing_C10_1000deg_p180_m30.dat", + "pointing_C10_1000deg_p320_m40.dat", +] +star_file_list = [ + "CSST_C10_1000sqd_1.hdf5", + "CSST_C10_1000sqd_2_Fornax.hdf5", + "CSST_C10_1000sqd_3.hdf5", + "CSST_C10_1000sqd_4.hdf5", +] def get_galaxy_qso_list(config): cat_dir = config["catalog_options"]["input_path"]["cat_dir"] - with open(cat_dir+"galcat_C10/qsocat/gal_C10_file", 'r', encoding='utf-8') as fn1: + with open(cat_dir + "galcat_C10/qsocat/gal_C10_file", "r", encoding="utf-8") as fn1: fn1_list = fn1.readlines() bundle_file_list = [line.strip() for line in fn1_list] - with open(cat_dir+"galcat_C10/qsocat/qso_sed_C10_file", 'r', encoding='utf-8') as fn2: + with open( + cat_dir + "galcat_C10/qsocat/qso_sed_C10_file", "r", encoding="utf-8" + ) as fn2: fn2_list = fn2.readlines() qsosed_file_list = [line.strip() for line in fn2_list] return bundle_file_list, qsosed_file_list @@ -51,16 +62,19 @@ def get_galaxy_qso_list(config): class StarParm(ctypes.Structure): _fields_ = [ - ('logte', ctypes.c_float), - ('logg', ctypes.c_float), - ('Mass', ctypes.c_float), - ('Av', ctypes.c_float), - ('mu0', ctypes.c_float), - ('Z', ctypes.c_float)] + ("logte", ctypes.c_float), + ("logg", ctypes.c_float), + ("Mass", ctypes.c_float), + ("Av", ctypes.c_float), + ("mu0", ctypes.c_float), + ("Z", ctypes.c_float), + ] def get_star_cat(config): - idx = pointing_file_list.index(os.path.basename(config['obs_setting']['pointing_file'])) + idx = pointing_file_list.index( + os.path.basename(config["obs_setting"]["pointing_file"]) + ) return_star_path = star_file_list[idx] return return_star_path @@ -68,13 +82,13 @@ def get_star_cat(config): def get_bundleIndex(healpixID_ring, bundleOrder=4, healpixOrder=7): assert NSIDE == 2**healpixOrder shift = healpixOrder - bundleOrder - shift = 2*shift + shift = 2 * shift nside_bundle = 2**bundleOrder nside_healpix = 2**healpixOrder healpixID_nest = hp.ring2nest(nside_healpix, healpixID_ring) - bundleID_nest = (healpixID_nest >> shift) + bundleID_nest = healpixID_nest >> shift bundleID_ring = hp.nest2ring(nside_bundle, bundleID_nest) return bundleID_ring @@ -96,48 +110,71 @@ class Catalog(CatalogBase): self.filt = filt self.logger = chip_output.logger - with pkg_resources.path('catalog.data', 'SLOAN_SDSS.g.fits') as filter_path: + with pkg_resources.path("catalog.data", "SLOAN_SDSS.g.fits") as filter_path: self.normF_star = Table.read(str(filter_path)) self.config = config self.chip = chip self.pointing = pointing - self.max_size = 0. + self.max_size = 0.0 # [TODO] Milky Way extinction - if "enable_mw_ext_gal" in config["catalog_options"] and config["catalog_options"]["enable_mw_ext_gal"]: + if ( + "enable_mw_ext_gal" in config["catalog_options"] + and config["catalog_options"]["enable_mw_ext_gal"] + ): if "planck_ebv_map" not in config["catalog_options"]: raise ValueError( - "Planck dust map must be given to enable Milky Way extinction calculation for galaxies.") + "Planck dust map must be given to enable Milky Way extinction calculation for galaxies." + ) self.mw_ext = ExtinctionMW() self.mw_ext.init_ext_model(model_name="odonnell") self.mw_ext.load_Planck_ext( - file_path=config["catalog_options"]["planck_ebv_map"]) + file_path=config["catalog_options"]["planck_ebv_map"] + ) - if "star_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["star_cat"] and not config["catalog_options"]["galaxy_only"]: + if ( + "star_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["star_cat"] + and not config["catalog_options"]["galaxy_only"] + ): # Get the cloest star catalog file star_file_name = get_star_cat(self.config) - star_path = os.path.join(config["catalog_options"]["input_path"]["star_cat"], star_file_name) + star_path = os.path.join( + config["catalog_options"]["input_path"]["star_cat"], star_file_name + ) self.star_path = os.path.join(self.cat_dir, star_path) - self.star_SED_path = config["catalog_options"]["SED_templates_path"]["star_SED"] + self.star_SED_path = config["catalog_options"]["SED_templates_path"][ + "star_SED" + ] self._load_SED_lib_star() - if "galaxy_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["galaxy_cat"] and not config["catalog_options"]["star_only"]: + if ( + "galaxy_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["galaxy_cat"] + and not config["catalog_options"]["star_only"] + ): galaxy_dir = config["catalog_options"]["input_path"]["galaxy_cat"] self.galaxy_path = os.path.join(self.cat_dir, galaxy_dir) - self.galaxy_SED_path = config["catalog_options"]["SED_templates_path"]["galaxy_SED"] + self.galaxy_SED_path = config["catalog_options"]["SED_templates_path"][ + "galaxy_SED" + ] self._load_SED_lib_gals() self.agn_seds = {} - if "AGN_SED" in config["catalog_options"]["SED_templates_path"] and not config["catalog_options"]["star_only"]: - self.AGN_SED_path = config["catalog_options"]["SED_templates_path"]["AGN_SED"] + if ( + "AGN_SED" in config["catalog_options"]["SED_templates_path"] + and not config["catalog_options"]["star_only"] + ): + self.AGN_SED_path = config["catalog_options"]["SED_templates_path"][ + "AGN_SED" + ] if "rotateEll" in config["catalog_options"]: - self.rotation = np.radians( - float(config["catalog_options"]["rotateEll"])) + self.rotation = np.radians(float(config["catalog_options"]["rotateEll"])) else: - self.rotation = 0. + self.rotation = 0.0 # Update output .cat header with catalog specific output columns self._add_output_columns_header() @@ -147,23 +184,30 @@ class Catalog(CatalogBase): def _add_output_columns_header(self): self.add_hdr = " av stellarmass dm teff logg feh" - self.add_hdr += " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + self.add_hdr += ( + " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + ) self.add_fmt = "%8.4f %8.4f %8.4f %8.4f %8.4f %8.4f" - self.add_fmt += " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " - self.chip_output.update_output_header( - additional_column_names=self.add_hdr) + self.add_fmt += ( + " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " + ) + self.chip_output.update_output_header(additional_column_names=self.add_hdr) def _get_healpix_list(self): self.sky_coverage = self.chip.getSkyCoverageEnlarged( - self.chip.img.wcs, margin=0.2) - ra_min, ra_max, dec_min, dec_max = self.sky_coverage.xmin, self.sky_coverage.xmax, self.sky_coverage.ymin, self.sky_coverage.ymax + self.chip.img.wcs, margin=0.2 + ) + ra_min, ra_max, dec_min, dec_max = ( + self.sky_coverage.xmin, + self.sky_coverage.xmax, + self.sky_coverage.ymin, + self.sky_coverage.ymax, + ) ra = np.deg2rad(np.array([ra_min, ra_max, ra_max, ra_min])) dec = np.deg2rad(np.array([dec_max, dec_max, dec_min, dec_min])) self.pix_list = hp.query_polygon( - NSIDE, - hp.ang2vec(np.radians(90.) - dec, ra), - inclusive=True + NSIDE, hp.ang2vec(np.radians(90.0) - dec, ra), inclusive=True ) if self.logger is not None: msg = str(("HEALPix List: ", self.pix_list)) @@ -184,44 +228,64 @@ class Catalog(CatalogBase): # self.tempSED_star = h5.File(self.star_SED_path,'r') def _load_SED_lib_star(self): # self.tempSED_star = h5.File(self.star_SED_path,'r') - with pkg_resources.path('catalog.data', 'starSpecInterp.so') as ddl_path: + with pkg_resources.path("catalog.data", "starSpecInterp.so") as ddl_path: self.starDDL = ctypes.CDLL(str(ddl_path)) self.starDDL.loadSpecLibs.argtypes = [ctypes.c_char_p, ctypes.c_char_p] self.starDDL.loadExts.argtypes = [ctypes.c_char_p] - nwv = self.starDDL.loadSpecLibs(str.encode(os.path.join( - self.star_SED_path, 'file_BT-Settl_CSST_wl1000-24000_R1000.par')), str.encode(self.star_SED_path)) - self.starDDL.loadExts(str.encode(os.path.join( - self.star_SED_path, "Ext_odonnell94_R3.1_CSST_wl1000-24000_R1000.fits"))) + nwv = self.starDDL.loadSpecLibs( + str.encode( + os.path.join( + self.star_SED_path, "file_BT-Settl_CSST_wl1000-24000_R1000.par" + ) + ), + str.encode(self.star_SED_path), + ) + self.starDDL.loadExts( + str.encode( + os.path.join( + self.star_SED_path, + "Ext_odonnell94_R3.1_CSST_wl1000-24000_R1000.fits", + ) + ) + ) self.star_spec_len = nwv def _interp_star_sed(self, obj): - spec = (ctypes.c_float*self.star_spec_len)() - wave = (ctypes.c_float*self.star_spec_len)() + spec = (ctypes.c_float * self.star_spec_len)() + wave = (ctypes.c_float * self.star_spec_len)() self.starDDL.interpSingleStar.argtypes = [ - ctypes.Structure, ctypes.POINTER(ctypes.c_float)] + ctypes.Structure, + ctypes.POINTER(ctypes.c_float), + ] # s=Star(obj.param['teff'], obj.param['grav''], obj.paramstar['mwmsc_mass'], obj.param['AV'], obj.param['DM'], obj.param['z_met']) - s = StarParm(obj.param['teff'], obj.param['logg'], obj.param['stellarMass'], - obj.param['av'], obj.param['DM'], obj.param['feh']) + s = StarParm( + obj.param["teff"], + obj.param["logg"], + obj.param["stellarMass"], + obj.param["av"], + obj.param["DM"], + obj.param["feh"], + ) self.starDDL.interpSingleStar(s, spec, wave) - rv_c = obj.param['rv']/(atcons.c.value/1000.) - Doppler_factor = np.sqrt((1+rv_c)/(1-rv_c)) - wave_RV = wave*Doppler_factor + rv_c = obj.param["rv"] / (atcons.c.value / 1000.0) + Doppler_factor = np.sqrt((1 + rv_c) / (1 - rv_c)) + wave_RV = wave * Doppler_factor return wave_RV, np.power(10, spec[:]) def _load_SED_lib_gals(self): pcs = h5.File(os.path.join(self.galaxy_SED_path, "pcs.h5"), "r") lamb = h5.File(os.path.join(self.galaxy_SED_path, "lamb.h5"), "r") - self.lamb_gal = lamb['lamb'][()] - self.pcs = pcs['pcs'][()] + self.lamb_gal = lamb["lamb"][()] + self.pcs = pcs["pcs"][()] def _load_gals(self, gals, pix_id=None, cat_id=0, agnsed_file=""): - ngals = len(gals['ra']) + ngals = len(gals["ra"]) # Apply astrometric modeling - ra_arr = gals['ra'][:] - dec_arr = gals['dec'][:] + ra_arr = gals["ra"][:] + dec_arr = gals["dec"][:] if self.config["obs_setting"]["enable_astrometric_model"]: ra_list = ra_arr.tolist() dec_list = dec_arr.tolist() @@ -248,11 +312,14 @@ class Catalog(CatalogBase): input_vz=self.pointing.sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) # [TODO] get Milky Way extinction AVs - if "enable_mw_ext_gal" in self.config["catalog_options"] and self.config["catalog_options"]["enable_mw_ext_gal"]: + if ( + "enable_mw_ext_gal" in self.config["catalog_options"] + and self.config["catalog_options"]["enable_mw_ext_gal"] + ): MW_Av_arr = self.mw_ext.Av_from_Planck(ra=ra_arr, dec=dec_arr) else: MW_Av_arr = np.zeros(len(ra_arr)) @@ -263,128 +330,165 @@ class Catalog(CatalogBase): # break param = self.initialize_param() - param['ra'] = ra_arr[igals] - param['dec'] = dec_arr[igals] - param['ra_orig'] = gals['ra'][igals] - param['dec_orig'] = gals['dec'][igals] + param["ra"] = ra_arr[igals] + param["dec"] = dec_arr[igals] + param["ra_orig"] = gals["ra"][igals] + param["dec_orig"] = gals["dec"][igals] # [TODO] get Milky Way extinction AVs - param['mw_Av'] = MW_Av_arr[igals] + param["mw_Av"] = MW_Av_arr[igals] - if not self.chip.isContainObj(ra_obj=param['ra'], dec_obj=param['dec'], margin=200): + if not self.chip.isContainObj( + ra_obj=param["ra"], dec_obj=param["dec"], margin=200 + ): continue # param['mag_use_normal'] = gals['mag_csst_%s'%(self.filt.filter_type)][igals] # if self.filt.is_too_dim(mag=param['mag_use_normal'], margin=self.config["obs_setting"]["mag_lim_margin"]): # continue - param['z'] = gals['redshift'][igals] - param['model_tag'] = 'None' - param['g1'] = gals['shear'][igals][0] - param['g2'] = gals['shear'][igals][1] - param['kappa'] = gals['kappa'][igals] - param['e1'] = gals['ellipticity_true'][igals][0] - param['e2'] = gals['ellipticity_true'][igals][1] + param["z"] = gals["redshift"][igals] + param["model_tag"] = "None" + param["g1"] = gals["shear"][igals][0] + param["g2"] = gals["shear"][igals][1] + param["kappa"] = gals["kappa"][igals] + param["e1"] = gals["ellipticity_true"][igals][0] + param["e2"] = gals["ellipticity_true"][igals][1] # For shape calculation - param['e1'], param['e2'], param['ell_total'] = self.rotate_ellipticity( - e1=gals['ellipticity_true'][igals][0], - e2=gals['ellipticity_true'][igals][1], + param["e1"], param["e2"], param["ell_total"] = self.rotate_ellipticity( + e1=gals["ellipticity_true"][igals][0], + e2=gals["ellipticity_true"][igals][1], rotation=self.rotation, - unit='radians') + unit="radians", + ) # param['ell_total'] = np.sqrt(param['e1']**2 + param['e2']**2) - if param['ell_total'] > 0.9: + if param["ell_total"] > 0.9: continue # phi_e = cmath.phase(complex(param['e1'], param['e2'])) # param['e1'] = param['ell_total'] * np.cos(phi_e + 2*self.rotation) # param['e2'] = param['ell_total'] * np.sin(phi_e + 2*self.rotation) - param['e1_disk'] = param['e1'] - param['e2_disk'] = param['e2'] - param['e1_bulge'] = param['e1'] - param['e2_bulge'] = param['e2'] + param["e1_disk"] = param["e1"] + param["e2_disk"] = param["e2"] + param["e1_bulge"] = param["e1"] + param["e2_bulge"] = param["e2"] - param['delta_ra'] = 0 - param['delta_dec'] = 0 + param["delta_ra"] = 0 + param["delta_dec"] = 0 # Masses - param['bulgemass'] = gals['bulgemass'][igals] - param['diskmass'] = gals['diskmass'][igals] + param["bulgemass"] = gals["bulgemass"][igals] + param["diskmass"] = gals["diskmass"][igals] - np.random.seed(int(pix_id)+cat_id+igals) - param['size'] = (gals['size'][igals] * ((1.+gals['redshift'][igals])**(0.4))) * np.random.uniform(0.7, 1.3) - if param['size'] > self.max_size: - self.max_size = param['size'] + np.random.seed(int(pix_id) + cat_id + igals) + param["size"] = ( + gals["size"][igals] * (1.0 + gals["redshift"][igals]) + ) * np.random.uniform(0.7, 1.3) + if param["size"] > self.max_size: + self.max_size = param["size"] # Sersic index - param['disk_sersic_idx'] = 1. - param['bulge_sersic_idx'] = 4. + param["disk_sersic_idx"] = 1.0 + param["bulge_sersic_idx"] = 4.0 # Sizes - param['bfrac'] = param['bulgemass'] / \ - (param['bulgemass'] + param['diskmass']) - if param['bfrac'] >= 0.6: - param['hlr_bulge'] = param['size'] - param['hlr_disk'] = param['size'] * (1. - param['bfrac']) + param["bfrac"] = param["bulgemass"] / ( + param["bulgemass"] + param["diskmass"] + ) + if param["bfrac"] >= 0.6: + param["hlr_bulge"] = param["size"] + param["hlr_disk"] = param["size"] * (1.0 - param["bfrac"]) else: - param['hlr_disk'] = param['size'] - param['hlr_bulge'] = param['size'] * param['bfrac'] + param["hlr_disk"] = param["size"] + param["hlr_bulge"] = param["size"] * param["bfrac"] # SED coefficients - param['coeff'] = gals['coeff'][igals] - param['detA'] = gals['detA'][igals] + param["coeff"] = gals["coeff"][igals] + param["detA"] = gals["detA"][igals] # Others - param['galType'] = gals['type'][igals] - param['veldisp'] = gals['veldisp'][igals] + param["galType"] = gals["type"][igals] + param["veldisp"] = gals["veldisp"][igals] # TEST no redening and no extinction - param['av'] = 0.0 - param['redden'] = 0 + param["av"] = 0.0 + param["redden"] = 0 # TEMP self.ids += 1 - param['id'] = '%06d' % (int(pix_id)) + \ - '%06d' % (cat_id) + '%08d' % (igals) + param["id"] = "%06d" % (int(pix_id)) + "%06d" % (cat_id) + "%08d" % (igals) # Is this an Quasar? - param['qsoindex'] = gals['qsoindex'][igals] - if param['qsoindex'] == -1: - param['star'] = 0 # Galaxy - param['agnsed_file'] = "" + param["qsoindex"] = gals["qsoindex"][igals] + if param["qsoindex"] == -1: + param["star"] = 0 # Galaxy + param["agnsed_file"] = "" obj = Galaxy(param, logger=self.logger) else: param_qso = copy.deepcopy(param) - param_qso['star'] = 2 # Quasar - param_qso['agnsed_file'] = agnsed_file + param_qso["star"] = 2 # Quasar + param_qso["agnsed_file"] = agnsed_file # First add QSO model obj = Quasar(param_qso, logger=self.logger) # Need to deal with additional output columns - obj.additional_output_str = self.add_fmt % (0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., - 0, 0.) + obj.additional_output_str = self.add_fmt % ( + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0, + 0.0, + ) self.objs.append(obj) # Then add host galaxy model - param['star'] = 0 # Galaxy - param['agnsed_file'] = "" + param["star"] = 0 # Galaxy + param["agnsed_file"] = "" obj = Galaxy(param, logger=self.logger) # Need to deal with additional output columns for (host) galaxy - obj.additional_output_str = self.add_fmt % (0., 0., 0., 0., 0., 0., - param['bulgemass'], param['diskmass'], param['detA'], - param['e1'], param['e2'], param['kappa'], param['g1'], param['g2'], param['size'], - param['galType'], param['veldisp']) + obj.additional_output_str = self.add_fmt % ( + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + param["bulgemass"], + param["diskmass"], + param["detA"], + param["e1"], + param["e2"], + param["kappa"], + param["g1"], + param["g2"], + param["size"], + param["galType"], + param["veldisp"], + ) self.objs.append(obj) def _load_stars(self, stars, pix_id=None): - nstars = len(stars['RA']) + nstars = len(stars["RA"]) # Apply astrometric modeling ra_arr = stars["RA"][:] dec_arr = stars["DEC"][:] - pmra_arr = stars['pmra'][:] - pmdec_arr = stars['pmdec'][:] - rv_arr = stars['RV'][:] - parallax_arr = stars['parallax'][:] + pmra_arr = stars["pmra"][:] + pmdec_arr = stars["pmdec"][:] + rv_arr = stars["RV"][:] + parallax_arr = stars["parallax"][:] if self.config["obs_setting"]["enable_astrometric_model"]: ra_list = ra_arr.tolist() dec_list = dec_arr.tolist() @@ -411,7 +515,7 @@ class Catalog(CatalogBase): input_vz=self.pointing.sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) for istars in range(nstars): # # (TEST) @@ -419,33 +523,35 @@ class Catalog(CatalogBase): # break param = self.initialize_param() - param['ra'] = ra_arr[istars] - param['dec'] = dec_arr[istars] - param['ra_orig'] = stars["RA"][istars] - param['dec_orig'] = stars["DEC"][istars] - param['pmra'] = pmra_arr[istars] - param['pmdec'] = pmdec_arr[istars] - param['rv'] = rv_arr[istars] - param['parallax'] = parallax_arr[istars] - if not self.chip.isContainObj(ra_obj=param['ra'], dec_obj=param['dec'], margin=200): + param["ra"] = ra_arr[istars] + param["dec"] = dec_arr[istars] + param["ra_orig"] = stars["RA"][istars] + param["dec_orig"] = stars["DEC"][istars] + param["pmra"] = pmra_arr[istars] + param["pmdec"] = pmdec_arr[istars] + param["rv"] = rv_arr[istars] + param["parallax"] = parallax_arr[istars] + if not self.chip.isContainObj( + ra_obj=param["ra"], dec_obj=param["dec"], margin=200 + ): continue - param['mag_use_normal'] = stars['app_SDSS_g'][istars] + param["mag_use_normal"] = stars["app_SDSS_g"][istars] self.ids += 1 - param['id'] = '%06d' % (int(pix_id)) + '%08d' % (istars) + param["id"] = "%06d" % (int(pix_id)) + "%08d" % (istars) # param['sed_type'] = istars # param['model_tag'] = '' - param['teff'] = stars['teff'][istars] - param['logg'] = stars['grav'][istars] - param['feh'] = stars['z_met'][istars] - param['stellarMass'] = stars['mass'][istars] + param["teff"] = stars["teff"][istars] + param["logg"] = stars["grav"][istars] + param["feh"] = stars["z_met"][istars] + param["stellarMass"] = stars["mass"][istars] - param['av'] = stars['AV'][istars] - param['DM'] = stars['DM'][istars] + param["av"] = stars["AV"][istars] + param["DM"] = stars["DM"][istars] # param['z_met'] = stars['z_met'][istars] - param['z'] = 0.0 - param['star'] = 1 # Star + param["z"] = 0.0 + param["star"] = 1 # Star try: obj = Star(param, logger=self.logger) @@ -453,8 +559,25 @@ class Catalog(CatalogBase): print(e) # Append additional output columns to the .cat file - obj.additional_output_str = self.add_fmt % (param["av"], param['stellarMass'], param['DM'], param['teff'], param['logg'], param['feh'], - 0., 0., 0., 0., 0., 0., 0., 0., 0., -1, 0.) + obj.additional_output_str = self.add_fmt % ( + param["av"], + param["stellarMass"], + param["DM"], + param["teff"], + param["logg"], + param["feh"], + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1, + 0.0, + ) self.objs.append(obj) @@ -466,8 +589,12 @@ class Catalog(CatalogBase): self.objs = [] self.ids = 0 - if "star_cat" in self.config["catalog_options"]["input_path"] and self.config["catalog_options"]["input_path"]["star_cat"] and not self.config["catalog_options"]["galaxy_only"]: - star_cat = h5.File(self.star_path, 'r')['star_catalog'] + if ( + "star_cat" in self.config["catalog_options"]["input_path"] + and self.config["catalog_options"]["input_path"]["star_cat"] + and not self.config["catalog_options"]["galaxy_only"] + ): + star_cat = h5.File(self.star_path, "r")["star_catalog"] for pix in self.pix_list: try: stars = star_cat[str(pix)] @@ -477,20 +604,26 @@ class Catalog(CatalogBase): self.logger.error(str(e)) # print(e) - if "galaxy_cat" in self.config["catalog_options"]["input_path"] and self.config["catalog_options"]["input_path"]["galaxy_cat"] and not self.config["catalog_options"]["star_only"]: + if ( + "galaxy_cat" in self.config["catalog_options"]["input_path"] + and self.config["catalog_options"]["input_path"]["galaxy_cat"] + and not self.config["catalog_options"]["star_only"] + ): for pix in self.pix_list: try: bundleID = get_bundleIndex(pix) bundle_file = "galaxies_C10_bundle{:06}.h5".format(bundleID) file_path = os.path.join(self.galaxy_path, bundle_file) - gals_cat = h5.File(file_path, 'r')['galaxies'] + gals_cat = h5.File(file_path, "r")["galaxies"] gals = gals_cat[str(pix)] # Get corresponding AGN SED file agnsed_file = get_agnsed_file(bundle_file, self.config) agnsed_path = os.path.join(self.AGN_SED_path, agnsed_file) self.agn_seds[agnsed_file] = fits.open(agnsed_path)[0].data - self._load_gals(gals, pix_id=pix, cat_id=bundleID, agnsed_file=agnsed_file) + self._load_gals( + gals, pix_id=pix, cat_id=bundleID, agnsed_file=agnsed_file + ) del gals except Exception as e: traceback.print_exc() @@ -499,13 +632,12 @@ class Catalog(CatalogBase): if self.logger is not None: self.logger.info("maximum galaxy size: %.4f" % (self.max_size)) - self.logger.info("number of objects in catalog: %d" % - (len(self.objs))) + self.logger.info("number of objects in catalog: %d" % (len(self.objs))) else: print("number of objects in catalog: ", len(self.objs)) def load_sed(self, obj, **kwargs): - if obj.type == 'star': + if obj.type == "star": # _, wave, flux = tag_sed( # h5file=self.tempSED_star, # model_tag=obj.param['model_tag'], @@ -514,39 +646,41 @@ class Catalog(CatalogBase): # feh=obj.param['feh'] # ) wave, flux = self._interp_star_sed(obj) - elif obj.type == 'galaxy' or obj.type == 'quasar': - factor = 10**(-.4 * self.cosmo.distmod(obj.z).value) - if obj.type == 'galaxy': + elif obj.type == "galaxy" or obj.type == "quasar": + factor = 10 ** (-0.4 * self.cosmo.distmod(obj.z).value) + if obj.type == "galaxy": flux = np.matmul(self.pcs, obj.coeff) * factor # if np.any(flux < 0): # raise ValueError("Glaxy %s: negative SED fluxes"%obj.id) - flux[flux < 0] = 0. + flux[flux < 0] = 0.0 sedcat = np.vstack((self.lamb_gal, flux)).T sed_data = getObservedSED( sedCat=sedcat, redshift=obj.z, av=obj.param["av"], - redden=obj.param["redden"] + redden=obj.param["redden"], ) wave, flux = sed_data[0], sed_data[1] - elif obj.type == 'quasar': - flux = self.agn_seds[obj.agnsed_file][int( - obj.qsoindex)] * 1e-17 - flux[flux < 0] = 0. + elif obj.type == "quasar": + flux = self.agn_seds[obj.agnsed_file][int(obj.qsoindex)] * 1e-17 + flux[flux < 0] = 0.0 wave = self.lamb_gal * (1.0 + obj.z) else: raise ValueError("Object type not known") - speci = interpolate.interp1d(wave, flux, fill_value=0., bounds_error=False) - lamb = np.arange(2000, 11001+0.5, 0.5) + speci = interpolate.interp1d(wave, flux, fill_value=0.0, bounds_error=False) + lamb = np.arange(2000, 11001 + 0.5, 0.5) y = speci(lamb) # [TODO] Apply Milky Way extinction - if obj.type != 'star' and ("enable_mw_ext_gal" in self.config["catalog_options"] and self.config["catalog_options"]["enable_mw_ext_gal"]): + if obj.type != "star" and ( + "enable_mw_ext_gal" in self.config["catalog_options"] + and self.config["catalog_options"]["enable_mw_ext_gal"] + ): y = self.mw_ext.apply_extinction(y, Av=obj.mw_Av) # erg/s/cm2/A --> photon/s/m2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - sed = Table(np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX")) # if obj.type == 'quasar': # # integrate to get the magnitudes # sed_photon = np.array([sed['WAVELENGTH'], sed['FLUX']]).T @@ -562,16 +696,20 @@ class Catalog(CatalogBase): # # print("mag diff = %.3f"%(mag - obj.param['mag_use_normal'])) # integrate to get the magnitudes - if obj.type == 'quasar' or obj.type == 'galaxy': - sed_photon = np.array([sed['WAVELENGTH'], sed['FLUX']]).T - sed_photon = galsim.LookupTable(x=np.array(sed_photon[:, 0]), f=np.array( - sed_photon[:, 1]), interpolant='nearest') + if obj.type == "quasar" or obj.type == "galaxy": + sed_photon = np.array([sed["WAVELENGTH"], sed["FLUX"]]).T + sed_photon = galsim.LookupTable( + x=np.array(sed_photon[:, 0]), + f=np.array(sed_photon[:, 1]), + interpolant="nearest", + ) sed_photon = galsim.SED( - sed_photon, wave_type='A', flux_type='1', fast=False) + sed_photon, wave_type="A", flux_type="1", fast=False + ) interFlux = integrate_sed_bandpass( - sed=sed_photon, bandpass=self.filt.bandpass_full) - obj.param['mag_use_normal'] = getABMAG( - interFlux, self.filt.bandpass_full) + sed=sed_photon, bandpass=self.filt.bandpass_full + ) + obj.param["mag_use_normal"] = getABMAG(interFlux, self.filt.bandpass_full) # mag = getABMAG(interFlux, self.filt.bandpass_full) # print("mag diff = %.3f"%(mag - obj.param['mag_use_normal'])) del wave diff --git a/catalog/C9_Catalog.py b/catalog/C9_Catalog.py index 6f3a7f420bb24f8f3b46585db85e81ac268d13c9..1518a9a3a190f8ceed25b1fbfa5962946a5cb501 100644 --- a/catalog/C9_Catalog.py +++ b/catalog/C9_Catalog.py @@ -1,24 +1,25 @@ import os import galsim -import random import copy import numpy as np import h5py as h5 import healpy as hp import astropy.constants as cons import traceback -from astropy.coordinates import spherical_to_cartesian from astropy.table import Table from scipy import interpolate from datetime import datetime from observation_sim.mock_objects import CatalogBase, Star, Galaxy, Quasar, ExtinctionMW -from observation_sim.mock_objects._util import tag_sed, getObservedSED, getABMAG, integrate_sed_bandpass, comoving_dist +from observation_sim.mock_objects._util import ( + getObservedSED, + getABMAG, + integrate_sed_bandpass, +) from observation_sim.astrometry.Astrometry_util import on_orbit_obs_position # (TEST) from astropy.cosmology import FlatLambdaCDM -from astropy import constants from astropy import units as U from astropy.coordinates import SkyCoord from astropy.io import fits @@ -34,40 +35,113 @@ except ImportError: NSIDE = 128 -bundle_file_list = ['galaxies_C6_bundle000199.h5', 'galaxies_C6_bundle000200.h5', 'galaxies_C6_bundle000241.h5', 'galaxies_C6_bundle000242.h5', 'galaxies_C6_bundle000287.h5', 'galaxies_C6_bundle000288.h5', 'galaxies_C6_bundle000714.h5', 'galaxies_C6_bundle000715.h5', 'galaxies_C6_bundle000778.h5', 'galaxies_C6_bundle000779.h5', 'galaxies_C6_bundle000842.h5', 'galaxies_C6_bundle000843.h5', 'galaxies_C6_bundle002046.h5', 'galaxies_C6_bundle002110.h5', 'galaxies_C6_bundle002111.h5', - 'galaxies_C6_bundle002173.h5', 'galaxies_C6_bundle002174.h5', 'galaxies_C6_bundle002238.h5', 'galaxies_C6_bundle002596.h5', 'galaxies_C6_bundle002597.h5', 'galaxies_C6_bundle002656.h5', 'galaxies_C6_bundle002657.h5', 'galaxies_C6_bundle002711.h5', 'galaxies_C6_bundle002712.h5', 'galaxies_C6_bundle002844.h5', 'galaxies_C6_bundle002845.h5', 'galaxies_C6_bundle002884.h5', 'galaxies_C6_bundle002885.h5', 'galaxies_C6_bundle002921.h5', 'galaxies_C6_bundle002922.h5'] - -qsosed_file_list = ['quickspeclib_interp1d_run1.fits', 'quickspeclib_interp1d_run2.fits', 'quickspeclib_interp1d_run3.fits', 'quickspeclib_interp1d_run4.fits', 'quickspeclib_interp1d_run5.fits', 'quickspeclib_interp1d_run6.fits', 'quickspeclib_interp1d_run7.fits', 'quickspeclib_interp1d_run8.fits', 'quickspeclib_interp1d_run9.fits', 'quickspeclib_interp1d_run10.fits', 'quickspeclib_interp1d_run11.fits', 'quickspeclib_interp1d_run12.fits', 'quickspeclib_interp1d_run13.fits', 'quickspeclib_interp1d_run14.fits', 'quickspeclib_interp1d_run15.fits', - 'quickspeclib_interp1d_run16.fits', 'quickspeclib_interp1d_run17.fits', 'quickspeclib_interp1d_run18.fits', 'quickspeclib_interp1d_run19.fits', 'quickspeclib_interp1d_run20.fits', 'quickspeclib_interp1d_run21.fits', 'quickspeclib_interp1d_run22.fits', 'quickspeclib_interp1d_run23.fits', 'quickspeclib_interp1d_run24.fits', 'quickspeclib_interp1d_run25.fits', 'quickspeclib_interp1d_run26.fits', 'quickspeclib_interp1d_run27.fits', 'quickspeclib_interp1d_run28.fits', 'quickspeclib_interp1d_run29.fits', 'quickspeclib_interp1d_run30.fits'] +bundle_file_list = [ + "galaxies_C6_bundle000199.h5", + "galaxies_C6_bundle000200.h5", + "galaxies_C6_bundle000241.h5", + "galaxies_C6_bundle000242.h5", + "galaxies_C6_bundle000287.h5", + "galaxies_C6_bundle000288.h5", + "galaxies_C6_bundle000714.h5", + "galaxies_C6_bundle000715.h5", + "galaxies_C6_bundle000778.h5", + "galaxies_C6_bundle000779.h5", + "galaxies_C6_bundle000842.h5", + "galaxies_C6_bundle000843.h5", + "galaxies_C6_bundle002046.h5", + "galaxies_C6_bundle002110.h5", + "galaxies_C6_bundle002111.h5", + "galaxies_C6_bundle002173.h5", + "galaxies_C6_bundle002174.h5", + "galaxies_C6_bundle002238.h5", + "galaxies_C6_bundle002596.h5", + "galaxies_C6_bundle002597.h5", + "galaxies_C6_bundle002656.h5", + "galaxies_C6_bundle002657.h5", + "galaxies_C6_bundle002711.h5", + "galaxies_C6_bundle002712.h5", + "galaxies_C6_bundle002844.h5", + "galaxies_C6_bundle002845.h5", + "galaxies_C6_bundle002884.h5", + "galaxies_C6_bundle002885.h5", + "galaxies_C6_bundle002921.h5", + "galaxies_C6_bundle002922.h5", +] + +qsosed_file_list = [ + "quickspeclib_interp1d_run1.fits", + "quickspeclib_interp1d_run2.fits", + "quickspeclib_interp1d_run3.fits", + "quickspeclib_interp1d_run4.fits", + "quickspeclib_interp1d_run5.fits", + "quickspeclib_interp1d_run6.fits", + "quickspeclib_interp1d_run7.fits", + "quickspeclib_interp1d_run8.fits", + "quickspeclib_interp1d_run9.fits", + "quickspeclib_interp1d_run10.fits", + "quickspeclib_interp1d_run11.fits", + "quickspeclib_interp1d_run12.fits", + "quickspeclib_interp1d_run13.fits", + "quickspeclib_interp1d_run14.fits", + "quickspeclib_interp1d_run15.fits", + "quickspeclib_interp1d_run16.fits", + "quickspeclib_interp1d_run17.fits", + "quickspeclib_interp1d_run18.fits", + "quickspeclib_interp1d_run19.fits", + "quickspeclib_interp1d_run20.fits", + "quickspeclib_interp1d_run21.fits", + "quickspeclib_interp1d_run22.fits", + "quickspeclib_interp1d_run23.fits", + "quickspeclib_interp1d_run24.fits", + "quickspeclib_interp1d_run25.fits", + "quickspeclib_interp1d_run26.fits", + "quickspeclib_interp1d_run27.fits", + "quickspeclib_interp1d_run28.fits", + "quickspeclib_interp1d_run29.fits", + "quickspeclib_interp1d_run30.fits", +] # star_file_list = ['C7_Gaia_Galaxia_RA170DECm23_healpix.hdf5', 'C7_Gaia_Galaxia_RA180DECp60_healpix.hdf5', 'C7_Gaia_Galaxia_RA240DECp30_healpix.hdf5', 'C7_Gaia_Galaxia_RA300DECm60_healpix.hdf5', 'C7_Gaia_Galaxia_RA30DECm48_healpix.hdf5'] -star_center_list = [(170., -23.), (180., 60.), (240., 30.), - (300., -60.), (30., -48.), [246.5, 40]] - -star_file_list = ['C9_RA170_DECm23_calmag_Nside_128_healpix.hdf5', 'C9_RA180_DECp60_calmag_Nside_128_healpix.hdf5', 'C9_RA240_DECp30_calmag_Nside_128_healpix.hdf5', - 'C9_RA300_DECm60_calmag_Nside_128_healpix.hdf5', 'C9_RA30_DECm48_calmag_Nside_128_healpix.hdf5', 'trilegal_calMag_mpi_Nside_128_healpix.hdf5'] +star_center_list = [ + (170.0, -23.0), + (180.0, 60.0), + (240.0, 30.0), + (300.0, -60.0), + (30.0, -48.0), + [246.5, 40], +] + +star_file_list = [ + "C9_RA170_DECm23_calmag_Nside_128_healpix.hdf5", + "C9_RA180_DECp60_calmag_Nside_128_healpix.hdf5", + "C9_RA240_DECp30_calmag_Nside_128_healpix.hdf5", + "C9_RA300_DECm60_calmag_Nside_128_healpix.hdf5", + "C9_RA30_DECm48_calmag_Nside_128_healpix.hdf5", + "trilegal_calMag_mpi_Nside_128_healpix.hdf5", +] class StarParm(ctypes.Structure): _fields_ = [ - ('logte', ctypes.c_float), - ('logg', ctypes.c_float), - ('Mass', ctypes.c_float), - ('Av', ctypes.c_float), - ('mu0', ctypes.c_float), - ('Z', ctypes.c_float)] + ("logte", ctypes.c_float), + ("logg", ctypes.c_float), + ("Mass", ctypes.c_float), + ("Av", ctypes.c_float), + ("mu0", ctypes.c_float), + ("Z", ctypes.c_float), + ] def get_bundleIndex(healpixID_ring, bundleOrder=4, healpixOrder=7): assert NSIDE == 2**healpixOrder shift = healpixOrder - bundleOrder - shift = 2*shift + shift = 2 * shift nside_bundle = 2**bundleOrder nside_healpix = 2**healpixOrder healpixID_nest = hp.ring2nest(nside_healpix, healpixID_ring) - bundleID_nest = (healpixID_nest >> shift) + bundleID_nest = healpixID_nest >> shift bundleID_ring = hp.nest2ring(nside_bundle, bundleID_nest) return bundleID_ring @@ -78,11 +152,11 @@ def get_agnsed_file(bundle_file_name): def get_star_cat(ra_pointing, dec_pointing): - pointing_c = SkyCoord(ra=ra_pointing*U.deg, dec=dec_pointing*U.deg) + pointing_c = SkyCoord(ra=ra_pointing * U.deg, dec=dec_pointing * U.deg) max_dist = 10 return_star_path = None for star_file, center in zip(star_file_list, star_center_list): - center_c = SkyCoord(ra=center[0]*U.deg, dec=center[1]*U.deg) + center_c = SkyCoord(ra=center[0] * U.deg, dec=center[1] * U.deg) dist = pointing_c.separation(center_c).to(U.deg).value if dist < max_dist: return_star_path = star_file @@ -101,50 +175,73 @@ class Catalog(CatalogBase): self.filt = filt self.logger = chip_output.logger - with pkg_resources.path('catalog.data', 'SLOAN_SDSS.g.fits') as filter_path: + with pkg_resources.path("catalog.data", "SLOAN_SDSS.g.fits") as filter_path: self.normF_star = Table.read(str(filter_path)) self.config = config self.chip = chip self.pointing = pointing - self.max_size = 0. + self.max_size = 0.0 # [TODO] Milky Way extinction - if "enable_mw_ext_gal" in config["catalog_options"] and config["catalog_options"]["enable_mw_ext_gal"]: + if ( + "enable_mw_ext_gal" in config["catalog_options"] + and config["catalog_options"]["enable_mw_ext_gal"] + ): if "planck_ebv_map" not in config["catalog_options"]: raise ValueError( - "Planck dust map must be given to enable Milky Way extinction calculation for galaxies.") + "Planck dust map must be given to enable Milky Way extinction calculation for galaxies." + ) self.mw_ext = ExtinctionMW() self.mw_ext.init_ext_model(model_name="odonnell") self.mw_ext.load_Planck_ext( - file_path=config["catalog_options"]["planck_ebv_map"]) + file_path=config["catalog_options"]["planck_ebv_map"] + ) - if "star_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["star_cat"] and not config["catalog_options"]["galaxy_only"]: + if ( + "star_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["star_cat"] + and not config["catalog_options"]["galaxy_only"] + ): # Get the cloest star catalog file star_file_name = get_star_cat( - ra_pointing=self.pointing.ra, dec_pointing=self.pointing.dec) + ra_pointing=self.pointing.ra, dec_pointing=self.pointing.dec + ) star_path = os.path.join( - config["catalog_options"]["input_path"]["star_cat"], star_file_name) + config["catalog_options"]["input_path"]["star_cat"], star_file_name + ) self.star_path = os.path.join(self.cat_dir, star_path) - self.star_SED_path = config["catalog_options"]["SED_templates_path"]["star_SED"] + self.star_SED_path = config["catalog_options"]["SED_templates_path"][ + "star_SED" + ] self._load_SED_lib_star() - if "galaxy_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["galaxy_cat"] and not config["catalog_options"]["star_only"]: + if ( + "galaxy_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["galaxy_cat"] + and not config["catalog_options"]["star_only"] + ): galaxy_dir = config["catalog_options"]["input_path"]["galaxy_cat"] self.galaxy_path = os.path.join(self.cat_dir, galaxy_dir) - self.galaxy_SED_path = config["catalog_options"]["SED_templates_path"]["galaxy_SED"] + self.galaxy_SED_path = config["catalog_options"]["SED_templates_path"][ + "galaxy_SED" + ] self._load_SED_lib_gals() self.agn_seds = {} - if "AGN_SED" in config["catalog_options"]["SED_templates_path"] and not config["catalog_options"]["star_only"]: - self.AGN_SED_path = config["catalog_options"]["SED_templates_path"]["AGN_SED"] + if ( + "AGN_SED" in config["catalog_options"]["SED_templates_path"] + and not config["catalog_options"]["star_only"] + ): + self.AGN_SED_path = config["catalog_options"]["SED_templates_path"][ + "AGN_SED" + ] if "rotateEll" in config["catalog_options"]: - self.rotation = np.radians( - float(config["catalog_options"]["rotateEll"])) + self.rotation = np.radians(float(config["catalog_options"]["rotateEll"])) else: - self.rotation = 0. + self.rotation = 0.0 # Update output .cat header with catalog specific output columns self._add_output_columns_header() @@ -154,23 +251,30 @@ class Catalog(CatalogBase): def _add_output_columns_header(self): self.add_hdr = " av stellarmass dm teff logg feh" - self.add_hdr += " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + self.add_hdr += ( + " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + ) self.add_fmt = "%8.4f %8.4f %8.4f %8.4f %8.4f %8.4f" - self.add_fmt += " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " - self.chip_output.update_output_header( - additional_column_names=self.add_hdr) + self.add_fmt += ( + " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " + ) + self.chip_output.update_output_header(additional_column_names=self.add_hdr) def _get_healpix_list(self): self.sky_coverage = self.chip.getSkyCoverageEnlarged( - self.chip.img.wcs, margin=0.2) - ra_min, ra_max, dec_min, dec_max = self.sky_coverage.xmin, self.sky_coverage.xmax, self.sky_coverage.ymin, self.sky_coverage.ymax + self.chip.img.wcs, margin=0.2 + ) + ra_min, ra_max, dec_min, dec_max = ( + self.sky_coverage.xmin, + self.sky_coverage.xmax, + self.sky_coverage.ymin, + self.sky_coverage.ymax, + ) ra = np.deg2rad(np.array([ra_min, ra_max, ra_max, ra_min])) dec = np.deg2rad(np.array([dec_max, dec_max, dec_min, dec_min])) self.pix_list = hp.query_polygon( - NSIDE, - hp.ang2vec(np.radians(90.) - dec, ra), - inclusive=True + NSIDE, hp.ang2vec(np.radians(90.0) - dec, ra), inclusive=True ) if self.logger is not None: msg = str(("HEALPix List: ", self.pix_list)) @@ -191,44 +295,64 @@ class Catalog(CatalogBase): # self.tempSED_star = h5.File(self.star_SED_path,'r') def _load_SED_lib_star(self): # self.tempSED_star = h5.File(self.star_SED_path,'r') - with pkg_resources.path('catalog.data', 'starSpecInterp.so') as ddl_path: + with pkg_resources.path("catalog.data", "starSpecInterp.so") as ddl_path: self.starDDL = ctypes.CDLL(str(ddl_path)) self.starDDL.loadSpecLibs.argtypes = [ctypes.c_char_p, ctypes.c_char_p] self.starDDL.loadExts.argtypes = [ctypes.c_char_p] - nwv = self.starDDL.loadSpecLibs(str.encode(os.path.join( - self.star_SED_path, 'file_BT-Settl_CSST_wl1000-24000_R1000.par')), str.encode(self.star_SED_path)) - self.starDDL.loadExts(str.encode(os.path.join( - self.star_SED_path, "Ext_odonnell94_R3.1_CSST_wl1000-24000_R1000.fits"))) + nwv = self.starDDL.loadSpecLibs( + str.encode( + os.path.join( + self.star_SED_path, "file_BT-Settl_CSST_wl1000-24000_R1000.par" + ) + ), + str.encode(self.star_SED_path), + ) + self.starDDL.loadExts( + str.encode( + os.path.join( + self.star_SED_path, + "Ext_odonnell94_R3.1_CSST_wl1000-24000_R1000.fits", + ) + ) + ) self.star_spec_len = nwv def _interp_star_sed(self, obj): - spec = (ctypes.c_float*self.star_spec_len)() - wave = (ctypes.c_float*self.star_spec_len)() + spec = (ctypes.c_float * self.star_spec_len)() + wave = (ctypes.c_float * self.star_spec_len)() self.starDDL.interpSingleStar.argtypes = [ - ctypes.Structure, ctypes.POINTER(ctypes.c_float)] + ctypes.Structure, + ctypes.POINTER(ctypes.c_float), + ] # s=Star(obj.param['teff'], obj.param['grav''], obj.paramstar['mwmsc_mass'], obj.param['AV'], obj.param['DM'], obj.param['z_met']) - s = StarParm(obj.param['teff'], obj.param['logg'], obj.param['stellarMass'], - obj.param['av'], obj.param['DM'], obj.param['feh']) + s = StarParm( + obj.param["teff"], + obj.param["logg"], + obj.param["stellarMass"], + obj.param["av"], + obj.param["DM"], + obj.param["feh"], + ) self.starDDL.interpSingleStar(s, spec, wave) - rv_c = obj.param['rv']/(atcons.c.value/1000.) - Doppler_factor = np.sqrt((1+rv_c)/(1-rv_c)) - wave_RV = wave*Doppler_factor + rv_c = obj.param["rv"] / (atcons.c.value / 1000.0) + Doppler_factor = np.sqrt((1 + rv_c) / (1 - rv_c)) + wave_RV = wave * Doppler_factor return wave_RV, np.power(10, spec[:]) def _load_SED_lib_gals(self): pcs = h5.File(os.path.join(self.galaxy_SED_path, "pcs.h5"), "r") lamb = h5.File(os.path.join(self.galaxy_SED_path, "lamb.h5"), "r") - self.lamb_gal = lamb['lamb'][()] - self.pcs = pcs['pcs'][()] + self.lamb_gal = lamb["lamb"][()] + self.pcs = pcs["pcs"][()] def _load_gals(self, gals, pix_id=None, cat_id=0, agnsed_file=""): - ngals = len(gals['ra']) + ngals = len(gals["ra"]) # Apply astrometric modeling - ra_arr = gals['ra'][:] - dec_arr = gals['dec'][:] + ra_arr = gals["ra"][:] + dec_arr = gals["dec"][:] if self.config["obs_setting"]["enable_astrometric_model"]: ra_list = ra_arr.tolist() dec_list = dec_arr.tolist() @@ -255,11 +379,14 @@ class Catalog(CatalogBase): input_vz=self.pointing.sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) # [TODO] get Milky Way extinction AVs - if "enable_mw_ext_gal" in self.config["catalog_options"] and self.config["catalog_options"]["enable_mw_ext_gal"]: + if ( + "enable_mw_ext_gal" in self.config["catalog_options"] + and self.config["catalog_options"]["enable_mw_ext_gal"] + ): MW_Av_arr = self.mw_ext.Av_from_Planck(ra=ra_arr, dec=dec_arr) else: MW_Av_arr = np.zeros(len(ra_arr)) @@ -270,132 +397,174 @@ class Catalog(CatalogBase): # break param = self.initialize_param() - param['ra'] = ra_arr[igals] - param['dec'] = dec_arr[igals] - param['ra_orig'] = gals['ra'][igals] - param['dec_orig'] = gals['dec'][igals] + param["ra"] = ra_arr[igals] + param["dec"] = dec_arr[igals] + param["ra_orig"] = gals["ra"][igals] + param["dec_orig"] = gals["dec"][igals] # [TODO] get Milky Way extinction AVs - param['mw_Av'] = MW_Av_arr[igals] + param["mw_Av"] = MW_Av_arr[igals] - if not self.chip.isContainObj(ra_obj=param['ra'], dec_obj=param['dec'], margin=200): + if not self.chip.isContainObj( + ra_obj=param["ra"], dec_obj=param["dec"], margin=200 + ): continue # param['mag_use_normal'] = gals['mag_csst_%s'%(self.filt.filter_type)][igals] - if self.filt.filter_type == 'NUV': - param['mag_use_normal'] = gals['mag_csst_nuv'][igals] + if self.filt.filter_type == "NUV": + param["mag_use_normal"] = gals["mag_csst_nuv"][igals] else: - param['mag_use_normal'] = gals['mag_csst_%s' % - (self.filt.filter_type)][igals] - if self.filt.is_too_dim(mag=param['mag_use_normal'], margin=self.config["obs_setting"]["mag_lim_margin"]): + param["mag_use_normal"] = gals["mag_csst_%s" % (self.filt.filter_type)][ + igals + ] + if self.filt.is_too_dim( + mag=param["mag_use_normal"], + margin=self.config["obs_setting"]["mag_lim_margin"], + ): continue - param['z'] = gals['redshift'][igals] - param['model_tag'] = 'None' - param['g1'] = gals['shear'][igals][0] - param['g2'] = gals['shear'][igals][1] - param['kappa'] = gals['kappa'][igals] - param['e1'] = gals['ellipticity_true'][igals][0] - param['e2'] = gals['ellipticity_true'][igals][1] + param["z"] = gals["redshift"][igals] + param["model_tag"] = "None" + param["g1"] = gals["shear"][igals][0] + param["g2"] = gals["shear"][igals][1] + param["kappa"] = gals["kappa"][igals] + param["e1"] = gals["ellipticity_true"][igals][0] + param["e2"] = gals["ellipticity_true"][igals][1] # For shape calculation - param['e1'], param['e2'], param['ell_total'] = self.rotate_ellipticity( - e1=gals['ellipticity_true'][igals][0], - e2=gals['ellipticity_true'][igals][1], + param["e1"], param["e2"], param["ell_total"] = self.rotate_ellipticity( + e1=gals["ellipticity_true"][igals][0], + e2=gals["ellipticity_true"][igals][1], rotation=self.rotation, - unit='radians') + unit="radians", + ) # param['ell_total'] = np.sqrt(param['e1']**2 + param['e2']**2) - if param['ell_total'] > 0.9: + if param["ell_total"] > 0.9: continue # phi_e = cmath.phase(complex(param['e1'], param['e2'])) # param['e1'] = param['ell_total'] * np.cos(phi_e + 2*self.rotation) # param['e2'] = param['ell_total'] * np.sin(phi_e + 2*self.rotation) - param['e1_disk'] = param['e1'] - param['e2_disk'] = param['e2'] - param['e1_bulge'] = param['e1'] - param['e2_bulge'] = param['e2'] + param["e1_disk"] = param["e1"] + param["e2_disk"] = param["e2"] + param["e1_bulge"] = param["e1"] + param["e2_bulge"] = param["e2"] - param['delta_ra'] = 0 - param['delta_dec'] = 0 + param["delta_ra"] = 0 + param["delta_dec"] = 0 # Masses - param['bulgemass'] = gals['bulgemass'][igals] - param['diskmass'] = gals['diskmass'][igals] + param["bulgemass"] = gals["bulgemass"][igals] + param["diskmass"] = gals["diskmass"][igals] - param['size'] = gals['size'][igals] - if param['size'] > self.max_size: - self.max_size = param['size'] + np.random.seed(int(pix_id) + cat_id + igals) + param["size"] = ( + gals["size"][igals] * (1.0 + gals["redshift"][igals]) + ) * np.random.uniform(0.7, 1.3) + if param["size"] > self.max_size: + self.max_size = param["size"] # Sersic index - param['disk_sersic_idx'] = 1. - param['bulge_sersic_idx'] = 4. + param["disk_sersic_idx"] = 1.0 + param["bulge_sersic_idx"] = 4.0 # Sizes - param['bfrac'] = param['bulgemass'] / \ - (param['bulgemass'] + param['diskmass']) - if param['bfrac'] >= 0.6: - param['hlr_bulge'] = param['size'] - param['hlr_disk'] = param['size'] * (1. - param['bfrac']) + param["bfrac"] = param["bulgemass"] / ( + param["bulgemass"] + param["diskmass"] + ) + if param["bfrac"] >= 0.6: + param["hlr_bulge"] = param["size"] + param["hlr_disk"] = param["size"] * (1.0 - param["bfrac"]) else: - param['hlr_disk'] = param['size'] - param['hlr_bulge'] = param['size'] * param['bfrac'] + param["hlr_disk"] = param["size"] + param["hlr_bulge"] = param["size"] * param["bfrac"] # SED coefficients - param['coeff'] = gals['coeff'][igals] - param['detA'] = gals['detA'][igals] + param["coeff"] = gals["coeff"][igals] + param["detA"] = gals["detA"][igals] # Others - param['galType'] = gals['type'][igals] - param['veldisp'] = gals['veldisp'][igals] + param["galType"] = gals["type"][igals] + param["veldisp"] = gals["veldisp"][igals] # TEST no redening and no extinction - param['av'] = 0.0 - param['redden'] = 0 + param["av"] = 0.0 + param["redden"] = 0 # TEMP self.ids += 1 - param['id'] = '%06d' % (int(pix_id)) + \ - '%06d' % (cat_id) + '%08d' % (igals) + param["id"] = "%06d" % (int(pix_id)) + "%06d" % (cat_id) + "%08d" % (igals) # Is this an Quasar? - param['qsoindex'] = gals['qsoindex'][igals] - if param['qsoindex'] == -1: - param['star'] = 0 # Galaxy - param['agnsed_file'] = "" + param["qsoindex"] = gals["qsoindex"][igals] + if param["qsoindex"] == -1: + param["star"] = 0 # Galaxy + param["agnsed_file"] = "" obj = Galaxy(param, logger=self.logger) else: param_qso = copy.deepcopy(param) - param_qso['star'] = 2 # Quasar - param_qso['agnsed_file'] = agnsed_file + param_qso["star"] = 2 # Quasar + param_qso["agnsed_file"] = agnsed_file # First add QSO model obj = Quasar(param_qso, logger=self.logger) # Need to deal with additional output columns - obj.additional_output_str = self.add_fmt % (0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., - 0, 0.) + obj.additional_output_str = self.add_fmt % ( + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0, + 0.0, + ) self.objs.append(obj) # Then add host galaxy model - param['star'] = 0 # Galaxy - param['agnsed_file'] = "" + param["star"] = 0 # Galaxy + param["agnsed_file"] = "" obj = Galaxy(param, logger=self.logger) # Need to deal with additional output columns for (host) galaxy - obj.additional_output_str = self.add_fmt % (param['mw_Av'], 0., 0., 0., 0., 0., - param['bulgemass'], param['diskmass'], param['detA'], - param['e1'], param['e2'], param['kappa'], param['g1'], param['g2'], param['size'], - param['galType'], param['veldisp']) + obj.additional_output_str = self.add_fmt % ( + param["mw_Av"], + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + param["bulgemass"], + param["diskmass"], + param["detA"], + param["e1"], + param["e2"], + param["kappa"], + param["g1"], + param["g2"], + param["size"], + param["galType"], + param["veldisp"], + ) self.objs.append(obj) def _load_stars(self, stars, pix_id=None): - nstars = len(stars['RA']) + nstars = len(stars["RA"]) # Apply astrometric modeling ra_arr = stars["RA"][:] dec_arr = stars["DEC"][:] - pmra_arr = stars['pmra'][:] - pmdec_arr = stars['pmdec'][:] - rv_arr = stars['RV'][:] - parallax_arr = stars['parallax'][:] + pmra_arr = stars["pmra"][:] + pmdec_arr = stars["pmdec"][:] + rv_arr = stars["RV"][:] + parallax_arr = stars["parallax"][:] if self.config["obs_setting"]["enable_astrometric_model"]: ra_list = ra_arr.tolist() dec_list = dec_arr.tolist() @@ -422,7 +591,7 @@ class Catalog(CatalogBase): input_vz=self.pointing.sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) for istars in range(nstars): # # (TEST) @@ -430,33 +599,35 @@ class Catalog(CatalogBase): # break param = self.initialize_param() - param['ra'] = ra_arr[istars] - param['dec'] = dec_arr[istars] - param['ra_orig'] = stars["RA"][istars] - param['dec_orig'] = stars["DEC"][istars] - param['pmra'] = pmra_arr[istars] - param['pmdec'] = pmdec_arr[istars] - param['rv'] = rv_arr[istars] - param['parallax'] = parallax_arr[istars] - if not self.chip.isContainObj(ra_obj=param['ra'], dec_obj=param['dec'], margin=200): + param["ra"] = ra_arr[istars] + param["dec"] = dec_arr[istars] + param["ra_orig"] = stars["RA"][istars] + param["dec_orig"] = stars["DEC"][istars] + param["pmra"] = pmra_arr[istars] + param["pmdec"] = pmdec_arr[istars] + param["rv"] = rv_arr[istars] + param["parallax"] = parallax_arr[istars] + if not self.chip.isContainObj( + ra_obj=param["ra"], dec_obj=param["dec"], margin=200 + ): continue - param['mag_use_normal'] = stars['app_sdss_g'][istars] + param["mag_use_normal"] = stars["app_sdss_g"][istars] self.ids += 1 - param['id'] = '%06d' % (int(pix_id)) + '%08d' % (istars) + param["id"] = "%06d" % (int(pix_id)) + "%08d" % (istars) # param['sed_type'] = istars # param['model_tag'] = '' - param['teff'] = stars['teff'][istars] - param['logg'] = stars['grav'][istars] - param['feh'] = stars['z_met'][istars] - param['stellarMass'] = stars['mass'][istars] + param["teff"] = stars["teff"][istars] + param["logg"] = stars["grav"][istars] + param["feh"] = stars["z_met"][istars] + param["stellarMass"] = stars["mass"][istars] - param['av'] = stars['AV'][istars] - param['DM'] = stars['DM'][istars] + param["av"] = stars["AV"][istars] + param["DM"] = stars["DM"][istars] # param['z_met'] = stars['z_met'][istars] - param['z'] = 0.0 - param['star'] = 1 # Star + param["z"] = 0.0 + param["star"] = 1 # Star try: obj = Star(param, logger=self.logger) @@ -464,8 +635,25 @@ class Catalog(CatalogBase): print(e) # Append additional output columns to the .cat file - obj.additional_output_str = self.add_fmt % (param["av"], param['stellarMass'], param['DM'], param['teff'], param['logg'], param['feh'], - 0., 0., 0., 0., 0., 0., 0., 0., 0., -1, 0.) + obj.additional_output_str = self.add_fmt % ( + param["av"], + param["stellarMass"], + param["DM"], + param["teff"], + param["logg"], + param["feh"], + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + -1, + 0.0, + ) self.objs.append(obj) @@ -478,8 +666,12 @@ class Catalog(CatalogBase): self.objs = [] self.ids = 0 - if "star_cat" in self.config["catalog_options"]["input_path"] and self.config["catalog_options"]["input_path"]["star_cat"] and not self.config["catalog_options"]["galaxy_only"]: - star_cat = h5.File(self.star_path, 'r')['star_catalog'] + if ( + "star_cat" in self.config["catalog_options"]["input_path"] + and self.config["catalog_options"]["input_path"]["star_cat"] + and not self.config["catalog_options"]["galaxy_only"] + ): + star_cat = h5.File(self.star_path, "r")["star_catalog"] for pix in self.pix_list: try: stars = star_cat[str(pix)] @@ -489,13 +681,17 @@ class Catalog(CatalogBase): self.logger.error(str(e)) # print(e) - if "galaxy_cat" in self.config["catalog_options"]["input_path"] and self.config["catalog_options"]["input_path"]["galaxy_cat"] and not self.config["catalog_options"]["star_only"]: + if ( + "galaxy_cat" in self.config["catalog_options"]["input_path"] + and self.config["catalog_options"]["input_path"]["galaxy_cat"] + and not self.config["catalog_options"]["star_only"] + ): for pix in self.pix_list: try: bundleID = get_bundleIndex(pix) bundle_file = "galaxies_C6_bundle{:06}.h5".format(bundleID) file_path = os.path.join(self.galaxy_path, bundle_file) - gals_cat = h5.File(file_path, 'r')['galaxies'] + gals_cat = h5.File(file_path, "r")["galaxies"] gals = gals_cat[str(pix)] # Get corresponding AGN SED file @@ -503,8 +699,9 @@ class Catalog(CatalogBase): agnsed_path = os.path.join(self.AGN_SED_path, agnsed_file) self.agn_seds[agnsed_file] = fits.open(agnsed_path)[0].data - self._load_gals(gals, pix_id=pix, - cat_id=bundleID, agnsed_file=agnsed_file) + self._load_gals( + gals, pix_id=pix, cat_id=bundleID, agnsed_file=agnsed_file + ) del gals except Exception as e: @@ -514,13 +711,12 @@ class Catalog(CatalogBase): if self.logger is not None: self.logger.info("maximum galaxy size: %.4f" % (self.max_size)) - self.logger.info("number of objects in catalog: %d" % - (len(self.objs))) + self.logger.info("number of objects in catalog: %d" % (len(self.objs))) else: print("number of objects in catalog: ", len(self.objs)) def load_sed(self, obj, **kwargs): - if obj.type == 'star': + if obj.type == "star": # _, wave, flux = tag_sed( # h5file=self.tempSED_star, # model_tag=obj.param['model_tag'], @@ -529,39 +725,41 @@ class Catalog(CatalogBase): # feh=obj.param['feh'] # ) wave, flux = self._interp_star_sed(obj) - elif obj.type == 'galaxy' or obj.type == 'quasar': - factor = 10**(-.4 * self.cosmo.distmod(obj.z).value) - if obj.type == 'galaxy': + elif obj.type == "galaxy" or obj.type == "quasar": + factor = 10 ** (-0.4 * self.cosmo.distmod(obj.z).value) + if obj.type == "galaxy": flux = np.matmul(self.pcs, obj.coeff) * factor # if np.any(flux < 0): # raise ValueError("Glaxy %s: negative SED fluxes"%obj.id) - flux[flux < 0] = 0. + flux[flux < 0] = 0.0 sedcat = np.vstack((self.lamb_gal, flux)).T sed_data = getObservedSED( sedCat=sedcat, redshift=obj.z, av=obj.param["av"], - redden=obj.param["redden"] + redden=obj.param["redden"], ) wave, flux = sed_data[0], sed_data[1] - elif obj.type == 'quasar': - flux = self.agn_seds[obj.agnsed_file][int( - obj.qsoindex)] * 1e-17 - flux[flux < 0] = 0. + elif obj.type == "quasar": + flux = self.agn_seds[obj.agnsed_file][int(obj.qsoindex)] * 1e-17 + flux[flux < 0] = 0.0 wave = self.lamb_gal * (1.0 + obj.z) else: raise ValueError("Object type not known") speci = interpolate.interp1d(wave, flux) - lamb = np.arange(2000, 11001+0.5, 0.5) + lamb = np.arange(2000, 11001 + 0.5, 0.5) y = speci(lamb) # [TODO] Apply Milky Way extinction - if obj.type != 'star' and ("enable_mw_ext_gal" in self.config["catalog_options"] and self.config["catalog_options"]["enable_mw_ext_gal"]): + if obj.type != "star" and ( + "enable_mw_ext_gal" in self.config["catalog_options"] + and self.config["catalog_options"]["enable_mw_ext_gal"] + ): y = self.mw_ext.apply_extinction(y, Av=obj.mw_Av) # erg/s/cm2/A --> photon/s/m2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - sed = Table(np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX")) # if obj.type == 'quasar': # # integrate to get the magnitudes # sed_photon = np.array([sed['WAVELENGTH'], sed['FLUX']]).T @@ -577,16 +775,20 @@ class Catalog(CatalogBase): # # print("mag diff = %.3f"%(mag - obj.param['mag_use_normal'])) # integrate to get the magnitudes - if obj.type == 'quasar' or obj.type == 'galaxy': - sed_photon = np.array([sed['WAVELENGTH'], sed['FLUX']]).T - sed_photon = galsim.LookupTable(x=np.array(sed_photon[:, 0]), f=np.array( - sed_photon[:, 1]), interpolant='nearest') + if obj.type == "quasar" or obj.type == "galaxy": + sed_photon = np.array([sed["WAVELENGTH"], sed["FLUX"]]).T + sed_photon = galsim.LookupTable( + x=np.array(sed_photon[:, 0]), + f=np.array(sed_photon[:, 1]), + interpolant="nearest", + ) sed_photon = galsim.SED( - sed_photon, wave_type='A', flux_type='1', fast=False) + sed_photon, wave_type="A", flux_type="1", fast=False + ) interFlux = integrate_sed_bandpass( - sed=sed_photon, bandpass=self.filt.bandpass_full) - obj.param['mag_use_normal'] = getABMAG( - interFlux, self.filt.bandpass_full) + sed=sed_photon, bandpass=self.filt.bandpass_full + ) + obj.param["mag_use_normal"] = getABMAG(interFlux, self.filt.bandpass_full) # mag = getABMAG(interFlux, self.filt.bandpass_full) # print("mag diff = %.3f"%(mag - obj.param['mag_use_normal'])) del wave diff --git a/catalog/Catalog_example.py b/catalog/Catalog_example.py index d3a6f28c1e050bf95fb4c4c1e9d8cf36671c5179..760da8eeaea357f3cca55966feb65f9bb1baee83 100644 --- a/catalog/Catalog_example.py +++ b/catalog/Catalog_example.py @@ -4,7 +4,7 @@ import astropy.constants as cons from astropy.table import Table from scipy import interpolate -from observation_sim.mock_objects import CatalogBase, Star, Galaxy, Quasar +from observation_sim.mock_objects import CatalogBase, Star class Catalog(CatalogBase): @@ -54,14 +54,17 @@ class Catalog(CatalogBase): super().__init__() self.cat_dir = os.path.join( - config["data_dir"], config["catalog_options"]["input_path"]["cat_dir"]) + config["data_dir"], config["catalog_options"]["input_path"]["cat_dir"] + ) self.chip = chip - if "star_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["star_cat"]: + if ( + "star_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["star_cat"] + ): star_file = config["catalog_options"]["input_path"]["star_cat"] star_SED_file = config["catalog_options"]["SED_templates_path"]["star_SED"] self.star_path = os.path.join(self.cat_dir, star_file) - self.star_SED_path = os.path.join( - config["data_dir"], star_SED_file) + self.star_SED_path = os.path.join(config["data_dir"], star_SED_file) # NOTE: must call _load() method here to read in all objects self.objs = [] self._load() @@ -142,17 +145,17 @@ class Catalog(CatalogBase): """ stars = Table.read(self.star_path) - nstars = stars['sourceID'].size + nstars = stars["sourceID"].size for istars in range(nstars): param = self.initialize_param() - param['id'] = istars + 1 - param['ra'] = stars['RA'][istars] - param['dec'] = stars['Dec'][istars] - param['sed_type'] = stars['sourceID'][istars] - param['model_tag'] = stars['model_tag'][istars] - param['z'] = 0.0 - param['star'] = 1 # Star - param['mag_use_normal'] = stars['app_sdss_g'][istars] + param["id"] = istars + 1 + param["ra"] = stars["RA"][istars] + param["dec"] = stars["Dec"][istars] + param["sed_type"] = stars["sourceID"][istars] + param["model_tag"] = stars["model_tag"][istars] + param["z"] = 0.0 + param["star"] = 1 # Star + param["mag_use_normal"] = stars["app_sdss_g"][istars] obj = Star(param) self.objs.append(obj) @@ -176,11 +179,10 @@ class Catalog(CatalogBase): NOTE: the range of wavelengthes must at least cover [2450 - 11000] Angstorms """ - if obj.type == 'star': - wave = Table.read(self.star_SED_path, - path=f"/SED/wave_{obj.model_tag}") + if obj.type == "star": + wave = Table.read(self.star_SED_path, path=f"/SED/wave_{obj.model_tag}") flux = Table.read(self.star_SED_path, path=f"/SED/{obj.sed_type}") - wave, flux = wave['col0'].data, flux['col0'].data + wave, flux = wave["col0"].data, flux["col0"].data else: raise ValueError("Object type not known") speci = interpolate.interp1d(wave, flux) @@ -188,7 +190,7 @@ class Catalog(CatalogBase): y = speci(lamb) # erg/s/cm^2/A --> photons/s/m^2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - sed = Table(np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX")) return sed def load_norm_filt(self, obj): diff --git a/catalog/__init__.py b/catalog/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/catalog/testCat_fits.py b/catalog/testCat_fits.py index 9717b22ab0315d594491a9ca54ac39eaf2686dbf..cc84852a1ab27484e8d26d8b772e5faa3103ce9c 100644 --- a/catalog/testCat_fits.py +++ b/catalog/testCat_fits.py @@ -5,23 +5,21 @@ import numpy as np import h5py as h5 import healpy as hp import astropy.constants as cons -import traceback -from astropy.coordinates import spherical_to_cartesian from astropy.table import Table from scipy import interpolate from datetime import datetime -from observation_sim.mock_objects import CatalogBase, Star, Galaxy, Quasar, Stamp -from observation_sim.mock_objects._util import tag_sed, getObservedSED, getABMAG, integrate_sed_bandpass, comoving_dist +from observation_sim.mock_objects import CatalogBase, Stamp +from observation_sim.mock_objects._util import ( + getObservedSED, +) from observation_sim.astrometry.Astrometry_util import on_orbit_obs_position import astropy.io.fits as fitsio -from observation_sim.mock_objects._util import seds, sed_assign, extAv +from observation_sim.mock_objects._util import seds, sed_assign # (TEST) from astropy.cosmology import FlatLambdaCDM -from astropy import constants -from astropy import units as U try: import importlib.resources as pkg_resources @@ -45,24 +43,30 @@ class Catalog(CatalogBase): self.filt = filt self.logger = chip_output.logger - with pkg_resources.path('catalog.data', 'SLOAN_SDSS.g.fits') as filter_path: + with pkg_resources.path("catalog.data", "SLOAN_SDSS.g.fits") as filter_path: self.normF_star = Table.read(str(filter_path)) - with pkg_resources.path('catalog.data', 'lsst_throuput_g.fits') as filter_path: + with pkg_resources.path("catalog.data", "lsst_throuput_g.fits") as filter_path: self.normF_galaxy = Table.read(str(filter_path)) self.config = config self.chip = chip self.pointing = pointing - self.max_size = 0. + self.max_size = 0.0 - if "stamp_cat" in config["catalog_options"]["input_path"] and config["catalog_options"]["input_path"]["stamp_cat"] and config["catalog_options"]["stamp_yes"]: + if ( + "stamp_cat" in config["catalog_options"]["input_path"] + and config["catalog_options"]["input_path"]["stamp_cat"] + and config["catalog_options"]["stamp_yes"] + ): stamp_file = config["catalog_options"]["input_path"]["stamp_cat"] self.stamp_path = os.path.join(self.cat_dir, stamp_file) # self.stamp_SED_path = os.path.join(config["data_dir"], config["SED_templates_path"]["stamp_SED"]) ###shoule be stamp-SED # self._load_SED_lib_stamps() ###shoule be stamp-SED self.tempSed_gal, self.tempRed_gal = seds( - "galaxy.list", seddir="/public/home/chengliang/CSSOSDataProductsSims/testCats/Templates/Galaxy/") # only for test + "galaxy.list", + seddir="/public/home/chengliang/CSSOSDataProductsSims/testCats/Templates/Galaxy/", + ) # only for test self._add_output_columns_header() self._get_healpix_list() @@ -70,24 +74,31 @@ class Catalog(CatalogBase): def _add_output_columns_header(self): self.add_hdr = " model_tag teff logg feh" - self.add_hdr += " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + self.add_hdr += ( + " bulgemass diskmass detA e1 e2 kappa g1 g2 size galType veldisp " + ) self.add_fmt = " %10s %8.4f %8.4f %8.4f" - self.add_fmt += " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " - self.chip_output.update_output_header( - additional_column_names=self.add_hdr) + self.add_fmt += ( + " %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %8.4f %4d %8.4f " + ) + self.chip_output.update_output_header(additional_column_names=self.add_hdr) def _get_healpix_list(self): self.sky_coverage = self.chip.getSkyCoverageEnlarged( - self.chip.img.wcs, margin=0.2) - ra_min, ra_max, dec_min, dec_max = self.sky_coverage.xmin, self.sky_coverage.xmax, self.sky_coverage.ymin, self.sky_coverage.ymax + self.chip.img.wcs, margin=0.2 + ) + ra_min, ra_max, dec_min, dec_max = ( + self.sky_coverage.xmin, + self.sky_coverage.xmax, + self.sky_coverage.ymin, + self.sky_coverage.ymax, + ) ra = np.deg2rad(np.array([ra_min, ra_max, ra_max, ra_min])) dec = np.deg2rad(np.array([dec_max, dec_max, dec_min, dec_min])) # vertices = spherical_to_cartesian(1., dec, ra) self.pix_list = hp.query_polygon( - NSIDE, - hp.ang2vec(np.radians(90.) - dec, ra), - inclusive=True + NSIDE, hp.ang2vec(np.radians(90.0) - dec, ra), inclusive=True ) # self.pix_list = hp.query_polygon(NSIDE, np.array(vertices).T, inclusive=True) if self.logger is not None: @@ -104,7 +115,7 @@ class Catalog(CatalogBase): def _load_stamps(self, stamps, pix_id=None): print("debug:: load_stamps") - nstamps = len(stamps['filename']) + nstamps = len(stamps["filename"]) self.rng_sedGal = random.Random() # Use healpix index as the random seed self.rng_sedGal.seed(float(pix_id)) @@ -113,31 +124,32 @@ class Catalog(CatalogBase): for istamp in range(nstamps): print("debug::", istamp) fitsfile = os.path.join( - self.cat_dir, "stampCats/"+stamps['filename'][istamp].decode('utf-8')) + self.cat_dir, "stampCats/" + stamps["filename"][istamp].decode("utf-8") + ) print("debug::", istamp, fitsfile) hdu = fitsio.open(fitsfile) param = self.initialize_param() - param['id'] = hdu[0].header['index'] # istamp - param['star'] = 3 # Stamp type in .cat file - param['ra'] = hdu[0].header['ra'] - param['dec'] = hdu[0].header['dec'] - param['pixScale'] = hdu[0].header['pixScale'] + param["id"] = hdu[0].header["index"] # istamp + param["star"] = 3 # Stamp type in .cat file + param["ra"] = hdu[0].header["ra"] + param["dec"] = hdu[0].header["dec"] + param["pixScale"] = hdu[0].header["pixScale"] # param['srcGalaxyID'] = hdu[0].header['srcGID'] # param['mu']= hdu[0].header['mu'] # param['PA']= hdu[0].header['PA'] # param['bfrac']= hdu[0].header['bfrac'] # param['z']= hdu[0].header['z'] # gals['mag_true_g_lsst'] - param['mag_use_normal'] = hdu[0].header['mag_g'] + param["mag_use_normal"] = hdu[0].header["mag_g"] # Apply astrometric modeling # in C3 case only aberration - param['ra_orig'] = param['ra'] - param['dec_orig'] = param['dec'] + param["ra_orig"] = param["ra"] + param["dec_orig"] = param["dec"] if self.config["obs_setting"]["enable_astrometric_model"]: - ra_list = [param['ra']] # ra_arr.tolist() - dec_list = [param['dec']] # dec_arr.tolist() + ra_list = [param["ra"]] # ra_arr.tolist() + dec_list = [param["dec"]] # dec_arr.tolist() pmra_list = np.zeros(1).tolist() pmdec_list = np.zeros(1).tolist() rv_list = np.zeros(1).tolist() @@ -161,25 +173,26 @@ class Catalog(CatalogBase): input_vz=self.pointing.sat_vz, input_epoch="J2015.5", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) - param['ra'] = ra_arr[0] - param['dec'] = dec_arr[0] + param["ra"] = ra_arr[0] + param["dec"] = dec_arr[0] # Assign each galaxy a template SED - param['sed_type'] = sed_assign( - phz=param['z'], btt=param['bfrac'], rng=self.rng_sedGal) - param['redden'] = self.tempRed_gal[param['sed_type']] - param['av'] = 0.0 - param['redden'] = 0 - param['mu'] = 1 + param["sed_type"] = sed_assign( + phz=param["z"], btt=param["bfrac"], rng=self.rng_sedGal + ) + param["redden"] = self.tempRed_gal[param["sed_type"]] + param["av"] = 0.0 + param["redden"] = 0 + param["mu"] = 1 # param["CSSTmag"]= True # param["mag_r"] = 20. # param[''] # more keywords for stamp# - param['image'] = hdu[0].data - param['image'] = param['image']/(np.sum(param['image'])) + param["image"] = hdu[0].data + param["image"] = param["image"] / (np.sum(param["image"])) obj = Stamp(param) self.objs.append(obj) @@ -187,8 +200,12 @@ class Catalog(CatalogBase): self.objs = [] self.ids = 0 - if "stamp_cat" in self.config["catalog_options"]["input_path"] and self.config["catalog_options"]["input_path"]["stamp_cat"] and self.config["catalog_options"]["stamp_yes"]: - stamps_cat = h5.File(self.stamp_path, 'r')['Stamps'] + if ( + "stamp_cat" in self.config["catalog_options"]["input_path"] + and self.config["catalog_options"]["input_path"]["stamp_cat"] + and self.config["catalog_options"]["stamp_yes"] + ): + stamps_cat = h5.File(self.stamp_path, "r")["Stamps"] print("debug::", stamps_cat.keys()) for pix in self.pix_list: @@ -203,28 +220,27 @@ class Catalog(CatalogBase): if self.logger is not None: self.logger.info("maximum galaxy size: %.4f" % (self.max_size)) - self.logger.info("number of objects in catalog: %d" % - (len(self.objs))) + self.logger.info("number of objects in catalog: %d" % (len(self.objs))) else: print("number of objects in catalog: ", len(self.objs)) def load_sed(self, obj, **kwargs): - if obj.type == 'stamp': + if obj.type == "stamp": sed_data = getObservedSED( sedCat=self.tempSed_gal[obj.sed_type], redshift=obj.z, av=obj.param["av"], - redden=obj.param["redden"] + redden=obj.param["redden"], ) wave, flux = sed_data[0], sed_data[1] else: raise ValueError("Object type not known") speci = interpolate.interp1d(wave, flux) - lamb = np.arange(2000, 11001+0.5, 0.5) + lamb = np.arange(2000, 11001 + 0.5, 0.5) y = speci(lamb) # erg/s/cm2/A --> photon/s/m2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - sed = Table(np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX")) del wave del flux diff --git a/config/config_overall.yaml b/config/config_overall.yaml index 867fcf18813e4677b29a28da2fa4d97635bbda6f..111b584e06795b14f5dbae2abf5d7c808ae3c994 100644 --- a/config/config_overall.yaml +++ b/config/config_overall.yaml @@ -10,12 +10,13 @@ # Base diretories and naming setup # can add some of the command-line arguments here as well; # ok to pass either way or both, as long as they are consistent -work_dir: "/public/home/fangyuedong/project/workplace/" -run_name: "ext_on" +# work_dir: "/public/home/fangyuedong/project/workplace/" +work_dir: "/home/yuedong/Work/CSST_sim/output/" +run_name: "test_ghost" data_set: "csst-msc-c11-1000sqdeg-wide-v1" # Project cycle and run counter are used to name the outputs -project_cycle: 11 +project_cycle: 9 run_counter: 1 # Run options @@ -32,14 +33,17 @@ run_option: # in the corresponding (user defined) 'Catalog' class catalog_options: input_path: - cat_dir: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/" + # cat_dir: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/" + # star_cat: "starcat_C9/" + # galaxy_cat: "qsocat/cat2CSSTSim_bundle-50sqDeg/" + cat_dir: "/home/yuedong/Work/CSST_sim/input_data/" star_cat: "starcat_C9/" - galaxy_cat: "qsocat/cat2CSSTSim_bundle-50sqDeg/" + galaxy_cat: "galaxycat_C9/cat2CSSTSim_bundle-50sqDeg" SED_templates_path: - star_SED: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/starcat_C9/" - galaxy_SED: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/sedlibs/" - AGN_SED: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/qsocat/qsosed/" + star_SED: "/home/yuedong/Work/CSST_sim/input_data/starcat_C9/" + galaxy_SED: "/home/yuedong/Work/CSST_sim/input_data/galaxycat_C9/cat2CSSTSim_bundle-50sqDeg/galaxySED/" + AGN_SED: "/home/yuedong/Work/CSST_sim/input_data/galaxycat_C9/qsosed/" # Only simulate stars? star_only: NO @@ -51,8 +55,8 @@ catalog_options: rotateEll: 0. # [degree] # Whether to apply milky way extinction to galaxies - enable_mw_ext_gal: YES - planck_ebv_map: "/public/home/fangyuedong/project/ext_maps/planck/HFI_CompMap_ThermalDustModel_2048_R1.20.fits" + enable_mw_ext_gal: NO + # planck_ebv_map: "/public/home/fangyuedong/project/ext_maps/planck/HFI_CompMap_ThermalDustModel_2048_R1.20.fits" ############################################### # Observation setting @@ -62,15 +66,15 @@ obs_setting: # if you just want to run default pointing: # - pointing_dir: null # - pointing_file: null - pointing_file: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/pointing50_C9/pointing_50_1_n.dat" + pointing_file: "/home/yuedong/Work/CSST_sim/input_data/pointing50_C9/pointing_50_1_n.dat" - obs_config_file: "/public/home/fangyuedong/project/csst_msc_sim/config/obs_config_SCI.yaml" + obs_config_file: "/home/yuedong/Work/csst_test/csst_msc_sim/config/obs_config_SCI.yaml" # Run specific pointing(s): # - give a list of indexes of pointings: [ip_1, ip_2...] # - run all pointings: null # Note: only valid when a pointing list is specified - run_pointings: [0, 1, 2, 3, 4] + run_pointings: [0] # Whether to enable astrometric modeling enable_astrometric_model: YES @@ -79,8 +83,8 @@ obs_setting: cut_in_band: "z" # saturation magnitude margin - mag_sat_margin: -2.5 - # mag_sat_margin: -15. + # mag_sat_margin: -2.5 + mag_sat_margin: -15. # limiting magnitude margin mag_lim_margin: +1.0 @@ -103,9 +107,9 @@ psf_setting: # path to PSF data # NOTE: only valid for "Interp" PSF # PSF models for photometry survey simulation - psf_pho_dir: "/public/share/yangxuliu/CSSOSDataProductsSims/dataC6/psfCube1" + psf_pho_dir: "/home/yuedong/Work/CSST_sim/input_data/PSFCube/set1_dynamic/" # PSF models for slitless spectrum survey simulation - psf_sls_dir: "/public/share/yangxuliu/CSSOSDataProductsSims/data_50sqDeg/SLS_PSF_PCA_fp/" + psf_sls_dir: "/home/yuedong/Work/CSST_sim/input_data/PSFCube/SLS_PSF_PCA_fp/" ############################################### # Shear setting @@ -125,7 +129,7 @@ shear_setting: # Output options ############################################### output_setting: - output_format: "channels" # Whether to export as 16 channels (subimages) with pre- and over-scan ("image"/"channels") + # format_output: NO # Whether to export as 16 channels (subimages) with pre- and over-scan ("image"/"channels") shutter_output: NO # Whether to export shutter effect 16-bit image prnu_output: NO # Whether to export the PRNU (pixel-to-pixel flat-fielding) files diff --git a/config/obs_config_SCI.yaml b/config/obs_config_SCI.yaml index fb8f538b2bbdfd3b42ec0cacb5e6c3538a014fa0..16f7434296b7a058d86eab978b9a16ee24abcada 100644 --- a/config/obs_config_SCI.yaml +++ b/config/obs_config_SCI.yaml @@ -9,14 +9,14 @@ ############################################### # Observation type -obs_type: "SCI" +obs_type: "WIDE" obs_type_code: "101" obs_id: "00000001" # this setting will only be used if pointing list file is not given # Define list of chips # run_chips: [6,7,8,9,11,12,13,14,15,16,17,18,19,20,22,23,24,25] # Photometric chips #run_chips: [1,2,3,4,5,10,21,26,27,28,29,30] # Spectroscopic chips -run_chips: [17, 22] +run_chips: [17] # Define observation sequence call_sequence: @@ -28,6 +28,11 @@ call_sequence: shutter_effect: YES flat_fielding: YES field_dist: YES + + ghost_SCI: + mag_threshold: 16 + field_dist: YES + # Accumulate fluxes from sky background sky_background: # [Optional]: exposure time of the pointing will be used as default. @@ -50,16 +55,17 @@ call_sequence: # exptime: 150. # [s] save_cosmic_img: YES # # Whether to export cosmic ray image # Add Poission noise and dark current - poisson_and_dark: - # [Optional]: exposure time of the pointing will be used as default. - # Set it here is you want to override the default - # exptime: 150. # [s] - add_dark: YES + # poisson_and_dark: + # # [Optional]: exposure time of the pointing will be used as default. + # # Set it here is you want to override the default + # # exptime: 150. # [s] + # add_dark: YES + # input_dark: NO # Simulate brighter fatter effects bright_fatter: {} # Add detector defects: hot/warm pixels, bad columns detector_defects: - hot_pixels: YES + hot_pixels: NO dead_pixels: YES bad_columns: YES # Apply CCD Saturation & Blooming @@ -83,5 +89,5 @@ call_sequence: gain_16channel: YES # Output the final image quantization_and_output: - format_output: YES + format_output: NO ... diff --git a/install.sh b/install.sh index cc6e7ee565c1170b9441b2d13ed9b1e4d12a07d2..11e6c4f343412305162e85b5b3c2a3253f0e28ef 100644 --- a/install.sh +++ b/install.sh @@ -1,9 +1,5 @@ # conda create -n csstsim_py311 python==3.11 # conda activate csstsim_py311 conda config --add channels conda-forge -conda install numpy==1.26.4 -conda install cython==3.0.6 -conda install sep==1.2.1 -conda install mpi4py==3.1.6 -conda install cfitsio==3.470 +conda install -c conda-forge "cfitsio=3.*" pip install -e . diff --git a/observation_sim/ObservationSim.py b/observation_sim/ObservationSim.py index 22fc4880cbe1bb0a8351b7245bdfcfa88f5b8295..fd463235a299b743736b08030daf0c8d59fc20aa 100755 --- a/observation_sim/ObservationSim.py +++ b/observation_sim/ObservationSim.py @@ -23,11 +23,14 @@ class Observation(object): self.filter_param = FilterParam() self.Catalog = Catalog - def prepare_chip_for_exposure(self, chip, ra_cen, dec_cen, pointing, wcs_fp=None, slsPSFOptim=False): + def prepare_chip_for_exposure( + self, chip, ra_cen, dec_cen, pointing, wcs_fp=None, slsPSFOptim=False + ): # Get WCS for the focal plane if wcs_fp is None: wcs_fp = self.focal_plane.getTanWCS( - ra_cen, dec_cen, pointing.img_pa, chip.pix_scale) + ra_cen, dec_cen, pointing.img_pa, chip.pix_scale + ) # Create chip Image chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) @@ -41,14 +44,13 @@ class Observation(object): gn = chip_utils.getChipSLSGratingID(chip.chipID)[id1] orders = {} # for id2 in ['-2','-1','0','1','2']: - for id2 in ['0', '1']: - o_n = "order"+id2 + for id2 in ["0", "1"]: + o_n = "order" + id2 allbands = {} - for id3 in ['1', '2', '3', '4']: - w_n = "w"+id3 + for id3 in ["1", "2", "3", "4"]: + w_n = "w" + id3 allbands[w_n] = galsim.ImageF(chip.npix_x, chip.npix_y) - allbands[w_n].setOrigin( - chip.bound.xmin, chip.bound.ymin) + allbands[w_n].setOrigin(chip.bound.xmin, chip.bound.ymin) allbands[w_n].wcs = wcs_fp orders[o_n] = allbands chip.img_stack[gn] = orders @@ -57,39 +59,70 @@ class Observation(object): # Get random generators for this chip chip.rng_poisson, chip.poisson_noise = chip_utils.get_poisson( - seed=int(self.config["random_seeds"]["seed_poisson"]) + pointing.id*30 + chip.chipID, sky_level=0.) + seed=int(self.config["random_seeds"]["seed_poisson"]) + + pointing.id * 30 + + chip.chipID, + sky_level=0.0, + ) # Get flat, shutter, and PRNU images chip.flat_img, _ = chip_utils.get_flat( - img=chip.img, seed=int(self.config["random_seeds"]["seed_flat"])) + img=chip.img, seed=int(self.config["random_seeds"]["seed_flat"]) + ) if chip.chipID <= 30: - chip.flat_img = chip.flat_img*chip_utils.get_innerflat(chip=chip) + chip.flat_img = chip.flat_img * chip_utils.get_innerflat(chip=chip) if chip.chipID > 30: chip.shutter_img = np.ones_like(chip.img.array) else: chip.shutter_img = effects.ShutterEffectArr( - chip.img, t_exp=pointing.exp_time, t_shutter=1.5, dist_bearing=735, dt=1E-3) - chip.prnu_img = effects.PRNU_Img(xsize=chip.npix_x, ysize=chip.npix_y, sigma=0.007, - seed=int(self.config["random_seeds"]["seed_prnu"]+chip.chipID)) + chip.img, + t_exp=pointing.exp_time, + t_shutter=1.5, + dist_bearing=735, + dt=1e-3, + ) + chip.prnu_img = effects.PRNU_Img( + xsize=chip.npix_x, + ysize=chip.npix_y, + sigma=0.007, + seed=int(self.config["random_seeds"]["seed_prnu"] + chip.chipID), + ) return chip - def run_one_chip(self, chip, filt, pointing, chip_output, wcs_fp=None, psf_model=None, cat_dir=None, sed_dir=None): + def run_one_chip( + self, + chip, + filt, + pointing, + chip_output, + wcs_fp=None, + psf_model=None, + cat_dir=None, + sed_dir=None, + ): chip_output.Log_info( - ':::::::::::::::::::Current Pointing Information::::::::::::::::::') + ":::::::::::::::::::Current Pointing Information::::::::::::::::::" + ) chip_output.Log_info("RA: %f, DEC; %f" % (pointing.ra, pointing.dec)) - chip_output.Log_info("Time: %s" % datetime.utcfromtimestamp( - pointing.timestamp).isoformat()) + chip_output.Log_info( + "Time: %s" % datetime.utcfromtimestamp(pointing.timestamp).isoformat() + ) chip_output.Log_info("Exposure time: %f" % pointing.exp_time) - chip_output.Log_info("Satellite Position (x, y, z): (%f, %f, %f)" % ( - pointing.sat_x, pointing.sat_y, pointing.sat_z)) - chip_output.Log_info("Satellite Velocity (x, y, z): (%f, %f, %f)" % ( - pointing.sat_vx, pointing.sat_vy, pointing.sat_vz)) + chip_output.Log_info( + "Satellite Position (x, y, z): (%f, %f, %f)" + % (pointing.sat_x, pointing.sat_y, pointing.sat_z) + ) + chip_output.Log_info( + "Satellite Velocity (x, y, z): (%f, %f, %f)" + % (pointing.sat_vx, pointing.sat_vy, pointing.sat_vz) + ) chip_output.Log_info("Position Angle: %f" % pointing.img_pa.deg) - chip_output.Log_info('Chip : %d' % chip.chipID) + chip_output.Log_info("Chip : %d" % chip.chipID) chip_output.Log_info( - ':::::::::::::::::::::::::::END:::::::::::::::::::::::::::::::::::') + ":::::::::::::::::::::::::::END:::::::::::::::::::::::::::::::::::" + ) # Apply astrometric simulation for pointing if self.config["obs_setting"]["enable_astrometric_model"]: @@ -99,9 +132,9 @@ class Observation(object): ra_cen, dec_cen = on_orbit_obs_position( input_ra_list=[pointing.ra], input_dec_list=[pointing.dec], - input_pmra_list=[0.], - input_pmdec_list=[0.], - input_rv_list=[0.], + input_pmra_list=[0.0], + input_pmdec_list=[0.0], + input_rv_list=[0.0], input_parallax_list=[1e-9], input_nstars=1, input_x=pointing.sat_x, @@ -112,33 +145,37 @@ class Observation(object): input_vz=pointing.sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) - ra_offset, dec_offset = pointing.ra - \ - ra_cen[0], pointing.dec - dec_cen[0] + ra_offset, dec_offset = pointing.ra - ra_cen[0], pointing.dec - dec_cen[0] else: - ra_offset, dec_offset = 0., 0. + ra_offset, dec_offset = 0.0, 0.0 ra_cen = pointing.ra dec_cen = pointing.dec slsPSFOpt = False # Prepare necessary chip properties for simulation chip = self.prepare_chip_for_exposure( - chip, ra_cen, dec_cen, pointing, slsPSFOptim=slsPSFOpt) + chip, ra_cen, dec_cen, pointing, slsPSFOptim=slsPSFOpt + ) # Initialize SimSteps - sim_steps = SimSteps(overall_config=self.config, - chip_output=chip_output, - all_filters=self.all_filters, - ra_offset=ra_offset, - dec_offset=dec_offset) + sim_steps = SimSteps( + overall_config=self.config, + chip_output=chip_output, + all_filters=self.all_filters, + ra_offset=ra_offset, + dec_offset=dec_offset, + ) for step in pointing.obs_param["call_sequence"]: if self.config["run_option"]["out_cat_only"]: if step != "scie_obs": continue - chip_output.Log_info("Starting simulation step: %s, calling function: %s" % ( - step, SIM_STEP_TYPES[step])) + chip_output.Log_info( + "Starting simulation step: %s, calling function: %s" + % (step, SIM_STEP_TYPES[step]) + ) obs_param = pointing.obs_param["call_sequence"][step] step_name = SIM_STEP_TYPES[step] try: @@ -149,7 +186,8 @@ class Observation(object): tel=self.tel, pointing=pointing, catalog=self.Catalog, - obs_param=obs_param) + obs_param=obs_param, + ) chip_output.Log_info("Finished simulation step: %s" % (step)) except Exception as e: traceback.print_exc() @@ -160,8 +198,15 @@ class Observation(object): del chip.flat_img del chip.prnu_img del chip.shutter_img - chip_output.Log_info("check running:1: pointing-%d chip-%d pid-%d memory-%6.2fGB" % (pointing.id, - chip.chipID, os.getpid(), (psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024 / 1024))) + chip_output.Log_info( + "check running:1: pointing-%d chip-%d pid-%d memory-%6.2fGB" + % ( + pointing.id, + chip.chipID, + os.getpid(), + (psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024 / 1024), + ) + ) def runExposure_MPI_PointingList(self, pointing_list, chips=None): comm = MPI.COMM_WORLD @@ -176,10 +221,10 @@ class Observation(object): pointing_ID = pointing.obs_id pointing.make_output_pointing_dir( - overall_config=self.config, copy_obs_config=True) + overall_config=self.config, copy_obs_config=True + ) - self.focal_plane = FocalPlane( - chip_list=pointing.obs_param["run_chips"]) + self.focal_plane = FocalPlane(chip_list=pointing.obs_param["run_chips"]) # Make Chip & Filter lists self.chip_list = [] self.filter_list = [] @@ -191,7 +236,8 @@ class Observation(object): filt = Filter( filter_id=filter_id, filter_type=filter_type, - filter_param=self.filter_param) + filter_param=self.filter_param, + ) if not self.focal_plane.isIgnored(chipID=chipID): self.chip_list.append(chip) self.filter_list.append(filt) @@ -224,20 +270,16 @@ class Observation(object): filt = run_filts[ichip] chip_output = ChipOutput( - config=self.config, - chip=chip, - filt=filt, - pointing=pointing + config=self.config, chip=chip, filt=filt, pointing=pointing ) - chip_output.Log_info("running pointing#%d, chip#%d, at PID#%d..." % ( - int(pointing_ID), chip.chipID, pid)) - self.run_one_chip( - chip=chip, - filt=filt, - chip_output=chip_output, - pointing=pointing) chip_output.Log_info( - "finished running chip#%d..." % (chip.chipID)) + "running pointing#%d, chip#%d, at PID#%d..." + % (int(pointing_ID), chip.chipID, pid) + ) + self.run_one_chip( + chip=chip, filt=filt, chip_output=chip_output, pointing=pointing + ) + chip_output.Log_info("finished running chip#%d..." % (chip.chipID)) for handler in chip_output.logger.handlers[:]: chip_output.logger.removeHandler(handler) del chip_output diff --git a/observation_sim/_util.py b/observation_sim/_util.py index 7c2312750dce7ec35ea665ec67432889ffdcfd4f..c580acc2b6bfd2d1cae813914da54d4ba1f6b498 100755 --- a/observation_sim/_util.py +++ b/observation_sim/_util.py @@ -1,46 +1,53 @@ -import numpy as np import os from datetime import datetime import argparse -from astropy.time import Time from observation_sim.config import Pointing def parse_args(): - ''' + """ Parse command line arguments. Many of the following can be set in the .yaml config file as well. - ''' + """ parser = argparse.ArgumentParser() - parser.add_argument('--config_file', type=str, required=True, - help='.yaml config file for simulation settings.') - parser.add_argument('--catalog', type=str, - help='name of the catalog interface class to be loaded.') - parser.add_argument('-c', '--config_dir', type=str, - help='Directory that houses the .yaml config file.') - parser.add_argument('-d', '--data_dir', type=str, - help='Directory that houses the input data.') - parser.add_argument('-w', '--work_dir', type=str, - help='The path for output.') + parser.add_argument( + "--config_file", + type=str, + required=True, + help=".yaml config file for simulation settings.", + ) + parser.add_argument( + "--catalog", type=str, help="name of the catalog interface class to be loaded." + ) + parser.add_argument( + "-c", + "--config_dir", + type=str, + help="Directory that houses the .yaml config file.", + ) + parser.add_argument( + "-d", "--data_dir", type=str, help="Directory that houses the input data." + ) + parser.add_argument("-w", "--work_dir", type=str, help="The path for output.") return parser.parse_args() -def generate_pointing_list(config, pointing_filename=None, data_dir=None, dataset=''): +def generate_pointing_list(config, pointing_filename=None, data_dir=None, dataset=""): pointing_list = [] # Only valid when the pointing list does not contain time stamp column t0 = datetime(2021, 5, 25, 12, 0, 0) - delta_t = 10. # Time elapsed between exposures (minutes) + delta_t = 10.0 # Time elapsed between exposures (minutes) # Calculate starting time(s) for CAL exposures # NOTE: temporary implementation t = datetime.timestamp(t0) ipoint = 0 - run_pointings = config['obs_setting']['run_pointings'] - if "obs_config_file" in config['obs_setting']: - obs_config_file = config['obs_setting']["obs_config_file"] + run_pointings = config["obs_setting"]["run_pointings"] + if "obs_config_file" in config["obs_setting"]: + obs_config_file = config["obs_setting"]["obs_config_file"] else: obs_config_file = None @@ -50,12 +57,12 @@ def generate_pointing_list(config, pointing_filename=None, data_dir=None, datase pointing_file = os.path.join(data_dir, pointing_filename) else: pointing_file = pointing_filename - f = open(pointing_file, 'r') + f = open(pointing_file, "r") # for _ in range(1): # header = f.readline() iline = 0 for line in f: - if len(line.strip()) == 0 or line[0] == '#': + if len(line.strip()) == 0 or line[0] == "#": continue if run_pointings and iline not in run_pointings: iline += 1 @@ -63,10 +70,9 @@ def generate_pointing_list(config, pointing_filename=None, data_dir=None, datase continue line = line.strip() columns = line.split() - pointing = Pointing( - obs_config_file=obs_config_file, dataset=dataset) + pointing = Pointing(obs_config_file=obs_config_file, dataset=dataset) pointing.read_pointing_columns(columns=columns, id=ipoint) - t += delta_t * 60. + t += delta_t * 60.0 pointing_list.append(pointing) iline += 1 ipoint += 1 @@ -75,7 +81,7 @@ def generate_pointing_list(config, pointing_filename=None, data_dir=None, datase if config["obs_setting"]["exp_time"]: exp_time = config["obs_setting"]["exp_time"] else: - exp_time = 150. + exp_time = 150.0 pointing = Pointing( id=ipoint, ra=config["obs_setting"]["ra_center"], @@ -83,11 +89,11 @@ def generate_pointing_list(config, pointing_filename=None, data_dir=None, datase img_pa=config["obs_setting"]["image_rot"], timestamp=t, exp_time=exp_time, - pointing_type='SCI', + pointing_type="SCI", obs_config_file=obs_config_file, - dataset=dataset + dataset=dataset, ) - t += delta_t * 60. + t += delta_t * 60.0 pointing_list.append(pointing) ipoint += 1 return pointing_list @@ -115,7 +121,7 @@ def make_output_pointing_dir(path_dict, config, pointing_ID=0): os.makedirs(imgDir, exist_ok=True) except OSError: pass - prefix = "MSC_" + str(pointing_ID).rjust(8, '0') + prefix = "MSC_" + str(pointing_ID).rjust(8, "0") subImgdir = os.path.join(imgDir, prefix) if not os.path.exists(subImgdir): try: @@ -126,7 +132,7 @@ def make_output_pointing_dir(path_dict, config, pointing_ID=0): def get_shear_field(config): - if not config["shear_setting"]["shear_type"] in ["constant", "catalog"]: + if config["shear_setting"]["shear_type"] not in ["constant", "catalog"]: raise ValueError("Please set a right 'shear_method' parameter.") if config["shear_setting"]["shear_type"] == "constant": @@ -135,6 +141,6 @@ def get_shear_field(config): nshear = 1 # TODO logging else: - g1, g2 = 0., 0. + g1, g2 = 0.0, 0.0 nshear = 0 return g1, g2, nshear diff --git a/observation_sim/astrometry/Astrometry_util.py b/observation_sim/astrometry/Astrometry_util.py index bdfe613905658df3ec967f8ce80e06b1ce2c34be..31eaa1a41c978fa90d8851958c847ba57afc7b4d 100644 --- a/observation_sim/astrometry/Astrometry_util.py +++ b/observation_sim/astrometry/Astrometry_util.py @@ -14,14 +14,32 @@ def checkInputList(input_list, n): for i in input_list: if type(i) is not type(1.1): if type(i) is not type(1): - raise TypeError( - "Input list's element is not float or int!", input_list) + raise TypeError("Input list's element is not float or int!", input_list) if len(input_list) != n: raise RuntimeError( - "Length of input list is not equal to stars' number!", input_list) + "Length of input list is not equal to stars' number!", input_list + ) -def on_orbit_obs_position(input_ra_list, input_dec_list, input_pmra_list, input_pmdec_list, input_rv_list, input_parallax_list, input_nstars, input_x, input_y, input_z, input_vx, input_vy, input_vz, input_epoch, input_date_str, input_time_str, lib_path=None): +def on_orbit_obs_position( + input_ra_list, + input_dec_list, + input_pmra_list, + input_pmdec_list, + input_rv_list, + input_parallax_list, + input_nstars, + input_x, + input_y, + input_z, + input_vx, + input_vy, + input_vz, + input_epoch, + input_date_str, + input_time_str, + lib_path=None, +): # Check input parameters if not isinstance(input_nstars, int): raise TypeError("Parameter 7 is not int!", input_nstars) @@ -46,12 +64,12 @@ def on_orbit_obs_position(input_ra_list, input_dec_list, input_pmra_list, input_ if not isinstance(input_vz, float): raise TypeError("Parameter 13 is not double!", input_vz) # Convert km -> m - input_x = input_x*1000.0 - input_y = input_y*1000.0 - input_z = input_z*1000.0 - input_vx = input_vx*1000.0 - input_vy = input_vy*1000.0 - input_vz = input_vz*1000.0 + input_x = input_x * 1000.0 + input_y = input_y * 1000.0 + input_z = input_z * 1000.0 + input_vx = input_vx * 1000.0 + input_vy = input_vy * 1000.0 + input_vz = input_vz * 1000.0 if not isinstance(input_date_str, str): raise TypeError("Parameter 15 is not string!", input_date_str) @@ -62,20 +80,18 @@ def on_orbit_obs_position(input_ra_list, input_dec_list, input_pmra_list, input_ else: tmp = input_date_str.split("-") if len(tmp) != 3: - raise TypeError( - "Parameter 15 format error (2)!", input_date_str) + raise TypeError("Parameter 15 format error (2)!", input_date_str) input_year = int(tmp[0]) input_month = int(tmp[1]) input_day = int(tmp[2]) if not (input_year >= 1900 and input_year <= 2100): raise TypeError( - "Parameter 15 year range error [1900 ~ 2100]!", input_year) + "Parameter 15 year range error [1900 ~ 2100]!", input_year + ) if not (input_month >= 1 and input_month <= 12): - raise TypeError( - "Parameter 15 month range error [1 ~ 12]!", input_month) + raise TypeError("Parameter 15 month range error [1 ~ 12]!", input_month) if not (input_day >= 1 and input_day <= 31): - raise TypeError( - "Parameter 15 day range error [1 ~ 31]!", input_day) + raise TypeError("Parameter 15 day range error [1 ~ 31]!", input_day) if not isinstance(input_time_str, str): raise TypeError("Parameter 16 is not string!", input_time_str) @@ -86,36 +102,60 @@ def on_orbit_obs_position(input_ra_list, input_dec_list, input_pmra_list, input_ else: tmp = input_time_str.split(":") if len(tmp) != 3: - raise TypeError( - "Parameter 16 format error (2)!", input_time_str) + raise TypeError("Parameter 16 format error (2)!", input_time_str) input_hour = int(tmp[0]) input_minute = int(tmp[1]) input_second = float(tmp[2]) if not (input_hour >= 0 and input_hour <= 23): - raise TypeError( - "Parameter 16 hour range error [0 ~ 23]!", input_hour) + raise TypeError("Parameter 16 hour range error [0 ~ 23]!", input_hour) if not (input_minute >= 0 and input_minute <= 59): raise TypeError( - "Parameter 16 minute range error [0 ~ 59]!", input_minute) + "Parameter 16 minute range error [0 ~ 59]!", input_minute + ) if not (input_second >= 0 and input_second < 60.0): raise TypeError( - "Parameter 16 second range error [0 ~ 60)!", input_second) + "Parameter 16 second range error [0 ~ 60)!", input_second + ) # Inital dynamic lib try: - with pkg_resources.files('observation_sim.astrometry.lib').joinpath("libshao.so") as lib_path: + with pkg_resources.files("observation_sim.astrometry.lib").joinpath( + "libshao.so" + ) as lib_path: shao = cdll.LoadLibrary(lib_path) except AttributeError: - with pkg_resources.path('observation_sim.astrometry.lib', "libshao.so") as lib_path: + with pkg_resources.path( + "observation_sim.astrometry.lib", "libshao.so" + ) as lib_path: shao = cdll.LoadLibrary(lib_path) shao.onOrbitObs.restype = c_int d3 = c_double * 3 - shao.onOrbitObs.argtypes = [c_double, c_double, c_double, c_double, c_double, c_double, - c_int, c_int, c_int, c_int, c_int, c_double, - c_double, POINTER(d3), POINTER(d3), - c_int, c_int, c_int, c_int, c_int, c_double, - POINTER(c_double), POINTER(c_double)] + shao.onOrbitObs.argtypes = [ + c_double, + c_double, + c_double, + c_double, + c_double, + c_double, + c_int, + c_int, + c_int, + c_int, + c_int, + c_double, + c_double, + POINTER(d3), + POINTER(d3), + c_int, + c_int, + c_int, + c_int, + c_int, + c_double, + POINTER(c_double), + POINTER(c_double), + ] output_ra_list = list() output_dec_list = list() for i in range(input_nstars): @@ -137,11 +177,31 @@ def on_orbit_obs_position(input_ra_list, input_dec_list, input_pmra_list, input_ DAT = c_double(37.0) output_ra = c_double(0.0) output_dec = c_double(0.0) - rs = shao.onOrbitObs(input_ra, input_dec, input_parallax, input_pmra, input_pmdec, input_rv, - input_year_c, input_month_c, input_day_c, input_hour_c, input_minute_c, input_second_c, - DAT, byref(p3), byref(v3), - input_year_c, input_month_c, input_day_c, input_hour_c, input_minute_c, input_second_c, - byref(output_ra), byref(output_dec)) + rs = shao.onOrbitObs( + input_ra, + input_dec, + input_parallax, + input_pmra, + input_pmdec, + input_rv, + input_year_c, + input_month_c, + input_day_c, + input_hour_c, + input_minute_c, + input_second_c, + DAT, + byref(p3), + byref(v3), + input_year_c, + input_month_c, + input_day_c, + input_hour_c, + input_minute_c, + input_second_c, + byref(output_ra), + byref(output_dec), + ) if rs != 0: raise RuntimeError("Calculate error!") output_ra_list.append(output_ra.value) diff --git a/observation_sim/cli.py b/observation_sim/cli.py new file mode 100644 index 0000000000000000000000000000000000000000..59e036b3532ffeafe654443c0bb638f0d44e75c8 --- /dev/null +++ b/observation_sim/cli.py @@ -0,0 +1,5 @@ +from observation_sim.run import run_sim + + +def main(): + run_sim() diff --git a/observation_sim/config/ChipOutput.py b/observation_sim/config/ChipOutput.py index 87aa566de5dfb7cb84cf3a166022e7fe3aded14a..0aec5d0668e0d8e9c1c13c004d80120cb4348421 100755 --- a/observation_sim/config/ChipOutput.py +++ b/observation_sim/config/ChipOutput.py @@ -10,7 +10,7 @@ class ChipOutput(object): self.chip = chip self.filt = filt self.pointing_type = pointing.pointing_type - self.chip_label = str(chip.chipID).rjust(2, '0') + self.chip_label = str(chip.chipID).rjust(2, "0") # Get primary header based on chip and pointing self.h_prim = generatePrimaryHeader( @@ -31,30 +31,40 @@ class ChipOutput(object): run_counter=self.config["run_counter"], chip_name=self.chip_label, obstype=pointing.pointing_type, - dataset=pointing.dataset) + dataset=pointing.dataset, + ) - obs_id = _util.get_obs_id(img_type=self.pointing_type, project_cycle=config["project_cycle"], run_counter=config[ - "run_counter"], pointing_id=pointing.obs_id, pointing_type_code=pointing.pointing_type_code) + obs_id = _util.get_obs_id( + img_type=self.pointing_type, + project_cycle=config["project_cycle"], + run_counter=config["run_counter"], + pointing_id=pointing.obs_id, + pointing_type_code=pointing.pointing_type_code, + ) self.subdir = pointing.output_dir - self.cat_name = self.h_prim['FILENAME'] + '.cat' + self.cat_name = self.h_prim["FILENAME"] + ".cat" if logger_filename is None: - logger_filename = self.h_prim['FILENAME'] + '.log' + logger_filename = self.h_prim["FILENAME"] + ".log" self.logger = logging.getLogger() - fh = logging.FileHandler(os.path.join( - self.subdir, logger_filename), mode='w+', encoding='utf-8') + fh = logging.FileHandler( + os.path.join(self.subdir, logger_filename), mode="w+", encoding="utf-8" + ) fh.setLevel(logging.DEBUG) self.logger.setLevel(logging.DEBUG) - logging.getLogger('numba').setLevel(logging.WARNING) + logging.getLogger("numba").setLevel(logging.WARNING) formatter = logging.Formatter( - '%(asctime)s - %(msecs)d - %(levelname)-8s - [%(filename)s:%(lineno)d] - %(message)s') + "%(asctime)s - %(msecs)d - %(levelname)-8s - [%(filename)s:%(lineno)d] - %(message)s" + ) fh.setFormatter(formatter) self.logger.addHandler(fh) hdr1 = "# obj_ID ID_chip filter xImage yImage ra dec ra_orig dec_orig z mag obj_type " hdr2 = "pm_ra pm_dec RV parallax" - fmt1 = "%20s %4d %5s %10.3f %10.3f %15.8f %15.8f %15.8f %15.8f %7.4f %8.4f %15s " + fmt1 = ( + "%20s %4d %5s %10.3f %10.3f %15.8f %15.8f %15.8f %15.8f %7.4f %8.4f %15s " + ) fmt2 = "%15.8f %15.8f %15.8f %15.8f" self.hdr = hdr1 + hdr2 self.fmt = fmt1 + fmt2 @@ -73,22 +83,44 @@ class ChipOutput(object): self.hdr += additional_column_names def create_output_file(self): - if self.pointing_type == 'WIDE' or self.pointing_type == 'DEEP' or self.pointing_type == 'CALF' or self.pointing_type == 'CALSP' or self.pointing_type == 'CALSS': + if ( + self.pointing_type == "WIDE" + or self.pointing_type == "DEEP" + or self.pointing_type == "CALF" + or self.pointing_type == "CALSP" + or self.pointing_type == "CALSS" + ): self.cat = open(os.path.join(self.subdir, self.cat_name), "w") - self.logger.info("Creating catalog file %s ...\n" % - (os.path.join(self.subdir, self.cat_name))) + self.logger.info( + "Creating catalog file %s ...\n" + % (os.path.join(self.subdir, self.cat_name)) + ) if not self.hdr.endswith("\n"): self.hdr += "\n" self.cat.write(self.hdr) - def cat_add_obj(self, obj, pos_img, pos_shear, ra_offset=0., dec_offset=0.): + def cat_add_obj(self, obj, pos_img, pos_shear, ra_offset=0.0, dec_offset=0.0): ximg = obj.real_pos.x + 1.0 yimg = obj.real_pos.y + 1.0 line = self.fmt % ( - obj.id, int(self.chip_label), self.filt.filter_type, ximg, yimg, obj.ra + ra_offset, obj.dec + dec_offset, obj.ra_orig, obj.dec_orig, obj.z, obj.getMagFilter( - self.filt), obj.type, - obj.pmra, obj.pmdec, obj.rv, obj.parallax) + obj.id, + int(self.chip_label), + self.filt.filter_type, + ximg, + yimg, + obj.ra + ra_offset, + obj.dec + dec_offset, + obj.ra_orig, + obj.dec_orig, + obj.z, + obj.getMagFilter(self.filt), + obj.type, + obj.pmra, + obj.pmdec, + obj.rv, + obj.parallax, + ) line += obj.additional_output_str if not line.endswith("\n"): line += "\n" diff --git a/observation_sim/config/Pointing.py b/observation_sim/config/Pointing.py index 9841c0f70645e09c3e213996fb2b3213f1500563..d45cce28497f86cff4d5c412c716ca0e31be8241 100644 --- a/observation_sim/config/Pointing.py +++ b/observation_sim/config/Pointing.py @@ -5,12 +5,35 @@ import galsim import numpy as np from astropy.time import Time -from observation_sim.config._util import get_obs_id import observation_sim.instruments._util as _util class Pointing(object): - def __init__(self, id=0, ra=0., dec=0., img_pa=0., timestamp=1621915200, sat_x=0., sat_y=0., sat_z=0., sun_x=0., sun_y=0., sun_z=0., sat_vx=0., sat_vy=0., sat_vz=0., exp_time=150., pointing_type='WIDE', pointing_type_code='101', pointing_id='00000001', obs_config_file=None, t_shutter_open=1.3, t_shutter_close=1.3, dataset='csst-msc-c9-25sqdeg-v3'): + def __init__( + self, + id=0, + ra=0.0, + dec=0.0, + img_pa=0.0, + timestamp=1621915200, + sat_x=0.0, + sat_y=0.0, + sat_z=0.0, + sun_x=0.0, + sun_y=0.0, + sun_z=0.0, + sat_vx=0.0, + sat_vy=0.0, + sat_vz=0.0, + exp_time=150.0, + pointing_type="WIDE", + pointing_type_code="101", + pointing_id="00000001", + obs_config_file=None, + t_shutter_open=1.3, + t_shutter_close=1.3, + dataset="csst-msc-c9-25sqdeg-v3", + ): self.id = id self.ra = ra self.dec = dec @@ -23,8 +46,8 @@ class Pointing(object): self.pointing_type = pointing_type self.pointing_type_code = pointing_type_code self.obs_id = pointing_id - self.survey_field_type = 'WIDE' - self.jdt = 0. + self.survey_field_type = "WIDE" + self.jdt = 0.0 self.obs_config_file = obs_config_file self.t_shutter_open = t_shutter_open self.t_shutter_close = t_shutter_close @@ -44,28 +67,28 @@ class Pointing(object): self.dataset = dataset def get_full_depth_exptime(self, filter_type): - if self.survey_field_type == 'WIDE': + if self.survey_field_type == "WIDE": if filter_type in _util.SPEC_FILTERS: - return 150. * 4 + return 150.0 * 4 else: - if filter_type.lower() in ['nuv', 'y']: - return 150. * 4 - elif filter_type.lower() in ['u', 'g', 'r', 'i', 'z']: - return 150. * 2 + if filter_type.lower() in ["nuv", "y"]: + return 150.0 * 4 + elif filter_type.lower() in ["u", "g", "r", "i", "z"]: + return 150.0 * 2 else: - return max(150., self.exp_time) # [TODO] for FGS - elif self.survey_field_type == 'DEEP': + return max(150.0, self.exp_time) # [TODO] for FGS + elif self.survey_field_type == "DEEP": if filter_type in _util.SPEC_FILTERS: - return 250. * 4 * 4 + return 250.0 * 4 * 4 else: - if filter_type.lower() in ['nuv', 'y']: - return 250. * 4 * 4 - elif filter_type.lower() in ['u', 'g', 'r', 'i', 'z']: - return 250. * 2 * 4 + if filter_type.lower() in ["nuv", "y"]: + return 250.0 * 4 * 4 + elif filter_type.lower() in ["u", "g", "r", "i", "z"]: + return 250.0 * 2 * 4 else: - return max(150., self.exp_time) # [TODO] for FGS + return max(150.0, self.exp_time) # [TODO] for FGS - def read_pointing_columns(self, columns, id=0, t=1621915200, pointing_type='SCI'): + def read_pointing_columns(self, columns, id=0, t=1621915200, pointing_type="SCI"): self.id = id col_len = len(columns) self.ra = float(columns[0]) @@ -74,7 +97,7 @@ class Pointing(object): # self.pointing_type = pointing_type if col_len > 5: jdt = np.double(columns[5]) - t_temp = Time(jdt, format='jd') + t_temp = Time(jdt, format="jd") self.jdt = jdt self.timestamp = t_temp.unix self.sat_x = float(columns[6]) @@ -111,8 +134,7 @@ class Pointing(object): self.timestamp = t def make_output_pointing_dir(self, overall_config, copy_obs_config=False): - run_dir = os.path.join( - overall_config["work_dir"], overall_config["run_name"]) + run_dir = os.path.join(overall_config["work_dir"], overall_config["run_name"]) if not os.path.exists(run_dir): try: os.makedirs(run_dir, exist_ok=True) @@ -124,7 +146,7 @@ class Pointing(object): # run_counter=overall_config["run_counter"], # pointing_id=self.obs_id, # pointing_type_code=self.pointing_type_code) - self.output_prefix = self.pointing_type_code+self.obs_id + self.output_prefix = self.pointing_type_code + self.obs_id self.output_dir = os.path.join(run_dir, self.output_prefix) if not os.path.exists(self.output_dir): try: @@ -133,7 +155,8 @@ class Pointing(object): pass if copy_obs_config and self.obs_config_file: obs_config_output_path = os.path.join( - self.output_dir, os.path.basename(self.obs_config_file)) + self.output_dir, os.path.basename(self.obs_config_file) + ) if not os.path.exists(obs_config_output_path): try: shutil.copy(self.obs_config_file, self.output_dir) diff --git a/observation_sim/config/_util.py b/observation_sim/config/_util.py index 7ab0f59d663cf9ad36bf5c63abb9c63e2cd4cdb0..c8e78f7343e8c00c7fa516d17bf854c5345329e9 100644 --- a/observation_sim/config/_util.py +++ b/observation_sim/config/_util.py @@ -1,14 +1,30 @@ - -def get_obs_id(img_type='SCI', project_cycle=6, run_counter=0, pointing_id='00000001', pointing_type_code='101'): +def get_obs_id( + img_type="SCI", + project_cycle=6, + run_counter=0, + pointing_id="00000001", + pointing_type_code="101", +): # obs_type = {'SCI': '01', 'BIAS': '03', 'DARK': '07', 'FLAT': '11', 'CRS': '98', 'CRD': '99'} # obs_type = {'SCIE': '01', 'BIAS': '03', 'DARK': '07', 'FLAT': '11', 'CRS': '98', 'CRD': '99', 'CAL': '01'} - obs_id = pointing_type_code + \ - str(int(project_cycle)).rjust(2, '0') + \ - str(int(run_counter)) + pointing_id + obs_id = ( + pointing_type_code + + str(int(project_cycle)).rjust(2, "0") + + str(int(run_counter)) + + pointing_id + ) return obs_id -def get_file_type(img_type='SCI'): - file_type = {'SCI': 'SCI', 'BIAS': 'BIAS', 'DARK': 'DARK', 'FLAT': 'FLAT', - 'CRS': 'CRS', 'CRD': 'CRD', 'CALS': 'CALS', 'CALF': 'CALF'} +def get_file_type(img_type="SCI"): + file_type = { + "SCI": "SCI", + "BIAS": "BIAS", + "DARK": "DARK", + "FLAT": "FLAT", + "CRS": "CRS", + "CRD": "CRD", + "CALS": "CALS", + "CALF": "CALF", + } return file_type[img_type] diff --git a/observation_sim/config/header/ImageHeader.py b/observation_sim/config/header/ImageHeader.py index 69f34fb03dbf224c61bc3048ca84bc2711f60807..bece2f2940e418a98ca7560d84fd43f73e8320f7 100644 --- a/observation_sim/config/header/ImageHeader.py +++ b/observation_sim/config/header/ImageHeader.py @@ -1,32 +1,29 @@ """ generate image header """ + import numpy as np from astropy.io import fits -import astropy.wcs as pywcs from collections import OrderedDict # from scipy import math -import random import os -import sys import astropy.coordinates as coord from astropy.coordinates import SkyCoord from astropy.wcs.utils import fit_wcs_from_points from astropy.time import Time from astropy import wcs -from observation_sim.config._util import get_obs_id, get_file_type from datetime import datetime, timezone + # import socket import platform -import toml def chara2digit(char): - """ Function to judge and convert characters to digitals + """Function to judge and convert characters to digitals Parameters ---------- @@ -43,8 +40,8 @@ def chara2digit(char): return data -def read_header_parameter(filename='global_header.param'): - """ Function to read the header parameters +def read_header_parameter(filename="global_header.param"): + """Function to read the header parameters Parameters ---------- @@ -56,15 +53,15 @@ def read_header_parameter(filename='global_header.param'): description = [] for line in open(filename): line = line.strip("\n") - arr = line.split('|') -# csvReader = csv.reader(csvDataFile) -# for arr in csvReader: + arr = line.split("|") + # csvReader = csv.reader(csvDataFile) + # for arr in csvReader: name.append(arr[0]) # print(arr[0],arr[1]) value.append(chara2digit(arr[1])) description.append(arr[2]) -# print(value) + # print(value) return name, value, description @@ -104,8 +101,8 @@ def calcaluteSLSRotSkyCoor(pix_xy=None, rot_angle=1, xlen=9216, ylen=9232, w=Non mat = np.zeros((2, 2)) mat[0, :] = np.array([np.cos(rad), -np.sin(rad)]) mat[1, :] = np.array([np.sin(rad), np.cos(rad)]) - center = np.array([xlen/2, ylen/2]) - rot_pix = np.dot(mat, pix_xy-center) + center + center = np.array([xlen / 2, ylen / 2]) + rot_pix = np.dot(mat, pix_xy - center) + center skyCoor = w.wcs_pix2world(np.array([rot_pix]), 1) return skyCoor @@ -176,21 +173,37 @@ def calcaluteSLSRotSkyCoor(pix_xy=None, rot_angle=1, xlen=9216, ylen=9232, w=Non # 9232 9216 898 534 1309 60 -40 -23.4333 -def WCS_def(xlen=9216, ylen=9232, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=60, dec_ref=-40, pa=-23.433, pixel_scale=0.074, pixel_size=1e-2, - rotate_chip=0., filter='GI', row_num=None, col_num=None, xcen=None, ycen=None): - """ Creat a wcs frame for CCST with multiple extensions +def WCS_def( + xlen=9216, + ylen=9232, + gapy=898.0, + gapx1=534, + gapx2=1309, + ra_ref=60, + dec_ref=-40, + pa=-23.433, + pixel_scale=0.074, + pixel_size=1e-2, + rotate_chip=0.0, + filter="GI", + row_num=None, + col_num=None, + xcen=None, + ycen=None, +): + """Creat a wcs frame for CCST with multiple extensions Parameters ---------- """ r_dat = OrderedDict() - r_dat['EQUINOX'] = 2000.0 - r_dat['WCSDIM'] = 2.0 - r_dat['CTYPE1'] = 'RA---TAN' - r_dat['CTYPE2'] = 'DEC--TAN' - r_dat['CRVAL1'] = ra_ref - r_dat['CRVAL2'] = dec_ref + r_dat["EQUINOX"] = 2000.0 + r_dat["WCSDIM"] = 2.0 + r_dat["CTYPE1"] = "RA---TAN" + r_dat["CTYPE2"] = "DEC--TAN" + r_dat["CRVAL1"] = ra_ref + r_dat["CRVAL2"] = dec_ref flag_x = [0, 1, -1, 1, -1] flag_y = [0, 1, 1, -1, -1] @@ -199,10 +212,9 @@ def WCS_def(xlen=9216, ylen=9232, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=60, pa_aper = pa if (row_num is not None) and (col_num is not None): - x_num = 6 y_num = 5 - detector_num = x_num*y_num + detector_num = x_num * y_num detector_size_x = xlen detector_size_y = ylen @@ -212,45 +224,52 @@ def WCS_def(xlen=9216, ylen=9232, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=60, gap_x1_num = 3 gap_x2_num = 2 - y_center = (detector_size_y*y_num+gap_y*(y_num-1))/2 + y_center = (detector_size_y * y_num + gap_y * (y_num - 1)) / 2 - x_center = (detector_size_x*x_num + - gap_x[0]*gap_x1_num+gap_x[1]*gap_x2_num)/2 + x_center = ( + detector_size_x * x_num + gap_x[0] * gap_x1_num + gap_x[1] * gap_x2_num + ) / 2 - gap_x_map = np.array([[0, 0, 0, 0, 0], [gap_x[0], gap_x[1], gap_x[1], gap_x[1], gap_x[1]], [gap_x[1], gap_x[0], gap_x[0], gap_x[0], gap_x[0]], [ - gap_x[0], gap_x[0], gap_x[0], gap_x[0], gap_x[0]], [gap_x[0], gap_x[0], gap_x[0], gap_x[0], gap_x[1]], [gap_x[1], gap_x[1], gap_x[1], gap_x[1], gap_x[0]]]) + gap_x_map = np.array([ + [0, 0, 0, 0, 0], + [gap_x[0], gap_x[1], gap_x[1], gap_x[1], gap_x[1]], + [gap_x[1], gap_x[0], gap_x[0], gap_x[0], gap_x[0]], + [gap_x[0], gap_x[0], gap_x[0], gap_x[0], gap_x[0]], + [gap_x[0], gap_x[0], gap_x[0], gap_x[0], gap_x[1]], + [gap_x[1], gap_x[1], gap_x[1], gap_x[1], gap_x[0]], + ]) j = row_num i = col_num # ccdnum = str((j-1)*5+i) - x_ref, y_ref = detector_size_x*i + \ - sum(gap_x_map[0:i, j-1]) - detector_size_x / \ - 2., (detector_size_y+gap_y)*j-gap_y-detector_size_y/2 + x_ref, y_ref = ( + detector_size_x * i + sum(gap_x_map[0:i, j - 1]) - detector_size_x / 2.0, + (detector_size_y + gap_y) * j - gap_y - detector_size_y / 2, + ) for k in range(1, 2): - - cd = np.array([[pixel_scale, 0], [0, pixel_scale]]) / \ - 3600.*flag_x[k] + cd = np.array([[pixel_scale, 0], [0, pixel_scale]]) / 3600.0 * flag_x[k] cd_rot = rotate_CD_matrix(cd, pa_aper) # f = open("CCD"+ccdnum.rjust(2,'0')+"_extension"+str(k)+"_wcs.param","w") - r_dat['CRPIX1'] = flag_ext_x[k] * \ - ((x_ref+flag_ext_x[k]*detector_size_x/2)-x_center) - r_dat['CRPIX2'] = flag_ext_y[k] * \ - ((y_ref+flag_ext_y[k]*detector_size_y/2)-y_center) - r_dat['CD1_1'] = cd_rot[0, 0] - r_dat['CD1_2'] = cd_rot[0, 1] - r_dat['CD2_1'] = cd_rot[1, 0] - r_dat['CD2_2'] = cd_rot[1, 1] - - if filter in ['GU', 'GV', 'GI']: - + r_dat["CRPIX1"] = flag_ext_x[k] * ( + (x_ref + flag_ext_x[k] * detector_size_x / 2) - x_center + ) + r_dat["CRPIX2"] = flag_ext_y[k] * ( + (y_ref + flag_ext_y[k] * detector_size_y / 2) - y_center + ) + r_dat["CD1_1"] = cd_rot[0, 0] + r_dat["CD1_2"] = cd_rot[0, 1] + r_dat["CD2_1"] = cd_rot[1, 0] + r_dat["CD2_2"] = cd_rot[1, 1] + + if filter in ["GU", "GV", "GI"]: w = wcs.WCS(naxis=2) - w.wcs.crpix = [r_dat['CRPIX1'], r_dat['CRPIX2']] + w.wcs.crpix = [r_dat["CRPIX1"], r_dat["CRPIX2"]] w.wcs.cd = cd_rot w.wcs.crval = [ra_ref, dec_ref] - w.wcs.ctype = [r_dat['CTYPE1'], r_dat['CTYPE2']] + w.wcs.ctype = [r_dat["CTYPE1"], r_dat["CTYPE2"]] # test_center_o = w.wcs_pix2world(np.array([[xlen / 2, ylen / 2]]), 1) @@ -275,47 +294,55 @@ def WCS_def(xlen=9216, ylen=9232, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=60, pix_coor = np.array([x, y]) sc1 = calcaluteSLSRotSkyCoor( - pix_xy=pix_coor, rot_angle=sls_rot, xlen=xlen, ylen=ylen, w=w) + pix_xy=pix_coor, + rot_angle=sls_rot, + xlen=xlen, + ylen=ylen, + w=w, + ) # print(sc1[0,0],sc1[0,1]) sky_coors.append((sc1[0, 0], sc1[0, 1])) - wcs_new = fit_wcs_from_points(xy=np.array([x_pixs, y_pixs]), - world_coords=SkyCoord(sky_coors, frame="icrs", unit="deg"), projection='TAN') + wcs_new = fit_wcs_from_points( + xy=np.array([x_pixs, y_pixs]), + world_coords=SkyCoord(sky_coors, frame="icrs", unit="deg"), + projection="TAN", + ) # print(wcs_new) # test_center = wcs_new.wcs_pix2world(np.array([[xlen / 2, ylen / 2]]), 1) # # print(test_center - test_center_o) - r_dat['CD1_1'] = wcs_new.wcs.cd[0, 0] - r_dat['CD1_2'] = wcs_new.wcs.cd[0, 1] - r_dat['CD2_1'] = wcs_new.wcs.cd[1, 0] - r_dat['CD2_2'] = wcs_new.wcs.cd[1, 1] - r_dat['CRPIX1'] = wcs_new.wcs.crpix[0] - r_dat['CRPIX2'] = wcs_new.wcs.crpix[1] + r_dat["CD1_1"] = wcs_new.wcs.cd[0, 0] + r_dat["CD1_2"] = wcs_new.wcs.cd[0, 1] + r_dat["CD2_1"] = wcs_new.wcs.cd[1, 0] + r_dat["CD2_2"] = wcs_new.wcs.cd[1, 1] + r_dat["CRPIX1"] = wcs_new.wcs.crpix[0] + r_dat["CRPIX2"] = wcs_new.wcs.crpix[1] - r_dat['CRVAL1'] = wcs_new.wcs.crval[0] - r_dat['CRVAL2'] = wcs_new.wcs.crval[1] + r_dat["CRVAL1"] = wcs_new.wcs.crval[0] + r_dat["CRVAL2"] = wcs_new.wcs.crval[1] elif (xcen is not None) and (ycen is not None): - xcen, ycen = xcen/pixel_size, ycen/pixel_size - x1, y1 = xcen - xlen/2., ycen - ylen/2. - r_dat['CRPIX1'] = -x1 - r_dat['CRPIX2'] = -y1 + xcen, ycen = xcen / pixel_size, ycen / pixel_size + x1, y1 = xcen - xlen / 2.0, ycen - ylen / 2.0 + r_dat["CRPIX1"] = -x1 + r_dat["CRPIX2"] = -y1 # cd = np.array([[ pixel_scale, 0], [0, pixel_scale]])/3600.*flag_x[1] - cd = np.array([[pixel_scale, 0], [0, -pixel_scale]])/3600. + cd = np.array([[pixel_scale, 0], [0, -pixel_scale]]) / 3600.0 cd_rot = rotate_CD_matrix(cd, pa_aper) - r_dat['CD1_1'] = cd_rot[0, 0] - r_dat['CD1_2'] = cd_rot[0, 1] - r_dat['CD2_1'] = cd_rot[1, 0] - r_dat['CD2_2'] = cd_rot[1, 1] + r_dat["CD1_1"] = cd_rot[0, 0] + r_dat["CD1_2"] = cd_rot[0, 1] + r_dat["CD2_1"] = cd_rot[1, 0] + r_dat["CD2_2"] = cd_rot[1, 1] w = wcs.WCS(naxis=2) - w.wcs.crpix = [r_dat['CRPIX1'], r_dat['CRPIX2']] + w.wcs.crpix = [r_dat["CRPIX1"], r_dat["CRPIX2"]] w.wcs.cd = cd_rot w.wcs.crval = [ra_ref, dec_ref] - w.wcs.ctype = [r_dat['CTYPE1'], r_dat['CTYPE2']] + w.wcs.ctype = [r_dat["CTYPE1"], r_dat["CTYPE2"]] sn_x = 30 sn_y = 30 @@ -331,31 +358,53 @@ def WCS_def(xlen=9216, ylen=9232, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=60, y_pixs[i_pix] = y pix_coor = np.array([x, y]) sc1 = calcaluteSLSRotSkyCoor( - pix_xy=pix_coor, rot_angle=rotate_chip, xlen=xlen, ylen=ylen, w=w) + pix_xy=pix_coor, rot_angle=rotate_chip, xlen=xlen, ylen=ylen, w=w + ) sky_coors.append((sc1[0, 0], sc1[0, 1])) - wcs_new = fit_wcs_from_points(xy=np.array([x_pixs, y_pixs]), - world_coords=SkyCoord( - sky_coors, frame="icrs", unit="deg"), - projection='TAN') - r_dat['CD1_1'] = wcs_new.wcs.cd[0, 0] - r_dat['CD1_2'] = wcs_new.wcs.cd[0, 1] - r_dat['CD2_1'] = wcs_new.wcs.cd[1, 0] - r_dat['CD2_2'] = wcs_new.wcs.cd[1, 1] - r_dat['CRPIX1'] = wcs_new.wcs.crpix[0] - r_dat['CRPIX2'] = wcs_new.wcs.crpix[1] - - r_dat['CRVAL1'] = wcs_new.wcs.crval[0] - r_dat['CRVAL2'] = wcs_new.wcs.crval[1] + wcs_new = fit_wcs_from_points( + xy=np.array([x_pixs, y_pixs]), + world_coords=SkyCoord(sky_coors, frame="icrs", unit="deg"), + projection="TAN", + ) + r_dat["CD1_1"] = wcs_new.wcs.cd[0, 0] + r_dat["CD1_2"] = wcs_new.wcs.cd[0, 1] + r_dat["CD2_1"] = wcs_new.wcs.cd[1, 0] + r_dat["CD2_2"] = wcs_new.wcs.cd[1, 1] + r_dat["CRPIX1"] = wcs_new.wcs.crpix[0] + r_dat["CRPIX2"] = wcs_new.wcs.crpix[1] + + r_dat["CRVAL1"] = wcs_new.wcs.crval[0] + r_dat["CRVAL2"] = wcs_new.wcs.crval[1] else: raise ValueError( - 'In function WCS_def(): Either (row_num, col_num) or (xcen, ycen, pixel_size) should be given') + "In function WCS_def(): Either (row_num, col_num) or (xcen, ycen, pixel_size) should be given" + ) return r_dat # TODO project_cycle is temporary， is not in header defined, delete in future -def generatePrimaryHeader(xlen=9216, ylen=9232, pointing_id='00000001', pointing_type_code='101', ra=60, dec=-40, pixel_scale=0.074, time_pt=None, im_type='SCI', exptime=150., sat_pos=[0., 0., 0.], sat_vel=[0., 0., 0.], pa=23.5, project_cycle=6, run_counter=0, chip_name="01", obstype='WIDE', dataset='csst-msc-c9-25sqdeg-v3'): +def generatePrimaryHeader( + xlen=9216, + ylen=9232, + pointing_id="00000001", + pointing_type_code="101", + ra=60, + dec=-40, + pixel_scale=0.074, + time_pt=None, + im_type="SCI", + exptime=150.0, + sat_pos=[0.0, 0.0, 0.0], + sat_vel=[0.0, 0.0, 0.0], + pa=23.5, + project_cycle=6, + run_counter=0, + chip_name="01", + obstype="WIDE", + dataset="csst-msc-c9-25sqdeg-v3", +): # array_size1, array_size2, flux, sigma = int(argv[1]), int(argv[2]), 1000.0, 5.0 @@ -365,18 +414,16 @@ def generatePrimaryHeader(xlen=9216, ylen=9232, pointing_id='00000001', pointing datetime_obs = datetime.utcfromtimestamp(time_pt) datetime_obs = datetime_obs.replace(tzinfo=timezone.utc) # print(datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]) - datetime_obs = datetime.utcfromtimestamp( - np.round(datetime_obs.timestamp(), 1)) + datetime_obs = datetime.utcfromtimestamp(np.round(datetime_obs.timestamp(), 1)) # print(datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]) # date_obs = datetime_obs.strftime("%y%m%d") # time_obs = datetime_obs.strftime("%H%M%S%f")[:-5] - g_header_fn = os.path.split(os.path.realpath(__file__))[ - 0] + '/csst_msc_l0_ms.fits' - f = open(os.path.split(os.path.realpath(__file__))[0] + '/filter.lst') + g_header_fn = os.path.split(os.path.realpath(__file__))[0] + "/csst_msc_l0_ms.fits" + f = open(os.path.split(os.path.realpath(__file__))[0] + "/filter.lst") s = f.readline() s = s.strip("\n") - filters = s.split(' ') + filters = s.split(" ") s = f.readline() s = s.strip("\n") filterID = s.split() @@ -402,15 +449,15 @@ def generatePrimaryHeader(xlen=9216, ylen=9232, pointing_id='00000001', pointing # h_prim['PIXSIZE1'] = xlen # h_prim['PIXSIZE2'] = ylen - h_prim['DATE'] = datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] - h_prim['DATE-OBS'] = datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + h_prim["DATE"] = datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + h_prim["DATE-OBS"] = datetime_obs.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] # h_prim['DATE'] = '20'+date[0:2]+'-' + date[2:4]+'-'+date[4:6] + 'T' + time_obs[0:2]+':'+time_obs[2:4]+':'+time_obs[4:6] # h_prim['TIME'] = time_obs[0:2]+':'+time_obs[2:4]+':'+time_obs[4:6] # h_prim['DATE-OBS'] = '20'+date[0:2]+'-' + date[2:4]+'-'+date[4:6] + 'T' + time_obs[0:2]+':'+time_obs[2:4]+':'+time_obs[4:6] # h_prim['TIME-OBS'] = time_obs[0:2]+':'+time_obs[2:4]+':'+time_obs[4:6] # h_prim['DETECTOR'] = 'CHIP'+CCDID[k-1].rjust(2,'0') - h_prim['RA_OBJ'] = ra - h_prim['DEC_OBJ'] = dec + h_prim["RA_OBJ"] = ra + h_prim["DEC_OBJ"] = dec # obs_type = {'SCI': '01', 'BIAS': '03', 'DARK': '07', 'FLAT': '11', 'CRS': '98', 'CRD': '99'} @@ -418,96 +465,146 @@ def generatePrimaryHeader(xlen=9216, ylen=9232, pointing_id='00000001', pointing # OBS_id = '1'+ obs_type[im_type] + str(int(project_cycle)) + str(int(run_counter)).rjust(2, '0') + pointNum.rjust(5,'0') # OBS_id = get_obs_id(img_type=im_type, project_cycle='project_cycle', run_counter=run_counter, # pointing_id=pointing_id, pointing_type_code=pointing_type_code) - OBS_id = pointing_type_code+pointing_id + OBS_id = pointing_type_code + pointing_id # h_prim['OBJECT'] = str(int(project_cycle)) + pointNum.rjust(7, '0') - h_prim['OBJECT'] = pointing_id - h_prim['OBSID'] = OBS_id + h_prim["OBJECT"] = pointing_id + h_prim["OBSID"] = OBS_id # h_prim['TELFOCUS'] = 'f/14' - h_prim['EXPTIME'] = exptime + h_prim["EXPTIME"] = exptime # # Define file types # file_type = {'SCI':'SCIE', 'BIAS':'BIAS', 'DARK':'DARK', 'FLAT':'FLAT', 'CRS':'CRS', 'CRD':'CRD','CALS':'CALS','CALF':'CALF'} # h_prim['FILETYPE'] = file_type[im_type] # h_prim['FILETYPE'] = get_file_type(img_type=im_type) # h_prim['FILETYPE'] = im_type - h_prim['OBSTYPE'] = obstype - h_prim['DATASET'] = dataset - co = coord.SkyCoord(ra, dec, unit='deg') - - ra_hms = format(co.ra.hms.h, '02.0f') + format(co.ra.hms.m, - '02.0f') + format(co.ra.hms.s, '04.1f') - dec_hms = format(co.dec.dms.d, '02.0f') + format(abs(co.dec.dms.m), - '02.0f') + format(abs(co.dec.dms.s), '02.0f') + h_prim["OBSTYPE"] = obstype + h_prim["DATASET"] = dataset + co = coord.SkyCoord(ra, dec, unit="deg") + + ra_hms = ( + format(co.ra.hms.h, "02.0f") + + format(co.ra.hms.m, "02.0f") + + format(co.ra.hms.s, "04.1f") + ) + dec_hms = ( + format(co.dec.dms.d, "02.0f") + + format(abs(co.dec.dms.m), "02.0f") + + format(abs(co.dec.dms.s), "02.0f") + ) if dec >= 0: - h_prim['TARGET'] = ra_hms + '+' + dec_hms + h_prim["TARGET"] = ra_hms + "+" + dec_hms else: - h_prim['TARGET'] = ra_hms + dec_hms + h_prim["TARGET"] = ra_hms + dec_hms # # h_prim['RA_NOM'] = ra_hms # h_prim['DEC_NOM'] = dec_hms - h_prim['RA_PNT0'] = ra - h_prim['DEC_PNT0'] = dec - h_prim['RA_PNT1'] = ra - h_prim['DEC_PNT1'] = dec + h_prim["RA_PNT0"] = ra + h_prim["DEC_PNT0"] = dec + h_prim["RA_PNT1"] = ra + h_prim["DEC_PNT1"] = dec # h_prim['PIXSCAL1'] = pixel_scale # h_prim['PIXSCAL2'] = pixel_scale - ttt = h_prim['DATE'] + ttt = h_prim["DATE"] tstart = Time(ttt) - h_prim['EXPSTART'] = round(tstart.mjd, 5) - h_prim['CABSTART'] = h_prim['EXPSTART'] + h_prim["EXPSTART"] = round(tstart.mjd, 5) + h_prim["CABSTART"] = h_prim["EXPSTART"] # tend = Time(tstart.cxcsec + h_prim['EXPTIME'], format="cxcsec") - tend = Time(tstart.mjd + h_prim['EXPTIME']/86400., format="mjd") - h_prim['EXPEND'] = round(tend.mjd, 5) - h_prim['CABEND'] = h_prim['EXPEND'] + tend = Time(tstart.mjd + h_prim["EXPTIME"] / 86400.0, format="mjd") + h_prim["EXPEND"] = round(tend.mjd, 5) + h_prim["CABEND"] = h_prim["EXPEND"] # file_start_time = '20' + date[0:6] + time_obs[0:6] file_start_time = datetime_obs.strftime("%Y%m%d%H%M%S") end_time_str = str(tend.datetime) - file_end_time = end_time_str[0:4] + end_time_str[5:7]+end_time_str[8:10] + \ - end_time_str[11:13] + end_time_str[14:16] + end_time_str[17:19] + file_end_time = ( + end_time_str[0:4] + + end_time_str[5:7] + + end_time_str[8:10] + + end_time_str[11:13] + + end_time_str[14:16] + + end_time_str[17:19] + ) # h_prim['FILENAME'] = 'CSST_MSC_MS_' + im_type + '_' + file_start_time + '_' + file_end_time + '_' + OBS_id + '_' + CCDID[ # k - 1].rjust(2, '0') + '_L0_V01' - h_prim['FILENAME'] = 'CSST_MSC_MS_' + obstype + '_' + \ - file_start_time + '_' + file_end_time + \ - '_' + OBS_id + '_' + chip_name + '_L0_V01' - - h_prim['POS_ANG0'] = pa - h_prim['POSI0_X'] = sat_pos[0] - h_prim['POSI0_Y'] = sat_pos[1] - h_prim['POSI0_Z'] = sat_pos[2] - - h_prim['VELO0_X'] = sat_vel[0] - h_prim['VELO0_Y'] = sat_vel[1] - h_prim['VELO0_Z'] = sat_vel[2] + h_prim["FILENAME"] = ( + "CSST_MSC_MS_" + + obstype + + "_" + + file_start_time + + "_" + + file_end_time + + "_" + + OBS_id + + "_" + + chip_name + + "_L0_V01" + ) + + h_prim["POS_ANG0"] = pa + h_prim["POSI0_X"] = sat_pos[0] + h_prim["POSI0_Y"] = sat_pos[1] + h_prim["POSI0_Z"] = sat_pos[2] + + h_prim["VELO0_X"] = sat_vel[0] + h_prim["VELO0_Y"] = sat_vel[1] + h_prim["VELO0_Z"] = sat_vel[2] # h_prim['RA_PNT0'] = ra_hms # h_prim['DEC_PNT0'] = dec_hms # Get version of CSSTSim Package - from pkg_resources import get_distribution + from importlib.metadata import version + # h_prim['SIM_VER'] = (get_distribution("CSSTSim").version, "Version of CSST MSC simulation software") currentDateAndTime = datetime.now() compute_name = platform.node() - h_prim['FITSSWV'] = get_distribution( - "csst_msc_sim").version + '_' + currentDateAndTime.strftime("%Y%m%d") + '_' + compute_name - h_prim['EPOCH'] = round( - (Time(h_prim['EXPSTART'], format='mjd', scale='tcb')).jyear, 1) + h_prim["FITSSWV"] = ( + version("csst_msc_sim") + + "_" + + currentDateAndTime.strftime("%Y%m%d") + + "_" + + compute_name + ) + h_prim["EPOCH"] = round( + (Time(h_prim["EXPSTART"], format="mjd", scale="tcb")).jyear, 1 + ) return h_prim -def generateExtensionHeader(chip, xlen=9216, ylen=9232, ra=60, dec=-40, pa=-23.433, gain=1.0, readout=5.0, dark=0.02, saturation=90000, pixel_scale=0.074, pixel_size=1e-2, - extName='SCIE', row_num=None, col_num=None, xcen=None, ycen=None, timestamp=1621915200, exptime=150., readoutTime=40., t_shutter_open=1.3, t_shutter_close=1.3): - - e_header_fn = os.path.split(os.path.realpath(__file__))[ - 0] + '/csst_msc_l0_ms.fits' - f = open(os.path.split(os.path.realpath(__file__))[0] + '/filter.lst') +def generateExtensionHeader( + chip, + xlen=9216, + ylen=9232, + ra=60, + dec=-40, + pa=-23.433, + gain=1.0, + readout=5.0, + dark=0.02, + saturation=90000, + pixel_scale=0.074, + pixel_size=1e-2, + extName="SCIE", + row_num=None, + col_num=None, + xcen=None, + ycen=None, + timestamp=1621915200, + exptime=150.0, + readoutTime=40.0, + t_shutter_open=1.3, + t_shutter_close=1.3, +): + + e_header_fn = os.path.split(os.path.realpath(__file__))[0] + "/csst_msc_l0_ms.fits" + f = open(os.path.split(os.path.realpath(__file__))[0] + "/filter.lst") s = f.readline() s = s.strip("\n") - filters = s.split(' ') + filters = s.split(" ") s = f.readline() s = s.strip("\n") filterID = s.split() @@ -534,104 +631,162 @@ def generateExtensionHeader(chip, xlen=9216, ylen=9232, ra=60, dec=-40, pa=-23.4 # h_ext['CCDCHIP'] = CCDID[k - 1].rjust(2, '0') # h_ext['CCDLABEL'] = filters[k-1] + '-' + filterID[k-1] # h_ext['FILTER'] = filters[k-1] - h_ext['DETECTOR'] = str(chip.chipID).rjust(2, '0') - h_ext['DETLABEL'] = chip.chip_name - h_ext['FILTER'] = chip.filter_type - h_ext['NAXIS1'] = xlen - h_ext['NAXIS2'] = ylen - h_ext['EXTNAME'] = extName - h_ext['GAIN01'] = chip.gain_channel[0] - h_ext['GAIN02'] = chip.gain_channel[1] - h_ext['GAIN03'] = chip.gain_channel[2] - h_ext['GAIN04'] = chip.gain_channel[3] - h_ext['GAIN05'] = chip.gain_channel[4] - h_ext['GAIN06'] = chip.gain_channel[5] - h_ext['GAIN07'] = chip.gain_channel[6] - h_ext['GAIN08'] = chip.gain_channel[7] - h_ext['GAIN09'] = chip.gain_channel[8] - h_ext['GAIN10'] = chip.gain_channel[9] - h_ext['GAIN11'] = chip.gain_channel[10] - h_ext['GAIN12'] = chip.gain_channel[11] - h_ext['GAIN13'] = chip.gain_channel[12] - h_ext['GAIN14'] = chip.gain_channel[13] - h_ext['GAIN15'] = chip.gain_channel[14] - h_ext['GAIN16'] = chip.gain_channel[15] - h_ext['PSCAN1'] = chip.prescan_x - h_ext['PSCAN2'] = chip.prescan_y - h_ext['OSCAN1'] = chip.overscan_x - h_ext['OSCAN2'] = chip.overscan_y - h_ext['RON01'] = readout - h_ext['RON02'] = readout - h_ext['RON03'] = readout - h_ext['RON04'] = readout - h_ext['RON05'] = readout - h_ext['RON06'] = readout - h_ext['RON07'] = readout - h_ext['RON08'] = readout - h_ext['RON09'] = readout - h_ext['RON10'] = readout - h_ext['RON11'] = readout - h_ext['RON12'] = readout - h_ext['RON13'] = readout - h_ext['RON14'] = readout - h_ext['RON15'] = readout - h_ext['RON16'] = readout - - h_ext['PIXSCAL1'] = pixel_scale - h_ext['PIXSCAL2'] = pixel_scale - h_ext['EXPTIME'] = exptime - h_ext['DARKTIME'] = exptime + h_ext["DETECTOR"] = str(chip.chipID).rjust(2, "0") + h_ext["DETLABEL"] = chip.chip_name + h_ext["FILTER"] = chip.filter_type + h_ext["NAXIS1"] = xlen + h_ext["NAXIS2"] = ylen + h_ext["EXTNAME"] = extName + h_ext["GAIN01"] = chip.gain_channel[0] + h_ext["GAIN02"] = chip.gain_channel[1] + h_ext["GAIN03"] = chip.gain_channel[2] + h_ext["GAIN04"] = chip.gain_channel[3] + h_ext["GAIN05"] = chip.gain_channel[4] + h_ext["GAIN06"] = chip.gain_channel[5] + h_ext["GAIN07"] = chip.gain_channel[6] + h_ext["GAIN08"] = chip.gain_channel[7] + h_ext["GAIN09"] = chip.gain_channel[8] + h_ext["GAIN10"] = chip.gain_channel[9] + h_ext["GAIN11"] = chip.gain_channel[10] + h_ext["GAIN12"] = chip.gain_channel[11] + h_ext["GAIN13"] = chip.gain_channel[12] + h_ext["GAIN14"] = chip.gain_channel[13] + h_ext["GAIN15"] = chip.gain_channel[14] + h_ext["GAIN16"] = chip.gain_channel[15] + h_ext["PSCAN1"] = chip.prescan_x + h_ext["PSCAN2"] = chip.prescan_y + h_ext["OSCAN1"] = chip.overscan_x + h_ext["OSCAN2"] = chip.overscan_y + h_ext["RON01"] = readout + h_ext["RON02"] = readout + h_ext["RON03"] = readout + h_ext["RON04"] = readout + h_ext["RON05"] = readout + h_ext["RON06"] = readout + h_ext["RON07"] = readout + h_ext["RON08"] = readout + h_ext["RON09"] = readout + h_ext["RON10"] = readout + h_ext["RON11"] = readout + h_ext["RON12"] = readout + h_ext["RON13"] = readout + h_ext["RON14"] = readout + h_ext["RON15"] = readout + h_ext["RON16"] = readout + + h_ext["PIXSCAL1"] = pixel_scale + h_ext["PIXSCAL2"] = pixel_scale + h_ext["EXPTIME"] = exptime + h_ext["DARKTIME"] = exptime datetime_obs = datetime.utcfromtimestamp(timestamp) datetime_obs = datetime_obs.replace(tzinfo=timezone.utc) tstart = Time(datetime_obs) t_shutter_os = tstart - t_shutter_oe = Time(tstart.mjd + t_shutter_open / 86400., format="mjd") - t_shutter_co = Time(tstart.mjd + exptime / 86400., format="mjd") + t_shutter_oe = Time(tstart.mjd + t_shutter_open / 86400.0, format="mjd") + t_shutter_co = Time(tstart.mjd + exptime / 86400.0, format="mjd") t_shutter_ce = Time( - tstart.mjd + (exptime + t_shutter_close) / 86400., format="mjd") - t_shutter_os1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_shutter_os.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['SHTOPEN0'] = t_shutter_os1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] - t_shutter_oe1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_shutter_oe.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['SHTOPEN1'] = t_shutter_oe1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] - t_shutter_co1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_shutter_co.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['SHTCLOS0'] = t_shutter_co1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] - t_shutter_ce1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_shutter_ce.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['SHTCLOS1'] = t_shutter_ce1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] - - tstart_read = Time(tstart.mjd + exptime / 86400., format="mjd") - tend_read = Time(tstart.mjd + (exptime + readoutTime) / - 86400., format="mjd") + tstart.mjd + (exptime + t_shutter_close) / 86400.0, format="mjd" + ) + t_shutter_os1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_shutter_os.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["SHTOPEN0"] = t_shutter_os1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + t_shutter_oe1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_shutter_oe.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["SHTOPEN1"] = t_shutter_oe1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + t_shutter_co1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_shutter_co.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["SHTCLOS0"] = t_shutter_co1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + t_shutter_ce1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_shutter_ce.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["SHTCLOS1"] = t_shutter_ce1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + + tstart_read = Time(tstart.mjd + exptime / 86400.0, format="mjd") + tend_read = Time(tstart.mjd + (exptime + readoutTime) / 86400.0, format="mjd") # tstart1=tstart.datetime.replace(microsecond=round(tstart.datetime.microsecond, -5)) - tstart1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - tstart_read.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['ROTIME0'] = tstart1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + tstart1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(tstart_read.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["ROTIME0"] = tstart1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] # tend_read1 = tend_read.datetime.replace(microsecond=round(tend_read.datetime.microsecond, -5)) - tend_read1 = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - tend_read.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - h_ext['ROTIME1'] = tend_read1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] + tend_read1 = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(tend_read.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + h_ext["ROTIME1"] = tend_read1.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5] # h_ext['POS_ANG'] = pa - header_wcs = WCS_def(xlen=xlen, ylen=ylen, gapy=898.0, gapx1=534, gapx2=1309, ra_ref=ra, dec_ref=dec, pa=pa, pixel_scale=pixel_scale, pixel_size=pixel_size, - rotate_chip=chip.rotate_angle, filter=h_ext['FILTER'], row_num=row_num, col_num=col_num, xcen=xcen, ycen=ycen) - - h_ext['CRPIX1'] = header_wcs['CRPIX1'] - h_ext['CRPIX2'] = header_wcs['CRPIX2'] - h_ext['CRVAL1'] = header_wcs['CRVAL1'] - h_ext['CRVAL2'] = header_wcs['CRVAL2'] - h_ext['CD1_1'] = header_wcs['CD1_1'] - h_ext['CD1_2'] = header_wcs['CD1_2'] - h_ext['CD2_1'] = header_wcs['CD2_1'] - h_ext['CD2_2'] = header_wcs['CD2_2'] + header_wcs = WCS_def( + xlen=xlen, + ylen=ylen, + gapy=898.0, + gapx1=534, + gapx2=1309, + ra_ref=ra, + dec_ref=dec, + pa=pa, + pixel_scale=pixel_scale, + pixel_size=pixel_size, + rotate_chip=chip.rotate_angle, + filter=h_ext["FILTER"], + row_num=row_num, + col_num=col_num, + xcen=xcen, + ycen=ycen, + ) + + h_ext["CRPIX1"] = header_wcs["CRPIX1"] + h_ext["CRPIX2"] = header_wcs["CRPIX2"] + h_ext["CRVAL1"] = header_wcs["CRVAL1"] + h_ext["CRVAL2"] = header_wcs["CRVAL2"] + h_ext["CD1_1"] = header_wcs["CD1_1"] + h_ext["CD1_2"] = header_wcs["CD1_2"] + h_ext["CD2_1"] = header_wcs["CD2_1"] + h_ext["CD2_2"] = header_wcs["CD2_2"] # h_ext['EQUINOX'] = header_wcs['EQUINOX'] # h_ext['WCSDIM'] = header_wcs['WCSDIM'] - h_ext['CTYPE1'] = header_wcs['CTYPE1'] - h_ext['CTYPE2'] = header_wcs['CTYPE2'] + h_ext["CTYPE1"] = header_wcs["CTYPE1"] + h_ext["CTYPE2"] = header_wcs["CTYPE2"] - h_ext['EXTNAME'] = 'IMAGE' + h_ext["EXTNAME"] = "IMAGE" h_ext.comments["XTENSION"] = "image extension" return h_ext @@ -653,17 +808,38 @@ def main(argv): dark = float(argv[12]) fw = float(argv[13]) - h_prim = generatePrimaryHeader(xlen=xlen, ylen=ylen, ra=ra, dec=dec, psize=pSize, - row_num=ccd_row_num, col_num=ccd_col_num, pointNum=pointingNum) - - h_ext = generateExtensionHeader(xlen=xlen, ylen=ylen, ra=ra, dec=dec, pa=pa_aper, gain=gain, - readout=readout, dark=dark, saturation=fw, psize=pSize, row_num=ccd_row_num, col_num=ccd_col_num) + h_prim = generatePrimaryHeader( + xlen=xlen, + ylen=ylen, + ra=ra, + dec=dec, + psize=pSize, + row_num=ccd_row_num, + col_num=ccd_col_num, + pointNum=pointingNum, + ) + + h_ext = generateExtensionHeader( + xlen=xlen, + ylen=ylen, + ra=ra, + dec=dec, + pa=pa_aper, + gain=gain, + readout=readout, + dark=dark, + saturation=fw, + psize=pSize, + row_num=ccd_row_num, + col_num=ccd_col_num, + ) hdu1 = fits.PrimaryHDU(header=h_prim) hdu2 = fits.ImageHDU(np.zeros([ylen, xlen]), header=h_ext) hdul = fits.HDUList([hdu1, hdu2]) - hdul.writeto(h_prim['FILENAME']+'.fits', output_verify='ignore') + hdul.writeto(h_prim["FILENAME"] + ".fits", output_verify="ignore") + # if __name__ == "__main__": # main(sys.argv) diff --git a/observation_sim/instruments/Filter.py b/observation_sim/instruments/Filter.py index 9e5083d8a7ff3ed38dcb0a18a4da6756cbf460c0..6f4c00aab6f7d9972ce0023adfcdb0c4edee7973 100755 --- a/observation_sim/instruments/Filter.py +++ b/observation_sim/instruments/Filter.py @@ -1,11 +1,8 @@ import galsim -import pylab as pl -import os import numpy as np import gc import observation_sim.instruments._util as _util -from observation_sim.instruments.FilterParam import FilterParam from observation_sim.sky_background import Straylight try: @@ -49,67 +46,81 @@ class Filter(object): def is_too_dim(self, mag, margin=1.0): return mag >= self.mag_limiting + margin - def _get_bandpasses(self, filter_dir=None, unit='A'): + def _get_bandpasses(self, filter_dir=None, unit="A"): if self.filter_id < 7: # Photometric try: - with pkg_resources.files('observation_sim.instruments.data.filters').joinpath(self.filter_type.lower() + '.txt') as filter_file: + with pkg_resources.files( + "observation_sim.instruments.data.filters" + ).joinpath(self.filter_type.lower() + ".txt") as filter_file: self.filter_bandpass = galsim.Bandpass( - str(filter_file), wave_type=unit) + str(filter_file), wave_type=unit + ) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.filters', self.filter_type.lower() + '.txt') as filter_file: + with pkg_resources.path( + "observation_sim.instruments.data.filters", + self.filter_type.lower() + ".txt", + ) as filter_file: self.filter_bandpass = galsim.Bandpass( - str(filter_file), wave_type=unit) + str(filter_file), wave_type=unit + ) try: - with pkg_resources.files('observation_sim.instruments.data.throughputs').joinpath(self.filter_type.lower() + '_throughput.txt') as filter_file: - bandpass_full = galsim.Bandpass( - str(filter_file), wave_type=unit) + with pkg_resources.files( + "observation_sim.instruments.data.throughputs" + ).joinpath(self.filter_type.lower() + "_throughput.txt") as filter_file: + bandpass_full = galsim.Bandpass(str(filter_file), wave_type=unit) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.throughputs', self.filter_type.lower() + '_throughput.txt') as filter_file: - bandpass_full = galsim.Bandpass( - str(filter_file), wave_type=unit) + with pkg_resources.path( + "observation_sim.instruments.data.throughputs", + self.filter_type.lower() + "_throughput.txt", + ) as filter_file: + bandpass_full = galsim.Bandpass(str(filter_file), wave_type=unit) # bandpass_full = bandpass_full * self.ccd_bandpass # Get sub-bandpasses bandpass_sub_list = [] try: - with pkg_resources.files('observation_sim.instruments.data.filters').joinpath(self.filter_type.lower() + "_sub.list") as wave_bin_file: + with pkg_resources.files( + "observation_sim.instruments.data.filters" + ).joinpath(self.filter_type.lower() + "_sub.list") as wave_bin_file: wave_points = open(wave_bin_file).read().splitlines() except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.filters', self.filter_type.lower() + "_sub.list") as wave_bin_file: + with pkg_resources.path( + "observation_sim.instruments.data.filters", + self.filter_type.lower() + "_sub.list", + ) as wave_bin_file: wave_points = open(wave_bin_file).read().splitlines() for i in range(2, len(wave_points), 2): - blim = max(float(wave_points[i-2]) - * 0.1, bandpass_full.blue_limit) - rlim = min(float(wave_points[i])*0.1, bandpass_full.red_limit) + blim = max(float(wave_points[i - 2]) * 0.1, bandpass_full.blue_limit) + rlim = min(float(wave_points[i]) * 0.1, bandpass_full.red_limit) if blim >= rlim: continue - bandpass = bandpass_full.truncate( - blue_limit=blim, red_limit=rlim) + bandpass = bandpass_full.truncate(blue_limit=blim, red_limit=rlim) bandpass_sub_list.append(bandpass) # print("num of sub-bandpasses for filter#%d(%s) = "%(self.filter_id, self.filter_type), len(bandpass_sub_list), flush=True) - else: # Spectroscopic + else: # Spectroscopic sls_lamb = np.linspace(self.blue_limit, self.red_limit, 100) sls_flux = np.ones_like(sls_lamb) - con_spec = galsim.LookupTable( - sls_lamb, sls_flux, interpolant='nearest') + con_spec = galsim.LookupTable(sls_lamb, sls_flux, interpolant="nearest") bandpass_full = galsim.Bandpass(con_spec, wave_type=unit) bandpass_sub_list = [] try: - with pkg_resources.files('observation_sim.instruments.data.filters').joinpath(self.filter_type.lower() + "_sub.list") as wave_bin_file: + with pkg_resources.files( + "observation_sim.instruments.data.filters" + ).joinpath(self.filter_type.lower() + "_sub.list") as wave_bin_file: wave_points = open(wave_bin_file).read().splitlines() except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.filters', self.filter_type.lower() + "_sub.list") as wave_bin_file: + with pkg_resources.path( + "observation_sim.instruments.data.filters", + self.filter_type.lower() + "_sub.list", + ) as wave_bin_file: wave_points = open(wave_bin_file).read().splitlines() for i in range(2, len(wave_points), 2): - blim = max(float(wave_points[i - 2]) - * 0.1, bandpass_full.blue_limit) - rlim = min(float(wave_points[i]) - * 0.1, bandpass_full.red_limit) + blim = max(float(wave_points[i - 2]) * 0.1, bandpass_full.blue_limit) + rlim = min(float(wave_points[i]) * 0.1, bandpass_full.red_limit) if blim >= rlim: continue - bandpass = bandpass_full.truncate( - blue_limit=blim, red_limit=rlim) + bandpass = bandpass_full.truncate(blue_limit=blim, red_limit=rlim) bandpass_sub_list.append(bandpass) # print("num of sub-bandpasses for filter#%d(%s) = " % (self.filter_id, self.filter_type), len(bandpass_sub_list), flush=True) @@ -118,22 +129,29 @@ class Filter(object): def getPhotonE(self): return _util.photonEnergy(self.effective_wavelength) - def getSkyNoise(self, exptime, gain=1.): + def getSkyNoise(self, exptime, gain=1.0): return self.sky_background * exptime / gain - def setFilterStrayLightPixel(self, jtime=2460843., sat_pos=np.array([0, 0, 0]), pointing_radec=np.array([0, 0]), sun_pos=np.array([0, 0, 0])): - sl = Straylight(jtime=jtime, sat_pos=sat_pos, - pointing_radec=pointing_radec, sun_pos=sun_pos) + def setFilterStrayLightPixel( + self, + jtime=2460843.0, + sat_pos=np.array([0, 0, 0]), + pointing_radec=np.array([0, 0]), + sun_pos=np.array([0, 0, 0]), + ): + sl = Straylight( + jtime=jtime, sat_pos=sat_pos, pointing_radec=pointing_radec, sun_pos=sun_pos + ) if self.filter_type in _util.SPEC_FILTERS: s_pix, spec = sl.calculateStrayLightGrating( - grating=self.filter_type.upper()) + grating=self.filter_type.upper() + ) if s_pix > 0.8: s_pix = 0.8 self.sky_background = s_pix self.zodical_spec = spec elif self.filter_type.lower() in [x.lower for x in _util.PHOT_FILTERS]: - s_pix = sl.calculateStrayLightFilter( - filter=self.filter_type.lower()) + s_pix = sl.calculateStrayLightFilter(filter=self.filter_type.lower()) if s_pix > 1: s_pix = 1 self.sky_background = s_pix @@ -142,7 +160,14 @@ class Filter(object): del sl gc.collect() - def update_limit_saturation_mags(self, exptime=150., psf_fwhm=0.1969, skyFn='sky_emiss_hubble_50_50_A.dat', chip=None, full_depth_exptime=None): + def update_limit_saturation_mags( + self, + exptime=150.0, + psf_fwhm=0.1969, + skyFn="sky_emiss_hubble_50_50_A.dat", + chip=None, + full_depth_exptime=None, + ): if self.filter_type in _util.SPEC_FILTERS: return if chip is not None: @@ -155,13 +180,32 @@ class Filter(object): read_noise = 5.0 dark_noise = 0.02 full_well = 90000 - throughput_file = self.filter_type.lower() + '_throughput.txt' + throughput_file = self.filter_type.lower() + "_throughput.txt" self.mag_limiting, self.mag_saturation = _util.calculateLimitMag( - psf_fwhm=psf_fwhm, pixelSize=pix_scale, throughputFn=throughput_file, readout=5.0, skyFn=skyFn, darknoise=dark_noise, exTime=exptime, fw=full_well) + psf_fwhm=psf_fwhm, + pixelSize=pix_scale, + throughputFn=throughput_file, + readout=5.0, + skyFn=skyFn, + darknoise=dark_noise, + exTime=exptime, + fw=full_well, + ) if full_depth_exptime is not None: - self.mag_limiting, _ = _util.calculateLimitMag(psf_fwhm=psf_fwhm, pixelSize=pix_scale, throughputFn=throughput_file, readout=5.0, skyFn=skyFn, darknoise=dark_noise, exTime=full_depth_exptime, fw=full_well) - - print("for filter %s: mag_limiting: %.3f, mag_saturation: %.3f" % - (self.filter_type, self.mag_limiting, self.mag_saturation)) + self.mag_limiting, _ = _util.calculateLimitMag( + psf_fwhm=psf_fwhm, + pixelSize=pix_scale, + throughputFn=throughput_file, + readout=5.0, + skyFn=skyFn, + darknoise=dark_noise, + exTime=full_depth_exptime, + fw=full_well, + ) + + print( + "for filter %s: mag_limiting: %.3f, mag_saturation: %.3f" + % (self.filter_type, self.mag_limiting, self.mag_saturation) + ) diff --git a/observation_sim/instruments/FilterParam.py b/observation_sim/instruments/FilterParam.py index ac433f36e0984dbe185497b2a72f60b7c65bbbee..350cbc825a6e4b18df2f80370cff44d4faf9f7e9 100755 --- a/observation_sim/instruments/FilterParam.py +++ b/observation_sim/instruments/FilterParam.py @@ -27,19 +27,18 @@ class FilterParam(object): # 8) dim end magnitude if filter_param is None: filtP = { - "NUV": [2867.7, 705.4, 2470.0, 3270.0, 0.1404, 0.004, 15.7, 25.4], - "u": [3601.1, 852.1, 3120.0, 4090.0, 0.2176, 0.021, 16.1, 25.4], - "g": [4754.5, 1569.8, 3900.0, 5620.0, 0.4640, 0.164, 17.2, 26.3], - "r": [6199.8, 1481.2, 5370.0, 7030.0, 0.5040, 0.207, 17.0, 26.0], - "i": [7653.2, 1588.1, 6760.0, 8550.0, 0.4960, 0.212, 16.7, 25.9], - "z": [9600.6, 2490.5, 8240.0, 11000.0, 0.2000, 0.123, 15.7, 25.2], - "y": [10051.0, 1590.6, 9130.0, 11000.0, 0.0960, 0.037, 14.4, 24.4], + "NUV": [2867.7, 705.4, 2470.0, 3270.0, 0.1404, 0.004, 15.7, 25.4], + "u": [3601.1, 852.1, 3120.0, 4090.0, 0.2176, 0.021, 16.1, 25.4], + "g": [4754.5, 1569.8, 3900.0, 5620.0, 0.4640, 0.164, 17.2, 26.3], + "r": [6199.8, 1481.2, 5370.0, 7030.0, 0.5040, 0.207, 17.0, 26.0], + "i": [7653.2, 1588.1, 6760.0, 8550.0, 0.4960, 0.212, 16.7, 25.9], + "z": [9600.6, 2490.5, 8240.0, 11000.0, 0.2000, 0.123, 15.7, 25.2], + "y": [10051.0, 1590.6, 9130.0, 11000.0, 0.0960, 0.037, 14.4, 24.4], # [TODO] - "FGS": [5000.0, 8000.0, 3000.0, 11000.0, 0.6500, 0.164, 0., 30.], - - "GU": [0.0, 0.0, 2550.0, 10000.0, 1.0, 0.037, 14.0, 26.0], - "GV": [0.0, 0.0, 3500.0, 6500.0, 1.0, 0.037, 14.0, 26.0], - "GI": [0.0, 0.0, 5000.0, 10000.0, 1.0, 0.037, 14.0, 26.0], + "FGS": [5000.0, 8000.0, 3000.0, 11000.0, 0.6500, 0.164, 0.0, 30.0], + "GU": [0.0, 0.0, 2550.0, 10000.0, 1.0, 0.037, 14.0, 26.0], + "GV": [0.0, 0.0, 3500.0, 6500.0, 1.0, 0.037, 14.0, 26.0], + "GI": [0.0, 0.0, 5000.0, 10000.0, 1.0, 0.037, 14.0, 26.0], } else: filtP = filter_param @@ -64,13 +63,14 @@ class FilterParam(object): filtlist = filtdir + filtlistn filtn = open(filtlist).read().splitlines() for itype in filtype: - ifiltn = filtdir+itype+".dat" + ifiltn = filtdir + itype + ".dat" iblim = filtParams[itype][2] irlim = filtParams[itype][3] ifilt = galsim.Bandpass( - ifiltn, wave_type=unit, blue_limit=iblim, red_limit=irlim) + ifiltn, wave_type=unit, blue_limit=iblim, red_limit=irlim + ) wave = ifilt.wave_list ifilt = ifilt.func(wave) - filts[itype] = np.transpose(np.array([wave*10.0, ifilt])) + filts[itype] = np.transpose(np.array([wave * 10.0, ifilt])) return filtype, filts diff --git a/observation_sim/instruments/FocalPlane.py b/observation_sim/instruments/FocalPlane.py index 8987fcb91cdf1bbf08dd9bd5ede8956049ee61db..69a157bcf5a6a200ba439e28fc67ada2709fc36e 100755 --- a/observation_sim/instruments/FocalPlane.py +++ b/observation_sim/instruments/FocalPlane.py @@ -3,9 +3,8 @@ import numpy as np class FocalPlane(object): - def __init__(self, chip_list=None, survey_type='Photometric', bad_chips=None): - """Get the focal plane layout - """ + def __init__(self, chip_list=None, survey_type="Photometric", bad_chips=None): + """Get the focal plane layout""" self.nchips = 42 self.ignore_chips = [] @@ -18,17 +17,17 @@ class FocalPlane(object): if chip_list is not None: for i in range(42): - if not (i+1 in chip_list): - self.ignore_chips.append(i+1) - elif survey_type == 'Photometric': + if i + 1 not in chip_list: + self.ignore_chips.append(i + 1) + elif survey_type == "Photometric": for i in range(5): - self.ignore_chips.append(i+1) - self.ignore_chips.append(i+26) + self.ignore_chips.append(i + 1) + self.ignore_chips.append(i + 26) self.ignore_chips.append(10) self.ignore_chips.append(21) for i in range(31, 43): self.ignore_chips.append(i) - elif survey_type == 'Spectroscopic': + elif survey_type == "Spectroscopic": for i in range(6, 26): if i == 10 or i == 21: continue @@ -36,7 +35,7 @@ class FocalPlane(object): self.ignore_chips.append(i) for i in range(31, 43): self.ignore_chips.append(i) - elif survey_type == 'FGS': + elif survey_type == "FGS": for i in range(1, 31): self.ignore_chips.append(i) @@ -57,15 +56,14 @@ class FocalPlane(object): return str("0%d" % chipID)[-2:] def isBadChip(self, chipID): - """Check if chip #(chipID) on the focal plane is bad or not - """ - return (chipID in self.bad_chips) + """Check if chip #(chipID) on the focal plane is bad or not""" + return chipID in self.bad_chips def isIgnored(self, chipID): - return (chipID in self.ignore_chips) + return chipID in self.ignore_chips def getTanWCS(self, ra, dec, img_rot, pix_scale, xcen=None, ycen=None, logger=None): - """ Get the WCS of the image mosaic using Gnomonic/TAN projection + """Get the WCS of the image mosaic using Gnomonic/TAN projection Parameter: ra, dec: float @@ -79,7 +77,8 @@ class FocalPlane(object): """ if logger is not None: logger.info( - " Construct the wcs of the entire image mosaic using Gnomonic/TAN projection") + " Construct the wcs of the entire image mosaic using Gnomonic/TAN projection" + ) if (xcen is None) or (ycen is None): xcen = self.cen_pix_x ycen = self.cen_pix_y @@ -91,7 +90,8 @@ class FocalPlane(object): moscen = galsim.PositionD(x=xcen, y=ycen) sky_center = galsim.CelestialCoord( - ra=ra*galsim.degrees, dec=dec*galsim.degrees) + ra=ra * galsim.degrees, dec=dec * galsim.degrees + ) affine = galsim.AffineTransform(dudx, dudy, dvdx, dvdy, origin=moscen) WCS = galsim.TanWCS(affine, sky_center, units=galsim.arcsec) @@ -101,12 +101,12 @@ class FocalPlane(object): """ The sky coverage of an area """ - r2d = 180.0/np.pi + r2d = 180.0 / np.pi s1 = wcs.toWorld(galsim.PositionD(x0, y0)) s2 = wcs.toWorld(galsim.PositionD(x0, y1)) s3 = wcs.toWorld(galsim.PositionD(x1, y0)) s4 = wcs.toWorld(galsim.PositionD(x1, y1)) - ra = [s1.ra.rad*r2d, s2.ra.rad*r2d, s3.ra.rad*r2d, s4.ra.rad*r2d] - dec = [s1.dec.rad*r2d, s2.dec.rad*r2d, s3.dec.rad*r2d, s4.dec.rad*r2d] + ra = [s1.ra.rad * r2d, s2.ra.rad * r2d, s3.ra.rad * r2d, s4.ra.rad * r2d] + dec = [s1.dec.rad * r2d, s2.dec.rad * r2d, s3.dec.rad * r2d, s4.dec.rad * r2d] return galsim.BoundsD(min(ra), max(ra), min(dec), max(dec)) diff --git a/observation_sim/instruments/Telescope.py b/observation_sim/instruments/Telescope.py index d6c1d2fabc50ab3540026b6655e58f55ad546794..f2147ea35bc1b722f1fb76ddc8bba84905778278 100755 --- a/observation_sim/instruments/Telescope.py +++ b/observation_sim/instruments/Telescope.py @@ -12,26 +12,30 @@ class Telescope(object): self.diameter = 2.0 # in unit of meter if param is not None: self.diameter = param["diameter"] - self.pupil_area = np.pi * (0.5 * self.diameter)**2 + self.pupil_area = np.pi * (0.5 * self.diameter) ** 2 if optEffCurve_path is not None: self.efficiency = self._get_efficiency(optEffCurve_path) else: try: - with pkg_resources.files('observation_sim.instruments.data').joinpath('mirror_ccdnote.txt') as optEffCurve_path: + with pkg_resources.files("observation_sim.instruments.data").joinpath( + "mirror_ccdnote.txt" + ) as optEffCurve_path: self.efficiency = self._get_efficiency(optEffCurve_path) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data', 'mirror_ccdnote.txt') as optEffCurve_path: + with pkg_resources.path( + "observation_sim.instruments.data", "mirror_ccdnote.txt" + ) as optEffCurve_path: self.efficiency = self._get_efficiency(optEffCurve_path) def _get_efficiency(self, effCurve_path): - """ Read in the efficiency of optics + """Read in the efficiency of optics for each band Parameters: effCurve_path: the path for efficiency file Returns: opticsEff: a dictionary of efficiency (a scalar) for each band """ - f = open(effCurve_path, 'r') + f = open(effCurve_path, "r") for _ in range(2): header = f.readline() diff --git a/observation_sim/instruments/_util.py b/observation_sim/instruments/_util.py index 0947b0a83bab377e781fcebd90807ca47f2c4f87..38df9ecb465ada0e24227cf4dcb880cddb7b2b54 100755 --- a/observation_sim/instruments/_util.py +++ b/observation_sim/instruments/_util.py @@ -1,7 +1,7 @@ import numpy as np -import os import math -from pylab import * + +# from pylab import * from scipy import interpolate try: @@ -10,29 +10,29 @@ except ImportError: # Try backported to PY<37 'importlib_resources' import importlib_resources as pkg_resources -VC_A = 2.99792458e+18 # speed of light: A/s -VC_M = 2.99792458e+8 # speed of light: m/s +VC_A = 2.99792458e18 # speed of light: A/s +VC_M = 2.99792458e8 # speed of light: m/s H_PLANK = 6.626196e-27 # Plank constant: erg s ALL_FILTERS = ["NUV", "u", "g", "r", "i", "z", "y", "GU", "GV", "GI", "FGS"] -PHOT_FILTERS = ["NUV", "u", "g", 'r', 'i', 'z', 'y', 'FGS'] +PHOT_FILTERS = ["NUV", "u", "g", "r", "i", "z", "y", "FGS"] SPEC_FILTERS = ["GI", "GV", "GU"] def rotate_conterclockwise(x0, y0, x, y, angle): """ -Rotate a point counterclockwise by a given angle around a given origin. + Rotate a point counterclockwise by a given angle around a given origin. -The angle should be given in radians. -""" + The angle should be given in radians. + """ angle = np.deg2rad(angle) - qx = x0 + np.cos(angle)*(x - x0) - np.sin(angle) * (y - y0) - qy = y0 + np.sin(angle)*(x - x0) + np.cos(angle) * (y - y0) + qx = x0 + np.cos(angle) * (x - x0) - np.sin(angle) * (y - y0) + qy = y0 + np.sin(angle) * (x - x0) + np.cos(angle) * (y - y0) return qx, qy def photonEnergy(lambd): - """ The energy of photon at a given wavelength + """The energy of photon at a given wavelength Parameter: lambd: the wavelength in unit of Angstrom @@ -44,8 +44,20 @@ def photonEnergy(lambd): return eph -def calculateLimitMag(aperture=2.0, psf_fwhm=0.1969, pixelSize=0.074, pmRation=0.8, throughputFn='i_throughput.txt', readout=5.0, skyFn='sky_emiss_hubble_50_50_A.dat', darknoise=0.02, exTime=150, exNum=1, fw=90000): - ''' +def calculateLimitMag( + aperture=2.0, + psf_fwhm=0.1969, + pixelSize=0.074, + pmRation=0.8, + throughputFn="i_throughput.txt", + readout=5.0, + skyFn="sky_emiss_hubble_50_50_A.dat", + darknoise=0.02, + exTime=150, + exNum=1, + fw=90000, +): + """ description: param {*} aperture: unit m, default 2 m param {*} psf_fwhm: psf fwhm, default 0.1969" @@ -59,76 +71,109 @@ def calculateLimitMag(aperture=2.0, psf_fwhm=0.1969, pixelSize=0.074, pmRation=0 param {*} exNum: exposure number, defautl 1 param {*} fw, full well value( or saturation value),default 90000e-/pixel return {*} limit mag and saturation mag - ''' + """ try: - with pkg_resources.files('observation_sim.instruments.data.throughputs').joinpath(throughputFn) as data_file: + with pkg_resources.files( + "observation_sim.instruments.data.throughputs" + ).joinpath(throughputFn) as data_file: throughput_f = np.loadtxt(data_file) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.throughputs', throughputFn) as data_file: + with pkg_resources.path( + "observation_sim.instruments.data.throughputs", throughputFn + ) as data_file: throughput_f = np.loadtxt(data_file) thr_i = interpolate.interp1d( - throughput_f[:, 0]/10, throughput_f[:, 1]) # wavelength in anstrom + throughput_f[:, 0] / 10, throughput_f[:, 1] + ) # wavelength in anstrom f_s = 200 f_e = 1100 delt_f = 0.5 - data_num = int((f_e-f_s)/delt_f+1) + data_num = int((f_e - f_s) / delt_f + 1) eff = np.zeros([data_num, 2]) - eff[:, 0] = np.arange(f_s, f_e+delt_f, delt_f) + eff[:, 0] = np.arange(f_s, f_e + delt_f, delt_f) eff[:, 1] = thr_i(eff[:, 0]) - wave = np.arange(f_s, f_e+delt_f, delt_f) + wave = np.arange(f_s, f_e + delt_f, delt_f) wavey = np.ones(wave.shape[0]) try: - with pkg_resources.files('observation_sim.instruments.data.throughputs').joinpath(skyFn) as data_file: + with pkg_resources.files( + "observation_sim.instruments.data.throughputs" + ).joinpath(skyFn) as data_file: skydata = np.loadtxt(data_file) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.throughputs', skyFn) as data_file: + with pkg_resources.path( + "observation_sim.instruments.data.throughputs", skyFn + ) as data_file: skydata = np.loadtxt(data_file) - skydatai = interpolate.interp1d(skydata[:, 0]/10, skydata[:, 1]*10) + skydatai = interpolate.interp1d(skydata[:, 0] / 10, skydata[:, 1] * 10) sky_data = np.zeros([data_num, 2]) - sky_data[:, 0] = np.arange(f_s, f_e+delt_f, delt_f) + sky_data[:, 0] = np.arange(f_s, f_e + delt_f, delt_f) sky_data[:, 1] = skydatai(sky_data[:, 0]) - flux_sky = trapz((sky_data[:, 1])*eff[:, 1], sky_data[:, 0]) - skyPix = flux_sky*pixelSize*pixelSize*pi*(aperture*aperture/4) + flux_sky = np.trapz((sky_data[:, 1]) * eff[:, 1], sky_data[:, 0]) + skyPix = flux_sky * pixelSize * pixelSize * math.pi * (aperture * aperture / 4) # limit mag - r_pix = psf_fwhm*0.7618080243778568/pixelSize # radius RE80, pixel + r_pix = psf_fwhm * 0.7618080243778568 / pixelSize # radius RE80, pixel cnum = math.pi * r_pix * r_pix sn = 5 - d = skyPix*exTime*exNum*cnum + darknoise * \ - exTime*exNum*cnum+readout*readout*cnum*exNum + d = ( + skyPix * exTime * exNum * cnum + + darknoise * exTime * exNum * cnum + + readout * readout * cnum * exNum + ) a = 1 - b = -sn*sn - c = -sn*sn*d - - flux = (-b+sqrt(b*b-4*a*c))/(2*a)/pmRation - limitMag = -2.5*log10(flux/(54799275581.04437 * trapz(wavey * - eff[:, 1]/wave, wave, 0.1)*exTime*exNum*pi*(aperture/2)*(aperture/2))) + b = -sn * sn + c = -sn * sn * d + + flux = (-b + np.sqrt(b * b - 4 * a * c)) / (2 * a) / pmRation + limitMag = -2.5 * np.log10( + flux + / ( + 54799275581.04437 + * np.trapz(wavey * eff[:, 1] / wave, wave, 0.1) + * exTime + * exNum + * math.pi + * (aperture / 2) + * (aperture / 2) + ) + ) # saturation mag from astropy.modeling.models import Gaussian2D - m_size = int(20 * psf_fwhm/pixelSize) + + m_size = int(20 * psf_fwhm / pixelSize) if m_size % 2 == 0: m_size + 1 - m_cen = m_size//2 - psf_sigma = psf_fwhm/2.355/pixelSize + m_cen = m_size // 2 + psf_sigma = psf_fwhm / 2.355 / pixelSize gaussShape = Gaussian2D(1, m_cen, m_cen, psf_sigma, psf_sigma) yp, xp = np.mgrid[0:m_size, 0:m_size] psfMap = gaussShape(xp, yp) - maxRatio = np.amax(psfMap)/np.sum(psfMap) + maxRatio = np.amax(psfMap) / np.sum(psfMap) # print(maxRatio) - flux_sat = fw/maxRatio*exNum - satMag = -2.5*log10(flux_sat/(54799275581.04437 * trapz(wavey * - eff[:, 1]/wave, wave, 0.1)*exTime*exNum*pi*(aperture/2)*(aperture/2))) + flux_sat = fw / maxRatio * exNum + satMag = -2.5 * np.log10( + flux_sat + / ( + 54799275581.04437 + * np.trapz(wavey * eff[:, 1] / wave, wave, 0.1) + * exTime + * exNum + * math.pi + * (aperture / 2) + * (aperture / 2) + ) + ) return limitMag, satMag diff --git a/observation_sim/instruments/chip/Chip.py b/observation_sim/instruments/chip/Chip.py index b995da41f818eba4c47659e57afc2e0e16d8b7f2..14ad3da4a8045c338c33b4d1e58318fe7c863bb1 100755 --- a/observation_sim/instruments/chip/Chip.py +++ b/observation_sim/instruments/chip/Chip.py @@ -1,20 +1,12 @@ import galsim -import os import numpy as np import pickle import json from astropy.table import Table -from numpy.random import Generator, PCG64 -from astropy.io import fits -from datetime import datetime import observation_sim.instruments._util as _util -from observation_sim.instruments.chip import effects from observation_sim.instruments.FocalPlane import FocalPlane -from observation_sim.config.header import generatePrimaryHeader, generateExtensionHeader -from observation_sim.instruments._util import rotate_conterclockwise from observation_sim.instruments.chip import chip_utils -from observation_sim.instruments.chip.libCTI.CTI_modeling import CTI_sim try: import importlib.resources as pkg_resources @@ -24,7 +16,16 @@ except ImportError: class Chip(FocalPlane): - def __init__(self, chipID, ccdEffCurve_dir=None, CRdata_dir=None, sls_dir=None, config=None, treering_func=None, logger=None): + def __init__( + self, + chipID, + ccdEffCurve_dir=None, + CRdata_dir=None, + sls_dir=None, + config=None, + treering_func=None, + logger=None, + ): # Get focal plane (instance of paraent class) info super().__init__() self.nsecy = 2 @@ -36,7 +37,7 @@ class Chip(FocalPlane): # A chip ID must be assigned self.chipID = int(chipID) - self.chip_name = str(chipID).rjust(2, '0') + self.chip_name = str(chipID).rjust(2, "0") # Get corresponding filter info self.filter_id, self.filter_type = self.getChipFilter() @@ -47,11 +48,15 @@ class Chip(FocalPlane): # Set the relavent specs for detectors try: - with pkg_resources.files('observation_sim.instruments.data.ccd').joinpath("chip_definition.json") as chip_definition: + with pkg_resources.files("observation_sim.instruments.data.ccd").joinpath( + "chip_definition.json" + ) as chip_definition: with open(chip_definition, "r") as f: chip_dict = json.load(f)[str(self.chipID)] except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.ccd', "chip_definition.json") as chip_definition: + with pkg_resources.path( + "observation_sim.instruments.data.ccd", "chip_definition.json" + ) as chip_definition: with open(chip_definition, "r") as f: chip_dict = json.load(f)[str(self.chipID)] for key in chip_dict: @@ -61,21 +66,33 @@ class Chip(FocalPlane): if self.filter_type == "FGS": fgs_name = self.chip_name[0:4] try: - with pkg_resources.files('observation_sim.instruments.data.field_distortion').joinpath("FieldDistModelGlobal_pr4_%s.pickle" % (fgs_name.lower())) as field_distortion: + with pkg_resources.files( + "observation_sim.instruments.data.field_distortion" + ).joinpath( + "FieldDistModelGlobal_pr4_%s.pickle" % (fgs_name.lower()) + ) as field_distortion: with open(field_distortion, "rb") as f: self.fdModel = pickle.load(f) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.field_distortion', "FieldDistModelGlobal_pr4_%s.pickle" % (fgs_name.lower())) as field_distortion: + with pkg_resources.path( + "observation_sim.instruments.data.field_distortion", + "FieldDistModelGlobal_pr4_%s.pickle" % (fgs_name.lower()), + ) as field_distortion: with open(field_distortion, "rb") as f: self.fdModel = pickle.load(f) else: # Get the corresponding field distortion model try: - with pkg_resources.files('observation_sim.instruments.data.field_distortion').joinpath("FieldDistModel_v2.0.pickle") as field_distortion: + with pkg_resources.files( + "observation_sim.instruments.data.field_distortion" + ).joinpath("FieldDistModel_v2.0.pickle") as field_distortion: with open(field_distortion, "rb") as f: self.fdModel = pickle.load(f) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.field_distortion', "FieldDistModelGlobal_mainFP_v1.0.pickle") as field_distortion: + with pkg_resources.path( + "observation_sim.instruments.data.field_distortion", + "FieldDistModelGlobal_mainFP_v1.0.pickle", + ) as field_distortion: with open(field_distortion, "rb") as f: self.fdModel = pickle.load(f) @@ -88,25 +105,37 @@ class Chip(FocalPlane): slsconfs = chip_utils.getChipSLSConf(chipID=self.chipID) if np.size(slsconfs) == 1: try: - with pkg_resources.files('observation_sim.instruments.data.sls_conf').joinpath(slsconfs) as conf_path: + with pkg_resources.files( + "observation_sim.instruments.data.sls_conf" + ).joinpath(slsconfs) as conf_path: self.sls_conf = str(conf_path) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.sls_conf', slsconfs) as conf_path: + with pkg_resources.path( + "observation_sim.instruments.data.sls_conf", slsconfs + ) as conf_path: self.sls_conf = str(conf_path) else: # self.sls_conf = [os.path.join(self.sls_dir, slsconfs[0]), os.path.join(self.sls_dir, slsconfs[1])] self.sls_conf = [] try: - with pkg_resources.files('observation_sim.instruments.data.sls_conf').joinpath(slsconfs[0]) as conf_path: + with pkg_resources.files( + "observation_sim.instruments.data.sls_conf" + ).joinpath(slsconfs[0]) as conf_path: self.sls_conf.append(str(conf_path)) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.sls_conf', slsconfs[0]) as conf_path: + with pkg_resources.path( + "observation_sim.instruments.data.sls_conf", slsconfs[0] + ) as conf_path: self.sls_conf.append(str(conf_path)) try: - with pkg_resources.files('observation_sim.instruments.data.sls_conf').joinpath(slsconfs[1]) as conf_path: + with pkg_resources.files( + "observation_sim.instruments.data.sls_conf" + ).joinpath(slsconfs[1]) as conf_path: self.sls_conf.append(str(conf_path)) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.sls_conf', slsconfs[1]) as conf_path: + with pkg_resources.path( + "observation_sim.instruments.data.sls_conf", slsconfs[1] + ) as conf_path: self.sls_conf.append(str(conf_path)) self.effCurve = self._getChipEffCurve(self.filter_type) @@ -119,11 +148,12 @@ class Chip(FocalPlane): def _set_attributes_from_config(self, config): # Default setting - self.read_noise = 4.5 # e/pix + self.read_noise = 4.5 # e/pix self.dark_noise = 0.02 # e/pix/s - self.rotate_angle = 0. + self.rotate_angle = 0.0 self.overscan = 1000 self.badfraction = 5e-5 + self.chip_type = "default" # Override default values # for key in ["gain", "bias_level, dark_exptime", "flat_exptime", "readout_time", "full_well", "read_noise", "dark_noise", "overscan"]: # if key in config["ins_effects"]: @@ -147,30 +177,39 @@ class Chip(FocalPlane): def _getChipEffCurve(self, filter_type): # CCD efficiency curves - if filter_type in ['NUV', 'u', 'GU']: - filename = 'UV0.txt' - if filter_type in ['g', 'r', 'GV', 'FGS']: + if filter_type in ["NUV", "u", "GU"]: + filename = "UV0.txt" + if filter_type in ["g", "r", "GV", "FGS"]: # TODO, need to switch to the right efficiency curvey for FGS CMOS - filename = 'Astro_MB.txt' - if filter_type in ['i', 'z', 'y', 'GI']: - filename = 'Basic_NIR.txt' + filename = "Astro_MB.txt" + if filter_type in ["i", "z", "y", "GI"]: + filename = "Basic_NIR.txt" try: - with pkg_resources.files('observation_sim.instruments.data.ccd').joinpath(filename) as ccd_path: - table = Table.read(ccd_path, format='ascii') + with pkg_resources.files("observation_sim.instruments.data.ccd").joinpath( + filename + ) as ccd_path: + table = Table.read(ccd_path, format="ascii") except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.ccd', filename) as ccd_path: - table = Table.read(ccd_path, format='ascii') + with pkg_resources.path( + "observation_sim.instruments.data.ccd", filename + ) as ccd_path: + table = Table.read(ccd_path, format="ascii") throughput = galsim.LookupTable( - x=table['col1'], f=table['col2'], interpolant='linear') - bandpass = galsim.Bandpass(throughput, wave_type='nm') + x=table["col1"], f=table["col2"], interpolant="linear" + ) + bandpass = galsim.Bandpass(throughput, wave_type="nm") return bandpass def _getCRdata(self): try: - with pkg_resources.files('observation_sim.instruments.data').joinpath("wfc-cr-attachpixel.dat") as cr_path: + with pkg_resources.files("observation_sim.instruments.data").joinpath( + "wfc-cr-attachpixel.dat" + ) as cr_path: self.attachedSizes = np.loadtxt(cr_path) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data', "wfc-cr-attachpixel.dat") as cr_path: + with pkg_resources.path( + "observation_sim.instruments.data", "wfc-cr-attachpixel.dat" + ) as cr_path: self.attachedSizes = np.loadtxt(cr_path) # def loadSLSFLATCUBE(self, flat_fn='flat_cube.fits'): @@ -189,8 +228,7 @@ class Chip(FocalPlane): # self.flat_cube[i] = flat_fits[i].data def getChipFilter(self, chipID=None): - """Return the filter index and type for a given chip #(chipID) - """ + """Return the filter index and type for a given chip #(chipID)""" filter_type_list = _util.ALL_FILTERS if chipID is None: chipID = self.chipID @@ -219,7 +257,7 @@ class Chip(FocalPlane): if chipID in [3, 4, 27, 28]: filter_type = "GU" if chipID in range(31, 43): - filter_type = 'FGS' + filter_type = "FGS" filter_id = filter_type_list.index(filter_type) return filter_id, filter_type @@ -236,43 +274,58 @@ class Chip(FocalPlane): xmin, xmax, ymin, ymax = 1e10, -1e10, 1e10, -1e10 xcen = self.x_cen / self.pix_size ycen = self.y_cen / self.pix_size - sign_x = [-1., 1., -1., 1.] - sign_y = [-1., -1., 1., 1.] + sign_x = [-1.0, 1.0, -1.0, 1.0] + sign_y = [-1.0, -1.0, 1.0, 1.0] for i in range(4): - x = xcen + sign_x[i] * self.npix_x / 2. - y = ycen + sign_y[i] * self.npix_y / 2. + x = xcen + sign_x[i] * self.npix_x / 2.0 + y = ycen + sign_y[i] * self.npix_y / 2.0 x, y = _util.rotate_conterclockwise( - x0=xcen, y0=ycen, x=x, y=y, angle=self.rotate_angle) + x0=xcen, y0=ycen, x=x, y=y, angle=self.rotate_angle + ) xmin, xmax = min(xmin, x), max(xmax, x) ymin, ymax = min(ymin, y), max(ymax, y) return galsim.BoundsD(xmin, xmax, ymin, ymax) def getSkyCoverage(self, wcs): # print("In getSkyCoverage: xmin = %.3f, xmax = %.3f, ymin = %.3f, ymax = %.3f"%(self.bound.xmin, self.bound.xmax, self.bound.ymin, self.bound.ymax)) - return super().getSkyCoverage(wcs, self.bound.xmin, self.bound.xmax, self.bound.ymin, self.bound.ymax) + return super().getSkyCoverage( + wcs, self.bound.xmin, self.bound.xmax, self.bound.ymin, self.bound.ymax + ) def getSkyCoverageEnlarged(self, wcs, margin=0.5): - """The enlarged sky coverage of the chip - """ + """The enlarged sky coverage of the chip""" margin /= 60.0 bound = self.getSkyCoverage(wcs) - return galsim.BoundsD(bound.xmin - margin, bound.xmax + margin, bound.ymin - margin, bound.ymax + margin) - - def isContainObj(self, ra_obj=None, dec_obj=None, x_image=None, y_image=None, wcs=None, margin=1): + return galsim.BoundsD( + bound.xmin - margin, + bound.xmax + margin, + bound.ymin - margin, + bound.ymax + margin, + ) + + def isContainObj( + self, ra_obj=None, dec_obj=None, x_image=None, y_image=None, wcs=None, margin=1 + ): # magin in number of pix if (ra_obj is not None) and (dec_obj is not None): if wcs is None: wcs = self.img.wcs - pos_obj = wcs.toImage(galsim.CelestialCoord( - ra=ra_obj*galsim.degrees, dec=dec_obj*galsim.degrees)) + pos_obj = wcs.toImage( + galsim.CelestialCoord( + ra=ra_obj * galsim.degrees, dec=dec_obj * galsim.degrees + ) + ) x_image, y_image = pos_obj.x, pos_obj.y elif (x_image is None) or (y_image is None): raise ValueError( - "Either (ra_obj, dec_obj) or (x_image, y_image) should be given") + "Either (ra_obj, dec_obj) or (x_image, y_image) should be given" + ) xmin, xmax = self.bound.xmin - margin, self.bound.xmax + margin ymin, ymax = self.bound.ymin - margin, self.bound.ymax + margin - if (x_image - xmin) * (x_image - xmax) > 0.0 or (y_image - ymin) * (y_image - ymax) > 0.0: + if (x_image - xmin) * (x_image - xmax) > 0.0 or (y_image - ymin) * ( + y_image - ymax + ) > 0.0: return False return True diff --git a/observation_sim/instruments/chip/chip_utils.py b/observation_sim/instruments/chip/chip_utils.py index e2a6c1326013e999434c00f600109b06ee45f335..c46e19b6c4b5813a1b3445eeeb8e73dba6e92367 100644 --- a/observation_sim/instruments/chip/chip_utils.py +++ b/observation_sim/instruments/chip/chip_utils.py @@ -2,8 +2,8 @@ import os import galsim import ctypes import numpy as np +from scipy import stats from astropy.io import fits -from datetime import datetime from observation_sim.instruments.chip import effects from observation_sim.config.header import generatePrimaryHeader, generateExtensionHeader @@ -23,64 +23,71 @@ def log_info(msg, logger=None): def getChipSLSGratingID(chipID): - gratingID = ['', ''] + gratingID = ["", ""] if chipID == 1: - gratingID = ['GI2', 'GI1'] + gratingID = ["GI2", "GI1"] if chipID == 2: - gratingID = ['GV4', 'GV3'] + gratingID = ["GV4", "GV3"] if chipID == 3: - gratingID = ['GU2', 'GU1'] + gratingID = ["GU2", "GU1"] if chipID == 4: - gratingID = ['GU4', 'GU3'] + gratingID = ["GU4", "GU3"] if chipID == 5: - gratingID = ['GV2', 'GV1'] + gratingID = ["GV2", "GV1"] if chipID == 10: - gratingID = ['GI4', 'GI3'] + gratingID = ["GI4", "GI3"] if chipID == 21: - gratingID = ['GI6', 'GI5'] + gratingID = ["GI6", "GI5"] if chipID == 26: - gratingID = ['GV8', 'GV7'] + gratingID = ["GV8", "GV7"] if chipID == 27: - gratingID = ['GU6', 'GU5'] + gratingID = ["GU6", "GU5"] if chipID == 28: - gratingID = ['GU8', 'GU7'] + gratingID = ["GU8", "GU7"] if chipID == 29: - gratingID = ['GV6', 'GV5'] + gratingID = ["GV6", "GV5"] if chipID == 30: - gratingID = ['GI8', 'GI7'] + gratingID = ["GI8", "GI7"] return gratingID def getChipSLSConf(chipID): - confFile = '' + confFile = "" if chipID == 1: - confFile = ['CSST_GI2.conf', 'CSST_GI1.conf'] + confFile = ["CSST_GI2.conf", "CSST_GI1.conf"] if chipID == 2: - confFile = ['CSST_GV4.conf', 'CSST_GV3.conf'] + confFile = ["CSST_GV4.conf", "CSST_GV3.conf"] if chipID == 3: - confFile = ['CSST_GU2.conf', 'CSST_GU1.conf'] + confFile = ["CSST_GU2.conf", "CSST_GU1.conf"] if chipID == 4: - confFile = ['CSST_GU4.conf', 'CSST_GU3.conf'] + confFile = ["CSST_GU4.conf", "CSST_GU3.conf"] if chipID == 5: - confFile = ['CSST_GV2.conf', 'CSST_GV1.conf'] + confFile = ["CSST_GV2.conf", "CSST_GV1.conf"] if chipID == 10: - confFile = ['CSST_GI4.conf', 'CSST_GI3.conf'] + confFile = ["CSST_GI4.conf", "CSST_GI3.conf"] if chipID == 21: - confFile = ['CSST_GI6.conf', 'CSST_GI5.conf'] + confFile = ["CSST_GI6.conf", "CSST_GI5.conf"] if chipID == 26: - confFile = ['CSST_GV8.conf', 'CSST_GV7.conf'] + confFile = ["CSST_GV8.conf", "CSST_GV7.conf"] if chipID == 27: - confFile = ['CSST_GU6.conf', 'CSST_GU5.conf'] + confFile = ["CSST_GU6.conf", "CSST_GU5.conf"] if chipID == 28: - confFile = ['CSST_GU8.conf', 'CSST_GU7.conf'] + confFile = ["CSST_GU8.conf", "CSST_GU7.conf"] if chipID == 29: - confFile = ['CSST_GV6.conf', 'CSST_GV5.conf'] + confFile = ["CSST_GV6.conf", "CSST_GV5.conf"] if chipID == 30: - confFile = ['CSST_GI8.conf', 'CSST_GI7.conf'] + confFile = ["CSST_GI8.conf", "CSST_GI7.conf"] return confFile -def generateHeader(chip, pointing, img_type=None, img_type_code=None, project_cycle='9', run_counter='1'): +def generateHeader( + chip, + pointing, + img_type=None, + img_type_code=None, + project_cycle="9", + run_counter="1", +): if (img_type is None) or (img_type_code is None): img_type = pointing.survey_field_type img_type_code = pointing.pointing_type_code @@ -101,8 +108,9 @@ def generateHeader(chip, pointing, img_type=None, img_type_code=None, project_cy pa=(pointing.img_pa.deg + 180) % 360 - 180, project_cycle=project_cycle, run_counter=run_counter, - chip_name=str(chip.chipID).rjust(2, '0'), - dataset=pointing.dataset) + chip_name=str(chip.chipID).rjust(2, "0"), + dataset=pointing.dataset, + ) h_ext = generateExtensionHeader( chip=chip, xlen=chip.npix_x, @@ -123,30 +131,41 @@ def generateHeader(chip, pointing, img_type=None, img_type_code=None, project_cy exptime=pointing.exp_time, readoutTime=chip.readout_time, t_shutter_open=pointing.t_shutter_open, - t_shutter_close=pointing.t_shutter_close) + t_shutter_close=pointing.t_shutter_close, + ) return h_prim, h_ext -def output_fits_image(chip, pointing, img, output_dir, img_type=None, img_type_code=None, project_cycle='9', run_counter='1'): +def output_fits_image( + chip, + pointing, + img, + output_dir, + img_type=None, + img_type_code=None, + project_cycle="9", + run_counter="1", +): h_prim, h_ext = generateHeader( chip=chip, pointing=pointing, img_type=img_type, img_type_code=img_type_code, project_cycle=project_cycle, - run_counter=run_counter) + run_counter=run_counter, + ) hdu1 = fits.PrimaryHDU(header=h_prim) hdu1.add_checksum() - hdu1.header.comments['CHECKSUM'] = 'HDU checksum' - hdu1.header.comments['DATASUM'] = 'data unit checksum' + hdu1.header.comments["CHECKSUM"] = "HDU checksum" + hdu1.header.comments["DATASUM"] = "data unit checksum" hdu2 = fits.ImageHDU(img.array, header=h_ext) hdu2.add_checksum() - hdu2.header.comments['XTENSION'] = 'extension type' - hdu2.header.comments['CHECKSUM'] = 'HDU checksum' - hdu2.header.comments['DATASUM'] = 'data unit checksum' + hdu2.header.comments["XTENSION"] = "extension type" + hdu2.header.comments["CHECKSUM"] = "HDU checksum" + hdu2.header.comments["DATASUM"] = "data unit checksum" hdu1 = fits.HDUList([hdu1, hdu2]) - fname = os.path.join(output_dir, h_prim['FILENAME']+'.fits') - hdu1.writeto(fname, output_verify='ignore', overwrite=True) + fname = os.path.join(output_dir, h_prim["FILENAME"] + ".fits") + hdu1.writeto(fname, output_verify="ignore", overwrite=True) def add_sky_background(img, filt, exptime, sky_map=None, tel=None): @@ -168,15 +187,14 @@ def add_sky_background(img, filt, exptime, sky_map=None, tel=None): def get_flat(img, seed): - flat_img = effects.MakeFlatSmooth( - GSBounds=img.bounds, - seed=seed) + flat_img = effects.MakeFlatSmooth(GSBounds=img.bounds, seed=seed) flat_normal = flat_img / np.mean(flat_img.array) return flat_img, flat_normal def get_innerflat(chip=None, filt=None): from observation_sim.mock_objects import FlatLED + led_obj = FlatLED(chip, filt) flat_img = led_obj.getInnerFlat() return flat_img @@ -184,12 +202,14 @@ def get_innerflat(chip=None, filt=None): def add_cosmic_rays(img, chip, exptime=150, seed=0): cr_map, cr_event_num = effects.produceCR_Map( - xLen=chip.npix_x, yLen=chip.npix_y, - exTime=exptime+0.5*chip.readout_time, - cr_pixelRatio=0.003*(exptime+0.5*chip.readout_time)/600., + xLen=chip.npix_x, + yLen=chip.npix_y, + exTime=exptime + 0.5 * chip.readout_time, + cr_pixelRatio=0.003 * (exptime + 0.5 * chip.readout_time) / 600.0, gain=chip.gain, attachedSizes=chip.attachedSizes, - seed=seed) # seed: obj-imaging:+0; bias:+1; dark:+2; flat:+3; + seed=seed, + ) # seed: obj-imaging:+0; bias:+1; dark:+2; flat:+3; img += cr_map cr_map[cr_map > 65535] = 65535 cr_map[cr_map < 0] = 0 @@ -200,70 +220,129 @@ def add_cosmic_rays(img, chip, exptime=150, seed=0): def add_PRNU(img, chip, seed=0): prnu_img = effects.PRNU_Img( - xsize=chip.npix_x, - ysize=chip.npix_y, - sigma=0.01, - seed=seed) + xsize=chip.npix_x, ysize=chip.npix_y, sigma=0.01, seed=seed + ) img *= prnu_img return img, prnu_img -def get_poisson(seed=0, sky_level=0.): +def get_poisson(seed=0, sky_level=0.0): rng_poisson = galsim.BaseDeviate(seed) poisson_noise = galsim.PoissonNoise(rng_poisson, sky_level=sky_level) return rng_poisson, poisson_noise -def get_base_img(img, chip, read_noise, readout_time, dark_noise, exptime=150., InputDark=None): - if InputDark is None: +def get_base_img( + img, chip, read_noise, readout_time, dark_noise, exptime=150.0, InputDark=False +): + if not InputDark: # base_level = read_noise**2 + dark_noise*(exptime+0.5*readout_time) # base_level = dark_noise*(exptime+0.5*readout_time) - base_level = dark_noise*(exptime) + base_level = dark_noise * (exptime) base_img1 = base_level * np.ones_like(img.array) + map_dc = base_img1 / exptime + + ny = int(chip.npix_y / 2) + nx = chip.npix_x + arr = np.arange(ny).reshape(ny, 1) + arr = np.broadcast_to(arr, (ny, nx)) + base_img2 = np.zeros_like(img.array) + base_img2[:ny, :] = arr + base_img2[ny:, :] = arr[::-1, :] + base_img2[:, :] = base_img2[:, :] * (readout_time / ny) * dark_noise else: - base_img1 = np.zeros_like(img.array) - - ny = int(chip.npix_y/2) - nx = chip.npix_x - arr = np.arange(ny).reshape(ny, 1) - arr = np.broadcast_to(arr, (ny, nx)) - base_img2 = np.zeros_like(img.array) - base_img2[:ny, :] = arr - base_img2[ny:, :] = arr[::-1, :] - base_img2[:, :] = base_img2[:, :]*(readout_time/ny)*dark_noise - return base_img1+base_img2 - - -def add_poisson(img, chip, exptime=150., seed=0, sky_level=0., poisson_noise=None, dark_noise=None, InputDark=None): + # base_img1 = np.zeros_like(img.array) + histfile = "hist_dark_{:}.npz".format(chip.chip_type) + try: + with pkg_resources.files("observation_sim.instruments.data.ccd").joinpath( + histfile + ) as dark_histogram: + hist_dc = np.load(dark_histogram) + except AttributeError: + with pkg_resources.path( + "observation_sim.instruments.data.ccd", histfile + ) as dark_histogram: + hist_dc = np.load(dark_histogram) + hist = stats.rv_histogram((hist_dc["counts"], hist_dc["bins"]), density=True) + xsize = chip.npix_x + ysize = chip.npix_y + seed = chip.chipID + samples = hist.rvs(size=xsize * ysize, random_state=seed) + map_dc = np.reshape(samples, [ysize, xsize]) + base_img1 = map_dc * exptime + + # base_img2 + map_temp = np.zeros((ysize // 2, 2 * xsize)) + map_temp[:, :xsize] = map_dc[: ysize // 2, :] + map_temp[:, xsize:] = map_dc[: ysize // 2 - 1 : -1, :] + + dt = readout_time / (ysize / 2.0) + A = map_temp + A_adjusted = np.vstack([np.zeros((1, A.shape[1])), A[:-1]]) + cumsum_A = np.cumsum(A_adjusted, axis=0) + B = cumsum_A * dt + base_img2 = np.vstack([B[:, :xsize], B[::-1, xsize:]]) + del hist_dc + del hist + del samples + del map_temp + del A + del A_adjusted + del cumsum_A + del B + return base_img1 + base_img2, map_dc + + +def add_poisson( + img, + chip, + exptime=150.0, + seed=0, + sky_level=0.0, + poisson_noise=None, + dark_noise=None, + InputDark=False, +): if poisson_noise is None: _, poisson_noise = get_poisson(seed=seed, sky_level=sky_level) read_noise = chip.read_noise if dark_noise is None: dark_noise = chip.dark_noise - base_img = get_base_img(img=img, chip=chip, read_noise=read_noise, readout_time=chip.readout_time, - dark_noise=dark_noise, exptime=exptime, InputDark=InputDark) + base_img, map_dc = get_base_img( + img=img, + chip=chip, + read_noise=read_noise, + readout_time=chip.readout_time, + dark_noise=dark_noise, + exptime=exptime, + InputDark=InputDark, + ) img += base_img img.addNoise(poisson_noise) # img -= read_noise**2 - - if InputDark is not None: - # "Instrument/data/dark/dark_1000s_example_0.fits" - hdu = fits.open(InputDark) - img += hdu[0].data/hdu[0].header['exptime']*exptime - hdu.close() - return img, base_img + del base_img + return img, map_dc def add_brighter_fatter(img): # Inital dynamic lib try: - with pkg_resources.files('observation_sim.instruments.chip.libBF').joinpath("libmoduleBF.so") as lib_path: + with pkg_resources.files("observation_sim.instruments.chip.libBF").joinpath( + "libmoduleBF.so" + ) as lib_path: lib_bf = ctypes.CDLL(lib_path) except AttributeError: - with pkg_resources.path('observation_sim.instruments.chip.libBF', "libmoduleBF.so") as lib_path: + with pkg_resources.path( + "observation_sim.instruments.chip.libBF", "libmoduleBF.so" + ) as lib_path: lib_bf = ctypes.CDLL(lib_path) - lib_bf.addEffects.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.POINTER( - ctypes.c_float), ctypes.POINTER(ctypes.c_float), ctypes.c_int] + lib_bf.addEffects.argtypes = [ + ctypes.c_int, + ctypes.c_int, + ctypes.POINTER(ctypes.c_float), + ctypes.POINTER(ctypes.c_float), + ctypes.c_int, + ] # Set bit flag bit_flag = 1 @@ -271,8 +350,8 @@ def add_brighter_fatter(img): nx, ny = img.array.shape nn = nx * ny - arr_ima = (ctypes.c_float*nn)() - arr_imc = (ctypes.c_float*nn)() + arr_ima = (ctypes.c_float * nn)() + arr_imc = (ctypes.c_float * nn)() arr_ima[:] = img.array.reshape(nn) arr_imc[:] = np.zeros(nn) @@ -299,49 +378,61 @@ def add_inputdark(img, chip, exptime): def AddPreScan(GSImage, pre1=27, pre2=4, over1=71, over2=80, nsecy=2, nsecx=8): img = GSImage.array ny, nx = img.shape - dx = int(nx/nsecx) - dy = int(ny/nsecy) - - imgt = np.zeros( - [int(nsecy*nsecx), int(ny/nsecy+pre2+over2), int(nx/nsecx+pre1+over1)]) + dx = int(nx / nsecx) + dy = int(ny / nsecy) + + imgt = np.zeros([ + int(nsecy * nsecx), + int(ny / nsecy + pre2 + over2), + int(nx / nsecx + pre1 + over1), + ]) for iy in range(nsecy): for ix in range(nsecx): if iy % 2 == 0: tx = ix else: - tx = (nsecx-1)-ix + tx = (nsecx - 1) - ix ty = iy # chunk1-[1,2,3,4], chunk2-[5,6,7,8], chunk3-[9,10,11,12], chunk4-[13,14,15,16] - chunkidx = int(tx+ty*nsecx) - - imgtemp = np.zeros( - [int(ny/nsecy+pre2+over2), int(nx/nsecx+pre1+over1)]) - if int(chunkidx/4) == 0: - imgtemp[pre2:pre2+dy, pre1:pre1 + - dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + chunkidx = int(tx + ty * nsecx) + + imgtemp = np.zeros([ + int(ny / nsecy + pre2 + over2), + int(nx / nsecx + pre1 + over1), + ]) + if int(chunkidx / 4) == 0: + imgtemp[pre2 : pre2 + dy, pre1 : pre1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp - if int(chunkidx/4) == 1: - imgtemp[pre2:pre2+dy, over1:over1 + - dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 1: + imgtemp[pre2 : pre2 + dy, over1 : over1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [:, ::-1] - if int(chunkidx/4) == 2: - imgtemp[over2:over2+dy, over1:over1 + - dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 2: + imgtemp[over2 : over2 + dy, over1 : over1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [::-1, ::-1] - if int(chunkidx/4) == 3: - imgtemp[over2:over2+dy, pre1:pre1 + - dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 3: + imgtemp[over2 : over2 + dy, pre1 : pre1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [::-1, :] # hstack chunk(1,2)-[1,2,3,4,5,6,7,8] imgtx1 = np.hstack(imgt[:nsecx:, :, :]) # hstack chunk(4,3)-[16,15,14,13,12,11,,10,9] - imgtx2 = np.hstack(imgt[:(nsecx-1):-1, :, :]) + imgtx2 = np.hstack(imgt[: (nsecx - 1) : -1, :, :]) - newimg = galsim.Image(int(nx+(pre1+over1)*nsecx), - int(ny+(pre2+over2)*nsecy), init_value=0) - newimg.array[:, :] = np.concatenate( - [imgtx1, imgtx2]) # vstack chunk(1,2) & chunk(4,3) + newimg = galsim.Image( + int(nx + (pre1 + over1) * nsecx), int(ny + (pre2 + over2) * nsecy), init_value=0 + ) + newimg.array[:, :] = np.concatenate([ + imgtx1, + imgtx2, + ]) # vstack chunk(1,2) & chunk(4,3) newimg.wcs = GSImage.wcs return newimg @@ -350,14 +441,17 @@ def AddPreScan(GSImage, pre1=27, pre2=4, over1=71, over2=80, nsecy=2, nsecx=8): def AddPreScanFO(GSImage, pre1=27, pre2=4, over1=71, over2=80, nsecy=1, nsecx=16): img = GSImage.array ny, nx = img.shape - dx = int(nx/nsecx) - dy = int(ny/nsecy) + dx = int(nx / nsecx) + dy = int(ny / nsecy) - newimg = galsim.Image(int(nx+(pre1+over1)*nsecx), - int(ny+(pre2+over2)*nsecy), init_value=0) + newimg = galsim.Image( + int(nx + (pre1 + over1) * nsecx), int(ny + (pre2 + over2) * nsecy), init_value=0 + ) for ix in range(nsecx): - newimg.array[pre2:pre2+dy, pre1+ix * - (dx+pre1+over1):pre1+dx+ix*(dx+pre1+over1)] = img[0:dy, 0+ix*dx:dx+ix*dx] + newimg.array[ + pre2 : pre2 + dy, + pre1 + ix * (dx + pre1 + over1) : pre1 + dx + ix * (dx + pre1 + over1), + ] = img[0:dy, 0 + ix * dx : dx + ix * dx] newimg.wcs = GSImage.wcs return newimg @@ -366,33 +460,41 @@ def AddPreScanFO(GSImage, pre1=27, pre2=4, over1=71, over2=80, nsecy=1, nsecx=16 def formatOutput(GSImage, nsecy=2, nsecx=8): img = GSImage.array ny, nx = img.shape - dx = int(nx/nsecx) - dy = int(ny/nsecy) + dx = int(nx / nsecx) + dy = int(ny / nsecy) - imgt = np.zeros([int(nsecx*nsecy), dy, dx]) + imgt = np.zeros([int(nsecx * nsecy), dy, dx]) for iy in range(nsecy): for ix in range(nsecx): if iy % 2 == 0: tx = ix else: - tx = (nsecx-1)-ix + tx = (nsecx - 1) - ix ty = iy - chunkidx = int(tx+ty*nsecx) - if int(chunkidx/4) == 0: - imgt[chunkidx, :, :] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] - if int(chunkidx/4) == 1: - imgt[chunkidx, :, :] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] - if int(chunkidx/4) == 2: - imgt[chunkidx, :, :] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] - if int(chunkidx/4) == 3: - imgt[chunkidx, :, :] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + chunkidx = int(tx + ty * nsecx) + if int(chunkidx / 4) == 0: + imgt[chunkidx, :, :] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] + if int(chunkidx / 4) == 1: + imgt[chunkidx, :, :] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] + if int(chunkidx / 4) == 2: + imgt[chunkidx, :, :] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] + if int(chunkidx / 4) == 3: + imgt[chunkidx, :, :] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgttx0 = np.hstack(imgt[0:4:, :, :]) imgttx1 = np.hstack(imgt[4:8:, :, ::-1]) imgttx2 = np.hstack(imgt[8:12:, ::-1, ::-1]) imgttx3 = np.hstack(imgt[12:16:, ::-1, :]) - newimg = galsim.Image(int(dx*nsecx*nsecy), dy, init_value=0) + newimg = galsim.Image(int(dx * nsecx * nsecy), dy, init_value=0) newimg.array[:, :] = np.hstack([imgttx0, imgttx1, imgttx2, imgttx3]) return newimg @@ -400,25 +502,30 @@ def formatOutput(GSImage, nsecy=2, nsecx=8): def formatRevert(GSImage, nsecy=1, nsecx=16): img = GSImage.array ny, nx = img.shape - dx = int(nx/nsecx) - dy = int(ny/nsecy) + dx = int(nx / nsecx) + dy = int(ny / nsecy) - newimg = galsim.Image(int(dx*8), int(dy*2), init_value=0) + newimg = galsim.Image(int(dx * 8), int(dy * 2), init_value=0) for ix in range(0, 4): tx = ix - newimg.array[0:dy, 0+tx*dx:dx+tx*dx] = img[:, 0+ix*dx:dx+ix*dx] + newimg.array[0:dy, 0 + tx * dx : dx + tx * dx] = img[ + :, 0 + ix * dx : dx + ix * dx + ] for ix in range(4, 8): tx = ix - newimg.array[0:dy, 0+tx*dx:dx+tx * - dx] = img[:, 0+ix*dx:dx+ix*dx][:, ::-1] + newimg.array[0:dy, 0 + tx * dx : dx + tx * dx] = img[ + :, 0 + ix * dx : dx + ix * dx + ][:, ::-1] for ix in range(8, 12): - tx = 7-(ix-8) - newimg.array[0+dy:dy+dy, 0+tx*dx:dx+tx * - dx] = img[:, 0+ix*dx:dx+ix*dx][::-1, ::-1] + tx = 7 - (ix - 8) + newimg.array[0 + dy : dy + dy, 0 + tx * dx : dx + tx * dx] = img[ + :, 0 + ix * dx : dx + ix * dx + ][::-1, ::-1] for ix in range(12, 16): - tx = 7-(ix-8) - newimg.array[0+dy:dy+dy, 0+tx*dx:dx+tx * - dx] = img[:, 0+ix*dx:dx+ix*dx][::-1, :] + tx = 7 - (ix - 8) + newimg.array[0 + dy : dy + dy, 0 + tx * dx : dx + tx * dx] = img[ + :, 0 + ix * dx : dx + ix * dx + ][::-1, :] return newimg diff --git a/observation_sim/instruments/chip/effects.py b/observation_sim/instruments/chip/effects.py index af696f7c5004f0b7fdaa2767f2b103c53a0a7f09..29921fc60b5186e847e2eb40aa01012af4fdb1f5 100644 --- a/observation_sim/instruments/chip/effects.py +++ b/observation_sim/instruments/chip/effects.py @@ -1,7 +1,5 @@ import galsim -from matplotlib.pyplot import flag import numpy as np -from numpy.core.fromnumeric import mean, size from numpy.random import Generator, PCG64 import math import copy @@ -9,7 +7,16 @@ from numba import jit from astropy import stats -def AddOverscan(GSImage, overscan=1000, gain=1, widthl=27, widthr=27, widtht=8, widthb=8, read_noise=5): +def AddOverscan( + GSImage, + overscan=1000, + gain=1, + widthl=27, + widthr=27, + widtht=8, + widthb=8, + read_noise=5, +): """ Add overscan/gain; gain=e-/ADU widthl: left pre-scan width @@ -18,12 +25,16 @@ def AddOverscan(GSImage, overscan=1000, gain=1, widthl=27, widthr=27, widtht=8, widthb: bottom over-scan width (the bottom of nd-array with large row-index) """ imgshape = GSImage.array.shape - newimg = galsim.Image(imgshape[1]+widthl+widthr, imgshape[0]+widtht+widthb, init_value=0) + newimg = galsim.Image( + imgshape[1] + widthl + widthr, imgshape[0] + widtht + widthb, init_value=0 + ) rng = galsim.UniformDeviate() NoiseOS = galsim.GaussianNoise(rng, sigma=read_noise) newimg.addNoise(NoiseOS) - newimg = (newimg+overscan)/gain - newimg.array[widtht:(widtht+imgshape[0]), widthl:(widthl+imgshape[1])] = GSImage.array + newimg = (newimg + overscan) / gain + newimg.array[widtht : (widtht + imgshape[0]), widthl : (widthl + imgshape[1])] = ( + GSImage.array + ) newimg.wcs = GSImage.wcs # if GSImage.wcs is not None: # newwcs = GSImage.wcs.withOrigin(galsim.PositionD(widthl,widtht)) @@ -33,13 +44,15 @@ def AddOverscan(GSImage, overscan=1000, gain=1, widthl=27, widthr=27, widtht=8, return newimg -def DefectivePixels(GSImage, IfHotPix=True, IfDeadPix=True, fraction=1E-4, seed=20210304, biaslevel=0): +def DefectivePixels( + GSImage, IfHotPix=True, IfDeadPix=True, fraction=1e-4, seed=20210304, biaslevel=0 +): # Also called bad pixels, including hot pixels and dead pixels # Hot Pixel > 20e-/s # Dead Pixel < 70%*Mean - rgf = Generator(PCG64(int(seed*1.1))) + rgf = Generator(PCG64(int(seed * 1.1))) if IfHotPix is True and IfDeadPix is True: - HotFraction = rgf.random() # fraction in total bad pixels + HotFraction = rgf.random() # fraction in total bad pixels elif IfHotPix is False and IfDeadPix is False: return GSImage elif IfHotPix is True: @@ -48,65 +61,100 @@ def DefectivePixels(GSImage, IfHotPix=True, IfDeadPix=True, fraction=1E-4, seed= HotFraction = 0 NPix = GSImage.array.size - NPixBad = int(NPix*fraction) - NPixHot = int(NPix*fraction*HotFraction) - NPixDead = NPixBad-NPixHot + NPixBad = int(NPix * fraction) + NPixHot = int(NPix * fraction * HotFraction) + NPixDead = NPixBad - NPixHot NPix_y, NPix_x = GSImage.array.shape mean = np.mean(GSImage.array) rgp = Generator(PCG64(int(seed))) yxposfrac = rgp.random((NPixBad, 2)) - YPositHot = np.array(NPix_y*yxposfrac[0:NPixHot, 0]).astype(np.int32) - XPositHot = np.array(NPix_x*yxposfrac[0:NPixHot, 1]).astype(np.int32) - YPositDead = np.array(NPix_y*yxposfrac[NPixHot:, 0]).astype(np.int32) - XPositDead = np.array(NPix_x*yxposfrac[NPixHot:, 1]).astype(np.int32) + YPositHot = np.array(NPix_y * yxposfrac[0:NPixHot, 0]).astype(np.int32) + XPositHot = np.array(NPix_x * yxposfrac[0:NPixHot, 1]).astype(np.int32) + YPositDead = np.array(NPix_y * yxposfrac[NPixHot:, 0]).astype(np.int32) + XPositDead = np.array(NPix_x * yxposfrac[NPixHot:, 1]).astype(np.int32) - rgh = Generator(PCG64(int(seed*1.2))) - rgd = Generator(PCG64(int(seed*1.3))) + rgh = Generator(PCG64(int(seed * 1.2))) + rgd = Generator(PCG64(int(seed * 1.3))) if IfHotPix is True: - GSImage.array[YPositHot, XPositHot] += rgh.gamma(2, 25*150, size=NPixHot) + GSImage.array[YPositHot, XPositHot] += rgh.gamma(2, 25 * 150, size=NPixHot) if IfDeadPix is True: - GSImage.array[YPositDead, XPositDead] = rgd.random(NPixDead)*(mean-biaslevel)*0.7+biaslevel+rgp.standard_normal()*5 + GSImage.array[YPositDead, XPositDead] = ( + rgd.random(NPixDead) * (mean - biaslevel) * 0.7 + + biaslevel + + rgp.standard_normal() * 5 + ) return GSImage def BadColumns(GSImage, seed=20240309, chipid=1, logger=None): # Set bad column values ysize, xsize = GSImage.array.shape - subarr = GSImage.array[int(ysize*0.1):int(ysize*0.12), int(xsize*0.1):int(xsize*0.12)] - subarr = stats.sigma_clip(subarr, sigma=4, cenfunc='median', maxiters=3, masked=False) + subarr = GSImage.array[ + int(ysize * 0.1) : int(ysize * 0.12), int(xsize * 0.1) : int(xsize * 0.12) + ] + subarr = stats.sigma_clip( + subarr, sigma=4, cenfunc="median", maxiters=3, masked=False + ) meanimg = np.median(subarr) stdimg = np.std(subarr) seed += chipid rgn = Generator(PCG64(int(seed))) - rgcollen = Generator(PCG64(int(seed*1.1))) - rgxpos = Generator(PCG64(int(seed*1.2))) - rgdn = Generator(PCG64(int(seed*1.3))) + rgcollen = Generator(PCG64(int(seed * 1.1))) + rgxpos = Generator(PCG64(int(seed * 1.2))) + rgdn = Generator(PCG64(int(seed * 1.3))) nbadsecA, nbadsecD = rgn.integers(low=1, high=5, size=2) - collen = rgcollen.integers(low=int(ysize*0.1), high=int(ysize*0.5), size=(nbadsecA+nbadsecD)) - xposit = rgxpos.integers(low=int(xsize*0.05), high=int(xsize*0.95), size=(nbadsecA+nbadsecD)) + collen = rgcollen.integers( + low=int(ysize * 0.1), high=int(ysize * 0.5), size=(nbadsecA + nbadsecD) + ) + xposit = rgxpos.integers( + low=int(xsize * 0.05), high=int(xsize * 0.95), size=(nbadsecA + nbadsecD) + ) if logger is not None: - logger.info(xposit+1) + logger.info(xposit + 1) else: - print(xposit+1) + print(xposit + 1) # signs = 2*rgdn.integers(0,2,size=(nbadsecA+nbadsecD))-1 # if meanimg>0: # dn = rgdn.integers(low=np.abs(meanimg)*1.3+50, high=np.abs(meanimg)*2+150, size=(nbadsecA+nbadsecD)) # *signs - dn = rgdn.integers(low=50, high=30000, size=(nbadsecA+nbadsecD)) # *signs + dn = rgdn.integers(low=50, high=30000, size=(nbadsecA + nbadsecD)) # *signs # elif meanimg<0: # dn = rgdn.integers(low=meanimg*2-150, high=meanimg*1.3-50, size=(nbadsecA+nbadsecD)) #*signs for badcoli in range(nbadsecA): - GSImage.array[(ysize-collen[badcoli]):ysize, xposit[badcoli]:(xposit[badcoli]+1)] = (np.abs(np.random.normal(0, 8.58*np.exp(0.0378*dn[badcoli]**0.5), (collen[badcoli], 1)))+dn[badcoli]) # (np.abs(np.random.normal(0, stdimg*2, (collen[badcoli], 1)))+dn[badcoli]) + GSImage.array[ + (ysize - collen[badcoli]) : ysize, xposit[badcoli] : (xposit[badcoli] + 1) + ] = ( + np.abs( + np.random.normal( + 0, 8.58 * np.exp(0.0378 * dn[badcoli] ** 0.5), (collen[badcoli], 1) + ) + ) + + dn[badcoli] + ) # (np.abs(np.random.normal(0, stdimg*2, (collen[badcoli], 1)))+dn[badcoli]) for badcoli in range(nbadsecD): - GSImage.array[0:collen[badcoli+nbadsecA], xposit[badcoli+nbadsecA]:(xposit[badcoli+nbadsecA]+1)] = (np.abs(np.random.normal(0, 8.58*np.exp(0.0378*dn[badcoli+nbadsecA]**0.5), (collen[badcoli+nbadsecA], 1)))+dn[badcoli+nbadsecA]) # (np.abs(np.random.normal(0, stdimg*2, (collen[badcoli+nbadsecA], 1)))+dn[badcoli+nbadsecA]) + GSImage.array[ + 0 : collen[badcoli + nbadsecA], + xposit[badcoli + nbadsecA] : (xposit[badcoli + nbadsecA] + 1), + ] = ( + np.abs( + np.random.normal( + 0, + 8.58 * np.exp(0.0378 * dn[badcoli + nbadsecA] ** 0.5), + (collen[badcoli + nbadsecA], 1), + ) + ) + + dn[badcoli + nbadsecA] + ) # (np.abs(np.random.normal(0, stdimg*2, (collen[badcoli+nbadsecA], 1)))+dn[badcoli+nbadsecA]) return GSImage -def AddBiasNonUniform16(GSImage, bias_level=500, nsecy=2, nsecx=8, seed=202102, logger=None): +def AddBiasNonUniform16( + GSImage, bias_level=500, nsecy=2, nsecx=8, seed=202102, logger=None +): # Generate Bias and its non-uniformity, and add the 16 bias values to the GS-Image rg = Generator(PCG64(int(seed))) - Random16 = (rg.random(nsecy*nsecx)-0.5)*20 + Random16 = (rg.random(nsecy * nsecx) - 0.5) * 20 if int(bias_level) == 0: BiasLevel = np.zeros((nsecy, nsecx)) elif bias_level > 0: @@ -117,33 +165,48 @@ def AddBiasNonUniform16(GSImage, bias_level=500, nsecy=2, nsecx=8, seed=202102, else: print(" Biases of 16 channels:\n", BiasLevel) arrshape = GSImage.array.shape - secsize_x = int(arrshape[1]/nsecx) - secsize_y = int(arrshape[0]/nsecy) + secsize_x = int(arrshape[1] / nsecx) + secsize_y = int(arrshape[0] / nsecy) for rowi in range(nsecy): for coli in range(nsecx): - GSImage.array[rowi*secsize_y:(rowi+1)*secsize_y, coli*secsize_x:(coli+1)*secsize_x] += BiasLevel[rowi, coli] + GSImage.array[ + rowi * secsize_y : (rowi + 1) * secsize_y, + coli * secsize_x : (coli + 1) * secsize_x, + ] += BiasLevel[rowi, coli] return GSImage -def MakeBiasNcomb(npix_x, npix_y, bias_level=500, ncombine=1, read_noise=5, gain=1, seed=202102, logger=None): +def MakeBiasNcomb( + npix_x, + npix_y, + bias_level=500, + ncombine=1, + read_noise=5, + gain=1, + seed=202102, + logger=None, +): # Start with 0 value bias GS-Image ncombine = int(ncombine) BiasSngImg0 = galsim.Image(npix_x, npix_y, init_value=0) - BiasSngImg = AddBiasNonUniform16(BiasSngImg0, - bias_level=bias_level, - nsecy=2, nsecx=8, - seed=int(seed), - logger=logger) - BiasCombImg = BiasSngImg*ncombine + BiasSngImg = AddBiasNonUniform16( + BiasSngImg0, + bias_level=bias_level, + nsecy=2, + nsecx=8, + seed=int(seed), + logger=logger, + ) + BiasCombImg = BiasSngImg * ncombine rng = galsim.UniformDeviate() - NoiseBias = galsim.GaussianNoise(rng=rng, sigma=read_noise*ncombine**0.5) + NoiseBias = galsim.GaussianNoise(rng=rng, sigma=read_noise * ncombine**0.5) BiasCombImg.addNoise(NoiseBias) if ncombine == 1: - BiasTag = 'Single' + BiasTag = "Single" pass elif ncombine > 1: BiasCombImg /= ncombine - BiasTag = 'Combine' + BiasTag = "Combine" # BiasCombImg.replaceNegative(replace_value=0) # BiasCombImg.quantize() return BiasCombImg, BiasTag @@ -152,34 +215,37 @@ def MakeBiasNcomb(npix_x, npix_y, bias_level=500, ncombine=1, read_noise=5, gain def ApplyGainNonUniform16(GSImage, gain=1, nsecy=2, nsecx=8, seed=202102, logger=None): # Generate Gain non-uniformity, and multipy the different factors (mean~1 with sigma~1%) to the GS-Image rg = Generator(PCG64(int(seed))) - Random16 = (rg.random(nsecy*nsecx)-0.5)*0.04+1 # sigma~1% - Gain16 = Random16.reshape((nsecy, nsecx))/gain - gain_array = np.ones(nsecy*nsecx)*gain + Random16 = (rg.random(nsecy * nsecx) - 0.5) * 0.04 + 1 # sigma~1% + Gain16 = Random16.reshape((nsecy, nsecx)) / gain + gain_array = np.ones(nsecy * nsecx) * gain if logger is not None: msg = str("Gain of 16 channels: " + str(Gain16)) logger.info(msg) else: print("Gain of 16 channels: ", Gain16) arrshape = GSImage.array.shape - secsize_x = int(arrshape[1]/nsecx) - secsize_y = int(arrshape[0]/nsecy) + secsize_x = int(arrshape[1] / nsecx) + secsize_y = int(arrshape[0] / nsecy) for rowi in range(nsecy): for coli in range(nsecx): - GSImage.array[rowi*secsize_y:(rowi+1)*secsize_y, coli*secsize_x:(coli+1)*secsize_x] *= Gain16[rowi, coli] - gain_array[rowi*nsecx+coli] = 1/Gain16[rowi, coli] + GSImage.array[ + rowi * secsize_y : (rowi + 1) * secsize_y, + coli * secsize_x : (coli + 1) * secsize_x, + ] *= Gain16[rowi, coli] + gain_array[rowi * nsecx + coli] = 1 / Gain16[rowi, coli] return GSImage, gain_array def GainsNonUniform16(GSImage, gain=1, nsecy=2, nsecx=8, seed=202102, logger=None): # Generate Gain non-uniformity, and multipy the different factors (mean~1 with sigma~1%) to the GS-Image rg = Generator(PCG64(int(seed))) - Random16 = (rg.random(nsecy*nsecx)-0.5)*0.04+1 # sigma~1% - Gain16 = Random16.reshape((nsecy, nsecx))/gain + Random16 = (rg.random(nsecy * nsecx) - 0.5) * 0.04 + 1 # sigma~1% + Gain16 = Random16.reshape((nsecy, nsecx)) / gain if logger is not None: - msg = str(seed-20210202, "Gains of 16 channels: " + str(Gain16)) + msg = str(seed - 20210202, "Gains of 16 channels: " + str(Gain16)) logger.info(msg) else: - print(seed-20210202, "Gains of 16 channels:\n", Gain16) + print(seed - 20210202, "Gains of 16 channels:\n", Gain16) # arrshape = GSImage.array.shape # secsize_x = int(arrshape[1]/nsecx) # secsize_y = int(arrshape[0]/nsecy) @@ -193,69 +259,104 @@ def GainsNonUniform16(GSImage, gain=1, nsecy=2, nsecx=8, seed=202102, logger=Non def MakeFlatSmooth(GSBounds, seed): rg = Generator(PCG64(int(seed))) r1, r2, r3, r4 = rg.random(4) - a1 = -0.5 + 0.2*r1 - a2 = -0.5 + 0.2*r2 - a3 = r3+5 - a4 = r4+5 - xmin, xmax, ymin, ymax = GSBounds.getXMin(), GSBounds.getXMax(), GSBounds.getYMin(), GSBounds.getYMax() - Flty, Fltx = np.mgrid[ymin:(ymax+1), xmin:(xmax+1)] + a1 = -0.5 + 0.2 * r1 + a2 = -0.5 + 0.2 * r2 + a3 = r3 + 5 + a4 = r4 + 5 + xmin, xmax, ymin, ymax = ( + GSBounds.getXMin(), + GSBounds.getXMax(), + GSBounds.getYMin(), + GSBounds.getYMax(), + ) + Flty, Fltx = np.mgrid[ymin : (ymax + 1), xmin : (xmax + 1)] rg = Generator(PCG64(int(seed))) p1, p2, bg = rg.poisson(1000, 3) - Fltz = 0.6*1e-7*(a1 * (Fltx-p1) ** 2 + a2 * (Flty-p2) ** 2 - a3*Fltx - a4*Flty) + bg*20 + Fltz = ( + 0.6 + * 1e-7 + * (a1 * (Fltx - p1) ** 2 + a2 * (Flty - p2) ** 2 - a3 * Fltx - a4 * Flty) + + bg * 20 + ) FlatImg = galsim.ImageF(Fltz) return FlatImg -def MakeFlatNcomb(flat_single_image, ncombine=1, read_noise=5, gain=1, overscan=500, biaslevel=500, seed_bias=20210311, logger=None): +def MakeFlatNcomb( + flat_single_image, + ncombine=1, + read_noise=5, + gain=1, + overscan=500, + biaslevel=500, + seed_bias=20210311, + logger=None, +): ncombine = int(ncombine) - FlatCombImg = flat_single_image*ncombine + FlatCombImg = flat_single_image * ncombine rng = galsim.UniformDeviate() NoiseFlatPoi = galsim.PoissonNoise(rng=rng, sky_level=0) FlatCombImg.addNoise(NoiseFlatPoi) - NoiseFlatReadN = galsim.GaussianNoise(rng=rng, sigma=read_noise*ncombine**0.5) + NoiseFlatReadN = galsim.GaussianNoise(rng=rng, sigma=read_noise * ncombine**0.5) FlatCombImg.addNoise(NoiseFlatReadN) # NoiseFlat = galsim.CCDNoise(rng, gain=gain, read_noise=read_noise*ncombine**0.5, sky_level=0) for i in range(ncombine): FlatCombImg = AddBiasNonUniform16( FlatCombImg, bias_level=biaslevel, - nsecy=2, nsecx=8, + nsecy=2, + nsecx=8, seed=seed_bias, - logger=logger) + logger=logger, + ) if ncombine == 1: - FlatTag = 'Single' + FlatTag = "Single" pass elif ncombine > 1: FlatCombImg /= ncombine - FlatTag = 'Combine' + FlatTag = "Combine" # FlatCombImg.replaceNegative(replace_value=0) # FlatCombImg.quantize() return FlatCombImg, FlatTag -def MakeDarkNcomb(npix_x, npix_y, overscan=500, bias_level=500, seed_bias=202102, darkpsec=0.02, exptime=150, ncombine=10, read_noise=5, gain=1, logger=None): +def MakeDarkNcomb( + npix_x, + npix_y, + overscan=500, + bias_level=500, + seed_bias=202102, + darkpsec=0.02, + exptime=150, + ncombine=10, + read_noise=5, + gain=1, + logger=None, +): ncombine = int(ncombine) - darkpix = darkpsec*exptime + darkpix = darkpsec * exptime DarkSngImg = galsim.Image(npix_x, npix_y, init_value=darkpix) rng = galsim.UniformDeviate() NoiseDarkPoi = galsim.PoissonNoise(rng=rng, sky_level=0) - NoiseReadN = galsim.GaussianNoise(rng=rng, sigma=read_noise*ncombine**0.5) - DarkCombImg = DarkSngImg*ncombine + NoiseReadN = galsim.GaussianNoise(rng=rng, sigma=read_noise * ncombine**0.5) + DarkCombImg = DarkSngImg * ncombine DarkCombImg.addNoise(NoiseDarkPoi) DarkCombImg.addNoise(NoiseReadN) for i in range(ncombine): DarkCombImg = AddBiasNonUniform16( DarkCombImg, bias_level=bias_level, - nsecy=2, nsecx=8, + nsecy=2, + nsecx=8, seed=int(seed_bias), - logger=logger) + logger=logger, + ) if ncombine == 1: - DarkTag = 'Single' + DarkTag = "Single" pass elif ncombine > 1: DarkCombImg /= ncombine - DarkTag = 'Combine' + DarkTag = "Combine" # DarkCombImg.replaceNegative(replace_value=0) # DarkCombImg.quantize() return DarkCombImg, DarkTag @@ -272,7 +373,7 @@ def NonLinear_f(x, beta_1, beta_2): return x - beta_1 * x * x + beta_2 * x * x * x -def NonLinearity(GSImage, beta1=5E-7, beta2=0): +def NonLinearity(GSImage, beta1=5e-7, beta2=0): # NonLinear_f = lambda x, beta_1, beta_2: x - beta_1*x*x + beta_2*x*x*x GSImage.applyNonlinearity(NonLinear_f, beta1, beta2) return GSImage @@ -285,9 +386,12 @@ def BleedingTrail(aa, yy): else: pass try: - fy = 0.5*(math.exp(math.log(yy+1)**3/aa)+np.exp(-1*math.log(yy+1)**3/aa)) - faa = 0.5*(math.e+1/math.e) - trail_frac = 1-0.1*(fy-1)/(faa-1) + fy = 0.5 * ( + math.exp(math.log(yy + 1) ** 3 / aa) + + np.exp(-1 * math.log(yy + 1) ** 3 / aa) + ) + faa = 0.5 * (math.e + 1 / math.e) + trail_frac = 1 - 0.1 * (fy - 1) / (faa - 1) except Exception as e: print(e) trail_frac = 1 @@ -295,31 +399,33 @@ def BleedingTrail(aa, yy): return trail_frac -def MakeTrail(imgarr, satuyxtuple, charge, fullwell=9e4, direction='up', trailcutfrac=0.9): - ''' +def MakeTrail( + imgarr, satuyxtuple, charge, fullwell=9e4, direction="up", trailcutfrac=0.9 +): + """ direction: "up" or "down". For "up", bleeds along Y-decreasing direction; for "down", bleeds along Y-increasing direction. - ''' + """ yi, xi = satuyxtuple - aa = np.log(charge/fullwell)**3 # scale length of the bleeding trail + aa = np.log(charge / fullwell) ** 3 # scale length of the bleeding trail yy = 1 while charge > 0: - if yi < 0 or yi > imgarr.shape[0]-1: + if yi < 0 or yi > imgarr.shape[0] - 1: break - if yi == 0 or yi == imgarr.shape[0]-1: + if yi == 0 or yi == imgarr.shape[0] - 1: imgarr[yi, xi] = fullwell break - if direction == 'up': - if imgarr[yi-1, xi] >= fullwell: + if direction == "up": + if imgarr[yi - 1, xi] >= fullwell: imgarr[yi, xi] = fullwell yi -= 1 # [TEST] charge in the middle if yi == (imgarr.shape[0] // 2 - 1): break continue - elif direction == 'down': - if imgarr[yi+1, xi] >= fullwell: + elif direction == "down": + if imgarr[yi + 1, xi] >= fullwell: imgarr[yi, xi] = fullwell yi += 1 if yi == (imgarr.shape[0] // 2): @@ -328,19 +434,19 @@ def MakeTrail(imgarr, satuyxtuple, charge, fullwell=9e4, direction='up', trailcu if aa <= 1: while imgarr[yi, xi] >= fullwell: imgarr[yi, xi] = fullwell - if direction == 'up': - imgarr[yi-1, xi] += charge - charge = imgarr[yi-1, xi]-fullwell + if direction == "up": + imgarr[yi - 1, xi] += charge + charge = imgarr[yi - 1, xi] - fullwell yi -= 1 # if yi < 0: - if yi < 0 or yi == (imgarr.shape[0]//2 - 1): + if yi < 0 or yi == (imgarr.shape[0] // 2 - 1): break - elif direction == 'down': - imgarr[yi+1, xi] += charge - charge = imgarr[yi+1, xi]-fullwell + elif direction == "down": + imgarr[yi + 1, xi] += charge + charge = imgarr[yi + 1, xi] - fullwell yi += 1 # if yi > imgarr.shape[0]: - if yi > imgarr.shape[0] or yi == (imgarr.shape[0]//2): + if yi > imgarr.shape[0] or yi == (imgarr.shape[0] // 2): break else: # calculate bleeding trail: @@ -349,44 +455,44 @@ def MakeTrail(imgarr, satuyxtuple, charge, fullwell=9e4, direction='up', trailcu # put charge upwards if trail_frac >= 0.99: imgarr[yi, xi] = fullwell - if direction == 'up': + if direction == "up": yi -= 1 - if yi == (imgarr.shape[0]//2 - 1): + if yi == (imgarr.shape[0] // 2 - 1): break - elif direction == 'down': + elif direction == "down": yi += 1 - if yi == (imgarr.shape[0]//2): + if yi == (imgarr.shape[0] // 2): break yy += 1 else: if trail_frac < trailcutfrac: break - charge = fullwell*trail_frac + charge = fullwell * trail_frac imgarr[yi, xi] += charge if imgarr[yi, xi] > fullwell: imgarr[yi, xi] = fullwell - if direction == 'up': + if direction == "up": yi -= 1 - if yi == (imgarr.shape[0]//2 - 1): + if yi == (imgarr.shape[0] // 2 - 1): break - elif direction == 'down': + elif direction == "down": yi += 1 - if yi == (imgarr.shape[0]//2): + if yi == (imgarr.shape[0] // 2): break yy += 1 return imgarr -def ChargeFlow(imgarr, fullwell=9E4): +def ChargeFlow(imgarr, fullwell=9e4): size_y, size_x = imgarr.shape satupos_y, satupos_x = np.where(imgarr > fullwell) if satupos_y.shape[0] == 0: # make no change for the image array return imgarr - elif satupos_y.shape[0]/imgarr.size > 0.5: + elif satupos_y.shape[0] / imgarr.size > 0.5: imgarr.fill(fullwell) return imgarr @@ -394,23 +500,37 @@ def ChargeFlow(imgarr, fullwell=9E4): imgarrorig = copy.deepcopy(imgarr) for yi, xi in zip(satupos_y, satupos_x): - yxidx = ''.join([str(yi), str(xi)]) - chargedict[yxidx] = imgarrorig[yi, xi]-fullwell + yxidx = "".join([str(yi), str(xi)]) + chargedict[yxidx] = imgarrorig[yi, xi] - fullwell for yi, xi in zip(satupos_y, satupos_x): - yxidx = ''.join([str(yi), str(xi)]) + yxidx = "".join([str(yi), str(xi)]) satcharge = chargedict[yxidx] - chargeup = ((np.random.random()-0.5)*0.05+0.5)*satcharge + chargeup = ((np.random.random() - 0.5) * 0.05 + 0.5) * satcharge chargedn = satcharge - chargeup try: # Charge Clump moves up if yi >= 0 and yi < imgarr.shape[0]: - imgarr = MakeTrail(imgarr, (yi, xi), chargeup, fullwell=9e4, direction='up', trailcutfrac=0.9) + imgarr = MakeTrail( + imgarr, + (yi, xi), + chargeup, + fullwell=9e4, + direction="up", + trailcutfrac=0.9, + ) # Charge Clump moves down - imgarr = MakeTrail(imgarr, (yi, xi), chargedn, fullwell=9e4, direction='down', trailcutfrac=0.9) + imgarr = MakeTrail( + imgarr, + (yi, xi), + chargedn, + fullwell=9e4, + direction="down", + trailcutfrac=0.9, + ) except Exception as e: - print(e, '@pix ', (yi+1, xi+1)) + print(e, "@pix ", (yi + 1, xi + 1)) return imgarr return imgarr @@ -427,19 +547,24 @@ def SaturBloom(GSImage, nsect_x=1, nsect_y=1, fullwell=9e4): """ imgarr = GSImage.array size_y, size_x = imgarr.shape - subsize_y = int(size_y/nsect_y) - subsize_x = int(size_x/nsect_x) + subsize_y = int(size_y / nsect_y) + subsize_x = int(size_x / nsect_x) for i in range(nsect_y): for j in range(nsect_x): - subimg = imgarr[subsize_y*i:subsize_y*(i+1), subsize_x*j:subsize_x*(j+1)] + subimg = imgarr[ + subsize_y * i : subsize_y * (i + 1), subsize_x * j : subsize_x * (j + 1) + ] subimg = ChargeFlow(subimg, fullwell=fullwell) - imgarr[subsize_y*i:subsize_y*(i+1), subsize_x*j:subsize_x*(j+1)] = subimg + imgarr[ + subsize_y * i : subsize_y * (i + 1), subsize_x * j : subsize_x * (j + 1) + ] = subimg return GSImage + # Saturation & Bleeding End # @@ -453,53 +578,101 @@ def readout16(GSImage, rowi=0, coli=0, overscan_value=0): # ... # return: GS Image Object npix_y, npix_x = GSImage.array.shape - subheight = int(8+npix_y/2+8) - subwidth = int(16+npix_x/8+27) + subheight = int(8 + npix_y / 2 + 8) + subwidth = int(16 + npix_x / 8 + 27) OutputSubimg = galsim.ImageUS(subwidth, subheight, init_value=overscan_value) if rowi < 4 and coli == 0: - subbounds = galsim.BoundsI(1, int(npix_x/2), int(npix_y/8*rowi+1), int(npix_y/8*(rowi+1))) - subbounds = subbounds.shift(galsim.PositionI(GSImage.bounds.getXMin()-1, GSImage.bounds.getYMin()-1)) + subbounds = galsim.BoundsI( + 1, int(npix_x / 2), int(npix_y / 8 * rowi + 1), int(npix_y / 8 * (rowi + 1)) + ) + subbounds = subbounds.shift( + galsim.PositionI(GSImage.bounds.getXMin() - 1, GSImage.bounds.getYMin() - 1) + ) subimg = GSImage[subbounds] - OutputSubimg.array[27:int(npix_y/8)+27, 8:int(npix_x/2)+8] = subimg.array + OutputSubimg.array[27 : int(npix_y / 8) + 27, 8 : int(npix_x / 2) + 8] = ( + subimg.array + ) elif rowi < 4 and coli == 1: - subbounds = galsim.BoundsI(npix_x/2+1, npix_x, npix_y/8*rowi+1, npix_y/8*(rowi+1)) - subbounds = subbounds.shift(galsim.PositionI(GSImage.bounds.getXMin()-1, GSImage.bounds.getYMin()-1)) + subbounds = galsim.BoundsI( + npix_x / 2 + 1, npix_x, npix_y / 8 * rowi + 1, npix_y / 8 * (rowi + 1) + ) + subbounds = subbounds.shift( + galsim.PositionI(GSImage.bounds.getXMin() - 1, GSImage.bounds.getYMin() - 1) + ) subimg = GSImage[subbounds] - OutputSubimg.array[27:int(npix_y/8)+27, 8:int(npix_x/2)+8] = subimg.array + OutputSubimg.array[27 : int(npix_y / 8) + 27, 8 : int(npix_x / 2) + 8] = ( + subimg.array + ) elif rowi >= 4 and rowi < 8 and coli == 0: - subbounds = galsim.BoundsI(1, npix_x/2, npix_y/8*rowi+1, npix_y/8*(rowi+1)) - subbounds = subbounds.shift(galsim.PositionI(GSImage.bounds.getXMin()-1, GSImage.bounds.getYMin()-1)) + subbounds = galsim.BoundsI( + 1, npix_x / 2, npix_y / 8 * rowi + 1, npix_y / 8 * (rowi + 1) + ) + subbounds = subbounds.shift( + galsim.PositionI(GSImage.bounds.getXMin() - 1, GSImage.bounds.getYMin() - 1) + ) subimg = GSImage[subbounds] - OutputSubimg.array[16:int(npix_y/8)+16, 8:int(npix_x/2)+8] = subimg.array + OutputSubimg.array[16 : int(npix_y / 8) + 16, 8 : int(npix_x / 2) + 8] = ( + subimg.array + ) elif rowi >= 4 and rowi < 8 and coli == 1: - subbounds = galsim.BoundsI(npix_x/2+1, npix_x, npix_y/8*rowi+1, npix_y/8*(rowi+1)) - subbounds = subbounds.shift(galsim.PositionI(GSImage.bounds.getXMin()-1, GSImage.bounds.getYMin()-1)) + subbounds = galsim.BoundsI( + npix_x / 2 + 1, npix_x, npix_y / 8 * rowi + 1, npix_y / 8 * (rowi + 1) + ) + subbounds = subbounds.shift( + galsim.PositionI(GSImage.bounds.getXMin() - 1, GSImage.bounds.getYMin() - 1) + ) subimg = GSImage[subbounds] - OutputSubimg.array[16:int(npix_y/8)+16, 8:int(npix_x/2)+8] = subimg.array + OutputSubimg.array[16 : int(npix_y / 8) + 16, 8 : int(npix_x / 2) + 8] = ( + subimg.array + ) else: - print("\n\033[31mError: "+"Wrong rowi or coli assignment. Permitted: 0<=rowi<=7, 0<=coli<=1."+"\033[0m\n") + print( + "\n\033[31mError: " + + "Wrong rowi or coli assignment. Permitted: 0<=rowi<=7, 0<=coli<=1." + + "\033[0m\n" + ) return OutputSubimg return OutputSubimg -def CTE_Effect(GSImage, threshold=27, direction='column'): +def CTE_Effect(GSImage, threshold=27, direction="column"): # Devide the image into 4 sections and apply CTE effect with different trail directions. # GSImage: a GalSim Image object. size_y, size_x = GSImage.array.shape - size_sect_y = int(size_y/2) - size_sect_x = int(size_x/2) + size_sect_y = int(size_y / 2) + size_sect_x = int(size_x / 2) imgarr = GSImage.array - if direction == 'column': - imgarr[0:size_sect_y, :] = CTEModelColRow(imgarr[0:size_sect_y, :], trail_direction='down', direction='column', threshold=threshold) - imgarr[size_sect_y:size_y, :] = CTEModelColRow(imgarr[size_sect_y:size_y, :], trail_direction='up', direction='column', threshold=threshold) - elif direction == 'row': - imgarr[:, 0:size_sect_x] = CTEModelColRow(imgarr[:, 0:size_sect_x], trail_direction='right', direction='row', threshold=threshold) - imgarr[:, size_sect_x:size_x] = CTEModelColRow(imgarr[:, size_sect_x:size_x], trail_direction='left', direction='row', threshold=threshold) + if direction == "column": + imgarr[0:size_sect_y, :] = CTEModelColRow( + imgarr[0:size_sect_y, :], + trail_direction="down", + direction="column", + threshold=threshold, + ) + imgarr[size_sect_y:size_y, :] = CTEModelColRow( + imgarr[size_sect_y:size_y, :], + trail_direction="up", + direction="column", + threshold=threshold, + ) + elif direction == "row": + imgarr[:, 0:size_sect_x] = CTEModelColRow( + imgarr[:, 0:size_sect_x], + trail_direction="right", + direction="row", + threshold=threshold, + ) + imgarr[:, size_sect_x:size_x] = CTEModelColRow( + imgarr[:, size_sect_x:size_x], + trail_direction="left", + direction="row", + threshold=threshold, + ) return GSImage @jit() -def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): +def CTEModelColRow(img, trail_direction="up", direction="column", threshold=27): # total trail flux vs (pixel flux)^1/2 is approximately linear # total trail flux = trail_a * (pixel flux)^1/2 + trail_b @@ -509,7 +682,7 @@ def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): sh1 = img.shape[0] sh2 = img.shape[1] - n_img = img*0 + n_img = img * 0 idx = np.where(img < threshold) if len(idx[0]) == 0: pass @@ -523,7 +696,7 @@ def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): # print(index) for i, j in zip(yidx, xidx): f = img[i, j] - trail_f = (np.sqrt(f)*trail_a + trail_b)*0.5 + trail_f = (np.sqrt(f) * trail_a + trail_b) * 0.5 # trail_f=5E-5*f**1.5 xy_num = 10 all_trail = np.zeros(xy_num) @@ -542,7 +715,7 @@ def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): # t_pow = 0 am = 1 bm = 1 - t_pow = am*np.exp(-bm*m) + t_pow = am * np.exp(-bm * m) # if m < 5: # t_pow = a1*np.exp(-b1*m)+c1 # else: @@ -552,23 +725,23 @@ def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): all_trail_pix += t_pow all_trail[m] = t_pow - trail_pix_eff = trail_f/all_trail_pix - all_trail = trail_pix_eff*all_trail + trail_pix_eff = trail_f / all_trail_pix + all_trail = trail_pix_eff * all_trail all_trail[0] = f - trail_f for m in np.arange(0, xy_num, 1): - if direction == 'column': - if trail_direction == 'down': + if direction == "column": + if trail_direction == "down": y_pos = i + m - elif trail_direction == 'up': + elif trail_direction == "up": y_pos = i - m if y_pos < 0 or y_pos >= sh1: break n_img[y_pos, j] = n_img[y_pos, j] + all_trail[m] - elif direction == 'row': - if trail_direction == 'left': + elif direction == "row": + if trail_direction == "left": x_pos = j - m - elif trail_direction == 'right': + elif trail_direction == "right": x_pos = j + m if x_pos < 0 or x_pos >= sh2: break @@ -581,27 +754,27 @@ def CTEModelColRow(img, trail_direction='up', direction='column', threshold=27): # ---------- Zhang Xin ---------------------------- def getYValue(collection, x): index = 0 - if (collection.shape[1] == 2): - while (x > collection[index, 0] and index < collection.shape[0]): + if collection.shape[1] == 2: + while x > collection[index, 0] and index < collection.shape[0]: index = index + 1 - if (index == collection.shape[0] or index == 0): + if index == collection.shape[0] or index == 0: return 0 - deltX = collection[index, 0] - collection[index-1, 0] - deltY = collection[index, 1] - collection[index-1, 1] + deltX = collection[index, 0] - collection[index - 1, 0] + deltY = collection[index, 1] - collection[index - 1, 1] if deltX == 0: - return (collection[index, 1] + collection[index-1, 1])/2.0 + return (collection[index, 1] + collection[index - 1, 1]) / 2.0 else: - a = deltY/deltX - return a * (x - collection[index-1, 0]) + collection[index-1, 1] + a = deltY / deltX + return a * (x - collection[index - 1, 0]) + collection[index - 1, 1] return 0 def selectCosmicRayCollection(attachedSizes, xLen, yLen, cr_pixelRatio, CR_max_size): normalRay = 0.90 - nnormalRay = 1-normalRay + nnormalRay = 1 - normalRay max_nrayLen = 100 pixelNum = int(xLen * yLen * cr_pixelRatio * normalRay) pixelNum_n = int(xLen * yLen * cr_pixelRatio * nnormalRay) @@ -612,16 +785,16 @@ def selectCosmicRayCollection(attachedSizes, xLen, yLen, cr_pixelRatio, CR_max_s cr_event_num = 0 CRs = np.zeros(pixelNum) - while (CRPixelNum < pixelNum): + while CRPixelNum < pixelNum: x = CR_max_size * np.random.random() y = maxValue * np.random.random() - if (y <= getYValue(attachedSizes, x)): + if y <= getYValue(attachedSizes, x): CRs[cr_event_num] = np.ceil(x) cr_event_num = cr_event_num + 1 CRPixelNum = CRPixelNum + round(x) - while (CRPixelNum < pixelNum + pixelNum_n): - nx = np.random.random()*(max_nrayLen-CR_max_size)+CR_max_size + while CRPixelNum < pixelNum + pixelNum_n: + nx = np.random.random() * (max_nrayLen - CR_max_size) + CR_max_size CRs[cr_event_num] = np.ceil(nx) cr_event_num = cr_event_num + 1 CRPixelNum = CRPixelNum + np.ceil(nx) @@ -631,6 +804,7 @@ def selectCosmicRayCollection(attachedSizes, xLen, yLen, cr_pixelRatio, CR_max_s def defineEnergyForCR(cr_event_size, seed=12345): import random + sigma = 0.6 / 2.355 mean = 3.3 random.seed(seed) @@ -645,18 +819,18 @@ def convCR(CRmap=None, addPSF=None, sp_n=4): sh = CRmap.shape # sp_n = 4 - subCRmap = np.zeros(np.array(sh)*sp_n) - pix_v0 = 1/(sp_n*sp_n) + subCRmap = np.zeros(np.array(sh) * sp_n) + pix_v0 = 1 / (sp_n * sp_n) for i in np.arange(sh[0]): - i_st = sp_n*i + i_st = sp_n * i for j in np.arange(sh[1]): if CRmap[i, j] == 0: continue - j_st = sp_n*j - pix_v1 = CRmap[i, j]*pix_v0 + j_st = sp_n * j + pix_v1 = CRmap[i, j] * pix_v0 for m in np.arange(sp_n): for n in np.arange(sp_n): - subCRmap[i_st+m, j_st + n] = pix_v1 + subCRmap[i_st + m, j_st + n] = pix_v1 m_size = addPSF.shape[0] @@ -665,32 +839,36 @@ def convCR(CRmap=None, addPSF=None, sp_n=4): for i in np.arange(subCRmap.shape[0]): for j in np.arange(subCRmap.shape[1]): if subCRmap[i, j] > 0: - convPix = addPSF*subCRmap[i, j] - subCRmap_n[i:i+m_size, j:j+m_size] += convPix + convPix = addPSF * subCRmap[i, j] + subCRmap_n[i : i + m_size, j : j + m_size] += convPix - CRmap_n = np.zeros((np.array(subCRmap_n.shape)/sp_n).astype(np.int32)) + CRmap_n = np.zeros((np.array(subCRmap_n.shape) / sp_n).astype(np.int32)) sh_n = CRmap_n.shape for i in np.arange(sh_n[0]): - i_st = sp_n*i + i_st = sp_n * i for j in np.arange(sh_n[1]): p_v = 0 - j_st = sp_n*j + j_st = sp_n * j for m in np.arange(sp_n): for n in np.arange(sp_n): - p_v += subCRmap_n[i_st+m, j_st + n] + p_v += subCRmap_n[i_st + m, j_st + n] CRmap_n[i, j] = p_v return CRmap_n -def produceCR_Map(xLen, yLen, exTime, cr_pixelRatio, gain, attachedSizes, seed=20210317): +def produceCR_Map( + xLen, yLen, exTime, cr_pixelRatio, gain, attachedSizes, seed=20210317 +): # Return: an 2-D numpy array # attachedSizes = np.loadtxt('./wfc-cr-attachpixel.dat'); np.random.seed(seed) CR_max_size = 20.0 - cr_size = selectCosmicRayCollection(attachedSizes, xLen, yLen, cr_pixelRatio, CR_max_size) + cr_size = selectCosmicRayCollection( + attachedSizes, xLen, yLen, cr_pixelRatio, CR_max_size + ) cr_event_size = cr_size.shape[0] cr_energys = defineEnergyForCR(cr_event_size, seed=seed) @@ -699,16 +877,17 @@ def produceCR_Map(xLen, yLen, exTime, cr_pixelRatio, gain, attachedSizes, seed=2 # produce conv kernel from astropy.modeling.models import Gaussian2D + o_size = 4 sp_n = 8 - m_size = o_size*sp_n+1 - m_cen = o_size*sp_n/2 - sigma_psf = 0.2*sp_n + m_size = o_size * sp_n + 1 + m_cen = o_size * sp_n / 2 + sigma_psf = 0.2 * sp_n addPSF_ = Gaussian2D(1, m_cen, m_cen, sigma_psf, sigma_psf) yp, xp = np.mgrid[0:m_size, 0:m_size] addPSF = addPSF_(xp, yp) - convKernel = addPSF/addPSF.sum() + convKernel = addPSF / addPSF.sum() # --------------------------------- for i in np.arange(cr_event_size): @@ -717,16 +896,16 @@ def produceCR_Map(xLen, yLen, exTime, cr_pixelRatio, gain, attachedSizes, seed=2 cr_lens = int(cr_size[i]) if cr_lens == 0: continue - pix_energy = cr_energys[i]/gain/cr_lens - pos_angle = 1/2*math.pi*np.random.random() + pix_energy = cr_energys[i] / gain / cr_lens + pos_angle = 1 / 2 * math.pi * np.random.random() - crMatrix = np.zeros([cr_lens+1, cr_lens + 1]) + crMatrix = np.zeros([cr_lens + 1, cr_lens + 1]) for j in np.arange(cr_lens): - x_n = int(np.cos(pos_angle)*j - np.sin(pos_angle)*0) + x_n = int(np.cos(pos_angle) * j - np.sin(pos_angle) * 0) if x_n < 0: - x_n = x_n + cr_lens+1 - y_n = int(np.sin(pos_angle)*j + np.cos(pos_angle)*0) + x_n = x_n + cr_lens + 1 + y_n = int(np.sin(pos_angle) * j + np.cos(pos_angle) * 0) if x_n < 0 or x_n > cr_lens or y_n < 0 or y_n > cr_lens: continue crMatrix[y_n, x_n] = pix_energy @@ -741,13 +920,13 @@ def produceCR_Map(xLen, yLen, exTime, cr_pixelRatio, gain, attachedSizes, seed=2 okx = (xpix >= 0) & (xpix < sh[1]) oky = (ypix >= 0) & (ypix < sh[0]) - sly = slice(ypix[oky].min(), ypix[oky].max()+1) - slx = slice(xpix[okx].min(), xpix[okx].max()+1) + sly = slice(ypix[oky].min(), ypix[oky].max() + 1) + slx = slice(xpix[okx].min(), xpix[okx].max() + 1) CRmap[sly, slx] += crMatrix_n[oky, :][:, okx] return CRmap.astype(np.int32), cr_event_size -def ShutterEffectArr(GSImage, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1E-3): +def ShutterEffectArr(GSImage, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1e-3): # Generate Shutter-Effect normalized image # t_shutter: time of shutter movement # dist_bearing: distance between two bearings of shutter leaves @@ -755,14 +934,14 @@ def ShutterEffectArr(GSImage, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1E- from scipy import interpolate - SampleNumb = int(t_shutter/dt+1) - DistHalf = dist_bearing/2 + SampleNumb = int(t_shutter / dt + 1) + DistHalf = dist_bearing / 2 - t = np.arange(SampleNumb)*dt - a_arr = 5.84*np.sin(2*math.pi/t_shutter*t) + t = np.arange(SampleNumb) * dt + a_arr = 5.84 * np.sin(2 * math.pi / t_shutter * t) v = np.zeros(SampleNumb) theta = np.zeros(SampleNumb) - x = np.arange(SampleNumb)/(SampleNumb-1)*dist_bearing + x = np.arange(SampleNumb) / (SampleNumb - 1) * dist_bearing s = np.zeros(SampleNumb) s1 = np.zeros(SampleNumb) s2 = np.zeros(SampleNumb) @@ -775,21 +954,21 @@ def ShutterEffectArr(GSImage, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1E- v[0] = 0 theta[0] = 0 - for i in range(SampleNumb-1): - v[i+1] = v[i]+a_arr[i]*dt - theta[i+1] = theta[i]+v[i]*dt - s1[i] = DistHalf*np.cos(theta[i]) - s2[i] = dist_bearing-DistHalf*np.cos(theta[i]) - s1idx[i] = int(s1[i]/dist_bearing*(SampleNumb)) - s2idx[i] = int(s2[i]/dist_bearing*(SampleNumb)) + for i in range(SampleNumb - 1): + v[i + 1] = v[i] + a_arr[i] * dt + theta[i + 1] = theta[i] + v[i] * dt + s1[i] = DistHalf * np.cos(theta[i]) + s2[i] = dist_bearing - DistHalf * np.cos(theta[i]) + s1idx[i] = int(s1[i] / dist_bearing * (SampleNumb)) + s2idx[i] = int(s2[i] / dist_bearing * (SampleNumb)) brt[(idx > s1idx[i]) & (idx < s2idx[i])] += dt - if t_exp > t_shutter*2: - brt = brt*2+(t_exp-t_shutter*2) + if t_exp > t_shutter * 2: + brt = brt * 2 + (t_exp - t_shutter * 2) else: - brt = brt*2 + brt = brt * 2 - x = (x-dist_bearing/2)*100 + x = (x - dist_bearing / 2) * 100 intp = interpolate.splrep(x, brt, s=0) @@ -801,9 +980,9 @@ def ShutterEffectArr(GSImage, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1E- raise LookupError("Out of focal-plane bounds in X-direction.") if ymin < -25331 or ymax > 25331: raise LookupError("Out of focal-plane bounds in Y-direction.") - sizex = xmax-xmin+1 - sizey = ymax-ymin+1 - xnewgrid = np.mgrid[xmin:(xmin+sizex)] + sizex = xmax - xmin + 1 + sizey = ymax - ymin + 1 + xnewgrid = np.mgrid[xmin : (xmin + sizex)] expeffect = interpolate.splev(xnewgrid, intp, der=0) expeffect /= t_exp exparrnormal = np.tile(expeffect, (sizey, 1)) diff --git a/observation_sim/instruments/chip/libCTI/CTI_modeling.py b/observation_sim/instruments/chip/libCTI/CTI_modeling.py index fdf8b41a99c544f291f6970083ffaca0c9b49849..23133d9dfbc812a728b0fd65c7cdff25551be771 100644 --- a/observation_sim/instruments/chip/libCTI/CTI_modeling.py +++ b/observation_sim/instruments/chip/libCTI/CTI_modeling.py @@ -1,10 +1,5 @@ -from ctypes import CDLL, POINTER, c_int, c_double, c_float, c_long, c_char_p -from numpy.ctypeslib import ndpointer -import numpy.ctypeslib as clb +from ctypes import CDLL, POINTER, c_int, c_float import numpy as np -from astropy.io import fits -from scipy.stats import randint -from glob import glob from datetime import datetime import os @@ -13,10 +8,24 @@ lib_path = os.path.dirname(os.path.realpath(__file__)) # lib_path += "/add_CTI.so" lib_path += "/libmoduleCTI.so" lib = CDLL(lib_path) -CTI_simul = lib.__getattr__('CTI_simul') -CTI_simul.argtypes = [POINTER(POINTER(c_int)), c_int, c_int, c_int, c_int, POINTER(c_float), POINTER(c_float), - c_float, c_float, c_float, c_int, POINTER(c_int), c_int, POINTER(POINTER(c_int))] -''' +CTI_simul = lib.__getattr__("CTI_simul") +CTI_simul.argtypes = [ + POINTER(POINTER(c_int)), + c_int, + c_int, + c_int, + c_int, + POINTER(c_float), + POINTER(c_float), + c_float, + c_float, + c_float, + c_int, + POINTER(c_int), + c_int, + POINTER(POINTER(c_int)), +] +""" get_trap_h = lib.__getattr__('save_trap_map') get_trap_h.argtypes = [POINTER(c_int), c_int, c_int, c_int, c_int, POINTER(c_float), c_float, c_float, c_char_p] @@ -44,11 +53,11 @@ def bin2fits(bin_file,fits_dir,nsp,nx,ny,nmax): h[np.where(ntrap 0.001 + if hasattr(norm_filt, "bandpass_full"): + norm_filt = Table( + np.array( + np.array([ + norm_filt.bandpass_full.wave_list * 10.0, + norm_filt.bandpass_full.func( + norm_filt.bandpass_full.wave_list + ), + ]) + ).T, + names=(["WAVELENGTH", "SENSITIVITY"]), + ) + norm_thr_rang_ids = norm_filt["SENSITIVITY"] > 0.001 else: norm_filt = Table( - np.array(np.array([bandpass.wave_list*10.0, bandpass.func( - bandpass.wave_list)])).T, names=(['WAVELENGTH', 'SENSITIVITY']) + np.array( + np.array([ + bandpass.wave_list * 10.0, + bandpass.func(bandpass.wave_list), + ]) + ).T, + names=(["WAVELENGTH", "SENSITIVITY"]), ) - norm_thr_rang_ids = norm_filt['SENSITIVITY'] > 0.001 + norm_thr_rang_ids = norm_filt["SENSITIVITY"] > 0.001 - sedNormFactor = getNormFactorForSpecWithABMAG(ABMag=mag, - spectrum=sed, - norm_thr=norm_filt, - sWave=np.floor( - norm_filt[norm_thr_rang_ids][0][0]), - eWave=np.ceil(norm_filt[norm_thr_rang_ids][-1][0])) + sedNormFactor = getNormFactorForSpecWithABMAG( + ABMag=mag, + spectrum=sed, + norm_thr=norm_filt, + sWave=np.floor(norm_filt[norm_thr_rang_ids][0][0]), + eWave=np.ceil(norm_filt[norm_thr_rang_ids][-1][0]), + ) sed_photon = copy.copy(sed) - sed_photon = np.array( - [sed_photon['WAVELENGTH'], sed_photon['FLUX']*sedNormFactor]).T - sed_photon = galsim.LookupTable(x=np.array(sed_photon[:, 0]), f=np.array( - sed_photon[:, 1] * mu), interpolant='nearest') + sed_photon = np.array([ + sed_photon["WAVELENGTH"], + sed_photon["FLUX"] * sedNormFactor, + ]).T + sed_photon = galsim.LookupTable( + x=np.array(sed_photon[:, 0]), + f=np.array(sed_photon[:, 1] * mu), + interpolant="nearest", + ) # Get magnitude - sed_photon = galsim.SED(sed_photon, wave_type='A', - flux_type='1', fast=False) + sed_photon = galsim.SED(sed_photon, wave_type="A", flux_type="1", fast=False) interFlux = integrate_sed_bandpass(sed=sed_photon, bandpass=bandpass) - mag_csst = getABMAG( - interFlux=interFlux, - bandpass=bandpass - ) + mag_csst = getABMAG(interFlux=interFlux, bandpass=bandpass) if target_filt.survey_type == "photometric": return sed_photon, mag_csst, interFlux elif target_filt.survey_type == "spectroscopic": diff --git a/observation_sim/mock_objects/ExtinctionMW.py b/observation_sim/mock_objects/ExtinctionMW.py index 85ae6928b35d9d4e80fb4f84de0d2a21efcc10ed..4a0c017b43ffbef11653a8ac88945378265c6fbf 100644 --- a/observation_sim/mock_objects/ExtinctionMW.py +++ b/observation_sim/mock_objects/ExtinctionMW.py @@ -1,4 +1,3 @@ -import os import numpy as np import healpy as hp from astropy.io import fits @@ -9,11 +8,13 @@ from astropy.coordinates import SkyCoord class ExtinctionMW(object): def __init__(self): self.Rv = 3.1 - self.lamb = np.arange(2000, 11001+0.5, 0.5) + self.lamb = np.arange(2000, 11001 + 0.5, 0.5) @staticmethod def radec2pix(ra, dec, NSIDE=2048): - return hp.pixelfunc.ang2pix(nside=NSIDE, theta=ra, phi=dec, nest=True, lonlat=True) + return hp.pixelfunc.ang2pix( + nside=NSIDE, theta=ra, phi=dec, nest=True, lonlat=True + ) def init_ext_model(self, model_name="odonnell", Av=1.0, Rv=3.1, lamb=None): self.model_name = model_name @@ -25,44 +26,93 @@ class ExtinctionMW(object): if self.model_name == "odonnell": alpha = np.zeros(self.lamb.shape) beta = np.zeros(self.lamb.shape) - x = 1.e4 / self.lamb - def f_alpha_1(x): return 0.574 * (x**1.61) - def f_beta_1(x): return -0.527 * (x**1.61) + x = 1.0e4 / self.lamb + + def f_alpha_1(x): + return 0.574 * (x**1.61) + + def f_beta_1(x): + return -0.527 * (x**1.61) def f_alpha_2(x): y = x - 1.82 - return 1 + 0.104*y - 0.609*(y**2) + 0.701*(y**3) + \ - 1.137*(y**4) - 1.718*(y**5) - 0.827 * \ - (y**6) + 1.647*(y**7) - 0.505*(y**8) + return ( + 1 + + 0.104 * y + - 0.609 * (y**2) + + 0.701 * (y**3) + + 1.137 * (y**4) + - 1.718 * (y**5) + - 0.827 * (y**6) + + 1.647 * (y**7) + - 0.505 * (y**8) + ) def f_beta_2(x): y = x - 1.82 - return 1.952*y + 2.908*(y**2) - 3.989*(y**3) - 7.985 * \ - (y**4) + 11.102*(y**5) + 5.491 * \ - (y**6) - 10.805*(y**7) + 3.347*(y**8) - - def f_alpha_3(x): return 1.752 - 0.316*x - \ - 0.104 / ((x - 4.67)**2 + 0.341) - - def f_beta_3(x): return -3.090 + 1.825*x + \ - 1.206 / ((x - 4.62)**2 + 0.262) - - def f_alpha_4(x): return f_alpha_3(x) + -0.04473 * \ - (x - 5.9)**2 - 0.009779 * (x - 5.9)**3 - - def f_beta_4(x): return f_beta_3(x) + 0.2130 * \ - (x - 5.9)**2 + 0.1207 * (x - 5.9)**3 - - def f_alpha_5(x): return -1.073 - 0.628*(x - 8) + \ - 0.137*(x - 8)**2 - 0.070*(x - 8)**3 - - def f_beta_5(x): return 13.670 + 4.257*(x - 8) - \ - 0.420*(x - 8)**2 + 0.374 * (x - 8)**3 + return ( + 1.952 * y + + 2.908 * (y**2) + - 3.989 * (y**3) + - 7.985 * (y**4) + + 11.102 * (y**5) + + 5.491 * (y**6) + - 10.805 * (y**7) + + 3.347 * (y**8) + ) + + def f_alpha_3(x): + return 1.752 - 0.316 * x - 0.104 / ((x - 4.67) ** 2 + 0.341) + + def f_beta_3(x): + return -3.090 + 1.825 * x + 1.206 / ((x - 4.62) ** 2 + 0.262) + + def f_alpha_4(x): + return ( + f_alpha_3(x) + -0.04473 * (x - 5.9) ** 2 - 0.009779 * (x - 5.9) ** 3 + ) + + def f_beta_4(x): + return f_beta_3(x) + 0.2130 * (x - 5.9) ** 2 + 0.1207 * (x - 5.9) ** 3 + + def f_alpha_5(x): + return ( + -1.073 + - 0.628 * (x - 8) + + 0.137 * (x - 8) ** 2 + - 0.070 * (x - 8) ** 3 + ) + + def f_beta_5(x): + return ( + 13.670 + + 4.257 * (x - 8) + - 0.420 * (x - 8) ** 2 + + 0.374 * (x - 8) ** 3 + ) alpha = np.piecewise( - x, [x <= 1.1, (x > 1.1)*(x <= 3.3), (x > 3.3)*(x <= 5.9), (x > 5.9)*(x <= 8.), (x > 8.)], [f_alpha_1, f_alpha_2, f_alpha_3, f_alpha_4, f_alpha_5]) + x, + [ + x <= 1.1, + (x > 1.1) * (x <= 3.3), + (x > 3.3) * (x <= 5.9), + (x > 5.9) * (x <= 8.0), + (x > 8.0), + ], + [f_alpha_1, f_alpha_2, f_alpha_3, f_alpha_4, f_alpha_5], + ) beta = np.piecewise( - x, [x <= 1.1, (x > 1.1)*(x <= 3.3), (x > 3.3)*(x <= 5.9), (x > 5.9)*(x <= 8.), (x > 8.)], [f_beta_1, f_beta_2, f_beta_3, f_beta_4, f_beta_5]) + x, + [ + x <= 1.1, + (x > 1.1) * (x <= 3.3), + (x > 3.3) * (x <= 5.9), + (x > 5.9) * (x <= 8.0), + (x > 8.0), + ], + [f_beta_1, f_beta_2, f_beta_3, f_beta_4, f_beta_5], + ) self.ext = (alpha + beta / self.Rv) * self.Av @@ -71,26 +121,27 @@ class ExtinctionMW(object): self.ebv_planck = hdu[1].data["EBV"] def Av_from_Planck(self, ra, dec): - if not hasattr(self, 'ebv_planck'): - raise ValueError( - "Need to load planck dust map first") + if not hasattr(self, "ebv_planck"): + raise ValueError("Need to load planck dust map first") # Convert to galactic coordinates - c = SkyCoord(ra=ra * u.degree, dec=dec * - u.degree, frame='icrs').galactic - l, b = c.l.radian, c.b.radian + c = SkyCoord(ra=ra * u.degree, dec=dec * u.degree, frame="icrs").galactic + phi, b = c.l.radian, c.b.radian NSIDE = hp.pixelfunc.get_nside(self.ebv_planck) pix = hp.pixelfunc.ang2pix( - nside=NSIDE, theta=np.pi/2. - b, phi=l, nest=True) + nside=NSIDE, theta=np.pi / 2.0 - b, phi=phi, nest=True + ) return self.ebv_planck[pix] * self.Rv def apply_extinction(self, spec, Av=1.0): if len(spec) != len(self.lamb): raise ValueError( - "Dimension of spec do not match that of ExtinctionMW.lamb: try initilize (init_ext_model) by the actual size of spec which you want to apply the extinction to") - if not hasattr(self, 'ext'): + "Dimension of spec do not match that of ExtinctionMW.lamb: try initilize (init_ext_model) by the actual size of spec which you want to apply the extinction to" + ) + if not hasattr(self, "ext"): raise ValueError( - "Need to initialize the extinction model (init_ext_model) first") - scale = 10**(-.4 * self.ext * Av) + "Need to initialize the extinction model (init_ext_model) first" + ) + scale = 10 ** (-0.4 * self.ext * Av) # print("scale = ", scale) spec *= scale return spec diff --git a/observation_sim/mock_objects/FlatLED.py b/observation_sim/mock_objects/FlatLED.py index 2674aa5c6ff8659db1c06a96e070d74db9c61d7d..2fca187fd2cbf69557cf435dacddb5fe51a17b8e 100755 --- a/observation_sim/mock_objects/FlatLED.py +++ b/observation_sim/mock_objects/FlatLED.py @@ -1,12 +1,8 @@ - - import galsim import os -import sys import numpy as np import time import math -import astropy.constants as cons from astropy.io import fits from scipy.interpolate import griddata from astropy.table import Table @@ -24,16 +20,71 @@ except ImportError: # flatDir = '/Volumes/EAGET/LED_FLAT/' -LED_name = ['LED1', 'LED2', 'LED3', 'LED4', 'LED5', 'LED6', 'LED7', 'LED8', 'LED9', 'LED10', 'LED11', 'LED12', 'LED13', - 'LED14'] -cwaves_name = {'LED1': '275', 'LED2': '310', 'LED3': '430', 'LED4': '505', 'LED5': '545', 'LED6': '590', 'LED7': '670', - 'LED8': '760', 'LED9': '940', 'LED10': '940', 'LED11': '1050', 'LED12': '1050', - 'LED13': '340', 'LED14': '365'} - -cwaves = {'LED1': 2750, 'LED2': 3100, 'LED3': 4300, 'LED4': 5050, 'LED5': 5250, 'LED6': 5900, 'LED7': 6700, - 'LED8': 7600, 'LED9': 8800, 'LED10': 9400, 'LED11': 10500, 'LED12': 15500, 'LED13': 3400, 'LED14': 3650} -cwaves_fwhm = {'LED1': 110, 'LED2': 120, 'LED3': 200, 'LED4': 300, 'LED5': 300, 'LED6': 130, 'LED7': 210, - 'LED8': 260, 'LED9': 400, 'LED10': 370, 'LED11': 500, 'LED12': 1400, 'LED13': 90, 'LED14': 100} +LED_name = [ + "LED1", + "LED2", + "LED3", + "LED4", + "LED5", + "LED6", + "LED7", + "LED8", + "LED9", + "LED10", + "LED11", + "LED12", + "LED13", + "LED14", +] +cwaves_name = { + "LED1": "275", + "LED2": "310", + "LED3": "430", + "LED4": "505", + "LED5": "545", + "LED6": "590", + "LED7": "670", + "LED8": "760", + "LED9": "940", + "LED10": "940", + "LED11": "1050", + "LED12": "1050", + "LED13": "340", + "LED14": "365", +} + +cwaves = { + "LED1": 2750, + "LED2": 3100, + "LED3": 4300, + "LED4": 5050, + "LED5": 5250, + "LED6": 5900, + "LED7": 6700, + "LED8": 7600, + "LED9": 8800, + "LED10": 9400, + "LED11": 10500, + "LED12": 15500, + "LED13": 3400, + "LED14": 3650, +} +cwaves_fwhm = { + "LED1": 110, + "LED2": 120, + "LED3": 200, + "LED4": 300, + "LED5": 300, + "LED6": 130, + "LED7": 210, + "LED8": 260, + "LED9": 400, + "LED10": 370, + "LED11": 500, + "LED12": 1400, + "LED13": 90, + "LED14": 100, +} # LED_QE = {'LED1': 0.3, 'LED2': 0.4, 'LED13': 0.5, 'LED14': 0.5, 'LED10': 0.4} # e-/ms # fluxLED = {'LED1': 0.16478729, 'LED2': 0.084220931, 'LED3': 2.263360617, 'LED4': 2.190623489, 'LED5': 0.703504768, @@ -42,16 +93,40 @@ cwaves_fwhm = {'LED1': 110, 'LED2': 120, 'LED3': 200, 'LED4': 300, 'LED5': 300, # 'LED14': 0.27842925} # e-/ms -fluxLED = {'LED1': 15, 'LED2': 15, 'LED3': 12.5, 'LED4': 9, 'LED5': 9, - 'LED6': 9, 'LED7': 9, 'LED8': 9, 'LED9': 9, 'LED10': 12.5, 'LED11': 15, 'LED12': 15, 'LED13': 12.5, - 'LED14': 12.5} +fluxLED = { + "LED1": 15, + "LED2": 15, + "LED3": 12.5, + "LED4": 9, + "LED5": 9, + "LED6": 9, + "LED7": 9, + "LED8": 9, + "LED9": 9, + "LED10": 12.5, + "LED11": 15, + "LED12": 15, + "LED13": 12.5, + "LED14": 12.5, +} # fluxLEDL = {'LED1': 10, 'LED2': 10, 'LED3': 10, 'LED4': 10, 'LED5': 10, # 'LED6': 10, 'LED7': 10, 'LED8': 10, 'LED9': 10, 'LED10': 10, 'LED11': 10, 'LED12':10, 'LED13': 10, # 'LED14': 10} -mirro_eff = {'GU': 0.61, 'GV': 0.8, 'GI': 0.8} - -bandtoLed = {'NUV': ['LED1', 'LED2'], 'u': ['LED13', 'LED14'], 'g': ['LED3', 'LED4', 'LED5'], 'r': ['LED6', 'LED7'], 'i': ['LED8'], 'z': ['LED9', 'LED10'], 'y': ['LED10'], 'GU': ['LED1', 'LED2', 'LED13', 'LED14'], 'GV': ['LED3', 'LED4', 'LED5', 'LED6'], 'GI': ['LED7', 'LED8', 'LED9', 'LED10']} +mirro_eff = {"GU": 0.61, "GV": 0.8, "GI": 0.8} + +bandtoLed = { + "NUV": ["LED1", "LED2"], + "u": ["LED13", "LED14"], + "g": ["LED3", "LED4", "LED5"], + "r": ["LED6", "LED7"], + "i": ["LED8"], + "z": ["LED9", "LED10"], + "y": ["LED10"], + "GU": ["LED1", "LED2", "LED13", "LED14"], + "GV": ["LED3", "LED4", "LED5", "LED6"], + "GI": ["LED7", "LED8", "LED9", "LED10"], +} # mirro_eff = {'GU':1, 'GV':1, 'GI':1} @@ -65,10 +140,14 @@ class FlatLED(MockObject): self.flatDir = flatDir else: try: - with pkg_resources.files('observation_sim.mock_objects.data.led').joinpath("") as ledDir: + with pkg_resources.files( + "observation_sim.mock_objects.data.led" + ).joinpath("") as ledDir: self.flatDir = ledDir.as_posix() except AttributeError: - with pkg_resources.path('observation_sim.mock_objects.data.led', "") as ledDir: + with pkg_resources.path( + "observation_sim.mock_objects.data.led", "" + ) as ledDir: self.flatDir = ledDir.as_posix() def getInnerFlat(self): @@ -76,17 +155,18 @@ class FlatLED(MockObject): iFlat = np.zeros([self.chip.npix_y, self.chip.npix_x]) for nled in ledflats: iFlat = iFlat + self.getLEDImage1(led_type=nled, LED_Img_flag=False) - iFlat = iFlat/len(ledflats) + iFlat = iFlat / len(ledflats) return iFlat ### # return LED flat, e/s ### - def getLEDImage(self, led_type='LED1', LED_Img_flag=True): + def getLEDImage(self, led_type="LED1", LED_Img_flag=True): # cwave = cwaves[led_type] - flat = fits.open(os.path.join(self.flatDir, 'model_' + - cwaves_name[led_type] + 'nm.fits')) - xlen = flat[0].header['NAXIS1'] + flat = fits.open( + os.path.join(self.flatDir, "model_" + cwaves_name[led_type] + "nm.fits") + ) + xlen = flat[0].header["NAXIS1"] ylen = 601 x = np.linspace(0, self.chip.npix_x * 6, xlen) y = np.linspace(0, self.chip.npix_y * 5, ylen) @@ -106,36 +186,50 @@ class FlatLED(MockObject): i = self.chip.rowID - 1 j = self.chip.colID - 1 - U = griddata(X_, Z_, ( - M[self.chip.npix_y * i:self.chip.npix_y * - (i + 1), self.chip.npix_x * j:self.chip.npix_x * (j + 1)], - N[self.chip.npix_y * i:self.chip.npix_y * (i + 1), self.chip.npix_x * j:self.chip.npix_x * (j + 1)]), - method='linear') - U = U/np.mean(U) + U = griddata( + X_, + Z_, + ( + M[ + self.chip.npix_y * i : self.chip.npix_y * (i + 1), + self.chip.npix_x * j : self.chip.npix_x * (j + 1), + ], + N[ + self.chip.npix_y * i : self.chip.npix_y * (i + 1), + self.chip.npix_x * j : self.chip.npix_x * (j + 1), + ], + ), + method="linear", + ) + U = U / np.mean(U) flatImage = U if LED_Img_flag: - flatImage = flatImage*fluxLED[led_type]*1000 + flatImage = flatImage * fluxLED[led_type] * 1000 gc.collect() return flatImage ### + # return LED flat, e/s ### - def getLEDImage1(self, led_type='LED1', LED_Img_flag=True): + def getLEDImage1(self, led_type="LED1", LED_Img_flag=True): # cwave = cwaves[led_type] - flat = fits.open(os.path.join(self.flatDir, 'model_' + - cwaves_name[led_type] + 'nm.fits')) - xlen = flat[0].header['NAXIS1'] + flat = fits.open( + os.path.join(self.flatDir, "model_" + cwaves_name[led_type] + "nm.fits") + ) + xlen = flat[0].header["NAXIS1"] ylen = 601 i = self.chip.rowID - 1 j = self.chip.colID - 1 - x = np.linspace(0, self.chip.npix_x, int(xlen/6.)) - y = np.linspace(0, self.chip.npix_y, int(ylen/5.)) + x = np.linspace(0, self.chip.npix_x, int(xlen / 6.0)) + y = np.linspace(0, self.chip.npix_y, int(ylen / 5.0)) xx, yy = np.meshgrid(x, y) - a1 = flat[0].data[int(ylen*i/5.):int(ylen*i/5.)+int(ylen/5.), - int(xlen*j/6.):int(xlen*j/6.)+int(xlen/6.)] + a1 = flat[0].data[ + int(ylen * i / 5.0) : int(ylen * i / 5.0) + int(ylen / 5.0), + int(xlen * j / 6.0) : int(xlen * j / 6.0) + int(xlen / 6.0), + ] # z = np.sin((xx+yy+xx**2+yy**2)) # fInterp = interp2d(xx, yy, z, kind='linear') X_ = np.hstack((xx.flatten()[:, None], yy.flatten()[:, None])) @@ -145,34 +239,65 @@ class FlatLED(MockObject): M, N = np.meshgrid(n_x, n_y) x_seg_len = 4 y_seg_len = 8 - x_seg = int(self.chip.npix_x/x_seg_len) - y_seg = int(self.chip.npix_y/y_seg_len) + x_seg = int(self.chip.npix_x / x_seg_len) + y_seg = int(self.chip.npix_y / y_seg_len) U = np.zeros([self.chip.npix_y, self.chip.npix_x], dtype=np.float32) for y_seg_i in np.arange(y_seg_len): for x_seg_i in np.arange(x_seg_len): - U[y_seg_i*y_seg:(y_seg_i+1)*y_seg, x_seg_i*x_seg:(x_seg_i+1)*x_seg] = griddata(X_, Z_, (M[y_seg_i*y_seg:(y_seg_i+1)*y_seg, x_seg_i*x_seg:(x_seg_i+1)*x_seg], N[y_seg_i*y_seg:(y_seg_i+1)*y_seg, x_seg_i*x_seg:(x_seg_i+1)*x_seg]), method='linear') + U[ + y_seg_i * y_seg : (y_seg_i + 1) * y_seg, + x_seg_i * x_seg : (x_seg_i + 1) * x_seg, + ] = griddata( + X_, + Z_, + ( + M[ + y_seg_i * y_seg : (y_seg_i + 1) * y_seg, + x_seg_i * x_seg : (x_seg_i + 1) * x_seg, + ], + N[ + y_seg_i * y_seg : (y_seg_i + 1) * y_seg, + x_seg_i * x_seg : (x_seg_i + 1) * x_seg, + ], + ), + method="linear", + ) # U = griddata(X_, Z_, ( # M[0:self.chip.npix_y, 0:self.chip.npix_x], # N[0:self.chip.npix_y, 0:self.chip.npix_x]), # method='nearest').astype(np.float32) - U = U/np.mean(U) + U = U / np.mean(U) flatImage = U if LED_Img_flag: - flatImage = U*fluxLED[led_type]*1000 + flatImage = U * fluxLED[led_type] * 1000 gc.collect() return flatImage - def drawObj_LEDFlat_img(self, led_type_list=['LED1'], exp_t_list=[0.1]): + def drawObj_LEDFlat_img(self, led_type_list=["LED1"], exp_t_list=[0.1]): if len(led_type_list) > len(exp_t_list): return np.ones([self.chip.npix_y, self.chip.npix_x]) ledFlat = np.zeros([self.chip.npix_y, self.chip.npix_x]) - ledStat = '00000000000000' - ledTimes = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, - 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] - - nledStat = '2' + ledStat = "00000000000000" + ledTimes = [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ] + + nledStat = "2" for i in np.arange(len(led_type_list)): led_type = led_type_list[i] exp_t = exp_t_list[i] @@ -182,39 +307,54 @@ class FlatLED(MockObject): led_wave = cwaves[led_type] led_fwhm = cwaves_fwhm[led_type] led_spec = self.gaussian1d_profile_led(led_wave, led_fwhm) - speci = interpolate.interp1d( - led_spec['WAVELENGTH'], led_spec['FLUX']) - w_list = np.arange(self.filt.blue_limit, - self.filt.red_limit, 0.5) # A + speci = interpolate.interp1d(led_spec["WAVELENGTH"], led_spec["FLUX"]) + w_list = np.arange(self.filt.blue_limit, self.filt.red_limit, 0.5) # A f_spec = speci(w_list) ccd_bp = self.chip._getChipEffCurve(self.chip.filter_type) - ccd_eff = ccd_bp.__call__(w_list / 10.) + ccd_eff = ccd_bp.__call__(w_list / 10.0) filt_bp = self.filt.filter_bandpass - fil_eff = filt_bp.__call__(w_list / 10.) - t_spec = np.trapz(f_spec*ccd_eff*fil_eff, w_list) + fil_eff = filt_bp.__call__(w_list / 10.0) + t_spec = np.trapz(f_spec * ccd_eff * fil_eff, w_list) # print(i, np.mean(unitFlatImg), t_spec, exp_t) unitFlatImg = unitFlatImg * t_spec # print("DEBUG1:---------------",np.mean(unitFlatImg)) - ledFlat = ledFlat+unitFlatImg*exp_t - - ledStat = ledStat[0:int(led_type[3:])-1] + \ - nledStat+ledStat[int(led_type[3:]):] - ledTimes[int(led_type[3:])-1] = exp_t * 1000 + ledFlat = ledFlat + unitFlatImg * exp_t + + ledStat = ( + ledStat[0 : int(led_type[3:]) - 1] + + nledStat + + ledStat[int(led_type[3:]) :] + ) + ledTimes[int(led_type[3:]) - 1] = exp_t * 1000 gc.collect() return ledFlat, ledStat, ledTimes - def drawObj_LEDFlat_slitless(self, led_type_list=['LED1'], exp_t_list=[0.1]): + def drawObj_LEDFlat_slitless(self, led_type_list=["LED1"], exp_t_list=[0.1]): if len(led_type_list) != len(exp_t_list): return np.ones([self.chip.npix_y, self.chip.npix_x]) ledFlat = np.zeros([self.chip.npix_y, self.chip.npix_x]) - ledStat = '00000000000000' - ledTimes = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, - 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] - - nledStat = '2' + ledStat = "00000000000000" + ledTimes = [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ] + + nledStat = "2" for i in np.arange(len(led_type_list)): led_type = led_type_list[i] @@ -222,7 +362,7 @@ class FlatLED(MockObject): # unitFlatImg = self.getLEDImage(led_type=led_type) unitFlatImg = self.getLEDImage1(led_type=led_type) # print("---------------TEST mem:",np.mean(unitFlatImg)) - ledFlat_ = unitFlatImg*exp_t + ledFlat_ = unitFlatImg * exp_t ledFlat_ = ledFlat_ / mirro_eff[self.filt.filter_type] ledFlat_.astype(np.float32) led_wave = cwaves[led_type] @@ -236,41 +376,63 @@ class FlatLED(MockObject): conf=self.chip.sls_conf, pixelSize=self.chip.pix_scale, isAlongY=0, - flat_cube=self.chip.flat_cube, led_spec=led_spec) + flat_cube=self.chip.flat_cube, + led_spec=led_spec, + ) ledFlat = ledFlat + ledspec_map - ledStat = ledStat[0:int(led_type[3:])-1] + \ - nledStat+ledStat[int(led_type[3:]):] - ledTimes[int(led_type[3:])-1] = exp_t * 1000 + ledStat = ( + ledStat[0 : int(led_type[3:]) - 1] + + nledStat + + ledStat[int(led_type[3:]) :] + ) + ledTimes[int(led_type[3:]) - 1] = exp_t * 1000 return ledFlat, ledStat, ledTimes - def drawObj_LEDFlat(self, led_type_list=['LED1'], exp_t_list=[0.1]): + def drawObj_LEDFlat(self, led_type_list=["LED1"], exp_t_list=[0.1]): if self.chip.survey_type == "photometric": - return self.drawObj_LEDFlat_img(led_type_list=led_type_list, exp_t_list=exp_t_list) + return self.drawObj_LEDFlat_img( + led_type_list=led_type_list, exp_t_list=exp_t_list + ) elif self.chip.survey_type == "spectroscopic": - return self.drawObj_LEDFlat_slitless(led_type_list=led_type_list, exp_t_list=exp_t_list) + return self.drawObj_LEDFlat_slitless( + led_type_list=led_type_list, exp_t_list=exp_t_list + ) def gaussian1d_profile_led(self, xc=5050, fwhm=300): - sigma = fwhm/2.355 - x_radii = int(5*sigma + 1) - xlist = np.arange(xc-x_radii, xc+x_radii, 0.5) + sigma = fwhm / 2.355 + x_radii = int(5 * sigma + 1) + xlist = np.arange(xc - x_radii, xc + x_radii, 0.5) xlist_ = np.zeros(len(xlist) + 2) xlist_[1:-1] = xlist xlist_[0] = 2000 xlist_[-1] = 18000 - ids1 = xlist > xc-fwhm - ids2 = xlist[ids1] < xc+fwhm - data = np.exp((-(xlist-xc)*(xlist-xc))/(2*sigma*sigma)) / \ - (np.sqrt(2*math.pi)*sigma) - scale = 1/np.trapz(data[ids1][ids2], xlist[ids1][ids2]) + ids1 = xlist > xc - fwhm + ids2 = xlist[ids1] < xc + fwhm + data = np.exp((-(xlist - xc) * (xlist - xc)) / (2 * sigma * sigma)) / ( + np.sqrt(2 * math.pi) * sigma + ) + scale = 1 / np.trapz(data[ids1][ids2], xlist[ids1][ids2]) data_ = np.zeros(len(xlist) + 2) - data_[1:-1] = data*scale + data_[1:-1] = data * scale # print("DEBUG:-------------------------------",np.sum(data_), scale) - return Table(np.array([xlist_.astype(np.float32), data_.astype(np.float32)]).T, names=('WAVELENGTH', 'FLUX')) - - def calculateLEDSpec(self, skyMap=None, blueLimit=4200, redLimit=6500, - conf=[''], pixelSize=0.074, isAlongY=0, - split_pos=3685, flat_cube=None, led_spec=None): + return Table( + np.array([xlist_.astype(np.float32), data_.astype(np.float32)]).T, + names=("WAVELENGTH", "FLUX"), + ) + + def calculateLEDSpec( + self, + skyMap=None, + blueLimit=4200, + redLimit=6500, + conf=[""], + pixelSize=0.074, + isAlongY=0, + split_pos=3685, + flat_cube=None, + led_spec=None, + ): conf1 = conf[0] conf2 = conf[0] @@ -318,23 +480,29 @@ class FlatLED(MockObject): sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s + sub_y_e) / 2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 sub_x_e = sub_x_end_arr[j] skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, - origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf2, - flat_cube=flat_cube) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf2, + flat_cube=flat_cube, + ) spec_orders = sdp.compute_spec_orders() @@ -350,7 +518,6 @@ class FlatLED(MockObject): fImg[bounds] = fImg[bounds] + ssImg[bounds] else: - # sdp.compute_spec_orders() y_len = skyMap.shape[0] x_len = skyMap.shape[1] @@ -370,7 +537,7 @@ class FlatLED(MockObject): sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s + sub_y_e) / 2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 @@ -378,16 +545,22 @@ class FlatLED(MockObject): # print(i,j,sub_y_s, sub_y_e,sub_x_s,sub_x_e) T1 = time.time() skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, - origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf1, - flat_cube=flat_cube) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf1, + flat_cube=flat_cube, + ) spec_orders = sdp.compute_spec_orders() @@ -404,7 +577,7 @@ class FlatLED(MockObject): T2 = time.time() - print('time: %s ms' % ((T2 - T1) * 1000)) + print("time: %s ms" % ((T2 - T1) * 1000)) delt_x = x_len - split_pos sub_x_start_arr = np.arange(split_pos, x_len, delt_x) @@ -415,7 +588,7 @@ class FlatLED(MockObject): sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s + sub_y_e) / 2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 @@ -425,16 +598,22 @@ class FlatLED(MockObject): T1 = time.time() skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, - origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf2, - flat_cube=flat_cube) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf2, + flat_cube=flat_cube, + ) spec_orders = sdp.compute_spec_orders() @@ -450,11 +629,12 @@ class FlatLED(MockObject): fImg[bounds] = fImg[bounds] + ssImg[bounds] T2 = time.time() - print('time: %s ms' % ((T2 - T1) * 1000)) + print("time: %s ms" % ((T2 - T1) * 1000)) if isAlongY == 1: fimg, tmx, tmy = SpecDisperser.rotate90( - array_orig=fImg.array, xc=0, yc=0, isClockwise=0) + array_orig=fImg.array, xc=0, yc=0, isClockwise=0 + ) else: fimg = fImg.array diff --git a/observation_sim/mock_objects/Galaxy.py b/observation_sim/mock_objects/Galaxy.py index 97df60cdf82ff9ad353ce95db490ba72a593d877..a477a6d4f1c28fe1a9a0245c91a64f9949a10e08 100755 --- a/observation_sim/mock_objects/Galaxy.py +++ b/observation_sim/mock_objects/Galaxy.py @@ -2,7 +2,11 @@ import numpy as np import galsim from astropy.table import Table -from observation_sim.mock_objects._util import eObs, integrate_sed_bandpass, getNormFactorForSpecWithABMAG +from observation_sim.mock_objects._util import ( + eObs, + integrate_sed_bandpass, + getNormFactorForSpecWithABMAG, +) from observation_sim.mock_objects.SpecDisperser import SpecDisperser from observation_sim.mock_objects.MockObject import MockObject @@ -14,32 +18,42 @@ class Galaxy(MockObject): super().__init__(param, logger=logger) if not hasattr(self, "disk_sersic_idx"): - self.disk_sersic_idx = 1. + self.disk_sersic_idx = 1.0 if not hasattr(self, "bulge_sersic_idx"): - self.bulge_sersic_idx = 4. + self.bulge_sersic_idx = 4.0 if not hasattr(self, "mu"): if hasattr(self, "detA"): - self.mu = 1./self.detA + self.mu = 1.0 / self.detA else: - self.mu = 1. + self.mu = 1.0 def unload_SED(self): - """(Test) free up SED memory - """ + """(Test) free up SED memory""" del self.sed - def getGSObj_multiband(self, tel, psf_list, bandpass_list, filt, nphotons_tot=None, g1=0, g2=0, exptime=150., fd_shear=None): + def getGSObj_multiband( + self, + tel, + psf_list, + bandpass_list, + filt, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + fd_shear=None, + ): if len(psf_list) != len(bandpass_list): raise ValueError( - "!!!The number of PSF profiles and the number of bandpasses must be equal.") + "!!!The number of PSF profiles and the number of bandpasses must be equal." + ) objs = [] if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) # print("nphotons_tot = ", nphotons_tot) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) if self.logger: @@ -55,19 +69,21 @@ class Galaxy(MockObject): self.logger.error(e) return -1 - ratio = sub/full + ratio = sub / full if not (ratio == -1 or (ratio != ratio)): nphotons = ratio * nphotons_tot else: return -1 psf = psf_list[i] - disk = galsim.Sersic(n=self.disk_sersic_idx, - half_light_radius=self.hlr_disk, flux=1.0) + disk = galsim.Sersic( + n=self.disk_sersic_idx, half_light_radius=self.hlr_disk, flux=1.0 + ) disk_shape = galsim.Shear(g1=self.e1_disk, g2=self.e2_disk) disk = disk.shear(disk_shape) - bulge = galsim.Sersic(n=self.bulge_sersic_idx, - half_light_radius=self.hlr_bulge, flux=1.0) + bulge = galsim.Sersic( + n=self.bulge_sersic_idx, half_light_radius=self.hlr_bulge, flux=1.0 + ) bulge_shape = galsim.Shear(g1=self.e1_bulge, g2=self.e2_bulge) bulge = bulge.shear(bulge_shape) @@ -91,14 +107,26 @@ class Galaxy(MockObject): final = galsim.Sum(objs) return final - def drawObj_multiband(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, exptime=150., fd_shear=None): + def drawObj_multiband( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + fd_shear=None, + ): if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) # print("nphotons_tot = ", nphotons_tot) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) if self.logger: @@ -115,13 +143,17 @@ class Galaxy(MockObject): # Set Galsim Parameters if self.getMagFilter(filt) <= 15 and (not big_galaxy): - folding_threshold = 5.e-8 + folding_threshold = 5.0e-8 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) @@ -135,11 +167,19 @@ class Galaxy(MockObject): # Model the galaxy as disk + bulge disk = galsim.Sersic( - n=self.disk_sersic_idx, half_light_radius=self.hlr_disk, flux=1.0, gsparams=gsp) + n=self.disk_sersic_idx, + half_light_radius=self.hlr_disk, + flux=1.0, + gsparams=gsp, + ) disk_shape = galsim.Shear(g1=self.e1_disk, g2=self.e2_disk) disk = disk.shear(disk_shape) bulge = galsim.Sersic( - n=self.bulge_sersic_idx, half_light_radius=self.hlr_bulge, flux=1.0, gsparams=gsp) + n=self.bulge_sersic_idx, + half_light_radius=self.hlr_bulge, + flux=1.0, + gsparams=gsp, + ) bulge_shape = galsim.Shear(g1=self.e1_bulge, g2=self.e2_bulge) bulge = bulge.shear(bulge_shape) @@ -159,7 +199,7 @@ class Galaxy(MockObject): if self.logger: self.logger.error(e) continue - ratio = sub/full + ratio = sub / full if not (ratio == -1 or (ratio != ratio)): nphotons = ratio * nphotons_tot else: @@ -171,10 +211,15 @@ class Galaxy(MockObject): # Get PSF model EXTRA = False - if self.getMagFilter(filt) <= filt.mag_saturation-1.: + if self.getMagFilter(filt) <= filt.mag_saturation - 1.0: EXTRA = True psf, pos_shear = psf_model.get_PSF( - chip=chip, pos_img=pos_img, bandpass=bandpass, folding_threshold=folding_threshold, extrapolate=EXTRA) + chip=chip, + pos_img=pos_img, + bandpass=bandpass, + folding_threshold=folding_threshold, + extrapolate=EXTRA, + ) if self.bfrac == 0: gal_temp = disk @@ -206,8 +251,7 @@ class Galaxy(MockObject): # ERROR happens return 2, pos_shear stamp.setCenter(x_nominal, y_nominal) - bounds = stamp.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + bounds = stamp.bounds & galsim.BoundsI(0, chip.npix_x - 1, 0, chip.npix_y - 1) if bounds.area() > 0: chip.img.setOrigin(0, 0) chip.img[bounds] += stamp[bounds] @@ -226,24 +270,46 @@ class Galaxy(MockObject): # print("nphotons_sum = ", nphotons_sum) return 1, pos_shear - def drawObj_slitless(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, - exptime=150., normFilter=None, grating_split_pos=3685, fd_shear=None): + def drawObj_slitless( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + normFilter=None, + grating_split_pos=3685, + fd_shear=None, + ): if normFilter is not None: - norm_thr_rang_ids = normFilter['SENSITIVITY'] > 0.001 - sedNormFactor = getNormFactorForSpecWithABMAG(ABMag=self.param['mag_use_normal'], spectrum=self.sed, - norm_thr=normFilter, - sWave=np.floor( - normFilter[norm_thr_rang_ids][0][0]), - eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0])) + norm_thr_rang_ids = normFilter["SENSITIVITY"] > 0.001 + sedNormFactor = getNormFactorForSpecWithABMAG( + ABMag=self.param["mag_use_normal"], + spectrum=self.sed, + norm_thr=normFilter, + sWave=np.floor(normFilter[norm_thr_rang_ids][0][0]), + eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0]), + ) if sedNormFactor == 0: return 2, None else: - sedNormFactor = 1. - normalSED = Table(np.array([self.sed['WAVELENGTH'], self.sed['FLUX'] * sedNormFactor]).T, - names=('WAVELENGTH', 'FLUX')) - - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + sedNormFactor = 1.0 + normalSED = Table( + np.array([self.sed["WAVELENGTH"], self.sed["FLUX"] * sedNormFactor]).T, + names=("WAVELENGTH", "FLUX"), + ) + + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) @@ -259,23 +325,28 @@ class Galaxy(MockObject): big_galaxy = True if self.getMagFilter(filt) <= 15 and (not big_galaxy): - folding_threshold = 5.e-4 + folding_threshold = 5.0e-4 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) # nphotons_sum = 0 flat_cube = chip.flat_cube - xOrderSigPlus = {'A': 1.3909419820029296, 'B': 1.4760376591236062, - 'C': 4.035447379743442, 'D': 5.5684364343742825, 'E': 16.260021029735388} + xOrderSigPlus = { + "A": 1.3909419820029296, + "B": 1.4760376591236062, + "C": 4.035447379743442, + "D": 5.5684364343742825, + "E": 16.260021029735388, + } grating_split_pos_chip = 0 + grating_split_pos branges = np.zeros([len(bandpass_list), 2]) # print(hasattr(psf_model, 'bandranges')) - if hasattr(psf_model, 'bandranges'): + if hasattr(psf_model, "bandranges"): if psf_model.bandranges is None: return 2, None if len(psf_model.bandranges) != len(bandpass_list): @@ -292,11 +363,19 @@ class Galaxy(MockObject): # psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img, bandpass=bandpass, folding_threshold=folding_threshold) disk = galsim.Sersic( - n=self.disk_sersic_idx, half_light_radius=self.hlr_disk, flux=1.0, gsparams=gsp) + n=self.disk_sersic_idx, + half_light_radius=self.hlr_disk, + flux=1.0, + gsparams=gsp, + ) disk_shape = galsim.Shear(g1=self.e1_disk, g2=self.e2_disk) disk = disk.shear(disk_shape) bulge = galsim.Sersic( - n=self.bulge_sersic_idx, half_light_radius=self.hlr_bulge, flux=1.0, gsparams=gsp) + n=self.bulge_sersic_idx, + half_light_radius=self.hlr_bulge, + flux=1.0, + gsparams=gsp, + ) bulge_shape = galsim.Shear(g1=self.e1_bulge, g2=self.e2_bulge) bulge = bulge.shear(bulge_shape) @@ -329,27 +408,41 @@ class Galaxy(MockObject): galImg_List = [] try: pos_img_local = [0, 0] - x_start = chip.x_cen/chip.pix_size - chip.npix_x / 2. - y_start = chip.y_cen/chip.pix_size - chip.npix_y / 2. + x_start = chip.x_cen / chip.pix_size - chip.npix_x / 2.0 + y_start = chip.y_cen / chip.pix_size - chip.npix_y / 2.0 pos_img_local[0] = pos_img.x - x_start pos_img_local[1] = pos_img.y - y_start nnx = 0 nny = 0 for order in ["A", "B"]: EXTRA = False - if self.getMagFilter(filt) <= filt.mag_saturation-2.: + if self.getMagFilter(filt) <= filt.mag_saturation - 2.0: EXTRA = True psf, pos_shear = psf_model.get_PSF( - chip, pos_img_local=pos_img_local, bandNo=i+1, galsimGSObject=True, g_order=order, grating_split_pos=grating_split_pos, extrapolate=EXTRA, ngg=3072) + chip, + pos_img_local=pos_img_local, + bandNo=i + 1, + galsimGSObject=True, + g_order=order, + grating_split_pos=grating_split_pos, + extrapolate=EXTRA, + ngg=3072, + ) star_p = galsim.Convolve(psf, gal) if nnx == 0: galImg = star_p.drawImage( - wcs=chip_wcs_local, offset=offset, method='no_pixel') + wcs=chip_wcs_local, offset=offset, method="no_pixel" + ) nnx = galImg.xmax - galImg.xmin + 1 nny = galImg.ymax - galImg.ymin + 1 else: galImg = star_p.drawImage( - nx=nnx, ny=nny, wcs=chip_wcs_local, offset=offset, method='no_pixel') + nx=nnx, + ny=nny, + wcs=chip_wcs_local, + offset=offset, + method="no_pixel", + ) galImg.setOrigin(0, 0) # n1 = np.sum(np.isinf(galImg.array)) # n2 = np.sum(np.isnan(galImg.array)) @@ -363,28 +456,30 @@ class Galaxy(MockObject): galImg_List.append(galImg) except: try: - psf, pos_shear = psf_model.get_PSF( - chip=chip, pos_img=pos_img) + psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img) star_p = galsim.Convolve(psf, gal) - galImg = star_p.drawImage( - wcs=chip_wcs_local, offset=offset) + galImg = star_p.drawImage(wcs=chip_wcs_local, offset=offset) galImg.setOrigin(0, 0) if np.sum(np.isnan(galImg.array)) > 0: # ERROR happens return 2, pos_shear for order in ["A", "B", "C", "D", "E"]: galImg_List.append(galImg) - except Exception as e: + except Exception: continue # starImg = gal.drawImage( # wcs=chip_wcs_local, offset=offset, method='real_space') - origin_star = [y_nominal - (galImg.center.y - galImg.ymin), - x_nominal - (galImg.center.x - galImg.xmin)] + origin_star = [ + y_nominal - (galImg.center.y - galImg.ymin), + x_nominal - (galImg.center.x - galImg.xmin), + ] galImg.setOrigin(0, 0) gal_origin = [origin_star[0], origin_star[1]] - gal_end = [origin_star[0] + galImg.array.shape[0] - - 1, origin_star[1] + galImg.array.shape[1] - 1] + gal_end = [ + origin_star[0] + galImg.array.shape[0] - 1, + origin_star[1] + galImg.array.shape[1] - 1, + ] if gal_origin[1] < grating_split_pos_chip < gal_end[1]: subSlitPos = int(grating_split_pos_chip - gal_origin[1]) @@ -392,25 +487,33 @@ class Galaxy(MockObject): star_p1s = [] for galImg in galImg_List: - subImg_p1 = galImg.array[:, 0:subSlitPos] star_p1 = galsim.Image(subImg_p1) star_p1.setOrigin(0, 0) star_p1s.append(star_p1) origin_p1 = origin_star - xcenter_p1 = min(x_nominal, grating_split_pos_chip-1) - 0 - ycenter_p1 = y_nominal-0 - - sdp_p1 = SpecDisperser(orig_img=star_p1s, xcenter=xcenter_p1, - ycenter=ycenter_p1, origin=origin_p1, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + xcenter_p1 = min(x_nominal, grating_split_pos_chip - 1) - 0 + ycenter_p1 = y_nominal - 0 + + sdp_p1 = SpecDisperser( + orig_img=star_p1s, + xcenter=xcenter_p1, + ycenter=ycenter_p1, + origin=origin_p1, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp_p1, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp_p1, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p1, chip=chip, pos_img_local=[xcenter_p1, ycenter_p1], # psf_model=psf_model, bandNo=i + 1, # grating_split_pos=grating_split_pos, @@ -418,9 +521,7 @@ class Galaxy(MockObject): star_p2s = [] for galImg in galImg_List: - - subImg_p2 = galImg.array[:, - subSlitPos:galImg.array.shape[1]] + subImg_p2 = galImg.array[:, subSlitPos : galImg.array.shape[1]] star_p2 = galsim.Image(subImg_p2) star_p2.setOrigin(0, 0) star_p2s.append(star_p2) @@ -428,16 +529,25 @@ class Galaxy(MockObject): xcenter_p2 = max(x_nominal, grating_split_pos_chip) - 0 ycenter_p2 = y_nominal - 0 - sdp_p2 = SpecDisperser(orig_img=star_p2s, xcenter=xcenter_p2, - ycenter=ycenter_p2, origin=origin_p2, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp_p2 = SpecDisperser( + orig_img=star_p2s, + xcenter=xcenter_p2, + ycenter=ycenter_p2, + origin=origin_p2, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp_p2, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp_p2, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p2, chip=chip, pos_img_local=[xcenter_p2, ycenter_p2], # psf_model=psf_model, bandNo=i + 1, # grating_split_pos=grating_split_pos, @@ -446,30 +556,48 @@ class Galaxy(MockObject): del sdp_p1 del sdp_p2 elif grating_split_pos_chip <= gal_origin[1]: - sdp = SpecDisperser(orig_img=galImg_List, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=galImg_List, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], # psf_model=psf_model, bandNo=i + 1, # grating_split_pos=grating_split_pos, # local_wcs=chip_wcs_local, pos_img=pos_img) del sdp elif grating_split_pos_chip >= gal_end[1]: - sdp = SpecDisperser(orig_img=galImg_List, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=galImg_List, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], # psf_model=psf_model, bandNo=i + 1, # grating_split_pos=grating_split_pos, @@ -480,16 +608,18 @@ class Galaxy(MockObject): # del psf return 1, pos_shear - def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.): + def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.0): if flux is None: flux = self.getElectronFluxFilt(filt, tel, exptime) - disk = galsim.Sersic(n=self.disk_sersic_idx, - half_light_radius=self.hlr_disk, flux=1.0) + disk = galsim.Sersic( + n=self.disk_sersic_idx, half_light_radius=self.hlr_disk, flux=1.0 + ) disk_shape = galsim.Shear(g1=self.e1_disk, g2=self.e2_disk) disk = disk.shear(disk_shape) - bulge = galsim.Sersic(n=self.bulge_sersic_idx, - half_light_radius=self.hlr_bulge, flux=1.0) + bulge = galsim.Sersic( + n=self.bulge_sersic_idx, half_light_radius=self.hlr_bulge, flux=1.0 + ) bulge_shape = galsim.Shear(g1=self.e1_bulge, g2=self.e2_bulge) bulge = bulge.shear(bulge_shape) @@ -501,6 +631,5 @@ class Galaxy(MockObject): return final def getObservedEll(self, g1=0, g2=0): - e1_obs, e2_obs, e_obs, theta = eObs( - self.e1_total, self.e2_total, g1, g2) + e1_obs, e2_obs, e_obs, theta = eObs(self.e1_total, self.e2_total, g1, g2) return self.e1_total, self.e2_total, g1, g2, e1_obs, e2_obs diff --git a/observation_sim/mock_objects/Ghost.py b/observation_sim/mock_objects/Ghost.py new file mode 100644 index 0000000000000000000000000000000000000000..1b8b5dac3af9197b5d03efc1a56d3d8ae6fcd204 --- /dev/null +++ b/observation_sim/mock_objects/Ghost.py @@ -0,0 +1,205 @@ +""" +Ghost image calculation (single-class refactor) + +This module keeps the original behavior but wraps everything into ONE class: `Ghost`. + +All lengths are in meters. +""" + +from __future__ import annotations + +from typing import Dict, Tuple + +import numpy as np +import galsim + + +class Ghost: + """ + Single-class ghost image model. + + Usage + ----- + ghost = Ghost() # uses built-in defaults + result = ghost.calculate(detector_id=13, x0=0.0, y0=0.0) + + The returned dict contains: + - "ghosts": ndarray shape (3, 4) with rows [x, y, radius, relative_energy] + - "angle_x_deg": float + - "angle_y_deg": float + - "wave_name": str (band name) + """ + + # ----------- Default optical constants (meters) ----------- + DEFAULT_ZP = 4.593 + DEFAULT_RP = 0.16367 + DEFAULT_D1 = 0.005 + DEFAULT_D2 = 0.008 + DEFAULT_PIXEL = 1.0e-5 # kept for completeness; unused by formula + + def __init__( + self, + *, + Zp: float = DEFAULT_ZP, + Rp: float = DEFAULT_RP, + D1: float = DEFAULT_D1, + D2: float = DEFAULT_D2, + pixel: float = DEFAULT_PIXEL, + ) -> None: + self.Zp = float(Zp) + self.Rp = float(Rp) + self.D1 = float(D1) + self.D2 = float(D2) + self.pixel = float(pixel) + + # ---- Wave table (from original script) ---- + # Stored as: wave_num -> (wave_name, wave_length, nd, tf1, rf1, tf2, rf2, TD, RD) + self._waves: Dict[ + int, Tuple[str, float, float, float, float, float, float, float, float] + ] = { + 0: ("nuv", 2.9e-7, 1.495, 0.825, 0.0093, 0.825, 0.0093, 0.555, 0.15), + 1: ("u", 3.6e-7, 1.473, 0.954, 0.0138, 0.954, 0.0138, 0.56, 0.15), + 2: ("g", 4.8e-7, 1.463, 0.98, 0.0197, 0.98, 0.0197, 0.8, 0.2), + 3: ("r", 6.2e-7, 1.458, 0.98, 0.015, 0.98, 0.015, 0.91, 0.09), + 4: ("i", 7.5e-7, 1.454, 0.985, 0.01, 0.985, 0.01, 0.889, 0.11), + 5: ("z", 9.1e-7, 1.452, 0.99, 0.0085, 0.99, 0.0085, 0.6, 0.1), + 6: ("y", 9.7e-7, 1.451, 0.99, 0.0035, 0.99, 0.0035, 0.36, 0.1), + } + + # ---- Detector -> wave_num mapping (from original script) ---- + self._detector_to_wave_num: Dict[int, int] = { + 6: 6, + 7: 4, + 8: 2, + 9: 3, + 11: 5, + 12: 0, + 13: 0, + 14: 1, + 15: 6, + 16: 6, + 17: 1, + 18: 0, + 19: 0, + 20: 5, + 22: 3, + 23: 2, + 24: 4, + 25: 6, + } + + # ----------------------------- + # Helpers + # ----------------------------- + + def _get_wave_tuple( + self, detector_id: int + ) -> Tuple[str, float, float, float, float, float, float, float, float]: + """ + Return wave properties tuple for a detector_id. + Raises KeyError with a friendly message if unknown. + """ + try: + wave_num = self._detector_to_wave_num[detector_id] + except KeyError as e: + known = ", ".join(map(str, sorted(self._detector_to_wave_num.keys()))) + raise KeyError( + f"Unknown detector_id={detector_id}. Known ids: {known}" + ) from e + + return self._waves[wave_num] + + # ----------------------------- + # Public API + # ----------------------------- + + def calculate(self, detector_id: int, x0: float, y0: float) -> Dict[str, object]: + """ + Calculate ghost images for a given detector and focal-plane coordinate. + + Parameters + ---------- + detector_id : int + Detector identifier. + x0, y0 : float + Absolute focal-plane coordinates of the image point [m]. + + Returns + ------- + dict with keys: + ghosts: ndarray (3, 4) rows [ghost_x, ghost_y, ghost_radius, relative_energy] + angle_x_deg: float + angle_y_deg: float + wave_name: str + """ + wave_name, _wave_length, nd, tf1, rf1, tf2, rf2, TD, RD = self._get_wave_tuple( + detector_id + ) + + # Slope defined by pupil geometry + u = self.Rp / self.Zp + + # Ghost radii (physical lengths) + r1 = 2.0 * self.D1 * u / nd + r2 = 2.0 * self.D2 * u + r3 = 2.0 * (self.D1 / nd + self.D2) * u + + # Relative energy terms + m1 = rf1 * rf2 + m2 = rf2 * RD + m3 = rf1 * RD * (tf2**2) + + # NOTE: Original script uses a fixed +0.313 x-offset; preserve behavior. + denom = self.Zp - self.D1 * (1.0 - 1.0 / nd) + angle_x = (float(x0) + 0.313) / denom + angle_y = float(y0) / denom + + # Angles inside the filter + af_x = angle_x / nd + af_y = angle_y / nd + + # Lateral shifts on the focal plane + xd1 = np.tan(af_x) * 2.0 * self.D1 + yd1 = np.tan(af_y) * 2.0 * self.D1 + xd2 = np.tan(angle_x) * 2.0 * self.D2 + yd2 = np.tan(angle_y) * 2.0 * self.D2 + + # Combined shifts + xd3 = xd1 + xd2 + yd3 = yd1 + yd2 + + # Ghost positions + x1, y1 = float(x0) + xd1, float(y0) + yd1 + x2, y2 = float(x0) + xd2, float(y0) + yd2 + x3, y3 = float(x0) + xd3, float(y0) + yd3 + + ghosts = [ + [x1, y1, r1, m1], + [x2, y2, r2, m2], + [x3, y3, r3, m3], + ] + + return { + "ghosts": ghosts, + "angle_x_deg": float(angle_x * 180.0 / np.pi), + "angle_y_deg": float(angle_y * 180.0 / np.pi), + "wave_name": wave_name, + } + + def draw_on_chip(self, chip, pos_pix, r_pix, flux=1.0): + x_pix, y_pix = pos_pix.x, pos_pix.y + profile = galsim.TopHat(radius=r_pix, flux=flux) + chip.img.setOrigin(0, 0) + profile.drawImage( + image=chip.img, center=galsim.PositionD(x_pix, y_pix), add_to_image=True + ) + chip.img.setOrigin(chip.bound.xmin, chip.bound.ymin) + + +if __name__ == "__main__": + g = Ghost() + + # Original sanity checks + print(g.calculate(13, 0.0, 0.0)) + print(g.calculate(6, 0.19233, -0.24876)) + print(g.calculate(25, -0.19233, 0.24876)) diff --git a/observation_sim/mock_objects/MockObject.py b/observation_sim/mock_objects/MockObject.py index a1f2b1b35532cc58c48d6db7fc944795f7a1cc01..875ce35550e2ee9090bb95962d5b2efc104a20db 100755 --- a/observation_sim/mock_objects/MockObject.py +++ b/observation_sim/mock_objects/MockObject.py @@ -1,14 +1,16 @@ import os import galsim import numpy as np -import astropy.constants as cons -from astropy import wcs from astropy.table import Table import astropy.io.fits as fitsio -from observation_sim.mock_objects._util import magToFlux, VC_A, convolveGaussXorders, convolveImg -from observation_sim.mock_objects._util import integrate_sed_bandpass, getNormFactorForSpecWithABMAG, getObservedSED, \ - getABMAG +from observation_sim.mock_objects._util import ( + convolveImg, +) +from observation_sim.mock_objects._util import ( + integrate_sed_bandpass, + getNormFactorForSpecWithABMAG, +) from observation_sim.mock_objects.SpecDisperser import SpecDisperser from observation_sim.instruments.chip import chip_utils @@ -49,11 +51,11 @@ class MockObject(object): def getFluxFilter(self, filt): return self.param["flux_%s" % filt.filter_type.lower()] - def getNumPhotons(self, flux, tel, exptime=150.): - pupil_area = tel.pupil_area * (100.) ** 2 # m^2 to cm^2 + def getNumPhotons(self, flux, tel, exptime=150.0): + pupil_area = tel.pupil_area * (100.0) ** 2 # m^2 to cm^2 return flux * pupil_area * exptime - def getElectronFluxFilt(self, filt, tel, exptime=150.): + def getElectronFluxFilt(self, filt, tel, exptime=150.0): # photonEnergy = filt.getPhotonE() # flux = magToFlux(self.getMagFilter(filt)) # factor = 1.0e4 * flux/photonEnergy * VC_A * (1.0/filt.blue_limit - 1.0/filt.red_limit) @@ -61,12 +63,22 @@ class MockObject(object): flux = self.getFluxFilter(filt) return flux * tel.pupil_area * exptime - def getPosWorld(self, ra_offset=0., dec_offset=0.): + def getPosWorld(self, ra_offset=0.0, dec_offset=0.0): ra = self.param["ra"] + ra_offset dec = self.param["dec"] + dec_offset return galsim.CelestialCoord(ra=ra * galsim.degrees, dec=dec * galsim.degrees) - def getPosImg_Offset_WCS(self, img, fdmodel=None, chip=None, verbose=True, chip_wcs=None, img_header=None, ra_offset=0., dec_offset=0.): + def getPosImg_Offset_WCS( + self, + img, + fdmodel=None, + chip=None, + verbose=True, + chip_wcs=None, + img_header=None, + ra_offset=0.0, + dec_offset=0.0, + ): self.posImg = img.wcs.toImage(self.getPosWorld(ra_offset, dec_offset)) self.localWCS = img.wcs.local(self.posImg) # Apply field distortion model @@ -74,16 +86,19 @@ class MockObject(object): if verbose: print("\n") print("Before field distortion:\n") - print("x = %.2f, y = %.2f\n" % - (self.posImg.x, self.posImg.y), flush=True) + print( + "x = %.2f, y = %.2f\n" % (self.posImg.x, self.posImg.y), flush=True + ) self.posImg, self.fd_shear = fdmodel.get_distorted( - chip=chip, pos_img=self.posImg) + chip=chip, pos_img=self.posImg + ) if self.posImg is None: return None, None, None, None, None if verbose: print("After field distortion:\n") - print("x = %.2f, y = %.2f\n" % - (self.posImg.x, self.posImg.y), flush=True) + print( + "x = %.2f, y = %.2f\n" % (self.posImg.x, self.posImg.y), flush=True + ) x, y = self.posImg.x + 0.5, self.posImg.y + 0.5 self.x_nominal = int(np.floor(x + 0.5)) @@ -102,21 +117,32 @@ class MockObject(object): return self.posImg, self.offset, self.localWCS, self.chip_wcs, self.fd_shear - def getRealPos(self, img, global_x=0., global_y=0., img_real_wcs=None): + def getRealPos(self, img, global_x=0.0, global_y=0.0, img_real_wcs=None): img_global_pos = galsim.PositionD(global_x, global_y) cel_pos = img.wcs.toWorld(img_global_pos) realPos = img_real_wcs.toImage(cel_pos) return realPos - def drawObj_multiband(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, - exptime=150., fd_shear=None): + def drawObj_multiband( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + fd_shear=None, + ): if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) # print("nphotons_tot = ", nphotons_tot) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) if self.logger: @@ -124,14 +150,18 @@ class MockObject(object): return 2, None # Set Galsim Parameters if self.getMagFilter(filt) <= 15: - folding_threshold = 5.e-8 + folding_threshold = 5.0e-8 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) # Get real image position of object (deal with chip rotation w.r.t its center) - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) y_nominal = int(np.floor(y + 0.5)) @@ -163,24 +193,32 @@ class MockObject(object): # Get PSF model EXTRA = False - if self.getMagFilter(filt) <= filt.mag_saturation-1.: + if self.getMagFilter(filt) <= filt.mag_saturation - 1.0: EXTRA = True - psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img, bandpass=bandpass, - folding_threshold=folding_threshold, extrapolate=EXTRA) + psf, pos_shear = psf_model.get_PSF( + chip=chip, + pos_img=pos_img, + bandpass=bandpass, + folding_threshold=folding_threshold, + extrapolate=EXTRA, + ) # star = galsim.DeltaFunction(gsparams=gsp) # star = star.withFlux(nphotons) # star = galsim.Convolve(psf, star) star = psf.withFlux(nphotons) if EXTRA: - stamp = star.drawImage(method='no_pixel', wcs=chip_wcs_local, offset=offset) + stamp = star.drawImage( + method="no_pixel", wcs=chip_wcs_local, offset=offset + ) else: stamp = star.drawImage(wcs=chip_wcs_local, offset=offset) if np.sum(np.isnan(stamp.array)) > 0: continue stamp.setCenter(x_nominal, y_nominal) bounds = stamp.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + 0, chip.npix_x - 1, 0, chip.npix_y - 1 + ) if bounds.area() > 0: chip.img.setOrigin(0, 0) chip.img[bounds] += stamp[bounds] @@ -196,7 +234,9 @@ class MockObject(object): return 0, pos_shear return 1, pos_shear # Return code 1: draw sucesss - def addSLStoChipImage(self, sdp=None, chip=None, xOrderSigPlus=None, local_wcs=None): + def addSLStoChipImage( + self, sdp=None, chip=None, xOrderSigPlus=None, local_wcs=None + ): spec_orders = sdp.compute_spec_orders() for k, v in spec_orders.items(): img_s = v[0] @@ -228,18 +268,29 @@ class MockObject(object): stamp.setOrigin(origin_order_x, origin_order_y) bounds = stamp.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + 0, chip.npix_x - 1, 0, chip.npix_y - 1 + ) if bounds.area() == 0: continue chip.img.setOrigin(0, 0) - chip.img[bounds] = chip.img[bounds]+stamp[bounds] + chip.img[bounds] = chip.img[bounds] + stamp[bounds] chip.img.setOrigin(chip.bound.xmin, chip.bound.ymin) del stamp del spec_orders - def addSLStoChipImageWithPSF(self, sdp=None, chip=None, pos_img_local=[1, 1], psf_model=None, bandNo=1, grating_split_pos=3685, local_wcs=None, pos_img=None): + def addSLStoChipImageWithPSF( + self, + sdp=None, + chip=None, + pos_img_local=[1, 1], + psf_model=None, + bandNo=1, + grating_split_pos=3685, + local_wcs=None, + pos_img=None, + ): spec_orders = sdp.compute_spec_orders() - pos_shear = galsim.Shear(e=0., beta=(np.pi/2)*galsim.radians) + pos_shear = galsim.Shear(e=0.0, beta=(np.pi / 2) * galsim.radians) if chip.slsPSFOptim: for k, v in spec_orders.items(): img_s = v[0] @@ -260,23 +311,31 @@ class MockObject(object): specImg.setOrigin(origin_order_x, origin_order_y) bounds = specImg.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + 0, chip.npix_x - 1, 0, chip.npix_y - 1 + ) if bounds.area() == 0: continue # orders = {'A': 'order1', 'B': 'order0', 'C': 'order2', 'D': 'order-1', 'E': 'order-2'} - orders = {'A': 'order1', 'B': 'order0', - 'C': 'order0', 'D': 'order0', 'E': 'order0'} + orders = { + "A": "order1", + "B": "order0", + "C": "order0", + "D": "order0", + "E": "order0", + } gratingN = chip_utils.getChipSLSGratingID(chip.chipID)[1] if pos_img_local[0] < grating_split_pos: gratingN = chip_utils.getChipSLSGratingID(chip.chipID)[0] - chip.img_stack[gratingN][orders[k] - ]['w' + str(bandNo)].setOrigin(0, 0) - chip.img_stack[gratingN][orders[k]]['w' + str( - bandNo)][bounds] = chip.img_stack[gratingN][orders[k]]['w' + str(bandNo)][bounds] + specImg[bounds] - chip.img_stack[gratingN][orders[k]]['w' + - str(bandNo)].setOrigin(chip.bound.xmin, chip.bound.ymin) + chip.img_stack[gratingN][orders[k]]["w" + str(bandNo)].setOrigin(0, 0) + chip.img_stack[gratingN][orders[k]]["w" + str(bandNo)][bounds] = ( + chip.img_stack[gratingN][orders[k]]["w" + str(bandNo)][bounds] + + specImg[bounds] + ) + chip.img_stack[gratingN][orders[k]]["w" + str(bandNo)].setOrigin( + chip.bound.xmin, chip.bound.ymin + ) else: for k, v in spec_orders.items(): @@ -342,10 +401,15 @@ class MockObject(object): specImg.setOrigin(origin_order_x, origin_order_y) try: specImg = psf_model.get_PSF_AND_convolve_withsubImg( - chip, cutImg=specImg, pos_img_local=pos_img_local, bandNo=bandNo, g_order=k, grating_split_pos=grating_split_pos) + chip, + cutImg=specImg, + pos_img_local=pos_img_local, + bandNo=bandNo, + g_order=k, + grating_split_pos=grating_split_pos, + ) except: - psf, pos_shear = psf_model.get_PSF( - chip=chip, pos_img=pos_img) + psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img) psf_img = psf.drawImage(nx=100, ny=100, wcs=local_wcs) @@ -361,7 +425,8 @@ class MockObject(object): specImg.setOrigin(origin_order_x, origin_order_y) bounds = specImg.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + 0, chip.npix_x - 1, 0, chip.npix_y - 1 + ) if bounds.area() == 0: continue chip.img.setOrigin(0, 0) @@ -374,31 +439,53 @@ class MockObject(object): del spec_orders return pos_shear - def drawObj_slitless(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, - exptime=150., normFilter=None, grating_split_pos=3685, fd_shear=None): + def drawObj_slitless( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + normFilter=None, + grating_split_pos=3685, + fd_shear=None, + ): if normFilter is not None: - norm_thr_rang_ids = normFilter['SENSITIVITY'] > 0.001 - sedNormFactor = getNormFactorForSpecWithABMAG(ABMag=self.param['mag_use_normal'], spectrum=self.sed, - norm_thr=normFilter, - sWave=np.floor( - normFilter[norm_thr_rang_ids][0][0]), - eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0])) + norm_thr_rang_ids = normFilter["SENSITIVITY"] > 0.001 + sedNormFactor = getNormFactorForSpecWithABMAG( + ABMag=self.param["mag_use_normal"], + spectrum=self.sed, + norm_thr=normFilter, + sWave=np.floor(normFilter[norm_thr_rang_ids][0][0]), + eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0]), + ) if sedNormFactor == 0: return 2, None else: - sedNormFactor = 1. + sedNormFactor = 1.0 if self.getMagFilter(filt) <= 15: - folding_threshold = 5.e-8 + folding_threshold = 5.0e-8 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) - normalSED = Table(np.array([self.sed['WAVELENGTH'], self.sed['FLUX'] * sedNormFactor]).T, - names=('WAVELENGTH', 'FLUX')) + normalSED = Table( + np.array([self.sed["WAVELENGTH"], self.sed["FLUX"] * sedNormFactor]).T, + names=("WAVELENGTH", "FLUX"), + ) - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) @@ -411,13 +498,18 @@ class MockObject(object): flat_cube = chip.flat_cube - xOrderSigPlus = {'A': 1.3909419820029296, 'B': 1.4760376591236062, 'C': 4.035447379743442, - 'D': 5.5684364343742825, 'E': 16.260021029735388} + xOrderSigPlus = { + "A": 1.3909419820029296, + "B": 1.4760376591236062, + "C": 4.035447379743442, + "D": 5.5684364343742825, + "E": 16.260021029735388, + } grating_split_pos_chip = 0 + grating_split_pos branges = np.zeros([len(bandpass_list), 2]) - if hasattr(psf_model, 'bandranges'): + if hasattr(psf_model, "bandranges"): if psf_model.bandranges is None: return 2, None if len(psf_model.bandranges) != len(bandpass_list): @@ -442,30 +534,44 @@ class MockObject(object): starImg_List = [] try: pos_img_local = [0, 0] - x_start = chip.x_cen/chip.pix_size - chip.npix_x / 2. - y_start = chip.y_cen/chip.pix_size - chip.npix_y / 2. + x_start = chip.x_cen / chip.pix_size - chip.npix_x / 2.0 + y_start = chip.y_cen / chip.pix_size - chip.npix_y / 2.0 pos_img_local[0] = pos_img.x - x_start pos_img_local[1] = pos_img.y - y_start nnx = 0 nny = 0 for order in ["A", "B"]: EXTRA = False - if self.getMagFilter(filt) <= filt.mag_saturation-2.: + if self.getMagFilter(filt) <= filt.mag_saturation - 2.0: EXTRA = True nnx = 2048 nny = 2048 psf, pos_shear = psf_model.get_PSF( - chip, pos_img_local=pos_img_local, bandNo=i+1, galsimGSObject=True, g_order=order, grating_split_pos=grating_split_pos, extrapolate=EXTRA, ngg=3072) + chip, + pos_img_local=pos_img_local, + bandNo=i + 1, + galsimGSObject=True, + g_order=order, + grating_split_pos=grating_split_pos, + extrapolate=EXTRA, + ngg=3072, + ) # star_p = galsim.Convolve(psf, star) star_p = psf.withFlux(tel.pupil_area * exptime) if nnx == 0: starImg = star_p.drawImage( - wcs=chip_wcs_local, offset=offset, method='no_pixel') + wcs=chip_wcs_local, offset=offset, method="no_pixel" + ) nnx = starImg.xmax - starImg.xmin + 1 nny = starImg.ymax - starImg.ymin + 1 else: starImg = star_p.drawImage( - nx=nnx, ny=nny, wcs=chip_wcs_local, offset=offset, method='no_pixel') + nx=nnx, + ny=nny, + wcs=chip_wcs_local, + offset=offset, + method="no_pixel", + ) # n1 = np.sum(np.isinf(starImg.array)) # n2 = np.sum(np.isnan(starImg.array)) # if n1>0 or n2 > 0: @@ -478,7 +584,9 @@ class MockObject(object): psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img) # star_p = galsim.Convolve(psf, star) star_p = psf.withFlux(tel.pupil_area * exptime) - starImg = star_p.drawImage(wcs=chip_wcs_local, offset=offset, method='no_pixel') + starImg = star_p.drawImage( + wcs=chip_wcs_local, offset=offset, method="no_pixel" + ) starImg.setOrigin(0, 0) for order in ["A", "B", "C", "D", "E"]: starImg_List.append(starImg) @@ -489,18 +597,21 @@ class MockObject(object): # starImg = star.drawImage( # nx=60, ny=60, wcs=chip_wcs_local, offset=offset) - origin_star = [y_nominal - (starImg.center.y - starImg.ymin), - x_nominal - (starImg.center.x - starImg.xmin)] + origin_star = [ + y_nominal - (starImg.center.y - starImg.ymin), + x_nominal - (starImg.center.x - starImg.xmin), + ] gal_origin = [origin_star[0], origin_star[1]] - gal_end = [origin_star[0] + starImg.array.shape[0] - - 1, origin_star[1] + starImg.array.shape[1] - 1] + gal_end = [ + origin_star[0] + starImg.array.shape[0] - 1, + origin_star[1] + starImg.array.shape[1] - 1, + ] if gal_origin[1] < grating_split_pos_chip < gal_end[1]: subSlitPos = int(grating_split_pos_chip - gal_origin[1]) # part img disperse star_p1s = [] for starImg in starImg_List: - subImg_p1 = starImg.array[:, 0:subSlitPos] star_p1 = galsim.Image(subImg_p1) star_p1.setOrigin(0, 0) @@ -509,25 +620,32 @@ class MockObject(object): xcenter_p1 = min(x_nominal, grating_split_pos_chip - 1) - 0 ycenter_p1 = y_nominal - 0 - sdp_p1 = SpecDisperser(orig_img=star_p1s, xcenter=xcenter_p1, - ycenter=ycenter_p1, origin=origin_p1, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + sdp_p1 = SpecDisperser( + orig_img=star_p1s, + xcenter=xcenter_p1, + ycenter=ycenter_p1, + origin=origin_p1, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp_p1, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp_p1, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p1, chip=chip, pos_img_local=[xcenter_p1, ycenter_p1], # psf_model=psf_model, bandNo=i+1, grating_split_pos=grating_split_pos, # local_wcs=chip_wcs_local, pos_img=pos_img) star_p2s = [] for starImg in starImg_List: - - subImg_p2 = starImg.array[:, - subSlitPos:starImg.array.shape[1]] + subImg_p2 = starImg.array[:, subSlitPos : starImg.array.shape[1]] star_p2 = galsim.Image(subImg_p2) star_p2.setOrigin(0, 0) star_p2s.append(star_p2) @@ -535,16 +653,25 @@ class MockObject(object): xcenter_p2 = max(x_nominal, grating_split_pos_chip) - 0 ycenter_p2 = y_nominal - 0 - sdp_p2 = SpecDisperser(orig_img=star_p2s, xcenter=xcenter_p2, - ycenter=ycenter_p2, origin=origin_p2, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp_p2 = SpecDisperser( + orig_img=star_p2s, + xcenter=xcenter_p2, + ycenter=ycenter_p2, + origin=origin_p2, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp_p2, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp_p2, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p2, chip=chip, pos_img_local=[xcenter_p2, ycenter_p2], # psf_model=psf_model, bandNo=i + 1, grating_split_pos=grating_split_pos, # local_wcs=chip_wcs_local, pos_img=pos_img) @@ -552,29 +679,47 @@ class MockObject(object): del sdp_p1 del sdp_p2 elif grating_split_pos_chip <= gal_origin[1]: - sdp = SpecDisperser(orig_img=starImg_List, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=starImg_List, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], # psf_model=psf_model, bandNo=i + 1, grating_split_pos=grating_split_pos, # local_wcs=chip_wcs_local, pos_img=pos_img) del sdp elif grating_split_pos_chip >= gal_end[1]: - sdp = SpecDisperser(orig_img=starImg_List, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=starImg_List, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) self.addSLStoChipImage( - sdp=sdp, chip=chip, xOrderSigPlus=xOrderSigPlus, local_wcs=chip_wcs_local) + sdp=sdp, + chip=chip, + xOrderSigPlus=xOrderSigPlus, + local_wcs=chip_wcs_local, + ) # pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], # psf_model=psf_model, bandNo=i + 1, grating_split_pos=grating_split_pos, # local_wcs=chip_wcs_local, pos_img=pos_img) @@ -592,15 +737,27 @@ class MockObject(object): snr_obj = img_flux / sig_obj return snr_obj - def drawObj_PSF(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, - exptime=150., fd_shear=None, chip_output=None): + def drawObj_PSF( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + fd_shear=None, + chip_output=None, + ): if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) # print("nphotons_tot = ", nphotons_tot) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) if self.logger: @@ -609,14 +766,18 @@ class MockObject(object): # Set Galsim Parameters if self.getMagFilter(filt) <= 15: - folding_threshold = 5.e-4 + folding_threshold = 5.0e-4 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) # Get real image position of object (deal with chip rotation w.r.t its center) - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) y_nominal = int(np.floor(y + 0.5)) @@ -644,14 +805,18 @@ class MockObject(object): continue # Get PSF model - psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img, bandpass=bandpass, - folding_threshold=folding_threshold) + psf, pos_shear = psf_model.get_PSF( + chip=chip, + pos_img=pos_img, + bandpass=bandpass, + folding_threshold=folding_threshold, + ) star_temp = psf.withFlux(nphotons) if i == 0: star = star_temp else: - star = star+star_temp + star = star + star_temp pixelScale = 0.074 stamp = star.drawImage(wcs=chip_wcs_local, offset=offset) @@ -661,16 +826,21 @@ class MockObject(object): fn = chip_output.subdir + "/psfIDW" os.makedirs(fn, exist_ok=True) - fn = fn + "/ccd_{:}".format(chip.chipID) + \ - "_psf_"+str(self.param['id'])+".fits" + fn = ( + fn + + "/ccd_{:}".format(chip.chipID) + + "_psf_" + + str(self.param["id"]) + + ".fits" + ) if fn is not None: if os.path.exists(fn): os.remove(fn) hdu = fitsio.PrimaryHDU() hdu.data = stamp.array - hdu.header.set('name', self.type) - hdu.header.set('pixScale', pixelScale) - hdu.header.set('objID', self.param['id']) + hdu.header.set("name", self.type) + hdu.header.set("pixScale", pixelScale) + hdu.header.set("objID", self.param["id"]) hdu.writeto(fn) del stamp diff --git a/observation_sim/mock_objects/Quasar.py b/observation_sim/mock_objects/Quasar.py index 3589b0d1e80f3977b9fc83f7624b02557a0b24ae..794b5b58b789c70d63736c685f5b772779a4fcfc 100755 --- a/observation_sim/mock_objects/Quasar.py +++ b/observation_sim/mock_objects/Quasar.py @@ -1,13 +1,17 @@ import galsim import os -import sys import numpy as np import astropy.constants as cons from astropy.table import Table from scipy import interpolate from observation_sim.mock_objects.MockObject import MockObject -from observation_sim.mock_objects._util import integrate_sed_bandpass, getNormFactorForSpecWithABMAG, getObservedSED, getABMAG +from observation_sim.mock_objects._util import ( + integrate_sed_bandpass, + getNormFactorForSpecWithABMAG, + getObservedSED, + getABMAG, +) class Quasar(MockObject): @@ -15,27 +19,38 @@ class Quasar(MockObject): super().__init__(param, logger=logger) if not hasattr(self, "mu"): if hasattr(self, "detA"): - self.mu = 1./self.detA + self.mu = 1.0 / self.detA else: - self.mu = 1. + self.mu = 1.0 - def load_SED(self, survey_type, sed_path=None, cosids=None, objtypes=None, sed_templates=None, normFilter=None, target_filt=None): - ''' + def load_SED( + self, + survey_type, + sed_path=None, + cosids=None, + objtypes=None, + sed_templates=None, + normFilter=None, + target_filt=None, + ): + """ -------------------------------------------------------------------- (Deprecated) Left over codes from cycle-1 to cycle-3 implementations -------------------------------------------------------------------- - ''' + """ if survey_type == "photometric": - norm_thr_rang_ids = normFilter['SENSITIVITY'] > 0.001 + norm_thr_rang_ids = normFilter["SENSITIVITY"] > 0.001 if sed_templates is None: # Read SED data directly itype = objtypes[cosids == self.sed_type][0] - sed_file = os.path.join( - sed_path, itype + "_ID%s.sed" % (self.sed_type)) + sed_file = os.path.join(sed_path, itype + "_ID%s.sed" % (self.sed_type)) if not os.path.exists(sed_file): raise ValueError("!!! No SED found.") sed_data = Table.read(sed_file, format="ascii") - wave, flux = sed_data["observedLambda"].data, sed_data["observedFlux"].data + wave, flux = ( + sed_data["observedLambda"].data, + sed_data["observedFlux"].data, + ) else: # Load SED from templates sed_data = sed_templates[self.sed_type] @@ -43,64 +58,72 @@ class Quasar(MockObject): sed_data = getObservedSED( sedCat=sed_data, redshift=self.z, - av=self.param['av'], - redden=self.param['redden']) + av=self.param["av"], + redden=self.param["redden"], + ) wave, flux = sed_data[0], sed_data[1] flux_photon = flux * (wave / (cons.h.value * cons.c.value)) * 1e-13 sed_photon = Table( - np.array([wave, flux_photon]).T, names=('WAVELENGTH', 'FLUX')) + np.array([wave, flux_photon]).T, names=("WAVELENGTH", "FLUX") + ) # Get scaling factor for SED - sedNormFactor = getNormFactorForSpecWithABMAG(ABMag=self.param['mag_use_normal'], - spectrum=sed_photon, - norm_thr=normFilter, - sWave=np.floor( - normFilter[norm_thr_rang_ids][0][0]), - eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0])) - sed_photon = np.array( - [sed_photon['WAVELENGTH'], sed_photon['FLUX']*sedNormFactor]).T + sedNormFactor = getNormFactorForSpecWithABMAG( + ABMag=self.param["mag_use_normal"], + spectrum=sed_photon, + norm_thr=normFilter, + sWave=np.floor(normFilter[norm_thr_rang_ids][0][0]), + eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0]), + ) + sed_photon = np.array([ + sed_photon["WAVELENGTH"], + sed_photon["FLUX"] * sedNormFactor, + ]).T # Convert to galsim.SED object - spec = galsim.LookupTable(x=np.array(sed_photon[:, 0]), f=np.array( - sed_photon[:, 1]), interpolant='nearest') - self.sed = galsim.SED(spec, wave_type='A', - flux_type='1', fast=False) + spec = galsim.LookupTable( + x=np.array(sed_photon[:, 0]), + f=np.array(sed_photon[:, 1]), + interpolant="nearest", + ) + self.sed = galsim.SED(spec, wave_type="A", flux_type="1", fast=False) # Get magnitude interFlux = integrate_sed_bandpass( - sed=self.sed, bandpass=target_filt.bandpass_full) - self.param['mag_%s' % target_filt.filter_type] = getABMAG( - interFlux=interFlux, - bandpass=target_filt.bandpass_full) + sed=self.sed, bandpass=target_filt.bandpass_full + ) + self.param["mag_%s" % target_filt.filter_type] = getABMAG( + interFlux=interFlux, bandpass=target_filt.bandpass_full + ) # print('mag_use_normal = ', self.param['mag_use_normal']) # print('mag_%s = '%target_filt.filter_type, self.param['mag_%s'%target_filt.filter_type]) elif survey_type == "spectroscopic": if sed_templates is None: - self.sedPhotons(sed_path=sed_path, - cosids=cosids, objtypes=objtypes) + self.sedPhotons(sed_path=sed_path, cosids=cosids, objtypes=objtypes) else: sed_data = sed_templates[self.sed_type] sed_data = getObservedSED( sedCat=sed_data, redshift=self.z, - av=self.param['av'], - redden=self.param['redden']) + av=self.param["av"], + redden=self.param["redden"], + ) speci = interpolate.interp1d(sed_data[0], sed_data[1]) lamb = np.arange(2500, 10001 + 0.5, 0.5) y = speci(lamb) # erg/s/cm2/A --> photo/s/m2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 self.sed = Table( - np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX") + ) def unload_SED(self): - """(Test) free up SED memory - """ + """(Test) free up SED memory""" del self.sed - def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.): + def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.0): if flux is None: flux = self.getElectronFluxFilt(filt, tel, exptime) - qso = galsim.Gaussian(sigma=1.e-8, flux=1.) + qso = galsim.Gaussian(sigma=1.0e-8, flux=1.0) qso = qso.withFlux(flux) final = galsim.Convolve(psf, qso) return final diff --git a/observation_sim/mock_objects/SpecDisperser/SpecDisperser.py b/observation_sim/mock_objects/SpecDisperser/SpecDisperser.py index 719a10c0c9a17eb850e4e36c0dd7027e6e7500f0..bd636fc59520f2b23ef4d8c56cf30564f4983af3 100644 --- a/observation_sim/mock_objects/SpecDisperser/SpecDisperser.py +++ b/observation_sim/mock_objects/SpecDisperser/SpecDisperser.py @@ -1,24 +1,19 @@ from astropy.table import Table -import astropy.constants as cons -import collections from collections import OrderedDict from scipy import interpolate -from astropy import units as u # from numpy import * import numpy as np -from pylab import * +# from pylab import * import galsim -import sys import os -import math def rotate90(array_orig=None, xc=0, yc=0, isClockwise=0): @@ -168,17 +163,17 @@ class SpecDisperser(object): ) # Account for pixel centering of the trace - yfrac_beam = ytrace_beam - floor(ytrace_beam + 0.5) + yfrac_beam = ytrace_beam - np.floor(ytrace_beam + 0.5) ysens = lam_beam * 0 - lam_index = argsort(lam_beam) + lam_index = np.argsort(lam_beam) conf_sens = self.grating_conf.sens[beam] - lam_intep = np.linspace(self.band_start, self.band_end, int( - (self.band_end - self.band_start) / 0.1)) + lam_intep = np.linspace( + self.band_start, self.band_end, int((self.band_end - self.band_start) / 0.1) + ) - thri = interpolate.interp1d( - conf_sens["WAVELENGTH"], conf_sens["SENSITIVITY"]) + thri = interpolate.interp1d(conf_sens["WAVELENGTH"], conf_sens["SENSITIVITY"]) spci = interpolate.interp1d(self.spec["WAVELENGTH"], self.spec["FLUX"]) beam_thr = thri(lam_intep) @@ -204,20 +199,19 @@ class SpecDisperser(object): sensitivity_beam = ysens len_spec_x = len(dx) - len_spec_y = int( - abs(ceil(ytrace_beam[-1]) - floor(ytrace_beam[0])) + 1) + len_spec_y = int(abs(np.ceil(ytrace_beam[-1]) - np.floor(ytrace_beam[0])) + 1) beam_sh = (self.img_sh[0] + len_spec_y, self.img_sh[1] + len_spec_x) - modelf = zeros(product(beam_sh), dtype=float) + modelf = np.zeros(np.product(beam_sh), dtype=float) model = modelf.reshape(beam_sh) - idx = np.arange(modelf.size, dtype=int64).reshape(beam_sh) - x0 = array((self.thumb_y, self.thumb_x), dtype=int64) + idx = np.arange(modelf.size, dtype=np.int64).reshape(beam_sh) + x0 = np.array((self.thumb_y, self.thumb_x), dtype=np.int64) dxpix = dx - dx[0] + x0[1] - dyc = cast[int](np.floor(ytrace_beam + 0.5)) + dyc = np.cast[int](np.floor(ytrace_beam + 0.5)) - # dypix = cast[int](np.floor(ytrace_beam - dyc[0] + x0[0] + 0.5)) + # dypix = np.cast[int](np.floor(ytrace_beam - dyc[0] + x0[0] + 0.5)) dypix = dyc - dyc[0] + x0[0] frac_ids = yfrac_beam < 0 @@ -229,7 +223,7 @@ class SpecDisperser(object): nonz = sensitivity_beam != 0 - origin_in = zeros_like(self.origin) + origin_in = np.zeros_like(self.origin) dx0_in = dx[0] dy0_in = dyc[0] if self.isAlongY == 1: @@ -248,10 +242,9 @@ class SpecDisperser(object): if self.flat_cube is None: beam_flat = None else: - beam_flat = zeros([len(modelf), len(self.flat_cube)]) + beam_flat = np.zeros([len(modelf), len(self.flat_cube)]) - sub_flat_cube = zeros( - [len(self.flat_cube), beam_sh[0], beam_sh[1]]) + sub_flat_cube = np.zeros([len(self.flat_cube), beam_sh[0], beam_sh[1]]) sub_flat_cube[0] = sub_flat_cube[0] + 1.0 overlap_flag = 1 @@ -281,11 +274,11 @@ class SpecDisperser(object): if overlap_flag == 1: sub_flat_cube[ :, - beam_y_s - originOut_y: beam_y_e - originOut_y + 1, - beam_x_s - originOut_x: beam_x_e - originOut_x + 1, - ] = self.flat_cube[:, beam_y_s: beam_y_e + 1, beam_x_s: beam_x_e + 1] + beam_y_s - originOut_y : beam_y_e - originOut_y + 1, + beam_x_s - originOut_x : beam_x_e - originOut_x + 1, + ] = self.flat_cube[:, beam_y_s : beam_y_e + 1, beam_x_s : beam_x_e + 1] - for i in arange(0, len(self.flat_cube), 1): + for i in np.arange(0, len(self.flat_cube), 1): beam_flat[:, i] = sub_flat_cube[i].flatten() # beam_flat = zeros([len(modelf), len(self.flat_cube)]) # flat_sh = self.flat_cube[0].shape @@ -306,8 +299,8 @@ class SpecDisperser(object): sensitivity_beam[nonz], modelf, x0, - array(self.img_sh, dtype=int64), - array(beam_sh, dtype=int64), + np.array(self.img_sh, dtype=np.int64), + np.array(beam_sh, dtype=np.int64), beam_flat, lam_beam[lam_index][nonz], ) @@ -335,11 +328,11 @@ class SpecDisperser(object): def writerSensitivityFile(self, conffile="", beam="", w=None, sens=None): orders = {"A": "1st", "B": "0st", "C": "2st", "D": "-1st", "E": "-2st"} - sens_file_name = conffile[0:-5] + \ - "_sensitivity_" + orders[beam] + ".fits" + sens_file_name = conffile[0:-5] + "_sensitivity_" + orders[beam] + ".fits" if os.path.exists(sens_file_name) is False: senstivity_out = Table( - array([w, sens]).T, names=("WAVELENGTH", "SENSITIVITY")) + np.array([w, sens]).T, names=("WAVELENGTH", "SENSITIVITY") + ) senstivity_out.write(sens_file_name, format="fits") @@ -436,13 +429,16 @@ class aXeConf: if self.orders[beam] > 0: self.beams.append(beam) self.dxlam[beam] = np.arange( - self.conf["BEAM{0}".format(beam)].min(), self.conf["BEAM{0}".format(beam)].max(), dtype=int + self.conf["BEAM{0}".format(beam)].min(), + self.conf["BEAM{0}".format(beam)].max(), + dtype=int, ) - self.nx[beam] = int(self.dxlam[beam].max() - - self.dxlam[beam].min()) + 1 + self.nx[beam] = int(self.dxlam[beam].max() - self.dxlam[beam].min()) + 1 self.sens[beam] = Table.read( - "{0}/{1}".format(os.path.dirname(self.conf_file), - self.conf["SENSITIVITY_{0}".format(beam)]) + "{0}/{1}".format( + os.path.dirname(self.conf_file), + self.conf["SENSITIVITY_{0}".format(beam)], + ) ) # self.sens[beam].wave = np.cast[np.double](self.sens[beam]['WAVELENGTH']) # self.sens[beam].sens = np.cast[np.double](self.sens[beam]['SENSITIVITY']) @@ -557,8 +553,7 @@ class aXeConf: dpfull = xfull * 0.0 lt0 = xfull < 0 if lt0.sum() > 1: - dpfull[lt0] = np.cumsum( - np.sqrt(1 + dyfull[lt0][::-1] ** 2))[::-1] + dpfull[lt0] = np.cumsum(np.sqrt(1 + dyfull[lt0][::-1] ** 2))[::-1] dpfull[lt0] *= -1 # @@ -602,10 +597,8 @@ class aXeConf: NORDER = self.orders[beam] + 1 xi, yi = x - self.xoff, y - self.yoff - xoff_beam = self.field_dependent( - xi, yi, self.conf["XOFF_{0}".format(beam)]) - yoff_beam = self.field_dependent( - xi, yi, self.conf["YOFF_{0}".format(beam)]) + xoff_beam = self.field_dependent(xi, yi, self.conf["XOFF_{0}".format(beam)]) + yoff_beam = self.field_dependent(xi, yi, self.conf["YOFF_{0}".format(beam)]) # y offset of trace (DYDX) dydx = np.zeros(NORDER) # 0 #+1.e-80 @@ -702,24 +695,40 @@ class aXeConf: s = 200 # marker size fig = plt.figure(figsize=[10, 3]) - plt.scatter(0, 0, marker="s", s=s, color="black", - edgecolor="0.8", label="Direct") + plt.scatter( + 0, 0, marker="s", s=s, color="black", edgecolor="0.8", label="Direct" + ) for beam in beams: if "XOFF_{0}".format(beam) not in self.conf.keys(): continue - xoff = self.field_dependent( - x0, x1, self.conf["XOFF_{0}".format(beam)]) + xoff = self.field_dependent(x0, x1, self.conf["XOFF_{0}".format(beam)]) dy, lam = self.get_beam_trace(x0, x1, dx=dx, beam=beam) xlim = self.conf["BEAM{0}".format(beam)] ok = (dx >= xlim[0]) & (dx <= xlim[1]) - plt.scatter(dx[ok] + xoff, dy[ok], c=lam[ok] / 1.0e4, - marker="s", s=s, alpha=0.5, edgecolor="None") - plt.text(np.median(dx[ok]), np.median( - dy[ok]) + 1, beam, ha="center", va="center", fontsize=14) - print("Beam {0}, lambda=({1:.1f} - {2:.1f})".format(beam, - lam[ok].min(), lam[ok].max())) + plt.scatter( + dx[ok] + xoff, + dy[ok], + c=lam[ok] / 1.0e4, + marker="s", + s=s, + alpha=0.5, + edgecolor="None", + ) + plt.text( + np.median(dx[ok]), + np.median(dy[ok]) + 1, + beam, + ha="center", + va="center", + fontsize=14, + ) + print( + "Beam {0}, lambda=({1:.1f} - {2:.1f})".format( + beam, lam[ok].min(), lam[ok].max() + ) + ) plt.grid() plt.xlabel(r"$\Delta x$") diff --git a/observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx b/observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx index 707c6707fa2f3044f915bf54c6f8087970452ee2..9782b20147d2fc767e53e1574fa5a3474be84d55 100644 --- a/observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx +++ b/observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx @@ -4,6 +4,8 @@ from __future__ import division import numpy as np cimport numpy as np +np.import_array() + DTYPE = np.double ITYPE = np.int64 diff --git a/observation_sim/mock_objects/SpecDisperser/setup_c.py b/observation_sim/mock_objects/SpecDisperser/setup_c.py index e5a4467b4145ef8b5b6a5edf2c5d116c1a0c75ee..9a96bd8a893e2f145a72f93d3792b4073127f4ba 100644 --- a/observation_sim/mock_objects/SpecDisperser/setup_c.py +++ b/observation_sim/mock_objects/SpecDisperser/setup_c.py @@ -1,19 +1,23 @@ from setuptools import setup from setuptools.extension import Extension -from setuptools.config import read_configuration from Cython.Build import cythonize import numpy extensions = [ - Extension("disperse_c.interp", ["disperse_c/interp.pyx"], - include_dirs=[numpy.get_include()], - libraries=["m"]), - - Extension("disperse_c.disperse", ["disperse_c/disperse.pyx"], - include_dirs=[numpy.get_include()], - libraries=["m"]), + Extension( + "disperse_c.interp", + ["disperse_c/interp.pyx"], + include_dirs=[numpy.get_include()], + libraries=["m"], + ), + Extension( + "disperse_c.disperse", + ["disperse_c/disperse.pyx"], + include_dirs=[numpy.get_include()], + libraries=["m"], + ), ] diff --git a/observation_sim/mock_objects/Stamp.py b/observation_sim/mock_objects/Stamp.py index d39ace7ec70edac6904dc2e992bb067a3c6c3872..4033d36effd42f47b5d80700ec86ff7d560e5963 100644 --- a/observation_sim/mock_objects/Stamp.py +++ b/observation_sim/mock_objects/Stamp.py @@ -1,14 +1,13 @@ -import os -import sys import numpy as np import galsim -import astropy.constants as cons from astropy.table import Table -from scipy import interpolate from observation_sim.mock_objects.MockObject import MockObject from observation_sim.mock_objects.SpecDisperser import SpecDisperser -from observation_sim.mock_objects._util import eObs, integrate_sed_bandpass, getNormFactorForSpecWithABMAG, getObservedSED, getABMAG, convolveGaussXorders +from observation_sim.mock_objects._util import ( + integrate_sed_bandpass, + getNormFactorForSpecWithABMAG, +) class Stamp(MockObject): @@ -16,17 +15,28 @@ class Stamp(MockObject): super().__init__(param, logger=logger) def unload_SED(self): - """(Test) free up SED memory - """ + """(Test) free up SED memory""" del self.sed - def drawObj_multiband(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, exptime=150., fd_shear=None): + def drawObj_multiband( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + fd_shear=None, + ): if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) self.logger.error(e) @@ -37,13 +47,17 @@ class Stamp(MockObject): # xmax, ymax = 0, 0 if self.getMagFilter(filt) <= 15: - folding_threshold = 5.e-4 + folding_threshold = 5.0e-4 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) @@ -69,7 +83,7 @@ class Stamp(MockObject): if self.logger: self.logger.error(e) continue - ratio = sub/full + ratio = sub / full if not (ratio == -1 or (ratio != ratio)): nphotons = ratio * nphotons_tot else: @@ -77,10 +91,14 @@ class Stamp(MockObject): # nphotons_sum += nphotons psf, pos_shear = psf_model.get_PSF( - chip=chip, pos_img=pos_img, bandpass=bandpass, folding_threshold=folding_threshold) - - _gal = self.param['image'] - galImg = galsim.ImageF(_gal, scale=self.param['pixScale']) + chip=chip, + pos_img=pos_img, + bandpass=bandpass, + folding_threshold=folding_threshold, + ) + + _gal = self.param["image"] + galImg = galsim.ImageF(_gal, scale=self.param["pixScale"]) gal_temp = galsim.InterpolatedImage(galImg) gal_temp = gal_temp.shear(gal_shear) gal_temp = gal_temp.withFlux(nphotons) @@ -98,8 +116,7 @@ class Stamp(MockObject): return 2, pos_shear stamp.setCenter(x_nominal, y_nominal) - bounds = stamp.bounds & galsim.BoundsI( - 0, chip.npix_x - 1, 0, chip.npix_y - 1) + bounds = stamp.bounds & galsim.BoundsI(0, chip.npix_x - 1, 0, chip.npix_y - 1) if bounds.area() > 0: chip.img.setOrigin(0, 0) @@ -116,24 +133,46 @@ class Stamp(MockObject): return 1, pos_shear - def drawObj_slitless(self, tel, pos_img, psf_model, bandpass_list, filt, chip, nphotons_tot=None, g1=0, g2=0, - exptime=150., normFilter=None, grating_split_pos=3685, fd_shear=None): + def drawObj_slitless( + self, + tel, + pos_img, + psf_model, + bandpass_list, + filt, + chip, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + normFilter=None, + grating_split_pos=3685, + fd_shear=None, + ): if normFilter is not None: - norm_thr_rang_ids = normFilter['SENSITIVITY'] > 0.001 - sedNormFactor = getNormFactorForSpecWithABMAG(ABMag=self.param['mag_use_normal'], spectrum=self.sed, - norm_thr=normFilter, - sWave=np.floor( - normFilter[norm_thr_rang_ids][0][0]), - eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0])) + norm_thr_rang_ids = normFilter["SENSITIVITY"] > 0.001 + sedNormFactor = getNormFactorForSpecWithABMAG( + ABMag=self.param["mag_use_normal"], + spectrum=self.sed, + norm_thr=normFilter, + sWave=np.floor(normFilter[norm_thr_rang_ids][0][0]), + eWave=np.ceil(normFilter[norm_thr_rang_ids][-1][0]), + ) if sedNormFactor == 0: return 2, None else: - sedNormFactor = 1. - normalSED = Table(np.array([self.sed['WAVELENGTH'], self.sed['FLUX'] * sedNormFactor]).T, - names=('WAVELENGTH', 'FLUX')) - - self.real_pos = self.getRealPos(chip.img, global_x=self.posImg.x, global_y=self.posImg.y, - img_real_wcs=self.chip_wcs) + sedNormFactor = 1.0 + normalSED = Table( + np.array([self.sed["WAVELENGTH"], self.sed["FLUX"] * sedNormFactor]).T, + names=("WAVELENGTH", "FLUX"), + ) + + self.real_pos = self.getRealPos( + chip.img, + global_x=self.posImg.x, + global_y=self.posImg.y, + img_real_wcs=self.chip_wcs, + ) x, y = self.real_pos.x + 0.5, self.real_pos.y + 0.5 x_nominal = int(np.floor(x + 0.5)) @@ -145,23 +184,28 @@ class Stamp(MockObject): chip_wcs_local = self.chip_wcs.local(self.real_pos) if self.getMagFilter(filt) <= 15: - folding_threshold = 5.e-4 + folding_threshold = 5.0e-4 else: - folding_threshold = 5.e-3 + folding_threshold = 5.0e-3 gsp = galsim.GSParams(folding_threshold=folding_threshold) # nphotons_sum = 0 flat_cube = chip.flat_cube - xOrderSigPlus = {'A': 1.3909419820029296, 'B': 1.4760376591236062, - 'C': 4.035447379743442, 'D': 5.5684364343742825, 'E': 16.260021029735388} + xOrderSigPlus = { + "A": 1.3909419820029296, + "B": 1.4760376591236062, + "C": 4.035447379743442, + "D": 5.5684364343742825, + "E": 16.260021029735388, + } grating_split_pos_chip = 0 + grating_split_pos branges = np.zeros([len(bandpass_list), 2]) # print(hasattr(psf_model, 'bandranges')) - if hasattr(psf_model, 'bandranges'): + if hasattr(psf_model, "bandranges"): if psf_model.bandranges is None: return 2, None if len(psf_model.bandranges) != len(bandpass_list): @@ -178,8 +222,8 @@ class Stamp(MockObject): # psf, pos_shear = psf_model.get_PSF(chip=chip, pos_img=pos_img, bandpass=bandpass, folding_threshold=folding_threshold) - _gal = self.param['image'] - galImg = galsim.ImageF(_gal, scale=self.param['pixScale']) + _gal = self.param["image"] + galImg = galsim.ImageF(_gal, scale=self.param["pixScale"]) gal = galsim.InterpolatedImage(galImg) # (TEST) Random knots @@ -201,14 +245,19 @@ class Stamp(MockObject): # # gal = gal.shear(fd_shear) starImg = gal.drawImage( - wcs=chip_wcs_local, offset=offset, method='real_space') + wcs=chip_wcs_local, offset=offset, method="real_space" + ) - origin_star = [y_nominal - (starImg.center.y - starImg.ymin), - x_nominal - (starImg.center.x - starImg.xmin)] + origin_star = [ + y_nominal - (starImg.center.y - starImg.ymin), + x_nominal - (starImg.center.x - starImg.xmin), + ] starImg.setOrigin(0, 0) gal_origin = [origin_star[0], origin_star[1]] - gal_end = [origin_star[0] + starImg.array.shape[0] - - 1, origin_star[1] + starImg.array.shape[1] - 1] + gal_end = [ + origin_star[0] + starImg.array.shape[0] - 1, + origin_star[1] + starImg.array.shape[1] - 1, + ] if gal_origin[1] < grating_split_pos_chip < gal_end[1]: subSlitPos = int(grating_split_pos_chip - gal_origin[1] + 1) @@ -218,74 +267,117 @@ class Stamp(MockObject): star_p1 = galsim.Image(subImg_p1) star_p1.setOrigin(0, 0) origin_p1 = origin_star - xcenter_p1 = min(x_nominal, grating_split_pos_chip-1) - 0 - ycenter_p1 = y_nominal-0 - - sdp_p1 = SpecDisperser(orig_img=star_p1, xcenter=xcenter_p1, - ycenter=ycenter_p1, origin=origin_p1, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + xcenter_p1 = min(x_nominal, grating_split_pos_chip - 1) - 0 + ycenter_p1 = y_nominal - 0 + + sdp_p1 = SpecDisperser( + orig_img=star_p1, + xcenter=xcenter_p1, + ycenter=ycenter_p1, + origin=origin_p1, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) # self.addSLStoChipImage(sdp=sdp_p1, chip=chip, xOrderSigPlus = xOrderSigPlus, local_wcs=chip_wcs_local) - pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p1, chip=chip, pos_img_local=[xcenter_p1, ycenter_p1], - psf_model=psf_model, bandNo=i + 1, - grating_split_pos=grating_split_pos, - local_wcs=chip_wcs_local, pos_img=pos_img) - - subImg_p2 = starImg.array[:, - subSlitPos+1:starImg.array.shape[1]] + pos_shear = self.addSLStoChipImageWithPSF( + sdp=sdp_p1, + chip=chip, + pos_img_local=[xcenter_p1, ycenter_p1], + psf_model=psf_model, + bandNo=i + 1, + grating_split_pos=grating_split_pos, + local_wcs=chip_wcs_local, + pos_img=pos_img, + ) + + subImg_p2 = starImg.array[:, subSlitPos + 1 : starImg.array.shape[1]] star_p2 = galsim.Image(subImg_p2) star_p2.setOrigin(0, 0) origin_p2 = [origin_star[0], grating_split_pos_chip] xcenter_p2 = max(x_nominal, grating_split_pos_chip - 1) - 0 ycenter_p2 = y_nominal - 0 - sdp_p2 = SpecDisperser(orig_img=star_p2, xcenter=xcenter_p2, - ycenter=ycenter_p2, origin=origin_p2, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp_p2 = SpecDisperser( + orig_img=star_p2, + xcenter=xcenter_p2, + ycenter=ycenter_p2, + origin=origin_p2, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) # self.addSLStoChipImage(sdp=sdp_p2, chip=chip, xOrderSigPlus = xOrderSigPlus, local_wcs=chip_wcs_local) - pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp_p2, chip=chip, pos_img_local=[xcenter_p2, ycenter_p2], - psf_model=psf_model, bandNo=i + 1, - grating_split_pos=grating_split_pos, - local_wcs=chip_wcs_local, pos_img=pos_img) + pos_shear = self.addSLStoChipImageWithPSF( + sdp=sdp_p2, + chip=chip, + pos_img_local=[xcenter_p2, ycenter_p2], + psf_model=psf_model, + bandNo=i + 1, + grating_split_pos=grating_split_pos, + local_wcs=chip_wcs_local, + pos_img=pos_img, + ) del sdp_p1 del sdp_p2 elif grating_split_pos_chip <= gal_origin[1]: - sdp = SpecDisperser(orig_img=starImg, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[1], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=starImg, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[1], + isAlongY=0, + flat_cube=flat_cube, + ) # self.addSLStoChipImage(sdp=sdp, chip=chip, xOrderSigPlus = xOrderSigPlus, local_wcs=chip_wcs_local) - pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], - psf_model=psf_model, bandNo=i + 1, - grating_split_pos=grating_split_pos, - local_wcs=chip_wcs_local, pos_img=pos_img) + pos_shear = self.addSLStoChipImageWithPSF( + sdp=sdp, + chip=chip, + pos_img_local=[x_nominal, y_nominal], + psf_model=psf_model, + bandNo=i + 1, + grating_split_pos=grating_split_pos, + local_wcs=chip_wcs_local, + pos_img=pos_img, + ) del sdp elif grating_split_pos_chip >= gal_end[1]: - sdp = SpecDisperser(orig_img=starImg, xcenter=x_nominal - 0, - ycenter=y_nominal - 0, origin=origin_star, - tar_spec=normalSED, - band_start=brange[0], band_end=brange[1], - conf=chip.sls_conf[0], - isAlongY=0, - flat_cube=flat_cube) + sdp = SpecDisperser( + orig_img=starImg, + xcenter=x_nominal - 0, + ycenter=y_nominal - 0, + origin=origin_star, + tar_spec=normalSED, + band_start=brange[0], + band_end=brange[1], + conf=chip.sls_conf[0], + isAlongY=0, + flat_cube=flat_cube, + ) # self.addSLStoChipImage(sdp=sdp, chip=chip, xOrderSigPlus = xOrderSigPlus, local_wcs=chip_wcs_local) - pos_shear = self.addSLStoChipImageWithPSF(sdp=sdp, chip=chip, pos_img_local=[x_nominal, y_nominal], - psf_model=psf_model, bandNo=i + 1, - grating_split_pos=grating_split_pos, - local_wcs=chip_wcs_local, pos_img=pos_img) + pos_shear = self.addSLStoChipImageWithPSF( + sdp=sdp, + chip=chip, + pos_img_local=[x_nominal, y_nominal], + psf_model=psf_model, + bandNo=i + 1, + grating_split_pos=grating_split_pos, + local_wcs=chip_wcs_local, + pos_img=pos_img, + ) del sdp # print(self.y_nominal, starImg.center.y, starImg.ymin) diff --git a/observation_sim/mock_objects/Star.py b/observation_sim/mock_objects/Star.py index c0aceb3d1ea5357f09a41fee421ca95c561e4311..af7e27a081369f7968820aee82d3cff9ec8848b5 100755 --- a/observation_sim/mock_objects/Star.py +++ b/observation_sim/mock_objects/Star.py @@ -1,12 +1,8 @@ import galsim -import os -import sys -import numpy as np -import astropy.constants as cons -from astropy.table import Table -from scipy import interpolate -from observation_sim.mock_objects._util import integrate_sed_bandpass, getNormFactorForSpecWithABMAG, getObservedSED, getABMAG, tag_sed +from observation_sim.mock_objects._util import ( + integrate_sed_bandpass, +) from observation_sim.mock_objects.MockObject import MockObject @@ -14,14 +10,13 @@ class Star(MockObject): def __init__(self, param, logger=None): super().__init__(param, logger=logger) if not hasattr(self, "mu"): - self.mu = 1. + self.mu = 1.0 def unload_SED(self): - """(Test) free up SED memory - """ + """(Test) free up SED memory""" del self.sed - def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.): + def getGSObj(self, psf, g1=0, g2=0, flux=None, filt=None, tel=None, exptime=150.0): if flux is None: flux = self.getElectronFluxFilt(filt, tel, exptime) # star = galsim.Gaussian(sigma=1.e-8, flux=1.) @@ -30,17 +25,27 @@ class Star(MockObject): final = galsim.Convolve(psf, star) return final - def getGSObj_multiband(self, tel, psf_list, bandpass_list, filt, nphotons_tot=None, g1=0, g2=0, exptime=150.): + def getGSObj_multiband( + self, + tel, + psf_list, + bandpass_list, + filt, + nphotons_tot=None, + g1=0, + g2=0, + exptime=150.0, + ): if len(psf_list) != len(bandpass_list): raise ValueError( - "!!!The number of PSF profiles and the number of bandpasses must be equal.") + "!!!The number of PSF profiles and the number of bandpasses must be equal." + ) objs = [] if nphotons_tot is None: nphotons_tot = self.getElectronFluxFilt(filt, tel, exptime) try: - full = integrate_sed_bandpass( - sed=self.sed, bandpass=filt.bandpass_full) + full = integrate_sed_bandpass(sed=self.sed, bandpass=filt.bandpass_full) except Exception as e: print(e) self.logger.error(e) @@ -56,7 +61,7 @@ class Star(MockObject): self.logger.error(e) return -1 - ratio = sub/full + ratio = sub / full if not (ratio == -1 or (ratio != ratio)): nphotons = ratio * nphotons_tot diff --git a/observation_sim/mock_objects/__init__.py b/observation_sim/mock_objects/__init__.py index c6c567f9332eb3b4b109b4ff7589903b24eb7ca9..b82341546ba93c6671e3cd5748dcc53998b8356c 100755 --- a/observation_sim/mock_objects/__init__.py +++ b/observation_sim/mock_objects/__init__.py @@ -6,3 +6,4 @@ from .Star import Star from .Stamp import Stamp from .FlatLED import FlatLED from .ExtinctionMW import ExtinctionMW +from .Ghost import Ghost diff --git a/observation_sim/mock_objects/_util.py b/observation_sim/mock_objects/_util.py index 5707fd4f561cb7ddaa7c8b86d86a397cfa7d6237..5043dbde5326bdd2083501e4113c652f1c5c0ac5 100755 --- a/observation_sim/mock_objects/_util.py +++ b/observation_sim/mock_objects/_util.py @@ -1,23 +1,23 @@ import numpy as np -import os from scipy.interpolate import InterpolatedUnivariateSpline, interp1d from scipy import interpolate, integrate from astropy.table import Table import galsim -VC_A = 2.99792458e+18 # speed of light: A/s -VC_M = 2.99792458e+8 # speed of light: m/s +VC_A = 2.99792458e18 # speed of light: A/s +VC_M = 2.99792458e8 # speed of light: m/s H_PLANK = 6.626196e-27 # Plank constant: erg s def comoving_dist(z, om_m=0.3111, om_L=0.6889, h=0.6766): # Return comving distance in pc - H0 = h*100. # km / (s Mpc) + H0 = h * 100.0 # km / (s Mpc) def dist_int(z): - return 1./np.sqrt(om_m*(1.+z)**3 + om_L) - res, err = integrate.quad(dist_int, 0., z) - return [res * (VC_M/1e3/H0) * 1e6, err * (VC_M/1e3/H0) * 1e6] + return 1.0 / np.sqrt(om_m * (1.0 + z) ** 3 + om_L) + + res, err = integrate.quad(dist_int, 0.0, z) + return [res * (VC_M / 1e3 / H0) * 1e6, err * (VC_M / 1e3 / H0) * 1e6] def magToFlux(mag): @@ -30,7 +30,7 @@ def magToFlux(mag): Return: flux: flux in unit of erg/s/cm^2/Hz """ - flux = 10**(-0.4*(mag+48.6)) + flux = 10 ** (-0.4 * (mag + 48.6)) return flux @@ -42,12 +42,15 @@ def extAv(nav, seed=1212123): np.random.seed(seed) tau = 0.4 peak, a = 0.1, 0.5 - b = a*(tau-peak) - def pav(av): return (a*av+b)*np.exp(-av/tau) - avmin, avmax = 0., 3. - avs = np.linspace(avmin, avmax, int((avmax-avmin)/0.001)+1) + b = a * (tau - peak) + + def pav(av): + return (a * av + b) * np.exp(-av / tau) + + avmin, avmax = 0.0, 3.0 + avs = np.linspace(avmin, avmax, int((avmax - avmin) / 0.001) + 1) norm = np.trapz(pav(avs), avs) - pav_base = pav(avs)/np.sum(pav(avs)) + pav_base = pav(avs) / np.sum(pav(avs)) rav = np.random.choice(avs, nav, p=pav_base) return rav @@ -70,10 +73,10 @@ def seds(sedlistn, seddir="./", unit="A"): reds = {} sedlist = seddir + sedlistn sedn = open(sedlist).read().splitlines() - sedtype = range(1, len(sedn)+1) + sedtype = range(1, len(sedn) + 1) for i in range(len(sedn)): xxx = sedn[i].split() - isedn = seddir+xxx[0] + isedn = seddir + xxx[0] itype = sedtype[i] ised = np.loadtxt(isedn) if unit == "nm": @@ -111,6 +114,7 @@ def sed_assign(phz, btt, rng): sedtype = 0 return sedtype + ########################################### @@ -141,30 +145,32 @@ def tflux(filt, sed, redshift=0.0, av=0.0, redden=0): sw, sf = sed[:, 0], sed[:, 1] # reddening sf = reddening(sw, sf, av=av, model=redden) - sw, sf = sw*z, sf*(z**3) + sw, sf = sw * z, sf * (z**3) # lyman forest correction sf = lyman_forest(sw, sf, redshift) sedxx = (sw.copy(), sf.copy()) - sw = VC_A/sw - sf = sf*(VC_A/sw**2) # convert flux unit to erg/s/cm^s/Hz + sw = VC_A / sw + sf = sf * (VC_A / sw**2) # convert flux unit to erg/s/cm^s/Hz sw, sf = sw[::-1], sf[::-1] - sfun = interp1d(sw, sf, kind='linear') + sfun = interp1d(sw, sf, kind="linear") fwave, fresp = filt[:, 0], filt[:, 1] - fwave = VC_A/fwave + fwave = VC_A / fwave fwave, fresp = fwave[::-1], fresp[::-1] tflux = sfun(fwave) - zpflux = 3.631*1.0e-20 + zpflux = 3.631 * 1.0e-20 - tflux = np.trapz(tflux*fresp/fwave, fwave) / \ - np.trapz(zpflux*fresp/fwave, fwave) + tflux = np.trapz(tflux * fresp / fwave, fwave) / np.trapz( + zpflux * fresp / fwave, fwave + ) # tflux = np.trapz(tflux*fresp,fwave)/np.trapz(zpflux*fresp,fwave) return tflux, sedxx + ########################################### @@ -178,22 +184,24 @@ def lyman_forest(wavelen, flux, z): flux0 = flux else: nw = 200 - istep = np.linspace(0, nw-1, nw) - w1a, w2a = 1050.0*(1.0+z), 1170.0*(1.0+z) - w1b, w2b = 920.0*(1.0+z), 1015.0*(1.0+z) - wstepa = (w2a-w1a)/float(nw) - wstepb = (w2b-w1b)/float(nw) + istep = np.linspace(0, nw - 1, nw) + w1a, w2a = 1050.0 * (1.0 + z), 1170.0 * (1.0 + z) + w1b, w2b = 920.0 * (1.0 + z), 1015.0 * (1.0 + z) + wstepa = (w2a - w1a) / float(nw) + wstepb = (w2b - w1b) / float(nw) - wtempa = w1a + istep*wstepa - ptaua = np.exp(-3.6e-03*(wtempa/1216.0)**3.46) + wtempa = w1a + istep * wstepa + ptaua = np.exp(-3.6e-03 * (wtempa / 1216.0) ** 3.46) - wtempb = w1b + istep*wstepb - ptaub = np.exp(-1.7e-3*(wtempb/1026.0)**3.46 - - 1.2e-3*(wtempb/972.50)**3.46 - - 9.3e-4*(wtempb/950.00)**3.46) + wtempb = w1b + istep * wstepb + ptaub = np.exp( + -1.7e-3 * (wtempb / 1026.0) ** 3.46 + - 1.2e-3 * (wtempb / 972.50) ** 3.46 + - 9.3e-4 * (wtempb / 950.00) ** 3.46 + ) - da = (1.0/(120.0*(1.0+z)))*np.trapz(ptaua, wtempa) - db = (1.0/(95.0*(1.0+z)))*np.trapz(ptaub, wtempb) + da = (1.0 / (120.0 * (1.0 + z))) * np.trapz(ptaua, wtempa) + db = (1.0 / (95.0 * (1.0 + z))) * np.trapz(ptaub, wtempb) if da > 1.0: da = 1.0 @@ -204,11 +212,12 @@ def lyman_forest(wavelen, flux, z): if db < 0.0: db = 0.0 flux0 = flux.copy() - id0 = wavelen <= 1026.0*(1.0+z) - id1 = np.logical_and(wavelen < 1216.0*(1.0+z), - wavelen >= 1026.0*(1.0+z)) - flux0[id0] = db*flux[id0] - flux0[id1] = da*flux[id1] + id0 = wavelen <= 1026.0 * (1.0 + z) + id1 = np.logical_and( + wavelen < 1216.0 * (1.0 + z), wavelen >= 1026.0 * (1.0 + z) + ) + flux0[id0] = db * flux[id0] + flux0[id1] = da * flux[id1] return flux0 @@ -239,145 +248,264 @@ def reddening(sw, sf, av=0.0, model=0): if model == 0 or av == 0.0: flux = sf elif model == 1: # Allen (1976) for the Milky Way - lambda0 = np.array([1000, 1110, 1250, 1430, 1670, - 2000, 2220, 2500, 2850, 3330, - 3650, 4000, 4400, 5000, 5530, - 6700, 9000, 10000, 20000, 100000], dtype=float) - kR = np.array([4.20, 3.70, 3.30, 3.00, 2.70, - 2.80, 2.90, 2.30, 1.97, 1.69, - 1.58, 1.45, 1.32, 1.13, 1.00, - 0.74, 0.46, 0.38, 0.11, 0.00], dtype=float) + lambda0 = np.array( + [ + 1000, + 1110, + 1250, + 1430, + 1670, + 2000, + 2220, + 2500, + 2850, + 3330, + 3650, + 4000, + 4400, + 5000, + 5530, + 6700, + 9000, + 10000, + 20000, + 100000, + ], + dtype=float, + ) + kR = np.array( + [ + 4.20, + 3.70, + 3.30, + 3.00, + 2.70, + 2.80, + 2.90, + 2.30, + 1.97, + 1.69, + 1.58, + 1.45, + 1.32, + 1.13, + 1.00, + 0.74, + 0.46, + 0.38, + 0.11, + 0.00, + ], + dtype=float, + ) ext0 = InterpolatedUnivariateSpline(lambda0, kR, k=1) - A_lambda = av*ext0(sw) + A_lambda = av * ext0(sw) A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) elif model == 2: # Seaton (1979) fit by Fitzpatrick (1986) for the Milky Way Rv = 3.1 al0, ga, c1, c2, c3, c4 = 4.595, 1.051, -0.38, 0.74, 3.96, 0.26 ff11 = __red(1100.0, al0, ga, c1, c2, c3, c4) ff12 = __red(1200.0, al0, ga, c1, c2, c3, c4) - slope = (ff12-ff11)/100.0 - lambda0 = np.array([3650, 4000, 4400, 5000, 5530, - 6700, 9000, 10000, 20000, 100000], dtype=float) - kR = np.array([1.58, 1.45, 1.32, 1.13, 1.00, - 0.74, 0.46, 0.38, 0.11, 0.00], dtype=float) - fun = interp1d(lambda0, kR, kind='linear') + slope = (ff12 - ff11) / 100.0 + lambda0 = np.array( + [3650, 4000, 4400, 5000, 5530, 6700, 9000, 10000, 20000, 100000], + dtype=float, + ) + kR = np.array( + [1.58, 1.45, 1.32, 1.13, 1.00, 0.74, 0.46, 0.38, 0.11, 0.00], dtype=float + ) + fun = interp1d(lambda0, kR, kind="linear") sw0 = sw[sw < 1200.0] - A_lambda0 = (ff11+(sw0-1100.0)*slope)/Rv+1.0 + A_lambda0 = (ff11 + (sw0 - 1100.0) * slope) / Rv + 1.0 sw1 = sw[np.logical_and(sw >= 1200.0, sw <= 3650.0)] ff = __red(sw1, al0, ga, c1, c2, c3, c4) - A_lambda1 = ff/Rv+1.0 + A_lambda1 = ff / Rv + 1.0 sw2 = sw[np.logical_and(sw > 3650.0, sw <= 100000.0)] A_lambda2 = fun(sw2) - A_lambda3 = sw[sw > 100000.0]*0.0 - A_lambda = av*np.hstack([A_lambda0, A_lambda1, A_lambda2, A_lambda3]) + A_lambda3 = sw[sw > 100000.0] * 0.0 + A_lambda = av * np.hstack([A_lambda0, A_lambda1, A_lambda2, A_lambda3]) A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) elif model == 3: # Fitzpatrick (1986) for the Large Magellanic Cloud (LMC) Rv = 3.1 al0, ga, c1, c2, c3, c4 = 4.608, 0.994, -0.69, 0.89, 2.55, 0.50 ff11 = __red(1100.0, al0, ga, c1, c2, c3, c4) ff12 = __red(1200.0, al0, ga, c1, c2, c3, c4) - slope = (ff12-ff11)/100.0 - lambda0 = np.array([3330, 3650, 4000, 4400, 5000, 5530, - 6700, 9000, 10000, 20000, 100000], dtype=float) - kR = np.array([1.682, 1.58, 1.45, 1.32, 1.13, 1.00, - 0.74, 0.46, 0.38, 0.11, 0.00], dtype=float) - fun = interp1d(lambda0, kR, kind='linear') + slope = (ff12 - ff11) / 100.0 + lambda0 = np.array( + [3330, 3650, 4000, 4400, 5000, 5530, 6700, 9000, 10000, 20000, 100000], + dtype=float, + ) + kR = np.array( + [1.682, 1.58, 1.45, 1.32, 1.13, 1.00, 0.74, 0.46, 0.38, 0.11, 0.00], + dtype=float, + ) + fun = interp1d(lambda0, kR, kind="linear") sw0 = sw[sw < 1200.0] - A_lambda0 = (ff11+(sw0-1100.0)*slope)/Rv+1.0 + A_lambda0 = (ff11 + (sw0 - 1100.0) * slope) / Rv + 1.0 sw1 = sw[np.logical_and(sw >= 1200.0, sw <= 3330.0)] ff = __red(sw1, al0, ga, c1, c2, c3, c4) - A_lambda1 = ff/Rv+1.0 + A_lambda1 = ff / Rv + 1.0 sw2 = sw[np.logical_and(sw > 3330.0, sw <= 100000.0)] A_lambda2 = fun(sw2) - A_lambda3 = sw[sw > 100000.0]*0.0 - A_lambda = av*np.hstack([A_lambda0, A_lambda1, A_lambda2, A_lambda3]) + A_lambda3 = sw[sw > 100000.0] * 0.0 + A_lambda = av * np.hstack([A_lambda0, A_lambda1, A_lambda2, A_lambda3]) A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) # Prevot et al (1984) and Bouchet (1985) for the Small Magellanic Cloud (SMC) elif model == 4: Rv = 2.72 - lambda0 = np.array([1275, 1330, 1385, 1435, 1490, 1545, - 1595, 1647, 1700, 1755, 1810, 1860, - 1910, 2000, 2115, 2220, 2335, 2445, - 2550, 2665, 2778, 2890, 2995, 3105, - 3704, 4255, 5291, 12500, 16500, 22000], dtype=float) - kR = np.array([13.54, 12.52, 11.51, 10.80, 9.84, 9.28, - 9.06, 8.49, 8.01, 7.71, 7.17, 6.90, 6.76, - 6.38, 5.85, 5.30, 4.53, 4.24, 3.91, 3.49, - 3.15, 3.00, 2.65, 2.29, 1.81, 1.00, 0.00, - -2.02, -2.36, -2.47], dtype=float) - kR = kR/Rv+1.0 + lambda0 = np.array( + [ + 1275, + 1330, + 1385, + 1435, + 1490, + 1545, + 1595, + 1647, + 1700, + 1755, + 1810, + 1860, + 1910, + 2000, + 2115, + 2220, + 2335, + 2445, + 2550, + 2665, + 2778, + 2890, + 2995, + 3105, + 3704, + 4255, + 5291, + 12500, + 16500, + 22000, + ], + dtype=float, + ) + kR = np.array( + [ + 13.54, + 12.52, + 11.51, + 10.80, + 9.84, + 9.28, + 9.06, + 8.49, + 8.01, + 7.71, + 7.17, + 6.90, + 6.76, + 6.38, + 5.85, + 5.30, + 4.53, + 4.24, + 3.91, + 3.49, + 3.15, + 3.00, + 2.65, + 2.29, + 1.81, + 1.00, + 0.00, + -2.02, + -2.36, + -2.47, + ], + dtype=float, + ) + kR = kR / Rv + 1.0 ext0 = InterpolatedUnivariateSpline(lambda0, kR, k=1) - A_lambda = av*ext0(sw) + A_lambda = av * ext0(sw) A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) elif model == 5: # Calzetti et al (2000) for starburst galaxies Rv = 4.05 - sw = sw*1.0e-04 # wavelength in microns + sw = sw * 1.0e-04 # wavelength in microns + + def fun1(x): + return 2.659 * (-2.156 + 1.509 / x - 0.198 / x**2 + 0.011 / x**3) + Rv - def fun1(x): return 2.659*(-2.156+1.509/x-0.198/x**2+0.011/x**3)+Rv - def fun2(x): return 2.659*(-1.857+1.040/x)+Rv + def fun2(x): + return 2.659 * (-1.857 + 1.040 / x) + Rv ff11, ff12 = fun1(0.11), fun1(0.12) - slope1 = (ff12-ff11)/0.01 + slope1 = (ff12 - ff11) / 0.01 ff99, ff100 = fun2(2.19), fun2(2.2) - slope2 = (ff100-ff99)/0.01 + slope2 = (ff100 - ff99) / 0.01 sw0 = sw[sw < 0.12] sw1 = sw[np.logical_and(sw >= 0.12, sw <= 0.63)] sw2 = sw[np.logical_and(sw > 0.63, sw <= 2.2)] sw3 = sw[sw > 2.2] - k_lambda0 = ff11+(sw0-0.11)*slope1 + k_lambda0 = ff11 + (sw0 - 0.11) * slope1 k_lambda1, k_lambda2 = fun1(sw1), fun2(sw2) - k_lambda3 = ff99+(sw3-2.19)*slope2 - A_lambda = av*np.hstack([k_lambda0, k_lambda1, - k_lambda2, k_lambda3])/Rv + k_lambda3 = ff99 + (sw3 - 2.19) * slope2 + A_lambda = av * np.hstack([k_lambda0, k_lambda1, k_lambda2, k_lambda3]) / Rv A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) elif model == 6: # Reddy et al (2015) for satr forming galaxies Rv = 2.505 - sw = sw*1.0e-04 + sw = sw * 1.0e-04 + + def fun1(x): + return -5.726 + 4.004 / x - 0.525 / x**2 + 0.029 / x**3 + Rv - def fun1(x): return -5.726+4.004/x-0.525/x**2+0.029/x**3+Rv - def fun2(x): return -2.672-0.010/x+1.532/x**2-0.412/x**3+Rv + def fun2(x): + return -2.672 - 0.010 / x + 1.532 / x**2 - 0.412 / x**3 + Rv ff11, ff12 = fun1(0.14), fun1(0.15) - slope1 = (ff12-ff11)/0.01 + slope1 = (ff12 - ff11) / 0.01 ff99, ff100 = fun2(2.84), fun2(2.85) - slope2 = (ff100-ff99)/0.01 + slope2 = (ff100 - ff99) / 0.01 sw0 = sw[sw < 0.15] sw1 = sw[np.logical_and(sw >= 0.15, sw < 0.60)] sw2 = sw[np.logical_and(sw >= 0.60, sw < 2.85)] sw3 = sw[sw >= 2.85] - k_lambda0 = ff11+(sw0-0.14)*slope1 + k_lambda0 = ff11 + (sw0 - 0.14) * slope1 k_lambda1, k_lambda2 = fun1(sw1), fun2(sw2) - k_lambda3 = ff99+(sw3-2.84)*slope2 - A_lambda = av*np.hstack([k_lambda0, k_lambda1, - k_lambda2, k_lambda3])/Rv + k_lambda3 = ff99 + (sw3 - 2.84) * slope2 + A_lambda = av * np.hstack([k_lambda0, k_lambda1, k_lambda2, k_lambda3]) / Rv A_lambda[A_lambda < 0.0] = 0.0 - flux = sf*10**(-0.4*A_lambda) + flux = sf * 10 ** (-0.4 * A_lambda) else: raise ValueError("!!! Please select a proper reddening model") return flux + ########################################### def __red(alan, al0, ga, c1, c2, c3, c4): - def fun1(x): return c3/(((x-(al0**2/x))**2)+ga*ga) - def fun2(x, cc): return cc*(0.539*((x-5.9)**2)+0.0564*((x-5.9)**3)) - def fun(x, cc): return c1+c2*x+fun1(x)+fun2(x, cc) + def fun1(x): + return c3 / (((x - (al0**2 / x)) ** 2) + ga * ga) + + def fun2(x, cc): + return cc * (0.539 * ((x - 5.9) ** 2) + 0.0564 * ((x - 5.9) ** 3)) + + def fun(x, cc): + return c1 + c2 * x + fun1(x) + fun2(x, cc) - ala = alan*1.0e-04 # wavelength in microns - p = 1.0/ala + ala = alan * 1.0e-04 # wavelength in microns + p = 1.0 / ala if np.size(p) > 1: p1, p2 = p[p >= 5.9], p[p < 5.9] ff = np.append(fun(p1, c4), fun(p2, 0.0)) @@ -389,6 +517,7 @@ def __red(alan, al0, ga, c1, c2, c3, c4): return return ff + ########################################### @@ -399,22 +528,25 @@ def sed2mag(mag_i, sedCat, filter_list, redshift=0.0, av=0.0, redden=0): aflux = np.zeros(nfilt) nid = -1 for k in range(nfilt): - if filter_list[k].filter_type == 'i': + if filter_list[k].filter_type == "i": nid = k bandpass = filter_list[k].bandpass_full ktrans = np.transpose( - np.array([bandpass.wave_list*10.0, bandpass.func(bandpass.wave_list)])) + np.array([bandpass.wave_list * 10.0, bandpass.func(bandpass.wave_list)]) + ) aflux[k], isedObs = tflux( - ktrans, sedCat, redshift=redshift, av=av, redden=redden) + ktrans, sedCat, redshift=redshift, av=av, redden=redden + ) # normalize to i-band aflux = aflux / aflux[nid] # magnitudes in all filters - amag = -2.5*np.log10(aflux) + mag_i - spec = galsim.LookupTable(x=np.array(isedObs[0]), f=np.array( - isedObs[1]), interpolant='nearest') - isedObs = galsim.SED(spec, wave_type='A', flux_type='1', fast=False) + amag = -2.5 * np.log10(aflux) + mag_i + spec = galsim.LookupTable( + x=np.array(isedObs[0]), f=np.array(isedObs[1]), interpolant="nearest" + ) + isedObs = galsim.SED(spec, wave_type="A", flux_type="1", fast=False) return amag, isedObs @@ -454,15 +586,15 @@ def eObs(e1, e2, g1, g2): e, gg = abs(e), abs(g) if gg <= 1.0: tt = e + g - bb = 1.0 + e*g.conjugate() - eobs = tt/bb + bb = 1.0 + e * g.conjugate() + eobs = tt / bb else: - tt = 1.0 + g*e.conjugate() + tt = 1.0 + g * e.conjugate() bb = e.conjugate() + g.conjugate() - eobs = tt/bb + eobs = tt / bb - # derive the orientation - dd = 0.5*np.arctan(abs(eobs.imag/eobs.real))*180.0/np.pi + # derive the orientation + dd = 0.5 * np.arctan(abs(eobs.imag / eobs.real)) * 180.0 / np.pi if eobs.imag > 0 and eobs.real > 0: dd = dd if eobs.imag > 0 and eobs.real < 0: @@ -477,8 +609,12 @@ def eObs(e1, e2, g1, g2): eeobs += [abs(eobs)] theta += [dd] - e1obs, e2obs, eeobs, theta = np.array(e1obs), np.array( - e2obs), np.array(eeobs), np.array(theta) + e1obs, e2obs, eeobs, theta = ( + np.array(e1obs), + np.array(e2obs), + np.array(eeobs), + np.array(theta), + ) if nobj == 1: e1obs, e2obs, eeobs, theta = e1obs[0], e2obs[0], eeobs[0], theta[0] @@ -491,7 +627,7 @@ def getObservedSED(sedCat, redshift=0.0, av=0.0, redden=0): # reddening sf = reddening(sw, sf, av=av, model=redden) # sw, sf = sw*z, sf*(z**3) - sw, sf = sw*z, sf/z + sw, sf = sw * z, sf / z # sw, sf = sw*z, sf # lyman forest correction @@ -502,30 +638,33 @@ def getObservedSED(sedCat, redshift=0.0, av=0.0, redden=0): def integrate_sed_bandpass(sed, bandpass): wave = np.linspace(bandpass.blue_limit, bandpass.red_limit, 1000) # in nm - flux_normalized = sed(wave)*bandpass(wave) + flux_normalized = sed(wave) * bandpass(wave) # print('in integrate_sed_bandpass', bandpass.blue_limit, bandpass.red_limit) - int_flux = np.trapz(y=flux_normalized, x=wave) * \ - 10. # convert to photons s-1 m-2 A-1 + int_flux = ( + np.trapz(y=flux_normalized, x=wave) * 10.0 + ) # convert to photons s-1 m-2 A-1 return int_flux def getABMAG(interFlux, bandpass): - throughtput = Table(np.array(np.array([bandpass.wave_list*10.0, bandpass.func( - bandpass.wave_list)])).T, names=(['WAVELENGTH', 'SENSITIVITY'])) - sWave = bandpass.blue_limit*10.0 - eWave = bandpass.red_limit*10.0 + throughtput = Table( + np.array( + np.array([bandpass.wave_list * 10.0, bandpass.func(bandpass.wave_list)]) + ).T, + names=(["WAVELENGTH", "SENSITIVITY"]), + ) + sWave = bandpass.blue_limit * 10.0 + eWave = bandpass.red_limit * 10.0 # print('in getABMAG', sWave, eWave) ABMAG_zero = getABMagAverageVal( - ABmag=0, - norm_thr=throughtput, - sWave=sWave, - eWave=eWave) - flux_ave = interFlux / (eWave-sWave) - ABMAG_spec = -2.5 * np.log10(flux_ave/ABMAG_zero) + ABmag=0, norm_thr=throughtput, sWave=sWave, eWave=eWave + ) + flux_ave = interFlux / (eWave - sWave) + ABMAG_spec = -2.5 * np.log10(flux_ave / ABMAG_zero) return ABMAG_spec -def getABMagAverageVal(ABmag=20., norm_thr=None, sWave=6840, eWave=8250): +def getABMagAverageVal(ABmag=20.0, norm_thr=None, sWave=6840, eWave=8250): """ norm_thr: astropy.table, 2 colum, 'WAVELENGTH', 'SENSITIVITY' @@ -533,12 +672,12 @@ def getABMagAverageVal(ABmag=20., norm_thr=None, sWave=6840, eWave=8250): the integerate flux of AB magnitude in the norm_thr(the throughtput of band), photos s-1 m-2 A-1 """ - inverseLambda = norm_thr['SENSITIVITY']/norm_thr['WAVELENGTH'] - norm_thr_i = interpolate.interp1d(norm_thr['WAVELENGTH'], inverseLambda) + inverseLambda = norm_thr["SENSITIVITY"] / norm_thr["WAVELENGTH"] + norm_thr_i = interpolate.interp1d(norm_thr["WAVELENGTH"], inverseLambda) - x = np.linspace(sWave, eWave, int(eWave)-int(sWave)+1) + x = np.linspace(sWave, eWave, int(eWave) - int(sWave) + 1) y = norm_thr_i(x) - AverageLamdaInverse = np.trapz(y, x)/(eWave-sWave) + AverageLamdaInverse = np.trapz(y, x) / (eWave - sWave) norm = 54798696332.52474 * pow(10.0, -0.4 * ABmag) * AverageLamdaInverse # print('AverageLamdaInverse = ', AverageLamdaInverse) # print('norm = ', norm) @@ -546,7 +685,9 @@ def getABMagAverageVal(ABmag=20., norm_thr=None, sWave=6840, eWave=8250): return norm -def getNormFactorForSpecWithABMAG(ABMag=20., spectrum=None, norm_thr=None, sWave=6840, eWave=8250): +def getNormFactorForSpecWithABMAG( + ABMag=20.0, spectrum=None, norm_thr=None, sWave=6840, eWave=8250 +): """ Use AB magnitude system (zero point, fv = 3631 janskys) in the normal band(norm_thr) normalize the spectrum by inpute ABMag @@ -560,20 +701,18 @@ def getNormFactorForSpecWithABMAG(ABMag=20., spectrum=None, norm_thr=None, sWave Return: the normalization factor flux of AB system(fix a band and magnitude ) /the flux of inpute spectrum(fix a band) """ - spectrumi = interpolate.interp1d(spectrum['WAVELENGTH'], spectrum['FLUX']) - norm_thri = interpolate.interp1d( - norm_thr['WAVELENGTH'], norm_thr['SENSITIVITY']) + spectrumi = interpolate.interp1d(spectrum["WAVELENGTH"], spectrum["FLUX"]) + norm_thri = interpolate.interp1d(norm_thr["WAVELENGTH"], norm_thr["SENSITIVITY"]) - x = np.linspace(sWave, eWave, int(eWave)-int(sWave)+1) + x = np.linspace(sWave, eWave, int(eWave) - int(sWave) + 1) y_spec = spectrumi(x) y_thr = norm_thri(x) - y = y_spec*y_thr + y = y_spec * y_thr - specAve = np.trapz(y, x)/(eWave-sWave) - norm = getABMagAverageVal( - ABmag=ABMag, norm_thr=norm_thr, sWave=sWave, eWave=eWave) + specAve = np.trapz(y, x) / (eWave - sWave) + norm = getABMagAverageVal(ABmag=ABMag, norm_thr=norm_thr, sWave=sWave, eWave=eWave) if specAve == 0: return 0 @@ -594,8 +733,10 @@ def tag_sed(h5file, model_tag, teff=5000, logg=2, feh=0): close_feh = 99 else: close_feh = feh_grid[np.argmin(np.abs(feh_grid - feh))] - path = model_tag_str + \ - f"_teff_{close_teff:.1f}_logg_{close_logg:.2f}_feh_{close_feh:.1f}" + path = ( + model_tag_str + + f"_teff_{close_teff:.1f}_logg_{close_logg:.2f}_feh_{close_feh:.1f}" + ) wave = np.array(h5file["wave"][model_tag_str][()]).ravel() flux = np.array(h5file["sed"][path][()]).ravel() return path, wave, flux @@ -604,25 +745,25 @@ def tag_sed(h5file, model_tag, teff=5000, logg=2, feh=0): def convolveGaussXorders(img=None, sigma=1): from astropy.modeling.models import Gaussian2D from scipy import signal - offset = int(np.ceil(sigma*10)) - g_size = 2*offset+1 - m_cen = int(g_size/2) + offset = int(np.ceil(sigma * 10)) + g_size = 2 * offset + 1 + + m_cen = int(g_size / 2) g_PSF_ = Gaussian2D(1, m_cen, m_cen, sigma, sigma) yp, xp = np.mgrid[0:g_size, 0:g_size] g_PSF = g_PSF_(xp, yp) - psf = g_PSF/g_PSF.sum() - convImg = signal.fftconvolve(img, psf, mode='full', axes=None) + psf = g_PSF / g_PSF.sum() + convImg = signal.fftconvolve(img, psf, mode="full", axes=None) return convImg, offset def convolveImg(img=None, psf=None): - from astropy.modeling.models import Gaussian2D from scipy import signal - convImg = signal.fftconvolve(img, psf, mode='full', axes=None) - offset_x = int(psf.shape[1]/2. + 0.5) - 1 - offset_y = int(psf.shape[0]/2. + 0.5) - 1 + convImg = signal.fftconvolve(img, psf, mode="full", axes=None) + offset_x = int(psf.shape[1] / 2.0 + 0.5) - 1 + offset_y = int(psf.shape[0] / 2.0 + 0.5) - 1 offset = [offset_x, offset_y] return convImg, offset diff --git a/observation_sim/psf/FieldDistortion.py b/observation_sim/psf/FieldDistortion.py index ff5faeab84a8237afca3471e400e13c9d8e2446c..f23fe913d3f4c3248abd57ca31684dcb0fcda70e 100644 --- a/observation_sim/psf/FieldDistortion.py +++ b/observation_sim/psf/FieldDistortion.py @@ -4,18 +4,17 @@ import cmath class FieldDistortion(object): - - def __init__(self, chip, fdModel=None, fdModel_path=None, img_rot=0.): + def __init__(self, chip, fdModel=None, fdModel_path=None, img_rot=0.0): if fdModel is None: - if hasattr(chip, 'fdModel'): + if hasattr(chip, "fdModel"): self.fdModel = chip.fdModel elif fdModel_path is not None: import pickle + with open(fdModel_path, "rb") as f: self.fdModel = pickle.load(f) else: - raise ValueError( - "Error: no field distortion model has been specified!") + raise ValueError("Error: no field distortion model has been specified!") else: self.fdModel = fdModel self.img_rot = img_rot @@ -28,8 +27,9 @@ class FieldDistortion(object): self.ify_dx = self.iyfdModel.partial_derivative(1, 0) self.ify_dy = self.iyfdModel.partial_derivative(0, 1) if "residual" in self.fdModel["wave1"]: - self.irsModel = self.fdModel["wave1"]["residual"]["ccd" + - chip.getChipLabel(chipID=chip.chipID)] + self.irsModel = self.fdModel["wave1"]["residual"][ + "ccd" + chip.getChipLabel(chipID=chip.chipID) + ] self.ixrsModel = self.irsModel["xResidual"] self.iyrsModel = self.irsModel["yResidual"] # first-order derivatives of the residual field distortion model @@ -48,7 +48,7 @@ class FieldDistortion(object): return True def get_distorted(self, chip, pos_img, bandpass=None, img_rot=None): - """ Get the distored position for an undistorted image position + """Get the distored position for an undistorted image position Parameters: chip: A 'Chip' object representing the @@ -106,8 +106,8 @@ class FieldDistortion(object): g_abs = np.sqrt(g1k_fd**2 + g2k_fd**2) phi = cmath.phase(complex(g1k_fd, g2k_fd)) # g_abs = 0.7 - g1k_fd = g_abs * np.cos(phi + 2*img_rot) - g2k_fd = g_abs * np.sin(phi + 2*img_rot) + g1k_fd = g_abs * np.cos(phi + 2 * img_rot) + g2k_fd = g_abs * np.sin(phi + 2 * img_rot) fd_shear = galsim.Shear(g1=g1k_fd, g2=g2k_fd) return galsim.PositionD(x, y), fd_shear diff --git a/observation_sim/psf/PSFGauss.py b/observation_sim/psf/PSFGauss.py index db8b3eac48d53645f75dade4734181115be40279..9cc4e15576e78351f27ea7ae6650c8601c60a2b9 100644 --- a/observation_sim/psf/PSFGauss.py +++ b/observation_sim/psf/PSFGauss.py @@ -1,13 +1,12 @@ import galsim import sep import numpy as np -from scipy.interpolate import interp1d from observation_sim.psf.PSFModel import PSFModel class PSFGauss(PSFModel): - def __init__(self, chip, fwhm=0.187, sigSpin=0., psfRa=None): + def __init__(self, chip, fwhm=0.187, sigSpin=0.0, psfRa=None): self.pix_size = chip.pix_scale self.chip = chip if psfRa is None: @@ -30,12 +29,18 @@ class PSFGauss(PSFModel): Return: the value of the pseudo CDF """ - def gaussFun(sigma, r): return 1.0/(np.sqrt(2.0*np.pi) - * sigma) * np.exp(-r**2/(2.0*sigma**2)) + + def gaussFun(sigma, r): + return ( + 1.0 + / (np.sqrt(2.0 * np.pi) * sigma) + * np.exp(-(r**2) / (2.0 * sigma**2)) + ) + nxx = 1000 rArr = np.linspace(0.0, r, nxx) gauss = gaussFun(sig, rArr) - erf = 2.0*np.trapz(gauss, rArr) + erf = 2.0 * np.trapz(gauss, rArr) return erf def fracGauss(self, sig, r=0.15, pscale=None): @@ -53,12 +58,13 @@ class PSFGauss(PSFModel): if pscale is None: pscale = self.pix_size gaussx = galsim.Gaussian(flux=1.0, sigma=sig) - gaussImg = gaussx.drawImage(scale=pscale, method='no_pixel') + gaussImg = gaussx.drawImage(scale=pscale, method="no_pixel") gaussImg = gaussImg.array size = np.size(gaussImg, axis=0) - cxy = 0.5*(size-1) + cxy = 0.5 * (size - 1) flux, ferr, flag = sep.sum_circle( - gaussImg, [cxy], [cxy], [r/pscale], subpix=0) + gaussImg, [cxy], [cxy], [r / pscale], subpix=0 + ) return flux def fwhmGauss(self, r=0.15, fr=0.8, pscale=None): @@ -72,24 +78,26 @@ class PSFGauss(PSFModel): pscale = self.pix_size err = 1.0e-3 nxx = 100 - sig = np.linspace(0.5*pscale, 1.0, nxx) + sig = np.linspace(0.5 * pscale, 1.0, nxx) frA = np.zeros(nxx) for i in range(nxx): frA[i] = self.fracGauss(sig[i], r=r, pscale=pscale) - index = [i for i in range(nxx-1) if (fr-frA[i]) - * (fr-frA[i+1]) <= 0.0][0] + index = [i for i in range(nxx - 1) if (fr - frA[i]) * (fr - frA[i + 1]) <= 0.0][ + 0 + ] - while abs(frA[index]-fr) > 1.0e-3: - sig = np.linspace(sig[index], sig[index+1], nxx) + while abs(frA[index] - fr) > 1.0e-3: + sig = np.linspace(sig[index], sig[index + 1], nxx) for i in range(nxx): frA[i] = self.fracGauss(sig[i], r=r, pscale=pscale) - index = [i for i in range( - nxx-1) if (fr-frA[i])*(fr-frA[i+1]) <= 0.0][0] + index = [ + i for i in range(nxx - 1) if (fr - frA[i]) * (fr - frA[i + 1]) <= 0.0 + ][0] - fwhm = 2.35482*sig[index] + fwhm = 2.35482 * sig[index] return fwhm - def get_PSF(self, pos_img, chip=None, bandpass=None, folding_threshold=5.e-3): + def get_PSF(self, pos_img, chip=None, bandpass=None, folding_threshold=5.0e-3): dx = pos_img.x - self.chip.cen_pix_x dy = pos_img.y - self.chip.cen_pix_y return self.PSFspin(dx, dy) @@ -105,21 +113,21 @@ class PSFGauss(PSFModel): Return: Spinned PSF: g1, g2 and axis ratio 'a/b' """ - a2Rad = np.pi/(60.0*60.0*180.0) + a2Rad = np.pi / (60.0 * 60.0 * 180.0) - ff = self.sigGauss * 0.107 * (1000.0/10.0) # in unit of [pixels] - rc = np.sqrt(x*x + y*y) - cpix = rc*(self.sigSpin*a2Rad) + ff = self.sigGauss * 0.107 * (1000.0 / 10.0) # in unit of [pixels] + rc = np.sqrt(x * x + y * y) + cpix = rc * (self.sigSpin * a2Rad) - beta = (np.arctan2(y, x) + np.pi/2) - ell = cpix**2/(2.0*ff**2+cpix**2) + beta = np.arctan2(y, x) + np.pi / 2 + ell = cpix**2 / (2.0 * ff**2 + cpix**2) # ell *= 10.0 - qr = np.sqrt((1.0+ell)/(1.0-ell)) + qr = np.sqrt((1.0 + ell) / (1.0 - ell)) # psfShape = galsim.Shear(e=ell, beta=beta) # g1, g2 = psfShape.g1, psfShape.g2 # qr = np.sqrt((1.0+ell)/(1.0-ell)) # return ell, beta, qr - PSFshear = galsim.Shear(e=ell, beta=beta*galsim.radians) + PSFshear = galsim.Shear(e=ell, beta=beta * galsim.radians) return self.psf.shear(PSFshear), PSFshear diff --git a/observation_sim/psf/PSFInterp.py b/observation_sim/psf/PSFInterp.py index d21e75fcc81d030f41022da26a40e8e460ddc809..2917b2af458aa6bbbcab07bea076b3193ece8217 100644 --- a/observation_sim/psf/PSFInterp.py +++ b/observation_sim/psf/PSFInterp.py @@ -1,12 +1,11 @@ -''' +""" PSF interpolation for CSST-Sim NOTE: [iccd, iwave, ipsf] are counted from 1 to n, but [tccd, twave, tpsf] are counted from 0 to n-1 -''' +""" import sys import time -import copy import numpy as np import scipy.spatial as spatial import galsim @@ -17,7 +16,7 @@ from observation_sim.psf._util import psf_extrapolate NPSF = 900 # ***# 30*30 -PixSizeInMicrons = 5. # ***# in microns +PixSizeInMicrons = 5.0 # ***# in microns # find neighbors-KDtree # @@ -47,36 +46,37 @@ def findNeighbors(tx, ty, px, py, dr=0.1, dn=1, OnlyDistance=True): while len(dataq) < dn: dataq = tree.query_ball_point([tx, ty], rr) rr += dr - dd = np.hypot(datax[dataq]-tx, datay[dataq]-ty) + dd = np.hypot(datax[dataq] - tx, datay[dataq] - ty) ddSortindx = np.argsort(dd) dataq = np.array(dataq)[ddSortindx[0:dn]] return dataq + # find neighbors-hoclist# def hocBuild(partx, party, nhocx, nhocy, dhocx, dhocy): - if np.max(partx) > nhocx*dhocx: - print('ERROR') + if np.max(partx) > nhocx * dhocx: + print("ERROR") sys.exit() - if np.max(party) > nhocy*dhocy: - print('ERROR') + if np.max(party) > nhocy * dhocy: + print("ERROR") sys.exit() npart = partx.size - hoclist = np.zeros(npart, dtype=np.int32)-1 - hoc = np.zeros([nhocy, nhocx], dtype=np.int32)-1 + hoclist = np.zeros(npart, dtype=np.int32) - 1 + hoc = np.zeros([nhocy, nhocx], dtype=np.int32) - 1 for ipart in range(npart): - ix = int(partx[ipart]/dhocx) - iy = int(party[ipart]/dhocy) + ix = int(partx[ipart] / dhocx) + iy = int(party[ipart] / dhocy) hoclist[ipart] = hoc[iy, ix] hoc[iy, ix] = ipart return hoc, hoclist def hocFind(px, py, dhocx, dhocy, hoc, hoclist): - ix = int(px/dhocx) - iy = int(py/dhocy) + ix = int(px / dhocx) + iy = int(py / dhocy) neigh = [] it = hoc[iy, ix] @@ -94,20 +94,20 @@ def findNeighbors_hoclist(px, py, tx=None, ty=None, dn=4, hoc=None, hoclist=None pyMin = np.min(py) pyMax = np.max(py) - dhocx = (pxMax - pxMin)/(nhocx-1) - dhocy = (pyMax - pyMin)/(nhocy-1) - partx = px - pxMin + dhocx/2 - party = py - pyMin + dhocy/2 + dhocx = (pxMax - pxMin) / (nhocx - 1) + dhocy = (pyMax - pyMin) / (nhocy - 1) + partx = px - pxMin + dhocx / 2 + party = py - pyMin + dhocy / 2 if hoc is None: hoc, hoclist = hocBuild(partx, party, nhocx, nhocy, dhocx, dhocy) return hoc, hoclist if hoc is not None: - tx = tx - pxMin + dhocx/2 - ty = ty - pyMin + dhocy/2 - itx = int(tx/dhocx) - ity = int(ty/dhocy) + tx = tx - pxMin + dhocx / 2 + ty = ty - pyMin + dhocy / 2 + itx = int(tx / dhocx) + ity = int(ty / dhocy) ps = [-1, 0, 1] neigh = [] @@ -119,9 +119,9 @@ def findNeighbors_hoclist(px, py, tx=None, ty=None, dn=4, hoc=None, hoclist=None continue if iy < 0: continue - if ix > nhocx-1: + if ix > nhocx - 1: continue - if iy > nhocy-1: + if iy > nhocy - 1: continue # neightt = myUtil.hocFind(ppx, ppy, dhocx, dhocy, hoc, hoclist) @@ -134,14 +134,25 @@ def findNeighbors_hoclist(px, py, tx=None, ty=None, dn=4, hoc=None, hoclist=None if dn != -1: ptx = np.array(partx[neigh]) pty = np.array(party[neigh]) - dd = np.hypot(ptx-tx, pty-ty) + dd = np.hypot(ptx - tx, pty - ty) idx = np.argsort(dd) neigh = np.array(neigh)[idx[0:dn]] return neigh # PSF-IDW# -def psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=True, hoc=None, hoclist=None, PSFCentroidWgt=False): +def psfMaker_IDW( + px, + py, + PSFMat, + cen_col, + cen_row, + IDWindex=2, + OnlyNeighbors=True, + hoc=None, + hoclist=None, + PSFCentroidWgt=False, +): """ psf interpolation by IDW @@ -166,11 +177,13 @@ def psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=Tru if OnlyNeighbors is True: if hoc is None: - neigh = findNeighbors(px, py, cen_col, cen_row, - dr=5., dn=4, OnlyDistance=False) + neigh = findNeighbors( + px, py, cen_col, cen_row, dr=5.0, dn=4, OnlyDistance=False + ) if hoc is not None: neigh = findNeighbors_hoclist( - cen_col, cen_row, tx=px, ty=py, dn=4, hoc=hoc, hoclist=hoclist) + cen_col, cen_row, tx=px, ty=py, dn=4, hoc=hoc, hoclist=hoclist + ) neighFlag = np.zeros(npsf) neighFlag[neigh] = 1 @@ -180,8 +193,7 @@ def psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=Tru if neighFlag[ipsf] != 1: continue - dist = np.sqrt((ref_col - cen_col[ipsf]) - ** 2 + (ref_row - cen_row[ipsf])**2) + dist = np.sqrt((ref_col - cen_col[ipsf]) ** 2 + (ref_row - cen_row[ipsf]) ** 2) if IDWindex == 1: psfWeight[ipsf] = dist if IDWindex == 2: @@ -191,7 +203,7 @@ def psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=Tru if IDWindex == 4: psfWeight[ipsf] = dist**4 psfWeight[ipsf] = max(psfWeight[ipsf], minimum_psf_weight) - psfWeight[ipsf] = 1./psfWeight[ipsf] + psfWeight[ipsf] = 1.0 / psfWeight[ipsf] psfWeight /= np.sum(psfWeight) psfMaker = np.zeros([ngy, ngx], dtype=np.float32) @@ -211,13 +223,27 @@ def psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=Tru # define PSFInterp# class PSFInterp(PSFModel): - def __init__(self, chip, npsf=NPSF, PSF_data=None, PSF_data_file=None, PSF_data_prefix="", sigSpin=0, psfRa=0.15, HocBuild=False, LOG_DEBUG=False): + def __init__( + self, + chip, + npsf=NPSF, + PSF_data=None, + PSF_data_file=None, + PSF_data_prefix="", + sigSpin=0, + psfRa=0.15, + HocBuild=False, + LOG_DEBUG=False, + ): self.LOG_DEBUG = LOG_DEBUG if self.LOG_DEBUG: - print('===================================================') - print('DEBUG: psf module for csstSim ' - + time.strftime("(%Y-%m-%d %H:%M:%S)", time.localtime()), flush=True) - print('===================================================') + print("===================================================") + print( + "DEBUG: psf module for csstSim " + + time.strftime("(%Y-%m-%d %H:%M:%S)", time.localtime()), + flush=True, + ) + print("===================================================") self.sigSpin = sigSpin self.sigGauss = psfRa @@ -225,25 +251,24 @@ class PSFInterp(PSFModel): self.iccd = int(chip.getChipLabel(chipID=chip.chipID)) # self.iccd = chip.chip_name if PSF_data_file is None: - print('Error - PSF_data_file is None') + print("Error - PSF_data_file is None") sys.exit() - self.nwave = self._getPSFwave( - self.iccd, PSF_data_file, PSF_data_prefix) + self.nwave = self._getPSFwave(self.iccd, PSF_data_file, PSF_data_prefix) self.npsf = npsf - self.PSF_data = self._loadPSF( - self.iccd, PSF_data_file, PSF_data_prefix) + self.PSF_data = self._loadPSF(self.iccd, PSF_data_file, PSF_data_prefix) if self.LOG_DEBUG: - print('nwave-{:} on ccd-{:}::'.format(self.nwave, - self.iccd), flush=True) - print('self.PSF_data ... ok', flush=True) + print("nwave-{:} on ccd-{:}::".format(self.nwave, self.iccd), flush=True) + print("self.PSF_data ... ok", flush=True) print( - 'Preparing self.[psfMat,cen_col,cen_row] for psfMaker ... ', end='', flush=True) + "Preparing self.[psfMat,cen_col,cen_row] for psfMaker ... ", + end="", + flush=True, + ) - ngy, ngx = self.PSF_data[0][0]['psfMat'].shape - self.psfMat = np.zeros( - [self.nwave, self.npsf, ngy, ngx], dtype=np.float32) + ngy, ngx = self.PSF_data[0][0]["psfMat"].shape + self.psfMat = np.zeros([self.nwave, self.npsf, ngy, ngx], dtype=np.float32) self.cen_col = np.zeros([self.nwave, self.npsf], dtype=np.float32) self.cen_row = np.zeros([self.nwave, self.npsf], dtype=np.float32) self.hoc = [] @@ -251,30 +276,35 @@ class PSFInterp(PSFModel): for twave in range(self.nwave): for tpsf in range(self.npsf): - self.psfMat[twave, tpsf, :, - :] = self.PSF_data[twave][tpsf]['psfMat'] - self.PSF_data[twave][tpsf]['psfMat'] = 0 # free psfMat - - self.pixsize = self.PSF_data[twave][tpsf]['pixsize']*1e-3 # mm - self.cen_col[twave, tpsf] = self.PSF_data[twave][tpsf]['image_x'] + \ - self.PSF_data[twave][tpsf]['centroid_x'] - self.cen_row[twave, tpsf] = self.PSF_data[twave][tpsf]['image_y'] + \ - self.PSF_data[twave][tpsf]['centroid_y'] + self.psfMat[twave, tpsf, :, :] = self.PSF_data[twave][tpsf]["psfMat"] + self.PSF_data[twave][tpsf]["psfMat"] = 0 # free psfMat + + self.pixsize = self.PSF_data[twave][tpsf]["pixsize"] * 1e-3 # mm + self.cen_col[twave, tpsf] = ( + self.PSF_data[twave][tpsf]["image_x"] + + self.PSF_data[twave][tpsf]["centroid_x"] + ) + self.cen_row[twave, tpsf] = ( + self.PSF_data[twave][tpsf]["image_y"] + + self.PSF_data[twave][tpsf]["centroid_y"] + ) if HocBuild: # hoclist on twave for neighborsFinding hoc, hoclist = findNeighbors_hoclist( - self.cen_col[twave], self.cen_row[twave]) + self.cen_col[twave], self.cen_row[twave] + ) self.hoc.append(hoc) self.hoclist.append(hoclist) if self.LOG_DEBUG: - print('ok', flush=True) + print("ok", flush=True) def _getPSFwave(self, iccd, PSF_data_file, PSF_data_prefix): # fq = h5py.File(PSF_data_file+'/' +PSF_data_prefix +'psfCube_ccd{:}.h5'.format(iccd), 'r') - fq = h5py.File(PSF_data_file+'/' + PSF_data_prefix + - 'psfCube_{:}.h5'.format(iccd), 'r') + fq = h5py.File( + PSF_data_file + "/" + PSF_data_prefix + "psfCube_{:}.h5".format(iccd), "r" + ) nwave = len(fq.keys()) fq.close() return nwave @@ -282,36 +312,36 @@ class PSFInterp(PSFModel): def _loadPSF(self, iccd, PSF_data_file, PSF_data_prefix): psfSet = [] # fq = h5py.File(PSF_data_file+'/' +PSF_data_prefix +'psfCube_ccd{:}.h5'.format(iccd), 'r') - fq = h5py.File(PSF_data_file+'/' + PSF_data_prefix + - 'psfCube_{:}.h5'.format(iccd), 'r') + fq = h5py.File( + PSF_data_file + "/" + PSF_data_prefix + "psfCube_{:}.h5".format(iccd), "r" + ) for ii in range(self.nwave): - iwave = ii+1 + iwave = ii + 1 psfWave = [] - fq_iwave = fq['w_{:}'.format(iwave)] + fq_iwave = fq["w_{:}".format(iwave)] for jj in range(self.npsf): - ipsf = jj+1 + ipsf = jj + 1 psfInfo = {} - psfInfo['wavelength'] = fq_iwave['wavelength'][()] - - fq_iwave_ipsf = fq_iwave['psf_{:}'.format(ipsf)] - psfInfo['pixsize'] = PixSizeInMicrons - psfInfo['field_x'] = fq_iwave_ipsf['field_x'][()] - psfInfo['field_y'] = fq_iwave_ipsf['field_y'][()] - psfInfo['image_x'] = fq_iwave_ipsf['image_x'][()] - psfInfo['image_y'] = fq_iwave_ipsf['image_y'][()] - psfInfo['centroid_x'] = fq_iwave_ipsf['cx'][()] - psfInfo['centroid_y'] = fq_iwave_ipsf['cy'][()] - psfInfo['psfMat'] = fq_iwave_ipsf['psfMat'][()] + psfInfo["wavelength"] = fq_iwave["wavelength"][()] + + fq_iwave_ipsf = fq_iwave["psf_{:}".format(ipsf)] + psfInfo["pixsize"] = PixSizeInMicrons + psfInfo["field_x"] = fq_iwave_ipsf["field_x"][()] + psfInfo["field_y"] = fq_iwave_ipsf["field_y"][()] + psfInfo["image_x"] = fq_iwave_ipsf["image_x"][()] + psfInfo["image_y"] = fq_iwave_ipsf["image_y"][()] + psfInfo["centroid_x"] = fq_iwave_ipsf["cx"][()] + psfInfo["centroid_y"] = fq_iwave_ipsf["cy"][()] + psfInfo["psfMat"] = fq_iwave_ipsf["psfMat"][()] psfWave.append(psfInfo) psfSet.append(psfWave) fq.close() if self.LOG_DEBUG: - print('psfSet has been loaded:', flush=True) - print('psfSet[iwave][ipsf][keys]:', - psfSet[0][0].keys(), flush=True) + print("psfSet has been loaded:", flush=True) + print("psfSet[iwave][ipsf][keys]:", psfSet[0][0].keys(), flush=True) return psfSet def _findWave(self, bandpass): @@ -320,12 +350,23 @@ class PSFInterp(PSFModel): return twave for twave in range(self.nwave): - bandwave = self.PSF_data[twave][0]['wavelength'] + bandwave = self.PSF_data[twave][0]["wavelength"] if bandpass.blue_limit < bandwave and bandwave < bandpass.red_limit: return twave return -1 - def get_PSF(self, chip, pos_img, bandpass, galsimGSObject=True, findNeighMode='treeFind', folding_threshold=5.e-3, pointing_pa=0.0, extrapolate=False, ngg=2048): + def get_PSF( + self, + chip, + pos_img, + bandpass, + galsimGSObject=True, + findNeighMode="treeFind", + folding_threshold=5.0e-3, + pointing_pa=0.0, + extrapolate=False, + ngg=2048, + ): """ Get the PSF at a given image position @@ -338,7 +379,7 @@ class PSFInterp(PSFModel): Returns: PSF: A 'galsim.GSObject'. """ - pixSize = np.rad2deg(self.pixsize*1e-3/28)*3600 # set psf pixsize + pixSize = np.rad2deg(self.pixsize * 1e-3 / 28) * 3600 # set psf pixsize # assert self.iccd == int(chip.getChipLabel(chipID=chip.chipID)), 'ERROR: self.iccd != chip.chipID' twave = self._findWave(bandpass) @@ -349,24 +390,82 @@ class PSFInterp(PSFModel): cen_col = self.cen_col[twave] cen_row = self.cen_row[twave] - px = (pos_img.x - chip.cen_pix_x)*0.01 - py = (pos_img.y - chip.cen_pix_y)*0.01 - if findNeighMode == 'treeFind': - imPSF = psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, - IDWindex=2, OnlyNeighbors=True, PSFCentroidWgt=True) - if findNeighMode == 'hoclistFind': - assert (self.hoc != 0), 'hoclist should be built correctly!' - imPSF = psfMaker_IDW(px, py, PSFMat, cen_col, cen_row, IDWindex=2, OnlyNeighbors=True, - hoc=self.hoc[twave], hoclist=self.hoclist[twave], PSFCentroidWgt=True) + px = (pos_img.x - chip.cen_pix_x) * 0.01 + py = (pos_img.y - chip.cen_pix_y) * 0.01 + if findNeighMode == "treeFind": + imPSF = psfMaker_IDW( + px, + py, + PSFMat, + cen_col, + cen_row, + IDWindex=2, + OnlyNeighbors=True, + PSFCentroidWgt=True, + ) + if findNeighMode == "hoclistFind": + assert self.hoc != 0, "hoclist should be built correctly!" + imPSF = psfMaker_IDW( + px, + py, + PSFMat, + cen_col, + cen_row, + IDWindex=2, + OnlyNeighbors=True, + hoc=self.hoc[twave], + hoclist=self.hoclist[twave], + PSFCentroidWgt=True, + ) if extrapolate is True: - ccdList = [6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25] - rr_trim_list = [72, 64, 96, 88, 64, 72, 72, 76, 72, 72, 76, 72, 72, 64, 88, 96, 64, 72] - imPSF = psf_extrapolate(imPSF, rr_trim=rr_trim_list[ccdList.index(chip.chipID)], ngg=ngg) + ccdList = [ + 6, + 7, + 8, + 9, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 22, + 23, + 24, + 25, + ] + rr_trim_list = [ + 72, + 64, + 96, + 88, + 64, + 72, + 72, + 76, + 72, + 72, + 76, + 72, + 72, + 64, + 88, + 96, + 64, + 72, + ] + imPSF = psf_extrapolate( + imPSF, rr_trim=rr_trim_list[ccdList.index(chip.chipID)], ngg=ngg + ) if galsimGSObject: if extrapolate is True: - imPSFt = np.zeros([ngg+1, ngg+1]) + imPSFt = np.zeros([ngg + 1, ngg + 1]) imPSFt[:-1, :-1] = imPSF else: imPSFt = np.zeros([257, 257]) @@ -381,11 +480,10 @@ class PSFInterp(PSFModel): self.psf = galsim.InterpolatedImage(img, gsparams=gsp) wcs = chip.img.wcs.local(pos_img) scale = galsim.PixelScale(0.074) - self.psf = wcs.toWorld(scale.toImage( - self.psf), image_pos=(pos_img)) + self.psf = wcs.toWorld(scale.toImage(self.psf), image_pos=(pos_img)) # return self.PSFspin(x=px/0.01, y=py/0.01) - return self.psf, galsim.Shear(e=0., beta=(np.pi/2)*galsim.radians) + return self.psf, galsim.Shear(e=0.0, beta=(np.pi / 2) * galsim.radians) return imPSF ''' diff --git a/observation_sim/psf/PSFInterpSLS.py b/observation_sim/psf/PSFInterpSLS.py index b62cfedaec63fd6220f1cb097a9e8c30ecdd4b8c..5632f256c97600dbc3d7faedb3684deba8252987 100644 --- a/observation_sim/psf/PSFInterpSLS.py +++ b/observation_sim/psf/PSFInterpSLS.py @@ -1,17 +1,13 @@ -''' +""" PSF interpolation for CSST-Sim NOTE: [iccd, iwave, ipsf] are counted from 1 to n, but [tccd, twave, tpsf] are counted from 0 to n-1 -''' +""" import yaml -import sys import time -import copy import numpy as np -import scipy.spatial as spatial import galsim -import h5py from observation_sim.instruments import Filter, FilterParam, Chip from observation_sim.psf.PSFModel import PSFModel @@ -23,14 +19,13 @@ from astropy.modeling.models import Gaussian2D from scipy import signal, interpolate import datetime import gc -from astropy.io import fits -from observation_sim.psf._util import psf_extrapolate, psf_extrapolate1 +from observation_sim.psf._util import psf_extrapolate1 # from jax import numpy as jnp LOG_DEBUG = False # ***# NPSF = 900 # ***# 30*30 -PIX_SIZE_MICRON = 5. # ***# in microns +PIX_SIZE_MICRON = 5.0 # ***# in microns # find neighbors-KDtree @@ -217,12 +212,17 @@ PIX_SIZE_MICRON = 5. # ***# in microns # define PSFInterp class PSFInterpSLS(PSFModel): - def __init__(self, chip, filt, PSF_data_prefix="", sigSpin=0, psfRa=0.15, pix_size=0.005): + def __init__( + self, chip, filt, PSF_data_prefix="", sigSpin=0, psfRa=0.15, pix_size=0.005 + ): if LOG_DEBUG: - print('===================================================') - print('DEBUG: psf module for csstSim ' - + time.strftime("(%Y-%m-%d %H:%M:%S)", time.localtime()), flush=True) - print('===================================================') + print("===================================================") + print( + "DEBUG: psf module for csstSim " + + time.strftime("(%Y-%m-%d %H:%M:%S)", time.localtime()), + flush=True, + ) + print("===================================================") self.sigSpin = sigSpin self.sigGauss = psfRa @@ -233,13 +233,13 @@ class PSFInterpSLS(PSFModel): self.pixsize = pix_size # um def getPSFDataFromFile(self, filt): - gratingInwavelist = {'GU': 0, 'GV': 1, 'GI': 2} - grating_orders = ['0', '1'] - waveListFn = self.data_folder + '/wavelist.dat' + gratingInwavelist = {"GU": 0, "GV": 1, "GI": 2} + grating_orders = ["0", "1"] + waveListFn = self.data_folder + "/wavelist.dat" wavelists = np.loadtxt(waveListFn) self.waveList = wavelists[:, gratingInwavelist[self.grating_type]] bandranges = np.zeros([4, 2]) - midBand = (self.waveList[0:3] + self.waveList[1:4])/2.*10000. + midBand = (self.waveList[0:3] + self.waveList[1:4]) / 2.0 * 10000.0 bandranges[0, 0] = filt.blue_limit bandranges[1:4, 0] = midBand bandranges[0:3, 1] = midBand @@ -248,42 +248,42 @@ class PSFInterpSLS(PSFModel): self.bandranges = bandranges self.grating1_data = {} - g_folder = self.data_folder + '/' + self.grating_ids[0] + '/' + g_folder = self.data_folder + "/" + self.grating_ids[0] + "/" for g_order in grating_orders: - g_folder_order = g_folder + 'PSF_Order_' + g_order + '/' + g_folder_order = g_folder + "PSF_Order_" + g_order + "/" grating_order_data = {} for bandi in [1, 2, 3, 4]: subBand_data = {} - subBand_data['bandrange'] = bandranges[bandi-1] - final_folder = g_folder_order + str(bandi) + '/' + subBand_data["bandrange"] = bandranges[bandi - 1] + final_folder = g_folder_order + str(bandi) + "/" print(final_folder) pca_fs = os.listdir(final_folder) for fname in pca_fs: - if ('_PCs.fits' in fname) and (fname[0] != '.'): + if ("_PCs.fits" in fname) and (fname[0] != "."): fname_ = final_folder + fname hdu = fits.open(fname_) - subBand_data['band_data'] = hdu - grating_order_data['band'+str(bandi)] = subBand_data - self.grating1_data['order'+g_order] = grating_order_data + subBand_data["band_data"] = hdu + grating_order_data["band" + str(bandi)] = subBand_data + self.grating1_data["order" + g_order] = grating_order_data self.grating2_data = {} - g_folder = self.data_folder + '/' + self.grating_ids[1] + '/' + g_folder = self.data_folder + "/" + self.grating_ids[1] + "/" for g_order in grating_orders: - g_folder_order = g_folder + 'PSF_Order_' + g_order + '/' + g_folder_order = g_folder + "PSF_Order_" + g_order + "/" grating_order_data = {} for bandi in [1, 2, 3, 4]: subBand_data = {} - subBand_data['bandrange'] = bandranges[bandi - 1] - final_folder = g_folder_order + str(bandi) + '/' + subBand_data["bandrange"] = bandranges[bandi - 1] + final_folder = g_folder_order + str(bandi) + "/" print(final_folder) pca_fs = os.listdir(final_folder) for fname in pca_fs: - if ('_PCs.fits' in fname) and (fname[0] != '.'): + if ("_PCs.fits" in fname) and (fname[0] != "."): fname_ = final_folder + fname hdu = fits.open(fname_) - subBand_data['band_data'] = hdu - grating_order_data['band' + str(bandi)] = subBand_data - self.grating2_data['order' + g_order] = grating_order_data + subBand_data["band_data"] = hdu + grating_order_data["band" + str(bandi)] = subBand_data + self.grating2_data["order" + g_order] = grating_order_data # # @@ -348,16 +348,27 @@ class PSFInterpSLS(PSFModel): offset = int(np.ceil(sigma * 3)) g_size = 2 * offset + 1 m_cen = int(g_size / 2) - print('-----', g_size) + print("-----", g_size) g_PSF_ = Gaussian2D(1, m_cen, m_cen, sigma, sigma) yp, xp = np.mgrid[0:g_size, 0:g_size] g_PSF = g_PSF_(xp, yp) psf = g_PSF / g_PSF.sum() - convImg = signal.fftconvolve(img, psf, mode='full', axes=None) - convImg = convImg/np.sum(convImg) + convImg = signal.fftconvolve(img, psf, mode="full", axes=None) + convImg = convImg / np.sum(convImg) return convImg - def get_PSF(self, chip, pos_img_local=[1000, 1000], bandNo=1, galsimGSObject=True, folding_threshold=5.e-3, g_order='A', grating_split_pos=3685, extrapolate=False, ngg=2048): + def get_PSF( + self, + chip, + pos_img_local=[1000, 1000], + bandNo=1, + galsimGSObject=True, + folding_threshold=5.0e-3, + g_order="A", + grating_split_pos=3685, + extrapolate=False, + ngg=2048, + ): """ Get the PSF at a given image position @@ -370,11 +381,14 @@ class PSFInterpSLS(PSFModel): Returns: PSF: A 'galsim.GSObject'. """ - order_IDs = {'A': '1', 'B': '0', 'C': '0', 'D': '0', 'E': '0'} - contam_order_sigma = {'C': 0.28032344707964174, - 'D': 0.39900182912061344, 'E': 1.1988309797685412} # arcsec - x_start = chip.x_cen/chip.pix_size - chip.npix_x / 2. - y_start = chip.y_cen/chip.pix_size - chip.npix_y / 2. + order_IDs = {"A": "1", "B": "0", "C": "0", "D": "0", "E": "0"} + contam_order_sigma = { + "C": 0.28032344707964174, + "D": 0.39900182912061344, + "E": 1.1988309797685412, + } # arcsec + x_start = chip.x_cen / chip.pix_size - chip.npix_x / 2.0 + y_start = chip.y_cen / chip.pix_size - chip.npix_y / 2.0 # print(pos_img.x - x_start) pos_img_x = pos_img_local[0] + x_start pos_img_y = pos_img_local[1] + y_start @@ -389,20 +403,21 @@ class PSFInterpSLS(PSFModel): # grating_order = '1' # if grating_order in ['0', '1']: - psf_order = psf_data['order'+grating_order] - psf_order_b = psf_order['band'+str(bandNo)] - psf_b_dat = psf_order_b['band_data'] + psf_order = psf_data["order" + grating_order] + psf_order_b = psf_order["band" + str(bandNo)] + psf_b_dat = psf_order_b["band_data"] pos_p = psf_b_dat[1].data pc_coeff = psf_b_dat[2].data pcs = psf_b_dat[0].data # print(max(pos_p[:,0]), min(pos_p[:,0]),max(pos_p[:,1]), min(pos_p[:,1])) # print(chip.x_cen, chip.y_cen) # print(pos_p) - px = pos_img.x*chip.pix_size - py = pos_img.y*chip.pix_size + px = pos_img.x * chip.pix_size + py = pos_img.y * chip.pix_size - dist2 = (pos_p[:, 1] - px)*(pos_p[:, 1] - px) + \ - (pos_p[:, 0] - py)*(pos_p[:, 0] - py) + dist2 = (pos_p[:, 1] - px) * (pos_p[:, 1] - px) + (pos_p[:, 0] - py) * ( + pos_p[:, 0] - py + ) temp_sort_dist = np.zeros([dist2.shape[0], 2]) temp_sort_dist[:, 0] = np.arange(0, dist2.shape[0], 1) temp_sort_dist[:, 1] = dist2 @@ -417,18 +432,21 @@ class PSFInterpSLS(PSFModel): nearest4p[i, 0] = pos_p[smaller_ids, 1] nearest4p[i, 1] = pos_p[smaller_ids, 0] pc_coeff_4p[:, i] = pc_coeff[:, smaller_ids] - idw_dist = 1/(np.sqrt((px-nearest4p[:, 0]) * (px-nearest4p[:, 0]) + ( - py-nearest4p[:, 1]) * (py-nearest4p[:, 1]))) + idw_dist = 1 / ( + np.sqrt( + (px - nearest4p[:, 0]) * (px - nearest4p[:, 0]) + + (py - nearest4p[:, 1]) * (py - nearest4p[:, 1]) + ) + ) coeff_int = np.zeros(pc_coeff.data.shape[0]) for i in np.arange(4): - coeff_int = coeff_int + pc_coeff_4p[:, i]*idw_dist[i] + coeff_int = coeff_int + pc_coeff_4p[:, i] * idw_dist[i] coeff_int = coeff_int / np.sum(coeff_int) npc = 10 - m_size = int(pcs.shape[0]**0.5) - PSF_int = np.dot(pcs[:, 0:npc], coeff_int[0:npc] - ).reshape(m_size, m_size) + m_size = int(pcs.shape[0] ** 0.5) + PSF_int = np.dot(pcs[:, 0:npc], coeff_int[0:npc]).reshape(m_size, m_size) # PSF_int = PSF_int/np.sum(PSF_int) PSF_int_trans = np.flipud(np.fliplr(PSF_int)) @@ -437,9 +455,9 @@ class PSFInterpSLS(PSFModel): # ids_szero = PSF_int_trans<0 # PSF_int_trans[ids_szero] = 0 # print(PSF_int_trans[ids_szero].shape[0],PSF_int_trans.shape) - PSF_int_trans = PSF_int_trans/np.sum(PSF_int_trans) - PSF_int_trans = PSF_int_trans-np.min(PSF_int_trans) - PSF_int_trans = PSF_int_trans/np.sum(PSF_int_trans) + PSF_int_trans = PSF_int_trans / np.sum(PSF_int_trans) + PSF_int_trans = PSF_int_trans - np.min(PSF_int_trans) + PSF_int_trans = PSF_int_trans / np.sum(PSF_int_trans) # fits.writeto('/home/zhangxin/CSST_SIM/CSST_sim_develop/psf_test/psf.fits',PSF_int_trans) # DEBGU ids_szero = PSF_int_trans < 0 @@ -448,8 +466,7 @@ class PSFInterpSLS(PSFModel): n1 = np.sum(np.isinf(PSF_int_trans)) n2 = np.sum(np.isnan(PSF_int_trans)) if n1 > 0 or n2 > 0: - print("DEBUG: PSFInterpSLS, inf:%d, nan:%d, 0 num:%d" % - (n1, n2, n01)) + print("DEBUG: PSFInterpSLS, inf:%d, nan:%d, 0 num:%d" % (n1, n2, n01)) if extrapolate is True: # for rep_i in np.arange(0, 2, 1): # PSF_int_trans[rep_i,:] = 1e9*pow(10,rep_i) @@ -480,7 +497,6 @@ class PSFInterpSLS(PSFModel): # chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) # chip.img.wcs = galsim.wcs.AffineTransform if galsimGSObject: - # imPSFt = np.zeros([257,257]) # imPSFt[0:256, 0:256] = imPSF # # imPSFt[120:130, 0:256] = 1. @@ -498,15 +514,22 @@ class PSFInterpSLS(PSFModel): # self.psf = galsim.Convolve(self.psf, add_psf) wcs = chip.img.wcs.local(pos_img) scale = galsim.PixelScale(0.074) - self.psf = wcs.toWorld(scale.toImage( - self.psf), image_pos=(pos_img)) + self.psf = wcs.toWorld(scale.toImage(self.psf), image_pos=(pos_img)) # return self.PSFspin(x=px/0.01, y=py/0.01) - return self.psf, galsim.Shear(e=0., beta=(np.pi/2)*galsim.radians) + return self.psf, galsim.Shear(e=0.0, beta=(np.pi / 2) * galsim.radians) return PSF_int_trans, PSF_int - def get_PSF_AND_convolve_withsubImg(self, chip, cutImg=None, pos_img_local=[1000, 1000], bandNo=1, g_order='A', grating_split_pos=3685): + def get_PSF_AND_convolve_withsubImg( + self, + chip, + cutImg=None, + pos_img_local=[1000, 1000], + bandNo=1, + g_order="A", + grating_split_pos=3685, + ): """ Get the PSF at a given image position @@ -519,11 +542,14 @@ class PSFInterpSLS(PSFModel): Returns: PSF: A 'galsim.GSObject'. """ - order_IDs = {'A': '1', 'B': '0', 'C': '0', 'D': '0', 'E': '0'} - contam_order_sigma = {'C': 0.28032344707964174, - 'D': 0.39900182912061344, 'E': 1.1988309797685412} # arcsec - x_start = chip.x_cen/chip.pix_size - chip.npix_x / 2. - y_start = chip.y_cen/chip.pix_size - chip.npix_y / 2. + order_IDs = {"A": "1", "B": "0", "C": "0", "D": "0", "E": "0"} + contam_order_sigma = { + "C": 0.28032344707964174, + "D": 0.39900182912061344, + "E": 1.1988309797685412, + } # arcsec + x_start = chip.x_cen / chip.pix_size - chip.npix_x / 2.0 + y_start = chip.y_cen / chip.pix_size - chip.npix_y / 2.0 # print(pos_img.x - x_start) # pos_img_x = pos_img_local[0] + x_start # pos_img_y = pos_img_local[1] + y_start @@ -539,22 +565,24 @@ class PSFInterpSLS(PSFModel): # grating_order = '1' # if grating_order in ['0', '1']: - psf_order = psf_data['order'+grating_order] - psf_order_b = psf_order['band'+str(bandNo)] - psf_b_dat = psf_order_b['band_data'] + psf_order = psf_data["order" + grating_order] + psf_order_b = psf_order["band" + str(bandNo)] + psf_b_dat = psf_order_b["band_data"] # pos_p = psf_b_dat[1].data - pos_p = psf_b_dat[1].data/chip.pix_size - np.array([y_start, x_start]) + pos_p = psf_b_dat[1].data / chip.pix_size - np.array([y_start, x_start]) pc_coeff = psf_b_dat[2].data pcs = psf_b_dat[0].data npc = 10 - m_size = int(pcs.shape[0]**0.5) + m_size = int(pcs.shape[0] ** 0.5) sumImg = np.sum(cutImg.array) - tmp_img = cutImg*0 + tmp_img = cutImg * 0 for j in np.arange(npc): - X_ = np.hstack((pos_p[:, 1].flatten()[:, None], pos_p[:, 0].flatten()[ - :, None]), dtype=np.float32) + X_ = np.hstack( + (pos_p[:, 1].flatten()[:, None], pos_p[:, 0].flatten()[:, None]), + dtype=np.float32, + ) Z_ = (pc_coeff[j].astype(np.float32)).flatten() # print(pc_coeff[j].shape[0], pos_p[:,1].shape[0], pos_p[:,0].shape[0]) cx_len = int(chip.npix_x) @@ -563,8 +591,8 @@ class PSFInterpSLS(PSFModel): n_y = np.arange(0, cy_len, 1, dtype=int) M, N = np.meshgrid(n_x, n_y) # t1=datetime.datetime.now() - # U = interpolate.griddata(X_, Z_, (M[0:cy_len, 0:cx_len],N[0:cy_len, 0:cx_len]), - # method='nearest',fill_value=1.0) + # U = interpolate.griddata(X_, Z_, (M[0:cy_len, 0:cx_len],N[0:cy_len, 0:cx_len]), + # method='nearest',fill_value=1.0) b_img = galsim.Image(cx_len, cy_len) b_img.setOrigin(0, 0) bounds = cutImg.bounds & b_img.bounds @@ -588,11 +616,16 @@ class PSFInterpSLS(PSFModel): # if xe - xs <=0: # continue ys = bounds.ymin - ye = bounds.ymax+1 + ye = bounds.ymax + 1 xs = bounds.xmin - xe = bounds.xmax+1 - U = interpolate.griddata(X_, Z_, (M[ys:ye, xs:xe], N[ys:ye, xs:xe]), - method='nearest', fill_value=1.0) + xe = bounds.xmax + 1 + U = interpolate.griddata( + X_, + Z_, + (M[ys:ye, xs:xe], N[ys:ye, xs:xe]), + method="nearest", + fill_value=1.0, + ) # t2=datetime.datetime.now() # print("time interpolate:", t2-t1) @@ -601,10 +634,11 @@ class PSFInterpSLS(PSFModel): # print('DEBUG:SHAPE',cutImg.ncol,cutImg.nrow,cutImg.xmin, cutImg.ymin) # continue img_tmp = cutImg - img_tmp[bounds] = img_tmp[bounds]*U + img_tmp[bounds] = img_tmp[bounds] * U psf = pcs[:, j].reshape(m_size, m_size) - tmp_img = tmp_img + \ - signal.fftconvolve(img_tmp.array, psf, mode='same', axes=None) + tmp_img = tmp_img + signal.fftconvolve( + img_tmp.array, psf, mode="same", axes=None + ) # t3=datetime.datetime.now() # print("time convole:", t3-t2) @@ -613,34 +647,35 @@ class PSFInterpSLS(PSFModel): if np.sum(tmp_img.array) == 0: tmp_img = cutImg else: - tmp_img = tmp_img/np.sum(tmp_img.array)*sumImg + tmp_img = tmp_img / np.sum(tmp_img.array) * sumImg return tmp_img - def convolveFullImgWithPCAPSF(self, chip, folding_threshold=5.e-3): + def convolveFullImgWithPCAPSF(self, chip, folding_threshold=5.0e-3): keys_L1 = chip_utils.getChipSLSGratingID(chip.chipID) # keys_L2 = ['order-2','order-1','order0','order1','order2'] - keys_L2 = ['order0', 'order1'] - keys_L3 = ['w1', 'w2', 'w3', 'w4'] + keys_L2 = ["order0", "order1"] + keys_L3 = ["w1", "w2", "w3", "w4"] npca = 10 - x_start = chip.x_cen/chip.pix_size - chip.npix_x / 2. - y_start = chip.y_cen/chip.pix_size - chip.npix_y / 2. + x_start = chip.x_cen / chip.pix_size - chip.npix_x / 2.0 + y_start = chip.y_cen / chip.pix_size - chip.npix_y / 2.0 for i, gt in enumerate(keys_L1): psfCo = self.grating1_data if i > 0: psfCo = self.grating2_data for od in keys_L2: - psfCo_L2 = psfCo['order1'] - if od in ['order-2', 'order-1', 'order0', 'order2']: - psfCo_L2 = psfCo['order0'] + psfCo_L2 = psfCo["order1"] + if od in ["order-2", "order-1", "order0", "order2"]: + psfCo_L2 = psfCo["order0"] for w in keys_L3: img = chip.img_stack[gt][od][w] - pcs = psfCo_L2['band'+w[1]]['band_data'][0].data - pos_p = psfCo_L2['band'+w[1]]['band_data'][1].data / \ - chip.pix_size - np.array([y_start, x_start]) - pc_coeff = psfCo_L2['band'+w[1]]['band_data'][2].data + pcs = psfCo_L2["band" + w[1]]["band_data"][0].data + pos_p = psfCo_L2["band" + w[1]]["band_data"][ + 1 + ].data / chip.pix_size - np.array([y_start, x_start]) + pc_coeff = psfCo_L2["band" + w[1]]["band_data"][2].data # print("DEBUG-----------",np.max(pos_p[:,1]),np.min(pos_p[:,1]), np.max(pos_p[:,0]),np.min(pos_p[:,0])) sum_img = np.sum(img.array) @@ -676,47 +711,55 @@ class PSFInterpSLS(PSFModel): # coeff_int = coeff_int + pc_coeff_4p[:, i]*idw_dist[i] # coeff_mat[:, m, n] = coeff_int - m_size = int(pcs.shape[0]**0.5) + m_size = int(pcs.shape[0] ** 0.5) tmp_img = np.zeros_like(img.array, dtype=np.float32) for j in np.arange(npca): print(gt, od, w, j) - X_ = np.hstack((pos_p[:, 1].flatten()[:, None], pos_p[:, 0].flatten()[ - :, None]), dtype=np.float32) + X_ = np.hstack( + ( + pos_p[:, 1].flatten()[:, None], + pos_p[:, 0].flatten()[:, None], + ), + dtype=np.float32, + ) Z_ = (pc_coeff[j].astype(np.float32)).flatten() # print(pc_coeff[j].shape[0], pos_p[:,1].shape[0], pos_p[:,0].shape[0]) sub_size = 4 - cx_len = int(chip.npix_x/sub_size) - cy_len = int(chip.npix_y/sub_size) + cx_len = int(chip.npix_x / sub_size) + cy_len = int(chip.npix_y / sub_size) n_x = np.arange(0, chip.npix_x, sub_size, dtype=int) n_y = np.arange(0, chip.npix_y, sub_size, dtype=int) M, N = np.meshgrid(n_x, n_y) t1 = datetime.datetime.now() - # U = interpolate.griddata(X_, Z_, (M[0:cy_len, 0:cx_len],N[0:cy_len, 0:cx_len]), - # method='nearest',fill_value=1.0) - U1 = interpolate.griddata(X_, Z_, (M, N), - method='nearest', fill_value=1.0) + # U = interpolate.griddata(X_, Z_, (M[0:cy_len, 0:cx_len],N[0:cy_len, 0:cx_len]), + # method='nearest',fill_value=1.0) + U1 = interpolate.griddata( + X_, Z_, (M, N), method="nearest", fill_value=1.0 + ) U = np.zeros_like(chip.img.array, dtype=np.float32) for mi in np.arange(cy_len): for mj in np.arange(cx_len): - U[mi*sub_size:(mi+1)*sub_size, mj * - sub_size:(mj+1)*sub_size] = U1[mi, mj] + U[ + mi * sub_size : (mi + 1) * sub_size, + mj * sub_size : (mj + 1) * sub_size, + ] = U1[mi, mj] t2 = datetime.datetime.now() - print("time interpolate:", t2-t1) + print("time interpolate:", t2 - t1) - img_tmp = img.array*U + img_tmp = img.array * U psf = pcs[:, j].reshape(m_size, m_size) - tmp_img = tmp_img + \ - signal.fftconvolve( - img_tmp, psf, mode='same', axes=None) + tmp_img = tmp_img + signal.fftconvolve( + img_tmp, psf, mode="same", axes=None + ) t3 = datetime.datetime.now() - print("time convole:", t3-t2) + print("time convole:", t3 - t2) del U del U1 - chip.img = chip.img + tmp_img*sum_img/np.sum(tmp_img) + chip.img = chip.img + tmp_img * sum_img / np.sum(tmp_img) del tmp_img gc.collect() @@ -766,6 +809,7 @@ class PSFInterpSLS(PSFModel): # # return self.PSFspin(x=px/0.01, y=py/0.01) # return self.psf, galsim.Shear(e=0., beta=(np.pi/2)*galsim.radians) # return imPSF + # # def PSFspin(self, x, y): # """ @@ -791,8 +835,8 @@ class PSFInterpSLS(PSFModel): # return self.psf.shear(PSFshear), PSFshear -if __name__ == '__main__': - configfn = '/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_new_sim/csst-simulation/config/config_C6_dev.yaml' +if __name__ == "__main__": + configfn = "/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_new_sim/csst-simulation/config/config_C6_dev.yaml" with open(configfn, "r") as stream: try: config = yaml.safe_load(stream) @@ -802,11 +846,12 @@ if __name__ == '__main__': print(exc) chip = Chip(chipID=1, config=config) filter_id, filter_type = chip.getChipFilter() - filt = Filter(filter_id=filter_id, - filter_type=filter_type, - filter_param=FilterParam()) + filt = Filter( + filter_id=filter_id, filter_type=filter_type, filter_param=FilterParam() + ) psf_i = PSFInterpSLS( - chip, filt, PSF_data_prefix="/Volumes/EAGET/CSST_PSF_data/SLS_PSF_PCA_fp/") + chip, filt, PSF_data_prefix="/Volumes/EAGET/CSST_PSF_data/SLS_PSF_PCA_fp/" + ) pos_img = galsim.PositionD(x=25155, y=-22060) - psf_im = psf_i.get_PSF(chip, pos_img=pos_img, g_order='1') + psf_im = psf_i.get_PSF(chip, pos_img=pos_img, g_order="1") diff --git a/observation_sim/psf/PSFModel.py b/observation_sim/psf/PSFModel.py index 604ff8b20130ea43ef26e096936b879e842da1f1..7a79e25a7ee60ae8784cc54a6947b5def27cf5bc 100755 --- a/observation_sim/psf/PSFModel.py +++ b/observation_sim/psf/PSFModel.py @@ -1,14 +1,9 @@ import galsim -import sep import numpy as np -from scipy.interpolate import interp1d -import pylab as pl -import os -import sys class PSFModel(object): - def __init__(self, sigSpin=0., psfRa=0.15): + def __init__(self, sigSpin=0.0, psfRa=0.15): # TODO: what are the nesseary fields in PSFModel class? pass @@ -23,21 +18,21 @@ class PSFModel(object): Return: Spinned PSF: g1, g2 and axis ratio 'a/b' """ - a2Rad = np.pi/(60.0*60.0*180.0) + a2Rad = np.pi / (60.0 * 60.0 * 180.0) - ff = sigGauss * 0.107 * (1000.0/10.0) # in unit of [pixels] - rc = np.sqrt(dx*dx + dy*dy) - cpix = rc*(sigSpin*a2Rad) + ff = sigGauss * 0.107 * (1000.0 / 10.0) # in unit of [pixels] + rc = np.sqrt(dx * dx + dy * dy) + cpix = rc * (sigSpin * a2Rad) - beta = (np.arctan2(dy, dx) + np.pi/2) - ell = cpix**2/(2.0*ff**2+cpix**2) + beta = np.arctan2(dy, dx) + np.pi / 2 + ell = cpix**2 / (2.0 * ff**2 + cpix**2) # ell *= 10.0 - qr = np.sqrt((1.0+ell)/(1.0-ell)) + qr = np.sqrt((1.0 + ell) / (1.0 - ell)) # psfShape = galsim.Shear(e=ell, beta=beta) # g1, g2 = psfShape.g1, psfShape.g2 # qr = np.sqrt((1.0+ell)/(1.0-ell)) # return ell, beta, qr - PSFshear = galsim.Shear(e=ell, beta=beta*galsim.radians) + PSFshear = galsim.Shear(e=ell, beta=beta * galsim.radians) return psf.shear(PSFshear), PSFshear diff --git a/observation_sim/psf/__init__.py b/observation_sim/psf/__init__.py index 43e878cdcabc19993b6f9c9c572b725b214daf50..4d17e29190be0538d339eba26ffd2096de81e876 100755 --- a/observation_sim/psf/__init__.py +++ b/observation_sim/psf/__init__.py @@ -1,5 +1,6 @@ from .PSFModel import PSFModel from .PSFGauss import PSFGauss + # from .PSFInterp.PSFInterp import PSFInterp from .PSFInterp import PSFInterp from .PSFInterpSLS import PSFInterpSLS diff --git a/observation_sim/psf/_util.py b/observation_sim/psf/_util.py index 1b7aa102c8f49dd26ddcf3aabb7e3056ac1b03b4..1ac2bc3860aeed6257b78a5c106f27a36d2382dd 100644 --- a/observation_sim/psf/_util.py +++ b/observation_sim/psf/_util.py @@ -3,14 +3,16 @@ from scipy.interpolate import interp1d def binningPSF(img, ngg): - imgX = img.reshape(ngg, img.shape[0]//ngg, ngg, img.shape[1]//ngg).mean(-1).mean(1) + imgX = ( + img.reshape(ngg, img.shape[0] // ngg, ngg, img.shape[1] // ngg).mean(-1).mean(1) + ) return imgX def radial_average_at_pixel(image, center_x, center_y, dr=10): # Get coordinates relative to the specified center pixel (x, y) y, x = np.indices(image.shape) - r = np.sqrt((x - center_x)**2 + (y - center_y)**2) + r = np.sqrt((x - center_x) ** 2 + (y - center_y) ** 2) # Set up bins max_radius = int(r.max()) # Maximum distance from the center pixel @@ -31,7 +33,7 @@ def psf_extrapolate(psf, rr_trim=64, ngg=256): # ngg = 256 # extrapolate PSF if True: - xim = np.arange(256)-128 + xim = np.arange(256) - 128 xim, yim = np.meshgrid(xim, xim) rim = np.sqrt(xim**2 + yim**2) @@ -44,24 +46,32 @@ def psf_extrapolate(psf, rr_trim=64, ngg=256): means_log = np.log(means[1:]) finite_mask = np.isfinite(means_log) - f_interp = interp1d(radii_log[finite_mask][:-1], means_log[finite_mask][:-1], kind='linear', fill_value="extrapolate") + f_interp = interp1d( + radii_log[finite_mask][:-1], + means_log[finite_mask][:-1], + kind="linear", + fill_value="extrapolate", + ) # ngg = 1024 - xim = np.arange(ngg)-int(ngg/2) + xim = np.arange(ngg) - int(ngg / 2) xim, yim = np.meshgrid(xim, xim) rim = np.sqrt(xim**2 + yim**2) - rim[int(ngg/2), int(ngg/2)] = np.finfo(float).eps # 1e-7 + rim[int(ngg / 2), int(ngg / 2)] = np.finfo(float).eps # 1e-7 rim_log = np.log(rim) y_new = f_interp(rim_log) arr = np.zeros([ngg, ngg]) - arr[int(ngg/2-128):int(ngg/2+128), int(ngg/2-128):int(ngg/2+128)] = np.log(psf_temp + np.finfo(float).eps) + arr[ + int(ngg / 2 - 128) : int(ngg / 2 + 128), + int(ngg / 2 - 128) : int(ngg / 2 + 128), + ] = np.log(psf_temp + np.finfo(float).eps) arr[rim > rr_trim] = 0 arr[arr == 0] = y_new[arr == 0] psf = np.exp(arr) - psf[rim > int(ngg/2)] = 0 + psf[rim > int(ngg / 2)] = 0 imPSF = psf # binningPSF(psf, int(ngg/2)) - imPSF = imPSF/np.nansum(imPSF) + imPSF = imPSF / np.nansum(imPSF) return imPSF @@ -70,15 +80,18 @@ def psf_extrapolate1(psf, rr_trim=64, ngg=256): # extrapolate PSF if True: psf_enlar = np.zeros([ngg, ngg]) - psf_enlar[int(ngg/2-128):int(ngg/2+128), int(ngg/2-128):int(ngg/2+128)] = psf - xim = np.arange(ngg)-ngg/2 + psf_enlar[ + int(ngg / 2 - 128) : int(ngg / 2 + 128), + int(ngg / 2 - 128) : int(ngg / 2 + 128), + ] = psf + xim = np.arange(ngg) - ngg / 2 xim, yim = np.meshgrid(xim, xim) rim = np.sqrt(xim**2 + yim**2) psf_temp = psf_enlar # psf_temp[rim >= rr_trim] = 0 - psf_temp[rim >= ngg/2-2] = np.finfo(float).eps - radii, means = radial_average_at_pixel(psf_temp, ngg/2, ngg/2, dr=2) + psf_temp[rim >= ngg / 2 - 2] = np.finfo(float).eps + radii, means = radial_average_at_pixel(psf_temp, ngg / 2, ngg / 2, dr=2) radii_log = np.log(radii[1:]) # radii_log = radii[1:] @@ -98,7 +111,12 @@ def psf_extrapolate1(psf, rr_trim=64, ngg=256): # means_log = np.log10(means[1:]) finite_mask = np.isfinite(means_log) - f_interp = interp1d(radii_log[finite_mask][:-1], means_log[finite_mask][:-1], kind='linear', fill_value="extrapolate") + f_interp = interp1d( + radii_log[finite_mask][:-1], + means_log[finite_mask][:-1], + kind="linear", + fill_value="extrapolate", + ) # ngg = 1024 # xim = np.arange(ngg)-int(ngg/2) @@ -109,8 +127,11 @@ def psf_extrapolate1(psf, rr_trim=64, ngg=256): y_new = f_interp(rim_log) arr = np.zeros([ngg, ngg]) - arr[int(ngg/2-128):int(ngg/2+128), int(ngg/2-128):int(ngg/2+128)] = np.log(psf + np.finfo(float).eps) - arr[rim > 128-2] = 0 + arr[ + int(ngg / 2 - 128) : int(ngg / 2 + 128), + int(ngg / 2 - 128) : int(ngg / 2 + 128), + ] = np.log(psf + np.finfo(float).eps) + arr[rim > 128 - 2] = 0 arr[arr == 0] = y_new[arr == 0] psf_n = np.exp(arr) # psf_n[int(ngg/2-128):int(ngg/2+128), int(ngg/2-128):int(ngg/2+128)] = psf diff --git a/observation_sim/run.py b/observation_sim/run.py new file mode 100644 index 0000000000000000000000000000000000000000..c67112d04e1e6fef432d5b20563953425951acab --- /dev/null +++ b/observation_sim/run.py @@ -0,0 +1,134 @@ +from observation_sim.ObservationSim import Observation +from observation_sim._util import parse_args, make_run_dirs, generate_pointing_list +from importlib.metadata import version +from pathlib import Path +import os +import sys +import yaml +import shutil +import datetime +import importlib + +import gc + +gc.enable() + + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) + + +def run_sim(): + """ + Main method for simulation call. + + Parameters + ---------- + Catalog : Class + a catalog class which is inherited from observation_sim.mock_objects.CatalogBase + + Returns + ---------- + None + """ + # Get version of the Package + __version__ = version("csst_msc_sim") + + # Get run datetime + now = datetime.datetime.now() + + args = parse_args() + if args.config_dir is None: + args.config_dir = "" + args.config_dir = os.path.abspath(args.config_dir) + args.config_file = os.path.join(args.config_dir, args.config_file) + with open(args.config_file, "r") as stream: + try: + config = yaml.safe_load(stream) + for key, value in config.items(): + print(key + " : " + str(value)) + except yaml.YAMLError as exc: + print(exc) + + # Overwrite the data and working directories + # if they are specified by the command line inputs + if args.data_dir is not None: + config["data_dir"] = args.data_dir + if args.work_dir is not None: + config["work_dir"] = args.work_dir + + if "data_dir" not in config: + config["data_dir"] = None + + # Some default values + if "mag_sat_margin" not in config["obs_setting"]: + config["obs_setting"]["mag_sat_margin"] = -2.5 + if "mag_lim_margin" not in config["obs_setting"]: + config["obs_setting"]["mag_lim_margin"] = 1.0 + if "project_cycle" not in config: + config["project_cycle"] = 6 + if "run_counter" not in config: + config["run_counter"] = 0 + + if "data_set" not in config: + config["data_set"] = "csst-msc" + + # Generate lists pointings based on the input pointing list (or default + # pointing RA, DEC) and "config["obs_setting"]["run_pointings"]". + # "config['obs_setting']['np_cal']"" is the number of CAL pointings which will be + # appended to the front. + # NOTE: the implementation of gerenating time_stamps is temporary. + pointing_dir = None + if "pointing_dir" in config["obs_setting"]: + pointing_dir = config["obs_setting"]["pointing_dir"] + pointing_list = generate_pointing_list( + config=config, + pointing_filename=config["obs_setting"]["pointing_file"], + data_dir=pointing_dir, + dataset=config["data_set"], + ) + + # Make the main output directories + run_dir = make_run_dirs( + work_dir=config["work_dir"], + run_name=config["run_name"], + pointing_list=pointing_list, + ) + + # Copy the config file to output directory & Write Run metadata + shutil.copy(args.config_file, run_dir) + run_meta = os.path.join(run_dir, "run_metadata.yaml") + with open(run_meta, "w") as config_out: + config_out.write("\n") + config_out.write("###############################################\n") + config_out.write('csst_msc_sim package version: "%s"\n' % __version__) + date_str = datetime.datetime.strftime(now, "%m/%d/%Y") + time_str = datetime.datetime.strftime(now, "%H:%M:%S") + config_out.write('Run_date: "%s"\n' % date_str) + config_out.write('Run_time: "%s"\n' % time_str) + config_out.write("###############################################\n") + + # Initialize the simulation + if args.catalog is not None: + catalog_module = importlib.import_module("catalog." + args.catalog) + obs = Observation( + config=config, + Catalog=catalog_module.Catalog, + work_dir=config["work_dir"], + data_dir=config["data_dir"], + ) + else: + catalog_module = None + obs = Observation( + config=config, + Catalog=None, + work_dir=config["work_dir"], + data_dir=config["data_dir"], + ) + + # Run simulation + obs.runExposure_MPI_PointingList(pointing_list=pointing_list) + + +if __name__ == "__main__": + run_sim() diff --git a/observation_sim/sim_steps/__init__.py b/observation_sim/sim_steps/__init__.py index 7669fe45bae94f3ed309840911e1d9c8660d89e7..9170c36129ffdddbba9fcf6710e040578c71b961 100644 --- a/observation_sim/sim_steps/__init__.py +++ b/observation_sim/sim_steps/__init__.py @@ -2,7 +2,9 @@ import os class SimSteps: - def __init__(self, overall_config, chip_output, all_filters, ra_offset=0., dec_offset=0.): + def __init__( + self, overall_config, chip_output, all_filters, ra_offset=0.0, dec_offset=0.0 + ): self.overall_config = overall_config self.chip_output = chip_output self.all_filters = all_filters @@ -10,17 +12,36 @@ class SimSteps: self.dec_offset = dec_offset from .prepare_headers import prepare_headers, updateHeaderInfo - from .add_sky_background import add_sky_background_sci, add_sky_flat_calibration, add_sky_background + from .add_sky_background import ( + add_sky_background_sci, + add_sky_flat_calibration, + add_sky_background, + ) from .add_objects import add_objects, _is_obj_valid from .add_cosmic_rays import add_cosmic_rays - from .add_pattern_noise import apply_PRNU, add_poisson_and_dark, add_detector_defects, add_nonlinearity, add_blooming, add_bias + from .add_pattern_noise import ( + apply_PRNU, + add_poisson_and_dark, + add_detector_defects, + add_nonlinearity, + add_blooming, + add_bias, + ) from .add_brighter_fatter_CTE import add_brighter_fatter, apply_CTE - from .readout_output import add_prescan_overscan, add_readout_noise, apply_gain, quantization_and_output, add_crosstalk + from .readout_output import ( + add_prescan_overscan, + add_readout_noise, + apply_gain, + quantization_and_output, + add_crosstalk, + ) from .add_LED_flat import add_LED_Flat + from .add_ghost import add_ghosts_SCI SIM_STEP_TYPES = { "scie_obs": "add_objects", + "ghost_SCI": "add_ghosts_SCI", "sky_background": "add_sky_background", "cosmic_rays": "add_cosmic_rays", "PRNU_effect": "apply_PRNU", @@ -37,5 +58,5 @@ SIM_STEP_TYPES = { "quantization_and_output": "quantization_and_output", "led_calib_model": "add_LED_Flat", "sky_flatField": "add_sky_flat_calibration", - "cross_talk": "add_crosstalk" + "cross_talk": "add_crosstalk", } diff --git a/observation_sim/sim_steps/add_LED_flat.py b/observation_sim/sim_steps/add_LED_flat.py index 7c5683c9478846afb652b9c3276c012ee94e9a11..97b6d22956f354f4649563e27ee79eabab3e26ef 100644 --- a/observation_sim/sim_steps/add_LED_flat.py +++ b/observation_sim/sim_steps/add_LED_flat.py @@ -10,7 +10,7 @@ import gc def add_LED_Flat(self, chip, filt, tel, pointing, catalog, obs_param): - if not hasattr(self, 'h_ext'): + if not hasattr(self, "h_ext"): _, _ = self.prepare_headers(chip=chip, pointing=pointing) chip_wcs = galsim.FitsWCS(header=self.h_ext) pf_map = np.zeros_like(chip.img.array) @@ -20,21 +20,41 @@ def add_LED_Flat(self, chip, filt, tel, pointing, catalog, obs_param): led_obj = FlatLED(chip, filt) led_flat, ledstat, letts = led_obj.drawObj_LEDFlat( - led_type_list=obs_param["LED_TYPE"], exp_t_list=obs_param["LED_TIME"]) + led_type_list=obs_param["LED_TYPE"], exp_t_list=obs_param["LED_TIME"] + ) pf_map = led_flat - self.updateHeaderInfo(header_flag='ext', keys=[ - 'LEDSTAT'], values=[ledstat]) - self.updateHeaderInfo(header_flag='ext', keys=['LEDT01', 'LEDT02', 'LEDT03', 'LEDT04', 'LEDT05', 'LEDT06', - 'LEDT07', 'LEDT08', 'LEDT09', 'LEDT10', 'LEDT11', 'LEDT12', 'LEDT13', 'LEDT14'], values=letts) + self.updateHeaderInfo(header_flag="ext", keys=["LEDSTAT"], values=[ledstat]) + self.updateHeaderInfo( + header_flag="ext", + keys=[ + "LEDT01", + "LEDT02", + "LEDT03", + "LEDT04", + "LEDT05", + "LEDT06", + "LEDT07", + "LEDT08", + "LEDT09", + "LEDT10", + "LEDT11", + "LEDT12", + "LEDT13", + "LEDT14", + ], + values=letts, + ) if obs_param["shutter_effect"] is True: pf_map = pf_map * chip.shutter_img - pf_map = np.array(pf_map, dtype='float32') - self.updateHeaderInfo(header_flag='ext', keys=[ - 'SHTSTAT'], values=[True]) + pf_map = np.array(pf_map, dtype="float32") + self.updateHeaderInfo(header_flag="ext", keys=["SHTSTAT"], values=[True]) else: - self.updateHeaderInfo(header_flag='ext', keys=['SHTSTAT', 'SHTOPEN1', 'SHTCLOS0'], values=[ - True, self.h_ext['SHTCLOS1'], self.h_ext['SHTOPEN0']]) + self.updateHeaderInfo( + header_flag="ext", + keys=["SHTSTAT", "SHTOPEN1", "SHTCLOS0"], + values=[True, self.h_ext["SHTCLOS1"], self.h_ext["SHTOPEN0"]], + ) chip.img = chip.img + pf_map @@ -44,16 +64,27 @@ def add_LED_Flat(self, chip, filt, tel, pointing, catalog, obs_param): t_obs = Time(datetime_obs) # ccd刷新2s,等待0.5s，开灯后等待0.5s,开始曝光 - t_obs_renew = Time(t_obs.mjd - (2.) / 86400., format="mjd") + t_obs_renew = Time(t_obs.mjd - (2.0) / 86400.0, format="mjd") - t_obs_utc = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_obs_renew.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - self.updateHeaderInfo(header_flag='prim', keys=[ - 'DATE-OBS'], values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]]) + t_obs_utc = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_obs_renew.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + self.updateHeaderInfo( + header_flag="prim", + keys=["DATE-OBS"], + values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]], + ) # dark time : - self.updateHeaderInfo(header_flag='ext', keys=[ - 'DARKTIME'], values=[pointing.exp_time]) + self.updateHeaderInfo( + header_flag="ext", keys=["DARKTIME"], values=[pointing.exp_time] + ) gc.collect() return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/add_brighter_fatter_CTE.py b/observation_sim/sim_steps/add_brighter_fatter_CTE.py index d235b4161f16c5da15d7e4c22f629f86fc00b5e5..e005e619b4ffa8f2330183cf0280a2d4fe9fe3e9 100644 --- a/observation_sim/sim_steps/add_brighter_fatter_CTE.py +++ b/observation_sim/sim_steps/add_brighter_fatter_CTE.py @@ -18,21 +18,38 @@ def apply_CTE(self, chip, filt, tel, pointing, catalog, obs_param): img_arr = img.array ny, nx = img_arr.shape - dx = int(nx/chip.nsecx) - dy = int(ny/chip.nsecy) - newimg = galsim.Image(nx, int(ny+chip.overscan_y), init_value=0) + dx = int(nx / chip.nsecx) + dy = int(ny / chip.nsecy) + newimg = galsim.Image(nx, int(ny + chip.overscan_y), init_value=0) for ichannel in range(16): - print('\n***add CTI effects: pointing-{:} chip-{:} channel-{:}***'.format( - pointing.id, chip.chipID, ichannel+1)) + print( + "\n***add CTI effects: pointing-{:} chip-{:} channel-{:}***".format( + pointing.id, chip.chipID, ichannel + 1 + ) + ) noverscan, nsp, nmax = chip.overscan_y, 3, 10 beta, w, c = 0.478, 84700, 0 t = np.array([0.74, 7.7, 37], dtype=np.float32) rho_trap = np.array([0.6, 1.6, 1.4], dtype=np.float32) - trap_seeds = np.array( - [0, 1000, 10000], dtype=np.int32) + ichannel + chip.chipID*16 - release_seed = 50 + ichannel + pointing.id*30 + chip.chipID*16 - newimg.array[:, 0+ichannel*dx:dx+ichannel*dx] = CTI_sim( - img_arr[:, 0+ichannel*dx:dx+ichannel*dx], dx, dy, noverscan, nsp, nmax, beta, w, c, t, rho_trap, trap_seeds, release_seed) + trap_seeds = ( + np.array([0, 1000, 10000], dtype=np.int32) + ichannel + chip.chipID * 16 + ) + release_seed = 50 + ichannel + pointing.id * 30 + chip.chipID * 16 + newimg.array[:, 0 + ichannel * dx : dx + ichannel * dx] = CTI_sim( + img_arr[:, 0 + ichannel * dx : dx + ichannel * dx], + dx, + dy, + noverscan, + nsp, + nmax, + beta, + w, + c, + t, + rho_trap, + trap_seeds, + release_seed, + ) newimg.wcs = img.wcs del img img = newimg diff --git a/observation_sim/sim_steps/add_cosmic_rays.py b/observation_sim/sim_steps/add_cosmic_rays.py index 77a470a3a4630175087ad8b2871df717ac36ab1b..15ec9885b6ca9666ece05a50591e47e5e82c35e2 100644 --- a/observation_sim/sim_steps/add_cosmic_rays.py +++ b/observation_sim/sim_steps/add_cosmic_rays.py @@ -14,18 +14,25 @@ def add_cosmic_rays(self, chip, filt, tel, pointing, catalog, obs_param): img=chip.img, chip=chip, exptime=exptime, - seed=self.overall_config["random_seeds"]["seed_CR"]+pointing.id*30+chip.chipID) + seed=self.overall_config["random_seeds"]["seed_CR"] + + pointing.id * 30 + + chip.chipID, + ) # Save cosmic ray image - if (obs_param) and ("save_cosmic_img" in obs_param) and (obs_param["save_cosmic_img"] is not None): + if ( + (obs_param) + and ("save_cosmic_img" in obs_param) + and (obs_param["save_cosmic_img"] is not None) + ): if obs_param["save_cosmic_img"]: chip_utils.output_fits_image( chip=chip, pointing=pointing, img=crmap_gsimg, output_dir=self.chip_output.subdir, - img_type='CRS', + img_type="CRS", img_type_code=pointing.pointing_type_code, project_cycle=self.overall_config["project_cycle"], - run_counter=self.overall_config["run_counter"] + run_counter=self.overall_config["run_counter"], ) return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/add_ghost.py b/observation_sim/sim_steps/add_ghost.py new file mode 100644 index 0000000000000000000000000000000000000000..c7909bba87e28d07c7f4e4071a72bb190610d7c1 --- /dev/null +++ b/observation_sim/sim_steps/add_ghost.py @@ -0,0 +1,107 @@ +import galsim +import traceback + +from observation_sim.mock_objects import Ghost +from observation_sim.psf import FieldDistortion + + +def add_ghosts_SCI(self, chip, filt, tel, pointing, catalog, obs_param): + # Load catalogues + if catalog is None: + self.chip_output.Log_error( + "Catalog interface class must be specified for SCIE-OBS" + ) + raise ValueError("Catalog interface class must be specified for SCIE-OBS") + cat = catalog( + config=self.overall_config, + chip=chip, + pointing=pointing, + chip_output=self.chip_output, + filt=filt, + ) + + # Apply field distortion model + if obs_param["field_dist"] is True: + fd_model = FieldDistortion(chip=chip, img_rot=pointing.img_pa.deg) + else: + fd_model = None + + # Get chip WCS + if not hasattr(self, "h_ext"): + _, _ = self.prepare_headers(chip=chip, pointing=pointing) + + chip_wcs = galsim.FitsWCS(header=self.h_ext) + ghost_model = Ghost() + + # Loop over objects + for j in range(len(cat.objs)): + obj = cat.objs[j] + + try: + sed_data = cat.load_sed(obj) + norm_filt = cat.load_norm_filt(obj) + + ( + obj.sed, + obj.param["mag_%s" % filt.filter_type.lower()], + obj.param["flux_%s" % filt.filter_type.lower()], + ) = cat.convert_sed( + mag=obj.param["mag_use_normal"], + sed=sed_data, + target_filt=filt, + norm_filt=norm_filt, + mu=obj.mu, + ) + except Exception as e: + traceback.print_exc() + self.chip_output.Log_error(e) + continue + + # Select only bright stars + if obj.type != "star" or obj.getMagFilter(filt) >= obs_param["mag_threshold"]: + continue + + # Get position of object on the focal plane + pos_img, _, _, _, fd_shear = obj.getPosImg_Offset_WCS( + img=chip.img, + fdmodel=fd_model, + chip=chip, + verbose=False, + chip_wcs=chip_wcs, + img_header=self.h_ext, + ra_offset=self.ra_offset, + dec_offset=self.dec_offset, + ) + + # [TODO] For now, only consider objects which their centers (after field distortion) are projected within the focal plane + if pos_img is None: + self.chip_output.Log_info( + "obj_ra = %.6f, obj_dec = %.6f, obj_ra_orig = %.6f, obj_dec_orig = %.6f" + % (obj.ra, obj.dec, obj.ra_orig, obj.dec_orig) + ) + self.chip_output.Log_error("Object missed: %s" % (obj.id)) + obj.unload_SED() + continue + + # Get number of photons for total photons for the object + nphotons_tot = obj.getElectronFluxFilt(filt, tel, pointing.exp_time) + + x_m = pos_img.x * chip.pix_size * 1e-3 + y_m = pos_img.y * chip.pix_size * 1e-3 + + ghost_list = ghost_model.calculate(chip.chipID, x_m, y_m)["ghosts"] + + for ghost in ghost_list: + x, y, r, ratio = ghost + factor = (1e-3) * chip.pix_size + x_pix = x / factor + y_pix = y / factor + r_pix = r / factor + ghost_pos = obj.getRealPos( + chip.img, global_x=x_pix, global_y=y_pix, img_real_wcs=obj.chip_wcs + ) + # print(f"star at {obj.getRealPos(chip.img, global_x=pos_img.x, global_y=pos_img.y, img_real_wcs=obj.chip_wcs)}") + # print("ghost info: ", ghost_pos.x, ghost_pos.y, r_pix, ratio * nphotons_tot) + ghost_model.draw_on_chip(chip, ghost_pos, r_pix, flux=ratio * nphotons_tot) + + return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/add_objects.py b/observation_sim/sim_steps/add_objects.py index 06bd712698c364bc54035a60ce9f27264acf30fd..4b8255db7ffb415d88293ac9c766e1a8498babd8 100644 --- a/observation_sim/sim_steps/add_objects.py +++ b/observation_sim/sim_steps/add_objects.py @@ -12,18 +12,17 @@ from datetime import datetime, timezone def _is_obj_valid(self, obj): - if obj.param['star'] == 4: + if obj.param["star"] == 4: # Currently there's no parameter checks for 'calib' type return True - pos_keys = ['ra', 'dec'] - shape_keys = ['hlr_bulge', 'hlr_disk', - 'e1_disk', 'e2_disk', 'e1_bulge', 'e2_bulge'] - if any(obj.param[key] == -999. for key in pos_keys): - msg = 'One or more positional information (ra, dec) is missing' + pos_keys = ["ra", "dec"] + shape_keys = ["hlr_bulge", "hlr_disk", "e1_disk", "e2_disk", "e1_bulge", "e2_bulge"] + if any(obj.param[key] == -999.0 for key in pos_keys): + msg = "One or more positional information (ra, dec) is missing" self.chip_output.Log_error(msg) return False - if obj.param['star'] == 0 and any(obj.param[key] == -999. for key in shape_keys): - msg = 'One or more shape information (hlr_bulge, hlr_disk, e1_disk, e2_disk, e1_bulge, e2_bulge) is missing' + if obj.param["star"] == 0 and any(obj.param[key] == -999.0 for key in shape_keys): + msg = "One or more shape information (hlr_bulge, hlr_disk, e1_disk, e2_disk, e1_bulge, e2_bulge) is missing" self.chip_output.Log_error(msg) return False return True @@ -40,11 +39,16 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): # Load catalogues if catalog is None: self.chip_output.Log_error( - "Catalog interface class must be specified for SCIE-OBS") - raise ValueError( - "Catalog interface class must be specified for SCIE-OBS") - cat = catalog(config=self.overall_config, chip=chip, - pointing=pointing, chip_output=self.chip_output, filt=filt) + "Catalog interface class must be specified for SCIE-OBS" + ) + raise ValueError("Catalog interface class must be specified for SCIE-OBS") + cat = catalog( + config=self.overall_config, + chip=chip, + pointing=pointing, + chip_output=self.chip_output, + filt=filt, + ) # Prepare output file(s) for this chip # [NOTE] Headers of output .cat file may be updated by Catalog instance @@ -54,14 +58,21 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): # Prepare the PSF model if self.overall_config["psf_setting"]["psf_model"] == "Gauss": psf_model = PSFGauss( - chip=chip, psfRa=self.overall_config["psf_setting"]["psf_rcont"]) + chip=chip, psfRa=self.overall_config["psf_setting"]["psf_rcont"] + ) elif self.overall_config["psf_setting"]["psf_model"] == "Interp": if chip.survey_type == "spectroscopic": psf_model = PSFInterpSLS( - chip, filt, PSF_data_prefix=self.overall_config["psf_setting"]["psf_sls_dir"]) + chip, + filt, + PSF_data_prefix=self.overall_config["psf_setting"]["psf_sls_dir"], + ) else: - psf_model = PSFInterp(chip=chip, npsf=chip.n_psf_samples, - PSF_data_file=self.overall_config["psf_setting"]["psf_pho_dir"]) + psf_model = PSFInterp( + chip=chip, + npsf=chip.n_psf_samples, + PSF_data_file=self.overall_config["psf_setting"]["psf_pho_dir"], + ) else: self.chip_output.Log_error("unrecognized PSF model type!!", flush=True) @@ -77,20 +88,24 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): temp_filter = self.all_filters[ifilt] temp_filter.update_limit_saturation_mags( exptime=pointing.exp_time, - full_depth_exptime=pointing.get_full_depth_exptime( - temp_filter.filter_type), - chip=chip) - if temp_filter.filter_type.lower() == self.overall_config["obs_setting"]["cut_in_band"].lower(): + full_depth_exptime=pointing.get_full_depth_exptime(temp_filter.filter_type), + chip=chip, + ) + if ( + temp_filter.filter_type.lower() + == self.overall_config["obs_setting"]["cut_in_band"].lower() + ): cut_filter = temp_filter # Read in shear values from configuration file if the constant shear type is used if self.overall_config["shear_setting"]["shear_type"] == "constant": g1_field, g2_field, _ = get_shear_field(config=self.overall_config) self.chip_output.Log_info( - "Use constant shear: g1=%.5f, g2=%.5f" % (g1_field, g2_field)) + "Use constant shear: g1=%.5f, g2=%.5f" % (g1_field, g2_field) + ) # Get chip WCS - if not hasattr(self, 'h_ext'): + if not hasattr(self, "h_ext"): _, _ = self.prepare_headers(chip=chip, pointing=pointing) chip_wcs = galsim.FitsWCS(header=self.h_ext) @@ -115,20 +130,28 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): sed_data = cat.load_sed(obj) norm_filt = cat.load_norm_filt(obj) - obj.sed, obj.param["mag_%s" % filt.filter_type.lower()], obj.param["flux_%s" % filt.filter_type.lower()] = cat.convert_sed( + ( + obj.sed, + obj.param["mag_%s" % filt.filter_type.lower()], + obj.param["flux_%s" % filt.filter_type.lower()], + ) = cat.convert_sed( mag=obj.param["mag_use_normal"], sed=sed_data, target_filt=filt, norm_filt=norm_filt, - mu=obj.mu + mu=obj.mu, ) - _, obj.param["mag_%s" % cut_filter.filter_type.lower()], obj.param["flux_%s" % cut_filter.filter_type.lower()] = cat.convert_sed( + ( + _, + obj.param["mag_%s" % cut_filter.filter_type.lower()], + obj.param["flux_%s" % cut_filter.filter_type.lower()], + ) = cat.convert_sed( mag=obj.param["mag_use_normal"], sed=sed_data, target_filt=cut_filter, norm_filt=(norm_filt if norm_filt else filt), - mu=obj.mu + mu=obj.mu, ) except Exception as e: traceback.print_exc() @@ -140,27 +163,41 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): # Exclude very bright/dim objects (for now) if cut_filter.is_too_bright( - mag=obj.param["mag_%s" % - self.overall_config["obs_setting"]["cut_in_band"].lower()], - margin=self.overall_config["obs_setting"]["mag_sat_margin"]): - self.chip_output.Log_info("obj %s too bright!! mag_%s = %.3f" % ( - obj.id, cut_filter.filter_type, obj.param["mag_%s" % self.overall_config["obs_setting"]["cut_in_band"].lower()])) + mag=obj.param[ + "mag_%s" % self.overall_config["obs_setting"]["cut_in_band"].lower() + ], + margin=self.overall_config["obs_setting"]["mag_sat_margin"], + ): + self.chip_output.Log_info( + "obj %s too bright!! mag_%s = %.3f" + % ( + obj.id, + cut_filter.filter_type, + obj.param[ + "mag_%s" + % self.overall_config["obs_setting"]["cut_in_band"].lower() + ], + ) + ) bright_obj += 1 obj.unload_SED() continue if filt.is_too_dim( - mag=obj.getMagFilter(filt), - margin=self.overall_config["obs_setting"]["mag_lim_margin"]): - self.chip_output.Log_info("obj %s too dim!! mag_%s = %.3f" % ( - obj.id, filt.filter_type, obj.getMagFilter(filt))) + mag=obj.getMagFilter(filt), + margin=self.overall_config["obs_setting"]["mag_lim_margin"], + ): + self.chip_output.Log_info( + "obj %s too dim!! mag_%s = %.3f" + % (obj.id, filt.filter_type, obj.getMagFilter(filt)) + ) dim_obj += 1 obj.unload_SED() continue # Get corresponding shear values if self.overall_config["shear_setting"]["shear_type"] == "constant": - if obj.type == 'star': - obj.g1, obj.g2 = 0., 0. + if obj.type == "star": + obj.g1, obj.g2 = 0.0, 0.0 else: # Figure out shear fields from overall configuration shear setting obj.g1, obj.g2 = g1_field, g2_field @@ -172,15 +209,25 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): # Get position of object on the focal plane pos_img, _, _, _, fd_shear = obj.getPosImg_Offset_WCS( - img=chip.img, fdmodel=fd_model, chip=chip, verbose=False, chip_wcs=chip_wcs, img_header=self.h_ext, ra_offset=self.ra_offset, dec_offset=self.dec_offset) + img=chip.img, + fdmodel=fd_model, + chip=chip, + verbose=False, + chip_wcs=chip_wcs, + img_header=self.h_ext, + ra_offset=self.ra_offset, + dec_offset=self.dec_offset, + ) # [TODO] For now, only consider objects which their centers (after field distortion) are projected within the focal plane # Otherwise they will be considered missed objects # if pos_img.x == -1 or pos_img.y == -1 or (not chip.isContainObj(x_image=pos_img.x, y_image=pos_img.y, margin=0.)): # if pos_img.x == -1 or pos_img.y == -1: if pos_img is None: - self.chip_output.Log_info('obj_ra = %.6f, obj_dec = %.6f, obj_ra_orig = %.6f, obj_dec_orig = %.6f' % ( - obj.ra, obj.dec, obj.ra_orig, obj.dec_orig)) + self.chip_output.Log_info( + "obj_ra = %.6f, obj_dec = %.6f, obj_ra_orig = %.6f, obj_dec_orig = %.6f" + % (obj.ra, obj.dec, obj.ra_orig, obj.dec_orig) + ) self.chip_output.Log_error("Object missed: %s" % (obj.id)) missed_obj += 1 obj.unload_SED() @@ -191,9 +238,16 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): if self.overall_config["run_option"]["out_cat_only"]: isUpdated = True obj.real_pos = obj.getRealPos( - chip.img, global_x=obj.posImg.x, global_y=obj.posImg.y, img_real_wcs=obj.chip_wcs) - pos_shear = 0. - elif chip.survey_type == "photometric" and not self.overall_config["run_option"]["out_cat_only"]: + chip.img, + global_x=obj.posImg.x, + global_y=obj.posImg.y, + img_real_wcs=obj.chip_wcs, + ) + pos_shear = 0.0 + elif ( + chip.survey_type == "photometric" + and not self.overall_config["run_option"]["out_cat_only"] + ): isUpdated, pos_shear = obj.drawObj_multiband( tel=tel, pos_img=pos_img, @@ -204,9 +258,13 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): g1=obj.g1, g2=obj.g2, exptime=exptime, - fd_shear=fd_shear) + fd_shear=fd_shear, + ) - elif chip.survey_type == "spectroscopic" and not self.overall_config["run_option"]["out_cat_only"]: + elif ( + chip.survey_type == "spectroscopic" + and not self.overall_config["run_option"]["out_cat_only"] + ): isUpdated, pos_shear = obj.drawObj_slitless( tel=tel, pos_img=pos_img, @@ -218,29 +276,45 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): g2=obj.g2, exptime=exptime, normFilter=norm_filt, - fd_shear=fd_shear) + fd_shear=fd_shear, + ) if isUpdated == 1: # TODO: add up stats self.chip_output.cat_add_obj( - obj, pos_img, pos_shear, ra_offset=self.ra_offset, dec_offset=self.dec_offset) + obj, + pos_img, + pos_shear, + ra_offset=self.ra_offset, + dec_offset=self.dec_offset, + ) pass elif isUpdated == 0: missed_obj += 1 self.chip_output.Log_error("Object missed: %s" % (obj.id)) else: - self.chip_output.Log_error( - "Draw error, object omitted: %s" % (obj.id)) + self.chip_output.Log_error("Draw error, object omitted: %s" % (obj.id)) continue except Exception as e: traceback.print_exc() self.chip_output.Log_error(e) self.chip_output.Log_error( - "pointing: #%d, chipID: %d" % (pointing.id, chip.chipID)) + "pointing: #%d, chipID: %d" % (pointing.id, chip.chipID) + ) if obj.type == "galaxy": - self.chip_output.Log_error("obj id: %s" % (obj.param['id'])) - self.chip_output.Log_error(" e1: %.5f\n e2: %.5f\n size: %f\n bfrac: %f\n detA: %f\n g1: %.5f\n g2: %.5f\n" % ( - obj.param['e1'], obj.param['e2'], obj.param['size'], obj.param['bfrac'], obj.param['detA'], obj.param['g1'], obj.param['g2'])) + self.chip_output.Log_error("obj id: %s" % (obj.param["id"])) + self.chip_output.Log_error( + " e1: %.5f\n e2: %.5f\n size: %f\n bfrac: %f\n detA: %f\n g1: %.5f\n g2: %.5f\n" + % ( + obj.param["e1"], + obj.param["e2"], + obj.param["size"], + obj.param["bfrac"], + obj.param["detA"], + obj.param["g1"], + obj.param["g2"], + ) + ) # Unload SED: obj.unload_SED() del obj @@ -248,7 +322,11 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): cat.free_mem() del cat - if chip.survey_type == "spectroscopic" and not self.overall_config["run_option"]["out_cat_only"] and chip.slsPSFOptim: + if ( + chip.survey_type == "spectroscopic" + and not self.overall_config["run_option"]["out_cat_only"] + and chip.slsPSFOptim + ): # from observation_sim.instruments.chip import chip_utils as chip_utils # gn = chip_utils.getChipSLSGratingID(chip.chipID)[0] # img1 = np.zeros([2,chip.img.array.shape[0],chip.img.array.shape[1]]) @@ -270,29 +348,40 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): del psf_model gc.collect() - self.chip_output.Log_info("Running checkpoint #1 (Object rendering finished): pointing-%d chip-%d pid-%d memory-%6.2fGB" % - (pointing.id, chip.chipID, os.getpid(), (psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024 / 1024))) + self.chip_output.Log_info( + "Running checkpoint #1 (Object rendering finished): pointing-%d chip-%d pid-%d memory-%6.2fGB" + % ( + pointing.id, + chip.chipID, + os.getpid(), + (psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024 / 1024), + ) + ) self.chip_output.Log_info( - "# objects that are too bright %d out of %d" % (bright_obj, nobj)) + "# objects that are too bright %d out of %d" % (bright_obj, nobj) + ) self.chip_output.Log_info( - "# objects that are too dim %d out of %d" % (dim_obj, nobj)) + "# objects that are too dim %d out of %d" % (dim_obj, nobj) + ) self.chip_output.Log_info( - "# objects that are missed %d out of %d" % (missed_obj, nobj)) + "# objects that are missed %d out of %d" % (missed_obj, nobj) + ) # Apply flat fielding (with shutter effects) flat_normal = np.ones_like(chip.img.array) if obs_param["flat_fielding"] is True: - flat_normal = flat_normal * chip.flat_img.array / \ - np.mean(chip.flat_img.array) + flat_normal = flat_normal * chip.flat_img.array / np.mean(chip.flat_img.array) if obs_param["shutter_effect"] is True: flat_normal = flat_normal * chip.shutter_img - flat_normal = np.array(flat_normal, dtype='float32') - self.updateHeaderInfo(header_flag='ext', keys=[ - 'SHTSTAT'], values=[True]) + flat_normal = np.array(flat_normal, dtype="float32") + self.updateHeaderInfo(header_flag="ext", keys=["SHTSTAT"], values=[True]) else: - self.updateHeaderInfo(header_flag='ext', keys=['SHTSTAT', 'SHTOPEN1', 'SHTCLOS0'], values=[ - True, self.h_ext['SHTCLOS1'], self.h_ext['SHTOPEN0']]) + self.updateHeaderInfo( + header_flag="ext", + keys=["SHTSTAT", "SHTOPEN1", "SHTCLOS0"], + values=[True, self.h_ext["SHTCLOS1"], self.h_ext["SHTOPEN0"]], + ) chip.img *= flat_normal del flat_normal @@ -302,15 +391,26 @@ def add_objects(self, chip, filt, tel, pointing, catalog, obs_param): t_obs = Time(datetime_obs) # ccd刷新2s,等待0.s，开始曝光 - t_obs_renew = Time(t_obs.mjd - (2.+0.) / 86400., format="mjd") - - t_obs_utc = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_obs_renew.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - self.updateHeaderInfo(header_flag='prim', keys=[ - 'DATE-OBS'], values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]]) + t_obs_renew = Time(t_obs.mjd - (2.0 + 0.0) / 86400.0, format="mjd") + + t_obs_utc = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_obs_renew.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + self.updateHeaderInfo( + header_flag="prim", + keys=["DATE-OBS"], + values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]], + ) # dark time : 曝光时间+刷新后等带时间0.s+关快门后读出前等待0.s - self.updateHeaderInfo(header_flag='ext', keys=[ - 'DARKTIME'], values=[0.+0.+pointing.exp_time]) + self.updateHeaderInfo( + header_flag="ext", keys=["DARKTIME"], values=[0.0 + 0.0 + pointing.exp_time] + ) return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/add_pattern_noise.py b/observation_sim/sim_steps/add_pattern_noise.py index 5fd4adab4a78613d2fbe614c255da8f28b4c298a..926bfcd86d6d62ccbce25571c2c1fb199629f48e 100644 --- a/observation_sim/sim_steps/add_pattern_noise.py +++ b/observation_sim/sim_steps/add_pattern_noise.py @@ -6,8 +6,10 @@ from observation_sim.instruments.chip import effects def apply_PRNU(self, chip, filt, tel, pointing, catalog, obs_param): chip.img *= chip.prnu_img if self.overall_config["output_setting"]["prnu_output"] is True: - chip.prnu_img.write("%s/FlatImg_PRNU_%s.fits" % - (self.chip_output.subdir, str(chip.chipID).rjust(2, '0'))) + chip.prnu_img.write( + "%s/FlatImg_PRNU_%s.fits" + % (self.chip_output.subdir, str(chip.chipID).rjust(2, "0")) + ) return chip, filt, tel, pointing @@ -20,66 +22,77 @@ def add_poisson_and_dark(self, chip, filt, tel, pointing, catalog, obs_param): exptime = pointing.exp_time if obs_param["add_dark"] is True: - chip.img, _ = chip_utils.add_poisson(img=chip.img, - chip=chip, - exptime=exptime, - poisson_noise=chip.poisson_noise, - InputDark=None) + if obs_param["input_dark"]: + self.chip_output.Log_info(" Applying input_dark") + chip.img, _ = chip_utils.add_poisson( + img=chip.img, + chip=chip, + exptime=exptime, + poisson_noise=chip.poisson_noise, + InputDark=obs_param["input_dark"], + ) else: - chip.img, _ = chip_utils.add_poisson(img=chip.img, - chip=chip, - exptime=exptime, - poisson_noise=chip.poisson_noise, - dark_noise=0.) + chip.img, _ = chip_utils.add_poisson( + img=chip.img, + chip=chip, + exptime=exptime, + poisson_noise=chip.poisson_noise, + dark_noise=0.0, + ) return chip, filt, tel, pointing def add_detector_defects(self, chip, filt, tel, pointing, catalog, obs_param): # Add Hot Pixels or/and Dead Pixels rgbadpix = Generator( - PCG64(int(self.overall_config["random_seeds"]["seed_defective"]+chip.chipID))) - badfraction = chip.badfraction*(rgbadpix.random()*0.5+0.7) # 5E-5*(rgbadpix.random()*0.5+0.7) + PCG64(int(self.overall_config["random_seeds"]["seed_defective"] + chip.chipID)) + ) + badfraction = chip.badfraction * ( + rgbadpix.random() * 0.5 + 0.7 + ) # 5E-5*(rgbadpix.random()*0.5+0.7) chip.img = effects.DefectivePixels( chip.img, IfHotPix=obs_param["hot_pixels"], IfDeadPix=obs_param["dead_pixels"], fraction=badfraction, - seed=self.overall_config["random_seeds"]["seed_defective"]+chip.chipID, biaslevel=0) + seed=self.overall_config["random_seeds"]["seed_defective"] + chip.chipID, + biaslevel=0, + ) # Apply Bad columns if obs_param["bad_columns"] is True: - chip.img = effects.BadColumns(chip.img, - seed=self.overall_config["random_seeds"]["seed_badcolumns"], - chipid=chip.chipID) + chip.img = effects.BadColumns( + chip.img, + seed=self.overall_config["random_seeds"]["seed_badcolumns"], + chipid=chip.chipID, + ) return chip, filt, tel, pointing def add_nonlinearity(self, chip, filt, tel, pointing, catalog, obs_param): self.chip_output.Log_info(" Applying Non-Linearity on the chip image") - chip.img = effects.NonLinearity(GSImage=chip.img, - beta1=5.e-7, - beta2=0) + chip.img = effects.NonLinearity(GSImage=chip.img, beta1=5.0e-7, beta2=0) return chip, filt, tel, pointing def add_blooming(self, chip, filt, tel, pointing, catalog, obs_param): self.chip_output.Log_info(" Applying CCD Saturation & Blooming") - chip.img = effects.SaturBloom(GSImage=chip.img, - nsect_x=1, - nsect_y=1, - fullwell=int(chip.full_well)) + chip.img = effects.SaturBloom( + GSImage=chip.img, nsect_x=1, nsect_y=1, fullwell=int(chip.full_well) + ) return chip, filt, tel, pointing def add_bias(self, chip, filt, tel, pointing, catalog, obs_param): - self.chip_output.Log_info( - " Adding Bias level and 16-channel non-uniformity") + self.chip_output.Log_info(" Adding Bias level and 16-channel non-uniformity") if obs_param["bias_16channel"] is True: - chip.img = effects.AddBiasNonUniform16(chip.img, - bias_level=float( - chip.bias_level), - nsecy=chip.nsecy, - nsecx=chip.nsecx, - seed=self.overall_config["random_seeds"]["seed_biasNonUniform"]+chip.chipID) + chip.img = effects.AddBiasNonUniform16( + chip.img, + bias_level=float(chip.bias_level), + nsecy=chip.nsecy, + nsecx=chip.nsecx, + seed=self.overall_config["random_seeds"]["seed_biasNonUniform"] + + chip.chipID, + ) elif obs_param["bias_16channel"] is False: chip.img += chip.bias_level return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/add_sky_background.py b/observation_sim/sim_steps/add_sky_background.py index 082e3aa50f3b87f74b20b5f8640538e70834c119..8a18f9b11dacc987effab900fb74ee83389e98f4 100644 --- a/observation_sim/sim_steps/add_sky_background.py +++ b/observation_sim/sim_steps/add_sky_background.py @@ -17,16 +17,17 @@ def add_sky_background_sci(self, chip, filt, tel, pointing, catalog, obs_param): flat_normal = np.ones_like(chip.img.array) if obs_param["flat_fielding"] is True: - flat_normal = flat_normal * chip.flat_img.array / \ - np.mean(chip.flat_img.array) + flat_normal = flat_normal * chip.flat_img.array / np.mean(chip.flat_img.array) if obs_param["shutter_effect"] is True: flat_normal = flat_normal * chip.shutter_img - flat_normal = np.array(flat_normal, dtype='float32') - self.updateHeaderInfo(header_flag='ext', keys=[ - 'SHTSTAT'], values=[True]) + flat_normal = np.array(flat_normal, dtype="float32") + self.updateHeaderInfo(header_flag="ext", keys=["SHTSTAT"], values=[True]) else: - self.updateHeaderInfo(header_flag='ext', keys=['SHTSTAT', 'SHTOPEN1', 'SHTCLOS0'], values=[ - True, self.h_ext['SHTCLOS1'], self.h_ext['SHTOPEN0']]) + self.updateHeaderInfo( + header_flag="ext", + keys=["SHTSTAT", "SHTOPEN1", "SHTCLOS0"], + values=[True, self.h_ext["SHTCLOS1"], self.h_ext["SHTOPEN0"]], + ) if obs_param["enable_straylight_model"]: # Filter.sky_background, Filter.zodical_spec will be updated @@ -34,11 +35,13 @@ def add_sky_background_sci(self, chip, filt, tel, pointing, catalog, obs_param): jtime=pointing.jdt, sat_pos=np.array([pointing.sat_x, pointing.sat_y, pointing.sat_z]), pointing_radec=np.array([pointing.ra, pointing.dec]), - sun_pos=np.array([pointing.sun_x, pointing.sun_y, pointing.sun_z])) + sun_pos=np.array([pointing.sun_x, pointing.sun_y, pointing.sun_z]), + ) + self.chip_output.Log_info("================================================") self.chip_output.Log_info( - "================================================") - self.chip_output.Log_info( - "sky background + stray light pixel flux value: %.5f" % (filt.sky_background)) + "sky background + stray light pixel flux value: %.5f" + % (filt.sky_background) + ) if chip.survey_type == "photometric": sky_map = filt.getSkyNoise(exptime=exptime) @@ -54,8 +57,9 @@ def add_sky_background_sci(self, chip, filt, tel, pointing, catalog, obs_param): pixelSize=chip.pix_scale, isAlongY=0, flat_cube=chip.flat_cube, - zoldial_spec=filt.zodical_spec) - sky_map = (sky_map + filt.sky_background)*exptime + zoldial_spec=filt.zodical_spec, + ) + sky_map = (sky_map + filt.sky_background) * exptime # sky_map = sky_map * tel.pupil_area * obs_param["exptime"] chip.img += sky_map @@ -64,7 +68,7 @@ def add_sky_background_sci(self, chip, filt, tel, pointing, catalog, obs_param): def add_sky_flat_calibration(self, chip, filt, tel, pointing, catalog, obs_param): - if not hasattr(self, 'h_ext'): + if not hasattr(self, "h_ext"): _, _ = self.prepare_headers(chip=chip, pointing=pointing) chip_wcs = galsim.FitsWCS(header=self.h_ext) @@ -78,39 +82,47 @@ def add_sky_flat_calibration(self, chip, filt, tel, pointing, catalog, obs_param filter_param = FilterParam() sky_level_filt = obs_param["flat_level_filt"] - norm_scaler = skyback_level/exptime / filter_param.param[sky_level_filt][5] + norm_scaler = skyback_level / exptime / filter_param.param[sky_level_filt][5] flat_normal = np.ones_like(chip.img.array) if obs_param["flat_fielding"] is True: - flat_normal = flat_normal * chip.flat_img.array / \ - np.mean(chip.flat_img.array) + flat_normal = flat_normal * chip.flat_img.array / np.mean(chip.flat_img.array) if obs_param["shutter_effect"] is True: flat_normal = flat_normal * chip.shutter_img - flat_normal = np.array(flat_normal, dtype='float32') + flat_normal = np.array(flat_normal, dtype="float32") # output 16-bit shutter effect image with pixel value <=65535 if self.overall_config["output_setting"]["shutter_output"] is True: - shutt_gsimg = galsim.ImageUS(chip.shutter_img*6E4) - shutt_gsimg.write("%s/ShutterEffect_%s_1.fits" % - (self.chip_output.subdir, str(chip.chipID).rjust(2, '0'))) - self.updateHeaderInfo(header_flag='ext', keys=[ - 'SHTSTAT'], values=[True]) + shutt_gsimg = galsim.ImageUS(chip.shutter_img * 6e4) + shutt_gsimg.write( + "%s/ShutterEffect_%s_1.fits" + % (self.chip_output.subdir, str(chip.chipID).rjust(2, "0")) + ) + self.updateHeaderInfo(header_flag="ext", keys=["SHTSTAT"], values=[True]) else: - self.updateHeaderInfo(header_flag='ext', keys=['SHTSTAT', 'SHTOPEN1', 'SHTCLOS0'], values=[ - True, self.h_ext['SHTCLOS1'], self.h_ext['SHTOPEN0']]) + self.updateHeaderInfo( + header_flag="ext", + keys=["SHTSTAT", "SHTOPEN1", "SHTCLOS0"], + values=[True, self.h_ext["SHTCLOS1"], self.h_ext["SHTOPEN0"]], + ) if chip.survey_type == "photometric": - sky_map = flat_normal * np.ones_like(chip.img.array) * norm_scaler * \ - filter_param.param[chip.filter_type][5] / tel.pupil_area * exptime + sky_map = ( + flat_normal + * np.ones_like(chip.img.array) + * norm_scaler + * filter_param.param[chip.filter_type][5] + / tel.pupil_area + * exptime + ) elif chip.survey_type == "spectroscopic": # flat_normal = np.ones_like(chip.img.array) if obs_param["flat_fielding"] is True: - - flat_normal = flat_normal * chip.flat_img.array / \ - np.mean(chip.flat_img.array) + flat_normal = ( + flat_normal * chip.flat_img.array / np.mean(chip.flat_img.array) + ) if obs_param["shutter_effect"] is True: - flat_normal = flat_normal * chip.shutter_img - flat_normal = np.array(flat_normal, dtype='float32') + flat_normal = np.array(flat_normal, dtype="float32") sky_map = calculateSkyMap_split_g( skyMap=flat_normal, blueLimit=filt.blue_limit, @@ -118,7 +130,8 @@ def add_sky_flat_calibration(self, chip, filt, tel, pointing, catalog, obs_param conf=chip.sls_conf, pixelSize=chip.pix_scale, isAlongY=0, - flat_cube=chip.flat_cube) + flat_cube=chip.flat_cube, + ) sky_map = sky_map * norm_scaler * exptime chip.img += sky_map @@ -126,20 +139,26 @@ def add_sky_flat_calibration(self, chip, filt, tel, pointing, catalog, obs_param def add_sky_background(self, chip, filt, tel, pointing, catalog, obs_param): - if not hasattr(self, 'h_ext'): + if not hasattr(self, "h_ext"): _, _ = self.prepare_headers(chip=chip, pointing=pointing) chip_wcs = galsim.FitsWCS(header=self.h_ext) if "flat_level" not in obs_param or "flat_level_filt" not in obs_param: chip, filt, tel, pointing = self.add_sky_background_sci( - chip, filt, tel, pointing, catalog, obs_param) + chip, filt, tel, pointing, catalog, obs_param + ) else: - if obs_param.get('flat_level') is None or obs_param.get('flat_level_filt') is None: + if ( + obs_param.get("flat_level") is None + or obs_param.get("flat_level_filt") is None + ): chip, filt, tel, pointing = self.add_sky_background_sci( - chip, filt, tel, pointing, catalog, obs_param) + chip, filt, tel, pointing, catalog, obs_param + ) else: chip, filt, tel, pointing = self.add_sky_flat_calibration( - chip, filt, tel, pointing, catalog, obs_param) + chip, filt, tel, pointing, catalog, obs_param + ) # renew header info datetime_obs = datetime.utcfromtimestamp(pointing.timestamp) @@ -147,15 +166,28 @@ def add_sky_background(self, chip, filt, tel, pointing, catalog, obs_param): t_obs = Time(datetime_obs) # ccd刷新2s,等待0.5s，开始曝光 - t_obs_renew = Time(t_obs.mjd - (2.+0.5) / 86400., format="mjd") - - t_obs_utc = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_obs_renew.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - self.updateHeaderInfo(header_flag='prim', keys=[ - 'DATE-OBS'], values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]]) + t_obs_renew = Time(t_obs.mjd - (2.0 + 0.5) / 86400.0, format="mjd") + + t_obs_utc = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_obs_renew.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + self.updateHeaderInfo( + header_flag="prim", + keys=["DATE-OBS"], + values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]], + ) # dark time : 曝光时间+刷新后等带时间0.5s+关闭快门时间1.5s+管快门后读出前等待0.5s - self.updateHeaderInfo(header_flag='ext', keys=['DARKTIME'], values=[ - 0.+0.0+0.0+pointing.exp_time]) + self.updateHeaderInfo( + header_flag="ext", + keys=["DARKTIME"], + values=[0.0 + 0.0 + 0.0 + pointing.exp_time], + ) return chip, filt, tel, pointing diff --git a/observation_sim/sim_steps/prepare_headers.py b/observation_sim/sim_steps/prepare_headers.py index 1fca53d907bcad124d1e5240c8a48f1f92c79ffa..6e7af72419839451b3a75839165c0d1d5e224312 100644 --- a/observation_sim/sim_steps/prepare_headers.py +++ b/observation_sim/sim_steps/prepare_headers.py @@ -18,9 +18,10 @@ def prepare_headers(self, chip, pointing): pa=(pointing.img_pa.deg + 180) % 360 - 180, project_cycle=self.overall_config["project_cycle"], run_counter=self.overall_config["run_counter"], - chip_name=str(chip.chipID).rjust(2, '0'), + chip_name=str(chip.chipID).rjust(2, "0"), obstype=pointing.pointing_type, - dataset=pointing.dataset) + dataset=pointing.dataset, + ) self.h_ext = generateExtensionHeader( chip=chip, xlen=chip.npix_x, @@ -41,15 +42,16 @@ def prepare_headers(self, chip, pointing): exptime=pointing.exp_time, readoutTime=chip.readout_time, t_shutter_open=pointing.t_shutter_open, - t_shutter_close=pointing.t_shutter_close) + t_shutter_close=pointing.t_shutter_close, + ) return self.h_prim, self.h_ext -def updateHeaderInfo(self, header_flag='prim', keys=['key'], values=[0]): - if header_flag == 'prim': +def updateHeaderInfo(self, header_flag="prim", keys=["key"], values=[0]): + if header_flag == "prim": for key, value in zip(keys, values): self.h_prim[key] = value - if header_flag == 'ext': + if header_flag == "ext": for key, value in zip(keys, values): self.h_ext[key] = value diff --git a/observation_sim/sim_steps/readout_output.py b/observation_sim/sim_steps/readout_output.py index db527de183037b8bf73f042784003fb326622934..2456ed88909158aba318d94325afc1b8e0fa7f5c 100644 --- a/observation_sim/sim_steps/readout_output.py +++ b/observation_sim/sim_steps/readout_output.py @@ -11,50 +11,326 @@ from datetime import datetime, timezone def add_prescan_overscan(self, chip, filt, tel, pointing, catalog, obs_param): self.chip_output.Log_info("Apply pre/over-scan") - chip.img = chip_utils.AddPreScan(GSImage=chip.img, - pre1=chip.prescan_x, - pre2=chip.prescan_y, - over1=chip.overscan_x, - over2=chip.overscan_y) + chip.img = chip_utils.AddPreScan( + GSImage=chip.img, + pre1=chip.prescan_x, + pre2=chip.prescan_y, + over1=chip.overscan_x, + over2=chip.overscan_y, + ) if obs_param["add_dark"] is True: - ny = int(chip.npix_y/2) - base_dark = (ny-1)*(chip.readout_time/ny)*chip.dark_noise - chip.img.array[(chip.prescan_y+ny):-(chip.prescan_y+ny), :] += base_dark + ny = int(chip.npix_y / 2) + base_dark = (ny - 1) * (chip.readout_time / ny) * chip.dark_noise + chip.img.array[(chip.prescan_y + ny) : -(chip.prescan_y + ny), :] += base_dark return chip, filt, tel, pointing def add_crosstalk(self, chip, filt, tel, pointing, catalog, obs_param): crosstalk = np.zeros([16, 16]) - crosstalk[0, :] = np.array([1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[1, :] = np.array([1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[2, :] = np.array([0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[3, :] = np.array([0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[4, :] = np.array([0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[5, :] = np.array([0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[6, :] = np.array([0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[7, :] = np.array([0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[8, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0.]) - crosstalk[9, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0.]) - crosstalk[10, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0.]) - crosstalk[11, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0.]) - crosstalk[12, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0.]) - crosstalk[13, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0.]) - crosstalk[14, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4]) - crosstalk[15, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1.]) + crosstalk[0, :] = np.array([ + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[1, :] = np.array([ + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[2, :] = np.array([ + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[3, :] = np.array([ + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[4, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[5, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[6, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[7, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[8, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[9, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[10, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[11, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[12, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + ]) + crosstalk[13, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + ]) + crosstalk[14, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + ]) + crosstalk[15, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + ]) # 2*8 -> 1*16 img = chip_utils.formatOutput(chip.img) ny, nx = img.array.shape nsecy = 1 nsecx = 16 - dy = int(ny/nsecy) - dx = int(nx/nsecx) + dy = int(ny / nsecy) + dx = int(nx / nsecx) newimg = galsim.Image(nx, ny, init_value=0) for i in range(16): for j in range(16): - newimg.array[:, int(i*dx):int(i*dx+dx)] += crosstalk[i, j]*img.array[:, int(j*dx):int(j*dx+dx)] + newimg.array[:, int(i * dx) : int(i * dx + dx)] += ( + crosstalk[i, j] * img.array[:, int(j * dx) : int(j * dx + dx)] + ) # 1*16 -> 2*8 newimg = chip_utils.formatRevert(newimg) @@ -64,11 +340,13 @@ def add_crosstalk(self, chip, filt, tel, pointing, catalog, obs_param): def add_readout_noise(self, chip, filt, tel, pointing, catalog, obs_param): - seed = int(self.overall_config["random_seeds"] - ["seed_readout"]) + pointing.id*30 + chip.chipID + seed = ( + int(self.overall_config["random_seeds"]["seed_readout"]) + + pointing.id * 30 + + chip.chipID + ) rng_readout = galsim.BaseDeviate(seed) - readout_noise = galsim.GaussianNoise( - rng=rng_readout, sigma=chip.read_noise) + readout_noise = galsim.GaussianNoise(rng=rng_readout, sigma=chip.read_noise) chip.img.addNoise(readout_noise) return chip, filt, tel, pointing @@ -76,42 +354,84 @@ def add_readout_noise(self, chip, filt, tel, pointing, catalog, obs_param): def apply_gain(self, chip, filt, tel, pointing, catalog, obs_param): self.chip_output.Log_info(" Applying Gain") if obs_param["gain_16channel"] is True: - chip.img, chip.gain_channel = effects.ApplyGainNonUniform16(chip.img, - gain=chip.gain, - nsecy=chip.nsecy, - nsecx=chip.nsecx, - seed=self.overall_config["random_seeds"]["seed_gainNonUniform"]+chip.chipID) + chip.img, chip.gain_channel = effects.ApplyGainNonUniform16( + chip.img, + gain=chip.gain, + nsecy=chip.nsecy, + nsecx=chip.nsecx, + seed=self.overall_config["random_seeds"]["seed_gainNonUniform"] + + chip.chipID, + ) elif obs_param["gain_16channel"] is False: chip.img /= chip.gain - chip.gain_channel = np.ones(chip.nsecy*chip.nsecx)*chip.gain + chip.gain_channel = np.ones(chip.nsecy * chip.nsecx) * chip.gain return chip, filt, tel, pointing def quantization_and_output(self, chip, filt, tel, pointing, catalog, obs_param): - if not hasattr(self, 'h_ext'): + if not hasattr(self, "h_ext"): _, _ = self.prepare_headers(chip=chip, pointing=pointing) - self.updateHeaderInfo(header_flag='ext', keys=['SHTSTAT', 'SHTOPEN1', 'SHTCLOS0', 'SHTCLOS1', 'EXPTIME'], values=[ - False, self.h_ext['SHTOPEN0'], self.h_ext['SHTOPEN0'], self.h_ext['SHTOPEN0'], 0.0]) + self.updateHeaderInfo( + header_flag="ext", + keys=["SHTSTAT", "SHTOPEN1", "SHTCLOS0", "SHTCLOS1", "EXPTIME"], + values=[ + False, + self.h_ext["SHTOPEN0"], + self.h_ext["SHTOPEN0"], + self.h_ext["SHTOPEN0"], + 0.0, + ], + ) # renew header info datetime_obs = datetime.utcfromtimestamp(pointing.timestamp) datetime_obs = datetime_obs.replace(tzinfo=timezone.utc) t_obs = Time(datetime_obs) # ccd刷新2s,等待0.5s，开灯后等待0.5s,开始曝光 - t_obs_renew = Time(t_obs.mjd - 2. / 86400., format="mjd") + t_obs_renew = Time(t_obs.mjd - 2.0 / 86400.0, format="mjd") - t_obs_utc = datetime.utcfromtimestamp(np.round(datetime.utcfromtimestamp( - t_obs_renew.unix).replace(tzinfo=timezone.utc).timestamp(), 1)) - self.updateHeaderInfo(header_flag='prim', keys=[ - 'DATE-OBS'], values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]]) + t_obs_utc = datetime.utcfromtimestamp( + np.round( + datetime + .utcfromtimestamp(t_obs_renew.unix) + .replace(tzinfo=timezone.utc) + .timestamp(), + 1, + ) + ) + self.updateHeaderInfo( + header_flag="prim", + keys=["DATE-OBS"], + values=[t_obs_utc.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-5]], + ) gains1 = list(chip.gain_channel[0:8]) gains2 = list(chip.gain_channel[8:]) gains2.reverse() gains = np.append(gains1, gains2) - self.updateHeaderInfo(header_flag='ext', keys=['GAIN01', 'GAIN02', 'GAIN03', 'GAIN04', 'GAIN05', 'GAIN06', 'GAIN07', - 'GAIN08', 'GAIN09', 'GAIN10', 'GAIN11', 'GAIN12', 'GAIN13', 'GAIN14', 'GAIN15', 'GAIN16'], values=gains) + self.updateHeaderInfo( + header_flag="ext", + keys=[ + "GAIN01", + "GAIN02", + "GAIN03", + "GAIN04", + "GAIN05", + "GAIN06", + "GAIN07", + "GAIN08", + "GAIN09", + "GAIN10", + "GAIN11", + "GAIN12", + "GAIN13", + "GAIN14", + "GAIN15", + "GAIN16", + ], + values=gains, + ) if obs_param["format_output"] is True: self.chip_output.Log_info(" Apply 1*16 format") @@ -123,8 +443,7 @@ def quantization_and_output(self, chip, filt, tel, pointing, catalog, obs_param) chip.img.replaceNegative(replace_value=0) chip.img.quantize() chip.img = galsim.Image(chip.img.array, dtype=np.uint16) - fname = os.path.join(self.chip_output.subdir, - self.h_prim['FILENAME'] + '.fits') + fname = os.path.join(self.chip_output.subdir, self.h_prim["FILENAME"] + ".fits") # f_name_size = 68 # if (len(self.h_prim['FILENAME']) > f_name_size): @@ -133,16 +452,19 @@ def quantization_and_output(self, chip, filt, tel, pointing, catalog, obs_param) hdu1 = fits.PrimaryHDU(header=self.h_prim) - self.updateHeaderInfo(header_flag='ext', keys=['DATASECT'], values=[ - str(chip.img.array.shape[1]) + 'x' + str(chip.img.array.shape[0])]) + self.updateHeaderInfo( + header_flag="ext", + keys=["DATASECT"], + values=[str(chip.img.array.shape[1]) + "x" + str(chip.img.array.shape[0])], + ) hdu2 = fits.ImageHDU(chip.img.array, header=self.h_ext) hdu2.header.comments["XTENSION"] = "image extension" hdu = fits.HDUList([hdu1, hdu2]) - hdu[0].add_datasum(when='data unit checksum') - hdu[0].add_checksum(when='HDU checksum', override_datasum=True) - hdu[1].add_datasum(when='data unit checksum') - hdu[1].add_checksum(when='HDU checksum', override_datasum=True) - hdu.writeto(fname, output_verify='ignore', overwrite=True) + hdu[0].add_datasum(when="data unit checksum") + hdu[0].add_checksum(when="HDU checksum", override_datasum=True) + hdu[1].add_datasum(when="data unit checksum") + hdu[1].add_checksum(when="HDU checksum", override_datasum=True) + hdu.writeto(fname, output_verify="ignore", overwrite=True) return chip, filt, tel, pointing diff --git a/observation_sim/sky_background/SkybackgroundMap.py b/observation_sim/sky_background/SkybackgroundMap.py index 1532784be4382ceb057c086b587d60fdbd46547e..b5ded60c7bfc4b0ffd5a249448452095aef9b65a 100644 --- a/observation_sim/sky_background/SkybackgroundMap.py +++ b/observation_sim/sky_background/SkybackgroundMap.py @@ -6,10 +6,8 @@ import numpy as np from astropy.table import Table from scipy import interpolate -import galsim import astropy.constants as cons -import os import time @@ -22,8 +20,19 @@ except ImportError: # calculate sky map by sky SED -def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='sky_emiss_hubble_50_50_A.dat', conf=[''], pixelSize=0.074, isAlongY=0, - split_pos=3685, flat_cube=None, zoldial_spec=None): + +def calculateSkyMap_split_g( + skyMap=None, + blueLimit=4200, + redLimit=6500, + skyfn="sky_emiss_hubble_50_50_A.dat", + conf=[""], + pixelSize=0.074, + isAlongY=0, + split_pos=3685, + flat_cube=None, + zoldial_spec=None, +): # skyMap = np.ones([yLen, xLen], dtype='float32') # # if isAlongY == 1: @@ -45,26 +54,30 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s fImg = galsim.Image(fimg) try: - with pkg_resources.files('observation_sim.sky_background.data.sky').joinpath(skyfn) as data_path: + with pkg_resources.files("observation_sim.sky_background.data.sky").joinpath( + skyfn + ) as data_path: skySpec = np.loadtxt(data_path) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.data.sky', skyfn) as data_path: + with pkg_resources.path( + "observation_sim.sky_background.data.sky", skyfn + ) as data_path: skySpec = np.loadtxt(data_path) # skySpec = np.loadtxt(skyfn) - spec = Table(np.array([skySpec[:, 0], skySpec[:, 1]] - ).T, names=('WAVELENGTH', 'FLUX')) + spec = Table( + np.array([skySpec[:, 0], skySpec[:, 1]]).T, names=("WAVELENGTH", "FLUX") + ) if zoldial_spec is not None: deltL = 0.5 lamb = np.arange(2000, 11000, deltL) - speci = interpolate.interp1d( - zoldial_spec['WAVELENGTH'], zoldial_spec['FLUX']) + speci = interpolate.interp1d(zoldial_spec["WAVELENGTH"], zoldial_spec["FLUX"]) y = speci(lamb) # erg/s/cm2/A --> photo/s/m2/A s_flux = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - spec = Table(np.array([lamb, s_flux]).T, names=('WAVELENGTH', 'FLUX')) + spec = Table(np.array([lamb, s_flux]).T, names=("WAVELENGTH", "FLUX")) if isAlongY == 0: directParm = 0 if isAlongY == 1: @@ -96,22 +109,30 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s+sub_y_e)/2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 sub_x_e = sub_x_end_arr[j] skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf2, - flat_cube=flat_cube, ignoreBeam=['D', 'E']) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf2, + flat_cube=flat_cube, + ignoreBeam=["D", "E"], + ) spec_orders = sdp.compute_spec_orders() @@ -146,7 +167,6 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s # fImg[bounds] = fImg[bounds] + ssImg[bounds] else: - # skyImg1 = galsim.Image(skyImg.array[:, 0:split_pos]) # origin1 = [0, 0] # skyImg2 = galsim.Image(skyImg.array[:, split_pos:]) @@ -167,7 +187,7 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s sub_y_end_arr = sub_y_start_arr + delt_y sub_y_end_arr[-1] = min(sub_y_end_arr[-1], y_len) - delt_x = split_pos-0 + delt_x = split_pos - 0 sub_x_start_arr = np.arange(0, split_pos, delt_x) sub_x_end_arr = sub_x_start_arr + delt_x sub_x_end_arr[-1] = min(sub_x_end_arr[-1], split_pos) @@ -176,7 +196,7 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s+sub_y_e)/2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 @@ -184,15 +204,22 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s # print(i,j,sub_y_s, sub_y_e,sub_x_s,sub_x_e) T1 = time.time() skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf1, - flat_cube=flat_cube) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf1, + flat_cube=flat_cube, + ) spec_orders = sdp.compute_spec_orders() @@ -209,9 +236,9 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s T2 = time.time() - print('time: %s ms' % ((T2 - T1)*1000)) + print("time: %s ms" % ((T2 - T1) * 1000)) - delt_x = x_len-split_pos + delt_x = x_len - split_pos sub_x_start_arr = np.arange(split_pos, x_len, delt_x) sub_x_end_arr = sub_x_start_arr + delt_x sub_x_end_arr[-1] = min(sub_x_end_arr[-1], x_len) @@ -220,7 +247,7 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s sub_y_s = k1 sub_y_e = sub_y_end_arr[i] - sub_y_center = (sub_y_s + sub_y_e) / 2. + sub_y_center = (sub_y_s + sub_y_e) / 2.0 for j, k2 in enumerate(sub_x_start_arr): sub_x_s = k2 @@ -230,15 +257,22 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s T1 = time.time() skyImg_sub = galsim.Image( - skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e]) + skyImg.array[sub_y_s:sub_y_e, sub_x_s:sub_x_e] + ) origin_sub = [sub_y_s, sub_x_s] - sub_x_center = (sub_x_s + sub_x_e) / 2. - - sdp = SpecDisperser(orig_img=skyImg_sub, xcenter=sub_x_center, ycenter=sub_y_center, origin=origin_sub, - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf2, - flat_cube=flat_cube) + sub_x_center = (sub_x_s + sub_x_e) / 2.0 + + sdp = SpecDisperser( + orig_img=skyImg_sub, + xcenter=sub_x_center, + ycenter=sub_y_center, + origin=origin_sub, + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf2, + flat_cube=flat_cube, + ) spec_orders = sdp.compute_spec_orders() @@ -254,11 +288,10 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s fImg[bounds] = fImg[bounds] + ssImg[bounds] T2 = time.time() - print('time: %s ms' % ((T2 - T1)*1000)) + print("time: %s ms" % ((T2 - T1) * 1000)) if isAlongY == 1: - fimg, tmx, tmy = rotate90( - array_orig=fImg.array, xc=0, yc=0, isClockwise=0) + fimg, tmx, tmy = rotate90(array_orig=fImg.array, xc=0, yc=0, isClockwise=0) else: fimg = fImg.array @@ -267,12 +300,20 @@ def calculateSkyMap_split_g(skyMap=None, blueLimit=4200, redLimit=6500, skyfn='s return fimg -def calculateSkyMap(xLen=9232, yLen=9126, blueLimit=4200, redLimit=6500, - skyfn='sky_emiss_hubble_50_50_A.dat', conf='', pixelSize=0.074, isAlongY=0): - skyMap = np.ones([yLen, xLen], dtype='float32') +def calculateSkyMap( + xLen=9232, + yLen=9126, + blueLimit=4200, + redLimit=6500, + skyfn="sky_emiss_hubble_50_50_A.dat", + conf="", + pixelSize=0.074, + isAlongY=0, +): + skyMap = np.ones([yLen, xLen], dtype="float32") if isAlongY == 1: - skyMap = np.ones([xLen, yLen], dtype='float32') + skyMap = np.ones([xLen, yLen], dtype="float32") skyImg = galsim.Image(skyMap) @@ -282,20 +323,31 @@ def calculateSkyMap(xLen=9232, yLen=9126, blueLimit=4200, redLimit=6500, fimg = np.zeros_like(skyMap) fImg = galsim.Image(fimg) try: - with pkg_resources.files('observation_sim.sky_background.data.sky').joinpath(skyfn) as data_path: + with pkg_resources.files("observation_sim.sky_background.data.sky").joinpath( + skyfn + ) as data_path: skySpec = np.loadtxt(data_path) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.data.sky', skyfn) as data_path: + with pkg_resources.path( + "observation_sim.sky_background.data.sky", skyfn + ) as data_path: skySpec = np.loadtxt(data_path) # skySpec = np.loadtxt(skyfn) - spec = Table(np.array([skySpec[:, 0], skySpec[:, 1]] - ).T, names=('WAVELENGTH', 'FLUX')) - - sdp = SpecDisperser(orig_img=skyImg, xcenter=skyImg.center.x, ycenter=skyImg.center.y, origin=[1, 1], - tar_spec=spec, - band_start=tbstart, band_end=tbend, - conf=conf) + spec = Table( + np.array([skySpec[:, 0], skySpec[:, 1]]).T, names=("WAVELENGTH", "FLUX") + ) + + sdp = SpecDisperser( + orig_img=skyImg, + xcenter=skyImg.center.x, + ycenter=skyImg.center.y, + origin=[1, 1], + tar_spec=spec, + band_start=tbstart, + band_end=tbend, + conf=conf, + ) spec_orders = sdp.compute_spec_orders() @@ -309,8 +361,7 @@ def calculateSkyMap(xLen=9232, yLen=9126, blueLimit=4200, redLimit=6500, fImg[bounds] = fImg[bounds] + ssImg[bounds] if isAlongY == 1: - fimg, tmx, tmy = rotate90( - array_orig=fImg.array, xc=0, yc=0, isClockwise=0) + fimg, tmx, tmy = rotate90(array_orig=fImg.array, xc=0, yc=0, isClockwise=0) else: fimg = fImg.array diff --git a/observation_sim/sky_background/Straylight.py b/observation_sim/sky_background/Straylight.py index d461e79424707b7fea424f0ca24627fce03f7a14..fd89955f965697fbe91a2625e60839a70d918641 100644 --- a/observation_sim/sky_background/Straylight.py +++ b/observation_sim/sky_background/Straylight.py @@ -15,18 +15,49 @@ except ImportError: # Try backported to PY<37 'importlib_resources' import importlib_resources as pkg_resources -filterPivotWave = {'nuv': 2875.5, 'u': 3629.6, 'g': 4808.4, - 'r': 6178.2, 'i': 7609.0, 'z': 9012.9, 'y': 9627.9} -filterIndex = {'nuv': 0, 'u': 1, 'g': 2, 'r': 3, 'i': 4, 'z': 5, 'y': 6} -filterCCD = {'nuv': 'UV0', 'u': 'UV0', 'g': 'Astro_MB', - 'r': 'Astro_MB', 'i': 'Basic_NIR', 'z': 'Basic_NIR', 'y': 'Basic_NIR'} -bandRange = {'nuv': [2504.0, 3230.0], 'u': [3190.0, 4039.0], 'g': [3989.0, 5498.0], 'r': [5438.0, 6956.0], 'i': [ - 6886.0, 8469.0], 'z': [8379.0, 10855.0], 'y': [9217.0, 10900.0], 'GU': [2550, 4000], 'GV': [4000, 6200], 'GI': [6200, 10000]} +filterPivotWave = { + "nuv": 2875.5, + "u": 3629.6, + "g": 4808.4, + "r": 6178.2, + "i": 7609.0, + "z": 9012.9, + "y": 9627.9, +} +filterIndex = {"nuv": 0, "u": 1, "g": 2, "r": 3, "i": 4, "z": 5, "y": 6} +filterCCD = { + "nuv": "UV0", + "u": "UV0", + "g": "Astro_MB", + "r": "Astro_MB", + "i": "Basic_NIR", + "z": "Basic_NIR", + "y": "Basic_NIR", +} +bandRange = { + "nuv": [2504.0, 3230.0], + "u": [3190.0, 4039.0], + "g": [3989.0, 5498.0], + "r": [5438.0, 6956.0], + "i": [6886.0, 8469.0], + "z": [8379.0, 10855.0], + "y": [9217.0, 10900.0], + "GU": [2550, 4000], + "GV": [4000, 6200], + "GI": [6200, 10000], +} # Instrument_dir = '/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_C6/straylight/straylight/Instrument/' -SpecOrder = ['-2', '-1', '0', '1', '2'] +SpecOrder = ["-2", "-1", "0", "1", "2"] -filterMirrorEff = {'nuv': 0.54, 'u': 0.68, 'g': 0.8, - 'r': 0.8, 'i': 0.8, 'z': 0.8, 'y': 0.8} +filterMirrorEff = { + "nuv": 0.54, + "u": 0.68, + "g": 0.8, + "r": 0.8, + "i": 0.8, + "z": 0.8, + "y": 0.8, +} def transRaDec2D(ra, dec): @@ -41,45 +72,52 @@ def getAngle132(x1=0, y1=0, z1=0, x2=0, y2=0, z2=0, x3=0, y3=0, z3=0): cosValue = 0 angle = 0 - x11 = x1-x3 - y11 = y1-y3 - z11 = z1-z3 + x11 = x1 - x3 + y11 = y1 - y3 + z11 = z1 - z3 - x22 = x2-x3 - y22 = y2-y3 - z22 = z2-z3 + x22 = x2 - x3 + y22 = y2 - y3 + z22 = z2 - z3 - tt = np.sqrt((x11*x11 + y11*y11 + z11 * z11) - * (x22*x22 + y22*y22 + z22*z22)) - if (tt == 0): + tt = np.sqrt( + (x11 * x11 + y11 * y11 + z11 * z11) * (x22 * x22 + y22 * y22 + z22 * z22) + ) + if tt == 0: return 0 - cosValue = (x11*x22+y11*y22+z11*z22)/tt + cosValue = (x11 * x22 + y11 * y22 + z11 * z22) / tt - if (cosValue > 1): + if cosValue > 1: cosValue = 1 - if (cosValue < -1): + if cosValue < -1: cosValue = -1 angle = math.acos(cosValue) return angle * 360 / (2 * math.pi) -def calculateAnglePwithEarth(sat=np.array([0, 0, 0]), pointing=np.array([0, 0, 0]), sun=np.array([0, 0, 0])): - modSat = np.sqrt(sat[0]*sat[0] + sat[1]*sat[1]+sat[2]*sat[2]) - modPoint = np.sqrt(pointing[0]*pointing[0] + - pointing[1]*pointing[1] + pointing[2]*pointing[2]) +def calculateAnglePwithEarth( + sat=np.array([0, 0, 0]), pointing=np.array([0, 0, 0]), sun=np.array([0, 0, 0]) +): + modSat = np.sqrt(sat[0] * sat[0] + sat[1] * sat[1] + sat[2] * sat[2]) + modPoint = np.sqrt( + pointing[0] * pointing[0] + + pointing[1] * pointing[1] + + pointing[2] * pointing[2] + ) withLocalZenithAngle = ( - pointing[0] * sat[0] + pointing[1] * sat[1] + pointing[2] * sat[2]) / (modPoint*modSat) + pointing[0] * sat[0] + pointing[1] * sat[1] + pointing[2] * sat[2] + ) / (modPoint * modSat) innerM_sat_sun = sat[0] * sun[0] + sat[1] * sun[1] + sat[2] * sun[2] - cosAngle = innerM_sat_sun / (modSat * cons.au.value/1000) + cosAngle = innerM_sat_sun / (modSat * cons.au.value / 1000) isInSunSide = 1 - if (cosAngle < -0.3385737): # cos109.79 + if cosAngle < -0.3385737: # cos109.79 isInSunSide = -1 elif cosAngle >= -0.3385737 and cosAngle <= 0.3385737: isInSunSide = 0 - return math.acos(withLocalZenithAngle)*180/math.pi, isInSunSide + return math.acos(withLocalZenithAngle) * 180 / math.pi, isInSunSide # /** @@ -87,132 +125,201 @@ def calculateAnglePwithEarth(sat=np.array([0, 0, 0]), pointing=np.array([0, 0, 0 # */ def Cartesian2Equatorial(carCoor=np.array([0, 0, 0])): eCoor = np.zeros(2) - if (carCoor[0] > 0 and carCoor[1] >= 0): + if carCoor[0] > 0 and carCoor[1] >= 0: eCoor[0] = math.atan(carCoor[1] / carCoor[0]) * 360 / (2 * math.pi) - elif (carCoor[0] < 0): - eCoor[0] = (math.atan(carCoor[1] / carCoor[0]) + - math.pi) * 360 / (2 * math.pi) - elif (carCoor[0] > 0 and carCoor[1] < 0): - eCoor[0] = (math.atan(carCoor[1] / carCoor[0]) + - 2 * math.pi) * 360 / (2 * math.pi) - elif (carCoor[0] == 0 and carCoor[1] < 0): + elif carCoor[0] < 0: + eCoor[0] = (math.atan(carCoor[1] / carCoor[0]) + math.pi) * 360 / (2 * math.pi) + elif carCoor[0] > 0 and carCoor[1] < 0: + eCoor[0] = ( + (math.atan(carCoor[1] / carCoor[0]) + 2 * math.pi) * 360 / (2 * math.pi) + ) + elif carCoor[0] == 0 and carCoor[1] < 0: eCoor[0] = 270 - elif (carCoor[0] == 0 and carCoor[1] > 0): + elif carCoor[0] == 0 and carCoor[1] > 0: eCoor[0] = 90 - eCoor[1] = math.atan(carCoor[2] / np.sqrt(carCoor[0] * - carCoor[0] + carCoor[1] * carCoor[1])) * 360 / (2 * math.pi) + eCoor[1] = ( + math.atan( + carCoor[2] / np.sqrt(carCoor[0] * carCoor[0] + carCoor[1] * carCoor[1]) + ) + * 360 + / (2 * math.pi) + ) return eCoor class Straylight(object): - def __init__(self, jtime=2460843., sat_pos=np.array([0, 0, 0]), pointing_radec=np.array([0, 0]), sun_pos=np.array([0, 0, 0])): + def __init__( + self, + jtime=2460843.0, + sat_pos=np.array([0, 0, 0]), + pointing_radec=np.array([0, 0]), + sun_pos=np.array([0, 0, 0]), + ): self.jtime = jtime self.sat = sat_pos self.sun_pos = sun_pos self.equator = coord.SkyCoord( - pointing_radec[0]*u.degree, pointing_radec[1]*u.degree, frame='icrs') - self.ecliptic = self.equator.transform_to('barycentrictrueecliptic') + pointing_radec[0] * u.degree, pointing_radec[1] * u.degree, frame="icrs" + ) + self.ecliptic = self.equator.transform_to("barycentrictrueecliptic") self.pointing = transRaDec2D(pointing_radec[0], pointing_radec[1]) platForm = sys.platform - if platForm == 'darwin': + if platForm == "darwin": try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('libstraylight.dylib') as dllFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "libstraylight.dylib" + ) as dllFn: self.slcdll = ctypes.CDLL(dllFn) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'libstraylight.dylib') as dllFn: + with pkg_resources.path( + "observation_sim.sky_background.lib", "libstraylight.dylib" + ) as dllFn: self.slcdll = ctypes.CDLL(dllFn) - elif platForm == 'linux': + elif platForm == "linux": try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('libstraylight.so') as dllFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "libstraylight.so" + ) as dllFn: self.slcdll = ctypes.CDLL(dllFn) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'libstraylight.so') as dllFn: + with pkg_resources.path( + "observation_sim.sky_background.lib", "libstraylight.so" + ) as dllFn: self.slcdll = ctypes.CDLL(dllFn) # self.slcdll=ctypes.CDLL('./libstraylight.dylib') - self.slcdll.Calculate.argtypes = [ctypes.c_double, ctypes.POINTER(ctypes.c_double), - ctypes.POINTER(ctypes.c_double), ctypes.POINTER( - ctypes.c_double), - ctypes.POINTER(ctypes.c_double), ctypes.c_char_p] - - self.slcdll.PointSource.argtypes = [ctypes.c_double, ctypes.POINTER(ctypes.c_double), - ctypes.POINTER(ctypes.c_double), ctypes.POINTER( - ctypes.c_double), - ctypes.POINTER(ctypes.c_double), ctypes.c_char_p] - - self.slcdll.EarthShine.argtypes = [ctypes.c_double, ctypes.POINTER(ctypes.c_double), - ctypes.POINTER(ctypes.c_double), ctypes.POINTER( - ctypes.c_double), - ctypes.POINTER(ctypes.c_double)] - - self.slcdll.Zodiacal.argtypes = [ctypes.c_double, ctypes.POINTER(ctypes.c_double), - ctypes.POINTER(ctypes.c_double)] - self.slcdll.ComposeY.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), - ctypes.POINTER(ctypes.c_double)] + self.slcdll.Calculate.argtypes = [ + ctypes.c_double, + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.c_char_p, + ] + + self.slcdll.PointSource.argtypes = [ + ctypes.c_double, + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.c_char_p, + ] + + self.slcdll.EarthShine.argtypes = [ + ctypes.c_double, + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ] + + self.slcdll.Zodiacal.argtypes = [ + ctypes.c_double, + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ] + self.slcdll.ComposeY.argtypes = [ + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ctypes.POINTER(ctypes.c_double), + ] self.slcdll.Init.argtypes = [ - ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p] + ctypes.c_char_p, + ctypes.c_char_p, + ctypes.c_char_p, + ctypes.c_char_p, + ] try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('DE405') as tFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "DE405" + ) as tFn: self.deFn = tFn.as_uri()[7:] except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'DE405') as tFn: + with pkg_resources.path( + "observation_sim.sky_background.lib", "DE405" + ) as tFn: self.deFn = tFn.as_uri()[7:] try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('PST') as tFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "PST" + ) as tFn: self.PSTFn = tFn.as_uri()[7:] except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'PST') as tFn: + with pkg_resources.path("observation_sim.sky_background.lib", "PST") as tFn: self.PSTFn = tFn.as_uri()[7:] try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('R') as tFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "R" + ) as tFn: self.RFn = tFn.as_uri()[7:] except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'R') as tFn: + with pkg_resources.path("observation_sim.sky_background.lib", "R") as tFn: self.RFn = tFn.as_uri()[7:] try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('Zodiacal') as tFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "Zodiacal" + ) as tFn: self.ZolFn = tFn.as_uri()[7:] except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'Zodiacal') as tFn: + with pkg_resources.path( + "observation_sim.sky_background.lib", "Zodiacal" + ) as tFn: self.ZolFn = tFn.as_uri()[7:] try: - with pkg_resources.files('observation_sim.sky_background.lib').joinpath('BrightGaia_with_csst_mag') as tFn: + with pkg_resources.files("observation_sim.sky_background.lib").joinpath( + "BrightGaia_with_csst_mag" + ) as tFn: self.brightStarTabFn = tFn.as_uri()[7:] except AttributeError: - with pkg_resources.path('observation_sim.sky_background.lib', 'BrightGaia_with_csst_mag') as tFn: + with pkg_resources.path( + "observation_sim.sky_background.lib", "BrightGaia_with_csst_mag" + ) as tFn: self.brightStarTabFn = tFn.as_uri()[7:] print(self.deFn) - self.slcdll.Init(str.encode(self.deFn), str.encode( - self.PSTFn), str.encode(self.RFn), str.encode(self.ZolFn)) - - def getFilterAndCCD_Q(self, filter='i'): + self.slcdll.Init( + str.encode(self.deFn), + str.encode(self.PSTFn), + str.encode(self.RFn), + str.encode(self.ZolFn), + ) + + def getFilterAndCCD_Q(self, filter="i"): try: - with pkg_resources.files('observation_sim.instruments.data.ccd').joinpath(filterCCD[filter] + '.txt') as ccd_fn: + with pkg_resources.files("observation_sim.instruments.data.ccd").joinpath( + filterCCD[filter] + ".txt" + ) as ccd_fn: q_ccd_f = np.loadtxt(ccd_fn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.ccd', filterCCD[filter] + '.txt') as ccd_fn: + with pkg_resources.path( + "observation_sim.instruments.data.ccd", filterCCD[filter] + ".txt" + ) as ccd_fn: q_ccd_f = np.loadtxt(ccd_fn) try: - with pkg_resources.files('observation_sim.instruments.data.filters').joinpath(filter + '.txt') as filter_fn: + with pkg_resources.files( + "observation_sim.instruments.data.filters" + ).joinpath(filter + ".txt") as filter_fn: q_fil_f = np.loadtxt(filter_fn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.filters', filter + '.txt') as filter_fn: + with pkg_resources.path( + "observation_sim.instruments.data.filters", filter + ".txt" + ) as filter_fn: q_fil_f = np.loadtxt(filter_fn) band_s = 2000 band_e = 11000 - q_ccd_f[:, 0] = q_ccd_f[:, 0]*10 - q_ccd = np.zeros([q_ccd_f.shape[0]+2, q_ccd_f.shape[1]]) + q_ccd_f[:, 0] = q_ccd_f[:, 0] * 10 + q_ccd = np.zeros([q_ccd_f.shape[0] + 2, q_ccd_f.shape[1]]) q_ccd[1:-1, :] = q_ccd_f q_ccd[0] = [band_s, 0] q_ccd[-1] = [band_e, 0] - q_fil = np.zeros([q_fil_f.shape[0]+2, q_fil_f.shape[1]]) + q_fil = np.zeros([q_fil_f.shape[0] + 2, q_fil_f.shape[1]]) q_fil[1:-1, :] = q_fil_f q_fil[0] = [band_s, 0] q_fil[-1] = [band_e, 0] @@ -220,23 +327,25 @@ class Straylight(object): q_fil_i = interpolate.interp1d(q_fil[:, 0], q_fil[:, 1]) q_ccd_i = interpolate.interp1d(q_ccd[:, 0], q_ccd[:, 1]) bands = np.arange(bandRange[filter][0], bandRange[filter][1], 0.5) - q_ccd_fil = q_fil_i(bands)*q_ccd_i(bands) + q_ccd_fil = q_fil_i(bands) * q_ccd_i(bands) - return np.trapz(q_ccd_fil, bands)/(bandRange[filter][1]-bandRange[filter][0]) + return np.trapz(q_ccd_fil, bands) / ( + bandRange[filter][1] - bandRange[filter][0] + ) - def calculateEarthShineFilter(self, filter='i', pixel_size_phy=10): - sat = (ctypes.c_double*3)() + def calculateEarthShineFilter(self, filter="i", pixel_size_phy=10): + sat = (ctypes.c_double * 3)() sat[:] = self.sat - ob = (ctypes.c_double*3)() + ob = (ctypes.c_double * 3)() ob[:] = self.pointing - py1 = (ctypes.c_double*3)() - py2 = (ctypes.c_double*3)() + py1 = (ctypes.c_double * 3)() + py2 = (ctypes.c_double * 3)() self.slcdll.ComposeY(ob, py1, py2) - earth_e1 = (ctypes.c_double*7)() + earth_e1 = (ctypes.c_double * 7)() self.slcdll.EarthShine(self.jtime, sat, ob, py1, earth_e1) - earth_e2 = (ctypes.c_double*7)() + earth_e2 = (ctypes.c_double * 7)() self.slcdll.EarthShine(self.jtime, sat, ob, py2, earth_e2) band_earth_e1 = earth_e1[:][filterIndex[filter]] @@ -246,10 +355,24 @@ class Straylight(object): p_lambda = filterPivotWave[filter] c = cons.c.value h = cons.h.value - pix_earth_e1 = band_earth_e1 / \ - (h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q - pix_earth_e2 = band_earth_e2 / \ - (h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q + pix_earth_e1 = ( + band_earth_e1 + / (h * c / (p_lambda * 1e-10)) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + * q + ) + pix_earth_e2 = ( + band_earth_e2 + / (h * c / (p_lambda * 1e-10)) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + * q + ) if pix_earth_e1 < pix_earth_e2: return pix_earth_e1, py1[:] @@ -260,18 +383,18 @@ class Straylight(object): calculate zodiacal call c++ program, seems to have some problem """ - def calculateZodiacalFilter1(self, filter='i', pixel_size_phy=10): - sat = (ctypes.c_double*3)() + def calculateZodiacalFilter1(self, filter="i", pixel_size_phy=10): + sat = (ctypes.c_double * 3)() sat[:] = self.sat - ob = (ctypes.c_double*3)() + ob = (ctypes.c_double * 3)() ob[:] = self.pointing - zodical_e = (ctypes.c_double*7)() + zodical_e = (ctypes.c_double * 7)() self.slcdll.Zodiacal(self.jtime, ob, zodical_e) - ob1 = (ctypes.c_double*2)() + ob1 = (ctypes.c_double * 2)() ob1[:] = np.array([self.ecliptic.lon.value, self.ecliptic.lat.value]) - zodical_e1 = (ctypes.c_double*7)() + zodical_e1 = (ctypes.c_double * 7)() self.slcdll.Zodiacal1(ob1, zodical_e1) band_zodical_e = zodical_e[:][filterIndex[filter]] @@ -280,8 +403,15 @@ class Straylight(object): p_lambda = filterPivotWave[filter] c = cons.c.value h = cons.h.value - pix_zodical_e = band_zodical_e / \ - (h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q + pix_zodical_e = ( + band_zodical_e + / (h * c / (p_lambda * 1e-10)) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + * q + ) return pix_zodical_e, band_zodical_e @@ -289,23 +419,33 @@ class Straylight(object): calculate zodiacal use python """ - def calculateZodiacalFilter2(self, filter='i', aper=2, pixelsize=0.074, sun_pos=np.array([0, 0, 0])): + def calculateZodiacalFilter2( + self, filter="i", aper=2, pixelsize=0.074, sun_pos=np.array([0, 0, 0]) + ): spec, v_mag = self.calculateZodicalSpec( - longitude=self.ecliptic.lon.value, latitude=self.ecliptic.lat.value, sun_pos=sun_pos) + longitude=self.ecliptic.lon.value, + latitude=self.ecliptic.lat.value, + sun_pos=sun_pos, + ) # spec = self.calculateZodicalSpec(longitude = lon, latitude = lat) try: - with pkg_resources.files('observation_sim.instruments.data.throughputs').joinpath(filter + '_throughput.txt') as throughputFn: + with pkg_resources.files( + "observation_sim.instruments.data.throughputs" + ).joinpath(filter + "_throughput.txt") as throughputFn: throughput = np.loadtxt(throughputFn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.throughputs', filter + '_throughput.txt') as throughputFn: + with pkg_resources.path( + "observation_sim.instruments.data.throughputs", + filter + "_throughput.txt", + ) as throughputFn: throughput = np.loadtxt(throughputFn) deltL = 0.5 lamb = np.arange(bandRange[filter][0], bandRange[filter][1], deltL) - speci = interpolate.interp1d(spec['WAVELENGTH'], spec['FLUX']) + speci = interpolate.interp1d(spec["WAVELENGTH"], spec["FLUX"]) y = speci(lamb) # erg/s/cm2/A --> photo/s/m2/A @@ -315,20 +455,29 @@ class Straylight(object): throughput_ = throughput_i(lamb) - sky_pix = np.trapz(flux*throughput_, lamb) * \ - math.pi * aper*aper/4 * pixelsize * pixelsize + sky_pix = ( + np.trapz(flux * throughput_, lamb) + * math.pi + * aper + * aper + / 4 + * pixelsize + * pixelsize + ) # sky_pix_e = np.trapz(y, lamb) * math.pi * aper*aper/4 * pixelsize * pixelsize/(10*10*1e-6*1e-6)*1e-7*1e4 return sky_pix, v_mag # , sky_pix_e - def calculateStarLightFilter(self, filter='i', pointYaxis=np.array([1, 1, 1]), pixel_size_phy=10): - sat = (ctypes.c_double*3)() + def calculateStarLightFilter( + self, filter="i", pointYaxis=np.array([1, 1, 1]), pixel_size_phy=10 + ): + sat = (ctypes.c_double * 3)() sat[:] = self.sat - ob = (ctypes.c_double*3)() + ob = (ctypes.c_double * 3)() ob[:] = self.pointing - py = (ctypes.c_double*3)() + py = (ctypes.c_double * 3)() py[:] = pointYaxis q = self.getFilterAndCCD_Q(filter=filter) @@ -336,30 +485,40 @@ class Straylight(object): c = cons.c.value h = cons.h.value - star_e1 = (ctypes.c_double*7)() - self.slcdll.PointSource(self.jtime, sat, ob, py, - star_e1, str.encode(self.brightStarTabFn)) + star_e1 = (ctypes.c_double * 7)() + self.slcdll.PointSource( + self.jtime, sat, ob, py, star_e1, str.encode(self.brightStarTabFn) + ) band_star_e1 = star_e1[:][filterIndex[filter]] - pix_star_e1 = band_star_e1 / \ - (h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q + pix_star_e1 = ( + band_star_e1 + / (h * c / (p_lambda * 1e-10)) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + * q + ) return pix_star_e1 - def calculateEarthshineGrating(self, grating='GU', pixel_size_phy=10, normFilter='g', aper=2, pixelsize=0.074): - sat = (ctypes.c_double*3)() + def calculateEarthshineGrating( + self, grating="GU", pixel_size_phy=10, normFilter="g", aper=2, pixelsize=0.074 + ): + sat = (ctypes.c_double * 3)() sat[:] = self.sat - ob = (ctypes.c_double*3)() + ob = (ctypes.c_double * 3)() ob[:] = self.pointing - py1 = (ctypes.c_double*3)() - py2 = (ctypes.c_double*3)() + py1 = (ctypes.c_double * 3)() + py2 = (ctypes.c_double * 3)() self.slcdll.ComposeY(ob, py1, py2) - earth_e1 = (ctypes.c_double*7)() + earth_e1 = (ctypes.c_double * 7)() self.slcdll.EarthShine(self.jtime, sat, ob, py1, earth_e1) - earth_e2 = (ctypes.c_double*7)() + earth_e2 = (ctypes.c_double * 7)() self.slcdll.EarthShine(self.jtime, sat, ob, py2, earth_e2) # zodical_e = (ctypes.c_double*7)() # self.slcdll.Zodiacal(self.jtime,ob,zodical_e) @@ -381,48 +540,67 @@ class Straylight(object): p_lambda = filterPivotWave[normFilter] c = cons.c.value h = cons.h.value - pix_earth_e = band_earth_e / \ - (h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q + pix_earth_e = ( + band_earth_e + / (h * c / (p_lambda * 1e-10)) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + * q + ) # pix_earth_e2 = band_earth_e2/(h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q # pix_zodical_e = band_zodical_e/(h*c/(p_lambda*1e-10))*pixel_size_phy*1e-6*pixel_size_phy*1e-6*q # pix_earth_e = np.min([pix_earth_e1, pix_earth_e2]) earthshine_v, earthshine_spec = self.calculatEarthshineByHSTSpec( - filter=normFilter, aper=aper, pixelsize=pixelsize) + filter=normFilter, aper=aper, pixelsize=pixelsize + ) - lamb_earth = earthshine_spec['WAVELENGTH'] - flux_earth = earthshine_spec['FLUX']*pix_earth_e/earthshine_v + lamb_earth = earthshine_spec["WAVELENGTH"] + flux_earth = earthshine_spec["FLUX"] * pix_earth_e / earthshine_v # print(pix_earth_e,earthshine_v) earth_v_grating = 0 for s_order in SpecOrder: try: - with pkg_resources.files('observation_sim.instruments.data.sls_conf').joinpath( - grating + '.Throughput.' + s_order + 'st.fits') as thpFn: + with pkg_resources.files( + "observation_sim.instruments.data.sls_conf" + ).joinpath(grating + ".Throughput." + s_order + "st.fits") as thpFn: thp_ = Table.read(thpFn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.sls_conf', - grating + '.Throughput.' + s_order + 'st.fits') as thpFn: + with pkg_resources.path( + "observation_sim.instruments.data.sls_conf", + grating + ".Throughput." + s_order + "st.fits", + ) as thpFn: thp_ = Table.read(thpFn) - thpFn_i = interpolate.interp1d( - thp_['WAVELENGTH'], thp_['SENSITIVITY']) + thpFn_i = interpolate.interp1d(thp_["WAVELENGTH"], thp_["SENSITIVITY"]) thp = thpFn_i(lamb_earth) - beamsEarth = np.trapz(flux_earth*thp, lamb_earth) * \ - math.pi*aper*aper/4 * pixelsize * pixelsize + beamsEarth = ( + np.trapz(flux_earth * thp, lamb_earth) + * math.pi + * aper + * aper + / 4 + * pixelsize + * pixelsize + ) earth_v_grating = earth_v_grating + beamsEarth # print(beamsEarth) # print(earthshine_v, pix_earth_e, earth_v_grating) return earth_v_grating, py - def calculateStarLightGrating(self, grating='GU', pointYaxis=np.array([1, 1, 1]), pixel_size_phy=10): - sat = (ctypes.c_double*3)() + def calculateStarLightGrating( + self, grating="GU", pointYaxis=np.array([1, 1, 1]), pixel_size_phy=10 + ): + sat = (ctypes.c_double * 3)() sat[:] = self.sat - ob = (ctypes.c_double*3)() + ob = (ctypes.c_double * 3)() ob[:] = self.pointing - py = (ctypes.c_double*3)() + py = (ctypes.c_double * 3)() py[:] = pointYaxis # q=self.getFilterAndCCD_Q(filter=filter) @@ -430,9 +608,10 @@ class Straylight(object): c = cons.c.value h = cons.h.value - star_e1 = (ctypes.c_double*7)() - self.slcdll.PointSource(self.jtime, sat, ob, py, - star_e1, str.encode(self.brightStarTabFn)) + star_e1 = (ctypes.c_double * 7)() + self.slcdll.PointSource( + self.jtime, sat, ob, py, star_e1, str.encode(self.brightStarTabFn) + ) filterPivotWaveList = np.zeros(7) bandRangeList = np.zeros(7) @@ -443,10 +622,15 @@ class Straylight(object): filterMirrorEffList[i] = filterMirrorEff[filterNameList[i]] brange = bandRange[filterNameList[i]] bandRangeList[i] = brange[1] - brange[0] - filterFlux_lamb = star_e1[:]/bandRangeList / \ - filterMirrorEffList/(h*c/(filterPivotWaveList*1e-10)) + filterFlux_lamb = ( + star_e1[:] + / bandRangeList + / filterMirrorEffList + / (h * c / (filterPivotWaveList * 1e-10)) + ) filterFlux_lambi = interpolate.interp1d( - filterPivotWaveList, filterFlux_lamb, fill_value="extrapolate") + filterPivotWaveList, filterFlux_lamb, fill_value="extrapolate" + ) lamb_g = np.arange(bandRange[grating][0], bandRange[grating][1], 1) flux_g = filterFlux_lambi(lamb_g) @@ -454,19 +638,26 @@ class Straylight(object): starLight_grating = 0 for s_order in SpecOrder: try: - with pkg_resources.files('observation_sim.instruments.data.sls_conf').joinpath( - grating + '.Throughput.' + s_order + 'st.fits') as thpFn: + with pkg_resources.files( + "observation_sim.instruments.data.sls_conf" + ).joinpath(grating + ".Throughput." + s_order + "st.fits") as thpFn: thp_ = Table.read(thpFn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.sls_conf', - grating + '.Throughput.' + s_order + 'st.fits') as thpFn: + with pkg_resources.path( + "observation_sim.instruments.data.sls_conf", + grating + ".Throughput." + s_order + "st.fits", + ) as thpFn: thp_ = Table.read(thpFn) - thpFn_i = interpolate.interp1d( - thp_['WAVELENGTH'], thp_['SENSITIVITY']) + thpFn_i = interpolate.interp1d(thp_["WAVELENGTH"], thp_["SENSITIVITY"]) thp = thpFn_i(lamb_g) - beamsZol = np.trapz(flux_g*thp, lamb_g) * \ - pixel_size_phy*1e-6*pixel_size_phy*1e-6 + beamsZol = ( + np.trapz(flux_g * thp, lamb_g) + * pixel_size_phy + * 1e-6 + * pixel_size_phy + * 1e-6 + ) starLight_grating = starLight_grating + beamsZol # print(beamsZol) @@ -476,21 +667,31 @@ class Straylight(object): return starLight_grating - def calculatEarthshineByHSTSpec(self, filter='g', aper=2, pixelsize=0.074, s=2000, e=11000): + def calculatEarthshineByHSTSpec( + self, filter="g", aper=2, pixelsize=0.074, s=2000, e=11000 + ): try: - with pkg_resources.files('observation_sim.sky_background.data.sky').joinpath('earthShine.dat') as specFn: + with pkg_resources.files( + "observation_sim.sky_background.data.sky" + ).joinpath("earthShine.dat") as specFn: spec = np.loadtxt(specFn) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.data.sky', - 'earthShine.dat') as specFn: + with pkg_resources.path( + "observation_sim.sky_background.data.sky", "earthShine.dat" + ) as specFn: spec = np.loadtxt(specFn) try: - with pkg_resources.files('observation_sim.instruments.data.throughputs').joinpath(filter + '_throughput.txt') as throughputFn: + with pkg_resources.files( + "observation_sim.instruments.data.throughputs" + ).joinpath(filter + "_throughput.txt") as throughputFn: throughput = np.loadtxt(throughputFn) except AttributeError: - with pkg_resources.path('observation_sim.instruments.data.throughputs', filter + '_throughput.txt') as throughputFn: + with pkg_resources.path( + "observation_sim.instruments.data.throughputs", + filter + "_throughput.txt", + ) as throughputFn: throughput = np.loadtxt(throughputFn) deltL = 0.5 @@ -505,8 +706,15 @@ class Straylight(object): throughput_ = throughput_i(lamb) - sky_pix = np.trapz(flux*throughput_, lamb) * \ - math.pi * aper*aper/4 * pixelsize * pixelsize + sky_pix = ( + np.trapz(flux * throughput_, lamb) + * math.pi + * aper + * aper + / 4 + * pixelsize + * pixelsize + ) lamb = np.arange(s, e, deltL) speci = interpolate.interp1d(spec[:, 0], spec[:, 1]) @@ -514,18 +722,22 @@ class Straylight(object): # erg/s/cm2/A --> photo/s/m2/A flux = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - return sky_pix, Table(np.array([lamb, flux]).T, names=('WAVELENGTH', 'FLUX')) + return sky_pix, Table(np.array([lamb, flux]).T, names=("WAVELENGTH", "FLUX")) - def calculateZodicalSpec(self, longitude=50, latitude=60, sun_pos=np.array([0, 0, 0])): - from scipy.interpolate import interp2d + def calculateZodicalSpec( + self, longitude=50, latitude=60, sun_pos=np.array([0, 0, 0]) + ): from scipy.interpolate import griddata try: - with pkg_resources.files('observation_sim.sky_background.data').joinpath('Zodiacal_map1.dat') as z_map_fn: + with pkg_resources.files("observation_sim.sky_background.data").joinpath( + "Zodiacal_map1.dat" + ) as z_map_fn: ZL = np.loadtxt(z_map_fn) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.data', - 'Zodiacal_map1.dat') as z_map_fn: + with pkg_resources.path( + "observation_sim.sky_background.data", "Zodiacal_map1.dat" + ) as z_map_fn: ZL = np.loadtxt(z_map_fn) # zl_sh = ZL.shape @@ -538,72 +750,109 @@ class Straylight(object): sun_radec = Cartesian2Equatorial(sun_pos) sun_eclip = coord.SkyCoord( - sun_radec[0]*u.degree, sun_radec[1]*u.degree, frame='icrs') - sun_equtor = sun_eclip.transform_to('barycentrictrueecliptic') + sun_radec[0] * u.degree, sun_radec[1] * u.degree, frame="icrs" + ) + sun_equtor = sun_eclip.transform_to("barycentrictrueecliptic") - longitude = longitude - (sun_equtor.lon*u.degree).value + longitude = longitude - (sun_equtor.lon * u.degree).value longitude = np.abs(longitude) # print((sun_equtor.lon*u.degree).value) - if (longitude > 180): + if longitude > 180: longitude = 360 - longitude latitude = np.abs(latitude) lo = longitude la = latitude - zl = griddata((X.flatten(), Y.flatten()), - ZL[1:, 1:].flatten(), (la, lo), method='cubic').min() - zl = zl*(math.pi*math.pi)/(180*180)/(3600*3600)*1e-4*1e7*1e-8*1e-4 + zl = griddata( + (X.flatten(), Y.flatten()), ZL[1:, 1:].flatten(), (la, lo), method="cubic" + ).min() + zl = ( + zl + * (math.pi * math.pi) + / (180 * 180) + / (3600 * 3600) + * 1e-4 + * 1e7 + * 1e-8 + * 1e-4 + ) # print(zl , '\n') try: - with pkg_resources.files('observation_sim.sky_background.data.sky').joinpath('zodiacal.dat') as zodical_fn: + with pkg_resources.files( + "observation_sim.sky_background.data.sky" + ).joinpath("zodiacal.dat") as zodical_fn: spec = np.loadtxt(zodical_fn) except AttributeError: - with pkg_resources.path('observation_sim.sky_background.data.sky', - 'zodiacal.dat') as zodical_fn: + with pkg_resources.path( + "observation_sim.sky_background.data.sky", "zodiacal.dat" + ) as zodical_fn: spec = np.loadtxt(zodical_fn) speci = interpolate.interp1d(spec[:, 0], spec[:, 1]) flux5000 = speci(5000) - f_ration = zl/flux5000 + f_ration = zl / flux5000 - v_mag = np.log10(f_ration)*(-2.5)+22.1 + v_mag = np.log10(f_ration) * (-2.5) + 22.1 # print("factor:", v_mag, lo, la) - return Table(np.array([spec[:, 0], spec[:, 1]*f_ration]).T, names=('WAVELENGTH', 'FLUX')), v_mag + return Table( + np.array([spec[:, 0], spec[:, 1] * f_ration]).T, + names=("WAVELENGTH", "FLUX"), + ), v_mag - def calculateStrayLightFilter(self, filter='i', pixel_size_phy=10, pixel_scale=0.074): + def calculateStrayLightFilter( + self, filter="i", pixel_size_phy=10, pixel_scale=0.074 + ): e1, py = self.calculateEarthShineFilter( - filter=filter, pixel_size_phy=pixel_size_phy) + filter=filter, pixel_size_phy=pixel_size_phy + ) e2, _ = self.calculateZodiacalFilter2( - filter=filter, sun_pos=self.sun_pos, pixelsize=pixel_scale) + filter=filter, sun_pos=self.sun_pos, pixelsize=pixel_scale + ) e3 = self.calculateStarLightFilter( - filter=filter, pointYaxis=py, pixel_size_phy=pixel_size_phy) + filter=filter, pointYaxis=py, pixel_size_phy=pixel_size_phy + ) - return e1+e2+e3 + return e1 + e2 + e3 - def calculateStrayLightGrating(self, grating='GI', pixel_size_phy=10, normFilter_es='g'): + def calculateStrayLightGrating( + self, grating="GI", pixel_size_phy=10, normFilter_es="g" + ): e1, py = self.calculateEarthshineGrating( - grating=grating, pixel_size_phy=pixel_size_phy, normFilter=normFilter_es) + grating=grating, pixel_size_phy=pixel_size_phy, normFilter=normFilter_es + ) e2 = self.calculateStarLightGrating(grating=grating, pointYaxis=py) spec, _ = self.calculateZodicalSpec( - longitude=self.ecliptic.lon.value, latitude=self.ecliptic.lat.value, sun_pos=self.sun_pos) - - return e1+e2, spec - - -def testZodiacal(lon=285.04312526255366, lat=30.): - c_eclip = coord.SkyCoord(lon*u.degree, lat*u.degree, - frame='barycentrictrueecliptic') - c_equtor = c_eclip.transform_to('icrs') - - sl = Straylight(jtime=2459767.00354975, sat=np.array([]), radec=np.array( - [(c_equtor.ra*u.degree).value, (c_equtor.dec*u.degree).value])) - e_zol, v_mag = sl.calculateZodiacalFilter2(filter='i', sun_pos=np.array( - [-3.70939436e+07, 1.35334903e+08, 5.86673104e+07])) + longitude=self.ecliptic.lon.value, + latitude=self.ecliptic.lat.value, + sun_pos=self.sun_pos, + ) + + return e1 + e2, spec + + +def testZodiacal(lon=285.04312526255366, lat=30.0): + c_eclip = coord.SkyCoord( + lon * u.degree, lat * u.degree, frame="barycentrictrueecliptic" + ) + c_equtor = c_eclip.transform_to("icrs") + + sl = Straylight( + jtime=2459767.00354975, + sat=np.array([]), + radec=np.array([ + (c_equtor.ra * u.degree).value, + (c_equtor.dec * u.degree).value, + ]), + ) + e_zol, v_mag = sl.calculateZodiacalFilter2( + filter="i", sun_pos=np.array([-3.70939436e07, 1.35334903e08, 5.86673104e07]) + ) print(e_zol) + # ju=2.4608437604166665e+06 # sat = (ctypes.c_double*3)() # sat[:] = np.array([5873.752, -1642.066, 2896.744]) diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000000000000000000000000000000000000..10b12013ec709f64ff11f4e25f22a7c28c8ae60b --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,84 @@ +[build-system] +requires = [ + "setuptools<82", + "wheel", + "Cython==3.0.6", + "numpy==1.26.4" +] +build-backend = "setuptools.build_meta" + +[project] +name = "csst-msc-sim" +version = "3.4.0" +description = "CSST Main Survey image simulator" +readme = "README.md" +requires-python = ">=3.11" +license = { text = "MIT" } +authors = [ + { name = "CSST simulation team" } +] +dynamic = ["dependencies"] + +# dependencies = [ +# "numpy==1.26.4", +# "scipy>=1.10", +# "astropy>=5", +# "PyYAML>=6", +# "galsim>=2.5", +# "sep>=1.2", +# "healpy>=1.16", +# "h5py>=3.11", +# "Cython==3.0.6", +# "numba>=0.59.1", +# "psutil>=5.9", +# "lmfit>=1.2", +# "mpi4py>=4.1", +# "toml>=0.10", +# "matplotlib" +#] + +[project.urls] +Homepage = "https://csst-tb.bao.ac.cn/code/csst-sims/csst_msc_sim" + +[project.optional-dependencies] +dev = [ + "pytest", + "pytest-cov", + "build", + "twine", + "ruff", + "black", + "mypy" +] +docs = [ + "sphinx", + "furo" +] + +[project.scripts] +csst-run-sim = "observation_sim.cli:main" + +[tool.setuptools] +include-package-data = true + +[tool.setuptools.dynamic] +dependencies = {file = ["requirements.txt"]} + +[tool.setuptools.packages.find] +include = ["observation_sim*"] + +[tool.pytest.ini_options] +testpaths = ["tests"] + +[tool.ruff] +preview = true +line-length = 88 +target-version = "py311" + +[tool.ruff.lint] +select = ["E", "F", "W"] # E/W = PEP8, F = pyflakes +ignore = ["E501", "E203"] # example: ignore line length if using black + +[tool.ruff.format] +quote-style = "double" +indent-style = "space" \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 6128cf4f1b8656da6e29ef0bd3de57c6268cadf8..465e0d29e380a5d1385745ac3d0652c0a4a4ee45 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,13 +1,15 @@ numpy==1.26.4 -astropy==6.0.1 -scipy==1.11.4 -GalSim==2.5.2 +astropy==7.2.0 +scipy==1.17.1 +GalSim==2.8.4 PyYAML==6.0.1 -sep==1.2.1 -healpy==1.16.6 -h5py==3.11.0 +sep==1.4.1 +healpy==1.19.0 +h5py==3.16.0 Cython==3.0.6 -numba==0.59.1 -psutil==5.9.8 +numba==0.65.1 +psutil==7.2.2 toml==0.10.2 -lmfit==1.2.2 \ No newline at end of file +lmfit==1.3.4 +matplotlib==3.10.9 +mpi4py==4.1.1 \ No newline at end of file diff --git a/run_sim.py b/run_sim.py index 416cb295bc8afefdaa3bd3527f2fe87041d0801d..757de47ce1cf288099b588c185c353b606a0cc7e 100755 --- a/run_sim.py +++ b/run_sim.py @@ -8,6 +8,7 @@ import datetime import importlib import gc + gc.enable() @@ -32,7 +33,7 @@ def run_sim(): args = parse_args() if args.config_dir is None: - args.config_dir = '' + args.config_dir = "" args.config_dir = os.path.abspath(args.config_dir) args.config_file = os.path.join(args.config_dir, args.config_file) with open(args.config_file, "r") as stream: @@ -46,11 +47,11 @@ def run_sim(): # Overwrite the data and working directories # if they are specified by the command line inputs if args.data_dir is not None: - config['data_dir'] = args.data_dir + config["data_dir"] = args.data_dir if args.work_dir is not None: - config['work_dir'] = args.work_dir + config["work_dir"] = args.work_dir - if not ("data_dir" in config): + if "data_dir" not in config: config["data_dir"] = None # Some default values @@ -72,14 +73,21 @@ def run_sim(): # appended to the front. # NOTE: the implementation of gerenating time_stamps is temporary. pointing_dir = None - if "pointing_dir" in config['obs_setting']: - pointing_dir = config['obs_setting']["pointing_dir"] + if "pointing_dir" in config["obs_setting"]: + pointing_dir = config["obs_setting"]["pointing_dir"] pointing_list = generate_pointing_list( - config=config, pointing_filename=config['obs_setting']['pointing_file'], data_dir=pointing_dir, dataset=config["data_set"]) + config=config, + pointing_filename=config["obs_setting"]["pointing_file"], + data_dir=pointing_dir, + dataset=config["data_set"], + ) # Make the main output directories run_dir = make_run_dirs( - work_dir=config['work_dir'], run_name=config['run_name'], pointing_list=pointing_list) + work_dir=config["work_dir"], + run_name=config["run_name"], + pointing_list=pointing_list, + ) # Copy the config file to output directory & Write Run metadata shutil.copy(args.config_file, run_dir) @@ -87,27 +95,34 @@ def run_sim(): with open(run_meta, "w") as config_out: config_out.write("\n") config_out.write("###############################################\n") - config_out.write( - "csst_msc_sim package version: \"%s\"\n" % __version__) - date_str = datetime.datetime.strftime(now, '%m/%d/%Y') - time_str = datetime.datetime.strftime(now, '%H:%M:%S') - config_out.write("Run_date: \"%s\"\n" % date_str) - config_out.write("Run_time: \"%s\"\n" % time_str) + config_out.write('csst_msc_sim package version: "%s"\n' % __version__) + date_str = datetime.datetime.strftime(now, "%m/%d/%Y") + time_str = datetime.datetime.strftime(now, "%H:%M:%S") + config_out.write('Run_date: "%s"\n' % date_str) + config_out.write('Run_time: "%s"\n' % time_str) config_out.write("###############################################\n") # Initialize the simulation if args.catalog is not None: - catalog_module = importlib.import_module('catalog.'+args.catalog) - obs = Observation(config=config, Catalog=catalog_module.Catalog, - work_dir=config['work_dir'], data_dir=config['data_dir']) + catalog_module = importlib.import_module("catalog." + args.catalog) + obs = Observation( + config=config, + Catalog=catalog_module.Catalog, + work_dir=config["work_dir"], + data_dir=config["data_dir"], + ) else: catalog_module = None - obs = Observation(config=config, Catalog=None, - work_dir=config['work_dir'], data_dir=config['data_dir']) + obs = Observation( + config=config, + Catalog=None, + work_dir=config["work_dir"], + data_dir=config["data_dir"], + ) # Run simulation obs.runExposure_MPI_PointingList(pointing_list=pointing_list) -if __name__ == '__main__': +if __name__ == "__main__": run_sim() diff --git a/setup.py b/setup.py index 0d5ef40342fed1550221f73a81127b680006c587..0227287151e16502d37e18a2677280221c926f92 100644 --- a/setup.py +++ b/setup.py @@ -1,20 +1,15 @@ -''' -Author: Zhang Xin zhangx@bao.ac.cn -Date: 2023-08-07 11:23:28 -LastEditors: Zhang Xin zhangx@bao.ac.cn -LastEditTime: 2023-10-08 14:44:19 -FilePath: /undefined/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/setup.py -Description: 这是默认设置,请设置`customMade`, 打开koroFileHeader查看配置 进行设置: https://github.com/OBKoro1/koro1FileHeader/wiki/%E9%85%8D%E7%BD%AE -''' -from setuptools import setup, find_packages - -from setuptools.extension import Extension -from setuptools.config import read_configuration -from distutils.command.build_ext import build_ext - +# ''' +# Author: Zhang Xin zhangx@bao.ac.cn +# Date: 2023-08-07 11:23:28 +# LastEditors: Zhang Xin zhangx@bao.ac.cn +# LastEditTime: 2023-10-08 14:44:19 +# FilePath: /undefined/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/setup.py +# Description: 这是默认设置,请设置`customMade`, 打开koroFileHeader查看配置 进行设置: https://github.com/OBKoro1/koro1FileHeader/wiki/%E9%85%8D%E7%BD%AE +# ''' +from setuptools import setup, Extension from Cython.Build import cythonize - -import numpy +import numpy as np +from distutils.command.build_ext import build_ext class CTypes(Extension): @@ -22,7 +17,6 @@ class CTypes(Extension): class build_ext(build_ext): - def build_extension(self, ext): self._ctypes = isinstance(ext, CTypes) return super().build_extension(ext) @@ -34,86 +28,59 @@ class build_ext(build_ext): def get_ext_filename(self, ext_name): if self._ctypes: - return ext_name + '.so' + return ext_name + ".so" return super().get_ext_filename(ext_name) extensions = [ - Extension("observation_sim.mock_objects.SpecDisperser.disperse_c.interp", ["observation_sim/mock_objects/SpecDisperser/disperse_c/interp.pyx"], - include_dirs=[numpy.get_include()], - libraries=["m"]), - - Extension("observation_sim.mock_objects.SpecDisperser.disperse_c.disperse", ["observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx"], - include_dirs=[numpy.get_include()], - libraries=["m"]), + Extension( + "observation_sim.mock_objects.SpecDisperser.disperse_c.interp", + ["observation_sim/mock_objects/SpecDisperser/disperse_c/interp.pyx"], + include_dirs=[np.get_include()], + libraries=["m"], + ), + Extension( + "observation_sim.mock_objects.SpecDisperser.disperse_c.disperse", + ["observation_sim/mock_objects/SpecDisperser/disperse_c/disperse.pyx"], + include_dirs=[np.get_include()], + libraries=["m"], + ), ] -df_module = [CTypes('observation_sim.instruments.chip.libBF.libmoduleBF', - ['observation_sim/instruments/chip/libBF/diffusion_X1.c', - 'observation_sim/instruments/chip/libBF/nrutil.c'], - include_dirs=[ - 'observation_sim/instruments/chip/libBF/', '/usr/include'] - )] -cti_module = [CTypes('observation_sim.instruments.chip.libCTI.libmoduleCTI', - ['observation_sim/instruments/chip/libCTI/src/add_CTI.c', 'observation_sim/instruments/chip/libCTI/src/nrutil.c', 'observation_sim/instruments/chip/libCTI/src/ran1.c', 'observation_sim/instruments/chip/libCTI/src/ran2.c', 'observation_sim/instruments/chip/libCTI/src/poidev.c', - 'observation_sim/instruments/chip/libCTI/src/gammln.c', 'observation_sim/instruments/chip/libCTI/src/gasdev.c', 'observation_sim/instruments/chip/libCTI/src/sort.c', 'observation_sim/instruments/chip/libCTI/src/creattraps.c'], - include_dirs=[ - 'observation_sim/instruments/chip/libCTI/src/', '/usr/include'] - )] - - -# setup( -# name = "slssim_disperse", -# ext_modules = cythonize(extensions), -# ) -# 读取依赖列表requirements.txt -# 忽略#开头或者版本号不明确指定的条目 -with open("requirements.txt", "r") as f: - requirements = [ - req.strip() - for req in f.readlines() - if not req.startswith("#") and req.__contains__("==") - ] - -setup(name='csst_msc_sim', - version='3.3.1', - packages=find_packages(), - # install_requires=[ - # # 'numpy>=1.18.5', - # # 'galsim>=2.2.4', - # # 'pyyaml>=5.3.1', - # # 'astropy>=4.0.1', - # # 'scipy>=1.5.0', - # # 'mpi4py>=3.0.3', - # # 'sep>=1.0.3', - # # 'healpy>=1.14.0', - # # 'h5py>=2.10.0', - # # 'Cython>=0.29.21', - # # 'numba>=0.50.1' - # ], - - package_data={ - 'observation_sim.astrometry.lib': ['libshao.so'], - 'observation_sim.instruments.chip.libBF': ['libmoduleBF.so'], - 'observation_sim.instruments.chip.libCTI': ['libmoduleCTI.so'], - 'observation_sim.mock_objects.data': ['*.dat'], - 'observation_sim.mock_objects.data.led': ['*.fits'], - 'observation_sim.instruments.data': ['*.txt', '*.dat', '*.json'], - 'observation_sim.instruments.data.field_distortion': ['*.pickle'], - 'observation_sim.instruments.data.ccd': ['*.txt', '*.json'], - 'observation_sim.instruments.data.filters': ['*.txt', '*.list', '*.dat'], - 'observation_sim.instruments.data.throughputs': ['*.txt', '*.dat'], - 'observation_sim.instruments.data.sls_conf': ['*.conf', '*.fits'], - # 'observation_sim.Instrument.data.flatCube': ['*.fits'], - 'catalog.data': ['*.fits', '*.so'], - 'observation_sim.config.header': ['*.fits', '*.lst'], - 'observation_sim.sky_background.data': ['*.dat'], - 'observation_sim.sky_background.data.sky': ['*.dat'], - 'observation_sim.sky_background.lib': ['*'], - }, - python_requires=">=3.11", # Python版本要求 - install_requires=requirements, +df_module = [ + CTypes( + "observation_sim.instruments.chip.libBF.libmoduleBF", + [ + "observation_sim/instruments/chip/libBF/diffusion_X1.c", + "observation_sim/instruments/chip/libBF/nrutil.c", + ], + include_dirs=["observation_sim/instruments/chip/libBF/", "/usr/include"], + ) +] +cti_module = [ + CTypes( + "observation_sim.instruments.chip.libCTI.libmoduleCTI", + [ + "observation_sim/instruments/chip/libCTI/src/add_CTI.c", + "observation_sim/instruments/chip/libCTI/src/nrutil.c", + "observation_sim/instruments/chip/libCTI/src/ran1.c", + "observation_sim/instruments/chip/libCTI/src/ran2.c", + "observation_sim/instruments/chip/libCTI/src/poidev.c", + "observation_sim/instruments/chip/libCTI/src/gammln.c", + "observation_sim/instruments/chip/libCTI/src/gasdev.c", + "observation_sim/instruments/chip/libCTI/src/sort.c", + "observation_sim/instruments/chip/libCTI/src/creattraps.c", + ], + include_dirs=["observation_sim/instruments/chip/libCTI/src/", "/usr/include"], + ) +] - ext_modules=cythonize(extensions) + df_module + cti_module, - cmdclass={'build_ext': build_ext} - ) +setup( + ext_modules=cythonize( + extensions, + language_level="3", + ) + + df_module + + cti_module, + cmdclass={"build_ext": build_ext}, +) diff --git a/test_run.sh b/test_run.sh index 436bfaa90c61cce627534e4923440b06fe19c46e..82c036e607aa2fa3574c3a9d008decabeaa628d7 100755 --- a/test_run.sh +++ b/test_run.sh @@ -2,8 +2,5 @@ date -python3 /public/home/fangyuedong/project/csst_msc_sim/run_sim.py \ - --config_file config_overall.yaml \ - -c /public/home/fangyuedong/project/csst_msc_sim/config \ - --catalog C9_Catalog +csst-run-sim --config_file config_overall.yaml -c /home/yuedong/Work/csst_test/csst_msc_sim/config --catalog C9_Catalog diff --git a/tests/test_BF_CTE.py b/tests/test_BF_CTE.py index a37bf39cc8d42e9734ea67e8799a92966cef23a8..e1faa04c6cf4e8b2e07e0ea3a4478e919e055a9d 100644 --- a/tests/test_BF_CTE.py +++ b/tests/test_BF_CTE.py @@ -1,9 +1,6 @@ import unittest -import sys import os -import math -from itertools import islice import numpy as np import copy import ctypes @@ -12,7 +9,6 @@ import yaml from astropy.io import fits from observation_sim.instruments import Chip, Filter, FilterParam, FocalPlane -from observation_sim.instruments.chip import chip_utils from observation_sim.instruments.chip.libCTI.CTI_modeling import CTI_sim try: @@ -26,13 +22,22 @@ except ImportError: def add_brighter_fatter(img): # Inital dynamic lib try: - with pkg_resources.files('observation_sim.instruments.chip.libBF').joinpath("libmoduleBF.so") as lib_path: + with pkg_resources.files("observation_sim.instruments.chip.libBF").joinpath( + "libmoduleBF.so" + ) as lib_path: lib_bf = ctypes.CDLL(lib_path) except AttributeError: - with pkg_resources.path('observation_sim.instruments.chip.libBF', "libmoduleBF.so") as lib_path: + with pkg_resources.path( + "observation_sim.instruments.chip.libBF", "libmoduleBF.so" + ) as lib_path: lib_bf = ctypes.CDLL(lib_path) - lib_bf.addEffects.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.POINTER( - ctypes.c_float), ctypes.POINTER(ctypes.c_float), ctypes.c_int] + lib_bf.addEffects.argtypes = [ + ctypes.c_int, + ctypes.c_int, + ctypes.POINTER(ctypes.c_float), + ctypes.POINTER(ctypes.c_float), + ctypes.c_int, + ] # Set bit flag bit_flag = 1 @@ -40,8 +45,8 @@ def add_brighter_fatter(img): nx, ny = img.array.shape nn = nx * ny - arr_ima = (ctypes.c_float*nn)() - arr_imc = (ctypes.c_float*nn)() + arr_ima = (ctypes.c_float * nn)() + arr_imc = (ctypes.c_float * nn)() arr_ima[:] = img.array.reshape(nn) arr_imc[:] = np.zeros(nn) @@ -50,6 +55,8 @@ def add_brighter_fatter(img): img.array[:, :] = np.reshape(arr_imc, [nx, ny]) del arr_ima, arr_imc return img + + # test FUNCTION --- END # @@ -66,7 +73,8 @@ def defineCCD(iccd, config_file): chip.img = galsim.ImageF(400, 200) focal_plane = FocalPlane(chip_list=[iccd]) chip.img.wcs = focal_plane.getTanWCS( - 192.8595, 27.1283, -113.4333*galsim.degrees, chip.pix_scale) + 192.8595, 27.1283, -113.4333 * galsim.degrees, chip.pix_scale + ) return chip @@ -77,21 +85,23 @@ def defineFilt(chip): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list return filt class detModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(detModule_coverage, self).__init__(methodName) # self.dataPath = "/public/home/chengliang/CSSOSDataProductsSims/csst-simulation/tests/UNIT_TEST_DATA" ##os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_fz_gc1') self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) self.iccd = 1 def test_add_brighter_fatter(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) filt = defineFilt(chip) print(chip.chipID) @@ -100,31 +110,31 @@ class detModule_coverage(unittest.TestCase): # objA-lowSFB obj = galsim.Gaussian(sigma=0.2, flux=1000) arr = obj.drawImage(nx=64, ny=64, scale=0.074).array - chip.img.array[(100-32):(100+32), (200-32):(200+32)] = arr[:, :] + chip.img.array[(100 - 32) : (100 + 32), (200 - 32) : (200 + 32)] = arr[:, :] img_old = copy.deepcopy(chip.img) img_new = add_brighter_fatter(img=chip.img) arr1 = img_old.array arr2 = img_new.array - deltaA_max = np.max(np.abs(arr2-arr1)) - print('deltaA-max:', np.max(np.abs(arr2-arr1))) - print('deltaA-min:', np.min(np.abs(arr2-arr1))) + deltaA_max = np.max(np.abs(arr2 - arr1)) + print("deltaA-max:", np.max(np.abs(arr2 - arr1))) + print("deltaA-min:", np.min(np.abs(arr2 - arr1))) # objB-highSFB obj = galsim.Gaussian(sigma=0.2, flux=10000) arr = obj.drawImage(nx=64, ny=64, scale=0.074).array - chip.img.array[(100-32):(100+32), (200-32):(200+32)] = arr[:, :] + chip.img.array[(100 - 32) : (100 + 32), (200 - 32) : (200 + 32)] = arr[:, :] img_old = copy.deepcopy(chip.img) img_new = add_brighter_fatter(img=chip.img) arr3 = img_old.array arr4 = img_new.array - deltaB_max = np.max(np.abs(arr4-arr3)) - print('deltaB-max:', np.max(np.abs(arr4-arr3))) - print('deltaB-min:', np.min(np.abs(arr4-arr3))) + deltaB_max = np.max(np.abs(arr4 - arr3)) + print("deltaB-max:", np.max(np.abs(arr4 - arr3))) + print("deltaB-min:", np.min(np.abs(arr4 - arr3))) self.assertTrue(deltaB_max > deltaA_max) def test_apply_CTE(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) filt = defineFilt(chip) print(chip.chipID) @@ -138,15 +148,32 @@ class detModule_coverage(unittest.TestCase): rho_trap = np.array([0.6, 1.6, 1.4], dtype=np.float32) trap_seeds = np.array([0, 100, 1000], dtype=np.int32) release_seed = 500 - image = fits.getdata(os.path.join( - self.dataPath, "testCTE_image_before.fits")).astype(np.int32) + image = fits.getdata( + os.path.join(self.dataPath, "testCTE_image_before.fits") + ).astype(np.int32) # get_trap_map(trap_seeds,nx,ny,nmax,rho_trap,beta,c,".") # bin2fits("trap.bin",".",nsp,nx,ny,nmax) - image_cti = CTI_sim(image, nx, ny, noverscan, nsp, nmax, - beta, w, c, t, rho_trap, trap_seeds, release_seed) - fits.writeto(os.path.join( - self.dataPath, "testCTE_image_after.fits"), data=image_cti, overwrite=True) - - -if __name__ == '__main__': + image_cti = CTI_sim( + image, + nx, + ny, + noverscan, + nsp, + nmax, + beta, + w, + c, + t, + rho_trap, + trap_seeds, + release_seed, + ) + fits.writeto( + os.path.join(self.dataPath, "testCTE_image_after.fits"), + data=image_cti, + overwrite=True, + ) + + +if __name__ == "__main__": unittest.main() diff --git a/tests/test_PSFmodule.py b/tests/test_PSFmodule.py index 434c2ec3de4366a8b9131b84003cf2219604ffc8..34888c02c29cbdde9d46fcd8073c4fd6ddba9015 100644 --- a/tests/test_PSFmodule.py +++ b/tests/test_PSFmodule.py @@ -1,9 +1,6 @@ import unittest -import sys import os -import math -from itertools import islice import numpy as np import galsim import yaml @@ -24,7 +21,8 @@ def defineCCD(iccd, config_file): chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) focal_plane = FocalPlane(chip_list=[iccd]) chip.img.wcs = focal_plane.getTanWCS( - 192.8595, 27.1283, -113.4333*galsim.degrees, chip.pix_scale) + 192.8595, 27.1283, -113.4333 * galsim.degrees, chip.pix_scale + ) return chip @@ -35,20 +33,22 @@ def defineFilt(chip): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list return bandpass_list class PSFInterpModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(PSFInterpModule_coverage, self).__init__(methodName) self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) self.iccd = 8 def test_loadPSFSet(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) bandpass = defineFilt(chip) print(chip.chipID) @@ -56,25 +56,38 @@ class PSFInterpModule_coverage(unittest.TestCase): # "/public/share/yangxuliu/CSSOSDataProductsSims/psfCube/set1_dynamic/" pathTemp = self.dataPath - psfModel = PSFInterp(chip, npsf=900, PSF_data_file=pathTemp, - PSF_data_prefix="", HocBuild=True, LOG_DEBUG=True) + psfModel = PSFInterp( + chip, + npsf=900, + PSF_data_file=pathTemp, + PSF_data_prefix="", + HocBuild=True, + LOG_DEBUG=True, + ) # imgPos[iobj, :] # try get the PSF at some location (1234, 1234) on the chip x, y = 4096, 4096 - x = x+chip.bound.xmin - y = y+chip.bound.ymin + x = x + chip.bound.xmin + y = y + chip.bound.ymin pos_img = galsim.PositionD(x, y) psf, _ = psfModel.get_PSF( - chip=chip, pos_img=pos_img, bandpass=0, galsimGSObject=True) + chip=chip, pos_img=pos_img, bandpass=0, galsimGSObject=True + ) psfA = psfModel.get_PSF( - chip=chip, pos_img=pos_img, bandpass=bandpass[0], galsimGSObject=False) + chip=chip, pos_img=pos_img, bandpass=bandpass[0], galsimGSObject=False + ) psfB = psfModel.get_PSF( - chip=chip, pos_img=pos_img, findNeighMode='hoclistFind', bandpass=bandpass[0], galsimGSObject=False) + chip=chip, + pos_img=pos_img, + findNeighMode="hoclistFind", + bandpass=bandpass[0], + galsimGSObject=False, + ) self.assertTrue(psf is not None) - self.assertTrue(np.max(np.abs(psfA-psfB)) < 1e-6) + self.assertTrue(np.max(np.abs(psfA - psfB)) < 1e-6) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_SpecDisperse.py b/tests/test_SpecDisperse.py index df7a7801ff447661e94f6e6aa243d96e3b41abd8..a5696ea4ce566e57bb0608879a67c3067afdc9b3 100644 --- a/tests/test_SpecDisperse.py +++ b/tests/test_SpecDisperse.py @@ -6,9 +6,8 @@ import unittest from observation_sim.mock_objects.SpecDisperser import rotate90, SpecDisperser -from observation_sim.config import ChipOutput from observation_sim.instruments import Telescope, Chip, FilterParam, Filter, FocalPlane -from observation_sim.mock_objects import MockObject, Star +from observation_sim.mock_objects import Star from observation_sim.psf import PSFGauss import numpy as np @@ -38,31 +37,32 @@ def getAngle132(x1=0, y1=0, z1=0, x2=0, y2=0, z2=0, x3=0, y3=0, z3=0): y22 = y2 - y3 z22 = z2 - z3 - tt = np.sqrt((x11 * x11 + y11 * y11 + z11 * z11) - * (x22 * x22 + y22 * y22 + z22 * z22)) - if (tt == 0): + tt = np.sqrt( + (x11 * x11 + y11 * y11 + z11 * z11) * (x22 * x22 + y22 * y22 + z22 * z22) + ) + if tt == 0: return 0 cosValue = (x11 * x22 + y11 * y22 + z11 * z22) / tt - if (cosValue > 1): + if cosValue > 1: cosValue = 1 - if (cosValue < -1): + if cosValue < -1: cosValue = -1 angle = math.acos(cosValue) return angle * 360 / (2 * math.pi) def fit_SingleGauss(xX, yX, contmX, iHa0): - background = LinearModel(prefix='line_') + background = LinearModel(prefix="line_") pars = background.make_params(intercept=yX.max(), slope=0) pars = background.guess(yX, x=xX) - gauss = GaussianModel(prefix='g_') + gauss = GaussianModel(prefix="g_") pars.update(gauss.make_params()) - pars['g_center'].set(iHa0, min=iHa0 - 3, max=iHa0 + 3) - pars['g_amplitude'].set(50, min=0) - pars['g_sigma'].set(12, min=0.0001) + pars["g_center"].set(iHa0, min=iHa0 - 3, max=iHa0 + 3) + pars["g_amplitude"].set(50, min=0) + pars["g_sigma"].set(12, min=0.0001) mod = gauss + background init = mod.eval(pars, x=xX) @@ -72,30 +72,68 @@ def fit_SingleGauss(xX, yX, contmX, iHa0): # print outX.params['g_center'] outX.fit_report(min_correl=0.25) # print(outX.fit_report(min_correl=0.25)) - line_slopeX = float(outX.fit_report(min_correl=0.25).split( - 'line_slope:')[1].split('+/-')[0]) * contmX - err_line_slopeX = float( - outX.fit_report(min_correl=0.25).split('line_slope:')[1].split('+/-')[1].split('(')[0]) * contmX - - line_interceptX = float(outX.fit_report(min_correl=0.25).split( - 'line_intercept:')[1].split('+/-')[0]) * contmX - err_line_interceptX = float( - outX.fit_report(min_correl=0.25).split('line_intercept:')[1].split('+/-')[1].split('(')[0]) * contmX - - sigmaX = float(outX.fit_report(min_correl=0.25).split( - 'g_sigma:')[1].split('+/-')[0]) - err_sigmaX = float(outX.fit_report(min_correl=0.25).split( - 'g_sigma:')[1].split('+/-')[1].split('(')[0]) - - fwhmX = float(outX.fit_report(min_correl=0.25).split( - 'g_fwhm:')[1].split('+/-')[0]) - err_fwhmX = float(outX.fit_report(min_correl=0.25).split( - 'g_fwhm:')[1].split('+/-')[1].split('(')[0]) - - centerX = float(outX.fit_report(min_correl=0.25).split( - 'g_center:')[1].split('+/-')[0]) - err_centerX = float(outX.fit_report(min_correl=0.25).split( - 'g_center:')[1].split('+/-')[1].split('(')[0]) + line_slopeX = ( + float(outX.fit_report(min_correl=0.25).split("line_slope:")[1].split("+/-")[0]) + * contmX + ) + err_line_slopeX = ( + float( + outX + .fit_report(min_correl=0.25) + .split("line_slope:")[1] + .split("+/-")[1] + .split("(")[0] + ) + * contmX + ) + + line_interceptX = ( + float( + outX.fit_report(min_correl=0.25).split("line_intercept:")[1].split("+/-")[0] + ) + * contmX + ) + err_line_interceptX = ( + float( + outX + .fit_report(min_correl=0.25) + .split("line_intercept:")[1] + .split("+/-")[1] + .split("(")[0] + ) + * contmX + ) + + sigmaX = float( + outX.fit_report(min_correl=0.25).split("g_sigma:")[1].split("+/-")[0] + ) + err_sigmaX = float( + outX + .fit_report(min_correl=0.25) + .split("g_sigma:")[1] + .split("+/-")[1] + .split("(")[0] + ) + + fwhmX = float(outX.fit_report(min_correl=0.25).split("g_fwhm:")[1].split("+/-")[0]) + err_fwhmX = float( + outX + .fit_report(min_correl=0.25) + .split("g_fwhm:")[1] + .split("+/-")[1] + .split("(")[0] + ) + + centerX = float( + outX.fit_report(min_correl=0.25).split("g_center:")[1].split("+/-")[0] + ) + err_centerX = float( + outX + .fit_report(min_correl=0.25) + .split("g_center:")[1] + .split("+/-")[1] + .split("(")[0] + ) return sigmaX, err_sigmaX, fwhmX, err_fwhmX, centerX, err_centerX @@ -113,31 +151,33 @@ def produceObj(x, y, chip, ra, dec, pa): obj = Star(param) - header_wcs = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - row_num=chip.rowID, - col_num=chip.colID, - pixel_scale=chip.pix_scale, - pixel_size=chip.pix_size, - xcen=chip.x_cen, - ycen=chip.y_cen, - extName='SCI') + header_wcs = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + row_num=chip.rowID, + col_num=chip.colID, + pixel_scale=chip.pix_scale, + pixel_size=chip.pix_size, + xcen=chip.x_cen, + ycen=chip.y_cen, + extName="SCI", + ) chip_wcs = galsim.FitsWCS(header=header_wcs) param["ra"] = chip_wcs.posToWorld(pos_img).ra.deg param["dec"] = chip_wcs.posToWorld(pos_img).dec.deg # pos_img, offset, local_wcs, _, _ = obj.getPosImg_Offset_WCS(img=chip.img, chip=chip, img_header=header_wcs) - pos_img, offset, local_wcs, real_wcs, fd_shear = obj.getPosImg_Offset_WCS(img=chip.img, - chip=chip, verbose=False, - chip_wcs=chip_wcs, img_header=header_wcs) + pos_img, offset, local_wcs, real_wcs, fd_shear = obj.getPosImg_Offset_WCS( + img=chip.img, chip=chip, verbose=False, chip_wcs=chip_wcs, img_header=header_wcs + ) wave = np.arange(2500, 11000.5, 0.5) # sedLen = wave.shape[0] flux = pow(wave, -2) * 1e8 @@ -145,63 +185,89 @@ def produceObj(x, y, chip, ra, dec, pa): flux[800] = flux[800] * 30 flux[2000] = flux[2000] * 5 - obj.sed = Table(np.array([wave, flux]).T, - names=('WAVELENGTH', 'FLUX')) + obj.sed = Table(np.array([wave, flux]).T, names=("WAVELENGTH", "FLUX")) return obj, pos_img class TestSpecDisperse(unittest.TestCase): - - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(TestSpecDisperse, self).__init__(methodName) - self.filePath('csst_msc_sim/test_sls_and_straylight') + self.filePath("csst_msc_sim/test_sls_and_straylight") # self.conff = conff # self.throughputf = throughputf def filePath(self, file_name): - fn = os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), file_name) - self.conff = os.path.join(fn, 'CSST_GI2.conf') - self.throughputf = os.path.join(fn, 'GI.Throughput.1st.fits') + fn = os.path.join(os.getenv("UNIT_TEST_DATA_ROOT"), file_name) + self.conff = os.path.join(fn, "CSST_GI2.conf") + self.throughputf = os.path.join(fn, "GI.Throughput.1st.fits") self.testDir = fn - self.outDataFn = os.path.join(fn, 'output') + self.outDataFn = os.path.join(fn, "output") if os.path.isdir(self.outDataFn): pass else: os.mkdir(self.outDataFn) def test_rotate901(self): - m = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [ - 16, 17, 18, 19, 20], [21, 22, 23, 24, 25]]) - m1 = np.array([[21, 16, 11, 6, 1], [22, 17, 12, 7, 2], [ - 23, 18, 13, 8, 3], [24, 19, 14, 9, 4], [25, 20, 15, 10, 5]]) - m2 = np.array([[5, 10, 15, 20, 25], [4, 9, 14, 19, 24], [ - 3, 8, 13, 18, 23], [2, 7, 12, 17, 22], [1, 6, 11, 16, 21]]) + m = np.array([ + [1, 2, 3, 4, 5], + [6, 7, 8, 9, 10], + [11, 12, 13, 14, 15], + [16, 17, 18, 19, 20], + [21, 22, 23, 24, 25], + ]) + m1 = np.array([ + [21, 16, 11, 6, 1], + [22, 17, 12, 7, 2], + [23, 18, 13, 8, 3], + [24, 19, 14, 9, 4], + [25, 20, 15, 10, 5], + ]) + m2 = np.array([ + [5, 10, 15, 20, 25], + [4, 9, 14, 19, 24], + [3, 8, 13, 18, 23], + [2, 7, 12, 17, 22], + [1, 6, 11, 16, 21], + ]) xc = 2 yc = 2 isClockwise = 0 - m1, xc1, yc1 = rotate90(array_orig=m, xc=xc, - yc=yc, isClockwise=isClockwise) - self.assertTrue(xc1-xc == 0) - self.assertTrue(yc1-yc == 0) - self.assertTrue(np.sum(m-m1) == 0) + m1, xc1, yc1 = rotate90(array_orig=m, xc=xc, yc=yc, isClockwise=isClockwise) + self.assertTrue(xc1 - xc == 0) + self.assertTrue(yc1 - yc == 0) + self.assertTrue(np.sum(m - m1) == 0) def test_rotate902(self): - m = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [ - 16, 17, 18, 19, 20], [21, 22, 23, 24, 25]]) - m1 = np.array([[21, 16, 11, 6, 1], [22, 17, 12, 7, 2], [ - 23, 18, 13, 8, 3], [24, 19, 14, 9, 4], [25, 20, 15, 10, 5]]) - m2 = np.array([[5, 10, 15, 20, 25], [4, 9, 14, 19, 24], [ - 3, 8, 13, 18, 23], [2, 7, 12, 17, 22], [1, 6, 11, 16, 21]]) + m = np.array([ + [1, 2, 3, 4, 5], + [6, 7, 8, 9, 10], + [11, 12, 13, 14, 15], + [16, 17, 18, 19, 20], + [21, 22, 23, 24, 25], + ]) + m1 = np.array([ + [21, 16, 11, 6, 1], + [22, 17, 12, 7, 2], + [23, 18, 13, 8, 3], + [24, 19, 14, 9, 4], + [25, 20, 15, 10, 5], + ]) + m2 = np.array([ + [5, 10, 15, 20, 25], + [4, 9, 14, 19, 24], + [3, 8, 13, 18, 23], + [2, 7, 12, 17, 22], + [1, 6, 11, 16, 21], + ]) xc = 2 yc = 2 isClockwise = 1 - m1, xc1, yc1 = rotate90(array_orig=m, xc=xc, - yc=yc, isClockwise=isClockwise) - self.assertTrue(xc1-xc == 0) - self.assertTrue(yc1-yc == 0) - self.assertTrue(np.sum(m-m2) == 0) + m1, xc1, yc1 = rotate90(array_orig=m, xc=xc, yc=yc, isClockwise=isClockwise) + self.assertTrue(xc1 - xc == 0) + self.assertTrue(yc1 - yc == 0) + self.assertTrue(np.sum(m - m2) == 0) def test_Specdistperse1(self): @@ -216,19 +282,28 @@ class TestSpecDisperse(unittest.TestCase): # flux[800] = flux[800] * 30 # flux[2000] = flux[2000] * 5 - sed = Table(np.array([wave, flux]).T, - names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([wave, flux]).T, names=("WAVELENGTH", "FLUX")) conff = self.conff throughput_f = self.throughputf thp = Table.read(throughput_f) - thp_i = interpolate.interp1d(thp['WAVELENGTH'], thp['SENSITIVITY']) - sdp = SpecDisperser(orig_img=starImg, xcenter=0, ycenter=0, origin=[100, 100], tar_spec=sed, band_start=6200, - band_end=10000, isAlongY=0, conf=conff, gid=0) + thp_i = interpolate.interp1d(thp["WAVELENGTH"], thp["SENSITIVITY"]) + sdp = SpecDisperser( + orig_img=starImg, + xcenter=0, + ycenter=0, + origin=[100, 100], + tar_spec=sed, + band_start=6200, + band_end=10000, + isAlongY=0, + conf=conff, + gid=0, + ) spec = sdp.compute_spec_orders() - Aimg = spec['A'][0] - wave_pix = spec['A'][5] - wave_pos = spec['A'][3] - sens = spec['A'][6] + Aimg = spec["A"][0] + wave_pix = spec["A"][5] + wave_pos = spec["A"][3] + sens = spec["A"][6] sh = Aimg.shape spec_pix = np.zeros(sh[1]) for i in range(sh[1]): @@ -248,12 +323,17 @@ class TestSpecDisperse(unittest.TestCase): else: f = 0 wave_flux[i] = f / deltW - sed_i = interpolate.interp1d(sed['WAVELENGTH'], sed['FLUX']) + sed_i = interpolate.interp1d(sed["WAVELENGTH"], sed["FLUX"]) ids = wave_pix < 9700 ids1 = wave_pix[ids] > 6500 - print('Spec disperse flux test') - self.assertTrue(np.mean( - (wave_flux[ids][ids1] - sed_i(wave_pix[ids][ids1]))/sed_i(wave_pix[ids][ids1])) < 0.004) + print("Spec disperse flux test") + self.assertTrue( + np.mean( + (wave_flux[ids][ids1] - sed_i(wave_pix[ids][ids1])) + / sed_i(wave_pix[ids][ids1]) + ) + < 0.004 + ) # plt.figure() # plt.plot(wave_pix, wave_flux) # plt.plot(sed['WAVELENGTH'], sed['FLUX']) @@ -280,19 +360,28 @@ class TestSpecDisperse(unittest.TestCase): flux[800] = flux[800] * 30 flux[2000] = flux[2000] * 5 - sed = Table(np.array([wave, flux]).T, - names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([wave, flux]).T, names=("WAVELENGTH", "FLUX")) conff = self.conff throughput_f = self.throughputf thp = Table.read(throughput_f) - thp_i = interpolate.interp1d(thp['WAVELENGTH'], thp['SENSITIVITY']) - sdp = SpecDisperser(orig_img=starImg, xcenter=0, ycenter=0, origin=[100, 100], tar_spec=sed, band_start=6200, - band_end=10000, isAlongY=0, conf=conff, gid=0) + thp_i = interpolate.interp1d(thp["WAVELENGTH"], thp["SENSITIVITY"]) + sdp = SpecDisperser( + orig_img=starImg, + xcenter=0, + ycenter=0, + origin=[100, 100], + tar_spec=sed, + band_start=6200, + band_end=10000, + isAlongY=0, + conf=conff, + gid=0, + ) spec = sdp.compute_spec_orders() - Aimg = spec['A'][0] - wave_pix = spec['A'][5] - wave_pos = spec['A'][3] - sens = spec['A'][6] + Aimg = spec["A"][0] + wave_pix = spec["A"][5] + wave_pos = spec["A"][3] + sens = spec["A"][6] sh = Aimg.shape spec_pix = np.zeros(sh[1]) for i in range(sh[1]): @@ -312,21 +401,22 @@ class TestSpecDisperse(unittest.TestCase): else: f = 0 wave_flux[i] = f / deltW - sed_i = interpolate.interp1d(sed['WAVELENGTH'], sed['FLUX']) + sed_i = interpolate.interp1d(sed["WAVELENGTH"], sed["FLUX"]) input_em_lam = 6600 - ids = wave_pix < input_em_lam+200 - ids1 = wave_pix[ids] > input_em_lam-200 - deltLamda_pix = (max( - wave_pix[ids][ids1]) - min(wave_pix[ids][ids1])) / (wave_pix[ids][ids1].shape[0] - 1) + ids = wave_pix < input_em_lam + 200 + ids1 = wave_pix[ids] > input_em_lam - 200 + deltLamda_pix = (max(wave_pix[ids][ids1]) - min(wave_pix[ids][ids1])) / ( + wave_pix[ids][ids1].shape[0] - 1 + ) _, _, fwhmx, fwhmx_err, center, center_err = fit_SingleGauss( - wave_pix[ids][ids1], wave_flux[ids][ids1], 1.0, 6600) + wave_pix[ids][ids1], wave_flux[ids][ids1], 1.0, 6600 + ) - print('Emission line position and shape test') + print("Emission line position and shape test") - self.assertTrue(input_em_lam-center < deltLamda_pix) + self.assertTrue(input_em_lam - center < deltLamda_pix) # print(fwhmx/deltLamda_pix*pix_scale - psf_fwhm) - self.assertTrue(fwhmx/deltLamda_pix*pix_scale - - psf_fwhm < np.abs(0.02)) + self.assertTrue(fwhmx / deltLamda_pix * pix_scale - psf_fwhm < np.abs(0.02)) # print('error is ',np.mean((wave_flux[ids][ids1] - sed_i(wave_pix[ids][ids1]))/sed_i(wave_pix[ids][ids1]))) # self.assertTrue(np.mean((wave_flux[ids][ids1] - sed_i(wave_pix[ids][ids1]))/sed_i(wave_pix[ids][ids1]))<0.004) # plt.figure() @@ -355,22 +445,41 @@ class TestSpecDisperse(unittest.TestCase): flux[800] = flux[800] * 30 flux[2000] = flux[2000] * 5 - sed = Table(np.array([wave, flux]).T, - names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([wave, flux]).T, names=("WAVELENGTH", "FLUX")) conff = self.conff throughput_f = self.throughputf thp = Table.read(throughput_f) - thp_i = interpolate.interp1d(thp['WAVELENGTH'], thp['SENSITIVITY']) - sdp = SpecDisperser(orig_img=starImg, xcenter=0, ycenter=0, origin=[100, 100], tar_spec=sed, band_start=6200, - band_end=8000, isAlongY=0, conf=conff, gid=0) - sdp1 = SpecDisperser(orig_img=starImg, xcenter=0, ycenter=0, origin=[100, 100], tar_spec=sed, band_start=8000, - band_end=10000, isAlongY=0, conf=conff, gid=0) + thp_i = interpolate.interp1d(thp["WAVELENGTH"], thp["SENSITIVITY"]) + sdp = SpecDisperser( + orig_img=starImg, + xcenter=0, + ycenter=0, + origin=[100, 100], + tar_spec=sed, + band_start=6200, + band_end=8000, + isAlongY=0, + conf=conff, + gid=0, + ) + sdp1 = SpecDisperser( + orig_img=starImg, + xcenter=0, + ycenter=0, + origin=[100, 100], + tar_spec=sed, + band_start=8000, + band_end=10000, + isAlongY=0, + conf=conff, + gid=0, + ) spec = sdp.compute_spec_orders() spec1 = sdp1.compute_spec_orders() - Aimg = spec['A'][0] + spec1['A'][0] - wave_pix = spec['A'][5] - wave_pos = spec['A'][3] - sens = spec['A'][6] + Aimg = spec["A"][0] + spec1["A"][0] + wave_pix = spec["A"][5] + wave_pos = spec["A"][3] + sens = spec["A"][6] sh = Aimg.shape spec_pix = np.zeros(sh[1]) for i in range(sh[1]): @@ -389,11 +498,21 @@ class TestSpecDisperse(unittest.TestCase): f = 0 wave_flux[i] = f / deltW - sdp2 = SpecDisperser(orig_img=starImg, xcenter=0, ycenter=0, origin=[100, 100], tar_spec=sed, band_start=6200, - band_end=10000, isAlongY=0, conf=conff, gid=0) + sdp2 = SpecDisperser( + orig_img=starImg, + xcenter=0, + ycenter=0, + origin=[100, 100], + tar_spec=sed, + band_start=6200, + band_end=10000, + isAlongY=0, + conf=conff, + gid=0, + ) spec2 = sdp2.compute_spec_orders() - Aimg2 = spec2['A'][0] + Aimg2 = spec2["A"][0] spec_pix2 = np.zeros(sh[1]) for i in range(sh[1]): @@ -415,8 +534,8 @@ class TestSpecDisperse(unittest.TestCase): r1_i = interpolate.interp1d(wave_pix, wave_flux2) r2_i = interpolate.interp1d(wave_pix, wave_flux) - print('Spec Splicing test') - self.assertTrue(r1_i(8000)-r2_i(8000) < np.abs(0.0001)) + print("Spec Splicing test") + self.assertTrue(r1_i(8000) - r2_i(8000) < np.abs(0.0001)) # self.assertTrue(fwhmx/deltLamda_pix*pix_scale - psf_fwhm < np.abs(0.01)) # print('error is ',np.mean((wave_flux[ids][ids1] - sed_i(wave_pix[ids][ids1]))/sed_i(wave_pix[ids][ids1]))) @@ -427,18 +546,18 @@ class TestSpecDisperse(unittest.TestCase): # plt.plot(sed['WAVELENGTH'], sed['FLUX']) plt.xlim(6200, 10000) plt.ylim(1, 4) - plt.yscale('log') - plt.xlabel(r'$\lambda$') - plt.ylabel(r'$F\lambda$') - plt.legend(['one spec', 'split in 8000 A']) + plt.yscale("log") + plt.xlabel(r"$\lambda$") + plt.ylabel(r"$F\lambda$") + plt.legend(["one spec", "split in 8000 A"]) plt.show() def test_double_disperse(self): # work_dir = "/public/home/fangyuedong/CSST_unittest/CSST/test/" # data_dir = "/Volumes/Extreme SSD/SimData/" # data_dir = "/data/simudata/CSSOSDataProductsSims/data/" - configFn = os.path.join(self.testDir, 'config_C6.yaml') - normFilterFn = os.path.join(self.testDir, 'SLOAN_SDSS.g.fits') + configFn = os.path.join(self.testDir, "config_C6.yaml") + normFilterFn = os.path.join(self.testDir, "SLOAN_SDSS.g.fits") norm_star = Table.read(normFilterFn) with open(configFn, "r") as stream: try: @@ -449,24 +568,37 @@ class TestSpecDisperse(unittest.TestCase): print(exc) filter_param = FilterParam() - focal_plane = FocalPlane( - survey_type=config["obs_setting"]["survey_type"]) + focal_plane = FocalPlane(survey_type=config["obs_setting"]["survey_type"]) chip = Chip(1, config=config) filter_id, filter_type = chip.getChipFilter() - filt = Filter(filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + filt = Filter( + filter_id=filter_id, + filter_type=filter_type, + filter_param=filter_param, + ccd_bandpass=chip.effCurve, + ) tel = Telescope() psf_model = PSFGauss(chip=chip) - wcs_fp = focal_plane.getTanWCS(float(config["obs_setting"]["ra_center"]), float( - config["obs_setting"]["dec_center"]), float(config["obs_setting"]["image_rot"]) * galsim.degrees, chip.pix_scale) + wcs_fp = focal_plane.getTanWCS( + float(config["obs_setting"]["ra_center"]), + float(config["obs_setting"]["dec_center"]), + float(config["obs_setting"]["image_rot"]) * galsim.degrees, + chip.pix_scale, + ) chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) chip.img.setOrigin(chip.bound.xmin, chip.bound.ymin) chip.img.wcs = wcs_fp - obj, pos_img = produceObj(2000, 4500, chip, float(config["obs_setting"]["ra_center"]), float( - config["obs_setting"]["dec_center"]), float(config["obs_setting"]["image_rot"])) + obj, pos_img = produceObj( + 2000, + 4500, + chip, + float(config["obs_setting"]["ra_center"]), + float(config["obs_setting"]["dec_center"]), + float(config["obs_setting"]["image_rot"]), + ) # print(pos_img,chip.pix_scale) obj.drawObj_slitless( tel=tel, @@ -478,10 +610,17 @@ class TestSpecDisperse(unittest.TestCase): g1=0, g2=0, exptime=150, - normFilter=norm_star) - - obj, pos_img = produceObj(3685, 6500, chip, float(config["obs_setting"]["ra_center"]), float( - config["obs_setting"]["dec_center"]), float(config["obs_setting"]["image_rot"])) + normFilter=norm_star, + ) + + obj, pos_img = produceObj( + 3685, + 6500, + chip, + float(config["obs_setting"]["ra_center"]), + float(config["obs_setting"]["dec_center"]), + float(config["obs_setting"]["image_rot"]), + ) obj.drawObj_slitless( tel=tel, pos_img=pos_img, @@ -492,10 +631,17 @@ class TestSpecDisperse(unittest.TestCase): g1=0, g2=0, exptime=150, - normFilter=norm_star) - - obj, pos_img = produceObj(5000, 2500, chip, float(config["obs_setting"]["ra_center"]), float( - config["obs_setting"]["dec_center"]), float(config["obs_setting"]["image_rot"])) + normFilter=norm_star, + ) + + obj, pos_img = produceObj( + 5000, + 2500, + chip, + float(config["obs_setting"]["ra_center"]), + float(config["obs_setting"]["dec_center"]), + float(config["obs_setting"]["image_rot"]), + ) obj.drawObj_slitless( tel=tel, pos_img=pos_img, @@ -506,12 +652,17 @@ class TestSpecDisperse(unittest.TestCase): g1=0, g2=0, exptime=150, - normFilter=norm_star) + normFilter=norm_star, + ) - print('Spec double disperse test') + print("Spec double disperse test") from astropy.io import fits - fits.writeto(os.path.join( - self.outDataFn, 'test_sls_doubleDisp.fits'), chip.img.array, overwrite=True) + + fits.writeto( + os.path.join(self.outDataFn, "test_sls_doubleDisp.fits"), + chip.img.array, + overwrite=True, + ) # plt.figure() # plt.imshow(chip.img.array) @@ -519,7 +670,8 @@ class TestSpecDisperse(unittest.TestCase): def test_SLSImage_rotation(self): from astropy.wcs import WCS - configFn = os.path.join(self.testDir, 'config_C6.yaml') + + configFn = os.path.join(self.testDir, "config_C6.yaml") with open(configFn, "r") as stream: try: @@ -535,20 +687,22 @@ class TestSpecDisperse(unittest.TestCase): pa = float(config["obs_setting"]["image_rot"]) chip.rotate_angle = 0 - header_wcs1 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs1 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) center = np.array([chip.npix_x / 2, chip.npix_y / 2]) h_wcs1 = WCS(header_wcs1) @@ -557,20 +711,22 @@ class TestSpecDisperse(unittest.TestCase): chip = Chip(1, config=config) rot_angle = 1 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -581,20 +737,22 @@ class TestSpecDisperse(unittest.TestCase): rot_angle = 10 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -604,20 +762,22 @@ class TestSpecDisperse(unittest.TestCase): rot_angle = 50 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -631,20 +791,22 @@ class TestSpecDisperse(unittest.TestCase): dec = float(config["obs_setting"]["dec_center"]) pa = float(config["obs_setting"]["image_rot"]) chip.rotate_angle = 0 - header_wcs1 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs1 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) center = np.array([chip.npix_x / 2, chip.npix_y / 2]) h_wcs1 = WCS(header_wcs1) @@ -653,20 +815,22 @@ class TestSpecDisperse(unittest.TestCase): rot_angle = 1 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -676,20 +840,22 @@ class TestSpecDisperse(unittest.TestCase): rot_angle = 10 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -699,20 +865,22 @@ class TestSpecDisperse(unittest.TestCase): rot_angle = 50 chip.rotate_angle = rot_angle - header_wcs2 = generateExtensionHeader(chip, - xlen=chip.npix_x, - ylen=chip.npix_y, - ra=ra, - dec=dec, - pa=pa, - gain=chip.gain, - readout=chip.read_noise, - dark=chip.dark_noise, - saturation=90000, - pixel_scale=chip.pix_scale, - row_num=chip.rowID, - col_num=chip.colID, - extName='raw') + header_wcs2 = generateExtensionHeader( + chip, + xlen=chip.npix_x, + ylen=chip.npix_y, + ra=ra, + dec=dec, + pa=pa, + gain=chip.gain, + readout=chip.read_noise, + dark=chip.dark_noise, + saturation=90000, + pixel_scale=chip.pix_scale, + row_num=chip.rowID, + col_num=chip.colID, + extName="raw", + ) h_wcs2 = WCS(header_wcs2) x2, y2 = h_wcs2.world_to_pixel(sky_1) @@ -721,22 +889,23 @@ class TestSpecDisperse(unittest.TestCase): self.assertTrue(rot_angle - angle < np.abs(0.001)) -if __name__ == '__main__': - - os.environ['UNIT_TEST_DATA_ROOT'] = "/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/tests/testData" - testDir = os.getenv('UNIT_TEST_DATA_ROOT') +if __name__ == "__main__": + os.environ["UNIT_TEST_DATA_ROOT"] = ( + "/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/tests/testData" + ) + testDir = os.getenv("UNIT_TEST_DATA_ROOT") # conff= os.path.join(testDir, 'CSST_GI2.conf') # throughputf= os.path.join(testDir, 'GI.Throughput.1st.fits') suit = unittest.TestSuite() - case1 = TestSpecDisperse('test_Specdistperse1') + case1 = TestSpecDisperse("test_Specdistperse1") suit.addTest(case1) - case2 = TestSpecDisperse('test_Specdistperse2') + case2 = TestSpecDisperse("test_Specdistperse2") suit.addTest(case2) - case3 = TestSpecDisperse('test_Specdistperse3') + case3 = TestSpecDisperse("test_Specdistperse3") suit.addTest(case3) - case4 = TestSpecDisperse('test_double_disperse') + case4 = TestSpecDisperse("test_double_disperse") suit.addTest(case4) - case5 = TestSpecDisperse('test_SLSImage_rotation') + case5 = TestSpecDisperse("test_SLSImage_rotation") suit.addTest(case5) unittest.TextTestRunner(verbosity=2).run(suit) diff --git a/tests/test_Straylight.py b/tests/test_Straylight.py index 63c1376a3a610d5f2fd3c9f11157b93dc9e5ede7..056e84a539157052228bedfdf464b4758088ab16 100644 --- a/tests/test_Straylight.py +++ b/tests/test_Straylight.py @@ -9,18 +9,29 @@ from observation_sim.sky_background import Straylight import numpy as np import math import astropy.constants as cons -import galsim -from astropy.table import Table -from scipy import interpolate import matplotlib.pyplot as plt import os -hubbleAverZodiacal = {'nuv': 0.0035, 'u': 0.0163, 'g': 0.1109, - 'r': 0.1471, 'i': 0.1568, 'z': 0.0953, 'y': 0.0283} -hubbleAverEarthShine = {'nuv': 0.00024, 'u': 0.0051, 'g': 0.0506, - 'r': 0.0591, 'i': 0.0568, 'z': 0.0315, 'y': 0.0090} +hubbleAverZodiacal = { + "nuv": 0.0035, + "u": 0.0163, + "g": 0.1109, + "r": 0.1471, + "i": 0.1568, + "z": 0.0953, + "y": 0.0283, +} +hubbleAverEarthShine = { + "nuv": 0.00024, + "u": 0.0051, + "g": 0.0506, + "r": 0.0591, + "i": 0.0568, + "z": 0.0315, + "y": 0.0090, +} # def transRaDec2D(ra, dec): # x1 = np.cos(dec / 57.2957795) * np.cos(ra / 57.2957795); @@ -41,54 +52,61 @@ def getAngle132(x1=0, y1=0, z1=0, x2=0, y2=0, z2=0, x3=0, y3=0, z3=0): y22 = y2 - y3 z22 = z2 - z3 - tt = np.sqrt((x11 * x11 + y11 * y11 + z11 * z11) - * (x22 * x22 + y22 * y22 + z22 * z22)) - if (tt == 0): + tt = np.sqrt( + (x11 * x11 + y11 * y11 + z11 * z11) * (x22 * x22 + y22 * y22 + z22 * z22) + ) + if tt == 0: return 0 cosValue = (x11 * x22 + y11 * y22 + z11 * z22) / tt - if (cosValue > 1): + if cosValue > 1: cosValue = 1 - if (cosValue < -1): + if cosValue < -1: cosValue = -1 angle = math.acos(cosValue) return angle * 360 / (2 * math.pi) -def calculateAnglePwithEarth(sat=np.array([0, 0, 0]), pointing=np.array([0, 0, 0]), sun=np.array([0, 0, 0])): - modSat = np.sqrt(sat[0]*sat[0] + sat[1]*sat[1]+sat[2]*sat[2]) - modPoint = np.sqrt(pointing[0]*pointing[0] + - pointing[1]*pointing[1] + pointing[2]*pointing[2]) +def calculateAnglePwithEarth( + sat=np.array([0, 0, 0]), pointing=np.array([0, 0, 0]), sun=np.array([0, 0, 0]) +): + modSat = np.sqrt(sat[0] * sat[0] + sat[1] * sat[1] + sat[2] * sat[2]) + modPoint = np.sqrt( + pointing[0] * pointing[0] + + pointing[1] * pointing[1] + + pointing[2] * pointing[2] + ) withLocalZenithAngle = ( - pointing[0] * sat[0] + pointing[1] * sat[1] + pointing[2] * sat[2]) / (modPoint*modSat) + pointing[0] * sat[0] + pointing[1] * sat[1] + pointing[2] * sat[2] + ) / (modPoint * modSat) innerM_sat_sun = sat[0] * sun[0] + sat[1] * sun[1] + sat[2] * sun[2] - cosAngle = innerM_sat_sun / (modSat * cons.au.value/1000) + cosAngle = innerM_sat_sun / (modSat * cons.au.value / 1000) isInSunSide = 1 - if (cosAngle < -0.3385737): # cos109.79 + if cosAngle < -0.3385737: # cos109.79 isInSunSide = -1 elif cosAngle >= -0.3385737 and cosAngle <= 0.3385737: isInSunSide = 0 - return math.acos(withLocalZenithAngle)*180/math.pi, isInSunSide + return math.acos(withLocalZenithAngle) * 180 / math.pi, isInSunSide class TestStraylight(unittest.TestCase): - - def __init__(self, methodName='runTest', filter='i', grating="GI"): + def __init__(self, methodName="runTest", filter="i", grating="GI"): super(TestStraylight, self).__init__(methodName) # print(file_name) # fn = os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), file_name) # self.pointingData = np.loadtxt(os.path.join(fn, 'Straylight_test.dat'), dtype=np.double) - self.filePath('csst_msc_sim/test_sls_and_straylight') + self.filePath("csst_msc_sim/test_sls_and_straylight") self.filter = filter self.grating = grating def filePath(self, file_name): - fn = os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), file_name) - self.pointingData = np.loadtxt(os.path.join( - fn, 'Straylight_test.dat'), dtype=np.double) + fn = os.path.join(os.getenv("UNIT_TEST_DATA_ROOT"), file_name) + self.pointingData = np.loadtxt( + os.path.join(fn, "Straylight_test.dat"), dtype=np.double + ) def test_EarthShineFilter(self): d_sh = self.pointingData.shape @@ -100,11 +118,21 @@ class TestStraylight(unittest.TestCase): ju = self.pointingData[i, 5] # pointing = transRaDec2D(self.pointingData[i, 0], self.pointingData[i, 1]) # print(ju, pointing, surveylist[i,3:9]) - sl = Straylight(jtime=ju, sat_pos=self.pointingData[i, 6:9], pointing_radec=np.array( - [self.pointingData[i, 0], self.pointingData[i, 1]]), sun_pos=self.pointingData[i, 9:12]) + sl = Straylight( + jtime=ju, + sat_pos=self.pointingData[i, 6:9], + pointing_radec=np.array([ + self.pointingData[i, 0], + self.pointingData[i, 1], + ]), + sun_pos=self.pointingData[i, 9:12], + ) e1, py = sl.calculateEarthShineFilter(filter=self.filter) earthZenithAngle, isInSunSide = calculateAnglePwithEarth( - sat=self.pointingData[i, 6:9], pointing=sl.pointing, sun=self.pointingData[i, 9:12]) + sat=self.pointingData[i, 6:9], + pointing=sl.pointing, + sun=self.pointingData[i, 9:12], + ) # e2, _ = sl.calculateZodiacalFilter2(filter='i', sun_pos=sl.sun_pos) # e3 = sl.calculateStarLightFilter(filter='i', pointYaxis=py) # e_all = sl.calculateStrayLightFilter(filter='i') @@ -113,16 +141,16 @@ class TestStraylight(unittest.TestCase): sl_e_pix[i, 1] = earthZenithAngle sl_e_pix[i, 2] = isInSunSide median = np.median(sl_e_pix[:, 0]) - print(' average Earthshine %s: %e' % (self.filter, median)) - self.assertTrue(median-hubbleAverEarthShine[self.filter] < 0.1) + print(" average Earthshine %s: %e" % (self.filter, median)) + self.assertTrue(median - hubbleAverEarthShine[self.filter] < 0.1) plt.figure() ids1 = sl_e_pix[:, 2] == 1 ids2 = sl_e_pix[:, 2] != 1 - plt.plot(sl_e_pix[ids1, 0], sl_e_pix[ids1, 1], 'r.') - plt.plot(sl_e_pix[ids2, 0], sl_e_pix[ids2, 1], 'b.') - plt.legend(['In Sun Side', 'In Earths shadow']) - plt.xlabel('straylight-earthshine(e-/pixel/s)') - plt.ylabel('Angle with local zenith(degree)') + plt.plot(sl_e_pix[ids1, 0], sl_e_pix[ids1, 1], "r.") + plt.plot(sl_e_pix[ids2, 0], sl_e_pix[ids2, 1], "b.") + plt.legend(["In Sun Side", "In Earths shadow"]) + plt.xlabel("straylight-earthshine(e-/pixel/s)") + plt.ylabel("Angle with local zenith(degree)") plt.show() def test_ZodiacalFilter(self): @@ -131,21 +159,36 @@ class TestStraylight(unittest.TestCase): for i in np.arange(d_sh[0]): ju = self.pointingData[i, 5] - sl = Straylight(jtime=ju, sat_pos=self.pointingData[i, 6:9], pointing_radec=np.array( - [self.pointingData[i, 0], self.pointingData[i, 1]]), sun_pos=self.pointingData[i, 9:12]) - e1, _ = sl.calculateZodiacalFilter2( - filter=self.filter, sun_pos=sl.sun_pos) + sl = Straylight( + jtime=ju, + sat_pos=self.pointingData[i, 6:9], + pointing_radec=np.array([ + self.pointingData[i, 0], + self.pointingData[i, 1], + ]), + sun_pos=self.pointingData[i, 9:12], + ) + e1, _ = sl.calculateZodiacalFilter2(filter=self.filter, sun_pos=sl.sun_pos) sl_e_pix[i, 0] = e1 - sl_e_pix[i, 1] = getAngle132(x1=self.pointingData[i, 9], y1=self.pointingData[i, 10], z1=self.pointingData[i, 11], x2=sl.pointing[0], - y2=sl.pointing[1], z2=sl.pointing[2], x3=0, y3=0, z3=0) + sl_e_pix[i, 1] = getAngle132( + x1=self.pointingData[i, 9], + y1=self.pointingData[i, 10], + z1=self.pointingData[i, 11], + x2=sl.pointing[0], + y2=sl.pointing[1], + z2=sl.pointing[2], + x3=0, + y3=0, + z3=0, + ) plt.figure() - plt.plot(sl_e_pix[:, 0], sl_e_pix[:, 1], 'r.') - plt.xlabel('straylight-zodiacal(e-/pixel/s)') - plt.ylabel('Angle between pointing and sun(degree)') + plt.plot(sl_e_pix[:, 0], sl_e_pix[:, 1], "r.") + plt.xlabel("straylight-zodiacal(e-/pixel/s)") + plt.ylabel("Angle between pointing and sun(degree)") plt.show() median = np.median(sl_e_pix[:, 0]) - print(' average Zodiacal %s: %f' % (self.filter, median)) - self.assertTrue(median-hubbleAverZodiacal[self.filter] < 0.1) + print(" average Zodiacal %s: %f" % (self.filter, median)) + self.assertTrue(median - hubbleAverZodiacal[self.filter] < 0.1) def test_StarFilter(self): d_sh = self.pointingData.shape @@ -158,8 +201,15 @@ class TestStraylight(unittest.TestCase): ju = self.pointingData[i, 5] # pointing = transRaDec2D(self.pointingData[i, 0], self.pointingData[i, 1]) # print(ju, pointing, surveylist[i,3:9]) - sl = Straylight(jtime=ju, sat_pos=self.pointingData[i, 6:9], pointing_radec=np.array( - [self.pointingData[i, 0], self.pointingData[i, 1]]), sun_pos=self.pointingData[i, 9:12]) + sl = Straylight( + jtime=ju, + sat_pos=self.pointingData[i, 6:9], + pointing_radec=np.array([ + self.pointingData[i, 0], + self.pointingData[i, 1], + ]), + sun_pos=self.pointingData[i, 9:12], + ) e1, py = sl.calculateEarthShineFilter(filter=self.filter) # e2, _ = sl.calculateZodiacalFilter2(filter='i', sun_pos=sl.sun_pos) e3 = sl.calculateStarLightFilter(filter=self.filter, pointYaxis=py) @@ -167,8 +217,8 @@ class TestStraylight(unittest.TestCase): # s_pix, spec = sl.calculateStrayLightGrating(grating='GI') sl_e_pix[i] = e3 median = np.median(sl_e_pix[0:tnum]) - print(' average Earthshine %s: %e' % (self.filter, median)) - self.assertTrue(median-hubbleAverEarthShine[self.filter] < 0.2) + print(" average Earthshine %s: %e" % (self.filter, median)) + self.assertTrue(median - hubbleAverEarthShine[self.filter] < 0.2) def test_GratingStraylight(self): d_sh = self.pointingData.shape @@ -181,8 +231,15 @@ class TestStraylight(unittest.TestCase): ju = self.pointingData[i, 5] # pointing = transRaDec2D(self.pointingData[i, 0], self.pointingData[i, 1]) # print(ju, pointing, surveylist[i,3:9]) - sl = Straylight(jtime=ju, sat_pos=self.pointingData[i, 6:9], pointing_radec=np.array( - [self.pointingData[i, 0], self.pointingData[i, 1]]), sun_pos=self.pointingData[i, 9:12]) + sl = Straylight( + jtime=ju, + sat_pos=self.pointingData[i, 6:9], + pointing_radec=np.array([ + self.pointingData[i, 0], + self.pointingData[i, 1], + ]), + sun_pos=self.pointingData[i, 9:12], + ) # e1, py = sl.calculateEarthShineFilter(filter=self.filter) # e2, _ = sl.calculateZodiacalFilter2(filter='i', sun_pos=sl.sun_pos) # e3 = sl.calculateStarLightFilter(filter=self.filter, pointYaxis=py) @@ -190,18 +247,20 @@ class TestStraylight(unittest.TestCase): s_pix, spec = sl.calculateStrayLightGrating(grating=self.grating) sl_e_pix[i] = s_pix plt.figure() - plt.plot(spec['WAVELENGTH'], spec['FLUX'], 'r') - plt.xlabel('WAVELENGTH') - plt.ylabel(r'F$\lambda$(erg/s/cm2/A/arcsec2)') + plt.plot(spec["WAVELENGTH"], spec["FLUX"], "r") + plt.xlabel("WAVELENGTH") + plt.ylabel(r"F$\lambda$(erg/s/cm2/A/arcsec2)") plt.xlim(2000, 10000) plt.show() median = np.median(sl_e_pix[0:tnum]) - print(' average Earthshine %s: %e' % (self.grating, median)) + print(" average Earthshine %s: %e" % (self.grating, median)) self.assertTrue(median < 0.8) -if __name__ == '__main__': - os.environ['UNIT_TEST_DATA_ROOT'] = "/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/tests/testData" +if __name__ == "__main__": + os.environ["UNIT_TEST_DATA_ROOT"] = ( + "/Users/zhangxin/Work/SlitlessSim/CSST_SIM/CSST_develop/csst-simulation/tests/testData" + ) # suit = unittest.TestSuite() # case1 = TestStraylight('test_EarthShineFilter', filter = 'i') diff --git a/tests/test_astrometry.py b/tests/test_astrometry.py index 3e84e7d4bac1508c9c371fe774d10339d39ccf12..6877289f9b4bc4aeb8af60b01612843a2914b70c 100644 --- a/tests/test_astrometry.py +++ b/tests/test_astrometry.py @@ -1,6 +1,4 @@ import unittest -import os -import sys from astropy.time import Time from datetime import datetime @@ -8,7 +6,7 @@ from observation_sim.astrometry.Astrometry_util import on_orbit_obs_position class TestAstrometry(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(TestAstrometry, self).__init__(methodName) # self.dataPath = os.path.join( # os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') @@ -16,9 +14,9 @@ class TestAstrometry(unittest.TestCase): def test_astrometry_method(self): ra_list = [300.061827] dec_list = [-60.132741] - pmra_list = [0.] - pmdec_list = [0.] - rv_list = [0.] + pmra_list = [0.0] + pmdec_list = [0.0] + rv_list = [0.0] parallax_list = [1e-9] sat_x = 5227.7501 sat_y = -1521.2218 @@ -26,8 +24,7 @@ class TestAstrometry(unittest.TestCase): sat_vx = 0.13581745522969868 sat_vy = 7.233066646238058 sat_vz = -2.5770060087052116 - dt = datetime.utcfromtimestamp( - Time(2461865.75468577, format='jd').unix) + dt = datetime.utcfromtimestamp(Time(2461865.75468577, format="jd").unix) date_str = dt.date().isoformat() time_str = dt.time().isoformat() ra_arr, dec_arr = on_orbit_obs_position( @@ -46,12 +43,12 @@ class TestAstrometry(unittest.TestCase): input_vz=sat_vz, input_epoch="J2000", input_date_str=date_str, - input_time_str=time_str + input_time_str=time_str, ) print(ra_arr[0], dec_arr[0]) self.assertTrue(ra_arr[0] != ra_list[0]) self.assertTrue(dec_arr[0] != dec_list[0]) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_crosstalk.py b/tests/test_crosstalk.py index bafe9fdc0c351e7f24d1343e1ddca4586e1fbc59..8a1e4173b1a36164f6bc1208afcdd8cf989da319 100644 --- a/tests/test_crosstalk.py +++ b/tests/test_crosstalk.py @@ -1,15 +1,8 @@ import unittest -import sys import os -import math -from itertools import islice import numpy as np -import copy -import ctypes import galsim -import yaml -from astropy.io import fits from observation_sim.instruments.chip import chip_utils import matplotlib.pyplot as plt @@ -18,83 +11,366 @@ import matplotlib.pyplot as plt # test FUNCTION --- START # def add_crosstalk(GSimg): crosstalk = np.zeros([16, 16]) - crosstalk[0, :] = np.array([1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[1, :] = np.array([1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[2, :] = np.array([0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[3, :] = np.array([0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[4, :] = np.array([0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[5, :] = np.array([0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[6, :] = np.array([0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[7, :] = np.array([0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0., 0., 0.]) - crosstalk[8, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0., 0., 0.]) - crosstalk[9, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0., 0., 0.]) - crosstalk[10, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0., 0., 0.]) - crosstalk[11, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0., 0., 0.]) - crosstalk[12, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4, 0., 0.]) - crosstalk[13, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1., 0., 0.]) - crosstalk[14, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1e-4]) - crosstalk[15, :] = np.array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1e-4, 1.]) + crosstalk[0, :] = np.array([ + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[1, :] = np.array([ + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[2, :] = np.array([ + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[3, :] = np.array([ + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[4, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[5, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[6, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[7, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[8, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[9, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[10, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[11, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + 0.0, + 0.0, + ]) + crosstalk[12, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + 0.0, + 0.0, + ]) + crosstalk[13, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + 0.0, + 0.0, + ]) + crosstalk[14, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1.0, + 1e-4, + ]) + crosstalk[15, :] = np.array([ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 1e-4, + 1.0, + ]) # 2*8 -> 1*16 img = chip_utils.formatOutput(GSimg) ny, nx = img.array.shape nsecy = 1 nsecx = 16 - dy = int(ny/nsecy) - dx = int(nx/nsecx) + dy = int(ny / nsecy) + dx = int(nx / nsecx) newimg = galsim.Image(nx, ny, init_value=0) for i in range(16): for j in range(16): - newimg.array[:, int(i*dx):int(i*dx+dx)] += crosstalk[i, j]*img.array[:, int(j*dx):int(j*dx+dx)] + newimg.array[:, int(i * dx) : int(i * dx + dx)] += ( + crosstalk[i, j] * img.array[:, int(j * dx) : int(j * dx + dx)] + ) # 1*16 -> 2*8 newimg = chip_utils.formatRevert(newimg) return newimg + + # test FUNCTION --- END # class detModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(detModule_coverage, self).__init__(methodName) self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) def test_add_crosstalk(self): nsecy = 2 nsecx = 8 ny, nx = 1024, 1024 - dy = int(ny/nsecy) - dx = int(nx/nsecx) - mapclip = np.zeros([dy, int(nsecx*nsecy*dx)]) - for i in range(int(nsecx*nsecy)): - mapclip[:, i*dx:dx+i*dx] = np.random.randint(10)+np.random.rand(int(dy*dx)).reshape([dy, dx]) + dy = int(ny / nsecy) + dx = int(nx / nsecx) + mapclip = np.zeros([dy, int(nsecx * nsecy * dx)]) + for i in range(int(nsecx * nsecy)): + mapclip[:, i * dx : dx + i * dx] = np.random.randint(10) + np.random.rand( + int(dy * dx) + ).reshape([dy, dx]) mapclip = galsim.ImageF(mapclip) nsecy = 1 nsecx = 16 ny, nx = mapclip.array.shape - dy = int(ny/nsecy) - dx = int(nx/nsecx) - for i in range(int(nsecy*nsecx)): + dy = int(ny / nsecy) + dx = int(nx / nsecx) + for i in range(int(nsecy * nsecx)): gal = galsim.Gaussian(sigma=0.2, flux=500000).drawImage(nx=32, ny=32).array - py = np.random.randint(450)+10 - mapclip.array[py:py+32, int(i*dx)+10:int(i*dx)+42] += gal + py = np.random.randint(450) + 10 + mapclip.array[py : py + 32, int(i * dx) + 10 : int(i * dx) + 42] += gal tmap = chip_utils.formatRevert(mapclip, nsecy=1, nsecx=16) # 1*16 -> 2*8 temp = add_crosstalk(tmap) fig = plt.figure(figsize=(20, 60)) ax = plt.subplot(311) - plt.imshow(np.log10(mapclip.array+1), origin='lower', cmap='gray') + plt.imshow(np.log10(mapclip.array + 1), origin="lower", cmap="gray") ax = plt.subplot(312) - plt.imshow(np.log10(temp.array+1), origin='lower', cmap='gray') + plt.imshow(np.log10(temp.array + 1), origin="lower", cmap="gray") ax = plt.subplot(313) - plt.imshow(np.log10(temp.array-tmap.array+1), origin='lower', cmap='gray') - plt.savefig(os.path.join(self.dataPath, "./test_crosstalk.png"), dpi=300, bbox_inches='tight') + plt.imshow(np.log10(temp.array - tmap.array + 1), origin="lower", cmap="gray") + plt.savefig( + os.path.join(self.dataPath, "./test_crosstalk.png"), + dpi=300, + bbox_inches="tight", + ) self.assertTrue(True) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_darknoise_func.py b/tests/test_darknoise_func.py index 58afd458d6a562d1ce19c0031629d11812a41ec8..1f3eaae24cc490ccb6218d951797b73d59156d74 100644 --- a/tests/test_darknoise_func.py +++ b/tests/test_darknoise_func.py @@ -1,9 +1,6 @@ import unittest -import sys import os -import math -from itertools import islice import numpy as np import galsim import yaml @@ -13,24 +10,28 @@ from observation_sim.instruments import Chip, Filter, FilterParam, FocalPlane # test FUNCTION --- START ### -def get_base_img(img, chip, read_noise, readout_time, dark_noise, exptime=150., InputDark=None): +def get_base_img( + img, chip, read_noise, readout_time, dark_noise, exptime=150.0, InputDark=None +): if InputDark is None: # base_level = read_noise**2 + dark_noise*(exptime+0.5*readout_time) # base_level = dark_noise*(exptime+0.5*readout_time) - base_level = dark_noise*(exptime) + base_level = dark_noise * (exptime) base_img1 = base_level * np.ones_like(img.array) else: base_img1 = np.zeros_like(img.array) - ny = int(chip.npix_y/2) + ny = int(chip.npix_y / 2) nx = chip.npix_x arr = np.arange(ny).reshape(ny, 1) arr = np.broadcast_to(arr, (ny, nx)) base_img2 = np.zeros_like(img.array) base_img2[:ny, :] = arr base_img2[ny:, :] = arr[::-1, :] - base_img2[:, :] = base_img2[:, :]*(readout_time/ny)*dark_noise - return base_img1+base_img2 + base_img2[:, :] = base_img2[:, :] * (readout_time / ny) * dark_noise + return base_img1 + base_img2 + + # test FUNCTION --- END ### @@ -46,7 +47,8 @@ def defineCCD(iccd, config_file): chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) focal_plane = FocalPlane(chip_list=[iccd]) chip.img.wcs = focal_plane.getTanWCS( - 192.8595, 27.1283, -113.4333*galsim.degrees, chip.pix_scale) + 192.8595, 27.1283, -113.4333 * galsim.degrees, chip.pix_scale + ) return chip @@ -57,39 +59,68 @@ def defineFilt(chip): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list return bandpass_list class detModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(detModule_coverage, self).__init__(methodName) self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) self.iccd = 1 def test_add_dark(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) bandpass = defineFilt(chip) print(chip.chipID) print(chip.cen_pix_x, chip.cen_pix_y) - exptime = 150. - base_img = get_base_img(img=chip.img, chip=chip, read_noise=chip.read_noise, - readout_time=chip.readout_time, dark_noise=chip.dark_noise, exptime=exptime, InputDark=None) - - ny = int(chip.npix_y/2) - self.assertTrue(np.abs(np.max(base_img) - (exptime*chip.dark_noise + - (ny-1)*(chip.readout_time/ny)*chip.dark_noise)) < 1e-6) + exptime = 150.0 + base_img = get_base_img( + img=chip.img, + chip=chip, + read_noise=chip.read_noise, + readout_time=chip.readout_time, + dark_noise=chip.dark_noise, + exptime=exptime, + InputDark=None, + ) + + ny = int(chip.npix_y / 2) + self.assertTrue( + np.abs( + np.max(base_img) + - ( + exptime * chip.dark_noise + + (ny - 1) * (chip.readout_time / ny) * chip.dark_noise + ) + ) + < 1e-6 + ) self.assertTrue(np.min(base_img) == 3) - base_img = get_base_img(img=chip.img, chip=chip, read_noise=chip.read_noise, - readout_time=chip.readout_time, dark_noise=chip.dark_noise, exptime=150., InputDark="testTag") - self.assertTrue(np.abs(np.max(base_img) - ((ny-1) * - (chip.readout_time/ny)*chip.dark_noise)) < 1e-6) - - -if __name__ == '__main__': + base_img = get_base_img( + img=chip.img, + chip=chip, + read_noise=chip.read_noise, + readout_time=chip.readout_time, + dark_noise=chip.dark_noise, + exptime=150.0, + InputDark="testTag", + ) + self.assertTrue( + np.abs( + np.max(base_img) + - ((ny - 1) * (chip.readout_time / ny) * chip.dark_noise) + ) + < 1e-6 + ) + + +if __name__ == "__main__": unittest.main() diff --git a/tests/test_effect_unit.py b/tests/test_effect_unit.py index 2a3bc46b9a029f8f767aa17ca9ae3be3b35dd6ba..35a00406f8ecee12004c1c20749fc4c6674a1046 100644 --- a/tests/test_effect_unit.py +++ b/tests/test_effect_unit.py @@ -4,74 +4,75 @@ from observation_sim.instruments.chip import effects import galsim import matplotlib.pyplot as plt import os -import sys -import math import copy from numpy.random import Generator, PCG64 import warnings -from astropy.io import fits -warnings.filterwarnings("ignore", '.*Numba.*',) +warnings.filterwarnings( + "ignore", + ".*Numba.*", +) width = 9216 height = 9232 class DetTest(unittest.TestCase): - - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(DetTest, self).__init__(methodName) - self.filePath('csst_msc_sim/test_sls_and_straylight') + self.filePath("csst_msc_sim/test_sls_and_straylight") def filePath(self, file_name): - self.datafn = os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), file_name) - self.outDataFn = os.path.join(self.datafn, 'output') + self.datafn = os.path.join(os.getenv("UNIT_TEST_DATA_ROOT"), file_name) + self.outDataFn = os.path.join(self.datafn, "output") if os.path.isdir(self.outDataFn): pass else: os.mkdir(self.outDataFn) def test_prnu(self): - ''' + """ Unit test for PRNU. Expected result: a randomized GS image contains PRNU with sigma=0.01, mean=1. - ''' - print('PRNU Test:') + """ + print("PRNU Test:") sigma = 0.01 seed = 20210911 prnuimg = effects.PRNU_Img(width, height, sigma=sigma, seed=seed) meanval, stdval = np.mean(prnuimg.array), np.std(prnuimg.array) - print(' Mean & STDDEV of PRNU image are %6.4f & %6.4f.' % - (meanval, stdval)) - print(' PRNU Image Array:') - print(' ', prnuimg.array) - self.assertTrue(np.abs(meanval-1) < 1e-6) - self.assertTrue(np.abs(stdval-sigma) < 0.002) - print('\nUnit test for PRNU has been passed.') + print(" Mean & STDDEV of PRNU image are %6.4f & %6.4f." % (meanval, stdval)) + print(" PRNU Image Array:") + print(" ", prnuimg.array) + self.assertTrue(np.abs(meanval - 1) < 1e-6) + self.assertTrue(np.abs(stdval - sigma) < 0.002) + print("\nUnit test for PRNU has been passed.") del prnuimg def test_dark(self): - ''' + """ Test add dark current to image. Expected result: an image with dark current 3.4 e- and noise=1.844 e-. - ''' + """ rng_poisson = galsim.BaseDeviate(20210911) dark_noise = galsim.DeviateNoise( - galsim.PoissonDeviate(rng_poisson, 0.02*(150+0.5*40))) + galsim.PoissonDeviate(rng_poisson, 0.02 * (150 + 0.5 * 40)) + ) img = galsim.Image(200, 200, dtype=np.float32, init_value=0) - print('Initial Mean & STD = %6.3f & %6.3f' % - (np.mean(img.array), np.std(img.array))) + print( + "Initial Mean & STD = %6.3f & %6.3f" + % (np.mean(img.array), np.std(img.array)) + ) img.addNoise(dark_noise) meanval = np.mean(img.array) stdval = np.std(img.array) - print('Dark added Mean & STD = %6.3f & %6.3f' % (meanval, stdval)) - self.assertTrue(np.abs(meanval-3.4) < 0.05) - self.assertTrue(np.abs(stdval-1.844) < 0.02) - print('\nUnit test for dark current has been passed.') + print("Dark added Mean & STD = %6.3f & %6.3f" % (meanval, stdval)) + self.assertTrue(np.abs(meanval - 3.4) < 0.05) + self.assertTrue(np.abs(stdval - 1.844) < 0.02) + print("\nUnit test for dark current has been passed.") del img def test_satu(self): - ''' + """ Test saturation and bleeding. Expected result: an image with bleeding effect. - ''' + """ img = galsim.Image(500, 500, dtype=np.float32) star = galsim.Gaussian(flux=60e5, fwhm=3) img = star.drawImage(image=img, center=(150, 200)) @@ -80,77 +81,99 @@ class DetTest(unittest.TestCase): img.addNoise(galsim.GaussianNoise(sigma=7)) # plt.imshow(img.array) # plt.show() - filename1 = os.path.join(self.outDataFn, 'test_satu_initimg.fits') + filename1 = os.path.join(self.outDataFn, "test_satu_initimg.fits") img.write(filename1) newimg = effects.SaturBloom(img, fullwell=9e4) # plt.imshow(newimg.array) # plt.show() - filename2 = os.path.join(self.outDataFn, 'test_satu_bleedimg.fits') + filename2 = os.path.join(self.outDataFn, "test_satu_bleedimg.fits") newimg.write(filename2) del img, newimg, star def test_nonlinear(self): - ''' + """ Test non-linear effect. Expected result: an image with non-linearity effect. - ''' + """ imgarr = np.arange(1, 9e4, 4).reshape((150, 150)) img = galsim.Image(copy.deepcopy(imgarr)) - filename1 = os.path.join(self.outDataFn, 'test_nonlinear_initimg.fits') + filename1 = os.path.join(self.outDataFn, "test_nonlinear_initimg.fits") img.write(filename1) - newimg = effects.NonLinearity(img, beta1=5E-7, beta2=0) - filename2 = os.path.join( - self.outDataFn, 'test_nonlinear_finalimg.fits') + newimg = effects.NonLinearity(img, beta1=5e-7, beta2=0) + filename2 = os.path.join(self.outDataFn, "test_nonlinear_finalimg.fits") newimg.write(filename2) plt.scatter(imgarr.flatten(), newimg.array.flatten(), s=2, alpha=0.5) - plt.plot([-1e3, 9e4], [-1e3, 9e4], color='black', lw=1, ls='--') - plt.xlabel('input (e-)') - plt.ylabel('output (e-)') - plt.savefig(os.path.join(self.outDataFn, - 'test_nonlinearity.png'), dpi=200) + plt.plot([-1e3, 9e4], [-1e3, 9e4], color="black", lw=1, ls="--") + plt.xlabel("input (e-)") + plt.ylabel("output (e-)") + plt.savefig(os.path.join(self.outDataFn, "test_nonlinearity.png"), dpi=200) plt.show() del img, newimg, imgarr def test_badpixel_HtrDtr(self): img = galsim.Image(500, 500, init_value=1000) rgbadpix = Generator(PCG64(20210911)) - badfraction = 5E-5*(rgbadpix.random()*0.5+0.7) + badfraction = 5e-5 * (rgbadpix.random() * 0.5 + 0.7) img = effects.DefectivePixels( - img, IfHotPix=True, IfDeadPix=True, fraction=badfraction, seed=20210911, biaslevel=0) - img.write(os.path.join(self.outDataFn, 'test_badpixel_HtrDtr.fits')) + img, + IfHotPix=True, + IfDeadPix=True, + fraction=badfraction, + seed=20210911, + biaslevel=0, + ) + img.write(os.path.join(self.outDataFn, "test_badpixel_HtrDtr.fits")) del img def test_badpixel_HfsDtr(self): img = galsim.Image(500, 500, init_value=1000) rgbadpix = Generator(PCG64(20210911)) - badfraction = 5E-5*(rgbadpix.random()*0.5+0.7) + badfraction = 5e-5 * (rgbadpix.random() * 0.5 + 0.7) img = effects.DefectivePixels( - img, IfHotPix=False, IfDeadPix=True, fraction=badfraction, seed=20210911, biaslevel=0) - img.write(os.path.join(self.outDataFn, 'test_badpixel_HfsDtr.fits')) + img, + IfHotPix=False, + IfDeadPix=True, + fraction=badfraction, + seed=20210911, + biaslevel=0, + ) + img.write(os.path.join(self.outDataFn, "test_badpixel_HfsDtr.fits")) del img def test_badpixel_HtrDfs(self): img = galsim.Image(500, 500, init_value=1000) rgbadpix = Generator(PCG64(20210911)) - badfraction = 5E-5*(rgbadpix.random()*0.5+0.7) + badfraction = 5e-5 * (rgbadpix.random() * 0.5 + 0.7) img = effects.DefectivePixels( - img, IfHotPix=True, IfDeadPix=False, fraction=badfraction, seed=20210911, biaslevel=0) - img.write(os.path.join(self.outDataFn, 'test_badpixel_HtrDfs.fits')) + img, + IfHotPix=True, + IfDeadPix=False, + fraction=badfraction, + seed=20210911, + biaslevel=0, + ) + img.write(os.path.join(self.outDataFn, "test_badpixel_HtrDfs.fits")) del img def test_badpixel_HfsDfs(self): img = galsim.Image(500, 500, init_value=1000) rgbadpix = Generator(PCG64(20210911)) - badfraction = 5E-5*(rgbadpix.random()*0.5+0.7) + badfraction = 5e-5 * (rgbadpix.random() * 0.5 + 0.7) img = effects.DefectivePixels( - img, IfHotPix=False, IfDeadPix=False, fraction=badfraction, seed=20210911, biaslevel=0) - img.write(os.path.join(self.outDataFn, 'test_badpixel_HfsDfs.fits')) + img, + IfHotPix=False, + IfDeadPix=False, + fraction=badfraction, + seed=20210911, + biaslevel=0, + ) + img.write(os.path.join(self.outDataFn, "test_badpixel_HfsDfs.fits")) del img def test_badlines(self): img = galsim.Image(500, 500, init_value=-1000) img.addNoise(galsim.GaussianNoise(sigma=7)) newimg = effects.BadColumns(copy.deepcopy(img), seed=20210911) - newimg.write(os.path.join(self.outDataFn, 'test_badlines.fits')) + newimg.write(os.path.join(self.outDataFn, "test_badlines.fits")) del newimg, img # def test_cte(self): @@ -169,52 +192,59 @@ class DetTest(unittest.TestCase): rng_readout = galsim.BaseDeviate(seed) readout_noise = galsim.GaussianNoise(rng=rng_readout, sigma=5) img.addNoise(readout_noise) - img.write(os.path.join(self.outDataFn, 'test_readnoise.fits')) + img.write(os.path.join(self.outDataFn, "test_readnoise.fits")) stdval = np.std(img.array) - self.assertTrue(np.abs(stdval-5) < 0.01*5) - print('\nUnit test for readout noise has been passed.') + self.assertTrue(np.abs(stdval - 5) < 0.01 * 5) + print("\nUnit test for readout noise has been passed.") del img def test_addbias(self): img = galsim.Image(200, 200, init_value=0) img = effects.AddBiasNonUniform16( - img, bias_level=500, nsecy=2, nsecx=8, seed=20210911) - img.write('./output/test_addbias.fits') + img, bias_level=500, nsecy=2, nsecx=8, seed=20210911 + ) + img.write("./output/test_addbias.fits") del img def test_apply16gains(self): img = galsim.Image(500, 500, init_value=100) img, _ = effects.ApplyGainNonUniform16( - img, gain=1.5, nsecy=2, nsecx=8, seed=202102) - img.write(os.path.join(self.outDataFn, 'test_apply16gains.fits')) - rightedge = int(500/8)*8 - print('gain=%6.2f' % 1.5) + img, gain=1.5, nsecy=2, nsecx=8, seed=202102 + ) + img.write(os.path.join(self.outDataFn, "test_apply16gains.fits")) + rightedge = int(500 / 8) * 8 + print("gain=%6.2f" % 1.5) meanimg = np.mean(img.array[:, :rightedge]) sigmaimg = np.std(img.array[:, :rightedge]) - print('mean, sigma = %6.2f, %6.2f' % (meanimg, sigmaimg)) - self.assertTrue(np.abs(meanimg-100/1.5) < 1) - self.assertTrue(np.abs(sigmaimg/meanimg-0.01) < 0.001) - print('\nUnit test for applying 16 channel gains has been passed.') + print("mean, sigma = %6.2f, %6.2f" % (meanimg, sigmaimg)) + self.assertTrue(np.abs(meanimg - 100 / 1.5) < 1) + self.assertTrue(np.abs(sigmaimg / meanimg - 0.01) < 0.001) + print("\nUnit test for applying 16 channel gains has been passed.") del img def test_cosmicray(self): - attachedSizes = np.loadtxt(os.path.join( - self.datafn, 'wfc-cr-attachpixel.dat')) + attachedSizes = np.loadtxt(os.path.join(self.datafn, "wfc-cr-attachpixel.dat")) cr_map, _ = effects.produceCR_Map( - xLen=500, yLen=500, exTime=150+0.5*40, - cr_pixelRatio=0.003*(1+0.5*40/150), - gain=1, attachedSizes=attachedSizes, seed=20210911) + xLen=500, + yLen=500, + exTime=150 + 0.5 * 40, + cr_pixelRatio=0.003 * (1 + 0.5 * 40 / 150), + gain=1, + attachedSizes=attachedSizes, + seed=20210911, + ) crimg = galsim.Image(cr_map) - crimg.write(os.path.join(self.outDataFn, 'test_cosmicray.fits')) + crimg.write(os.path.join(self.outDataFn, "test_cosmicray.fits")) del cr_map, crimg def test_shutter(self): img = galsim.Image(5000, 5000, init_value=1000) # shutter effect normalized image for this chip shuttimg = effects.ShutterEffectArr( - img, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1E-3) + img, t_exp=150, t_shutter=1.3, dist_bearing=735, dt=1e-3 + ) img *= shuttimg - img.write(os.path.join(self.outDataFn, 'test_shutter.fits')) + img.write(os.path.join(self.outDataFn, "test_shutter.fits")) del img def test_vignette(self): @@ -224,9 +254,9 @@ class DetTest(unittest.TestCase): img.setOrigin(10000, 10000) flat_img = effects.MakeFlatSmooth(img.bounds, 20210911) flat_normal = flat_img / np.mean(flat_img.array) - flat_normal.write(os.path.join(self.outDataFn, 'test_vignette.fits')) + flat_normal.write(os.path.join(self.outDataFn, "test_vignette.fits")) del flat_img, img, flat_normal -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_field_distortion.py b/tests/test_field_distortion.py index 0f77edbe4062a24d36f90bb07963045b700b4bc9..73c9d8407f0db72e7a8d95153ce8ccb6dad8de96 100644 --- a/tests/test_field_distortion.py +++ b/tests/test_field_distortion.py @@ -3,7 +3,6 @@ import unittest import numpy as np import galsim import os -import sys from astropy.table import Table from scipy import interpolate import pickle diff --git a/tests/test_focalplane.py b/tests/test_focalplane.py index 74ab61b7ece12f2aa1ca15d3f888f437f7e84bd1..7756bf2b0fe0d1b0cc3b0fb699066b54721be1f3 100644 --- a/tests/test_focalplane.py +++ b/tests/test_focalplane.py @@ -1,31 +1,29 @@ import unittest import os import galsim -from observation_sim.instruments import FocalPlane, Chip +from observation_sim.instruments import FocalPlane class TestFocalPlane(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(TestFocalPlane, self).__init__(methodName) self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') - self.focal_plane = FocalPlane( - chip_list=['8']) + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) + self.focal_plane = FocalPlane(chip_list=["8"]) self.assertTrue(self.focal_plane.cen_pix_x == 0) self.assertTrue(self.focal_plane.cen_pix_y == 0) - test_focal_plane_phot = FocalPlane(survey_type='Photometric') - test_focal_plane_spec = FocalPlane(survey_type='Spectroscopic') - test_focal_plane_FGS = FocalPlane(survey_type='FGS') + test_focal_plane_phot = FocalPlane(survey_type="Photometric") + test_focal_plane_spec = FocalPlane(survey_type="Spectroscopic") + test_focal_plane_FGS = FocalPlane(survey_type="FGS") test_focal_plane_bad_chips = FocalPlane(bad_chips=["1"]) def test_fp_method(self): - wcs = self.focal_plane.getTanWCS( - 192.8595, 0., 0.*galsim.degrees, 0.074) - sky_coverage = self.focal_plane.getSkyCoverage( - wcs, x0=-1, x1=0, y0=-1, y1=0) + wcs = self.focal_plane.getTanWCS(192.8595, 0.0, 0.0 * galsim.degrees, 0.074) + sky_coverage = self.focal_plane.getSkyCoverage(wcs, x0=-1, x1=0, y0=-1, y1=0) print(sky_coverage.area()) - self.assertTrue(abs(sky_coverage.area() - 0.074**2/(3600.**2)) < 1e13) + self.assertTrue(abs(sky_coverage.area() - 0.074**2 / (3600.0**2)) < 1e13) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_imaging.py b/tests/test_imaging.py index 5ba5cdc67d4b4d8b272a03d4a530f8e4905ff8c4..261ede44dbb55b26baf7d3e214033646a592511f 100644 --- a/tests/test_imaging.py +++ b/tests/test_imaging.py @@ -4,62 +4,67 @@ from scipy import interpolate import astropy.constants as cons from astropy.table import Table import h5py as h5 -import sys import os -import math -from itertools import islice import numpy as np import galsim import yaml -import copy from astropy.cosmology import FlatLambdaCDM -from astropy import constants -from astropy import units as U from observation_sim.mock_objects._util import getObservedSED from observation_sim.mock_objects import CatalogBase, Galaxy from observation_sim.instruments import Chip, Filter, FilterParam, FocalPlane -from observation_sim.psf.PSFInterp import PSFInterp -from observation_sim.mock_objects._util import integrate_sed_bandpass, getNormFactorForSpecWithABMAG, getABMAG +from observation_sim.mock_objects._util import ( + integrate_sed_bandpass, +) class Catalog(CatalogBase): def __init__(self): super().__init__() - self.rotation = 0. + self.rotation = 0.0 def load_norm_filt(self, obj): return None def load_sed(self, obj, **kward): - pcs = h5.File(os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), - 'csst_msc_sim/csst_fz_msc/sedlibs/pcs.h5'), "r") - lamb = h5.File(os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), - 'csst_msc_sim/csst_fz_msc/sedlibs/lamb.h5'), "r") - lamb_gal = lamb['lamb'][()] - pcs = pcs['pcs'][()] + pcs = h5.File( + os.path.join( + os.getenv("UNIT_TEST_DATA_ROOT"), + "csst_msc_sim/csst_fz_msc/sedlibs/pcs.h5", + ), + "r", + ) + lamb = h5.File( + os.path.join( + os.getenv("UNIT_TEST_DATA_ROOT"), + "csst_msc_sim/csst_fz_msc/sedlibs/lamb.h5", + ), + "r", + ) + lamb_gal = lamb["lamb"][()] + pcs = pcs["pcs"][()] cosmo = FlatLambdaCDM(H0=67.66, Om0=0.3111) - factor = 10**(-.4 * cosmo.distmod(obj.param['z']).value) - flux = np.matmul(pcs, obj.param['coeff']) * factor + factor = 10 ** (-0.4 * cosmo.distmod(obj.param["z"]).value) + flux = np.matmul(pcs, obj.param["coeff"]) * factor # if np.any(flux < 0): # raise ValueError("Glaxy %s: negative SED fluxes"%obj.id) - flux[flux < 0] = 0. + flux[flux < 0] = 0.0 sedcat = np.vstack((lamb_gal, flux)).T sed_data = getObservedSED( sedCat=sedcat, - redshift=obj.param['z'], + redshift=obj.param["z"], av=obj.param["av"], - redden=obj.param["redden"] + redden=obj.param["redden"], ) wave, flux = sed_data[0], sed_data[1] speci = interpolate.interp1d(wave, flux) - lamb = np.arange(2000, 11001+0.5, 0.5) + lamb = np.arange(2000, 11001 + 0.5, 0.5) y = speci(lamb) # erg/s/cm2/A --> photon/s/m2/A all_sed = y * lamb / (cons.h.value * cons.c.value) * 1e-13 - sed = Table(np.array([lamb, all_sed]).T, names=('WAVELENGTH', 'FLUX')) + sed = Table(np.array([lamb, all_sed]).T, names=("WAVELENGTH", "FLUX")) # obj.param["sed"] = sed del wave @@ -67,57 +72,57 @@ class Catalog(CatalogBase): return sed def _load(self, file_path): - gals_cat = h5.File(file_path, 'r')['galaxies'] + gals_cat = h5.File(file_path, "r")["galaxies"] for ikeys in gals_cat.keys(): gals = gals_cat[ikeys] param = self.initialize_param() igals = 9 - param['z'] = gals['redshift'][igals] - param['mag_use_normal'] = gals['mag_csst_g'][igals] - print(param['mag_use_normal']) + param["z"] = gals["redshift"][igals] + param["mag_use_normal"] = gals["mag_csst_g"][igals] + print(param["mag_use_normal"]) - param['e1'] = gals['ellipticity_true'][igals][0] - param['e2'] = gals['ellipticity_true'][igals][1] + param["e1"] = gals["ellipticity_true"][igals][0] + param["e2"] = gals["ellipticity_true"][igals][1] # For shape calculation - param['e1'], param['e2'], param['ell_total'] = self.rotate_ellipticity( - e1=gals['ellipticity_true'][igals][0], - e2=gals['ellipticity_true'][igals][1], + param["e1"], param["e2"], param["ell_total"] = self.rotate_ellipticity( + e1=gals["ellipticity_true"][igals][0], + e2=gals["ellipticity_true"][igals][1], rotation=self.rotation, - unit='radians') + unit="radians", + ) - param['e1_disk'] = param['e1'] - param['e2_disk'] = param['e2'] - param['e1_bulge'] = param['e1'] - param['e2_bulge'] = param['e2'] + param["e1_disk"] = param["e1"] + param["e2_disk"] = param["e2"] + param["e1_bulge"] = param["e1"] + param["e2_bulge"] = param["e2"] # Masses - param['bulgemass'] = gals['bulgemass'][igals] - param['diskmass'] = gals['diskmass'][igals] + param["bulgemass"] = gals["bulgemass"][igals] + param["diskmass"] = gals["diskmass"][igals] - param['size'] = gals['size'][igals] + param["size"] = gals["size"][igals] # Sersic index - param['disk_sersic_idx'] = 1. - param['bulge_sersic_idx'] = 4. + param["disk_sersic_idx"] = 1.0 + param["bulge_sersic_idx"] = 4.0 # Sizes - param['bfrac'] = param['bulgemass'] / \ - (param['bulgemass'] + param['diskmass']) - if param['bfrac'] >= 0.6: - param['hlr_bulge'] = param['size'] - param['hlr_disk'] = param['size'] * (1. - param['bfrac']) + param["bfrac"] = param["bulgemass"] / (param["bulgemass"] + param["diskmass"]) + if param["bfrac"] >= 0.6: + param["hlr_bulge"] = param["size"] + param["hlr_disk"] = param["size"] * (1.0 - param["bfrac"]) else: - param['hlr_disk'] = param['size'] - param['hlr_bulge'] = param['size'] * param['bfrac'] + param["hlr_disk"] = param["size"] + param["hlr_bulge"] = param["size"] * param["bfrac"] # SED coefficients - param['coeff'] = gals['coeff'][igals] + param["coeff"] = gals["coeff"][igals] # TEST no redening and no extinction - param['av'] = 0.0 - param['redden'] = 0 + param["av"] = 0.0 + param["redden"] = 0 obj = Galaxy(param) @@ -136,7 +141,8 @@ def defineCCD(iccd, config_file): chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) focal_plane = FocalPlane(chip_list=[iccd]) chip.img.wcs = focal_plane.getTanWCS( - 192.8595, 27.1283, -113.4333*galsim.degrees, chip.pix_scale) + 192.8595, 27.1283, -113.4333 * galsim.degrees, chip.pix_scale + ) return chip @@ -147,19 +153,21 @@ def defineFilt(chip): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) return filt class imagingModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(imagingModule_coverage, self).__init__(methodName) self.dataPath = os.path.join( - os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) self.iccd = 8 def test_imaging(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) bandpass = defineFilt(chip) filt = defineFilt(chip) @@ -168,8 +176,7 @@ class imagingModule_coverage(unittest.TestCase): catalog = Catalog() - obj = catalog._load(os.path.join( - self.dataPath, 'galaxies_C6_bundle000287.h5')) + obj = catalog._load(os.path.join(self.dataPath, "galaxies_C6_bundle000287.h5")) sed_data = catalog.load_sed(obj) norm_filt = catalog.load_norm_filt(obj) @@ -180,22 +187,25 @@ class imagingModule_coverage(unittest.TestCase): norm_filt=norm_filt, ) - pupil_area = np.pi * (0.5 * 2.)**2 - exptime = 150. + pupil_area = np.pi * (0.5 * 2.0) ** 2 + exptime = 150.0 # getElectronFluxFilt(filt, tel, exptime) - nphotons_tot = obj_flux*pupil_area * exptime + nphotons_tot = obj_flux * pupil_area * exptime full = integrate_sed_bandpass(sed=obj_sed, bandpass=filt.bandpass_full) print(full, nphotons_tot, obj_mag) for i in range(4): sub = integrate_sed_bandpass( - sed=obj_sed, bandpass=filt.bandpass_sub_list[i]) + sed=obj_sed, bandpass=filt.bandpass_sub_list[i] + ) ratio = sub / full nphotons = ratio * nphotons_tot disk = galsim.Sersic( - n=obj.param['disk_sersic_idx'], half_light_radius=obj.param['hlr_disk'], flux=1.0) - disk_shape = galsim.Shear( - g1=obj.param['e1_disk'], g2=obj.param['e2_disk']) + n=obj.param["disk_sersic_idx"], + half_light_radius=obj.param["hlr_disk"], + flux=1.0, + ) + disk_shape = galsim.Shear(g1=obj.param["e1_disk"], g2=obj.param["e2_disk"]) disk = disk.shear(disk_shape) gal_temp = disk gal_temp = gal_temp.withFlux(nphotons) @@ -212,5 +222,5 @@ class imagingModule_coverage(unittest.TestCase): self.assertTrue(gal is not None) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/tests/test_prescan_overscan_func.py b/tests/test_prescan_overscan_func.py index 01ec75ffd7bc66e721387256a88add2458ea982b..461d0b443537e9d460f94227357f39497d586a42 100644 --- a/tests/test_prescan_overscan_func.py +++ b/tests/test_prescan_overscan_func.py @@ -1,9 +1,6 @@ import unittest -import sys import os -import math -from itertools import islice import numpy as np import galsim import yaml @@ -15,42 +12,67 @@ from observation_sim.instruments import Chip, Filter, FilterParam, FocalPlane def AddPreScan(GSImage, pre1=27, pre2=4, over1=71, over2=80, nsecy=2, nsecx=8): img = GSImage.array ny, nx = img.shape - dx = int(nx/nsecx) - dy = int(ny/nsecy) - - imgt = np.zeros([int(nsecy*nsecx), int(ny/nsecy+pre2+over2), int(nx/nsecx+pre1+over1)]) + dx = int(nx / nsecx) + dy = int(ny / nsecy) + + imgt = np.zeros([ + int(nsecy * nsecx), + int(ny / nsecy + pre2 + over2), + int(nx / nsecx + pre1 + over1), + ]) for iy in range(nsecy): for ix in range(nsecx): if iy % 2 == 0: tx = ix else: - tx = (nsecx-1)-ix + tx = (nsecx - 1) - ix ty = iy - chunkidx = int(tx+ty*nsecx) # chunk1-[1,2,3,4], chunk2-[5,6,7,8], chunk3-[9,10,11,12], chunk4-[13,14,15,16] - - imgtemp = np.zeros([int(ny/nsecy+pre2+over2), int(nx/nsecx+pre1+over1)]) - if int(chunkidx/4) == 0: - imgtemp[pre2:pre2+dy, pre1:pre1+dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + chunkidx = int( + tx + ty * nsecx + ) # chunk1-[1,2,3,4], chunk2-[5,6,7,8], chunk3-[9,10,11,12], chunk4-[13,14,15,16] + + imgtemp = np.zeros([ + int(ny / nsecy + pre2 + over2), + int(nx / nsecx + pre1 + over1), + ]) + if int(chunkidx / 4) == 0: + imgtemp[pre2 : pre2 + dy, pre1 : pre1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp - if int(chunkidx/4) == 1: - imgtemp[pre2:pre2+dy, over1:over1+dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 1: + imgtemp[pre2 : pre2 + dy, over1 : over1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [:, ::-1] - if int(chunkidx/4) == 2: - imgtemp[over2:over2+dy, over1:over1+dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 2: + imgtemp[over2 : over2 + dy, over1 : over1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [::-1, ::-1] - if int(chunkidx/4) == 3: - imgtemp[over2:over2+dy, pre1:pre1+dx] = img[iy*dy:(iy+1)*dy, ix*dx:(ix+1)*dx] + if int(chunkidx / 4) == 3: + imgtemp[over2 : over2 + dy, pre1 : pre1 + dx] = img[ + iy * dy : (iy + 1) * dy, ix * dx : (ix + 1) * dx + ] imgt[chunkidx, :, :] = imgtemp # [::-1, :] imgtx1 = np.hstack(imgt[:nsecx:, :, :]) # hstack chunk(1,2)-[1,2,3,4,5,6,7,8] - imgtx2 = np.hstack(imgt[:(nsecx-1):-1, :, :]) # hstack chunk(4,3)-[16,15,14,13,12,11,,10,9] - - newimg = galsim.Image(int(nx+(pre1+over1)*nsecx), int(ny+(pre2+over2)*nsecy), init_value=0) - newimg.array[:, :] = np.concatenate([imgtx1, imgtx2]) # vstack chunk(1,2) & chunk(4,3) + imgtx2 = np.hstack( + imgt[: (nsecx - 1) : -1, :, :] + ) # hstack chunk(4,3)-[16,15,14,13,12,11,,10,9] + + newimg = galsim.Image( + int(nx + (pre1 + over1) * nsecx), int(ny + (pre2 + over2) * nsecy), init_value=0 + ) + newimg.array[:, :] = np.concatenate([ + imgtx1, + imgtx2, + ]) # vstack chunk(1,2) & chunk(4,3) newimg.wcs = GSImage.wcs return newimg + # test FUNCTION --- END # @@ -65,7 +87,9 @@ def defineCCD(iccd, config_file): chip = Chip(chipID=iccd, config=config) chip.img = galsim.ImageF(chip.npix_x, chip.npix_y) focal_plane = FocalPlane(chip_list=[iccd]) - chip.img.wcs = focal_plane.getTanWCS(192.8595, 27.1283, -113.4333*galsim.degrees, chip.pix_scale) + chip.img.wcs = focal_plane.getTanWCS( + 192.8595, 27.1283, -113.4333 * galsim.degrees, chip.pix_scale + ) return chip @@ -76,33 +100,44 @@ def defineFilt(chip): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list return bandpass_list class detModule_coverage(unittest.TestCase): - def __init__(self, methodName='runTest'): + def __init__(self, methodName="runTest"): super(detModule_coverage, self).__init__(methodName) - self.dataPath = os.path.join(os.getenv('UNIT_TEST_DATA_ROOT'), 'csst_msc_sim/csst_fz_msc') + self.dataPath = os.path.join( + os.getenv("UNIT_TEST_DATA_ROOT"), "csst_msc_sim/csst_fz_msc" + ) self.iccd = 1 def test_add_prescan_overscan(self): - config_file = os.path.join(self.dataPath, 'config_test.yaml') + config_file = os.path.join(self.dataPath, "config_test.yaml") chip = defineCCD(self.iccd, config_file) bandpass = defineFilt(chip) print(chip.chipID) print(chip.cen_pix_x, chip.cen_pix_y) - chip.img = AddPreScan(GSImage=chip.img, - pre1=chip.prescan_x, - pre2=chip.prescan_y, - over1=chip.overscan_x, - over2=chip.overscan_y) - - self.assertTrue((chip.prescan_x+chip.overscan_x)*8+chip.npix_x == np.shape(chip.img.array)[1]) - self.assertTrue((chip.prescan_y+chip.overscan_y)*2+chip.npix_y == np.shape(chip.img.array)[0]) - - -if __name__ == '__main__': + chip.img = AddPreScan( + GSImage=chip.img, + pre1=chip.prescan_x, + pre2=chip.prescan_y, + over1=chip.overscan_x, + over2=chip.overscan_y, + ) + + self.assertTrue( + (chip.prescan_x + chip.overscan_x) * 8 + chip.npix_x + == np.shape(chip.img.array)[1] + ) + self.assertTrue( + (chip.prescan_y + chip.overscan_y) * 2 + chip.npix_y + == np.shape(chip.img.array)[0] + ) + + +if __name__ == "__main__": unittest.main() diff --git a/tools/create_chip_json.py b/tools/create_chip_json.py index aca80c3ecc1cf2a6ee2eac9b938e8da4927fd857..4596ae02fef4e7432ba6eddd8f4960fc53c562c6 100644 --- a/tools/create_chip_json.py +++ b/tools/create_chip_json.py @@ -1,8 +1,7 @@ import json -import os import shutil -chip_filename = 'chip_definition.json' +chip_filename = "chip_definition.json" # chip_list = {} # chip_id = "39" @@ -35,12 +34,12 @@ def get_chip_center_main_fp(chip_id, pixel_size=1e-2): nchip_x = 6 nchip_y = 5 - xrem = 2*(col - 1) - (nchip_x - 1) - xcen = (npix_x//2 + gx1//2) * xrem + xrem = 2 * (col - 1) - (nchip_x - 1) + xcen = (npix_x // 2 + gx1 // 2) * xrem if chip_id >= 26 or chip_id == 21: - xcen = (npix_x//2 + gx1//2) * xrem - (gx2-gx1) + xcen = (npix_x // 2 + gx1 // 2) * xrem - (gx2 - gx1) if chip_id <= 5 or chip_id == 10: - xcen = (npix_x//2 + gx1//2) * xrem + (gx2-gx1) + xcen = (npix_x // 2 + gx1 // 2) * xrem + (gx2 - gx1) # ylim of a given CCD chip yrem = (row - 1) - nchip_y // 2 @@ -49,32 +48,122 @@ def get_chip_center_main_fp(chip_id, pixel_size=1e-2): def create_chip_dict_main_fp(chip_id, pixel_size=1e-2): - filter_list = ["GV", "GI", "y", "z", "y", "GI", "GU", "r", "u", "NUV", "i", "GV", "GU", "g", - "NUV", "NUV", "g", "GU", "GV", "i", "NUV", "u", "r", "GU", "GI", "y", "z", "y", "GI", "GV"] - chip_label_list = [3, 3, 3, 1, 1, 1, 3, 2, 2, 1, 1, 1, - 4, 2, 3, 2, 1, 1, 4, 2, 4, 1, 1, 2, 4, 2, 2, 4, 2, 2] - chip_id_list = [26, 21, 16, 11, 6, 1, 27, 22, 17, 12, 7, 2, 28, - 23, 18, 13, 8, 3, 29, 24, 19, 14, 9, 4, 30, 25, 20, 15, 10, 5] + filter_list = [ + "GV", + "GI", + "y", + "z", + "y", + "GI", + "GU", + "r", + "u", + "NUV", + "i", + "GV", + "GU", + "g", + "NUV", + "NUV", + "g", + "GU", + "GV", + "i", + "NUV", + "u", + "r", + "GU", + "GI", + "y", + "z", + "y", + "GI", + "GV", + ] + chip_label_list = [ + 3, + 3, + 3, + 1, + 1, + 1, + 3, + 2, + 2, + 1, + 1, + 1, + 4, + 2, + 3, + 2, + 1, + 1, + 4, + 2, + 4, + 1, + 1, + 2, + 4, + 2, + 2, + 4, + 2, + 2, + ] + chip_id_list = [ + 26, + 21, + 16, + 11, + 6, + 1, + 27, + 22, + 17, + 12, + 7, + 2, + 28, + 23, + 18, + 13, + 8, + 3, + 29, + 24, + 19, + 14, + 9, + 4, + 30, + 25, + 20, + 15, + 10, + 5, + ] npix_x = 9216 npix_y = 9232 idx = chip_id_list.index(chip_id) chip_name = filter_list[idx] + "-" + str(chip_label_list[idx]) xcen, ycen = get_chip_center_main_fp(chip_id, pixel_size) row, col = get_chip_row_col_main_fp(chip_id) - rotate_angle = 0. - if filter_list[idx] in ['GU', 'GV', 'GI']: - rotate_angle = 1. + rotate_angle = 0.0 + if filter_list[idx] in ["GU", "GV", "GI"]: + rotate_angle = 1.0 if col > 2: - rotate_angle = -1. * rotate_angle + rotate_angle = -1.0 * rotate_angle chip_dict = { - "chip_name": chip_name, - "pix_size": 1e-2, # [mm] - "pix_scale": 0.074, # [arcsec/pix] - "npix_x": npix_x, - "npix_y": npix_y, - "x_cen": xcen, # [mm] - "y_cen": ycen, # [mm] - "rotate_angle": rotate_angle, # [deg] + "chip_name": chip_name, + "pix_size": 1e-2, # [mm] + "pix_scale": 0.074, # [arcsec/pix] + "npix_x": npix_x, + "npix_y": npix_y, + "x_cen": xcen, # [mm] + "y_cen": ycen, # [mm] + "rotate_angle": rotate_angle, # [deg] "n_psf_samples": 900, "dark_exptime": 300, "flat_exptime": 150, @@ -82,7 +171,7 @@ def create_chip_dict_main_fp(chip_id, pixel_size=1e-2): "df_strength": 2.3, "bias_level": 500, "gain": 1.1, - "full_well": 90000 + "full_well": 90000, } return chip_dict @@ -92,12 +181,12 @@ def set_fgs_chips(filepath): data = json.load(f) for i in range(12): chip_id = str(31 + i) - data[chip_id]["dark_exptime"] = 300. - data[chip_id]["falt_exptime"] = 150. + data[chip_id]["dark_exptime"] = 300.0 + data[chip_id]["falt_exptime"] = 150.0 data[chip_id]["readout_time"] = 0.01 data[chip_id]["df_strength"] = 2.3 - data[chip_id]["bias_level"] = 2000. - data[chip_id]["gain"] = 1. + data[chip_id]["bias_level"] = 2000.0 + data[chip_id]["gain"] = 1.0 data[chip_id]["full_well"] = 90000 with open(filepath, "w") as f: json.dump(data, f, indent=4) @@ -111,7 +200,7 @@ def add_main_fp(filepath): def add_dict_to_json(filepath, key, value): - with open(filepath, 'r') as f: + with open(filepath, "r") as f: data = json.load(f) data[key] = value with open(filepath, "w") as f: diff --git a/tools/get_LED_Img.py b/tools/get_LED_Img.py index 57dd31cdc5adc342733abaf344500af3e5b73168..910eb1b688ea040cb19e683597773741135f22c5 100644 --- a/tools/get_LED_Img.py +++ b/tools/get_LED_Img.py @@ -1,4 +1,3 @@ -import numpy as np from observation_sim.instruments import Chip, Filter, FilterParam import astropy.io.fits as fits from observation_sim.mock_objects import FlatLED @@ -6,24 +5,25 @@ from observation_sim.mock_objects import FlatLED # 输入打开的LED名字列表led_type_list和对应的打开时间列表exp_t_list -def get_LED_Img(chipID=8, led_type_list=['LED1'], exp_t_list=[0.1]): +def get_LED_Img(chipID=8, led_type_list=["LED1"], exp_t_list=[0.1]): chip = Chip(chipID) filter_id, filter_type = chip.getChipFilter() filt = Filter( - filter_id=filter_id, - filter_type=filter_type, - filter_param=FilterParam()) + filter_id=filter_id, filter_type=filter_type, filter_param=FilterParam() + ) led_obj = FlatLED(chip, filt) led_flat, ledstat, letts = led_obj.drawObj_LEDFlat( - led_type_list=led_type_list, exp_t_list=exp_t_list) + led_type_list=led_type_list, exp_t_list=exp_t_list + ) return led_flat if __name__ == "__main__": chipid = 7 - led_type_list = ['LED5', 'LED6', 'LED7', 'LED8'] + led_type_list = ["LED5", "LED6", "LED7", "LED8"] exp_t_list = [0.1, 8, 9, 10] led_img = get_LED_Img( - chipID=chipid, led_type_list=led_type_list, exp_t_list=exp_t_list) - fits.writeto('test_led.fits', led_img) + chipID=chipid, led_type_list=led_type_list, exp_t_list=exp_t_list + ) + fits.writeto("test_led.fits", led_img) diff --git a/tools/get_PSF.py b/tools/get_PSF.py index de711c9267ad7e2e509aab15b54f52e8fd43c73c..682fc149936768abdeefec59890ca5db8422b3a7 100644 --- a/tools/get_PSF.py +++ b/tools/get_PSF.py @@ -8,13 +8,13 @@ import astropy.io.fits as fitsio # Setup PATH SIMPATH = "/share/simudata/CSSOSDataProductsSims/data/CSSTSimImage_C8/testRun_FGS" -config_filename = SIMPATH+"/config_C6_fits.yaml" -cat_filename = SIMPATH+"/MSC_00000000/MSC_10106100000000_chip_40_filt_FGS.cat" +config_filename = SIMPATH + "/config_C6_fits.yaml" +cat_filename = SIMPATH + "/MSC_00000000/MSC_10106100000000_chip_40_filt_FGS.cat" # Read cat file catFn = open(cat_filename, "r") line = catFn.readline() -print(cat_filename, '\n', line) +print(cat_filename, "\n", line) imgPos = [] chipID = -1 for line in catFn: @@ -30,7 +30,7 @@ for line in catFn: imgPos.append([ximg, yimg]) imgPos = np.array(imgPos) nobj = imgPos.shape[0] -print('chipID, nobj::', chipID, nobj) +print("chipID, nobj::", chipID, nobj) # Read config file with open(config_filename, "r") as stream: @@ -43,16 +43,17 @@ with open(config_filename, "r") as stream: # Setup Chip chip = Chip(chipID=chipID, config=config) -print('chip.bound::', chip.bound.xmin, chip.bound.xmax, - chip.bound.ymin, chip.bound.ymax) +print( + "chip.bound::", chip.bound.xmin, chip.bound.xmax, chip.bound.ymin, chip.bound.ymax +) for iobj in range(nobj): - print("\nget psf for iobj-", iobj, '\t', 'bandpass:', end=" ", flush=True) + print("\nget psf for iobj-", iobj, "\t", "bandpass:", end=" ", flush=True) # Setup Position on focalplane # try get the PSF at some location (1234, 1234) on the chip x, y = imgPos[iobj, :] - x = x+chip.bound.xmin - y = y+chip.bound.ymin + x = x + chip.bound.xmin + y = y + chip.bound.ymin pos_img = galsim.PositionD(x, y) @@ -64,7 +65,8 @@ for iobj in range(nobj): filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list for i in range(len(bandpass_list)): print(i, end=" ", flush=True) @@ -72,10 +74,12 @@ for iobj in range(nobj): bandpass = bandpass_list[i] # Get corresponding PSF model - psf_model = PSFInterp(chip=chip, npsf=100, - PSF_data_file=config["psf_setting"]["psf_dir"]) + psf_model = PSFInterp( + chip=chip, npsf=100, PSF_data_file=config["psf_setting"]["psf_dir"] + ) psf = psf_model.get_PSF( - chip=chip, pos_img=pos_img, bandpass=bandpass, galsimGSObject=False) + chip=chip, pos_img=pos_img, bandpass=bandpass, galsimGSObject=False + ) if True: fn = "psf_{:}.{:}.{:}.fits".format(chipID, iobj, i) @@ -84,5 +88,5 @@ for iobj in range(nobj): os.remove(fn) hdu = fitsio.PrimaryHDU() hdu.data = psf - hdu.header.set('pixScale', 5) + hdu.header.set("pixScale", 5) hdu.writeto(fn) diff --git a/tools/get_PSF_SLS.py b/tools/get_PSF_SLS.py index f921b92931ea15a31906fd8467f6fa74a2702097..c01e22781098ad7af02aacfd1c0960d8be1eefd4 100644 --- a/tools/get_PSF_SLS.py +++ b/tools/get_PSF_SLS.py @@ -1,10 +1,8 @@ -import numpy as np import observation_sim.psf.PSFInterpSLS as PSFInterpSLS import observation_sim.psf.PSFInterp as PSFInterp from observation_sim.instruments import Chip, Filter, FilterParam import astropy.io.fits as fitsio -from observation_sim.instruments import Chip, FilterParam, Filter import galsim # 计算 0级或1级光谱在某个波长位置的PSF, 返回值是过采样的PSF，像元大小有CSST图像像元大小的1/2 @@ -14,14 +12,19 @@ import galsim # wave: 波长，单位A -def get_SLS_PSF(chipID=1, pos_img=[6000, 4000], order=1, wave=8000, sls_psf_dir='/nfsdata/share/CSSOSDataProductsSims/data/SLS_PSF_PCA_fp_cd/'): - orders = {0: 'B', 1: 'A'} +def get_SLS_PSF( + chipID=1, + pos_img=[6000, 4000], + order=1, + wave=8000, + sls_psf_dir="/nfsdata/share/CSSOSDataProductsSims/data/SLS_PSF_PCA_fp_cd/", +): + orders = {0: "B", 1: "A"} chip = Chip(chipID) filter_id, filter_type = chip.getChipFilter() filt = Filter( - filter_id=filter_id, - filter_type=filter_type, - filter_param=FilterParam()) + filter_id=filter_id, filter_type=filter_type, filter_param=FilterParam() + ) psf_model = PSFInterpSLS(chip, filt, PSF_data_prefix=sls_psf_dir) bandNo = 1 @@ -31,29 +34,44 @@ def get_SLS_PSF(chipID=1, pos_img=[6000, 4000], order=1, wave=8000, sls_psf_dir= break psf1, _ = psf_model.get_PSF( - chip, pos_img_local=pos_img, bandNo=bandNo, galsimGSObject=False, g_order=orders[order]) + chip, + pos_img_local=pos_img, + bandNo=bandNo, + galsimGSObject=False, + g_order=orders[order], + ) fn = "psf.chip{:}.order{:}.wave{:}.fits".format(chipID, order, wave) hdu = fitsio.PrimaryHDU() hdu.data = psf1 - hdu.header.set('pixScale', 5) + hdu.header.set("pixScale", 5) hdu.writeto(fn, overwrite=True) + # 获得成像的PSF,PSF在波长区间分四段采样，返回4段的PSF，每个PSF代表不同band波长范围不同的四段psf，波长从小到大 # chipId 必须为[6,7,8,9,11,12,13,14,15,16,17,18,19,20,22,23,24,25] # pos_img 为直接成像在图像上的位置,[x, y] -def getPSFImage(chipID=6, pos_img=[6000, 4500], psf_pho_dir="/nfsdata/share/CSSOSDataProductsSims/data/psfcube/set1_dynamic/"): +def getPSFImage( + chipID=6, + pos_img=[6000, 4500], + psf_pho_dir="/nfsdata/share/CSSOSDataProductsSims/data/psfcube/set1_dynamic/", +): chip = Chip(chipID=chipID) - print('chip.bound::', chip.bound.xmin, chip.bound.xmax, - chip.bound.ymin, chip.bound.ymax) + print( + "chip.bound::", + chip.bound.xmin, + chip.bound.xmax, + chip.bound.ymin, + chip.bound.ymax, + ) # Setup Position on focalplane # try get the PSF at some location (1234, 1234) on the chip pos_img = pos_img x, y = pos_img - x = x+chip.bound.xmin - y = y+chip.bound.ymin + x = x + chip.bound.xmin + y = y + chip.bound.ymin pos_img = galsim.PositionD(x, y) @@ -65,7 +83,8 @@ def getPSFImage(chipID=6, pos_img=[6000, 4500], psf_pho_dir="/nfsdata/share/CSSO filter_id=filter_id, filter_type=filter_type, filter_param=filter_param, - ccd_bandpass=chip.effCurve) + ccd_bandpass=chip.effCurve, + ) bandpass_list = filt.bandpass_sub_list for i in range(len(bandpass_list)): print(i, end=" ", flush=True) @@ -73,14 +92,14 @@ def getPSFImage(chipID=6, pos_img=[6000, 4500], psf_pho_dir="/nfsdata/share/CSSO bandpass = bandpass_list[i] # Get corresponding PSF model - psf_model = PSFInterp(chip=chip, npsf=100, - PSF_data_file=psf_pho_dir) + psf_model = PSFInterp(chip=chip, npsf=100, PSF_data_file=psf_pho_dir) psf = psf_model.get_PSF( - chip=chip, pos_img=pos_img, bandpass=bandpass, galsimGSObject=False) + chip=chip, pos_img=pos_img, bandpass=bandpass, galsimGSObject=False + ) fn = "psf_chip{:}.wave{:}.fits".format(chipID, i) hdu = fitsio.PrimaryHDU() hdu.data = psf - hdu.header.set('pixScale', 5) + hdu.header.set("pixScale", 5) hdu.writeto(fn, overwrite=True) diff --git a/tools/get_pointing.py b/tools/get_pointing.py index 057fac67caa5920b134654f8b4626cd92e0d3869..db6eca9092aaa376f91a446b4c14b06a273579fa 100755 --- a/tools/get_pointing.py +++ b/tools/get_pointing.py @@ -16,10 +16,10 @@ # # -from tkinter.tix import INTEGER import astropy.coordinates as coord from astropy import units as u -from pylab import * + +# from pylab import * import numpy as np import galsim import math @@ -44,8 +44,10 @@ class Chip(object): self.npix_y = 9232 self.pix_scale = 0.074 - def getTanWCS(self, ra, dec, img_rot, pix_scale=None, xcen=None, ycen=None, logger=None): - """ Get the WCS of the image mosaic using Gnomonic/TAN projection + def getTanWCS( + self, ra, dec, img_rot, pix_scale=None, xcen=None, ycen=None, logger=None + ): + """Get the WCS of the image mosaic using Gnomonic/TAN projection Parameter: ra, dec: float @@ -59,7 +61,8 @@ class Chip(object): """ if logger is not None: logger.info( - " Construct the wcs of the entire image mosaic using Gnomonic/TAN projection") + " Construct the wcs of the entire image mosaic using Gnomonic/TAN projection" + ) if (xcen is None) or (ycen is None): xcen = self.cen_pix_x ycen = self.cen_pix_y @@ -81,7 +84,8 @@ class Chip(object): # dvdy = +np.sin(img_rot.rad) * pix_scale moscen = galsim.PositionD(x=xcen, y=ycen) sky_center = galsim.CelestialCoord( - ra=ra*galsim.degrees, dec=dec*galsim.degrees) + ra=ra * galsim.degrees, dec=dec * galsim.degrees + ) affine = galsim.AffineTransform(dudx, dudy, dvdx, dvdy, origin=moscen) WCS = galsim.TanWCS(affine, sky_center, units=galsim.arcsec) @@ -109,12 +113,12 @@ class Chip(object): gy = self.npix_gap_y # xlim of a given CCD chip - xrem = 2*(colID - 1) - (self.nchip_x - 1) - xcen = (self.npix_x//2 + gx1//2) * xrem + xrem = 2 * (colID - 1) - (self.nchip_x - 1) + xcen = (self.npix_x // 2 + gx1 // 2) * xrem if chipID >= 26 or chipID == 21: - xcen = (self.npix_x//2 + gx1//2) * xrem - (gx2-gx1) + xcen = (self.npix_x // 2 + gx1 // 2) * xrem - (gx2 - gx1) if chipID <= 5 or chipID == 10: - xcen = (self.npix_x//2 + gx1//2) * xrem + (gx2-gx1) + xcen = (self.npix_x // 2 + gx1 // 2) * xrem + (gx2 - gx1) # nx0 = xcen - self.npix_x//2 + 1 # nx1 = xcen + self.npix_x//2 @@ -137,70 +141,80 @@ def transRaDec2D(ra, dec): def getobsPA(ra, dec): l1 = np.array([0, 0, 1]) l2 = transRaDec2D(ra, dec) - polar_ec = coord.SkyCoord(0*u.degree, 90*u.degree, - frame='barycentrictrueecliptic') - polar_eq = polar_ec.transform_to('icrs') + polar_ec = coord.SkyCoord( + 0 * u.degree, 90 * u.degree, frame="barycentrictrueecliptic" + ) + polar_eq = polar_ec.transform_to("icrs") # print(polar_eq.ra.value,polar_eq.dec.value) polar_d = transRaDec2D(polar_eq.ra.value, polar_eq.dec.value) l1l2cross = np.cross(l2, l1) pdl2cross = np.cross(l2, polar_d) - angle = math.acos(np.dot(l1l2cross, pdl2cross) / - (np.linalg.norm(l1l2cross)*np.linalg.norm(pdl2cross))) + angle = math.acos( + np.dot(l1l2cross, pdl2cross) + / (np.linalg.norm(l1l2cross) * np.linalg.norm(pdl2cross)) + ) - angle = (angle)/math.pi*180 + angle = (angle) / math.pi * 180 angle = angle + 90 - if (ra < 90 or ra > 270): + if ra < 90 or ra > 270: angle = -angle return angle + # @jit() def getSelectPointingList(center=[60, -40], radius=2): - points = np.loadtxt('sky.dat') + points = np.loadtxt("sky.dat") center = center # ra dec radius = radius # degree radii_dec = 1 - radii_ra = 1/math.cos(math.pi*center[1]/180) + radii_ra = 1 / math.cos(math.pi * center[1] / 180) if radii_ra > 180: radii_ra = 180 c_eclip = coord.SkyCoord( - points[:, 2]*u.degree, points[:, 1]*u.degree, frame='barycentrictrueecliptic') - c_equtor = c_eclip.transform_to('icrs') + points[:, 2] * u.degree, + points[:, 1] * u.degree, + frame="barycentrictrueecliptic", + ) + c_equtor = c_eclip.transform_to("icrs") # print(np.min((c_equtor.ra*u.degree).value), np.max((c_equtor.ra*u.degree).value)) # c_equtor_sel = c_equtor # points_sel = points - ra_range_lo = center[0]-radii_ra - ra_range_hi = center[0]+radii_ra + ra_range_lo = center[0] - radii_ra + ra_range_hi = center[0] + radii_ra if ra_range_lo < 0: - ids1 = ((c_equtor.ra*u.degree).value < - ra_range_hi) | ((c_equtor.ra*u.degree).value > 360+ra_range_lo) + ids1 = ((c_equtor.ra * u.degree).value < ra_range_hi) | ( + (c_equtor.ra * u.degree).value > 360 + ra_range_lo + ) elif ra_range_hi > 360: - ids1 = ((c_equtor.ra*u.degree).value > - ra_range_lo) | ((c_equtor.ra*u.degree).value < ra_range_hi-360) + ids1 = ((c_equtor.ra * u.degree).value > ra_range_lo) | ( + (c_equtor.ra * u.degree).value < ra_range_hi - 360 + ) else: - ids1 = ((c_equtor.ra*u.degree).value > - ra_range_lo) & ((c_equtor.ra*u.degree).value < ra_range_hi) + ids1 = ((c_equtor.ra * u.degree).value > ra_range_lo) & ( + (c_equtor.ra * u.degree).value < ra_range_hi + ) - dec_range_lo = center[1]-radii_dec - if center[1]-radii_dec < -90: + dec_range_lo = center[1] - radii_dec + if center[1] - radii_dec < -90: dec_range_lo = -90 - dec_range_hi = center[1]+radii_dec - if center[1]+radii_dec > 90: + dec_range_hi = center[1] + radii_dec + if center[1] + radii_dec > 90: dec_range_hi = 90 - ids3 = (c_equtor[ids1].dec*u.degree).value > dec_range_lo - ids4 = (c_equtor[ids1][ids3].dec*u.degree).value < dec_range_hi + ids3 = (c_equtor[ids1].dec * u.degree).value > dec_range_lo + ids4 = (c_equtor[ids1][ids3].dec * u.degree).value < dec_range_hi num = points[ids1][ids3][ids4].shape[0] @@ -208,8 +222,8 @@ def getSelectPointingList(center=[60, -40], radius=2): i = 0 for p, p_ in zip(points[ids1][ids3][ids4], c_equtor[ids1][ids3][ids4]): - ra = (p_.ra*u.degree).value - dec = (p_.dec*u.degree).value + ra = (p_.ra * u.degree).value + dec = (p_.dec * u.degree).value # print(ra, dec) lon = p[2] lat = p[1] @@ -224,7 +238,7 @@ def getSelectPointingList(center=[60, -40], radius=2): return p_result -def findPointingbyChipID(chipID=8, ra=60., dec=-40.): +def findPointingbyChipID(chipID=8, ra=60.0, dec=-40.0): """_summary_ Args: @@ -245,11 +259,11 @@ def findPointingbyChipID(chipID=8, ra=60., dec=-40.): r_ra = ra r_dec = dec - r_rot = 0. + r_rot = 0.0 for i in np.arange(0, p_num, 1): ra_n = p_list[i, 0] dec_n = p_list[i, 1] - rot = p_list[i, 4]*galsim.degrees + rot = p_list[i, 4] * galsim.degrees chip_wcs = pchip.getTanWCS(ra_n, dec_n, rot) c_center = pchip.getChipCenter() @@ -259,7 +273,7 @@ def findPointingbyChipID(chipID=8, ra=60., dec=-40.): ra_s = c_world.ra.deg dec_s = c_world.dec.deg # print(ra, dec, ra_s, dec_s) - d = (ra_s - ra)*(ra_s - ra) + (dec_s - dec)*(dec_s - dec) + d = (ra_s - ra) * (ra_s - ra) + (dec_s - dec) * (dec_s - dec) if d < min_d: min_d = d r_ra = ra_n @@ -273,6 +287,6 @@ def findPointingbyChipID(chipID=8, ra=60., dec=-40.): if __name__ == "__main__": - tchip, tra, tdec = 13, 60., -40. + tchip, tra, tdec = 13, 60.0, -40.0 pointing = findPointingbyChipID(chipID=tchip, ra=tra, dec=tdec) print("[ra_center, dec_center, image_rot]: ", pointing) diff --git a/tools/get_pointing_accuracy.py b/tools/get_pointing_accuracy.py index 99aff912ab6371847970e210eec27785678a5a1a..c2a6e50990ea4fee4eabaab8c3b7da8a46cc178f 100644 --- a/tools/get_pointing_accuracy.py +++ b/tools/get_pointing_accuracy.py @@ -1,13 +1,9 @@ - -from pylab import * -import math -import sys +# from pylab import * import numpy as np -import astropy.coordinates as coord from astropy.coordinates import SkyCoord from astropy import wcs, units as u from observation_sim.config.header import ImageHeader -from observation_sim.instruments import Telescope, Chip, FilterParam, Filter, FocalPlane +from observation_sim.instruments import Chip def transRaDec2D(ra, dec): @@ -80,7 +76,9 @@ def cal_FoVcenter_1P_equatorial(ra_FieldCenter, dec_FieldCenter, chipID=1, pa=-2 return ra_PointCenter, dec_PointCenter, lon_ecl_PointCenter, lat_ecl_PointCenter -def cal_FoVcenter_1P_ecliptic(lon_ecl_FieldCenter, lat_ecl_FieldCenter, chipID=1, pa=-23.5): +def cal_FoVcenter_1P_ecliptic( + lon_ecl_FieldCenter, lat_ecl_FieldCenter, chipID=1, pa=-23.5 +): # [ra_FieldCenter, dec_FieldCenter] is the center ra, dec of calibration fileds, such as: NEP, NGC 6397, etc. # [ra_ChipCenter, dec_ChipCenter] is the center ra, dec of the Chip center. @@ -130,7 +128,7 @@ def cal_FoVcenter_1P_ecliptic(lon_ecl_FieldCenter, lat_ecl_FieldCenter, chipID=1 return ra_PointCenter, dec_PointCenter, lon_ecl_PointCenter, lat_ecl_PointCenter -def getChipCenterRaDec(chipID=1, p_ra=60., p_dec=-40., pa=23.5): +def getChipCenterRaDec(chipID=1, p_ra=60.0, p_dec=-40.0, pa=23.5): chip = Chip(chipID) h_ext = ImageHeader.generateExtensionHeader( @@ -156,17 +154,20 @@ def getChipCenterRaDec(chipID=1, p_ra=60., p_dec=-40., pa=23.5): return RA_chip, Dec_chip -if __name__ == '__main__': +if __name__ == "__main__": ra_input, dec_input = 270.00000, 66.56000 # NEP pa = 23.5 # chipid = 2 for chipid in np.arange(1, 31, 1): ra, dec, lon_ecl, lat_ecl = cal_FoVcenter_1P_equatorial( - ra_input, dec_input, chipID=chipid, pa=pa) + ra_input, dec_input, chipID=chipid, pa=pa + ) - print("chip id is %d, chip center [ra,dec] is [%f, %f], pointing center calculated [ra,dec] is [%f, %f]" % ( - chipid, ra_input, dec_input, ra, dec)) + print( + "chip id is %d, chip center [ra,dec] is [%f, %f], pointing center calculated [ra,dec] is [%f, %f]" + % (chipid, ra_input, dec_input, ra, dec) + ) # for check the result # testRA, testDec = getChipCenterRaDec(chipID = chipid, p_ra = ra, p_dec = dec) # print(ra_input-testRA, dec_input-testDec) diff --git a/tools/imgCropping.py b/tools/imgCropping.py index 42f4bd787988920c6235f37137687550520a0d44..16867540b56a119886e897ecf71b10b6da8c2195 100644 --- a/tools/imgCropping.py +++ b/tools/imgCropping.py @@ -12,19 +12,24 @@ def process_single_file(filename): with fits.open(filename) as hdul: data = hdul[1].data - data = data[PRESCAN_Y:PRESCAN_Y + IMAGE_Y, :] - blocks = [data[:, i * BLOCK_WIDTH + PRESCAN_X:(i + 1) * BLOCK_WIDTH - OVERSCAN_X] for i in range(16)] + data = data[PRESCAN_Y : PRESCAN_Y + IMAGE_Y, :] + blocks = [ + data[:, i * BLOCK_WIDTH + PRESCAN_X : (i + 1) * BLOCK_WIDTH - OVERSCAN_X] + for i in range(16) + ] blocks_a = np.concatenate(blocks[:4], axis=1) blocks_b = np.concatenate([np.fliplr(b) for b in blocks[4:8]], axis=1) - blocks_c = np.concatenate([np.flipud(np.fliplr(b)) for b in blocks[11:7:-1]], axis=1) + blocks_c = np.concatenate( + [np.flipud(np.fliplr(b)) for b in blocks[11:7:-1]], axis=1 + ) blocks_d = np.concatenate([np.flipud(b) for b in blocks[15:11:-1]], axis=1) blocks_dc = np.concatenate([blocks_d, blocks_c], axis=1) blocks_ab = np.concatenate([blocks_a, blocks_b], axis=1) blocks_final = np.concatenate([blocks_ab, blocks_dc], axis=0) - output_path = os.path.splitext(fn)[0]+'_cropping.fits' + output_path = os.path.splitext(fn)[0] + "_cropping.fits" fits.writeto(output_path, blocks_final, overwrite=True) print(f"OK:{output_path}") @@ -33,6 +38,6 @@ if __name__ == "__main__": if len(sys.argv) > 1: fn = sys.argv[1] else: - fn = 'CSST_MSC_MS_WIDE_20281024132057_20281024132327_10100484833_08_L0_V01.fits' + fn = "CSST_MSC_MS_WIDE_20281024132057_20281024132327_10100484833_08_L0_V01.fits" process_single_file(fn) diff --git a/tools/index_fits_hdf5.py b/tools/index_fits_hdf5.py index 779cbc0e550af590b9e28765067d1f20735d7ddd..f795135498cd0e93c4182786144f9bb223251e60 100644 --- a/tools/index_fits_hdf5.py +++ b/tools/index_fits_hdf5.py @@ -27,19 +27,20 @@ import galsim def test_fits(nfits=100, dir_cat=None): for ifits in range(nfits): gal = galsim.Gaussian(sigma=np.random.uniform(0.2, 0.3)).shear( - g1=np.random.uniform(-0.5, 0.5), g2=np.random.uniform(-0.5, 0.5)) + g1=np.random.uniform(-0.5, 0.5), g2=np.random.uniform(-0.5, 0.5) + ) arr = gal.drawImage(nx=64, ny=64, scale=0.074).array hdu = fitsio.PrimaryHDU() hdu.data = arr - hdu.header.set('index', ifits) - hdu.header.set('ra', 60.+np.random.uniform(-0.2, 0.2)) - hdu.header.set('dec', -40.+np.random.uniform(-0.2, 0.2)) - hdu.header.set('mag_g', 22+np.random.uniform(-1, 1)) - hdu.header.set('pixScale', 0.074) + hdu.header.set("index", ifits) + hdu.header.set("ra", 60.0 + np.random.uniform(-0.2, 0.2)) + hdu.header.set("dec", -40.0 + np.random.uniform(-0.2, 0.2)) + hdu.header.set("mag_g", 22 + np.random.uniform(-1, 1)) + hdu.header.set("pixScale", 0.074) - fout = dir_cat+"stampCats/testStamp_{:}.fits".format(ifits) + fout = dir_cat + "stampCats/testStamp_{:}.fits".format(ifits) if os.path.exists(fout): os.remove(fout) hdu.writeto(fout) @@ -48,59 +49,62 @@ def test_fits(nfits=100, dir_cat=None): def write_StampsIndex(dir_cat=None, DEBUG=False): MAXNUMBERINDEX = 10000 NSIDE = 128 - fp = h5py.File(dir_cat+'stampCatsIndex.hdf5', 'w') - grp1 = fp.create_group('Stamps') + fp = h5py.File(dir_cat + "stampCatsIndex.hdf5", "w") + grp1 = fp.create_group("Stamps") dataSet_Size = np.zeros(healpy.nside2npix(NSIDE), dtype=np.int64) - fitsList = os.listdir(dir_cat+'stampCats/') # 获取fits文件列表 + fitsList = os.listdir(dir_cat + "stampCats/") # 获取fits文件列表 for istamp in range(len(fitsList)): - print(istamp, ': ', fitsList[istamp], end='\r') + print(istamp, ": ", fitsList[istamp], end="\r") - hdu = fitsio.open(dir_cat+"stampCats/"+fitsList[istamp]) - tra = hdu[0].header['RA'] - tdec = hdu[0].header['DEC'] + hdu = fitsio.open(dir_cat + "stampCats/" + fitsList[istamp]) + tra = hdu[0].header["RA"] + tdec = hdu[0].header["DEC"] healpixID = healpy.ang2pix(NSIDE, tra, tdec, nest=False, lonlat=True) - if not (str(healpixID) in grp1): + if str(healpixID) not in grp1: grp2 = grp1.create_group(str(healpixID)) else: grp2 = grp1[str(healpixID)] - if not ('ra' in grp2): + if "ra" not in grp2: dset_ra = grp2.create_dataset( - 'ra', (0,), dtype='f16', maxshape=(MAXNUMBERINDEX, )) + "ra", (0,), dtype="f16", maxshape=(MAXNUMBERINDEX,) + ) dset_dec = grp2.create_dataset( - 'dec', (0,), dtype='f16', maxshape=(MAXNUMBERINDEX, )) + "dec", (0,), dtype="f16", maxshape=(MAXNUMBERINDEX,) + ) dt = h5py.special_dtype(vlen=str) dset_fn = grp2.create_dataset( - 'filename', (0,), dtype=dt, maxshape=(MAXNUMBERINDEX, )) + "filename", (0,), dtype=dt, maxshape=(MAXNUMBERINDEX,) + ) else: - dset_ra = grp2['ra'] - dset_dec = grp2['dec'] - dset_fn = grp2['filename'] - - dataSet_Size[healpixID] = dataSet_Size[healpixID]+1 - grp2['ra'].resize((dataSet_Size[healpixID],)) - grp2['dec'].resize((dataSet_Size[healpixID],)) - grp2['filename'].resize((dataSet_Size[healpixID],)) - - dset_ra[dataSet_Size[healpixID]-1] = tra - dset_dec[dataSet_Size[healpixID]-1] = tdec - dset_fn[dataSet_Size[healpixID]-1] = fitsList[istamp] + dset_ra = grp2["ra"] + dset_dec = grp2["dec"] + dset_fn = grp2["filename"] + + dataSet_Size[healpixID] = dataSet_Size[healpixID] + 1 + grp2["ra"].resize((dataSet_Size[healpixID],)) + grp2["dec"].resize((dataSet_Size[healpixID],)) + grp2["filename"].resize((dataSet_Size[healpixID],)) + + dset_ra[dataSet_Size[healpixID] - 1] = tra + dset_dec[dataSet_Size[healpixID] - 1] = tdec + dset_fn[dataSet_Size[healpixID] - 1] = fitsList[istamp] fp.close() if DEBUG: - print('\n') - ff = h5py.File(dir_cat+"stampCatsIndex.hdf5", "r") + print("\n") + ff = h5py.File(dir_cat + "stampCatsIndex.hdf5", "r") ss = 0 - for kk in ff['Stamps'].keys(): - print(kk, ff['Stamps'][kk]['ra'].size) - ss = ss+ff['Stamps'][kk]['ra'].size + for kk in ff["Stamps"].keys(): + print(kk, ff["Stamps"][kk]["ra"].size) + ss = ss + ff["Stamps"][kk]["ra"].size print(ss) -if __name__ == '__main__': +if __name__ == "__main__": dir_temp = "./Catalog_test/" # test_fits(dir_cat=dir_temp) write_StampsIndex(dir_cat=dir_temp) diff --git a/tools/plan_json_to_pointing.py b/tools/plan_json_to_pointing.py new file mode 100644 index 0000000000000000000000000000000000000000..583e6e87e6e507b725753ecaebc3919fd6340020 --- /dev/null +++ b/tools/plan_json_to_pointing.py @@ -0,0 +1,73 @@ +import json +import math +from pathlib import Path + + +def _get_nested(d, path, default=math.nan): + cur = d + for k in path: + if not isinstance(cur, dict) or k not in cur: + return default + cur = cur[k] + return cur + + +def plan_json_to_pointing_text(json_path, out_path): + json_path = Path(json_path) + with json_path.open("r", encoding="utf-8") as f: + data = json.load(f) + + if isinstance(data, dict) and "data" in data: + data = data["data"] + + if not isinstance(data, list): + raise TypeError(f"Unsupported JSON top-level type: {type(data).__name__}") + + header = ( + "# ra dec lon(ecliptic) lat(ecliptic) pos_angle time(julian) sat_x sat_y sat_z " + "sun_x sun_y sun_z moon_x moon_y moon_z sat_vx sat_vy sat_vz exp_time " + "isDeep obs_config id\n" + ) + + columns = [ + ("ra", ("simresult", "ra")), + ("dec", ("simresult", "dec")), + ("lon(ecliptic)", ("simresult", "lambda")), + ("lat(ecliptic)", ("simresult", "beta")), + ("pos_angle", ("params", "pa_obj")), + ("time(julian)", ("simresult", "jd")), + ("sat_x", ("simresult", "sat_x")), + ("sat_y", ("simresult", "sat_y")), + ("sat_z", ("simresult", "sat_z")), + ("sun_x", ("simresult", "sun_x")), + ("sun_y", ("simresult", "sun_y")), + ("sun_z", ("simresult", "sun_z")), + ("moon_x", ("simresult", "moon_x")), + ("moon_y", ("simresult", "moon_y")), + ("moon_z", ("simresult", "moon_z")), + ("sat_vx", ("simresult", "sat_vx")), + ("sat_vy", ("simresult", "sat_vy")), + ("sat_vz", ("simresult", "sat_vz")), + ("exp_time", ("exptime",)), + ("isDeep", ("simresult", "sky_type")), + ("obs_config", None), + ("id", ("obs_id",)), + ] + + out_path = Path(out_path) + with out_path.open("w", encoding="utf-8") as f: + f.write(header) + for item in data: + values = [] + for _, path in columns: + if path is None: + values.append("null") + else: + values.append(_get_nested(item, path)) + f.write(" ".join(map(str, values)) + "\n") + + +if __name__ == "__main__": + jsonFn = "test_2461876_1_plan.json" + outFn = "pointing_test.dat" + plan_json_to_pointing_text(jsonFn, outFn) diff --git a/tools/setConfig/reset_obs.py b/tools/setConfig/reset_obs.py index f05b9d2448458b35e6eb5c280d6eece1384d7991..8587cc70eb2d8595d46ade7bdd04a9b0a333c13e 100644 --- a/tools/setConfig/reset_obs.py +++ b/tools/setConfig/reset_obs.py @@ -6,50 +6,50 @@ app = Flask(__name__) key_type_map = { - 'obs_type': str, - 'obs_type_code': str, - 'obs_id': str, - 'run_chips': list, - 'call_sequence': { - 'scie_obs': { - 'shutter_effect': bool, - 'flat_fielding': bool, - 'field_dist': bool, + "obs_type": str, + "obs_type_code": str, + "obs_id": str, + "run_chips": list, + "call_sequence": { + "scie_obs": { + "shutter_effect": bool, + "flat_fielding": bool, + "field_dist": bool, }, - 'sky_background': { - 'shutter_effect': bool, - 'flat_fielding': bool, - 'enable_straylight_model': bool, - 'flat_level': None, - 'flat_level_filt': None, + "sky_background": { + "shutter_effect": bool, + "flat_fielding": bool, + "enable_straylight_model": bool, + "flat_level": None, + "flat_level_filt": None, }, - 'PRNU_effect': {}, - 'cosmic_rays': { - 'save_cosmic_img': bool, + "PRNU_effect": {}, + "cosmic_rays": { + "save_cosmic_img": bool, }, - 'poisson_and_dark': { - 'add_dark': bool, + "poisson_and_dark": { + "add_dark": bool, }, - 'bright_fatter': {}, - 'detector_defects': { - 'hot_pixels': bool, - 'dead_pixels': bool, - 'bad_columns': bool, + "bright_fatter": {}, + "detector_defects": { + "hot_pixels": bool, + "dead_pixels": bool, + "bad_columns": bool, }, - 'nonlinearity': {}, - 'blooming': {}, - 'prescan_overscan': { - 'add_dark': bool, + "nonlinearity": {}, + "blooming": {}, + "prescan_overscan": { + "add_dark": bool, }, - 'bias': { - 'bias_16channel': bool, + "bias": { + "bias_16channel": bool, }, - 'readout_noise': {}, - 'gain': { - 'gain_16channel': bool, + "readout_noise": {}, + "gain": { + "gain_16channel": bool, }, - 'quantization_and_output': { - 'format_output': bool, + "quantization_and_output": { + "format_output": bool, }, }, } @@ -65,7 +65,7 @@ def convert_dict_values(d, key_type_map): if key_type_map[key] is float: d[key] = float(value) if key_type_map[key] is bool: - if d[key].lower() == 'yes' or d[key].lower() == 'true': + if d[key].lower() == "yes" or d[key].lower() == "true": d[key] = True else: d[key] = False @@ -78,45 +78,47 @@ def convert_dict_values(d, key_type_map): def load_yaml(): - with open('templates/obs_config_SCI.yaml', 'r') as file: + with open("templates/obs_config_SCI.yaml", "r") as file: return yaml.safe_load(file) def save_yaml(data): convert_dict_values(data, key_type_map) - with open('config_reset/obs_config_SCI_reset.yaml', 'w') as file: + with open("config_reset/obs_config_SCI_reset.yaml", "w") as file: yaml.dump(data, file, default_flow_style=False, sort_keys=False) -def render_form(data, parent_key=''): - form_html = '' +def render_form(data, parent_key=""): + form_html = "" for key, value in data.items(): full_key = f"{parent_key}.{key}" if parent_key else key if isinstance(value, dict): # 处理字典 - form_html += f"

{key}

{render_form(value, full_key)}
" + form_html += ( + f"

{key}

{render_form(value, full_key)}
" + ) else: form_html += f"" form_html += f"
" return form_html -@app.route('/', methods=['GET', 'POST']) +@app.route("/", methods=["GET", "POST"]) def index(): - if request.method == 'POST': + if request.method == "POST": data = load_yaml() for key in request.form: - keys = key.split('.') + keys = key.split(".") temp = data for k in keys[:-1]: temp = temp[k] temp[keys[-1]] = request.form[key] save_yaml(data) - return redirect('/') + return redirect("/") data = load_yaml() form_html = render_form(data) - return render_template('index_obs.html', form_html=form_html) + return render_template("index_obs.html", form_html=form_html) -if __name__ == '__main__': +if __name__ == "__main__": app.run(debug=True) diff --git a/tools/setConfig/reset_overall.py b/tools/setConfig/reset_overall.py index cacb2049210ed85717c103baa26fc2dc01be32d2..61d942a289cdb965c994e759cbfd3b90550bd02c 100644 --- a/tools/setConfig/reset_overall.py +++ b/tools/setConfig/reset_overall.py @@ -5,65 +5,65 @@ import ast app = Flask(__name__) key_type_map = { - 'work_dir': str, - 'run_name': str, - 'project_cycle': int, - 'run_counter': int, - 'run_option': { - 'out_cat_only': bool, + "work_dir": str, + "run_name": str, + "project_cycle": int, + "run_counter": int, + "run_option": { + "out_cat_only": bool, }, - 'catalog_options': { - 'input_path': { - 'cat_dir': str, - 'star_cat': str, - 'galaxy_cat': str, + "catalog_options": { + "input_path": { + "cat_dir": str, + "star_cat": str, + "galaxy_cat": str, }, - 'SED_templates_path': { - 'star_SED': str, - 'galaxy_SED': str, - 'AGN_SED': str, + "SED_templates_path": { + "star_SED": str, + "galaxy_SED": str, + "AGN_SED": str, }, - 'star_only': bool, - 'galaxy_only': bool, - 'rotateEll': float, - 'enable_mw_ext_gal': bool, - 'planck_ebv_map': str, + "star_only": bool, + "galaxy_only": bool, + "rotateEll": float, + "enable_mw_ext_gal": bool, + "planck_ebv_map": str, }, - 'obs_setting': { - 'pointing_file': str, - 'obs_config_file': str, - 'run_pointings': list, - 'enable_astrometric_model': bool, - 'cut_in_band': str, - 'mag_sat_margin': float, - 'mag_lim_margin': float, + "obs_setting": { + "pointing_file": str, + "obs_config_file": str, + "run_pointings": list, + "enable_astrometric_model": bool, + "cut_in_band": str, + "mag_sat_margin": float, + "mag_lim_margin": float, }, - 'psf_setting': { - 'psf_model': str, - 'psf_pho_dir': str, - 'psf_sls_dir': str, + "psf_setting": { + "psf_model": str, + "psf_pho_dir": str, + "psf_sls_dir": str, }, - 'shear_setting': { - 'shear_type': str, - 'reduced_g1': float, - 'reduced_g2': float, + "shear_setting": { + "shear_type": str, + "reduced_g1": float, + "reduced_g2": float, }, - 'output_setting': { - 'output_format': str, - 'shutter_output': bool, - 'prnu_output': bool, + "output_setting": { + "output_format": str, + "shutter_output": bool, + "prnu_output": bool, }, - 'random_seeds': { - 'seed_poisson': int, - 'seed_CR': int, - 'seed_flat': int, - 'seed_prnu': int, - 'seed_gainNonUniform': int, - 'seed_biasNonUniform': int, - 'seed_rnNonUniform': int, - 'seed_badcolumns': int, - 'seed_defective': int, - 'seed_readout': int, + "random_seeds": { + "seed_poisson": int, + "seed_CR": int, + "seed_flat": int, + "seed_prnu": int, + "seed_gainNonUniform": int, + "seed_biasNonUniform": int, + "seed_rnNonUniform": int, + "seed_badcolumns": int, + "seed_defective": int, + "seed_readout": int, }, } @@ -78,7 +78,7 @@ def convert_dict_values(d, key_type_map): if key_type_map[key] is float: d[key] = float(value) if key_type_map[key] is bool: - if d[key].lower() == 'yes' or d[key].lower() == 'true': + if d[key].lower() == "yes" or d[key].lower() == "true": d[key] = True else: d[key] = False @@ -89,45 +89,47 @@ def convert_dict_values(d, key_type_map): def load_yaml(): - with open('templates/config_overall.yaml', 'r') as file: + with open("templates/config_overall.yaml", "r") as file: return yaml.safe_load(file) def save_yaml(data): convert_dict_values(data, key_type_map) - with open('config_reset/config_overall_reset.yaml', 'w') as file: + with open("config_reset/config_overall_reset.yaml", "w") as file: yaml.dump(data, file, default_flow_style=False, sort_keys=False) -def render_form(data, parent_key=''): - form_html = '' +def render_form(data, parent_key=""): + form_html = "" for key, value in data.items(): full_key = f"{parent_key}.{key}" if parent_key else key if isinstance(value, dict): # 处理字典 - form_html += f"

{key}

{render_form(value, full_key)}
" + form_html += ( + f"

{key}

{render_form(value, full_key)}
" + ) else: form_html += f"" form_html += f"
" return form_html -@app.route('/', methods=['GET', 'POST']) +@app.route("/", methods=["GET", "POST"]) def index(): - if request.method == 'POST': + if request.method == "POST": data = load_yaml() for key in request.form: - keys = key.split('.') + keys = key.split(".") temp = data for k in keys[:-1]: temp = temp[k] temp[keys[-1]] = request.form[key] save_yaml(data) - return redirect('/') + return redirect("/") data = load_yaml() form_html = render_form(data) - return render_template('index_overall.html', form_html=form_html) + return render_template("index_overall.html", form_html=form_html) -if __name__ == '__main__': +if __name__ == "__main__": app.run(debug=True) diff --git a/tools/target_location_check.py b/tools/target_location_check.py index 6e0f525ea3b4a35d95140f04cb2f7d470ff69eec..78a17ce2ea281dadc63042eb5cc8002eac5d992e 100644 --- a/tools/target_location_check.py +++ b/tools/target_location_check.py @@ -9,11 +9,16 @@ import galsim import numpy as np import argparse import matplotlib.pyplot as plt -import os -import sys -def focalPlaneInf(ra_target, dec_target, ra_point, dec_point, image_rot=-113.4333, figout="zTargetOnCCD.pdf"): +def focalPlaneInf( + ra_target, + dec_target, + ra_point, + dec_point, + image_rot=-113.4333, + figout="zTargetOnCCD.pdf", +): """ Input parameters: ra_target : right ascension of the target/input object; @@ -39,50 +44,62 @@ def focalPlaneInf(ra_target, dec_target, ra_point, dec_point, image_rot=-113.433 or type >> python TargetLocationCheck.py ra_target dec_target ra_point dec_point -image_rot=floatNum or type >> python TargetLocationCheck.py ra_target dec_target ra_point dec_point -image_rot=floatNum -figout=FigureName """ - print("^_^ Input target coordinate: [Ra, Dec] = [%10.6f, %10.6f]" % ( - ra_target, dec_target)) - print("^_^ Input telescope pointing center: [Ra, Dec] = [%10.6f, %10.6f]" % ( - ra_point, dec_point)) + print( + "^_^ Input target coordinate: [Ra, Dec] = [%10.6f, %10.6f]" + % (ra_target, dec_target) + ) + print( + "^_^ Input telescope pointing center: [Ra, Dec] = [%10.6f, %10.6f]" + % (ra_point, dec_point) + ) print("^_^ Input camera orientation: %12.6f degree(s)" % image_rot) print(" ") # load ccd parameters xsize, ysize, xchip, ychip, xgap, ygap, xnchip, ynchip = ccdParam() - print("^_^ Pixel range of focal plane: x = [%5d, %5d], y = [%5d, %5d]" % ( - -xsize/2, xsize/2, -ysize/2, ysize/2)) + print( + "^_^ Pixel range of focal plane: x = [%5d, %5d], y = [%5d, %5d]" + % (-xsize / 2, xsize / 2, -ysize / 2, ysize / 2) + ) # wcs wcs = getTanWCS(ra_point, dec_point, image_rot, pix_scale=0.074) skyObj = galsim.CelestialCoord( - ra=ra_target*galsim.degrees, dec=dec_target*galsim.degrees) + ra=ra_target * galsim.degrees, dec=dec_target * galsim.degrees + ) pixObj = wcs.toImage(skyObj) xpixObj = pixObj.x ypixObj = pixObj.y - print("^_^ Image position of target: [xImage, yImage] = [%9.3f, %9.3f]" % ( - xpixObj, ypixObj)) + print( + "^_^ Image position of target: [xImage, yImage] = [%9.3f, %9.3f]" + % (xpixObj, ypixObj) + ) # first determine if the target is in the focal plane - xin = (xpixObj+xsize/2)*(xpixObj-xsize/2) - yin = (ypixObj+ysize/2)*(ypixObj-ysize/2) + xin = (xpixObj + xsize / 2) * (xpixObj - xsize / 2) + yin = (ypixObj + ysize / 2) * (ypixObj - ysize / 2) if xin > 0 or yin > 0: raise ValueError("!!! Input target is out of the focal plane") # second determine the location of the target trigger = False for i in range(30): - ichip = i+1 + ichip = i + 1 ischip = str("0%d" % ichip)[-2:] fId, fType = getChipFilter(ichip) ix0, ix1, iy0, iy1 = getChipLim(ichip) - ixin = (xpixObj-ix0)*(xpixObj-ix1) - iyin = (ypixObj-iy0)*(ypixObj-iy1) + ixin = (xpixObj - ix0) * (xpixObj - ix1) + iyin = (ypixObj - iy0) * (ypixObj - iy1) if ixin <= 0 and iyin <= 0: trigger = True idx = xpixObj - ix0 idy = ypixObj - iy0 print(" ---------------------------------------------") - print(" ** Target locates in CHIP#%s with filter %s **" % - (ischip, fType)) print( - " ** Target position in the chip: [x, y] = [%7.2f, %7.2f]" % (idx, idy)) + " ** Target locates in CHIP#%s with filter %s **" % (ischip, fType) + ) + print( + " ** Target position in the chip: [x, y] = [%7.2f, %7.2f]" + % (idx, idy) + ) print(" ---------------------------------------------") break if not trigger: @@ -130,8 +147,7 @@ def getTanWCS(ra, dec, img_rot, pix_scale=0.074): dvdy = -np.cos(img_rot.rad) * pix_scale moscen = galsim.PositionD(x=xcen, y=ycen) - sky_center = galsim.CelestialCoord( - ra=ra*galsim.degrees, dec=dec*galsim.degrees) + sky_center = galsim.CelestialCoord(ra=ra * galsim.degrees, dec=dec * galsim.degrees) affine = galsim.AffineTransform(dudx, dudy, dvdx, dvdy, origin=moscen) WCS = galsim.TanWCS(affine, sky_center, units=galsim.arcsec) @@ -187,22 +203,22 @@ def getChipLim(chipID): colID = 6 - ((chipID - 1) // 5) # xlim of a given CCD chip - xrem = 2*(colID - 1) - (xnchip - 1) - xcen = (x0//2 + gx1//2) * xrem + xrem = 2 * (colID - 1) - (xnchip - 1) + xcen = (x0 // 2 + gx1 // 2) * xrem if chipID >= 26 or chipID == 21: - xcen = (x0//2 + gx1//2) * xrem - (gx2-gx1) + xcen = (x0 // 2 + gx1 // 2) * xrem - (gx2 - gx1) if chipID <= 5 or chipID == 10: - xcen = (x0//2 + gx1//2) * xrem + (gx2-gx1) - nx0 = xcen - x0//2 + 1 - nx1 = xcen + x0//2 + xcen = (x0 // 2 + gx1 // 2) * xrem + (gx2 - gx1) + nx0 = xcen - x0 // 2 + 1 + nx1 = xcen + x0 // 2 # ylim of a given CCD chip yrem = (rowID - 1) - ynchip // 2 ycen = (y0 + gy) * yrem - ny0 = ycen - y0//2 + 1 - ny1 = ycen + y0//2 + ny0 = ycen - y0 // 2 + 1 + ny1 = ycen + y0 // 2 - return nx0-1, nx1-1, ny0-1, ny1-1 + return nx0 - 1, nx1 - 1, ny0 - 1, ny1 - 1 def ccdLayout(xpixTar, ypixTar, figout="ccdLayout.pdf"): @@ -210,7 +226,7 @@ def ccdLayout(xpixTar, ypixTar, figout="ccdLayout.pdf"): ax = fig.add_axes([0.1, 0.1, 0.80, 0.80]) # plot the layout of the ccd distribution for i in range(30): - ichip = i+1 + ichip = i + 1 fId, fType = getChipFilter(ichip) ischip = str("0%d" % ichip)[-2:] ix0, ix1, iy0, iy1 = getChipLim(ichip) @@ -218,13 +234,14 @@ def ccdLayout(xpixTar, ypixTar, figout="ccdLayout.pdf"): ax.plot([ix0, ix1], [iy1, iy1], "k-", linewidth=2.5) ax.plot([ix0, ix0], [iy0, iy1], "k-", linewidth=2.5) ax.plot([ix1, ix1], [iy0, iy1], "k-", linewidth=2.5) - ax.text(ix0+500, iy0+1500, "%s#%s" % - (fType, ischip), fontsize=12, color="grey") + ax.text( + ix0 + 500, iy0 + 1500, "%s#%s" % (fType, ischip), fontsize=12, color="grey" + ) ax.plot(xpixTar, ypixTar, "r*", ms=12) ax.set_xlabel(r"$X\,[\mathrm{pixels}]$", fontsize=20) ax.set_ylabel(r"$Y\,[\mathrm{pixels}]$", fontsize=20) ax.invert_yaxis() - ax.axis('off') + ax.axis("off") plt.savefig(figout) @@ -233,14 +250,14 @@ def parseArguments(): parser = argparse.ArgumentParser() # Positional arguments - parser.add_argument("ra_target", type=float) - parser.add_argument("dec_target", type=float) - parser.add_argument("ra_point", type=float) + parser.add_argument("ra_target", type=float) + parser.add_argument("dec_target", type=float) + parser.add_argument("ra_point", type=float) parser.add_argument("dec_point", type=float) # Optional arguments parser.add_argument("-image_rot", type=float, default=-113.4333) - parser.add_argument("-figout", type=str, default="zTargetOnCCD.pdf") + parser.add_argument("-figout", type=str, default="zTargetOnCCD.pdf") # Parse arguments args = parser.parse_args() @@ -253,5 +270,11 @@ if __name__ == "__main__": args = parseArguments() # Run function - focalPlaneInf(args.ra_target, args.dec_target, args.ra_point, - args.dec_point, args.image_rot, args.figout) + focalPlaneInf( + args.ra_target, + args.dec_target, + args.ra_point, + args.dec_point, + args.image_rot, + args.figout, + )