README.md

**astropy 需升级至 5.3**  
**老写法同时兼容本地nas和云上s3，只要读写路径以s3:// 协议开头会自动识别**  

如果需要读写S3时，需要传入s3的密钥和endpoint等配置，有两种方法可选

### 方法1 环境变量
执行下面三个环境变量，本文档下面介绍到的所有方法都会尝试读取环境变量以获取配置
```python
s3_options = {
    'key': os.getenv('S3_KEY'),
    'secret': os.getenv('S3_SECRET'),
    'endpoint_url': os.getenv('S3_ENDPOINT_URL')
}
```

### 方法2 每次调用方法时传入 s3_options
```
在第一个kwargs参数位置指定s3_options, s3_options示例：
```json
s3_options = {
    "key": "minioadmin", 
    "secret": "minioadmin", 
    "endpoint_url": "http://localhost:9000"
}

```
## 本地到s3的上传与下载
### 上传
 ```python
from csst_fs import s3_fs
# single file，s3_options from env
s3_fs.put('requirements.txt', 's3://csst-prod/gaia/test/requirements.txt')
# single file，s3_options from function parameter
s3_fs.put('requirements.txt', 's3://csst-prod/gaia/test/requirements.txt', s3_options=s3_options)
# folder，to s3 s3://csst-prod/common
s3_fs.put('./common', 's3://csst-prod/', recursive=True)
s3_fs.put('./common', 's3://csst-prod/', s3_options=s3_options, recursive=True)

```

### 下载
```python
from csst_fs import s3_fs
# single file
s3_fs.get('s3://csst-prod/gaia/test/requirements.txt', 'requirements.txt')
s3_fs.get('s3://csst-prod/gaia/test/requirements.txt', 'requirements.txt', s3_options=s3_options)
# folder
s3_fs.get('s3://csst-prod/gaia/data', './', recursive=True)
s3_fs.get('s3://csst-prod/gaia/data', './', s3_options=s3_options, recursive=True)
# get file or folder info
s3_fs.info('s3://csst-prod/gaia/data')
s3_fs.info('s3://csst-prod/gaia/test/requirements.txt', s3_options=s3_options)
```

### Open for read/write
```python
from csst_fs import s3_fs
# open single file (s3 or local)
with s3_fs.open('s3://csst-prod/gaia/data') as file:
    file.read()
with s3_fs.open('s3://csst-prod/gaia/test/requirements.txt', s3_options=s3_options, mode='w') as file:
    file.write("CSST")
```

### Check if the given file path exists
```python
from csst_fs import fs
# local or on s3, depending on the given path
fs.isfile('requirements.txt')
fs.isfile('s3://csst-prod/test.txt')
fs.isfile('s3://csst-prod/test.txt', s3_options=s3_options)
```

### Delete a file from local or s3
```python
from csst_fs import fs
# local or on s3, depending on the given path
fs.delete('requirements.txt') # uses os.remove
fs.delete('test', dir_fd=1)
fs.delete('s3://csst-prod/test.txt') # uses fsspec.delete
fs.delete('s3://csst-prod/test.txt', recursive=True, maxdepth=3)
fs.delete('s3://csst-prod/test.txt', s3_options=s3_options)
```

## astropy直接读写s3的写法适配
### fits.open
#### 老写法
```python
fits.open(path)
```
usage reference:  
[https://docs.astropy.org/en/stable/io/fits/api/files.html#open](https://docs.astropy.org/en/stable/io/fits/api/files.html#open)  
#### 新写法
 ```python
from csst_fs import fsspec_fits
fsspec_fits.open("s3://csst-prod/gaia/xx.fits")
fsspec_fits.open("s3://csst-prod/gaia/xx.fits", s3_options=s3_options)
```

 ### fits.getheader
 #### 老写法
 ```python
fits.getheader(filename=in_image_path, ext=1)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/files.html#getheader](https://docs.astropy.org/en/stable/io/fits/api/files.html#getheader)  

#### 新写法
```python
from csst_fs import fsspec_fits
fsspec_fits.getheader(filename=in_image_path, ext=1)
fsspec_fits.getheader(filename=in_image_path, ext=1, s3_options=s3_options)
```


### fits.getdata
#### 老写法
 ```python
fits.getdata(in_ref_flat)
fits.getdata( in_ref_shutter, ext=1)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/files.html#getdata](https://docs.astropy.org/en/stable/io/fits/api/files.html#getdata)  

#### 新写法
```python
from csst_fs import fsspec_fits
fsspec_fits.getdata(in_ref_flat)
fsspec_fits.getdata(in_ref_flat, s3_options=s3_options)
fsspec_fits.getdata( in_ref_shutter, ext=1)
fsspec_fits.getdata( in_ref_shutter, s3_options=s3_options, ext=1)
```

### fits.getval
#### 老写法
 ```python
fits.getval(filename, keyword)
fits.getval(filename, keyword, ext=1)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/files.html#getdata](https://docs.astropy.org/en/stable/io/fits/api/files.html#getval)  

#### 新写法
```python
from csst_fs import fsspec_fits
fsspec_fits.getval(filename, keyword)
fsspec_fits.getval(filename, keyword, s3_options=s3_options)
fsspec_fits.getval(filename, keyword, ext=1)
fsspec_fits.getval(filename, keyword, s3_options=s3_options, ext=1)
```

### header.tofile
#### 老写法
```python
header.tofile(out_head_path)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/headers.html#astropy.io.fits.Header.tofile](https://docs.astropy.org/en/stable/io/fits/api/headers.html#astropy.io.fits.Header.tofile)  

#### 新写法
```python
from csst_fs import fsspec_header
fsspec_header.tofile(header, out_head_path)
fsspec_header.tofile(header, out_head_path, s3_options=s3_options)
```


 ### header.fromfile
 #### 老写法
 ```python
header.fromfile(filename)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/headers.html#astropy.io.fits.Header.fromfile](https://docs.astropy.org/en/stable/io/fits/api/headers.html#astropy.io.fits.Header.fromfile)  

#### 新写法
```python
from csst_fs import fsspec_header
fsspec_header.fromfile(filename)
fsspec_header.fromfile(filename, s3_options=s3_options)
```


### HDUList.writeto
#### 老写法
```python
hdul_img.writeto(hdul_img, out_combined_img, overwrite=True)
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/hdulists.html#astropy.io.fits.HDUList.writeto](https://docs.astropy.org/en/stable/io/fits/api/hdulists.html#astropy.io.fits.HDUList.writeto)  

#### 新写法
 ```python
from csst_fs import fsspec_HDUList
fsspec_HDUList.writeto(hdul_img, out_combined_img, overwrite=True)
fsspec_HDUList.writeto(hdul_img, out_combined_img, s3_options=s3_options, overwrite=True)
```


### HDUList.fromfile
#### 老写法
```python
hdul_img = fits.HDUList.fromfile("hdulist.fits")
```
usage reference:
[https://docs.astropy.org/en/stable/io/fits/api/hdulists.html#astropy.io.fits.HDUList.fromfile](https://docs.astropy.org/en/stable/io/fits/api/hdulists.html#astropy.io.fits.HDUList.fromfile)

#### 新写法
 ```python
from csst_fs import fsspec_HDUList
hdul_img = fsspec_HDUList.fromfile("hdulist.fits")
hdul_img = fsspec_HDUList.fromfile("hdulist.fits", cache=False, s3_options=s3_options)
```


### table.Table.read
#### 老写法
```python
from astropy import table
table.Table.read(out_gaia_ldac, hdu=2)
```
usage reference:
[https://docs.astropy.org/en/stable/api/astropy.table.Table.html#astropy.table.Table.read](https://docs.astropy.org/en/stable/api/astropy.table.Table.html#astropy.table.Table.read
)   

#### 新写法
```python
from csst_fs import fsspec_table
fsspec_table.read(out_gaia_ldac, hdu=2)
fsspec_table.read(out_gaia_ldac, s3_options=s3_options, hdu=2)
```

### table.Table.write
#### 老写法
 ```python
ps.write(ref, format='fits', overwrite=True)
```
usage reference:
[https://docs.astropy.org/en/stable/api/astropy.table.Table.html#astropy.table.Table.write](https://docs.astropy.org/en/stable/api/astropy.table.Table.html#astropy.table.Table.write)  

#### 新写法
```python
from csst_fs import fsspec_table
fsspec_table.write(ps, ref, format='fits', overwrite=True)
fsspec_table.write(ps, ref, format='fits', s3_options=s3_options, overwrite=True)
```