Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
csst-pipeline
csst_fs
Commits
d63100fe
Commit
d63100fe
authored
Jun 27, 2024
by
Matthias Weidenthaler
Browse files
Added 'open' functionality to s3_fs
parent
4e27be18
Changes
2
Hide whitespace changes
Inline
Side-by-side
README.md
View file @
d63100fe
...
...
@@ -52,6 +52,16 @@ s3_fs.info('s3://csst-prod/gaia/data')
s3_fs
.
info
(
's3://csst-prod/gaia/test/requirements.txt'
,
s3_options
=
s3_options
)
```
### Open for read/write
```
python
from
csst_fs
import
s3_fs
# open single file
with
s3_fs
.
open
(
's3://csst-prod/gaia/data'
)
as
file
:
file
.
read
()
with
s3_fs
.
open
(
's3://csst-prod/gaia/test/requirements.txt'
,
s3_options
=
s3_options
,
mode
=
'w'
)
as
file
:
file
.
write
(
"CSST"
)
```
## astropy直接读写s3的写法适配
### fits.open
#### 老写法
...
...
csst_fs/s3_fs.py
View file @
d63100fe
...
...
@@ -12,3 +12,7 @@ def get(rpath, lpath, recursive=False, callback=fsspec.callbacks.DEFAULT_CALLBAC
def
info
(
path
,
bucket
=
None
,
key
=
None
,
refresh
=
False
,
version_id
=
None
,
s3_options
=
load_s3_options
())
->
dict
:
s3_fs
=
fsspec
.
filesystem
(
's3'
,
**
s3_options
)
return
s3_fs
.
info
(
path
,
bucket
,
key
,
refresh
,
version_id
)
def
open
(
path
,
s3_options
=
load_s3_options
(),
**
kwargs
):
s3_fs
=
fsspec
.
filesystem
(
's3'
,
**
s3_options
)
return
s3_fs
.
open
(
path
,
**
kwargs
)
\ No newline at end of file
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment