Add remote storage#
Show code cell content
!lamin close
!yes | lamin delete pgtest
import lamindb_setup as ln_setup
from lamindb_setup._add_remote_storage import switch_default_storage
Add remote storage#
Show code cell content
import laminci
!docker stop pgtest && docker rm pgtest
pgurl = laminci.db.setup_local_test_postgres()
ln_setup.init(storage="./storage1", db=pgurl)
ln_setup.settings.storage.root
Local storage:
switch_default_storage("./storage_2")
ln_setup.settings.storage.root
ln_setup.settings.storage.root_as_str
Show code cell content
from pathlib import Path
assert ln_setup.settings.storage.root_as_str == f"{Path.cwd()}/storage_2"
Cloud storage:
switch_default_storage("s3://lamindb-ci")
ln_setup.settings.storage.root
See an overview:
Show code cell content
assert ln_setup.settings.storage.is_cloud
assert ln_setup.settings.storage.root_as_str == "s3://lamindb-ci"
assert ln_setup.settings.storage.region == "us-west-1"
# root.fs contains the underlying fsspec filesystem
assert (
ln_setup.settings.storage.root.fs.cache_regions # set by lamindb to True for s3 by default
)
You can set any additional fsspec filesystem arguments for cloud storage, such as profile or cache_regions (for s3 only), for example:
switch_default_storage("s3://lamindb-ci", cache_regions=False)
Show code cell content
# test cache_regions
assert not ln_setup.settings.storage.root.fs.cache_regions
# test setting storage not by owner
ln_setup.login("testuser2@lamin.ai", key="goeoNJKE61ygbz1vhaCVynGERaRrlviPBVQsjkhz")
switch_default_storage("./storage_3")
assert ln_setup.settings.storage.root_as_str == f"{Path.cwd()}/storage_3"
!docker stop pgtest && docker rm pgtest
Currently not possible: setting storage for SQLite instance#
If you try to set the storage for an sqlite instance, an error message is returned:
assert switch_default_storage("mydata_storage_2") == "set-storage-failed"