Uploading

from prescient_sdk.upload import upload
from prescient_sdk.client import PrescientClient

client = PrescientClient(env_file="config.env")

For a directory test_upload/ containing

test_upload
├── subdir
│   └── test.tif
├── test.csv
└── test.txt

to upload

upload("test_upload", prescient_client=client)

To exclude files from being uploaded with certain suffixes

upload("test_upload", prescient_client=client, exclude=["*.txt", "*.csv"])

To skip uploading of files that would overwrite an existing key (default behaviour) specify overwrite=False. You can see helpful logging messages that illustrate whether a particular file was uploaded or skipped.

import logging

logging.basicConfig(level=logging.INFO)
upload("test_upload", prescient_client=client, overwrite=False)

INFO:prescient_sdk.upload:found 3 files to upload
INFO:prescient_sdk.upload:skipping file test_upload/test.txt as it already exists at s3://upload.enexus.server.prescient.earth/test_upload/test.txt
INFO:prescient_sdk.upload:skipping file test_upload/test.csv as it already exists at s3://upload.enexus.server.prescient.earth/test_upload/test.csv
INFO:prescient_sdk.upload:skipping file test_upload/subdir/test.tif as it already exists at s3://upload.enexus.server.prescient.earth/test_upload/subdir/test.tif