Datasaur
Search…
Storage Options
Currently, these are the supported object storage. Each source attribute can be configured like this below.

Local

{
"files": {
"source": "local",
"path": <path-to-project-folder>
}
}

Google Cloud Storage

config/google-cloud-storage/config.json
{
"credentials": {
"gcs": { "gcsCredentialJson": "config/google-cloud-storage/credential.json" }
},
"create": {
"files": {
"source": "gcs",
"bucketName": "my-bucket",
"prefix": "projects"
}
}
}
To fully use Robosaur with a GCS bucket, we can use the Storage Object Admin role. The specific IAM permissions required are as follows:
  • storage.objects.list
  • storage.objects.get
  • storage.objects.create - to save export results to GCS bucket
  • storage.objects.delete - used with storage.objects.create to update the statefile

Amazon S3 Buckets

config/s3/config.json
{
"credentials": {
"s3": {
"s3Endpoint": "s3.amazonaws.com",
"s3Port": 443,
"s3AccessKey": "accesskey",
"s3SecretKey": "secretkey",
"s3UseSSL": true,
"s3Region": "bucket-region-or-null"
}
},
"projectState": {
"source": "s3",
"bucketName": "my-bucket",
"path": "path/to/stateFile.json"
}
}
s3Region is an optional parameter, indicating where your S3 bucket is located. However, we have encountered some cases where we got S3: Access Denied error when it is not defined. We recommend setting this property whenever possible. Usually, these are identified by access keys starting with ASIA...
To fully use Robosaur with S3 buckets, these are the IAM Roles required:
  • s3:GetObject
  • s3:GetObjectAcl
  • s3:PutObject
  • s3:PutObjectAcl
  • s3:DeleteObject

Azure Blob Storage

config/azure-blob-storage/config.json
{
"credentials": {
"azure": {
"connectionString": "my-connection-string-from-storage-account",
"containerName": "my-container-name"
}
},
"projectState": {
"source": "azure",
"bucketName": "my-container-name",
"path": "path/to/stateFile.json"
},
}
Both connectionString and containerName are required.
You can obtain your connectionString by copying one of the connection strings from your Azure Storage Account.
The containerName is where you would upload your projects, inside a projects folder.
Copy link
Outline