Using Storage Manager with IAM Role
AWS IAM Role Configuration
We have added a new feature in Columnstore 5.5.2 that allows you to use AWS IAM roles in order to connect to S3 buckets without explicitly entering credentials into the storagemanager.cnf config file.
You will need to modify the IAM role of your Amazon EC2 instance to allow for this. Please follow the AWS documentation before beginning this process.
For a complete list of AWS service endpoints, please visit the AWS reference guide.
Edit your Storage Manager configuration file located at /etc/columnstore/storagemanager.cnf in order to look similar to the example below (replacing those in the [S3] section with your own custom variables):
[ObjectStorage] service = S3 object_size = 5M metadata_path = /var/lib/columnstore/storagemanager/metadata journal_path = /var/lib/columnstore/storagemanager/journal max_concurrent_downloads = 21 max_concurrent_uploads = 21 common_prefix_depth = 3 [S3] ec2_iam_mode=enabled bucket = my_mcs_bucket region = us-west-2 endpoint = s3.us-west-2.amazonaws.com [LocalStorage] path = /var/lib/columnstore/storagemanager/fake-cloud fake_latency = n max_latency = 50000 [Cache] cache_size = 2g path = /var/lib/columnstore/storagemanager/cache
Note: This is an AWS only feature. For other deployment methods, see the example here.