Could Storage Gateway help here? EC2 fronts the FTP endpoint to a file server that is Storage Gateway? Source: almost 2 years ago

One option is https://aws.amazon.com/storagegateway/ (plus VPN). Source: almost 2 years ago

That helps, thank you. You might have a look at AWS' Storage Gateway, more specifically the S3 File Gateway. Source: almost 2 years ago

I *think* storage gateway can do this (https://aws.amazon.com/storagegateway/). Also, several of the Amazon FSx services can, I think. They all basically mean you have a remote file system to which you write, and that file system gets backed up to FSx. Source: about 2 years ago

You can deploy Storage gateway in your local environment, and copy the files to storage gateway, it will automatically sync the files to s3. Source: over 2 years ago

We use Kubernetes as our deployment platform. Any feedback on one of these open source data catalogs ? - https://atlas.apache.org/#/ - https://opendatadiscovery.org/ - https://open-metadata.org/ - https://marquezproject.github.io/marquez/ - https://datahubproject.io/ - https://www.amundsen.io/ - https://ckan.org/ - https://magda.io/. Source: over 1 year ago

CKAN (https://ckan.org/) is what data.gov and most state governments use. Source: almost 2 years ago

Our first instinct is to use [CKAN](https://ckan.org) for cataloging (and storage, with modifications), especially since we know it and know that it has been used successfully elsewhere. However, we suspect that more specialized/better tools exist for this, thus why I kindly ask for your insights. Source: over 2 years ago

We publish all our data on the [Data Portal](https://data.nhm.ac.uk), a Museum project that's been running since 2014. Instead of MediaWiki it runs on an open-source Python framework called [CKAN](https://ckan.org), which is designed for hosting datasets - though we've had to adapt it in various ways so that it can handle such large amounts of data. Source: over 3 years ago