Large files

We have an extra data integration where you can upload larger files. Instead of uploading your data directly to our File Explorer, you can follow the best practices by splitting the code and data, by connecting to datasets and adding your data files via our Integrations. Your notebooks and other code and config files are located in File Explorer and your data is in the Integrations sidebar.

You can upload your files into a Shared dataset by drag&dropping them into Shared dataset modal. Then you add a name for this Integration, and you're good to go. After creating a Shared dataset, click on "Connect" in the Integration sidebar and your files will be available in /datasets/{integration name} in your environment. Right now, you can also edit the data stored by adding/deleting/modifying them directly in the mounted integration folder.

Note that Shared Datasets are just GCS buckets that we create and manage for you. This might cause slower performance for a large number of files (e.g. 50+ images for training neural networks).