By default, Deepnote projects run in containers on Debian Buster with Python 3.7, and a number of pre-installed packages. It’s meant to "just work" for the majority of projects, and if you’re a beginner, you most likely don’t need to configure anything. However, for advanced users, Deepnote offers a large amount of flexibility.
There are 3 major ways of installing both system-level dependencies and Python packages: installing them ad-hoc, using a start-up script, and by changing the underlying image.
pip install <package_name> or
apt install <library>, either in the terminal or with the
! prefix in a notebook. This will immediately install the library for you to use, so you can see if it helps you. However, when the hardware restarts, the environment will revert back to the default state.
Whenever your project hardware starts up, we run some initializing Python code. This code is in the
init.ipynb notebook which can be accessed from the environment tab. By default, it installs packages from
./requirements.txt. Read more.
For bigger changes, you can select a different environment. Deepnote maintains a number of supported runtimes – a few versions of Python and R. Simply pick one from the dropdown, and the hardware will restart with the selected environment.
You can also choose the
Local ./Dockerfile option that will use the
Dockerfile file in the root of your project. A common use-case is to base this Dockerfile on one of our runtimes, which you can do by specifying the
FROM directive at the beginning of the file (such as
You can also bring your own Docker image. This is an extremely powerful feature because it allows you to pick any of the preconfigured images from public repositories, such as Dockerhub, or to bring your own internal ones configured by your company.
Teams can use Docker images from their private image repositories (we support Docker Hub, Google Cloud Repository and Amazon ECR). To use a private image repository, create an integration for respective provider to enter your credentials. Then connect the integration to any project where you wish to use your private Docker images.
The rest of the guide will describe the options for custom environments.
Deepnote supports any Python version >= 3.6. You can pick 3.7, 3.8 and 3.9 from the dropdown, or use a custom Docker image for a more specific selection.
You can also pick from R 3.5.2 and 4.0.3, or use a custom Docker image for a more specific selection. Read more.
You can run Deepnote on any Docker image, as long as it satisfies a few requirements. Just add a new image from a public repository, optionally give it a name, and it’s ready to use. The initial load might take slightly longer, while the image gets downloaded.
There must be an installation of Python with version >= 3.6, which can be referenced using the
There must be an installation of
pip, configured to install packages into a location visible to
The image must contain
If you want to run a different kernel, read more here.
When you add a custom image in your personal project, you will be able to use it in all other projects as well. When you add a custom image in a team project, all team members will be able to use it in other team projects as well.
Sometimes you may wish to make small modifications to the source image. You can build it directly in Deepnote, just pick the local Dockerfile option in the dropdown.
Even though the definition file is called Dockerfile, it does not support the full syntax you might be used to. In order to stay compatible with Deepnote, you can only use the following statements:
FROM clause. You can start from any public base image, or pick one of the Deepnote environments as a base.
RUN clause. This allows you to run arbitrary commands, for example, install libraries or tools you are using in your project. The commands run as root, so you don't need to use sudo.
ENV clause. You can use this to set environment variable or to define the default jupyter kernel for creating new notebook files.
The time-out for builds is 10 minutes.
You can also use images built locally or from a different platform, like your CI/CD system. Note, the local builds allow for faster iteration thanks to layer caching. You will still need to host the image on an online registry.
Check out this guide for a quickstart on building images locally and hosting them on Docker Hub.
The support for private image repositories, which require authentication, is coming soon as part of the Team plan.
The duplicates of a project with a custom environment will retain the custom environment configuration in these cases:
When the project is using one of the default Deepnote environments
When the project is using the local Dockerfile built within Deepnote
(Coming soon) When the project gets duplicated within the team space
If your project was using the default configuration, nothing changes for you.
If your project was using a custom-built Docker image, it will keep using it. The Dockerfile will move from a hidden folder into your regular workspace. In case of a conflict, the already-present Dockerfile will be renamed to Dockerfile.old.