This is a team/professional feature. To get access, please reach out in our support chat or via email.

The default way of customizing environment is installing programs and packages using terminal, where you can become root by running sudo command and therefore you can install any libraries or tools you may need. Deepnote also helps you keep track of installed python packages, and installs them when a project is restarted or accessed after longer idle period. It uses init.ipynb to achieve that - you can think of it as a start-up script. For more info, see docs.

However, init.ipynb runs every time after project executor is restarted, which might become time consuming or even annoying. Custom environment cache allows you to describe your environment in a simplified Docker syntax, build it and have it ready whenever you open your Deepnote project. Custom environment cache is defined in Dockerfileplaced in the environment tab.

Restrictions in custom environment cache Dockerfile

Even tough the custom environment cache definition file is called Dockerfile, it does not support the full syntax you might be used to from other projects. In order to stay compatible with Deepnote, you can only use the following statements:

• FROM clause, that needs to point to our base image:

  • The default image is:

    FROM gcr.io/deepnote-200602/templates/deepnote

  • If you are using GPUs, use this instead.:FROM gcr.io/deepnote-200602/templates/deepnote_gpu

• Otherwise, only RUN clause is permitted. This allows you to run arbitrary commands, for example install libraries or tools you are using in your project. If you need root privileges, you can use sudo in your RUN commands.

Duplicating projects with custom environment cache

The access to the feature is granted on a per-team basis. A duplicate of a project utilising custom environments, done by a Deepnote user outside of the team with access to the feature, will allow such user to use the pre-defined custom environment. However, no further edits of the environment cache will be possible, apart from reverting to default.

Tool/library versions and reproducibility

Even tough Custom environment cache is implemented using Docker images, it serves a different use case - the aim of the feature is to significantly speed up the start time of your project. In other words, you should consider it to be only a cache at this point.

Since Deepnote is still a fast evolving product, we might need to change the base image often and rebuild your caches to ship you the latest features and bugfixes. Hence, we do not recommend you to rely on the fact that the image will not be rebuilt at arbitrary times. If you require a specific version of a library, we recommend you to specify it in your RUN clause so you are not affected by rebuilds.

Default Dockerfile custom environment cache definitions

Default template

FROM gcr.io/deepnote-200602/templates/deepnote
# Your RUN commands (sudo apt-get install -y <package>)
# Your RUN commands (pip install <package>)

GPU template

FROM gcr.io/deepnote-200602/templates/deepnote_gpu
# Your RUN commands (sudo apt-get install -y <package>)
# Your RUN commands (pip install <package>)

Examples of Custom environment cache

TensorFlow example

# Copy this to Dockerfile in environment tab
FROM gcr.io/deepnote-200602/templates/deepnote
RUN pip install tensorflow==2.2.0

Face recognition repo​

# Copy this to Dockerfile in environment tab
​
FROM gcr.io/deepnote-200602/templates/deepnote
​
RUN sudo apt-get -y update
RUN sudo apt-get install -y --fix-missing \
    cmake \
    libgtk2.0-dev 
RUN sudo apt-get clean && rm -rf /tmp/* /var/tmp/*
​
RUN cd ~ && \
    mkdir -p dlib && \
    git clone -b 'v19.9' --single-branch https://github.com/davisking/dlib.git dlib/ && \
    cd  dlib/ && \
    python3 setup.py install --yes USE_AVX_INSTRUCTIONS
​
RUN pip install face_recognition
​

​