The default way of customizing environment is installing programs and packages using terminal, where you can become root by running sudo command and therefore you can install any libraries or tools you may need. Deepnote also helps you keep track of installed python packages, and installs them when a project is restarted or accessed after longer idle period. It uses init.ipynb to achieve that - you can think of it as a start-up script. For more info, see the init.ipynb documentation.

However, the init.ipynb notebook runs every time the project hardware starts up, which might become time consuming or even annoying. Our Custom environment cache allows you to describe your environment in a simplified Docker syntax, build it and have it ready whenever you open your Deepnote project. The Custom environment cache is defined in the Dockerfileplaced in the environment tab.

Restrictions in the Custom environment Dockerfile

Even though the Custom environment cache definition file is called Dockerfile, it does not support the full syntax you might be used to. In order to stay compatible with Deepnote, you can only use the following statements:

• FROM clause, that needs to point to our base image FROM gcr.io/deepnote-200602/templates/deepnote

• RUN clause. This allows you to run arbitrary commands, for example install libraries or tools you are using in your project. If you need root privileges, you can use sudo in your RUN commands.

• ENV clause. You can use this to set environment variable or to define the default jupyter kernel for creating new notebook files.

Duplicating projects with a Custom environment

The duplicates of a project with a custom environment will retain the custom environment configuration.

Tool and library versions, and reproducibility

Even though the Custom environment cache is implemented using Docker images, it doesn't primarily serve the reproducibility problem. The aim of the feature is to significantly speed up the start time of your projects. In other words, you should consider it to be only a cache at this point.

Since Deepnote is still a fast evolving product, we might need to change the base image often and rebuild your caches to ship you the latest features and bugfixes. Hence, we do not recommend you to rely on the fact that the image will not be rebuilt at arbitrary times. If you require a specific version of a library, we recommend you to specify it in your RUN clause so you are not affected by rebuilds.

Default Dockerfile custom environment cache definitions

FROM gcr.io/deepnote-200602/templates/deepnote
# Your RUN commands (sudo apt-get install -y <package>)
# Your RUN commands (pip install <package>)

Examples of Custom environment cache

TensorFlow example

# Copy this to Dockerfile in environment tab
FROM gcr.io/deepnote-200602/templates/deepnote
RUN pip install tensorflow==2.2.0

Face recognition repo​

Quick create new project via www.deepnote.com/launch?template=face_recognition​

# Copy this to Dockerfile in environment tab
​
FROM gcr.io/deepnote-200602/templates/deepnote
​
RUN sudo apt-get -y update
RUN sudo apt-get install -y --fix-missing \
    cmake \
    libgtk2.0-dev 
RUN sudo apt-get clean && rm -rf /tmp/* /var/tmp/*
​
RUN cd ~ && \
    mkdir -p dlib && \
    git clone -b 'v19.9' --single-branch https://github.com/davisking/dlib.git dlib/ && \
    cd  dlib/ && \
    python3 setup.py install --yes USE_AVX_INSTRUCTIONS
​
RUN pip install face_recognition
​

​