Spark

Pyspark

To use pyspark in Deepnote, you need to install a JDK first.
1
! sudo apt-get update
2
! sudo mkdir -p /usr/share/man/man1
3
! sudo apt-get install -y openjdk-11-jdk
4
! pip install pyspark
Copied!
Now you can test it in a notebook cell with:
1
from pyspark import SparkContext
2
sc = SparkContext("local", "First App")
Copied!
or in a terminal by opening an interactive shell:
1
pyspark
Copied!
Copy link