Podemos apagar o pyspark dataframe usando python

df.rdd.saveAsPickleFile(filename)
Relieved Rook