This is a problem that has plagued me a few times and is usually due to outputting too much in a single cell. The Notebook will run out of memory and crash, but it will contain the output that caused the notebook to crash. The notebook will crash every time you open it, for some browsers you may even see an ‘out of memory’ error message. The solution to this problem is to delete the vast amount of output, without opening the notebook. You may think this impossible, but there is a way via some code in a separate notebook.

The code you need is as follows:

Here’s a version you can copy and paste:

from nbformat import read, write

def Remove_Output(Book):

for cell in Book.cells:

if hasattr(cell, “outputs”):

cell.outputs = []

if hasattr(cell, “prompt_number”):

del cell[“prompt_number”]

Book= read(open(“Insert notebook name here.ipynb”), 4)
Remove_Output(Book)
write(Book, open(“Insert new name for notebook.ipynb”, “w”), 4)

To use the code you need to provide the name of the notebook you want to fix in the first bold section (underlined blue in print screen) and the name of what you want the fixed notebook to be called in the second bold section (underlined red in print screen). Then run the code and the output will be cleared and you have access to your notebook again, just remember to remove the code which caused it to crash in the first place!

If you want more detail on how this works, I have included the specifics below.

We make use of the “nbformat” package which contains the reference implementation of the Jupyter Notebook format, and Python APIs for working with notebooks. From this we can take the APIs for reading and writing Jupyter notebooks. So, using this we then go through each cell in the notebook and ask if it has certain attributes using the “hasattr” command, which will return true if we provide the wanted attribute. What we look for are the attributes of “outputs” (of which we would then set that cell to an empty list, thus deleting the output), and “prompt number” which we will delete. The function above is called upon the notebook in question which is opened via the nbformat read command. Once done we then use the nbformat write command to save the cleared-out notebook under a new name.

On a side note on what “prompt numbers” are: Jupyter notebooks caches inputs and outputs as we go along running our notebooks and prompt numbers are references that link information that has been cached to the cells. You can see this as the numbers that pop up by cells as you run them!

Tags: ,

« Previous Post
Next Post »