This is a problem that has plagued me a few times and is usually due to outputting too much in a single cell. The Notebook will run out of memory and crash, but it will contain the output that caused the notebook to crash. The notebook will crash every time you open it, for some browsers you may even see an ‘out of memory’ error message. The solution to this problem is to delete the vast amount of output, without opening the notebook. You may think this impossible, but there is a way via some code in a separate notebook.
The code you need is as follows:
Here’s a version you can copy and paste:
from nbformat import read, write
def Remove_Output(Book):
for cell in Book.cells:
if hasattr(cell, “outputs”):
cell.outputs = []
if hasattr(cell, “prompt_number”):
del cell[“prompt_number”]
Book= read(open(“Insert notebook name here.ipynb”), 4)
Remove_Output(Book)
write(Book, open(“Insert new name for notebook.ipynb”, “w”), 4)
To use the code you need to provide the name of the notebook you want to fix in the first bold section (underlined blue in print screen) and the name of what you want the fixed notebook to be called in the second bold section (underlined red in print screen). Then run the code and the output will be cleared and you have access to your notebook again, just remember to remove the code which caused it to crash in the first place!
If you want more detail on how this works, I have included the specifics below.
We make use of the “nbformat” package which contains the reference implementation of the Jupyter Notebook format, and Python APIs for working with notebooks. From this we can take the APIs for reading and writing Jupyter notebooks. So, using this we then go through each cell in the notebook and ask if it has certain attributes using the “hasattr” command, which will return true if we provide the wanted attribute. What we look for are the attributes of “outputs” (of which we would then set that cell to an empty list, thus deleting the output), and “prompt number” which we will delete. The function above is called upon the notebook in question which is opened via the nbformat read command. Once done we then use the nbformat write command to save the cleared-out notebook under a new name.
On a side note on what “prompt numbers” are: Jupyter notebooks caches inputs and outputs as we go along running our notebooks and prompt numbers are references that link information that has been cached to the cells. You can see this as the numbers that pop up by cells as you run them!
Really helpful. Was stuck on it for a few hours now, but it was just a quick fix. Thanks a lot!
Thanks a lot, helped me in crisis
Worked a treat, thanks
after running the code, I still get this error, where have i gone wrong?
Cell In[4], line 7
if hasattr(cell, “outputs”):
^
SyntaxError: invalid character ‘“’ (U+201C)
Hi Vincent,
The invalid character Python is talking about is your double apostrophes. This is an annoying error to have as technically you’ve done nothing wrong, but these apostrophes come in two types; curly (“ ”) and straight (” “). Python likes straight, but will break with curly. This may have come up if you’ve pasted into something like Microsoft Word and then into your editor, or it may be if you’ve copied directly from the blog post the word processor for the blog may have changed them itself. If you could get back to me on that I would really appreciate it as I’ll be more careful posting code in my blogs to avoid others getting the error in the future if the apostrophes are being converted when the blogs are published.
Regardless, if you go into the code and change your apostrophes for straight ones (” “) the code should then run.
Many thanks,
Lewis
Hi Vincent,
I’ve just confirmed it’s converting the apostrophes to curly when I publish as it’s done it with my reply just. If you insert the apostrophes yourself it should work.
Many thanks,
Lewis
Great aid. Thanks a lot.
this really works. Thanks a lot!
Thanks a lot! So helpful!
Your script has been really helpfull! Though, I have to state a matter that occured. I kept running the following, as shown:
from nbformat import read, write
def Remove_Output(Book):
for cell in Book.cells:
if hasattr(cell, “outputs”):
cell.outputs = []
if hasattr(cell,”prompt_number”):
del cell[“prompt_number”]
Book=read(open(“C:/Users/Despoina/Desktop/DIPLOMATIKI/PROBLEM_OF_ASSIGNMENT/TAXI+IX+TRUCK/DOKIMI.ipynb”),4)
Remove_Output(Book)
write(Book,open(“C:/Users/Despoina/Desktop/DIPLOMATIKI/PROBLEM_OF_ASSIGNMENT/TAXI+IX+TRUCK/DOKIMI2.ipynb”,”w”),4)
With the following result:
home() missing 1 required positional argument: as_version
Then, I tried the below mentioned and I had the so wanted result.
from nbformat import read, write
def Remove_Output(Book):
for cell in Book.cells:
if hasattr(cell, “outputs”):
cell.outputs = []
if hasattr(cell,”prompt_number”):
del cell[“prompt_number”]
Book=read(open(“C:/Users/Despoina/Desktop/DIPLOMATIKI/PROBLEM_OF_ASSIGNMENT/TAXI+IX+TRUCK/DOKIMI.ipynb”,’r’,encoding=’utf-8′,errors=’ignore’),4)
Remove_Output(Book)
write(Book,open(“C:/Users/Despoina/Desktop/DIPLOMATIKI/PROBLEM_OF_ASSIGNMENT/TAXI+IX+TRUCK/DOKIMI2.ipynb”,”w”,encoding=’utf-8′,errors=’ignore’),4)
Hi Despoina,
From what I can understand, I think the problem lies in the encoding of your notebook. The reason I believe this is because the ‘open()’ command in python by default will use ASCII encoding, but when you specified ‘UTF-8’ you got the code to work. Despite the error message pointing to the ‘read()’ and ‘write()’ commands not having the ‘as_version’ argument declared (we have infact declared this, it’s the ‘4’ at the end of the line of code), I think it was an error in the ‘open()’ we call within the ‘read()’ and ‘write()’ commands because of the encoding of your notebook that was causing the above error.
Many thanks,
Lewis
Thank you very much. It worked perfectly. For those directly copying from this post, make sure to change curled double quotes to straight double quotes and indent your code to avoid generating errors
This worked perfectly.
I opened a new notebook in the same directory and pasted and executed the code.
All I had to change were the curly double quotations – which you had already mentioned in another comment.
Then I opened the notebook with the new name.
I can’t thank you enough!
Did wonders!
Thanks 👍
This was super helpful.
name ‘read’ is not defined . i am not able to resolve it help will be appreciated on this
Hi Anish,
Just want to make sure you executed the following line:
‘from nbformat import read, write’
This is what brings through the functionality for the Read and Write commands, and I would expect to find the error you are having if these were not imported?
If you did execute that line, just make sure you get a message saying the libraries were imported correctly.
Failing this we want to make sure we are using a version of the library that are compatible with the version of Python you are using, it can happen from time to time where these libraries have not yet updated themselves to be in line with the most up to date Python version available. The usual solution if that is the case is to find out what version of Python the library author recommends and use that.
Hope this helps, if not then give this a reply and we can have another look!
Many thanks,
Lewis
I am getting this error : File “”, line 5
if hasattr(cell, “outputs”):
^
SyntaxError: invalid character in identifier
Hi Siladitya,
I’ve been a bit perplexed by this error to be honest, as there are no hidden characters in that string which is usually the cause when you get this error. After some extensive googling though I may have found the cause. Apparently “” don’t work if you are using a keyboard set to English US (international) as opposed to English US. Could you check this setting and see if that might be the problem please?
Many thanks,
Lewis
Thanks a ton! I was stuck on it for hours and tried several methods, but your code solved it in a jiffy. The best solution ever!