How to (intermittently) skip certain cells when running IPython notebook?
Asked Answered
G

11

80

I usually have to rerun (most parts of) a notebook when reopen it, in order to get access to previously defined variables and go on working.

However, sometimes I'd like to skip some of the cells, which have no influence to subsequent cells (e.g., they might comprise a branch of analysis that is finished) and could take very long time to run. These cells can be scattered throughout the notebook, so that something like "Run All Below" won't help much.

Is there a way to achieve this?

Ideally, those cells could be tagged with some special flags, so that they could be "Run" manually, but would be skipped when "Run All".

EDIT

%%cache (ipycache extension) as suggested by @Jakob solves the problem to some extent.

Actually, I don't even need to load any variables (which can be large but unnecessary for following cells) when re-run, only the stored output matters as analyzing results.

As a work-around, put %%cache folder/unique_identifier to the beginning of the cell. The code will be executed only once and no variables will be loaded when re-run unless you delete the unique_identifier file.

Unfortunately, all the output results are lost when re-run with %%cache...

EDIT II (Oct 14, 2013)

The master version of ipython+ipycache now pickles (and re-displays) the codecell output as well.

For rich display outputs including Latex, HTML(pandas DataFrame output), remember to use IPython's display() method, e.g., display(Latex(r'$\alpha_1$'))

Guizot answered 11/10, 2013 at 2:25 Comment(1)
If you don't need to redisplay the output, you can define your own skip magic like I did here: https://mcmap.net/q/138878/-simple-way-to-choose-which-cells-to-run-in-ipython-notebook-during-run-allFatherhood
I
44

Currently, there is no such feature included in the IPython notebook. Nevertheless, there are some possibilities to make your life easier, like:

  • use the %store or maybe better the %%cache magic (extension) to store the results of these intermittently cells, so they don't have to be recomputed (see https://github.com/rossant/ipycache)

  • add a if==0: before the cells you don't want to execute

  • convert these cells to raw cells (but you will loose the already stored output!)

(see discussion at https://github.com/ipython/ipython/issues/2125)

Illicit answered 11/10, 2013 at 12:41 Comment(5)
Great to know about ipycache extension, which functions similarly to what I'm doing (manually) now, but only better! The %%cache magic effectively serves as the special flag when "Run All", and manual re-run requires simply cache file deletion. Brilliant! Thanks, Jakob~Guizot
Indeed using %%cache is much more convenient than manually handling with %store. %store need to send one by one the variables to save, but can read all at time, little pointless. %%cache works perfect for whole cell, and %store works better for single variables.Augustina
ipycache is no longer maintained- do you know any other solutions?Hiltner
@Hiltner currently I use the freeze extension. You might ask Rossant directly.Illicit
IMHO, %%script echo skipping (or possibly %%script true) is a better and universal solution and should become the accepted answer :-)Shelashelagh
I
77

Though this isn't exactly what you seem to be looking for, if you wish to entirely omit the execution of a cell (where no cached results are loaded), you can add the following hack at the beginning of a cell (assuming you are using a Unix-based OS):

%%script false

or a variant (working as of early 2020 -- see here for explanation):

%%script false --no-raise-error

As noted in comments, the shorter

%%script true

also works, though maybe not on all platforms.

Improvement answered 18/5, 2016 at 19:9 Comment(8)
Anyone know how to do this on Windows?Cullan
It works perfectly on windows. The notebook output will be "Couldn't find program: 'false'".Vitriol
This stopped working some time in 2019. You can find new workarounds here.Shelf
As a heads up, this gets rid of existing output. Not useful if you want to keep previous output of the cell.Huntingdon
Here's another similar one (%%script echo): https://mcmap.net/q/259408/-how-to-intermittently-skip-certain-cells-when-running-ipython-notebookHypogastrium
instead use this as it works on all known platforms: %%script echo skippingShelashelagh
What about %%script true instead of %%script false --no-raise-error?Siren
I would put the option first to show that it belongs to the %%script magic, not the false command: %%script --no-raise-error false --- @Siren %%script true will probably fail on some non-POSIX platforms like Windows. Also the falsy return status could be probably used as an explicit indication that the cell did not (correctly) execute?Photomap
I
44

Currently, there is no such feature included in the IPython notebook. Nevertheless, there are some possibilities to make your life easier, like:

  • use the %store or maybe better the %%cache magic (extension) to store the results of these intermittently cells, so they don't have to be recomputed (see https://github.com/rossant/ipycache)

  • add a if==0: before the cells you don't want to execute

  • convert these cells to raw cells (but you will loose the already stored output!)

(see discussion at https://github.com/ipython/ipython/issues/2125)

Illicit answered 11/10, 2013 at 12:41 Comment(5)
Great to know about ipycache extension, which functions similarly to what I'm doing (manually) now, but only better! The %%cache magic effectively serves as the special flag when "Run All", and manual re-run requires simply cache file deletion. Brilliant! Thanks, Jakob~Guizot
Indeed using %%cache is much more convenient than manually handling with %store. %store need to send one by one the variables to save, but can read all at time, little pointless. %%cache works perfect for whole cell, and %store works better for single variables.Augustina
ipycache is no longer maintained- do you know any other solutions?Hiltner
@Hiltner currently I use the freeze extension. You might ask Rossant directly.Illicit
IMHO, %%script echo skipping (or possibly %%script true) is a better and universal solution and should become the accepted answer :-)Shelashelagh
S
28

Here's a simple and universal solution without the need for workarounds: Simply type this as the top line of the cell to skip the cell:

%%script echo skipping

On all tested platforms it will proceed without executing the code in the cell.

On Mac and Unix-like platforms it simply prints "skipping" as cell output to indicate that it has been skipped.

On Windows it displays an error message instead (something like "command echo not found"), unless you happen to have installed something that provides an echo command (such as cygwin, gnuwin32, bash, or git), but anyway this error message is ignored and harmless.

I guess converting the cell to a raw cell (as in accepted answer) sort of works, but I'm not a fan because it no longer formats the contents properly as code.

Some of the other proposed solutions are more platform-specific.

[EDIT (adding 2024-04-14 -- after four comments below which are thus comments solely on original %%script echo skipping above): As noted by some others, this one might be another alternative that is fairly universal (I'm not able to test it on multiple platforms at the moment):

%%script true

]

Shelashelagh answered 1/5, 2020 at 14:41 Comment(4)
Works in Hydrogen on UbuntuSiren
Works much better than %% false as it doesn't create an error. This is very good if you don't need to keep old output.Mullein
Works beautifully on Google Colab too...Monotint
Also works with Dataiku.Helenehelenka
G
13

If no cached results are expected to be loaded, I find the Freeze nbextension quite useful to this end.

enter image description here

Although unofficial, I strongly recommend to give these notebook extensions a try if you have never used them before.

To install the extension machinery,

$ pip install jupyter_contrib_nbextensions && jupyter contrib nbextension install

To enable the Freeze extension, launch jupyter notebook and open a new notebook, from the menu select Edit > nbextensions config, and then check Freeze.

Guizot answered 15/12, 2019 at 2:55 Comment(2)
Looks like the best solution. When hitting "Restart and Run All" or other commands, the frozen cells won't be re-executed but the cell outputs are still there.Boltonia
This is the best solution. The cell won't run, even you press run and the output will still be there.Katabolism
F
3

This question is a bit older, but the most convenient answer seems to be missing. You can use the 'initialization cells' from the NBextensions. Once installed/activated, you can in any notebook mark cells as 'initialization cells' which can then be run with a specific button.

  • Install NBextensions:here
  • Activate 'initialization cells' when you launch the jupyter dashboard

  • In your notebook, in the 'view' menu choose 'cell toolbar' then 'initialization cell'

  • Next to each cell there is now a checkbox. Mark all the cells you want to run for initialization
  • When reopening a notebook, click the button that looks like a pocket calculator to run all the initialization cells.
Friedly answered 18/5, 2020 at 8:3 Comment(0)
R
2

The simplest way to skip python code in jupyter notebook cell from running, I temporarily convert those cells to markdown.

Rakia answered 23/11, 2020 at 13:31 Comment(3)
but in this way you lose the outputEun
why someone needs output for the code he/she doesn't want to run?Rakia
In data science you don't want to run the whole notebook every time, but still you want to be able to see the resultsEun
Q
2

Simplest solution: just add % character as first(new) line of the cell and entire cell's content will be skipped.

Cons:

  • A single line error message will be shown under the cell:

    UsageError: Line magic function % not found.

Pros:

  • The error message is very short and therefore doesn’t impair the notebook legibility.
  • It's the quickest on/off solution for specific cells.
Quietude answered 14/5, 2023 at 17:19 Comment(0)
O
1

For the cells you wish to skip when you press Run All you can use try/except blocks, where you try to display the already calculated information in the try block, and perform the calculation in the except block.

Take this cell for example:

my_results = 2 # Assume this is a bigger calculation
print(my_results)
print('displayed my results.')

To skip the calculation for this cell in subsequent runs, change the contents of this cell to the following:

try:
  print(my_results)
  print('displayed state value')
except:
  my_results = 2 # Assume this is a bigger calculation
  print(my_results)
  print('displayed newly calculated value')

The first time you run this cell, it will attempt to print the value of the state variable my_results. This throws an exception so it jumps to the except block and does the actual calculation of my_results (which in this case just makes it equate to 2). At the end of the first run the output for this cell would be:

2
displayed newly calculated value

When you run the cell a second time (whether that be manually or via Run All), the try block will first execute but this time since the variable is available in the state, it does not throw an exception. Instead it displays the result and the except block is never run. At the end of the second run the output of this cell would be:

2
displayed state value

This doesn't meet your explicit criteria that the cell should be completely skipped, but effectively the calculation is skipped.

If the displaying of the information is more complex than using a single print or display, it would probably be cleaner if you make the displaying routine into a function like so:

def print_my_results(my_result):
  print(my_result)
  print('displayed my result.')

try:
  print_my_results(my_results)
except:
  my_results = 2 # Assume this is a bigger calculation
  print_my_results(my_results)
Organo answered 8/9, 2022 at 4:31 Comment(3)
this is clever, but can you achieve the same thing with a simple if...else block?Wall
Good observation @MattMontag, I had to think about it for a little bit but I would say this try/catch is better than an if/else block: With an if/else you need to track and account for every used variable in the if statement. There can be a situation where you forget to update that if statement. With a try/catch you don't need to worry about that. You can just focus on printing the variable and if you forget a calculation it will throw another exception in the except block prompting you to add the missing calculation upstream or in the except block.Organo
I like it! But in the try: block how about just two lines: my_results (which will throw the exception if undefined) and then print("Displaying previously-calculated results:"). Then instead of a print(my_results) in the except: block, just put my_results after the block (unindented) as the last line of the cell. This will be much better, in case my_results is something you don't want to print() per se, like a pandas dataframe.Shelashelagh
S
0

The %%script false solution stopped working some time in 2019.

Here are some other available workarounds. These are based on programs ignoring their arguments when you tell them not to expect any. Here are some easy examples:

Perl:

%%perl -e0
​
for i in range(10): print(i)

Here you're running: perl -e '0' cellcontents

A more memorable version:

%%perl -eat
​
for i in range(10): print(i)

Here you're running: perl -e 'at' cellcontents

Bash:

%%bash -c :

for i in range(10): print(i)

: is a no-op in bash, so you're running: bash -c : cellcontents

I haven't looked at the external magic implementation code, but I'm pretty sure "cellcontents" are passed as arguments and won't be interpreted by shell by mistake, say if you were to include ; in them and accidentally inject some bad code. But I can't guarantee you that.

I'm sure you can come up with other creative solutions, by looking at the supported programs here: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics

Shelf answered 19/1, 2020 at 0:38 Comment(3)
%%script false --no-raise-error works for the purpose on the Jupyter I tried todaySuboxide
indeed, it does, but it's a way too long to type and remember... Ideally, ipython should have a %%script noop or something similar, so it's a feature and not a workaround.Shelf
Why such complicated solutions. There is a POSIX command which always returns true. You can use: %%script truePhotomap
Z
0

The best solution I found is to define a custom magic function in Jupyter Notebook. See: (17-Apr-2024): How to Skip Cell Execution in Jupyter Notebook at: https://kioku-space.com/en/jupyter-skip-execution/

(1) How to Skip a Cell Execution?
To skip cell execution, create a custom magic command as follows:

  1. First, create a skip magic command that does nothing.
from IPython.core.magic import register_cell_magic
@register_cell_magic
def skip(line, cell):
    return
  1. To skip a cell, simply add %%skip at the top line of the cell.
%%skip
<code>

This Link also describes: (2) How to Conditionally Skip a Cell Execution, and (3) How to Skip All Subsequent Cells Based on a Condition

Zoochemistry answered 1/7 at 9:2 Comment(0)
B
-1

A workaround I sometimes use:

if 0:
    time_consuming_function()
Barehanded answered 15/6, 2023 at 10:10 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.