Highlight shell escapes inside timing and profiling cell magics

[ump45nose]

Fixes the case in the issue: a shell escape inside a cell magic whose body is IPython input was highlighted as an error instead of as a command.

The bug

%%time, %%timeit, %%capture and %%prun lexed their body with using(Python3Lexer) — the bare Python lexer, which knows nothing about IPython syntax. So the ! of a shell escape came out as Token.Error:

%%time
!cmd

context	tokens for the !
top level !cmd	Token.Operator
inside %%time (before)	Token.Error
inside %%time (after)	Token.Operator

Which magics get the fix, and why

I checked this against IPython itself rather than guessing, by executing each magic with a shell escape in the body (IPython 9.17.1):

magic	body runs as	!echo … reaches the shell?	change
%%time	IPython input	yes	applies the fix
%%timeit	IPython input	yes	applies the fix
%%capture	IPython input	yes (in captured.stdout)	applies the fix
%%prun	IPython input	yes	applies the fix
%%debug	body run under pdb	n/a	unchanged
%%python, %%python2, %%python3, %%pypy	separate interpreter	no	unchanged
%%writefile, %%file	literal file content	no	unchanged

That is why the change is limited to four rules: the other Python-running magics genuinely do lex plain Python, and the existing test for %%writefile still pins that.

The change

Those four now delegate to using(this) — the current lexer's own root state, which already contains the !-escape rule, the line-magic rules and everything else in ipython_tokens. Using this rather than naming a class keeps the rule correct for both the Python 2 and Python 3 IPython lexers, which share this token table.

Testing

python -m pytest -q
# 15 passed

The CI matrix is Python 3.8–3.13; nothing added uses syntax newer than 3.8.

Eight test cases are added, in the style of the existing test_shell_commands:

• test_shell_commands_inside_ipython_input_cell_magics (4 params) — the body tokenizes as the shell escape it is.
• test_no_shell_commands_where_the_body_is_not_ipython_input (3 params) — %%debug, %%python and %%writefile keep plain-Python highlighting, so the fix cannot silently widen to magics whose bodies are not IPython input.
• test_python_still_lexes_inside_a_timing_magic — delegating to this lexer's root keeps ordinary Python highlighting intact.

Red/green: with only the test changes applied, the four parametrizations of the first test fail and the other eleven pass — so the failure is specific to the reported behaviour, and the guards are genuinely pinning unchanged behaviour rather than passing by accident.

Final-interface output from the reporter's snippet, through the shipped ipython3 entry point with HtmlFormatter(nowrap=True):

%%time
!cmd

<span class="o">%%time</span>
<span class="o">!</span>cmd

and unchanged for a magic where the body is not IPython input:

%%writefile f.py
!cmd

<span class="o">%%writefile</span> f.py
<span class="err">!</span><span class="n">cmd</span>

Notes

• Open PR Keep exception message continuation lines as text #13 (also mine) edits ipython_pygments_lexers.py, but around line 232 in IPythonPartialTracebackLexer; this change is confined to lines 53–150 (ipython_tokens), so the two do not overlap textually.
  Closes failure on shell commands after %%time #11

• I did not change how %%writefile lexes its body. It treats the body as literal text, so Python highlighting of it is arguably also imprecise, but that is a separate behaviour from the one reported here.

---

This PR description was written with AI assistance.