TIL to use raw triple double quotes for doctests with escapes

Use r""“raw triple double quotes”"" if you use any backslashes in your docstrings.

You can also double escape (//), it seems, per Claude 2 (see below).

Below I found one of the clearest and most straightforward fails(?) of mine, that I’ve stumbled upon on ChatGPT-4. My perception of using the various LLMs is that I generally know when I am out of my depth or the system is outside its performance envelope. Many of my interactions are complicated enough it seems tough, without some tooling, to trace where the faultlines ruptured. But this was more clear.

I’d asked ChatGPT-4 (repeatedly): How do I do a doctest that requires sending an escaped quotation mark in the parameters? (I left feedback.)

I probably could have improved my prompt! Trying the first few variations (adding explicit notes to reference the python documentation, or explicit context of docstrings, and adding the exception itself) didn’t make a difference. But adding the full function and exception alongside my initial question (including my tried/tired typo) did provide the correct answer, implicitly (in the code provided in the response), though not explicitly called out.

But a Stack Overflow result appearing for g[use single and duoble quotes in a string in python doctest] (sic) led me to the answer that quoted from PEP 257 – Docstring Conventions

I then checked a few other search tools, just with my raw prompt:

Perplexity AI’s Copilot also failed, but did have the Stack Overflow question I found linked to as a Related question! I left this feedback:

Use r""“raw triple double quotes”"" if you use any backslashes in your docstrings.

PEP 257 – Docstring Conventions

Phind had helpful but not exactly correct suggestions. The examples use the raw r on the test string, not the full docstring (which didn’t work for me). It also provided a suggestion of double backslashes (though the code rendered with triple backslashes), which does work.

Double backslashes is also what Claude 2 suggested. [link to image b/c Claude 2 still doesn’t support sharing interactions]

You.com also struggled, with the search results not looking particularly relevant either.

And llama-2-70b-chat and codellama-34b-instruct struggled as well (link to image) (at labs.perplexity.ai

I’ll fully acknowledge it was late at night and I didn’t have a ‘good’ query/prompt. I get into plenty of struggles and rabbit holes and black holes in my various searches, I just thought this one seemed particularly straightforward.

Note also that searching the raw prompt on Google, like g[How do I do a doctest that requires sending an escaped quotation mark in the parameters? Like this: parameter: ‘“custom instructions” in Siri’ Tired: >>> slugify(“‘"custom instructions" in Siri’”, args) But I get a syntax error.], is striped of its quoates but also has a helpful Stack Overflow question and answer in the top results: https://stackoverflow.com/questions/11765401/doctest-involving-escape-characters

See also the ChainForge playground for a few of my prompt variations - Falcon.7B is wild! (Arawjo et al., 2023) - note: may not mobile friendly at the moment