"Scheherazade, you easily make up stories of Egypt or any country you please."

Reading Plato’s Phaedrus¹ again I was struck by this line in relation to discussion of fabrication or hallucination in tools like ChatGPT.

Phaedrus
Socrates, you easily make up stories of Egypt or any country you please.

Musing on that line I tweeted:

I ran the question through various generative search and search-like systems (pre-pending only “Please answer this question:”) and was surprised at how they all got it “wrong”:

• ChatGPT-3.5 & ChatGPT-4: Scheherazade

• Claude 2: Scheherazade

• Google’s Bard: ChatGPT

• Inflection AI’s Pi: Scheherazade

• llama-2-70b-chat (at Perplexity Labs): [DECLINED TO ANSWER]

• Perplexity AI: Scheherazade

  • w/ Copilot: Scheherazade
    • Note: I declined to answer Copilot’s clarifying question (which was great, something like ‘Which do you think it is?’).
      • Aside: Note the Copilot question is not retained for review by the user.
    • Note: I generally have the AI Model (in Perplexity’s Settings) set to Perplexity, rather than GPT-4 or Claude 2.

• You.com’s YouChat: Scheherazade

  • w/ GPT-4: ChatGPT

```