Here are the initial/top results from various generative search tools for the query:[ What is this type of encoding? ' ], re the character entity reference for an apostrophe.
Some of the tools have a correct or useful answer or top-link but render the entity reference in the message to the user as the apostrophe (ChatGPT, You.com). Some tools render the entity reference in the query itself as the apostrophe (Andi, Claude 2; Phind (though also rendering as queried in the response)). Some tools do not render it back to the user at all (or do not highlight it as they do other query terms; Google, Google SGE). Inflection AI Pi seems not to recognize the character at all.
This is largely only on base-versions of these tools (not GPT-4, not the GPT-4 version of You.com or Perplexity’s Copilot). This is using Claude 2, though.
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:37:13
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:39:05
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:39:17
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:39:31
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:40:01
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:40:38
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:41:16
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:43:12
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:47:09
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 11:47:44
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 13:29:57
Screenshot taken with GoFullPage (distortions possible) at: 2023-07-17 14:30:23
This is not a systematic search audit or test. I am sharing this as I explore my own process for understanding and sharing-out about these tools.
Reminders:
It is very difficult to compare SERPs outside of contexts-of-use.
We cannot pretend that one-SERP-fits-all.
It is difficult to compare initial responses in a query that is inclined towards reformulation and interaction (i.e. sometimes the initial limitations of results are more problematic than others—like:[ Had a seizure Now what? ] as compared to the initial steps of a more methodical and less time-sensitive search like the topic of this post).
For more on advice and search, see Grimmelmann (2014, p. 950):
↩︎
Grimmelmann, J. (2014). Speech engines. Minnesota Law Review, 98, 868. https://scholarship.law.cornell.edu/facpub/1488/ [grimmelmann2014speech]
Lurie, E., & Mulligan, D. K. (2021). Searching for representation: A sociotechnical audit of googling for members of U.S. Congress. https://arxiv.org/abs/2109.07012 [lurie2021searching_facctrec]