"We really need to talk more about monitoring search quality for public interest topics."

Guarino has been regularly sharing examples of "Google Search results for public benefits" on Twitter and discussing "public interest search quality" and "#PublicInterestSEO", see: Twitter[from:allafarce public interest search]. Here are a list of such queries, linked to his tweets about them:

• [reschedule calfresh interview los angeles county]
• [ca ebt change pin]
• [food stamp eligibility]
• [my calfresh was discontinued - what do I do?]
• [illinois snap asset limit] & [snap asset limit illinois]
• [snap asset limit]
• [calfresh]
• [food stamps]
• [food stamps new jersey]

He has shared about asking questions of various LLM models, including in ChainForge [@arawjo2023chainforge]: [What is the SNAP asset limit in Illinois?] and [north carolina dss fair hearing request phone number]

See also work looking at new evaluations needed with search engines shifting to look at "societal relevance" [@sundin2021relevance]

Here are examples just a few pieces doing some public interest search quality evaluation and search audits (though not necessarily providing ongoing monitoring and sometimes evaluating things much more contentious and broad):

• Google and abortion: @mejova2022googling & @guendelman2022exploring
• election information: @lurie2021searching_facctrec & @mustafaraj2020case & @zade2022auditing
• Aleksandra Urman and colleagues work on representation of migrants [@urman2022auditing], conspiracy [@urman2022earth], and sexualization [@urman2022foreign]
• And of course @noble2018algorithms.