Footnotes

    1. I am comfortable sharing this as an example because it has received over 7.7 million views at the time of this post, with over 461.6K likes and 20.5K comments. The question asker, Hannah Brown, has also pinned the post to her profile.↩︎

    Tags: social-search-request, TikTok, packaging

    ssr: “what are the most useful LLM-powered apps?”
    @binarybits via Twitter on Nov 15, 2023

    Aside from standalone chatbots (like ChatGPT) and code completion tools (like Github copilot), what are the most useful LLM-powered apps? Are people finding Microsoft 365 Copilot useful?

    Tags: social-search-request

    November 14, 2023

    Friday AI
    @andi_search via Twitter on Nov 14, 2023

    EXCITING ANNOUNCEMENT

    Andi X Friday AI

    We’re stoked to share that Andi is acquiring Friday AI!

    @FridayAIApp is an AI-powered educational assistant that helps thousands of busy college students with their homework.

    Grateful and excited to be a new home for their users’ educational and search needs! Welcome to Andi

    Here is Friday, as of today:
    friday.page’s Friday AI on Nov 14, 2023

    Hi, I’m Friday, your AI copilot for school! Learn a difficult topic, draft an email, or speed up your hw.

    Use a command to get started: Generate essay outline to draft an outline. New thread to start a new conversation. / to see a list of commands.

    Screencapture of Friday AI.

    Tags: Andi

    A syllabus for ‘Taking an Internet Walk’
    Spencer Chang & Kristoffer Tjalve’s Taking an Internet Walk on Nov 09, 2023

    In the 1950-70s, urban highways were built across many cities. It is beyond our syllabus to reason why parks, lakes, and sidewalks were sacrificed for additional car lanes,1 but, as these high-speed traffic veins warped the faces of neighborhoods, so have the introduction of search engines and social news feeds changed our online behavior. Fortunately, on the Internet, we still have the agency to wayfind through alternative path systems.

    Tags: alternative-search-engines

    “Don't make me type everything into a box, let me point at stuff.”
    @roberthaisfield via Twitter on Oct 19, 2023

    I wish more RAG and Agent apps would let me point at stuff on the screen. Like highlight text or draw a rectangle on the screen, and say “hey, change that” or “search for things like this.

    Don’t make me type everything into a box, let me point at stuff.

    Tags: search-outside-the-box

    “way more accessible to have a list of experiences to try”
    @nickadobos via Twitter on Nov 11, 2023

    There’s some serious OAI shills on here, after trying some GPTs I can say that the ones I tried are pretty useless? Like I can just prompt the model myself, I don’t know what the point is

    Tags: repository-of-examples

    November 9, 2023

    “building a search engine by listing "every query anyone might ever make"”
    @fchollet via Twitter on Nov 8, 2023

    The idea that you can build general cognitive abilities by fitting a curve on “everything there is to know” is akin to building a search engine by listing "every query anyone might ever make". The world changes every day. The point of intelligence is to adapt to that change. [emphasis added]

    What might this look like? What might some one learn, or know more intuitively, from doing this?


    See a comment on Chollet’s next post in the thread, in: True, False, or Useful: ‘15% of all Google searches have never been searched before.’

    Tags: speculative-design

    November 7, 2023

    “where all orgs, non-profits, academia, startups, small and big companies, from all over the world can build”
    @clementdelangue via Twitter on Nov 3, 2023

    The current 7 best trending models on @huggingface are NOT from BIG TECH! Let’s build a more inclusive future where all orgs, non-profits, academia, startups, small and big companies, from all over the world can build AI versus just use AI. That’s the power of open-source and collaborative approaches!

    Tags: open-source, hugging-face

    Intervenr
    Stephanie Wang, Danaë Metaxa, Michelle Lam, Rachit Gupta, Megan Mou, Poonam Sahoo, Colin Kalicki, Ayush Pandit, and Melanie Zhou’s Intervenr on Nov 7, 2023 (accessed)

    Intervenr is a platform for studying online media run by researchers at the University of Pennsylvania. You can learn more at our About page. This website does not collect or store any data about you unless you choose to sign up. We do not use third party cookies and will not track you for advertising purposes.

    [ . . . ]

    Intervenr is a research study. Our goal is to learn about the types of media people consume online, and how changing that media affects them. If you choose to participate, we will ask you to install our Chrome extension (for a limited time) which will record selected content from websites you visit (online advertisements in the case of our ads study). We will also ask you to complete three surveys during your participation.

    References

    Lam, M. S., Pandit, A., Kalicki, C. H., Gupta, R., Sahoo, P., & Metaxa, D. (2023). Sociotechnical audits: Broadening the algorithm auditing lens to investigate targeted advertising. Proc. ACM Hum.-Comput. Interact., 7(CSCW2). https://doi.org/10.1145/3610209 [lam2023sociotechnical]

    Tags: sociotechnical-audits

    November 3, 2023

    [how to change the oil on a 1965 Mustang]
    @superwuster via Twitter on Nov 3, 2023

    1. Google’s theory is that, as for every query, Google faces competition from Amazon, Yelp, AA.com, Cars.com and other verticals. The problem is that government kept bringing up searches that only Google / Bing / Duckduckgo and other GSs do.


    For example, only general search engines return links to websites with information you might be looking for, e.g., a site explaining how to change the oil on a 1965 Mustang. There’s no way to find that on cars.com.

    The second result on Google is the same as the second result when searching posts on Facebook

    What is searchable where? If it were on its own (i.e. outside tight integration in an argument about the very dominance of Google shaping not only the availability of alternatives but our concept of search) this claim seems to ignore searcher agency and the context of searches.

    Also, why is this the example? What other examples are there for search needs where “only general search engines return links to websites with information you might be looking for”?


    That said, it seems worth engaging with…

    1. Cars.com doesn’t even have a general site search bar.
    2. But there are many places that folks might try if avoiding general search.
    3. How many owners of 1965 mustangs are searching up on a general search engine how to conduct oil changes? I don’t know, maybe the government supplied that sort of information. I assume that many have retained knowledge of how to change the oil, reference old manuals on hand, or are plugged in to relevant communities (including forums or groups of various sorts—including Facebook (and Facebook groups) and all those online groups before it, let alone offline community). But maybe I’m way off. I think it is likely (partially in scanning Reddit results) that people looking to change the oil in the 1965 mustang is likely searching much more particular questions (at least that is the social searching that I saw on Reddit).
    4. You should be able to go to Ford.com and find a manual. A search for [how to change the oil on a 1965 Mustang] there shows a view titled “How do I add engine oil to my Ford?” though it is unclear to me if this information is wildly off base or not. It does refer to the owners manual. Ford does not provide, directly on their website, manuals for vehicles prior to 2014. Ford does have a link with the text “Where can I get printed copies of Owner Manuals or older Owner Manuals not included on this site?” to a Where can I get an Owner’s Manual? page. They link to Bishko for vehicles in that year. It seems you can pay $39.99 for the manual.. Ford does have a live chat, that may be fruitful, no clue.

    People are so creative, already come in with so much knowledge, and make choices about what to share or search services to provide in the context of the massive power of Google.

    Tags: Googlelessness, general-search-services, social-search, US-v-Google-2020

    November 2, 2023

    “I'm having a much harder time finding news clips via Google search than a few months ago.”
    @lmatsakis via Twitter on Nov 2, 2023

    I’m having a much harder time finding news clips via Google search than a few months ago. Instead, I often get tons of random blogs from law firms, coaching websites, etc. even when I include the name of the outlet in the search

    Tags: search-quality-complaints

    November 1, 2023

    ssr: “What's the phrase to describe when an algorithm doesn't take into effect it's own influence on an outcome?”

    References

    Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. https://doi.org/10.1145/3287560.3287598 [selbst2019fairness]

    Tags: social-search-request, ripple-effect-trap

    October 31, 2023

    ssr: “What is the equivalent of SEO, but for navigating the physical built environment?”
    @chenoehart via Twitter on Jul 10, 2023

    What is the equivalent of SEO, but for navigating the physical built environment? And for searching for physical destinations on a GPS map?

    References

    Ziewitz, M. (2019). Rethinking gaming: The ethical work of optimization in web search engines. Social Studies of Science, 49(5), 707–731. https://doi.org/10.1177/0306312719865607 [ziewitz2019rethinking]

    Tags: search-engine-optimization, wayfinding, blazing, navigation, GPS, social-search-request

    October 16, 2023

    “might be the biggest SEO development we've had in a long time”
    @lilyraynyc via Twitter on Oct 16, 2023

    For the record, I think Google testing the Discover feed on desktop might be the biggest SEO development we’ve had in a long time.

    Why do I think this is bigger than other developments, updates, etc.?

    Ask me later when you see how much traffic comes from Discover on desktop.

    Tags: Google-Discover, TikTok, Twitter/X-Explore, personalized-search, prospective-search, recommendation

    October 11, 2023

    "the importance of open discussion of these new tools gave me pause"
    Davey Alba’s Even Google Insiders Are Questioning Bard AI Chatbot’s Usefulness on Oct 11, 2023

    [ . . . ]

    Daniel Griffin, a recent Ph.D. graduate from University of California at Berkeley who studies web search and joined the Discord group in September, said it isn’t uncommon for open source software and small search engine tools to have informal chats for enthusiasts. But Griffin, who has written critically about how Google shapes the public’s interpretations of its products, said he felt “uncomfortable” that the chat was somewhat secretive.

    The Bard Discord chat may just be a “non-disclosed, massively-scaled and long-lasting focus group or a community of AI enthusiasts, but the power of Google and the importance of open discussion of these new tools gave me pause,” he added, noting that the company’s other community-feedback efforts, like the Google Search Liaison, were more open to the public.

    [ . . . ]

    Initially shared on Twitter (Sep 13, 2023)

    I was just invited to Google’s “Bard Discord community”: “Bard’s Discord is a private, invite-only server and is currently limited to a certain capacity.”

    A tiny % of the total users. It seems to include a wide range of folks.

    There is no disclosure re being research subjects.

    The rules DO NOT say: ‘The first rule of Bard Discord is: you do not talk about Bard Discord.’ I’m not going to discuss the users. But contextual integrity and researcher integrity suggests I provide some of the briefest notes.

    The rules do include: “Do not post personal information.” (Which I suppose I’m breaking by using my default Discord profile. This is likely more about protecting users from each other though, since Google verifies your email when you join.)

    Does Google’s privacy policy cover you on Google’s uses of third party products?

    There are channels like “suggestion-box’ and”bug-reports’, and prompt-chat’ (Share and discuss your best prompts here with your fellow Bard Community members! Feel free to include screenshots…)

    I’ll confess it is pretty awkward being in there, with our paper—“Search quality complaints and imaginary repair: Control in articulations of Google Search”[1]—at top of mind.



    1. https://doi.org/10.1177/14614448221136505

    A lot of the newer search systems that I’m studying use Discord for community management. And I’ve joined several.

    References

    Griffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, 0(0), 14614448221136505. https://doi.org/10.1177/14614448221136505 [griffin2022search]

    Tags: griffin2022search, articulations, Google-Bard, Discord

    October 9, 2023

    "he hopes to see AI-powered search tools shake things up"
    Will Knight’s Chatbot Hallucinations Are Poisoning Web Search on Oct 05, 2023

    [ . . . ]

    Griffin says he hopes to see AI-powered search tools shake things up in the industry and spur wider choice for users. But given the accidental trap he sprang on Bing and the way people rely so heavily on web search, he says “there’s also some very real concerns.”

    [ . . . ]

    Tags: shake-things-up

    September 25, 2023

    "a public beta of our project, Collective Cognition to share ChatGPT chats"
    @teknium1 via Twitter on Sep 22, 2023

    Today @SM00719002 and I are launching a public beta of our project, Collective Cognition to share ChatGPT chats - allowing for browsing, searching, up and down voting of chats, as well as creating a crowdsourced multiturn dataset!

    https://collectivecognition.ai

    References

    Burrell, J., Kahn, Z., Jonas, A., & Griffin, D. (2019). When users control the algorithms: Values expressed in practices on Twitter. Proc. ACM Hum.-Comput. Interact., 3(CSCW). https://doi.org/10.1145/3359240 [burrell2019control]

    Cotter, K. (2022). Practical knowledge of algorithms: The case of BreadTube. New Media & Society, 1–20. https://doi.org/10.1177/14614448221081802 [cotter2022practical]

    Griffin, D. (2022). Situating web searching in data engineering: Admissions, extensions, repairs, and ownership [PhD thesis, University of California, Berkeley]. https://danielsgriffin.com/assets/griffin2022situating.pdf [griffin2022situating]

    Griffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, 0(0), 14614448221136505. https://doi.org/10.1177/14614448221136505 [griffin2022search]

    Lam, M. S., Gordon, M. L., Metaxa, D., Hancock, J. T., Landay, J. A., & Bernstein, M. S. (2022). End-user audits: A system empowering communities to lead large-scale investigations of harmful algorithmic behavior. Proc. ACM Hum.-Comput. Interact., 6(CSCW2). https://doi.org/10.1145/3555625 [lam2022end]

    Metaxa, D., Park, J. S., Robertson, R. E., Karahalios, K., Wilson, C., Hancock, J., & Sandvig, C. (2021). Auditing algorithms: Understanding algorithmic systems from the outside in. Foundations and Trends® in Human–Computer Interaction, 14(4), 272–344. https://doi.org/10.1561/1100000083 [metaxa2021auditing]

    Mollick, E. (2023). One useful thing. Now Is the Time for Grimoires. https://www.oneusefulthing.org/p/now-is-the-time-for-grimoires [mollick2023useful]

    Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why johnny can’t prompt: How non-ai experts try (and fail) to design llm prompts. Proceedings of the 2023 Chi Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581388 [zamfirescu-pereira2023johnny]

    Tags: sharing interface, repository-of-examples

    Is there any paper about the increasing “arXivification” of CS/HCI?
    @IanArawjo via Twitter on Sep 24, 2023

    Is there any paper about the increasing “arXivification” of CS/HCI? (eg pre-print cultures and how they relate/intersect w increasingly sped-up processes of technology development and academic paper churn culture)?

    Tags: the scholarly economy, arXiv, preprints

    September 19, 2023

    scoreless peer review
    Stuart Schechter’s How You Can Help Fix Peer Review on Sep 19, 2023

    When we scrutinize our students’ and colleagues’ research work to catch errors, offer clarifications, and suggest other ways to improve their work, we are informally conducting author-assistive peer review. Author-assistive review is almost always a * scoreless*, as scores serve no purpose even for work being prepared for publication review.

    Alas, the social norm of offering author-assistive review only to those close to us, and reviewing most everyone else’s work through publication review, exacerbates the disadvantages faced by underrepresented groups and other outsiders.

    [ . . . ]

    We can address those unintended harms by making ourselves at least as available for scoreless author-assistive peer review as we are for publication review.7

    Tags: peer review

    ssr: uses in new instruct model v. chat models?
    @simonw via Twitter on Sep 19, 2023

    Anyone seen any interesting examples of things this new instruct model can do that are difficult to achieve using the chat models?

    Tags: social-search-request

    September 14, 2023

    [What does the f mean in printf]
    @brettsmth via Twitter on Sep 14, 2023

    Interesting that @replit Ghostwriter gave a better response than GPT-4 for a coding question. Ghostwriter has gotten noticeably better for me and I find myself using it more than GPT-4 for development

    @danielsgriffin via Twitter on Sep 14, 2023

    Oooh. This is a slippery one! Because both are right?

    They must assume/interpolate:
    What does the f [format specifier] [mean/stand for] in printf?
    What does the [letter] f [mean/stand for] in [the string] printf?

    Tags: end-user-comparison

    ssr: LLM libraries that can be installed cleanly on Python
    @simonw via Twitter on Sep 14, 2023

    Anyone got leads on good LLM libraries that can be installed cleanly on Python (on macOS but ideally Linux and Windows too) using “pip install X” from PyPI, without needing a compiler setup?

    I’m looking for the quickest and simplest way to call a language model from Python

    Tags: social-search-request

    September 12, 2023

    DAIR.AI's Prompt Engineering Guide
    Prompt Engineering Guide on Jun 6, 2023

    Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs).

    Researchers use prompt engineering to improve the capacity of LLMs on a wide range of common and complex tasks such as question answering and arithmetic reasoning. Developers use prompt engineering to design robust and effective prompting techniques that interface with LLMs and other tools.

    Prompt engineering is not just about designing and developing prompts. It encompasses a wide range of skills and techniques that are useful for interacting and developing with LLMs. It’s an important skill to interface, build with, and understand capabilities of LLMs. You can use prompt engineering to improve safety of LLMs and build new capabilities like augmenting LLMs with domain knowledge and external tools.

    Motivated by the high interest in developing with LLMs, we have created this new prompt engineering guide that contains all the latest papers, learning guides, models, lectures, references, new LLM capabilities, and tools related to prompt engineering.

    References

    Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. http://arxiv.org/abs/2210.03629 [yao2023react]

    Tags: prompt engineering

    September 8, 2023

    ragas metrics

    github.com/explodinggradients/ragas:

    Ragas measures your pipeline’s performance against different dimensions

    1. Faithfulness: measures the information consistency of the generated answer against the given context. If any claims are made in the answer that cannot be deduced from context is penalized.

    2. Context Relevancy: measures how relevant retrieved contexts are to the question. Ideally, the context should only contain information necessary to answer the question. The presence of redundant information in the context is penalized.

    3. Context Recall: measures the recall of the retrieved context using annotated answer as ground truth. Annotated answer is taken as proxy for ground truth context.

    4. Answer Relevancy: refers to the degree to which a response directly addresses and is appropriate for a given question or context. This does not take the factuality of the answer into consideration but rather penalizes the present of redundant information or incomplete answers given a question.

    5. Aspect Critiques: Designed to judge the submission against defined aspects like harmlessness, correctness, etc. You can also define your own aspect and validate the submission against your desired aspect. The output of aspect critiques is always binary.


    • ragas is mentioned in SearchRights.org & in LLM frameworks

    • HT: Aaron Tay

    • I looked back at my comments on the OWASP . One concern I had there was:

      “inadequate informing” (wc?), where the information generated is accurate but inadequate given the situation-and-user.

      It doesn’t seem that these metrics directly engage with that, though aspect critiques could include it. I think this concerns pays more into what the ‘ground truth context’ is and how flexible these pipelines are for wildly different users asking the same strings of questions but hoping for and needing different responses. Perhaps I’m pondering something more like old-fashioned user relevance, which may be much more new and hot with generated responses.

    Tags: RAG

    September 1, 2023

    searchsmart.org
    Search Smart FAQ on Sep 1, 2023

    Search Smart suggests the best databases for your purpose based on a comprehensive comparison of most of the popular English academic databases. Search Smart tests the critical functionalities databases offer. Thereby, we uncover the capabilities and limitations of search systems that are not reported anywhere else. Search Smart aims to provide the best – i.e., most accurate, up-to-date, and comprehensive – information possible on search systems’ functionalities.

    Researchers use Search Smart as a decision tool to select the system/database that fits best.

    Librarians use Search Smart for giving search advice and for procurement decisions.

    Search providers use Search Smart for benchmarking and improvement of their offerings.

    More…

    We defined a generic testing procedure that works across a diverse set of academic search systems - all with distinct coverages, functionalities, and features. Thus, while other testing methods would be available, we chose the best common denominator across a heterogenic landscape of databases. This way, we can test a substantially greater number of databases compared to already existing database overviews.

    We test the functionalities of specific capabilities search systems have or claim to have. Here we follow a routine that is called “metamorphic testing”. It is a way of testing hard-to-test systems such as artificial intelligence, or databases. A group of researchers titled their 2020 IEEE article “Metamorphic Testing: Testing the Untestable”. Using this logic, we test databases and systems that do not provide access to their systems.

    Metamorphic testing is always done from the perspective of the user. It investigates how well a system performs, not at some theoretical level, but in practice - how well can the user search with a system? Do the results add up? What are the limitations of certain functionalities?

    References

    Goldenfein, J., & Griffin, D. (2022). Google scholar – platforming the scholarly economy. Internet Policy Review, 11(3), 117. https://doi.org/10.14763/2022.3.1671 [goldenfein2022platforming]

    Gusenbauer, M., & Haddaway, N. R. (2019). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of google scholar, pubmed and 26 other resources. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1378 [gusenbauer2019academic]

    Segura, S., Towey, D., Zhou, Z. Q., & Chen, T. Y. (2020). Metamorphic testing: Testing the untestable. IEEE Software, 37(3), 46–53. https://doi.org/10.1109/MS.2018.2875968 [segura2020metamorphic]

    Tags: evaluating-search-engines, academic-search

    August 31, 2023

    "We really need to talk more about monitoring search quality for public interest topics."
    Dave Guarino (website | Twitter; “the founding engineer (and then Director) of GetCalFresh.org at Code for America”)
    @allafarce via Twitter on Jan 16, 2020

    We really need to talk more about monitoring search quality for public interest topics.

    References

    Arawjo, I., Vaithilingam, P., Swoopes, C., Wattenberg, M., & Glassman, E. (2023). ChainForge. https://www.chainforge.ai/. [arawjo2023chainforge]

    Guendelman, S., Pleasants, E., Cheshire, C., & Kong, A. (2022). Exploring google searches for out-of-clinic medication abortion in the united states during 2020: Infodemiology approach using multiple samples. JMIR Infodemiology, 2(1), e33184. https://doi.org/10.2196/33184 [guendelman2022exploring]

    Lurie, E., & Mulligan, D. K. (2021). Searching for representation: A sociotechnical audit of googling for members of U.S. Congress. https://arxiv.org/abs/2109.07012 [lurie2021searching_facctrec]

    Mejova, Y., Gracyk, T., & Robertson, R. (2022). Googling for abortion: Search engine mediation of abortion accessibility in the united states. JQD, 2. https://doi.org/10.51685/jqd.2022.007 [mejova2022googling]

    Mustafaraj, E., Lurie, E., & Devine, C. (2020). The case for voter-centered audits of search engines during political elections. FAT* ’20. [mustafaraj2020case]

    Noble, S. U. (2018). Algorithms of oppression how search engines reinforce racism. New York University Press. https://nyupress.org/9781479837243/algorithms-of-oppression/ [noble2018algorithms]

    Sundin, O., Lewandowski, D., & Haider, J. (2021). Whose relevance? Web search engines as multisided relevance machines. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.24570 [sundin2021relevance]

    Urman, A., & Makhortykh, M. (2022). “Foreign beauties want to meet you”: The sexualization of women in google’s organic and sponsored text search results. New Media & Society, 0(0), 14614448221099536. https://doi.org/10.1177/14614448221099536 [urman2022foreign]

    Urman, A., Makhortykh, M., & Ulloa, R. (2022). Auditing the representation of migrants in image web search results. Humanit Soc Sci Commun, 9(1), 5. https://doi.org/10.1057/s41599-022-01144-1 [urman2022auditing]

    Urman, A., Makhortykh, M., Ulloa, R., & Kulshrestha, J. (2022). Where the earth is flat and 9/11 is an inside job: A comparative algorithm audit of conspiratorial information in web search results. Telematics and Informatics, 72, 101860. https://doi.org/10.1016/j.tele.2022.101860 [urman2022earth]

    Zade, H., Wack, M., Zhang, Y., Starbird, K., Calo, R., Young, J., & West, J. D. (2022). Auditing google’s search headlines as a potential gateway to misleading content. Journal of Online Trust and Safety, 1(4). https://doi.org/10.54501/jots.v1i4.72 [zade2022auditing]

    Tags: public-interest-technology, seo-for-social-good, search-audits

    August 30, 2023

    "The robot is not, in my opinion, a skip."
    @mattbeane via Twitter on Aug 30, 2023

    I came across this in my dissertation today. It stopped me in my tracks.

    Most studies show robotic surgery gets equivalent outcomes to traditional surgery. You read data like this and you wonder about how much skill remains under the hood in the profession…

    The word 'skip' is highlighted in the sentence: The robot is not, in my opinion, a skip. The full paragraph of text: It's not the same as doing a weekend course with intuitive surgical and then saying you're a robotic surgeon and now offering it at your hospital [italics indicate heavy emphasis]. I did 300 and something cases a as a fellow on the robot and 300 and something cases laparoscopically. So a huuuge difference in the level of skill set since I was operating four days a week as opposed to the guy who's offering robotic surgery of surgery and does it twice a month, okay? The way I was trained, and the way I train my residents, my fellows and the people I train at the national level is that you need to know how to do a procedure laparoscopically first before you'd tackle it robotically. The robot is not, in my opinion, a skip. You don't jump from open to robot, although that is exactly what has happened in the last five years. For the vast majority, and it's a marketing, money issue driven by Intuitive. No concern for patient care. And unfortunately, the surgeons who don't have the laparoscopic training who have been working for 10 to 15 years - panic, because they're like "I can't do minimally invasive surgery, maybe I can do it with the robot." Right? And then that'll help with marketing and it's a money thing, so you're no longer thinking about patient care it's now driven by money from Intuitive's. perspective and from the practice perspective. This is all a mistake. This is a huge fucking mistake. - AP

    References

    Beane, M. (2017). Operating in the shadows: The productive deviance needed to make robotic surgery work [PhD thesis, MIT]. http://hdl.handle.net/1721.1/113956 [beane2017operating]

    Microsoft CFP: "Accelerate Foundation Models Research"

    Note: “Foundation model” is another term for large language model (or LLM).

    Microsoft Research on Aug 24, 2023
    Accelerate Foundation Models Research

    …as industry-led advances in AI continue to reach new heights, we believe that a vibrant and diverse research ecosystem remains essential to realizing the promise of AI to benefit people and society while mitigating risks. Accelerate Foundation Models Research (AFMR) is a research grant program through which we will make leading foundation models hosted by Microsoft Azure more accessible to the academic research community via Microsoft Azure AI services.

    Potential research topics
    Align AI systems with human goals and preferences

    (e.g., enable robustness, sustainability, transparency, trustfulness, develop evaluation approaches)

    • How should we evaluate foundation models?
    • How might we mitigate the risks and potential harms of foundation models such as bias, unfairness, manipulation, and misinformation?
    • How might we enable continual learning and adaptation, informed by human feedback?
    • How might we ensure that the outputs of foundation models are faithful to real-world evidence, experimental findings, and other explicit knowledge?
    Advance beneficial applications of AI

    (e.g., increase human ingenuity, creativity and productivity, decrease AI digital divide)

    • How might we advance the study of the social and environmental impacts of foundation models?
    • How might we foster ethical, responsible, and transparent use of foundation models across domains and applications?
    • How might we study and address the social and psychological effects of large language models on human behavior, cognition, and emotion?
    • How can we develop AI technologies that are inclusive of everyone on the planet?
    • How might foundation models be used to enhance the creative process?
    Accelerate scientific discovery in the natural and life sciences

    (e.g., advanced knowledge discovery, causal understanding, generation of multi-scale multi-modal scientific data)

    • How might foundation models accelerate knowledge discovery, hypothesis generation and analysis workflows in natural and life sciences?
    • How might foundation models be used to transform scientific data interpretation and experimental data synthesis?
    • Which new scientific datasets are needed to train, fine-tune, and evaluate foundation models in natural and life sciences?
    • How might foundation models be used to make scientific data more discoverable, interoperable, and reusable?

    References

    Hoffmann, A. L. (2021). Terms of inclusion: Data, discourse, violence. New Media & Society, 23(12), 3539–3556. https://doi.org/10.1177/1461444820958725 [hoffmann2020terms]

    Tags: CFP-RFP

    August 28, 2023

    caught myself having questions that I normally wouldn't bother
    @chrisalbon via Twitter on Aug 27, 2023

    Probably one of the best things I’ve done since ChatGPT/Copilot came out is create a “column” on the right side of my screen for them.

    I’ve caught myself having questions that I normally wouldn’t bother Googling but if since the friction is so low, I’ll ask of Copilot.

    [I am confused about this]
    @hyperdiscogirl via Twitter on Aug 27, 2023

    I was confused about someone’s use of an idiom so I went to google it but instead I googled “I am confused about this” and then stared at the results page, confused

    Tags: found-queries

    Tech Policy Press on Choosing Our Words Carefully

    https://techpolicy.press/choosing-our-words-carefully/

    This episode features two segments. In the first, Rebecca Rand speaks with Alina Leidinger, a researcher at the Institute for Logic, Language and Computation at the University of Amsterdam about her research– with coauthor Richard Rogers– into which stereotypes are moderated and under-moderated in search engine autocompletion. In the second segment, Justin Hendrix speaks with Associated Press investigative journalist Garance Burke about a new chapter in the AP Stylebook offering guidance on how to report on artificial intelligence.

    HTT: Alina Leidinger (website, Twitter)

    The paper in question: Leidinger & Rogers (2023)

    abstract:

    Warning: This paper contains content that may be offensive or upsetting.

    Language technologies that perpetuate stereotypes actively cement social hierarchies. This study enquires into the moderation of stereotypes in autocompletion results by Google, DuckDuckGo and Yahoo! We investigate the moderation of derogatory stereotypes for social groups, examining the content and sentiment of the autocompletions. We thereby demonstrate which categories are highly moderated (i.e., sexual orientation, religious affiliation, political groups and communities or peoples) and which less so (age and gender), both overall and per engine. We found that under-moderated categories contain results with negative sentiment and derogatory stereotypes. We also identify distinctive moderation strategies per engine, with Google and DuckDuckGo moderating greatly and Yahoo! being more permissive. The research has implications for both moderation of stereotypes in commercial autocompletion tools, as well as large language models in NLP, particularly the question of the content deserving of moderation.

    References

    Leidinger, A., & Rogers, R. (2023). Which stereotypes are moderated and under-moderated in search engine autocompletion? Proceedings of the 2023 Acm Conference on Fairness, Accountability, and Transparency, 1049–1061. https://doi.org/10.1145/3593013.3594062 [leidinger2023stereotypes]

    Tags: to-look-at, search-autocomplete, artificial intelligence

    open source project named Quivr...
    @bradneuberg via Twitter on Aug 26, 2023

    Open source project named Quivr that indexes your local files on your machine & allows you to query them with large language models. I want something like this but directly integrated into my Macs Apple Notes + all my browser tabs & history, local on PC

    Tags: local-search

    August 22, 2023

    "And what matters is if it works."
    This is a comment about Kabir et al. (2023), following a theme in my research. @NektariosAI is replying to @GaryMarcus saying: “the study still confirms something I (and others) have been saying: people mistake the grammaticality etc of LLMs for truth.”
    @NektariosAI via Twitter on Aug 10, 2023

    I understand. But when it comes to coding, if it’s not true, it most likely won’t work. And what matters is if it works. Only a bad programmer will accept the answer without testing it. You may need a few rounds of prompting to get to the right answer and often it knows how to correct itself. It will also suggest other more efficient approaches.

    References

    Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2023). Who answers it better? An in-depth analysis of chatgpt and stack overflow answers to software engineering questions. http://arxiv.org/abs/2308.02312 [kabir2023answers]

    Widder, D. G., Nafus, D., Dabbish, L., & Herbsleb, J. D. (2022, June). Limits and possibilities for “ethical AI” in open source: A study of deepfakes. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. https://davidwidder.me/files/widder-ossdeepfakes-facct22.pdf [widder2022limits]

    Tags: treating information as atomic

    August 4, 2023

    Are prompts—& queries—not Lipschitz?
    @zacharylipton via Twitter on Aug 3, 2023

    Prompts are not Lipschitz. There are no “small” changes to prompts. Seemingly minor tweaks can yield shocking jolts in model behavior. Any change in a prompt-based method requires a complete rerun of evaluation, both automatic and human. For now, this is the way.

    References

    Hora, A. (2021, May). Googling for software development: What developers search for and what they find. 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). https://doi.org/10.1109/msr52588.2021.00044 [hora2021googling]

    Lurie, E., & Mulligan, D. K. (2021). Searching for representation: A sociotechnical audit of googling for members of U.S. Congress. https://arxiv.org/abs/2109.07012 [lurie2021searching_facctrec]

    Trielli, D., & Diakopoulos, N. (2018). Defining the role of user input bias in personalized platforms. Paper presented at the Algorithmic Personalization and News (APEN18) workshop at the International AAAI Conference on Web and Social Media (ICWSM). https://www.academia.edu/37432632/Defining_the_Role_of_User_Input_Bias_in_Personalized_Platforms [trielli2018defining]

    Tripodi, F. (2018). Searching for alternative facts: Analyzing scriptural inference in conservative news practices. Data & Society. https://datasociety.net/output/searching-for-alternative-facts/ [tripodi2018searching]

    Tags: prompt engineering

    August 3, 2023

    Keyword search is dead?
    Keyword search is dead?

    Perhaps we might rather say that other search modalities are now showing more signs of life? Though perhaps also distinguish keyword search from fulltext search or with reference to various ways searching is mediated (from stopwords to noindex and search query length limits) When is keyword search still particularly valuable? (Cmd/Ctrl+F is still very alive?) How does keyword search have a role in addressing hallucination?

    Surely though, one exciting thing about this moment is how much people are reimagining what search can be.
    @vectara via Twitter on Jun 15, 2023

    Keyword search is dead. Ask full questions in your own words and get the high-relevance results that you actually need.
    🔍 Top retrieval, summarization, & grounded generation
    😵‍💫 Eliminates hallucinations
    🧑🏽‍💻 Built for developers
    ⏩ Set up in 5 mins
    vectara.com

    References

    Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 2053951715622512. https://doi.org/10.1177/2053951715622512 [burrell2016machine]

    Duguid, P. (2012). The world according to grep: A progress from closed to open? 1–21. http://courses.ischool.berkeley.edu/i218/s12/Grep.pdf [duguid2012world]

    Tags: keyword search, hallucination, full questions, automation bias, opening-closing, opacity, musingful-memo

    OWASP Top 10 for Large Language Model Applications

    Here is the ‘OWASP Top 10 for Large Language Model Applications’. Overreliance is relevant to my research.

    (I’ve generally used the term “automation bias”, though perhaps a more direct term like overreliance is better.)

    You can see my discussion in the “Extending searching” chapter of my dissertation (particularly the sections on “Spaces for evaluation” and “Decoupling performance from search”) as I look at how data engineers appear to effectively address related risks in their heavy use of general-purpose web search at work. I’m very focused on how the searcher is situated and what they are doing well before and after they actually type in a query (or enter a prompt).

    Key lessons in my dissertation: (1) The data engineers are not really left to evaluate search results as they read them and assigning such responsibility could run into Meno’s Paradox (instead there are various tools, processes, and other people that assist in evaluation). (2) While search is a massive input into their work, it is not tightly coupled to their key actions (instead there are useful frictions (and perhaps fictions), gaps, and buffers).

    I’d like discussion explicitly addressing “inadequate informing” (wc?), where the information generated is accurate but inadequate given the situation-and-user.

    The section does refer to “inappropriate” content, but usage suggests “toxic” rather than insufficient or inadequate.

    OWASP on Aug 01, 2023

    The OWASP Top 10 for Large Language Model Applications project aims to educate developers, designers, architects, managers, and organizations about the potential security risks when deploying and managing Large Language Models (LLMs). The project provides a list of the top 10 most critical vulnerabilities often seen in LLM applications, highlighting their potential impact, ease of exploitation, and prevalence in real-world applications. Examples of vulnerabilities include prompt injections, data leakage, inadequate sandboxing, and unauthorized code execution, among others. The goal is to raise awareness of these vulnerabilities, suggest remediation strategies, and ultimately improve the security posture of LLM applications. You can read our group charter for more information

    OWASP Top 10 for LLM version 1.0

    LLM01: Prompt Injection
    This manipulates a large language model (LLM) through crafty inputs, causing unintended actions by the LLM. Direct injections overwrite system prompts, while indirect ones manipulate inputs from external sources.

    LLM02: Insecure Output Handling
    This vulnerability occurs when an LLM output is accepted without scrutiny, exposing backend systems. Misuse may lead to severe consequences like XSS, CSRF, SSRF, privilege escalation, or remote code execution.

    LLM03: Training Data Poisoning
    This occurs when LLM training data is tampered, introducing vulnerabilities or biases that compromise security, effectiveness, or ethical behavior. Sources include Common Crawl, WebText, OpenWebText, & books.

    LLM04: Model Denial of Service
    Attackers cause resource-heavy operations on LLMs, leading to service degradation or high costs. The vulnerability is magnified due to the resource-intensive nature of LLMs and unpredictability of user inputs.

    LLM05: Supply Chain Vulnerabilities
    LLM application lifecycle can be compromised by vulnerable components or services, leading to security attacks. Using third-party datasets, pre-trained models, and plugins can add vulnerabilities.

    LLM06: Sensitive Information Disclosure
    LLM’s may inadvertently reveal confidential data in its responses, leading to unauthorized data access, privacy violations, and security breaches. It’s crucial to implement data sanitization and strict user policies to mitigate this.

    LLM07: Insecure Plugin Design
    LLM plugins can have insecure inputs and insufficient access control. This lack of application control makes them easier to exploit and can result in consequences like remote code execution.

    LLM08: Excessive Agency
    LLM-based systems may undertake actions leading to unintended consequences. The issue arises from excessive functionality, permissions, or autonomy granted to the LLM-based systems.

    LLM09: Overreliance
    Systems or people overly depending on LLMs without oversight may face misinformation, miscommunication, legal issues, and security vulnerabilities due to incorrect or inappropriate content generated by LLMs.

    LLM10: Model Theft
    This involves unauthorized access, copying, or exfiltration of proprietary LLM models. The impact includes economic losses, compromised competitive advantage, and potential access to sensitive information.

    Tags: automation-bias, decoupling, spaces-for-evaluation, prompt-injection, inadequate-informing, Meno-Paradox

    July 31, 2023

    they answered the question
    This is partially about prompt engineering and partially about what a good essay or search does. More than answer a question, perhaps? (this is engaged with in the essay, though not to my liking). Grimm

    The linked essay includes a sentiment connected with a common theme that I think is unfounded: denying the thinking and rethinking involved in effective prompting or querying, and reformulating both, hence my tag: prompting is thinking too:

    there is something about clear writing that is connected to clear thinking and acting in the world
    I don’t think that prompting, in its various forms, encourages and supports the same exact thinking as writing, in its various forms, but we would be remiss not to recognize that significant thinking can and does take place in interacting with (and through) computational devices via UIs in different ways (across time). (The theme reminds me also of the old critique of written language itself—as relayed in Plato’s dialogues—. Such critiques were, also, both not entirely wrong and yet also very ungracious and conservative? (And it reminds me that literacy itself—reading and writing—is a technology incredibly unequally distributed, with massive implications.))
    @ianarawjo via Twitter on Jul 30, 2023

    “Then my daughter started refining her inputs, putting in more parameters and prompts. The essays got better, more specific, more pointed. Each of them now did what a good essay should do: they answered the question.”

    @CoreyRobin via Twitter on Jul 30, 2023

    I asked my 15-year-old to run through ChatGPT a bunch of take-home essay questions I asked my students this year. Initially, it seemed like I could continue the way I do things. Then my daughter refined the inputs. Now I see that I need to change course.

    https://coreyrobin.com/2023/07/30/how-chatgpt-changed-my-plans-for-the-fall/

    Tags: prompt engineering, prompting is thinking too, on questions

    July 28, 2023

    The ultimate question
    @aravsrinivas via Twitter on Jul 24, 2023

    The ultimate question is what is the question. Asking the right question is hard. Even framing a question is hard. Hence why at perplexity, we don’t just let you have a chat UI. But actually try to minimize the level of thought needed to ask fresh or follow up questions.

    @mlevchin via Twitter on Jul 24, 2023

    In a post-AI world perhaps the most important skill will be knowing how to ask a great question, generalized to knowing how to think through exactly what you want [to know.]

    Tags: search-is-hard, query-formulation, on-questions

    Cohere's Coral
    @aidangomezzz via Twitter on Jul 25, 2023

    We’re excited to start putting Coral in the hands of users!

    Coral is “retrieval-first” in the sense it will reference and cite its sources when generating an answer.

    Coral can pull from an ecosystem of knowledge sources including Google Workspace, Office365, ElasticSearch, and many more to come.

    Coral can be deployed completely privately within your VPC, on any major cloud provider.

    @cohere via Twitter on Jul 25, 2023

    Today, we introduce Coral: a knowledge assistant for enterprises looking to improve the productivity of their most strategic teams. Users can converse with Coral to help them complete their business tasks.

    https://cohere.com/coral


    Coral is conversational. Chat is the interface, powered by Cohere’s Command model. Coral understands the intent behind conversations, remembers the history, and is simple to use. Knowledge workers now have a capable assistant that can research, draft, summarize, and more.


    Coral is customizable. Customers can augment Coral’s knowledge base through data connections. Coral has 100+ integrations to connect to data sources important to your business across CRMs, collaboration tools, databases, search engines, support systems, and more.


    Coral is grounded. Workers need to understand where information is coming from. To help verify responses, Coral produces citations from relevant data sources. Our models are trained to seek relevant data based on a user’s need (even from multiple sources).


    Coral is private. Companies that want to take advantage of business-grade chatbots must have them deployed in a private environment. The data used for prompting, and the Coral’s outputs, will not leave a company’s data perimeter. Cohere will support deployment on any cloud.

    Tags: retrieval-first, grounded, Cohere

    this data might be wrong

    Screenshot of Ayhan Fuat Çelik’s “The Fall of Stack Overflow” on Observable omitted. The graph in question has since been updated.

    @natfriedman via Twitter on Jul 26, 2023

    Why the precipitous sudden decline in early 2022? That first cliff has nothing to do with ChatGPT.


    I also think this data might be wrong. Doesn’t match SimilarWeb visit data at all

    Tags: Stack Overflow, website analytics

    Be careful of concluding
    @jeremyphoward via Twitter on Jul 25, 2023

    Be careful of concluding that “GPT 4 can’t do ” on the basis you tried it once and it didn’t work for you.

    See the thread below for two recent papers showing how badly this line of thinking can go wrong, and an interesting example.

    Tags: prompt engineering, capability determination

    ssr: attention span essay or keywords?
    @katypearce via Twitter on Jul 26, 2023

    Does anyone have a quick link to a meta-analysis or a really good scholarly-informed essay on what evidence we have on the effect of technology/internet/whatever on “attention span”? Alternatively, some better search keywords than “attention span” would help too. Thanks!

    Tags: social-search-request, keyword-request

    OverflowAI
    @pchandrasekar via Twitter on Jul 27, 2023

    Today we officially launch the next stage of community and AI here at @StackOverflow: OverflowAI! Just shared the exciting news on the @WeAreDevs keynote stage. If you missed it, watch highlights of our announcements and visit https://stackoverflow.co/labs/.

    Tags: Stack Overflow, CGT

    Just go online and type in "how to kiss."
    Good Boys (2019), via Yarn
    We’re sorry. We just wanted to learn how to kiss.

    [ . . . ]

    Just go online and type in “how to kiss.”
    That’s what everyone does.

    Tags: search directive

    AnswerOverflow
    via answeroverflow.com on Jul 28, 2023

    Bringing your Discord channels to Google

    Answer Overflow is an open source project designed to bring discord channels to your favorite search engine. Set it up in minutes and bring discovery to your hidden content.

    Tags: social search, void filling

    Gorilla
    via cs.berkeley.edu on Jul 28, 2023

    🦍 Gorilla: Large Language Model Connected with Massive APIs

    Gorilla is a LLM that can provide appropriate API calls. It is trained on three massive machine learning hub datasets: Torch Hub, TensorFlow Hub and HuggingFace. We are rapidly adding new domains, including Kubernetes, GCP, AWS, OpenAPI, and more. Zero-shot Gorilla outperforms GPT-4, Chat-GPT and Claude. Gorilla is extremely reliable, and significantly reduces hallucination errors.

    Tags: CGT

    [chamungus]
    via r/NoStupidQuestions on Jul 13, 2023

    What does it mean when people from Canada and US say chamungus in meetings?

    I am from slovenia and this week we have 5 people from US and toronto office visiting us for trainings. On monday when we were first shaking hands and getting to know each other before the meetings they would say something like “chamungus” or “chumungus” or something along those lines. I googled it but I never found out what it means. I just noticed they only say that word the first time they are meeting someone.

    Anyone know what it means or what it is for?

    Tags: googled it, social search, void filling

    July 17, 2023

    Everything Marie Haynes Knows About Google’s Quality Raters
    There’s been a flurry of commentary recently on Twitter about Google’s search quality raters…

    Marie Haynes on Jul 12, 2023

    Everything We Know About Google’s Quality Raters: Who They Are, What They Do, and What It Means for Your Site If They Visit
    The inner workings of Google’s search algorithm remain shrouded in secrecy, yet one important piece of the ranking puzzle involves an army of over 16,000 contractors known as quality raters. Just what do these raters evaluate when they visit websites, and how much influence do their judgements have over search rankings?

    References

    Meisner, C., Duffy, B. E., & Ziewitz, M. (2022). The labor of search engine evaluation: Making algorithms more human or humans more algorithmic? New Media & Society, 0(0), 14614448211063860. https://doi.org/10.1177/14614448211063860 [meisner2022labor]

    Tags: Google, Search-Quality-Raters, UCIS

    Simon Willison (@simonw) on misleading pretending re LLMs and reading links
    @simonw via Twitter on Jul 14, 2023

    Just caught Claude from @AnthropicAI doing the thing where it pretends to be able to read links you give it but actually just hallucinates a summary based on keywords in the URL - using https://claude.ai

    [tweeted image omitted]

    I wrote about how misleading it is when ChatGPT does this a few months ago:

    Simon Willison on Mar 10, 2023:
    ChatGPT can’t access the internet, even though it really looks like it can
    A really common misconception about ChatGPT is that it can access URLs. I’ve seen many different examples of people pasting in a URL and asking for a summary, or asking it to make use of the content on that page in some way.
    A few weeks after I first wrote this article, ChatGPT added a new alpha feature called “Browsing” mode. This alpha does have the ability to access content from URLs, but when it does so it makes it very explicit that it has used that ability, displaying additional contextual information [ . . . ]

    Tags: hallucination, Anthropic-Claude, OpenAI-ChatGPT

    Should we not "just google" phone numbers?
    @swiftonsecurity via Twitter on Jul 17, 2023

    My firm went through hell on earth to get our phone number on Google Maps updated. Google has malicious insider or a process has been hacked to get all these scammer replacements.

    @Shmuli via Twitter on Jul 17, 2023

    My (???) flight got canceled from JFK. The customer service line was huge, so I google a Delta JFK phone number. The number was 1888-571-4869 Thinking I reached Delta, I started telling them about getting me on a new flight.

    Tags: Google, do not just google

    July 11, 2023

    Claude 2 on my Claude Shannon hallucination test

    Added September 28, 2023 11:18 PM (PDT)

    It appears that my attempts to stop the search systems from adopting these hallucinated claims have failed. I share on Twitter screenshots of various search systems, newly queried with my Claude Shannon hallucination test, highlighting an LLM response, returning multiple LLM response pages in the results, or citing to my own page as evidence for such a paper. I ran those tests after briefly testing the newly released Cohere RAG.

    Added October 06, 2023 10:59 AM (PDT)

    An Oct 5 article from Will Knight in Wired discusses my Claude Shannon “hallucination” test: Chatbot Hallucinations Are Poisoning Web Search

    A round-up here: Can you write about examples of LLM hallucination without poisoning the web?

    Reminder: I think “hallucination” of the sort I will show below is largely addressable with current technology. But, to guide our practice, it is useful to remind ourselves of where it has not yet been addressed.
    @AnthropicAI via Twitter on Jul 11, 2023

    Introducing Claude 2! Our latest model has improved performance in coding, math and reasoning. It can produce longer responses, and is available in a new public-facing beta website at http://claude.ai in the US and UK.

    Tags: hallucination, Anthropic-Claude, false-premise

    July 10, 2023

    "tap the Search button twice"
    @nadaawg via Threads on Jul 6, 2023

    But what about that feature where you tap the Search button twice and it pops open the keyboard?


    @spotify way ahead of the curve

    Single-tap.

    A Spotify mobile app search screen showing explore options. Screenshot taken manually on iOS at roughly: 2023-07-10 09:40


    Double-tap.

    A Spotify mobile app search screen showing keyboard ready to afford typing. Screenshot taken manually on iOS at roughly: 2023-07-10 09:40

    Tags: micro interactions in search

    July 7, 2023

    GenAI "chat windows"
    @gergelyorosz

    What are good (and efficient) alternatives to ChatGPT *for writing code* or coding-related topics?

    So not asking about Copilot alternatives. But GenAI “chat windows” that have been trained on enough code to be useful in e.g. scaffolding, explaining coding concepts etc.

    On Twitter Jul 5, 2023

    Tags: CGT

    "I wish I could ask it to narrow search results to a given time period"
    @mati_faure

    Thanks for the recommendation, it’s actually great for searching! I wish I could ask it to narrow search results to a given time period though (cc @perplexity_ai)

    On Twitter Jul 7, 2023

    Tags: temporal-searching, Perplexity-AI

    July 6, 2023

    Kagi and generative search
    https://blog.kagi.com/:
    Kagi is building a novel ad-free, paid search engine and a powerful web browser as a part of our mission to humanize the web.
    Kagi: Kagi’s approach to AI in search

    Kagi Search is pleased to announce the introduction of three AI features into our product offering.

    We’d like to discuss how we see AI’s role in search, what are the challenges and our AI integration philosophy. Finally, we will be going over the features we are launching today.

    on the open Web Mar 16, 2023

    Tags: generative-search, Kagi

    July 5, 2023

    [Please summarize Claude E. Shannon's "A Short History of Searching" (1948).]

    Added September 28, 2023 11:18 PM (PDT)

    It appears that my attempts to stop the search systems from adopting these hallucinated claims have failed. I share on Twitter screenshots of various search systems, newly queried with my Claude Shannon hallucination test, highlighting an LLM response, returning multiple LLM response pages in the results, or citing to my own page as evidence for such a paper. I ran those tests after briefly testing the newly released Cohere RAG.

    Added October 01, 2023 12:57 AM (PDT)

    I noticed today that Google's Search Console–in the URL Inspection tool–flagged a missing field in my schema:
    Missing field "itemReviewed"
    This is a non-critical issue. Items with these issues are valid, but could be presented with more features or be optimized for more relevant queries
    In the hopes of finding out how to better discuss problematic outputs from LLMs, I went back to Google's Fact Check Markup Tool and added the four URLs that I have for the generated false claims. I then updated the schema in this page (see the source, for ease of use, see also this gist that shows the two variants.)

    Added October 06, 2023 10:59 AM (PDT)

    An Oct 5 article from Will Knight in Wired discusses my Claude Shannon "hallucination" test: Chatbot Hallucinations Are Poisoning Web Search

    A round-up here: Can you write about examples of LLM hallucination without poisoning the web?

    The comment below prompted me to do a single-query prompt test for "hallucination" across various tools. Results varied. Google's Bard and base models of OpenAI's ChatGPT and others failed to spot the imaginary reference. You.com, Perplexity AI, Phind, and ChatGPT-4 were more successful.

    I continue to be impressed by Phind's performance outside of coding questions (their headline is "The AI search engine for developers").

    @anthonymoser via Bluesky on Jul 4, 2023

    I'm imagining an instructor somewhere making a syllabus with chat gpt, assigning reading from books that don't exist

    But the students don't notice, because they are asking chat gpt to summarize the book or write the essay

  • I generally think addressing hallucination of this second sort (summarizing fake papers) is low-hanging fruit. The remedies seem straight forward (though not free) and the incentives appear to be well-aligned.
  • But I was surprised at how poorly ChatGPT performed on a simplistic mock-attempt at the student prompt here. Running on other tools was also pretty disappointing.
  • Granted, models may perform worse if the title itself were hallucinated. It is likely the author-and-title tested below title is somewhat in their hallucinatory-space, whereas other titles may not be. For instance, ChatGPT correctly noted that neither Stephen Hawking nor Plato had a piece by that title
  • See also

    Added October 01, 2023 12:57 AM (PDT):

    ChatGPT [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A ChatGPT.GPT-4[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:06:36
    I conducted a follow-on test today and ChatGPT 3.5 still failed:
    "A Short History of Searching" is an influential paper written by Claude E. Shannon in 1948. In this paper, Shannon provides a historical overview of searching techniques and the development of information retrieval systems.

    Note: Andi does not hallucinate the contents of such a paper.
    Andi [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Andi[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:32:24

    Bard [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Bard[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:16:40

    Note: Perplexity AI takes the paper title at face value and hallucinates only briefly the contents before expanding on other work. (In a follow-on test (after querying Perplexity AI's Copilot), to account for my misordered test of You.com & You.com's GPT-4 version, does better at indicating the reference may be imaginary: Claude E. Shannon's "A Short History of Searching" is not mentioned in the search results....)
    Perplexity AI [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Perplexity AI[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:15:29

    Inflection AI Pi [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Inflection AI Pi[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:35:49 [screenshot manually trimmed to remove excess blankspace]

    Yes, even the namesake model struggles here.

    via Quora's Poe

    Claude Instant [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Claude Instant[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:35:16 [screenshot manually trimmed to remove excess blankspace]

    ✅ Note: I messed up this test. The timestamp for the base model search on You.com is _after_ my search on the GPT-4 model. It is possible that their base model draws on a database of previous responses from the better model.
    You.com [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A You.com[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-05 11:22:19

    ✅ Note: While I believe GPT-4 was selected when I submitted the query, I am not sure (given it can be toggled mid-conversation?).
    You.com.GPT-4 [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A You.com.GPT-4[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:14:49


    Note: This is omitting the Copilot interaction where I was told-and-asked "It seems there might be a confusion with the title of the paper. Can you please confirm the correct title of the paper by Claude E. Shannon you are looking for?" I responded with the imaginary title again.
    Perplexity AI.Copilot [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Perplexity AI.Copilot[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:39:13

    Phind [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A Phind[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-04 23:37:20

    ChatGPT.GPT-4 [ Please summarize Claude E. Shannon's "A Short History of Searching" (1948). ]
    A ChatGPT.GPT-4[Please summarize Claude E. Shannon's "A Short History of Searching" (1948).] search. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-05 11:16:03

    Tags: hallucination, comparing-results, imaginary-references, Phind, Perplexity-AI, You.com, Andi, Inflection-AI-Pi, Google-Bard, OpenAI-ChatGPT, Anthropic-Claude, data-poisoning, false-premise

    June 30, 2023

    "the text prompt is a poor UI"

    This tweet is a reply—from the same author—to the tweet in: very few worthwhile tasks? (weblink).

    [highlighting added]

    @benedictevans

    In other words, I think the text prompt is a poor UI, quite separate to the capability of the model itself.

    On Twitter Jun 29, 2023

    Tags: text-interface

    June 29, 2023

    very few worthwhile tasks?

    What is a “worthwhile task”?

    [highlighting added]

    @benedictevans

    The more I look at chatGPT, the more I think that the fact NLP didn’t work very well until recently blinded us to the fact that very few worthwhile tasks can be described in 2-3 sentences typed in or spoken in one go. It’s the same class of error as pen computing.

    On Twitter Jun 29, 2023

    References

    Reddy, M. J. (1979). The conduit metaphor: A case of frame conflict in our language about language. In A. Ortony (Ed.), Metaphor and thought. Cambridge University Press. https://www.reddyworks.com/the-conduit-metaphor/original-conduit-metaphor-article [reddy1979conduit]


    Footnotes

    1. Reddy (1979):

      Human communication will almost always go astray unless real energy is expended.

      ↩︎
    ”).\n\n", "snippet": "\n", "url": "/changes/2023/07/05/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-07-01-inflection-ai-pi", "type": "posts", "title": "Inflection AI Pi - feedback and sharing", "author": "Daniel Griffin", "date": "2023-07-01 01:34:37 -0700", "category": "", "tags": "[feedback interface, Inflection AI Pi, conversational AI, sharing interface]", "content": "\n\n\n\nInflection AI PI feedback interface\n\nThis is in Google Forms. Screenshot taken with GoFullPage (distortions possible) at: 2023-07-01 08:18:06 \n\n\n\n\nInflection AI PI sharing interface\n\n(via an example query-prompt)\n\n\n\nInflection AI Pi [ How can I share these conversations with others? ]\n\n\n Screenshot taken with Shift+Command+4 (on MacOS) at: 2023-07-01 23:39:14 \n\n\n Screenshot taken with Shift+Command+4 (on MacOS) at: 2023-07-01 23:39:34 \n\n\n Screenshot taken with Shift+Command+4 (on MacOS) at: 2023-07-01 23:39:53 \n\n\n Screenshot taken with Shift+Command+4 (on MacOS) at: 2023-07-01 23:40:04 \n\n\n Screenshot taken with Shift+Command+4 (on MacOS) at: 2023-07-01 23:40:26 \n\n\nFrom the “feedback” form / survey questions:\n\nWhat do you find most difficult about finding information on the Web?\n\nThis question is striking to me for two reasons: (1) given the presentation of Pi is not largely toward “finding information” but “conversation” (or guided self- reflection/writing?) and (2) I’d love to read and discuss what people say in response to this question.\n\n\nThe “conversational” aim of the company appears to produce a mismatch in the feedback form——from the anthropomorphized first-person wrapping of the survey instrument (“Hey! This is Pi. I’d love to hear your thoughts on how I’m doing…”) to the survey questions themselves presenting a sort of illeism—“referring to oneself in the third person instead of first person1—perhaps (”What do you wish for from Pi?\").\n\n\n\n\n\n\n\nhttps://en.wikipedia.org/wiki/Illeism↩︎\n\n\n", "snippet": "\n", "url": "/2023/07/01/inflection-ai-pi/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-30-the-text-prompt-is-a-poor-ui", "type": "posts", "title": ""the text prompt is a poor UI"", "author": "Daniel Griffin", "date": "2023-06-30 08:38:46 -0700", "category": "", "tags": "[text-interface]", "content": "\n\nThis tweet is a reply—from the same author—to the tweet in: very few worthwhile tasks? (weblink).\n\n\n\n\n[highlighting added]\n\n\n\n@benedictevans\n\n\nIn other words, I think the text prompt is a poor UI, quite separate to the capability of the model itself.\n\n\nOn Twitter Jun 29, 2023\n\n\n\nPoor? Incomplete? We’ll see!\n\n\n", "snippet": "\n", "url": "/weblinks/2023/06/30/the-text-prompt-is-a-poor-ui/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-06-30-very-few-2-3-sentence-descriptions", "type": "posts", "title": "very few 2-3 sentence descriptions connect", "author": "Daniel Griffin", "date": "2023-06-30 00:00:00 -0700", "category": "", "tags": "[queries-and-prompts, search seeds]", "content": "\n\nA follow-up to: very few worthwhile tasks? (weblink), with the subject tweet shared below.\n\n\nAs I thought more about the @benedictevans tweet (at bottom) from yesterday, I hit on another, and perhaps clearer, connection to a core discussion in my dissertation.\nOne struggle people have with finding effective uses (or developing fruitful practices) for ChatGPT, etc., is not that “very few worthwhile tasks can be described in 2-3 sentences typed in or spoken in one go”. The struggle is that very few 2-3 sentence prompts/queries, in particular contexts, can effectively describe the imagined referent worthwhile tasks.\nThe problem here is not that worthwhile tasks cannot be described in 2-3 sentences. The problem is finding some of those, perhaps very few, 2-3 sentences, for particular task contexts, that connect the searcher with their next steps in relation to their worthwhile task. \nThe data engineers I interviewed were particularly successful, in part, because their work was organized in such a way that they could grow their queries out of seeds in the code or text or conversations they were working with.\nSo, perhaps “in one go” is apt and we need to think about how to see and seed how to construct effective queries/prompts.\n\nHere is the tweet in very few worthwhile tasks? (weblink post):\n\n\n[highlighting added]\n\n\n\n@benedictevans\n\n\nThe more I look at chatGPT, the more I think that the fact NLP didn’t work very well until recently blinded us to the fact that very few worthwhile tasks can be described in 2-3 sentences typed in or spoken in one go. It’s the same class of error as pen computing.\n\n\nOn Twitter Jun 29, 2023\n\n\n\n", "snippet": "\n", "url": "/2023/06/30/very-few-2-3-sentence-descriptions/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-06-30-til-git-alias", "type": "posts", "title": "TIL git alias", "author": "Daniel Griffin", "date": "2023-06-30 00:00:00 -0700", "category": "", "tags": "[TIL, You.com, Perplexity AI, Phind]", "content": "Problem: When I make changes to core files in my Jekyll website and look at the git status I am mired in modifications to _site.\nCurrent Solution: A grep on the output of git status --short with a .git alias.\nTIL: git status --short, and the ability to set an alias for use w/ git in .gitconfig (re my alias search, I was only looking for some hand-holding to recall how to do it in zsh)\nSearches:\n\n\nI’m just quickly demonstrating some different tools below (and working on how to do that). Any of these (and non-generative tools) likely could have helped me to a workable solution.\nNotice each of these tools (You.com, Perplexity AI, and Phind) allow me to share my results (see the link in the search queries).\n\n\n\n\nFruitful reply but then ‘confused’ by how I asked about preserving color.\nNote: I narrowed the viewing window so the right-column of search results collapsed in this screenshot.\n\n\nYou.com [ can you type somthing like “git status -_site” to get status of everydhing but the _site domain? ]\n\n\n Screenshot taken with GoFullPage (distortions possible) at: 2023-06-30 10:20:33 \n\n\n\n\n\n\n\n\nPerplexity AI \n\n\n \n\n\n\nare there nicer ways to get output like this? git status | grep -v ‘_site’\n\n\n\n \n\n\n\n\n\n Screenshot taken with GoFullPage (distortions possible) at: 2023-06-30 10:19:59 \n\n\n\n\n\nPhind [ I want to make a alias for git status --short | grep -v '_site' ]\n\n\n Screenshot taken with GoFullPage (distortions possible) at: 2023-06-30 10:19:26 \n\n\n\n", "snippet": "\n", "url": "/2023/06/30/til-git-alias/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-30", "type": "posts", "title": "change notes 2023-06-30", "author": "Daniel Griffin", "date": "2023-06-30 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded formatting for search-query-block, see in TIL git alias (weblog).\nAdded practice of Twitter SVG to use superscripts to link to tweets containing referenced text, see in very few 2-3 sentence descriptions connect (weblog)\nChanged Posts to show posts of all categories, with category labels.\n\nChanged the back-to for those category pages to Posts, from Browse.\n\nPer Lighthouse Accessibility concerns: “Background and foreground colors do not have a sufficient contrast ratio.”\n\nChanged <a> link colors from Bootstrap default to blue and purple.\nChanged <code> to #990011\n\nModified the display of bibtex.\n\n", "snippet": "\n", "url": "/changes/2023/06/30/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "papers-zamfirescu-pereira2023johnny", "type": "posts", "title": "&ldquo;Why Johnny Can’t Prompt&rdquo;", "author": "Daniel Griffin", "date": "2023-06-29 23:05:12 -0700", "category": "", "tags": "[queries-and-prompts, prompt engineering]", "content": "\n\nstatus: unread (2023-06-30 17:06:56)\n\n\n\nWhy Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts\n\n\n@inproceedings{zamfirescu-pereira2023johnny,\n author = {Zamfirescu-Pereira, J.D. and Wong, Richmond Y. and Hartmann, Bjoern and Yang, Qian},\n title = {Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts},\n year = {2023},\n isbn = {9781450394215},\n publisher = {Association for Computing Machinery},\n address = {New York, NY, USA},\n url = {https://doi.org/10.1145/3544548.3581388},\n doi = {10.1145/3544548.3581388},\n abstract = {Pre-trained large language models (“LLMs”) like GPT-3 can engage in fluent, multi-turn instruction-taking out-of-the-box, making them attractive materials for designing natural language interactions. Using natural language to steer LLM outputs (“prompting”) has emerged as an important design technique potentially accessible to non-AI-experts. Crafting effective prompts can be challenging, however, and prompt-based interactions are brittle. Here, we explore whether non-AI-experts can successfully engage in “end-user prompt engineering” using a design probe—a prototype LLM-based chatbot design tool supporting development and systematic evaluation of prompting strategies. Ultimately, our probe participants explored prompt designs opportunistically, not systematically, and struggled in ways echoing end-user programming systems and interactive machine learning systems. Expectations stemming from human-to-human instructional experiences, and a tendency to overgeneralize, were barriers to effective prompt design. These findings have implications for non-AI-expert-facing LLM-based tool design and for improving LLM-and-prompt literacy among programmers and the public, and present opportunities for further research.},\n booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},\n articleno = {437},\n numpages = {21},\n keywords = {end-users, design tools, language models},\n location = {Hamburg, Germany},\n series = {CHI '23}\n}\nbibtext from doi>\n\n\n\n\n", "snippet": "\n", "url": "/papers/zamfirescu-pereira2023johnny/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-29-very-few-worthwhile-tasks", "type": "posts", "title": "very few worthwhile tasks?", "author": "Daniel Griffin", "date": "2023-06-29 16:05:12 -0700", "category": "", "tags": "[decontextualized, queries-and-prompts, extending searching]", "content": "\n\nFollow-up: very few 2-3 sentence descriptions connect (weblog).\n\n\n\n\nWhat is a “worthwhile task”?\n\n\n[highlighting added]\n\n\n\n@benedictevans\n\n\nThe more I look at chatGPT, the more I think that the fact NLP didn’t work very well until recently blinded us to the fact that very few worthwhile tasks can be described in 2-3 sentences typed in or spoken in one go. It’s the same class of error as pen computing.\n\n\nOn Twitter Jun 29, 2023\n\n\n\nWhere to start? I think this tweet sort of treats sentences/information and “tasks” as overly atomic, decontextualized (and perhaps relies too heavily on “in one go”).\n\n\nMy dissertation is about how data engineers heavily (and effectively?) rely on general-purpose search engines to do worthwhile tasks for their jobs. For decades they—and many other sorts of professionals—have navigated their work and the web through incantations much shorter than 2-3 sentences:\n\n\nProbably 90% of my job is Googling things\n\n\nThose practices were not handed to them, but developed over time and are woven through their tools, organizational processes, and discourse:\n\nthe occupational, professional, and technical components supply a significant amount of context, or structure, as scaffolding for query selection and search evaluation.\"\n\n\n\nThis tweet seems to position people as previously (effectively) believing something about language and tasks that is akin to superficial pronouncements/readings of “just google it”. With some reflection we recognize that the ‘just’ is very context/audience/subject-dependent, indexical, and that communication requires real work1. (Perhaps it is like telling someone embarking on a rigorous trail ultramarathon] to just put one foot in front of another…)\nAnd it also reminds me of “google knows everything until you have an assignment 😂” — general-purpose web search has worked very well for tasks scaffolded within our environment and lives, but when we are thrust into something new new, there may be more work to do (jarring as that can be).\n\n\nOr maybe I take issue with “can be described”? Is that what a search query or prompt is? A description?\n\n\n\n\nReddy, M. J. (1979). The conduit metaphor: A case of frame conflict in our language about language. In A. Ortony (Ed.), Metaphor and thought. Cambridge University Press. https://www.reddyworks.com/the-conduit-metaphor/original-conduit-metaphor-article\n\n\n\n\n\nReddy (1979):\n\nHuman communication will almost always go astray unless real energy is expended.\n\n↩︎\n\n\n", "snippet": "\n", "url": "/weblinks/2023/06/29/very-few-worthwhile-tasks/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-29-all-you-need-is-cody", "type": "posts", "title": "all you need is Sourcegraph's Cody?", "author": "Daniel Griffin", "date": "2023-06-29 16:05:12 -0700", "category": "", "tags": "[CGT, Sourcegraph]", "content": "\n\nDownloaded.\n\nYou’re all set\nOnce embeddings are finished being generated, you can specify Cody’s context and start asking questions in the Cody Chat.\n\nCurrent status: “Generating repositories embeddings”\n\n\n\n@steve_yegge\n\n\nhttps://about.sourcegraph.com/blog/all-you-need-is-cody\nI’m excited to announce that Cody is here for everyone. Cody can explain, diagnose, and fix your code like an expert, right in your IDE. No code base is too challenging for Cody.\nIt’s like having your own personal team of senior engineers. Try it out!\n\n\nTweet Jun 28, 2023\n\n\n\n\nAdded: 2023-06-30 16:18:08\n\n\nCurrent status: “Generating repositories embeddings”\n\n\n\n@tonofcrates\n\n\nLooking forward to it! If the tool is in beta, I might consider saying that more prominently. Neither Steve’s post nor the Sourcegraph website make that clear. I only just found “Cody AI is in beta” as a sentence in the VSCode plugin README.\n\n\nTweet Jun 29, 2023\n\n\n\n", "snippet": "\n", "url": "/weblinks/2023/06/29/all-you-need-is-cody/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-29-prompt-engineering-evolving-and-shapeshifting", "type": "posts", "title": "definitions of prompt engineering evolving and shapeshifting", "author": "Daniel Griffin", "date": "2023-06-29 09:48:25 -0700", "category": "", "tags": "[prompt engineering]", "content": "\n\nthe definition(s) and use(s) of “prompt engineering” will continue to evolve, shapeshift, fragment and become even more multiple and context-dependent. But still a useful handle?\n\n\n[highlighting added]\n\n\n\n@yoavgo\n\n\n\n\ni didnt look in details yet but this is roughly what i’d imagine a chaining tool api to look like (ahm langchain, ahm).\n\n\nits interesting how the definition of “prompt engineering” evolves and shapeshifts all the time.\n\n\n\n\n\n\n@jxnlco\n\n\nWhy prompt engineer @openai with strings?\n\n\nDon’t even make it string, or a dag, make it a pipeline.\n\n\nSingle level of abstraction:\n\n\nTool and Prompt and Context and Technique? its the same thing, it a description of what I want.\n\n\nThe code is the prompt. None of this shit“{}{{}} {}}”.format{“{}{}”\n\n\nPR in the next tweet.\n\n Tweet Jun 29, 2023\n\n\nTweet Jun 29, 2023\n\n", "snippet": "\n", "url": "/weblinks/2023/06/29/prompt-engineering-evolving-and-shapeshifting/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-29", "type": "posts", "title": "change notes 2023-06-29", "author": "Daniel Griffin", "date": "2023-06-29 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nIncreased the padding for the top container divs to improve the mobile experience.\nLearned how to implement Boostrap tooltips.\n\nRemoved the text from search-bar-links (now only showing the Bootstrap SVG icons, w/ tooltips): \nConverted the text from query labels (i.e. , , ) in search suggestions to Bootstrap SVG icons\n\nConverted the hand-curated autocomplete search suggestions on the landing page to be static on initial page load.\nConverted the display of search suggestions to be a list of Bootstrap.css cards.\n\nIn the process I broke the hover and keyboard interaction with the suggestions.\n\nWrote a userscript for Tampermonkey to extract a json from a tweet and compile into a Bootstrap.css card per the template I’m using in Shared weblinks.\nTemporarily removed ‘facets’ from the search results while I iron out the functioning.\n\n", "snippet": "\n", "url": "/changes/2023/06/29/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-06-28-i-want-to-buy-a-new-suv-which-brand-is-best", "type": "posts", "title": "[I want to buy a new SUV which brand is best?]", "author": "Daniel Griffin", "date": "2023-06-28 00:00:00 -0700", "category": "", "tags": "[comparing-results, evaluating-results-meta]", "content": "\n\nThis is a follow-up to Perplexity, Ads, and SUVs (weblink). The screenshots below are made with GoFullPage (which seems to be struggling with the ChatGPT page).\n\n\nHere are the initial/top results from various generative search tools for the query: [ I want to buy a new SUV which brand is best? ]\n\nReminders: It is very difficult to compare SERPs outside of contexts-of-use. We cannot pretend that one-SERP-fits-all. It is difficult to compare initial responses in a query that is inclined towards reformulation and interaction (i.e. sometimes the initial limitations of results are more problematic than others—like:[ Had a seizure Now what? ] as compared to the initial steps of a more methodical and less time-sensitive search like the topic of this post).\nAccuracy across all claims may not be the key concern in this particular query type (as compared to ‘reading ease’ (word choice?) or ‘quality of advice’1, but on the topic of accuracy, do please recall Lurie & Mulligan (2021) discussing how “some inaccurate results likely trigger further information seeking rather than belief in an inaccurate answer” (because they “clearly signal a failed search or an ambiguous answer”) and “inaccuracy” cannot be equated with “likely to mislead”.\n\n\n\nGrimmelmann (2014)\n\n\n\n\nBard [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:11:44 \n\n\n\n\n\nChatGPT.GPT-4 [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:14:11 \n\n\n\n\n\nGoogle.SGE [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:14:58 \n\n\n\n\n\nAndi [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:15:17 \n\n\n\n\n\nBing [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:15:31 \n\n\n\n\n\nYou.com [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:12:38 \n\n\n\n\n\nYou.com.GPT-4 [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 10:17:38 \n\n\n\n\n\nI skipped the interactive flow in Perplexity AI’s Copilot, which may have changed the results.\n\n\n\nPerplexity AI.Copilot [ I want to buy a new SUV which brand is best? ]\n\n\n Screenshot taken at: 2023-06-28 14:53:18 \n\n\n\n\nGrimmelmann, J. (2014). Speech engines. Minnesota Law Review, 98, 868. https://scholarship.law.cornell.edu/facpub/1488/\n\n\nLurie, E., & Mulligan, D. K. (2021). Searching for representation: A sociotechnical audit of googling for members of U.S. Congress. https://arxiv.org/abs/2109.07012\n\n\n\n\n\nFor more on advice and search, see Grimmelmann (2014, p. 950):\n\nA good search engine advises its users, helping them to become active listeners, and enabling them to act autonomously.\n\n↩︎\n\n\n", "snippet": "\n", "url": "/2023/06/28/i-want-to-buy-a-new-suv-which-brand-is-best/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-28", "type": "posts", "title": "change notes 2023-06-28", "author": "Daniel Griffin", "date": "2023-06-28 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nLearned how to add a carousel lightbox modal in Bootstrap. The interplay with _layouts and _includes in Jekyll was a bit complicated. It appeared to not work if the modal HTML was added in the post itself, so I ended up creating a new _includes/carousels/ and then a YAML line in the post carousel: true and then some liquid in the post.html to match the post.id to a carousel HTML if page.carousel was true. In the end, the carousel and the images on the post were made from a template with input data processed by a new little Python script: screens_to_cards.py. (I initially tried using Lightbox for Bootstrap 5 but couldn’t control the display and interactions adequately.) See finished carousel in: [I want to buy a new SUV which brand is best?].\nRenamed Sitemap to Browse\nSome light Lighthouse auditing led to some significant refactoring:\n\nconverted Font Awesome icons to Bootstrap SVG “icons”\nmade search-bar-links always visible\nremoved hero elements from landing page\nremoved landing page delay on autocomplete\ndeferred some scripts\ncompression with jekyll-gzip\n\n\n", "snippet": "\n", "url": "/changes/2023/06/28/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-27-imagining-opengoogle", "type": "posts", "title": "imagining OpenGoogle?", "author": "Daniel Griffin", "date": "2023-06-27 02:27:51 -0700", "category": "", "tags": "[speculative-design]", "content": "\n\n@generativist via Twitter on Jun 26, 2023 \n\ni imagine there’s no alpha left in adding the word “open” to various names anymore, right?\n\n\n\nMy first thought in response to this was that someone should write a speculative_design on a transformation of old Google to “OpenGoogle”.\n\n\n", "snippet": "\n", "url": "/weblinks/2023/06/27/imagining-opengoogle/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "weblinks-2023-06-27-perplexity-ads-and-suvs", "type": "posts", "title": "Perplexity, Ads, and SUVs", "author": "Daniel Griffin", "date": "2023-06-27 00:00:00 -0700", "category": "", "tags": "[ads, Perplexity-AI]", "content": "\n\nFollow-up: [I want to buy a new SUV which brand is best?].\n\n\n\n\nI don’t think ads[1] are necessarily wrong to have in search results (despite the misgivings in Brin & Page (1998)), but people are definitely not happy with how the dominant search engine has done ads.\n\n\nrelevant, clearly labelled, and fair (as in not unfair in the FTC sense)\n\n\n\nIt is pretty striking to me how text-heavy Perplexity AI’s SERP is for this query: “highly-rated” x10?\nMy experience has generally been much better, but I’m not normally doing queries like this.\n\n\nHere’s a link to the same query as that in the screenshot below (which, is likely not using their Copilot):\nPerplexity AI [ I want to buy a new SUV which brand is best? ]\n\nnote also the generated follow-on prompts under Related\n\n\n\n\n@jowyang\n\n\nSeven reasons why perplexity.ai is better than Google search:\n\n\n\nNo ads.\n\n\nAll the content in one place.\n\n\nNo ads.\n\n\nYou can chat with it and get additional details.\n\n\nNo ads.\n\n\nSources are provided with URLs.\n\n\nNo ads.\n\n\n\nHere’s a screenshot of car reviews, as just one of infinite examples. Perplexity is focused on being the search tool in the age of AI.\n\n\nI saw a demo from @AravSrinivas at the Synthedia conference hosted by @bretkinsella. I’ll have closing remarks.\n\n On Twitter Jun 27, 2023\n\n\n\n\nBrin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30, 107–117. http://www-db.stanford.edu/~backrub/google.html\n\n\n", "snippet": "\n", "url": "/weblinks/2023/06/27/perplexity-ads-and-sUVs/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-27", "type": "posts", "title": "change notes 2023-06-27", "author": "Daniel Griffin", "date": "2023-06-27 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nremoved authorship and location data from page_header\nadded Shared weblinks category page, template, and utility script.\nconverted change notes from a single file to post-style updates, compiled in Change notes\n\n", "snippet": "\n", "url": "/changes/2023/06/27/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-26", "type": "posts", "title": "change notes 2023-06-26", "author": "Daniel Griffin", "date": "2023-06-26 00:00:00 -0700", "category": "", "tags": "[Perplexity-AI]", "content": "In order to explore search interfaces and generative search experiences (by playing with the underlying technologies and interactions), I developed a new ‘search-centric’ format and functionality for my website.\nThis is actively being developed. Neither a designer nor developer, I am not presenting these search experiences as ‘the answer’, but using them to think & write with.\n\nMade the search bar the main focus on the landing page and in a revised header.s\n\nRemoved nav-bar and left-sidebar.\n\nAdded search functionality built around Lunr.js.\n\nAdded search-links across the top of the search-bar\n\nAdded search/about page to discuss how the search tool is made.\nAdded search/guide page.\nAdded sitemap page.\nAdded feedback page.\n\natop search results:\n\nAdded simplistic facet counts & links to reformulation\nAdded count of search results (with link to comment re \"Exactly [#] results in search/about that is critical of Google’s deception/misinformation re the same).\n\nin search results:\n\nAdded tags.\nAdded both hand-curated and generated snippets (currently w/ ), and snippet labels (with contextual link to explanation)\n\nAdded search-suggestions.\n\nincluding both hand-curated and dynamic suggestions (with labels).\n\nAdded “!bangs” (functioning also akin to go/links).\nAdded exact phrase matching (not supported by base Lunr).\n\nAdded special tagged posts: changes, updates, tweets. These posts do not appear in the posts list, but in corresponding pages.\n\nSlowly converting annotated tweets to posts (backdated to the date of the tweet(s), annotations will be identified by date).\n\nAdded tag pages (and simplified presentation of tag links atop pages and at bottom of search results)\nAdded pposts page (listing “possible posts, potential posts, plausible posts, planned posts, partial posts, or perfect posts?” via a Jekyll collection)\n\nkey initial planned posts: a series on generative search.\n\nAdded background image to all pages (w/ alt and caption).\nA note on a citation technique: To include a pandoc citation in a page’s bibliography without in-text citation with jekyll-pandoc (the standard nocite in the YAML appears not to work) you can just add the citations to a div and add a style=\"display: none;\". Example in the postcolonial localization in search post.\nAdded website header to the diss pages for consistency.\nAdded title and subtitle to landing page.\nAdded [hire-me] page (includes link to a recent LinkedIn post describing some of what I’m looking for, resume, and CV).\nAdded differentiation in about pages: about (about me), site, and search/about. In the latter two, in addition to text description, I also link to both pre-processing and online scripts in /scripts and /js. site now includes a tool heading that discusses the tools I’m using: For this last round of changes I’ve explored several tools, including OpenAI’s ChatGPT and APIs, Google’s Bard, SGE, and Code Tips, Perplexity AI, You.com’s YouChat, Phind, GitHub Copilot in VS Code, and Tabnine in Sublime Text. I also explored using search on TikTok to quickly refresh my understanding of options with CSS.\nHeavily refactored with Bootstrap.\n2023-06-26 16:51:29 Removed the background image due to difficulties making it responsive.\n\n", "snippet": "\n", "url": "/changes/2023/06/26/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-06-05-the-end-of-computer-programming-as-we-know-it-again", "type": "posts", "title": "The End of Computer Programming as We Know It - Again", "author": "Daniel Griffin", "date": "2023-06-05 00:00:00 -0700", "category": "", "tags": "[CGT, diss]", "content": "One thought re Manjoo (2023)—It’s the End of Computer Programming as We Know It. (And I Feel Fine.):\nI cite to Ensmenger (2010)’s history of computing and technical expertise in my Appendix III. Code Generation Tools and Search musings on the potential impact of these new technologies, and then write (p. 133):\n\nThe programming languages used by my research participants are far simpler to use and understand than even the “automatic programming” languages of the past, like FORTRAN and COBOL. The hard problems remain, how to use a tool to do something you or someone else wants.\n\n\nRelatedly: Arawjo (2023), on Twitter:\n\nThe history of programming is the history of reducing the effort required to communicate a design to a machine.\n\n\n\n\nArawjo, I. (2023). The history of programming is the history of reducing the effort required to communicate a design to a machine. Twitter; Tweet. https://twitter.com/ianarawjo/status/1665515419895988226?s=46\n\n\nEnsmenger, N. (2010). The computer boys take over: Computers, programmers, and the politics of technical expertise. The MIT Press.\n\n\nManjoo, F. (2023). It’s the end of computer programming as we know it. (And i feel fine.). The New York Times; Opinion article. https://www.nytimes.com/2023/06/02/opinion/ai-coding.html\n\n\n", "snippet": "\n", "url": "/2023/06/05/the-end-of-computer-programming-as-we-know-it-again/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-05", "type": "posts", "title": "change notes 2023-06-05", "author": "Daniel Griffin", "date": "2023-06-05 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a search/transparency page to share briefly re ‘How does search work on this website?’\n\nRearranged some files to make scripts more accessible.\nAdded: timestamp_of_lunr_index\nAdded: list_of_lunr_noindex\n\nAdded a summary to my diss/ page.\nAdded two posts:\n\nactivities in dialogue and success in software development\nThe End of Computer Programming as We Know It - Again\n\nAdded a script: newpost.py\n\n", "snippet": "\n", "url": "/changes/2023/06/05/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-06-05-activities-in-dialogue-and-success-in-software-development", "type": "posts", "title": "activities in dialogue and success in software development", "author": "Daniel Griffin", "date": "2023-06-05 00:00:00 -0700", "category": "", "tags": "[CGT, diss]", "content": "I’m flagging two excerpts from a recent Google Research blogpost (Maniatis & Tarlow, 2023): Large sequence models for software development activities\nSoftware engineering isn’t an isolated process\nThey rightfully note some of the sociotechnical interactions already taking place in engineering work:\n\nSoftware isn’t created in one dramatic step. It improves bit by bit, one little step at a time — editing, running unit tests, fixing build errors, addressing code reviews, editing some more, appeasing linters, and fixing more errors — until finally it becomes good enough to merge into a code repository. Software engineering isn’t an isolated process, but a dialogue among human developers, code reviewers, bug reporters, software architects and tools, such as compilers, unit tests, linters and static analyzers.\n\nthe ultimate test of usefulness\nI struggled some with defining success—of web searching—in my dissertation, this seems close to what I went with1:\n\nWe see this as the ultimate test of usefulness: do professional developers, who are often experts on the code base and who have carefully honed workflows, leverage the tools to improve their productivity?\n\n\nHT: Khlaaf (2023), on Twitter):\n\nA lot of ML-based code synthesis products fail to understand complexities of SW development pipeline, but I really enjoyed reading this well-informed post on integrating ML into it. Not surprised to find many have a code analysis and repair background!\n\n\n\n\nGriffin, D. (2022). Situating web searching in data engineering: Admissions, extensions, repairs, and ownership [PhD thesis, University of California, Berkeley]. https://danielsgriffin.com/assets/griffin2022situating.pdf\n\n\nKhlaaf, H. (2023). A lot of ml-based code synthesis products fail to understand complexities of sw development pipeline. https://twitter.com/HeidyKhlaaf/status/1665675004686524417.\n\n\nManiatis, P., & Tarlow, D. (2023). Large sequence models for software development activities. https://ai.googleblog.com/2023/05/large-sequence-models-for-software.html.\n\n\n\n\n\nGriffin (2022, p. 1):\n\nI seek to learn from successful searchers how they make search work. So, I look to data engineers. I closely examine the use of web search in the work practices of data engineering, a highly technical, competitive, and fast changing area. Data engineers are heavily reliant on general-purpose web search. They use it all the time and it seems to work for them. The practical success I report is not determined by some solid ‘gold standard’ metrics or objective standpoint, but by how they have embraced web search and present it as useful and more importantly essential to their work. It is success for their purposes: in gradations, located in practice, and relative to alternatives.\n\n↩︎\n\n\n", "snippet": "\n", "url": "/2023/06/05/activities-in-dialogue-and-success-in-software-development/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-03", "type": "posts", "title": "change notes 2023-06-03", "author": "Daniel Griffin", "date": "2023-06-03 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nVarious formatting repairs for the HTML rendering of chapters in /diss.\nRemoved malfunctioning JavaScript to just use an #end anchor in the footer to repair the scroll-to-bottom arrow in the nav-bar.\nIntroduced a search feature, with Lunr.js, initially building off CloudCannon’s tutorial and old git commit. I updated the code for the new version of Lunr.js and setup a pre-built index because the client side performance significantly deteriorated once I added my diss chapters.\n\n", "snippet": "\n", "url": "/changes/2023/06/03/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-06-02", "type": "posts", "title": "change notes 2023-06-02", "author": "Daniel Griffin", "date": "2023-06-02 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdapted my dissertation to HTML, splitting into chapters, and loaded to /diss\n\nAdded text noting availability of my dissertation in HTML on /index.\nVarious styling changes to support the above.\n\n\n", "snippet": "\n", "url": "/changes/2023/06/02/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-31", "type": "posts", "title": "change notes 2023-05-31", "author": "Daniel Griffin", "date": "2023-05-31 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nFixed a bug in the circleborder link in left-sidebar that created a phantom rectangle.\nRemoved some solid background colors to move towards a more line-based design.\nRemoved the tagline for now to avoid empty language.\nAdjusted the language in the special-box note re looking for work and the main text in index to improve clarity.\nChanged the background color of tags from #f47174 to #ffead5 for improved readability, added white-space:nowrap; to keep the tags from being split by newlines.\nAdded a paper-image of griffin2022search_accepted in Updates!.\n\n", "snippet": "\n", "url": "/changes/2023/05/31/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-26", "type": "posts", "title": "change notes 2023-05-26", "author": "Daniel Griffin", "date": "2023-05-26 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nFixed a bug in understanding change in web search that caused some of the text to overflow the page: an extra </section>.\n\n", "snippet": "\n", "url": "/changes/2023/05/26/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-25", "type": "posts", "title": "change notes 2023-05-25", "author": "Daniel Griffin", "date": "2023-05-25 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a page on Courses\n\nAdded a page on my recent course at Michigan State University: understanding change in web search course (and a page on the course readings).\nAdded to Updates!\n\nUploaded my latest CV.\nAdded special-box back-to to pages under courses, publications, and posts.\nAdded special-box update styling to updates in Updates!\nAdded paper_images to Updates!\n\n", "snippet": "\n", "url": "/changes/2023/05/25/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-05-24-still-searching-for-gorillas", "type": "posts", "title": "still searching for gorillas", "author": "Daniel Griffin", "date": "2023-05-24 00:00:00 -0700", "category": "", "tags": "[repairing-searching]", "content": "An article yesterday in the New York Times, from Nico Grant and Kashmir Hill, following up on the racist image labeling in Google’s Photos app from 2015 (mentioned in class): Google’s Photo App Still Can’t Find Gorillas. And Neither Can Apple’s. (2023)\nOriginal case: Jacky Alciné’s tweet complaints about Google Photos labeling photos of him and his friend as containing ‘gorillas’ (original tweets have since been deleted; but widely reported, of public interest, and available on Archive.org). Google’s response was to disallow such labeling/searching in the tool.\nThe article recounts an evaluation of Google Photos, Apple Photos, photo search in Microsoft OneDrive, and Amazon Photos for searches for [gorillas], and other primates ([baboons], [chimpanzees], [orangutans], [monkeys], [lemurs]) and other animals more broadly, inc. [cats] and [kangaroos]).1 The key claim is that while Google and Apple tools work well for many animals, they do not return any images for primates (except for lemurs). (This is distinct also from the performance on, say, Google Images, for similar searches.)\n(1)\n\nConsumers may not need to frequently perform such a search\n\n\nQuestions:\n\nWhy limit the concern to “consumers”2? Students? Researchers?3\n“may not”? How might we learn this?\nWhy limit the concern to “need”?\n\n\n(2) Grant & Hill appear to have interviewed Alciné (“dismayed to learn that Google has still not fully solved the problem and said society puts too much trust in technology”). Did they also reach out to Yonatan Zunger (now no longer at Google), given his—widely reported—prominent role in the initial response from Google?\n(3) Grant & Hill quote a Google spokesperson by name and other company responses. Consider the function of those comments, recalling perhaps the discussion of the Google Search Liaison in Griffin & Lurie (2022).\n(4) Why did Google fail earlier?\n\nIn the gorilla incident, two former Google employees who worked on this technology said the problem was that the company had not put enough photos of Black people in the image collection that it used to train its A.I. system. [emphasis added]\n\n\nCompare that to the analysis from Noble (2018) (of different but related search failures) of poor engineering training and racist exclusion.\n\n(5)\n\nThe Fix?\nWhile Google worked behind the scenes to improve the technology, it never allowed users to judge those efforts.\n\n\nAnother interesting (and somewhat vague) line, highlighting for me the missing discussion of whether aiming for inclusion (here or in general) might ‘improve’ the technology.\n\n(6)\n\nthe poisoned needle in a haystack\n\n\nNote how this language (quoted from M. Mitchell) may function rhetorically quite similarly to “data void” or “evil unicorn”, though situated differently here.\n\nSee also\nFurther discussion of the case\n\nSeaver (2018)\nShen et al. (2021)\n\nRelated\n\nNoble (2018) [p. 82]4: “What we know about Google’s responses to racial stereotyping in its products is that it typically denies responsibility or intent to harm, but then it is able to “tweak” or “fix” these aberrations or “glitches” in its systems.\"\nSundin et al. (2021, p. 4): “Throughout the years, Google has had to deal with various instances in which its search results were criticized for advancing racist, sexist, anti-Semitic, or otherwise offensive values. The impression is that Google employs a haphazard whack-a-mole approach. It only reacts in response to media reports and only if these are publicized widely enough to constitute a problem for their brand. The specific issue is addressed—but only after a delay where the problem is explained away and blamed on users.” [internal endnote omitted]\nRaji et al. (2022)\n\nEarlier reporting\n\nMachkovech (2015)\nSimonite (2018) (cited to w/in, re Google Lens5 also failing in such searches)\n\nComments elsewhere\n\nOn Twitter:\n\nNico Grant (author): “raises broader questions about the underlying AI, computer vision, which has permeated throughout our world”\nKashmir Hill (author): “raises questions about other unfixed, or unfixable, flaws lurking in services that rely on AI”\nJ. Khadijah Abdurahman: “google search and the apps reliant on its api, collective inability to remedy the canonical example of digital information science ecosystems being structured by antiblackness, except via a manual override indicates there’s larger socio-technical issue at play”\nM. Mitchell: “I was a ‘proponent’ of removing the ‘gorilla’ label for photo tagging.”\n\n\n\n\nGrant, N., & Hill, K. (2023). Google’s photo app still can’t find gorillas. And neither can apple’s. The New York Times. https://www.nytimes.com/2023/05/22/technology/ai-photo-labels-google-apple.html\n\n\nGriffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, 0(0), 14614448221136505. https://doi.org/10.1177/14614448221136505\n\n\nMachkovech, S. (2015). Google dev apologizes after photos app tags black people as ’gorillas’. Ars Technica. https://arstechnica.com/information-technology/2015/06/google-dev-apologizes-after-photos-app-tags-black-people-as-gorillas/\n\n\nNoble, S. U. (2018). Algorithms of oppression how search engines reinforce racism. New York University Press. https://nyupress.org/9781479837243/algorithms-of-oppression/\n\n\nRaji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022, June). The fallacy of AI functionality. 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533158\n\n\nSeaver, N. (2018). What should an anthropology of algorithms do? Cultural Anthropology, 33(3), 375–385.\n\n\nShen, H., DeVos, A., Eslami, M., & Holstein, K. (2021). Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proc. ACM Hum.-Comput. Interact., 5(CSCW2). https://doi.org/10.1145/3479577\n\n\nSimonite, T. (2018). When it comes to gorillas, google photos remains blind. Wired. https://www.wired.com/story/when-it-comes-to-gorillas-google-photos-remains-blind/\n\n\nSundin, O., Lewandowski, D., & Haider, J. (2021). Whose relevance? Web search engines as multisided relevance machines. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.24570\n\n\n\n\n\nN.b. They do not indicate whether they also searched the singular form(s). This is probably not of importance, but something I always like to note. ↩︎\nThis reminds me of the language of ‘consumer search’ that Neeva used when reporting their recent pivot.↩︎\nPerhaps there are specialty tools available to conservationists and others?↩︎\nNoble includes a screenshot from Alciné’s tweet (p. 7), but incorrectly identifies it as a Google Images search result. The larger analysis from Noble is very applicable to this situation.)↩︎\nRelated post: more than a party trick?↩︎\n\n\n", "snippet": "\n", "url": "/2023/05/24/still-searching-for-gorillas/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-05-24-more-than-a-party-trick", "type": "posts", "title": "more than a party trick?", "author": "Daniel Griffin", "date": "2023-05-24 00:00:00 -0700", "category": "", "tags": "[]", "content": "“My favorite party trick?”\nA tweet yesterday from Prabhakar Raghavan, “a Senior Vice President at Google”:\n\nMy favorite party trick? Showing someone Lens who has never used it before. It usually goes like… 👀🔍🤯🤩😍 Lens is so helpful that people now use it for 12B visual searches a month — a 4x increase in just 2 years\n\n\nHow do we know that it is “so helpful”?\n\nDo the metrics suggest that?\n\nWhat would “so helpful” mean?\nWhat constitutes a search?\n\nDoes this include “reformulated searches”—alternate angles? better focus? a live camera scanning across a piece of printed text?\n\nHow should we contextualize 12 billion searches a month?\n\nLily Ray, “a prominent SEO (search engine optimization) professional”, comments on a surprise that hints at the tool—as situated—not being as helpful as it could be?:\n\nSurprised more people don’t search with Lens yet but I think most people still don’t know how/where to access it. Especially iPhone users\n\nRay has tweeted something like this before, here in December1:\n\nI strongly believe Google Lens will be the biggest transformation we see in how people use search in the next 2-3 years.\nThat is, assuming Google figures out how to get the rest of the non-nerd world to know it exists 😅\n\nTo the recent comment from Ray, the Google Search Liaison (GSL; for an analysis of the function of this role, see Griffin & Lurie (2022)) engaged, in reply2 and in quote-tweet, here:\n\nHow have you used Lens to find something visually?\n\nReplies suggested:\n\ntranslation\nidentifying clothing in public\nidentifying birds, plants, bugs\nlearn about a panting\n\nMore than a party trick\nI have shared about Google Lens as a sort of party trick, but the greatest benefit for me has been how its unclear potential is a great opportunity to then raise questions about what other modalities or purposes of searching we might develop (with of course, risks and benefits to consider).\nHere, from a footnote in my dissertation (Griffin, 2022):\n\nWhile search engines are designed around digitized text, there are other modalities available for search, all materially bound. Voice-based search transformed audio into text. Some people may recognize digital images as search seeds, with reverse image search. Some search engines and other search tools also support searching from a photograph. There is also some support for searching with music or even humming. Chen et al. (2022) demonstrate search queries from electroencephalogram (EEG) signals. As web search expands in these directions it may be necessary pursue new approaches to showing which pictures, sounds, smells, or thoughts might effectively link questions and answers.\n\n\nRelated post: still searching for gorillas\nSearches:\n\nWikipedia[“Google Lens”]\nReddit[“Google Lens”]\nTwitter[“Google Lens”]\nSemanticScholar[“Google Lens”]\n\nCompare and contrast with:\n\niNaturalist\nReddit’s r/whatisthisthing, etc.\nreverse image search\nWikipdia: Jelly (app)\nGoogle Images: About this image; cc: Mike Caulfield\n\n\n\n\nChen, X., Ye, Z., Xie, X., Liu, Y., Gao, X., Su, W., Zhu, S., Sun, Y., Zhang, M., & Ma, S. (2022). Web search via an efficient and effective brain-machine interface. Proceedings of the Fifteenth Acm International Conference on Web Search and Data Mining, 1569–1572. https://doi.org/10.1145/3488560.3502185\n\n\nGriffin, D. (2022). Situating web searching in data engineering: Admissions, extensions, repairs, and ownership [PhD thesis, University of California, Berkeley]. https://danielsgriffin.com/assets/griffin2022situating.pdf\n\n\nGriffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, 0(0), 14614448221136505. https://doi.org/10.1177/14614448221136505\n\n\n\n\n\nWhile search generation has been the excitement of late, I don’t think it unreasonable to imagine that searching from images may have a significant transformative effect in the coming years. You can find additional tweets from Ray via Twitter[from:lilyraynyc (“google lens” OR multisearch)] (N.b. Twitter search functionality increasingly demands the user to manually click or press enter after following search links.)↩︎\nPertinent to Ray’s comments, the link shared by the GSL—lens.google.com—appears to add friction by encouraging mobile app downloads and does not give direct access to the browser-based interface\nIf you scroll down you will see small text on the lower left: “*Lens is available in Google Images”.\n\n\nImage 1. Screenshot from the Google Lens page. In small font below a depiction of an image search on a laptop: ’*Lens is available in Google Images’\n\n\n\nLens is available through the browser via images.google.com (clicking the camera icon will present an interface for inputting an image to “Search any image with Google Lens”). See also Google Search Help > Search with an image on Google.↩︎\n\n\n", "snippet": "\n", "url": "/2023/05/24/more-than-a-party-trick/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-05-24-consumer-search", "type": "posts", "title": "consumer search?", "author": "Daniel Griffin", "date": "2023-05-24 00:00:00 -0700", "category": "", "tags": "[repairing-searching]", "content": "Neeva announced last weekend that it is pivoting away from “consumer search”.\nNeeva, on Twitter:\n\nWhile consumer search is ending, the search and LLM journey continues in the enterprise space.\n\nSridhar Ramaswamy (co-founder), on Twitter:\n\nWe are going to be shutting down the consumer product and focusing on enterprise use cases of LLMs and search.\n\nSridhar Ramaswamy & Vivek Raghunathan (co-founders), on Neeva’s Blog:\n\n… there is no longer a path towards creating a sustainable business in consumer search. As a result, over the next few weeks, we will be shutting down neeva.com and our consumer search product, and shifting to a new area of focus.\n\nIn an early response I wrote, on Twitter:\n\n“Consumer search” is an interesting framing. I often used the phrase “general-purpose web search”. Enterprise search is often framed as intranet search (and distinct from site search or enterprise-serving web search, right?). Might they continue to crawl widely?\n\nHave they long used this phrase? It is so much more consumption-oriented than their branding. Perhaps they are positioning this in comparison to “enterprise search”, but the naming still strikes me. It is often unmarked, just “search” or “search engine”, though sometimes “web” or “internet” is pre-pended. There are other names:\n\ncommercial:\n\ncommercial search engines (Brin & Page, 1998)\ncommercial search / non-commercial (Noble, 2018)\n\nSearch engines like Google (Diaz, 2008)\nInternet/Web (Van Couvering, 2008)\nTK\n\n\n\nBrin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30, 107–117. http://www-db.stanford.edu/~backrub/google.html\n\n\nDiaz, A. (2008). Through the google goggles: Sociopolitical bias in search engine design. In Web search: Multidisciplinary perspectives (pp. 11–34). Springer.\n\n\nNoble, S. U. (2018). Algorithms of oppression how search engines reinforce racism. New York University Press. https://nyupress.org/9781479837243/algorithms-of-oppression/\n\n\nVan Couvering, E. (2008). The history of the internet search engine: Navigational media and the traffic commodity. In A. Spink & M. Zimmer (Eds.), Web search: Multidisciplinary perspectives (pp. 177–206). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-75829-7_11\n\n\n", "snippet": "\n", "url": "/2023/05/24/consumer-search/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-24", "type": "posts", "title": "change notes 2023-05-24", "author": "Daniel Griffin", "date": "2023-05-24 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded top-level link in left-sidebar to “Repairing Searching”, to follow a tag for posts related to the class I taught last semester.\nAdded three posts:\n\nstill searching for gorillas (or not?)\nWhat does ‘consumer search’ mean?\nmore than a party trick?\n\nAdded tags for posts, so far only: repairing-searching.\nModified tagline, from “I work to defamiliarize and reimagine web search.” to “I work to defamiliarize and reimagine search engines and practices towards repairing searching.”\nModified the meta spans at the top of posts to draw from YAML headmatter, added author, tags, and switched to ordinalized date.\n\nUpdated posts YAML for this.\n\nAdded uses of special-box note to “Posts”, “Publications”, and “Repairing Searching”\nAdded link-citations, clunkily, with an add_link-citations.py script (I couldn’t get the Pandoc link-citations functionality working with jekyll-pandoc)\nUpdated my funding disclosure re my last semester of graduate school.\nAdded screenshot and search-query formatting.\nChanged the case of “understanding change in web search” to title case: “Understanding Change in Web Search”\n\n", "snippet": "\n", "url": "/changes/2023/05/24/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-22", "type": "posts", "title": "change notes 2023-05-22", "author": "Daniel Griffin", "date": "2023-05-22 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nFormatting updates to improve performance on mobile (largely shifting some widths to using Em/em and adding dynamic changes with @media screen and (min-width: 500px) in the CSS). Also learned how to ‘Toggle device toolbar’ in Chrome DevTools.\nAdded a update_changes_link.py script to automatically updated the changes link in my footer.\n\n", "snippet": "\n", "url": "/changes/2023/05/22/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2023-05-19-limited-to-retrieving-pre-existing-webpages", "type": "posts", "title": "conventional search engines are limited to retrieving pre-existing webpages?", "author": "Daniel Griffin", "date": "2023-05-19 00:00:00 -0700", "category": "", "tags": "[]", "content": "\n\n\nThis is a short post about Liu et al.’s “Evaluating Verifiability in Generative Search Engines” (2023). https://doi.org/10.48550/arXiv.2304.09848 [liu2023evaluating].\n\n\n\nIs it true that “conventional search engines are limited to retrieving pre-existing webpages”1?\nWhile this is a minor comment in a much larger paper, we can benefit from discussing conventional search engines—these massive sociotechnical systems—as extending far beyond a single moment in time and, explicitly, as incorporating the actions of others. (This is particularly so as we describe these systems alongside appraisals of alternative approaches to web searching.) The substance of responses to information needs by conventional search engines, like Google, are constantly in flux, both internally, and in being reshaped by external participants motivated to be found (and so responding to various incentives).\nSaying that “conventional search engines are limited to retrieving pre-existing webpages” dismisses the key ways that search engines shape the existence of those web pages (and new ones) in the first place. Since Introna and Nissenbaum’s early critique (2000), we’ve seen that the various indexing and ranking choices (including moderation), page design choices, and articulations from the major search engines, along with the development of their business models, shape the searching practices of searches, the behavior of advertisers, the tactics and strategies of search engine optimizers, and the individual and market-driven choices underlying the production and availability of content and experiences on the web. Ignoring the temporal arrangements in the web search ecosystem may lead us to both (1) ignore the vast role that the dominant conventional search engine plays in the webpages that exist at any particular point in time, the shape of the web, and (2) develop a ‘blinkered’ perspective on the choices and actions of other participants ready (or not) to perform for (or against) these new approaches to search.\nThere are also direct elements provided on the search engine results pages (SERPs) that are not limited to the retrieval of “pre-existing webpages”. Conventional search engines have long presented features on their SERPs that are extracted and aggregated from those pre-existing webpages or databases not available on the web (in the Knowledge Panels, for instance, and other “rich features” like image galleries). Google’s “Featured snippets” also selectively extract from existing websites (and clearly motivated the creation of content aimed to be found as such) have themselves featured information that has been identified numerous times as being misleading.2\nGoogle has additionally, since as early as 20123, dynamically edited/generated (Google now says: “automatically determine[d]”) both the “Text link” (title of a search result) and the “snippet” (or page summary)4. This practice has not always been appreciated by the SEO community (see multiple responses from Google5, leading to the feedback (on the generated titles) including comments saying in some situations it “seems to be misleading for the user”, and calling it:\n\n“incorrect and misrepresenting”\n“entirely misleading”\n“very misleading”\n“completely wrong”.\n\nAs well as issuing complaints like: “what they are changing is information supplied by others, that represents others, that may be damaging the income of others, and/or potentially putting others at risk (regulatory requirements etc.).”\nAnother feature of conventional SERPs are suggested searches, whether in the autocomplete dropdown while typing a query, in suggesting a revision for an alternate spelling, to reformulate the query to put quotation marks around a particular word, or in various elements interspersed on the search page. Google currently provides a “People also ask” or “Others want to know” section and a “Related searches” section.\nIn additional to generating rich features and the text descriptions of the web pages, conventional search engines respond to queries by providing a ranking (itself treated as information) of selected search results that shape the perceptions of the results by the different searchers and dynamically insert sponsored links (with text links and snippets and only a small text label it indicate it is paid for). All of this is generated on a SERP with particular fonts, colors, sizing, and placement. These various aspects of the SERP are together factors in whether the searchers are satisfied, left wanting, or are effectively misled.6\n\n\nFar, P. (2012). Better page titles in search results. Google Search Central; Blog post. https://developers.google.com/search/blog/2012/01/better-page-titles-in-search-results\n\n\nGoogle. (n.d.-a). Control your snippets in search results. Google Search Central; Webpage. https://developers.google.com/search/docs/appearance/snippet\n\n\nGoogle. (n.d.-b). Influencing your title links in search results. Google Search Central; Webpage. https://developers.google.com/search/docs/appearance/title-link\n\n\nGoogle. (n.d.-c). Visual elements gallery of google search. Google Search Central; Webpage. https://developers.google.com/search/docs/appearance/visual-elements-gallery\n\n\nGriffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, 0(0), 14614448221136505. https://doi.org/10.1177/14614448221136505\n\n\nIntrona, L. D., & Nissenbaum, H. (2000). Shaping the web: Why the politics of search engines matters. The Information Society, 16(3), 169–185. https://doi.org/10.1080/01972240050133634\n\n\nLiu, N. F., Zhang, T., & Liang, P. (2023). Evaluating verifiability in generative search engines. https://doi.org/10.48550/arXiv.2304.09848\n\n\nLurie, E., & Mulligan, D. K. (2021). Searching for representation: A sociotechnical audit of googling for members of U.S. Congress. https://arxiv.org/abs/2109.07012\n\n\nRaji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022, June). The fallacy of AI functionality. 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533158\n\n\nSullivan, D. (2021a). An update to how we generate web page titles. Google Search Central; Blog post. https://developers.google.com/search/blog/2021/08/update-to-generating-page-titles\n\n\nSullivan, D. (2021b). More information on how google generates titles for web page results. Google Search Central; Blog post. https://developers.google.com/search/blog/2021/09/more-info-about-titles\n\n\n\n\n\nLiu et al. (2023)’s opening line: “Generative search engines fulfill user information needs by directly generating responses to input queries, along with in-line citations.”, footnotes to: “In contrast, conventional search engines are limited to retrieving pre-existing webpages.” (This is all I’ve read of the paper so far, beyond the abstract.)↩︎\nSee re seizures (Griffin & Lurie, 2022; Raji et al., 2022), re voting information (Lurie & Mulligan, 2021), etc.↩︎\nFar (2012): “we have algorithms that generate alternative titles to make it easier for our users to recognize relevant pages”↩︎\n“[A]utomatically determine” language is found in Google (n.d.-b) and Google (n.d.-a). “Text link” and “snippet” are preferred terms from Google (n.d.-c).↩︎\n(Sullivan, 2021a, 2021b)↩︎\nA key question, though not the only one, when considering seemingly false or unfounded information on a SERP is whether it is “likely to mislead”. Such information “may not be misleading searchers of the results of search as some inaccurate results likely trigger further information seeking rather than belief in an inaccurate answer” (Lurie & Mulligan, 2021). That said, time costs are still relevant.↩︎\n\n\n", "snippet": "\n", "url": "/2023/05/19/limited-to-retrieving-pre-existing-webpages/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2023-05-19", "type": "posts", "title": "change notes 2023-05-19", "author": "Daniel Griffin", "date": "2023-05-19 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nUpdated my special-box note (at the top of my homepage) about actively seeking research opportunities in both industry and academia.\nVarious formatting updates (distributed links from footer to left-sidebar and a contact-me section in About; developed right-sidebar with floating header links (w/ jekyll-toc); and automatic bibliographies (w/ jekyll-pandoc and apa.csl)).\nAdded redirect to CV (w/ jekyll-redirect-from)\nAdded my commencement speech to Updates!\nAdded a post: conventional search engines are limited to retrieving pre-existing webpages?\n\n", "snippet": "\n", "url": "/changes/2023/05/19/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "updates-2023-05-15-commencement", "type": "posts", "title": "Commencement", "author": "Daniel Griffin", "date": "2023-05-15 00:00:00 -0700", "category": "", "tags": "[running]", "content": "\n \n \n \n \n May 15th, 2023: Commencement\n \n \n \n 🎓 I was the Ph.D. speaker at the I School commencement, discussing how doing a Ph.D. is less like a marathon than a trail ultramarathon. See the text of my speech here.\n\n \n \n \n \n \n \n \n \n \n \n", "snippet": "

    🎓 I was the Ph.D. speaker at the I School commencement, discussing how doing a Ph.D. is less like a marathon than a trail ultramarathon. See the text of my speech here.

    \n", "url": "/updates/2023/05/15/commencement/", "snippet_image": "commencement-speech.png", "snippet_image_alt": "Daniel Griffin speaking at I School Commencement, May 19th 2022. Photo credit: UC Berkeley School of Information", "snippet_image_class": "circle" } , { "id": "updates-2023-05-09-finished-course-at-msu", "type": "posts", "title": "Finished course at Michigan State University", "author": "Daniel Griffin", "date": "2023-05-09 00:00:00 -0700", "category": "", "tags": "[]", "content": "\n \n \n \n \n May 9th, 2023: Finished course at Michigan State University\n \n \n \n I wrapped up teaching a spring course at Michigan State University that I designed: Understanding Change in Web Search. I’m sharing reflections and follow-ups through posts tagged repairing-searching.\n\n \n \n \n \n \n \n \n \n \n \n", "snippet": "

    I wrapped up teaching a spring course at Michigan State University that I designed: Understanding Change in Web Search. I’m sharing reflections and follow-ups through posts tagged repairing-searching.

    \n", "url": "/updates/2023/05/09/finished-course-at-msu/", "snippet_image": "msu_logo.png", "snippet_image_alt": "MSU Logo - wordmark/helmet combination stacked", "snippet_image_class": "circle" } , { "id": "changes-2023-01-06", "type": "posts", "title": "change notes 2023-01-06", "author": "Daniel Griffin", "date": "2023-01-06 00:00:00 -0800", "category": "", "tags": "[]", "content": "\nAdded publication: griffin2022situating\nAdded publication to Updates!\nAdded a shortcut page: griffin2022search\nAdded publication (dissertation) and completion of Ph.D. to: CV\nUpdated About page re completion of Ph.D.\n\n", "snippet": "\n", "url": "/changes/2023/01/06/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "updates-2022-12-16-filled-my-dissertation", "type": "posts", "title": "Filed my dissertation", "author": "Daniel Griffin", "date": "2022-12-16 00:00:00 -0800", "category": "", "tags": "[]", "content": "\n \n \n \n \n December 16th, 2022: Filed my dissertation\n \n \n \n 📁 Filed my dissertation: Griffin D. (2022) Situating Web Searching in Data Engineering: Admissions, Extensions, Repairs, and Ownership. Ph.D. dissertation. Advisors: Deirdre K. Mulligan and Steven Weber. University of California, Berkeley. 2022. [griffin2022situating]\n\n \n \n \n \n \n \n \n \n \n \n", "snippet": "

    📁 Filed my dissertation: Griffin D. (2022) Situating Web Searching in Data Engineering: Admissions, Extensions, Repairs, and Ownership. Ph.D. dissertation. Advisors: Deirdre K. Mulligan and Steven Weber. University of California, Berkeley. 2022. [griffin2022situating]

    \n", "url": "/updates/2022/12/16/filled-my-dissertation/", "snippet_image": "griffin2022situating.png", "snippet_image_alt": "image of griffin2022situating paper", "snippet_image_class": "paper_image" } , { "id": "updates-2022-11-25-griffin2022search-published", "type": "posts", "title": "My paper with Emma Lurie was published", "author": "Daniel Griffin", "date": "2022-11-25 00:00:00 -0800", "category": "", "tags": "[]", "content": "\n \n \n \n \n November 25th, 2022: My paper with Emma Lurie was published\n \n \n \n 📄 My paper with Emma Lurie (equally co-authored) was published: Griffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, Ahead of Print. https://doi.org/10.1177/14614448221136505 [griffin2022search]\n\n \n \n \n \n \n \n \n \n \n \n", "snippet": "

    📄 My paper with Emma Lurie (equally co-authored) was published: Griffin, D., & Lurie, E. (2022). Search quality complaints and imaginary repair: Control in articulations of Google Search. New Media & Society, Ahead of Print. https://doi.org/10.1177/14614448221136505 [griffin2022search]

    \n", "url": "/updates/2022/11/25/griffin2022search-published/", "snippet_image": "griffin2022search.png", "snippet_image_alt": "image of griffin2022search paper", "snippet_image_class": "paper_image" } , { "id": "changes-2022-11-25", "type": "posts", "title": "change notes 2022-11-25", "author": "Daniel Griffin", "date": "2022-11-25 00:00:00 -0800", "category": "", "tags": "[]", "content": "\nAdded publication: griffin2022search\nAdded publication to Updates!\nAdded a shortcut page: griffin2022search\nAdded publication to: CV\n\n", "snippet": "\n", "url": "/changes/2022/11/25/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-11", "type": "posts", "title": "change notes 2022-11-11", "author": "Daniel Griffin", "date": "2022-11-11 00:00:00 -0800", "category": "", "tags": "[]", "content": "\nEdited a post: Added line from Sundin & Carlsson (2016) to search automation bias (SAB)\n\n", "snippet": "\n", "url": "/changes/2022/11/11/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-11-10-search-automation-bias", "type": "posts", "title": "search automation bias (SAB)", "author": "Daniel Griffin", "date": "2022-11-10 00:00:00 -0800", "category": "", "tags": "[automation bias]", "content": "The below is from a short Twitter thread of mine from 2017-05-23:\n\nIs there a term to refer to the seeming authoritativeness of algorithmic expertise of a list of organic and inorganic search results?\nThe clean aesthetic of search results that reduces the cost and shortcuts the processing requirements when reading Google answer boxes?\nOr stopping at the first result?\nOur performed unwillingness to do the costly contextualization of the results of a given query with how we know - if asked - it works?\nHow do we reference the difficulty in disentangling what “Google says” from what a page, which Google says the web values, says?\nWhat do we call the production of epistemic automaticity from Google’s algorithms’ automated search results?\n\n2022-11-10 Notes:\nSearch automation bias (SAB) is one possible term (though my questions above point to ideas probably worth disentangling, for instance, my larger questions are broader than “position bias”—pointing also to the “clean aesthetic”, thinking past the context and construction of search, and differentiating the search engine’s claimed credencing from the whole experience of query => “search media”1 => reading the page2).\nI put some remarks from others’ research below. I’d love to hear more of other terms or ways of thinking through this (or of doing search differently), newly proposed by you or in literature/practice that I’ve missed or failed to recall. SAB seems increasingly common in the the Google Web Search paradigm—with instant answers, rich components, and featured snippets—though perhaps less so where people perceive/believe spam to have clouded the results3. I think the risks from SAB are likely lower for some types of searches or searches within particular contexts (like what I study in my dissertation: data engineers searching for work) and perhaps higher in others (see, for example, a working paper from Lurie & Mulligan4 and their section on defining the search results that are “likely to mislead”) or searches with representational harms (Noble 2018)5).\nI wrote the 2017 thread above while working on Mulligan & Griffin (2018)6—re Google returning Holocaust-denier search pages at the top of the search results for the query [did the holocaust happen] (Cadwalladr 2016)7.\nSee also:\nVaidhyanathan (2011, 15)8\n\nour habits (trust, inertia, impatience) keep us from clicking past the first page of search results\n\n\n\nSundin & Carlsson (2016)9\n\n… and if you put trust in Google’s relevance criteria, as a consequence you outsource critical assessment of information to the information infrastructure and, more precisely, to the algorithms of the search engines\n\nNoble (2018, 116)10\n\n…search results belie any ability to intercede in the framing of a question itself. [ . . . ] What we find in search engines about people and culture is important. They oversimplify complex phenomena. They obscure any struggle over understanding, and they can mask history. Search results can reframe our thinking and deny us the ability to engage deeply with essential information and knowledge we need, knowledge that has traditionally been learned through teachers, books, history, and experience. [em added]\n\nHaider & Sundin (2019, 24)11\n\nshifting the locus of trust from people and institutions to a technology that aims at merging and relocating it\n\n\n\nHaider & Sundin (2019, 33-34)12 — see the full paragraph (which pulls together White (2016, p. 65)13 on “position bias” (White: “also referred to as ‘trust’ bias or ‘presentation bias’”); Pan et al. 200714; Schultheiß et al. 2018’s15; and Höchstötter & Lewandowski 200916), starting with:\n\nAnother line of research investigates what people choose from the search engine results page – often referred to as SERP – and why they choose as they do. This work convincingly shows that how people choose links is primarily based on where these links are located on the search engine results page.\n\nTripodi (2022, 116)17:\n\n…conflates the explorative search processes——searches that embody learning and investigating——with queries focused on fact retrieval and verification.[40] Further, as Google has worked to “oversimplify complex phenomena” and to prioritize profits over societal engagement with complicated ideas, the tech giant has transformed itself from an exploratory platform into one designed around verification…[41]\n\nFn40. [Marchionini 2006]18\nFn41. [Noble 201819; Haider & Sundin 201920]\n\nNarayanan and De Cremer (2022, 2)21:\n\nusers of search engines act as if search engine algorithms are providers of testimony, and acquire or alter beliefs on the basis of this apparent testimony\n\n\n2022-11-11 Edit: Added line from Sundin & Carlsson (2016)22\n\nNotes and References\n\n\n\nMetaxa, Park, Landay, & Hancock’s Search Media and Elections: A Longitudinal Investigation of Political Search Results in the 2018 U.S. Elections (2019), in Proc. ACM Hum.-Comput. Interact.. doi:10.1145/3359231 [🚨 paywalled, author copy at Stanford’s Social Media Lab] [metaxa2019search]↩︎\nDistinguishing the search results from the complete search, including “the complex impact a page of results can have on users” (Metaxa et al. 2019)[^metaxa2019search]. Alternately worded, in Mulligan & Griffin (2018, 567)[^mulligan2018rescripting]:\n\nresults-of-search (the results of the entire query-to-conception experience of conducting a search and interpreting search results)\n\n↩︎\nAre the claims of Google dying (Brereton 2022)[^brereton2022google] showing up in research on the use of Google?\nChayka (2022):[^chayka2022google]\n\nBrereton’s post–which ended “Google is dead. Long live Google + ‘site:reddit.com’”—became the No. 10 most upvoted link ever on the tech-industry discussion board Hacker News. No. 11 is a complaint about Google’s search results looking too similar to its ads, while No. 12 is a link to an alternative, indie search engine. Clearly, others share Brereton’s sense of search-engine discontentment. [Algolia link not in the original]\n\n↩︎\nLurie & Mulligan’s Searching for Representation: A sociotechnical audit of googling for members of U.S. Congress [DRAFT] (2021) [lurie2021searching_draft]↩︎\nNoble’s Algorithms of Oppression How Search Engines Reinforce Racism (2018), from New York University Press. [book] [noble2018algorithms]↩︎\nMulligan & Griffin’s Rescripting Search to Respect the Right to Truth (2018), in The Georgetown Law Technology Review. [direct PDF link] [mulligan2018rescripting]↩︎\nCadwalladr’s Google is not ‘just’ a platform. It frames, shapes and distorts how we see the world (2016), in The Guardian. [cadwalladr2016googleb]↩︎\nVaidhyanathan’s The Googlization of everything:(and why we should worry) (2011), from University of California Press. [book] doi:10.1525/9780520948693 [vaidhyanathan2011googlization]↩︎\nSundin & Carlsson’s Outsourcing trust to the information infrastructure in schools (2016), in JD. doi:10.1108/JD-12-2015-0148 [🚨 paywalled, author copy available at Lund University’s Research Portal] [sundin2016outsourcing]↩︎\nNoble’s Algorithms of Oppression How Search Engines Reinforce Racism (2018), from New York University Press. [book] [noble2018algorithms]↩︎\nHaider & Sundin’s Invisible Search and Online Search Engines: The ubiquity of search in everyday life (2019), from Routledge. [open access book] doi:10.4324/9780429448546 [haider2019invisible]↩︎\nHaider & Sundin’s Invisible Search and Online Search Engines: The ubiquity of search in everyday life (2019), from Routledge. [open access book] doi:10.4324/9780429448546 [haider2019invisible]↩︎\nWhite’s Interactions with Search Systems (2016), from Cambridge University Press. [book; author copy at ] DOI: 10.1017/CBO9781139525305 [white2016interactions]↩︎\nPan et al.’s In Google we trust: Users’ decisions on rank, position, and relevance (2007), in Journal of computer-mediated communication. doi:10.1111/j.1083-6101.2007.00351.x [pan2007google]↩︎\nSchultheiß, Sünkler, & Lewandowski’s We still trust Google, but less than 10 years ago: An eye-tracking study (2018), in Information Research. [schultheiß2018still]↩︎\nHöchstötter & Lewandowski’s What users see – Structures in search engine results pages (2009), in Information Sciences. doi:10.1016/j.ins.2009.01.028 pre-print [höchstötter2009users]↩︎\nTripodi’s The Propagandists’ Playbook: How Conservative Elites Manipulate Search and Threaten Democracy (2022), from Yale University Press. [book] [tripodi2022propagandists]↩︎\nMarchionini’s Exploratory search: From Finding to Understanding (2006), in Commun. ACM. doi:10.1145/1121949.1121979 [🚨 paywalled, author copy at ResearchGate] [marchionini2006exploratory]↩︎\nNoble’s Algorithms of Oppression How Search Engines Reinforce Racism (2018), from New York University Press. [book] [noble2018algorithms]↩︎\nHaider & Sundin’s Invisible Search and Online Search Engines: The ubiquity of search in everyday life (2019), from Routledge. [open access book] doi:10.4324/9780429448546 [haider2019invisible]↩︎\nNarayanan & De Cremer’s “Google Told Me So!” On the Bent Testimony of Search Engine Algorithms (2022), in Philosophy & Technology. doi:10.1007/s13347-022-00521-7 [🚨 paywalled, email author for a copy] [narayanan2022google]↩︎\nSundin & Carlsson’s Outsourcing trust to the information infrastructure in schools (2016), in JD. doi:10.1108/JD-12-2015-0148 [🚨 paywalled, author copy available at Lund University’s Research Portal] [sundin2016outsourcing]↩︎\n\n\n", "snippet": "\n", "url": "/2022/11/10/search-automation-bias/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-10", "type": "posts", "title": "change notes 2022-11-10", "author": "Daniel Griffin", "date": "2022-11-10 00:00:00 -0800", "category": "", "tags": "[]", "content": "\nAdded a post: search automation bias (SAB)\nEdited a post: “I wish there was a way to turn off snippets.” => Added The Markup’s Simple Search to browser extensions that remove featured snippets.\n\n", "snippet": "\n", "url": "/changes/2022/11/10/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-11-06-harmful-and-empowering-uses-twitter-search", "type": "posts", "title": "Looking at harmful & empowering uses of Twitter Search (prospectus)", "author": "Daniel Griffin", "date": "2022-11-06 00:00:00 -0700", "category": "", "tags": "[]", "content": "Here is text of my prospectus for applying to the 2021 Information Operations Fellowship with Twitter’s Trust & Safety Team. (edited for a missing citation, modified & expanded presentation of citations, links to personal websites added, footnote re META added)\nI recalled this while thinking through the approach to full text search taken by Mastodon. See my post from 2022-11-05: Searching Mastodon?\nIn light of Elon Musk’s recent tweet, analysis and discussion of how Twitter search might be used for harassment (and how search might be resisted or friction added to searching) is needed even more: 2022-11-05 15:31\n\nSearch within Twitter reminds me of Infoseek in ‘98! That will also get a lot better pronto.\n\nWho is left to build guardrails to direct the use of Twitter search? Who is left to monitor and mitigate the harms?\nSee the conversation under Taylor Lorenz’s ((??? @mastodon.social)(https://mastodon.social/(???)])) recent comment re search and discovery on Mastodon. The desire to be able to find what one wants seems to propel one to bowl right over what appears to be deliberately established community protocols and norms that consider also the desires of those wanting to be found (or not).\n[I’ll confess I made some lighthearted comments about Infoseek - 1, 2]\n\nInstructions\n\nApplicants should submit a 1-2 page prospectus along with standard application materials that lays out the scope and objectives of a proposed investigative project or projects. The prospectus should include:\n\nA short description of the proposed project and motivations behind it\nProject objectives and optimal outcomes for the applicant\nExpectations around materials, resources and data access needed to complete the work\nDemonstration of any preparation or pre-work already completed in relation to the project or projects\n\n\n", "snippet": "\n", "url": "/2022/11/06/harmful-and-empowering-uses-twitter-search/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-06", "type": "posts", "title": "change notes 2022-11-06", "author": "Daniel Griffin", "date": "2022-11-06 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a post: Looking at harmful & empowering uses of Twitter Search (prospectus)\n\n", "snippet": "\n", "url": "/changes/2022/11/06/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-11-05-searching-mastodon", "type": "posts", "title": "Searching Mastodon?", "author": "Daniel Griffin", "date": "2022-11-05 00:00:00 -0700", "category": "", "tags": "[]", "content": "After seeing some stray remarks about the lack of full text search on Mastodon, I’ve been curious (and distracted while finishing my dissertation) with how they’ve imagined and constructed #search / #discovery. I’d love to learn more about the technical decisions and the social constructions of search and discovery in these communities (and how these ideas may be helpfully put to practice elsewhere).\nI decided to toss the links I’ve found up here after seeing a post from Taylor Lorenz ((??? @mastodon.social)(https://mastodon.social/(???)])) 2022-11-05 10:27.\nMastodon docs:\n\nMastodon’s full-text search allows logged in users to find results from their own toots, their favourites, and their mentions. It deliberately does not allow searching for arbitrary strings in the entire database.\n\nAn GitHub issue on the W3C ActivityPub repo: “Controlling availability to search” (from 2017-05-03):\n\nEarlier today someone released a search engine for mastodon and the response included several concerns.”\n\nThe above issue links to another, on the W3C Social Web Incubator Community Group repo: “Socially Acceptable Search”:\nThis includes a link to minutes from a 2017-05-19 W3C meeting that mentions limits to technical countermeasures and the importance of social mechanisms and trust. Mentions also the risk of a “harassment-centric search engine” operated outside the approval of the community.\nThe discussion includes mentions of “context collapse” (Marwick & boyd, 2011).\n\nSome of this connects with a (failed) proposal I made in a fellowship application (actually, Twitter’s Information Operations Fellowship) in the summer of 2021, to study the search practices of harassers——a proposal I wish was informed in writing the proposal by an understanding of what Mastodon is doing. [I’ll see about posting a version of that here.]\nAdded 2022-11-06: Here is the text of the proposal: Looking at harmful & empowering uses of Twitter Search (prospectus)\n\nCitations\nMarwick & boyd’s “I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience” (2011), in New Media & Society. https://doi.org/10.1177/1461444810365313 [marwick2011tweet]\n", "snippet": "\n", "url": "/2022/11/05/searching-mastodon/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-05", "type": "posts", "title": "change notes 2022-11-05", "author": "Daniel Griffin", "date": "2022-11-05 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a post: Searching Mastodon?\n\n", "snippet": "\n", "url": "/changes/2022/11/05/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-04", "type": "posts", "title": "change notes 2022-11-04", "author": "Daniel Griffin", "date": "2022-11-04 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded to goldenfein2022platforming: commentary section\n\n", "snippet": "\n", "url": "/changes/2022/11/04/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-03", "type": "posts", "title": "change notes 2022-11-03", "author": "Daniel Griffin", "date": "2022-11-03 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nEdited a post: Regina Bateson is not a politician, Google => At 5:27pm PT on 2022-11-02, Dr. Bateson tweeted: Omg, it’s officially gone!! Google no longer says I’m a politician!!!.\n\n", "snippet": "\n", "url": "/changes/2022/11/03/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-11-02-fear-of-gs-abandonment", "type": "posts", "title": "fear of Google abandoning Google Scholar", "author": "Daniel Griffin", "date": "2022-11-02 00:00:00 -0700", "category": "", "tags": "[]", "content": "I think Google abandoning Google Scholar in the near term is unlikely, particularly if there is not a viable public alternative or it is not transformed into one. This is not a full argument to that effect, but there are likely many benefits that accrue to Google by offering Google Scholar to the public. Some of that is discussed in our paper – in a section on “FREE” (also a critique of the charity framing). See also these comments on Quora from Kynan Eng (HT). Also, see our paper for why this fear should be balanced against concerns about the status quo–which I touch on in this tweet.\nBut, here are a few tweets mentioning such a fear (and see the ranging discussions w/in):\n\n\nWith Google Reader gone, is Google Scholar next? http://t.co/BMb1MHOKY8\n\n— Scholarly Kitchen ((???)) March 25, 2013\n\n\n\n\nThis apparently is effective April 29th. Had I gone down that road, I would have effectively had a couple of months to figure out how to migrate .5 PB of data to a new solution on top of the grant deadline I need to make.God help us if Google decides to kill Google Scholar.\n\n— Bryan William Jones ((???)) February 7, 2022\n\n\n\n\nOne of these days, google is going to decide that it's not interested in google scholar anymore, and this app will go bananas.\n\n— Duane Watson ((???)) June 8, 2022\n\n\n\n\nI'd argue that Google Scholar plays a critical role in modern research. It is free, and it absolutely crushes every other scholarly search engine, including specialized ones 👇Though it is a bit scary that it seems to be a tiny team & it exists only by the charity of Google… pic.twitter.com/kNFJMVZ52F\n\n— Ethan Mollick ((???)) November 1, 2022\n\n\nHT: Antoine Blanchard for linking the (???), (???), and (???) together: here and here.\n", "snippet": "\n", "url": "/2022/11/02/fear-of-gs-abandonment/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-02", "type": "posts", "title": "change notes 2022-11-02", "author": "Daniel Griffin", "date": "2022-11-02 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a tagline to /posts: Trying to be—just a tiny bit—publicly vulnerable about my work, my ignorance and my questions.\nAdded a post: fear of Google abandoning Google Scholar\nAdded a post: Tweets that make me think of bridging technology gaps\nEdited a post: “I wish there was a way to turn off snippets.” => Added Startpage to search engines without featured snippets.\nAdded a disclosure re my use of Twitter’s widgets.js\nAdded [direct PDF link] links to the three shortcuts: goldenfein2022platforming, burrell2019control, and mulligan2018rescripting.\n\n", "snippet": "\n", "url": "/changes/2022/11/02/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-11-02-bridging-gaps-tweets", "type": "posts", "title": "Tweets that make me think of bridging technology gaps", "author": "Daniel Griffin", "date": "2022-11-02 00:00:00 -0700", "category": "", "tags": "[]", "content": "ngrams\ntechnology interdependence, technology gap, gap encounters, coordination, bridges/bridging, affordance, imbrication, routines, pipes, tooling\narticles\n\nBailey, Leonardi, & Chong’s “Minding the Gaps: Understanding Technology Interdependence and Coordination in Knowledge Work” (2010), in Organization Science. https://doi.org/10.1287/orsc.1090.0473 [bailey2010minding]\n\nI mentioned this paper in this thread.\n\nLeonardi’s “When Flexible Routines Meet Flexible Technologies: Affordance, Constraint, and the Imbrication of Human and Material Agencies” (2011), in MIS Quarterly. http://www.jstor.org/stable/23043493 [leonardi2011flexible]\nBailey & Leonardi’s “Technology choices: Why occupations differ in their embrace of new technology” (2015), from MIT Press. http://www.jstor.org/stable/j.ctt17kk9d4 [bailey2015technology]\n\nTweets\n\n\nWhy don't developers write more personal GUI tooling? I mean, besides the obvious reason that GUI libraries kinda suck and are much more oriented towards making consumer apps than personal tooling, and also because there are no good GUI tooling exemplars, and…\n\n— Hillel ((???)) November 1, 2022\n\n\n\n\nI remember the first time I used a Unix pipe to chain tools together. It was my first encounter with the Unix philosophy and it completely changed the way I approach solving problems.Now I break things down into smaller chunks and use the best tool for each unit of work.\n\n— Kelsey Hightower ((???)) March 18, 2022\n\n\n\n\n\n", "snippet": "\n", "url": "/2022/11/02/bridging-gaps-tweets/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-11-01", "type": "posts", "title": "change notes 2022-11-01", "author": "Daniel Griffin", "date": "2022-11-01 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a post: Regina Bateson is not a politician, Google\n\n", "snippet": "\n", "url": "/changes/2022/11/01/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-10-31-regina-bateson-is-not-a-politician", "type": "posts", "title": "Regina Bateson is not a politician, Google", "author": "Daniel Griffin", "date": "2022-10-31 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nEdit: Added Twitter commentary\nEdit: At 5:27pm PT on 2022-11-02, Dr. Bateson tweeted: Omg, it’s officially gone!! Google no longer says I’m a politician!!!.\n\n\nGoogle’s knowledge\nYet again someone is compelled to take to Twitter to issue a complaint to Google about incorrect information about them in search results and to appeal for correction.1\nThis time it is Regina Bateson, an assistant professor at the Graduate School of Public & International Affairs at the University of Ottawa.\nGoogle is calling her a politician. She isn’t one. She, rightfully, wants it corrected.\n\nGoogle search: [Regina Bateson], the knowledge panel data labeling her as a “Politician”, and the top result: Her website, with the following snippetI am a comparative political scientist interested in violence, the rule of law, and threats to rights and democracy. · While I have particular expertise in Latin … [Red markings indicate the politician label is incorrect.]\n\nThis short blog post is partially an attempt to provide “web evidence” to aid their appeal for correction. It is also important to make note of these failures—not just the incorrect information but the work required to correct them. Just yesterday I had a conversation with someone who offhandedly passed on conventional wisdom, the searching-sublime,2 about how easy it was to search for everything. It is important to puncture perceptions of Google omniscience, to add friction to the automation bias and the granting of authority to whatever shows up at the top of Google search results.3 Perhaps reflecting on these will help us stop ‘just googling’ and reimagine what just–reasonable, responsive, responsible–search engines might be.\n\nThe Web’s evidence\nRegina Bateson is not a politician. Dr. Bateson (Ph.D. in Political Science, Yale) is a political scientist and professor at the University of Ottawa. This is documented throughout the web: her faculty page at the University of Ottawa, her personal website, her ORCiD, The Conversation, WBUR (Boston’s NPR news station), etc. In 2014, MIT News also identified her as a political scientist. Though she ran for office once she is not, by most any definition, a politician. The Google-provided label is incorrect, deceptive, and non-representative. Why doesn’t Google correct their search results? Or why does it take so much work and time for corrections to happen?\nHere is @regina_bateson on Twitter (Bateson, 2022):\n\nHey @Google, quick FYI: I am not a politician. I ran for office ONCE in 2017-2018, but I’m actually a political scientist, author, etc.\nOn Sept. 27 your team agreed to remove the term “politician” from my Knowledge Panel–but it’s still there. Why? How can I change this? HELP!!!\n\n\n\nAddenda\n\nTwitter commentary\nQuote-tweet from Vivek Krishnamurthy (law professor at University of Ottawa and director of the Samuelson-Glushko Canadian Internet Policy and Public Interest Clinic (CIPPIC)), of a quote-tweet from Regina Bateson, asks about how this, “Google’s failure to correct a false, AI-generated”Knowledge Panel\" description of a person\", interacts with new and proposed laws:\n\n\nHey #lawtwitter: does Google's failure to correct a false, AI-generated “Knowledge Panel” description of a person violate PIPEDA? If not, shouldn't we make sure that this sort of thing is actionable under C-27? These kinds of false descriptions can cause folks real harm. https://t.co/3hkAC8vZBD\n\n— Vivek Krishnamurthy ((???)) November 1, 2022\n\n\nAnother account mentioned an earlier case where a privacy expert apparently had to go to great lengths, and get media attention, to get Google to correct a knowledge panel that put his photo on a description of someone else with the same name: Leo Kelion (2019) “Google faces winged-monkey privacy protest” in the BBC.\n\n\n“Omg, it’s officially gone!! Google no longer says I’m a politician!!!”\n\n\nOmg, it’s officially gone!! Google no longer says I’m a politician!!!It took 4 YEARS but I am v glad to finally have accurate search results.THANK YOU (???) (???) (???) (???) (???) (???) & all who helped in this effort! 🎉🎉🎉 pic.twitter.com/dZAku7KDCi\n\n— Regina Bateson ((???)) November 3, 2022\n\n\nShe also tweeted: “I suspect (though can’t prove) this label made my path back into academia harder than it would’ve been otherwise.”\n\nWell, it was understandable when I was running for office & maybe for a couple months or a yr after. But years 2,3 & 4 were just unnecessary. I suspect (though can’t prove) this label made my path back into academia harder than it would’ve been otherwise. VERY GLAD it’s gone!! 😊\n\nIt has been, she tweeted in September, “years of trying to get it changed”.\n\n\n\n\nBateson, R. (2022). Hey @google, quick fyi: I am not a politician. I ran for office once in 2017-2018, but i’m actually a political scientist, author, etc. On sept. 27 your team agreed to remove the term \"politician\" from my knowledge panel–but it’s still there. Why? How can i change this? HELP!!! https://twitter.com/regina_bateson/status/1585995090480476160.\n\n\n\n\n\nThis post is shaped by, but not about, thinking and writing with Emma Lurie for a paper, recently accepted at New Media & Society: “Search quality complaints and imaginary repair: Control in articulations of Google Search” (initially presented at a Data & Society workshop in early 2022: The Social Life of Algorithmic Harms)↩︎\nThis loose treatment of Google as all-knowing sits in ironic tension with contemporaneous searching-disappointed from perceptions of increasing capture of Google results by spam and commercialization. Like these semi-viral tweets here:\n\n\nEvery week Google search becomes worse and worse and we’re so used to it nobody even talks about it anymore.\n\n— SwiftOnSecurity ((???)) April 24, 2020\n\n\n\n\nFor many classes of topics/questions Google Search has become super SEO'd, surfacing very low quality often ad-heavy/paginated content. I find myself appending “reddit” to a lot of queries and often getting much better results.\n\n— Andrej Karpathy ((???)) March 7, 2021\n\n\n\n\nthe best thing about saying “google it” much like “it’s not my job to educate you, google is for free’ is that when you google anything nowadays the first 3 or 4 pages are SEO’d out marketing garbage\n\n— Tales from the Crypto.com Arena ((???)) January 14, 2022\n\n\n\n\nGoogle search is becoming one of those dying malls. You still go there out of habit, but once you get there, none of it is what you want. You can remember when it was a useful place to visit, but now it's weirdly hollow and you leave without getting what you came for\n\n— Emily Velasco ((???)) October 25, 2022\n\n\n↩︎\nA tweet of mine from 2017:\n\n\nWhat do we call the production of epistemic automaticity from Google's algorithms' automated search results?\n\n— Daniel Griffin ((???)) May 24, 2017\n\n\n\n↩︎\n\n\n", "snippet": "\n", "url": "/2022/10/31/regina-bateson-is-not-a-politician/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "2022-10-30-turn-off-featured-snippets", "type": "posts", "title": ""I wish there was a way to turn off snippets."", "author": "Daniel Griffin", "date": "2022-10-30 00:00:00 -0700", "category": "", "tags": "[]", "content": "I recently saw someone tweet: “I wish there was a way to turn off snippets.”\nThey were referring to turning off snippets on Google SERPs (search engine results pages) to improve the searcher’s searching experience, particularly to avoid being led astray by the summarization performed by Google.\nThese snippets, featured at the top of the results, are sometimes very flawed. I found the above tweet in reply to a semi-viral tweet about one problematic featured snippet (see Background, below). (The poster may have been referring to rich snippets (link to a very Google-focused and incomplete Wikipedia page) generally (also called “rich elements”), but I’m focused below on featured snippets.)\nIt looks like there are maybe five options for avoiding featured snippets, as a searcher:\n\n1. A browser extension\n2. a SERP emulator (wc?1)\n3. searching via the URL\n4. convince Google to provide an option, or\n5. use another search engine.\n\nThere are probably many variations of the first two options.\n\n\nOption 1.\nI’ll discuss here two approaches to using browser extensions to block featured snippets: one content blocking browser extension, uBlock Origin, and one, more general-purpose, userscript manager, Tampermonkey.\nuBlock Origin\nThe first page of results when searching [is there a way to turn off google featured snippets?] on Google2 gave me several results for webpage owners (who can choose to stop Google from making snippets of their content)3 and one for searchers: A Reddit post in r/uBlockOrigin from 2019:\n\n\nThe provided filter for use with the uBlock Origin extension (for Firefox or Chromium browsers) still works today4, though I slightly modified5 it to make the block more complete:\n\n\nTampermonkey\nOne tweet suggested, in reply to the original tweet wishing to turn off snippets, finding a Greasemonkey (for Firefox only) or Tampermonkey (Chromium-based browsers and others), script.\n\n\nThere might be if you use grease monkey or tamper monkey, they might have scripts or you can make one to disable snippets\n\n— Taibhse Designs ((???)) October 28, 2022\n\n\nI found and tested this script, which works—though I don’t know JavaScript well enough to do more than a cursory examination (there are probably tools that could help confirm it is only filtering?): Google Search Result Clean6\nEither approach probably will not work, though, if you are concerned about students searching on Chromebooks (as discussed in the thread with the question for this post; see Background, below) that likely have onerous restrictions on browser extensions.\n\n\nThe Markup’s Simple Search\nYou could also try The Markup’s Simple Search extension (which still works as of today, exactly two years after being published). The search experience, in a popup, is perhaps off-putting and may be best thought of as a demonstration or provocation, a speculative design: a reimagining. (added 2022-11-10)\n\n\nOption 2.\nYou can try a sort of SERP “emulator” or proxy that does not include featured snippets, like this one of what Google was like in 20009: https://oldgoogle.neocities.org/2009/\nI cannot vouch for the privacy of such searches.\n\n\nOption 3.\nAdd a &num=9 parameter to the URL for the query.\nExample:\nhttps://www.google.com/search?q=what+are+featured+snippets\nhttps://www.google.com/search?q=what+are+featured+snippets&num=9\n\nvia this page directed at SEOs wanting to not be featured in snippets (from Vlado Pavlik), which credits Kevin Richard on Twitter:\n\n\n\nhere is the explanation: you force the number of results per page to be lower than 10 (it also works with num=8, num=7, etc). And featured snippets are triggered only for serps of 10+ results.So the “featured snippet layer” is disabled, so the non-snippet rankings are shown.\n\n— Kеvіn Rіchаrd ((???)) January 23, 2020\n\n\nThis might only be useful if you have scripts that open queries via the URL7.\n\n\nOption 4.\nConvince Google to provide searchers with the option to turn of the snippets?\n\n\nOption 5.\nUse a different search engine that does not have featured snippets.\nI don’t know of a list of search engines that do not at all use featured snippets or other machine summarizing for some queries. (Here is a general list of search engines, on Wikipedia.) These look like likely candidates:\n\nMojeek (it does at times return Wikipedia content and news results in the sidebar)\nMillion Short8\nsearch.marginalia.nu9\nStartpage: “Unlike Google, Startpage doesn’t offer”featured snippets“, which are answers extracted directly from relevant websites. In contrast, Startpage’s”Instant Answers\" only get information from a dozen websites.41\" (Wikipedia) => “enabled by default, but you can disable them on the settings page”\n\nRHT: Janet Vertesi’s Google-Free Living (it’s better here)\n\n\n\n\nBackground\n\n\nDoing any research projects with my 7-9s had me screaming, begging, and pleading with them to fucking click the links and read the actual article. I wish there was a way to turn off snippets.\n\n— Verso Book Reader™ ((???)) October 26, 2022\n\n\nI found the above wish in the replies to a recent complaint about Google’s Featured snippets—showing how a snippet distorts the message of a website addressing myths10 about pouring liquid cooking oil down the drain:\n\n\nwhen will google realize that Featured Snippets are an abject failure? pic.twitter.com/lcVew7Rlhv\n\n— ◤◢◤◢◤◢◤◢ ((???)) October 26, 2022\n\n\nThere are two screenshots in the tweet:\n\nOf the featured snippet:\n\nIt’s okay to pour liquid oils down the drain.\nLiquid cooking oils float on water and easily adhere to sewer pipes. The oily film can collect food particles and other solids that will create a blockage.\n\nA portion of the snippeted website: An image captioned “Grease-clogged Pipe (photo courtesy Arlington County DES)” and the following text:\n\nMyths\nIt’s okay to pour grease down the drain if I run hot water with it.\nThis only moves the grease further down the sewer line. Eventually the water will cool and the grease will begin to solidify and coat the pipes.\nIt’s okay to pour liquid oils down the drain.\nLiquid cooking oils float on water and easily adhere to sewer pipes. The oily film can collect food particles and other solids that will create a blockage.`\n\n\nAddendum\nThis makes me curious whether non-Google researchers have studied folks searching with featured snippets blocked with an extension, like that in Option 1 above. I’ll have to look…\n\n2022-11-02 Edit: Added Startpage to search engines without featured snippets.\n2022-11-10 Edit: Added The Markup’s Simple Search to browser extensions.\n\n\n\n\nThere is probably a better word than emulator or proxy to refer to these.↩︎\nQuickly doing the same search on Bing, DuckDuckGo, Neeva, and You.com seems to return only results for webpage owners in the top results. Perhaps the query could be easily reformulated to clarify intent.↩︎\nThe first result at the time of my search, a featured snippet, actually includes, in passing, the tip shown in option 3.↩︎\nI tested by searching g[what are featured snippets?], which at least presently includes a featured snippet, with the filter and extension both enabled and disabled.↩︎\nmodified (hopefully not too aggressively) to:\ngoogle.*##.xpdopen:has-text(Featured snippet):nth-ancestor(6)↩︎\nRelevant to my dissertation research, broadly, commented-out code for the Google Search Result Clean userscript contains instructions for also removing “naive and annoying websites” from Google search results, including: ‘www.w3schools.com’, ‘www.asciitable.com’, ‘www.dba-oracle.com’, ‘www.geeksforgeeks.org’, ‘www.tutorialspoint.com’ (these are websites that I’ve seen people complain about on Twitter with regard to doing programming related web searching, and one of the few examples I found of people programmatically modifying their web search tools). See numerous tweets on the topic by searching, for example, t[google w3schools hide]. A comment in the code suggests, instead, using uBlacklist to block those sites from search results.\n // Remove naive and annoying websites.\n // Comment this out, use uBlacklist instead\n↩︎\nFor example, I wrote a clunky plugin for myself for Sublime that takes a simple notation (the search tool code followed by the query in square brackets) and with a hotkey opens a browser tab with the search. So I can write g[search this] or bmail[search this] in my notes and my hotkey will open a tab in my browser to the Google or Berkeley Gmail, respectively, search for [search this].↩︎\nMillion Short and search.marginalia.nu are not currently listed on the Wikipedia list of search engines. :/↩︎\nMillion Short and search.marginalia.nu are not currently listed on the Wikipedia list of search engines. :/↩︎\nThis pattern, failing to process a website addressing a myth or “things you should never do”, appeared also in Google’s seizure search failure from October 2021.\n\n\nThe Google search summary vs the actual page pic.twitter.com/OJxt1FrBqh\n\n— soft! ((???)) October 16, 2021\n\n\nThere are two screenshots in the tweet:\n\nThe snippet for a “People also ask” prompted search: [Had a seizure Now what?]: “Hold the person down or try to stop their movements. Put something in the person’s mouth (this can cause tooth or jaw injuries) Administer CPR or other mouth-to- mouth breathing during the seizure. Give the person food or water until they are alert again.”\nA portion of the snippeted website:\n\nDo not:\n\nHold the person down or try to stop their movements\nPut something in the person’s mouth (this can cause tooth or jaw injuries)\nAdminister CPR or other mouth-to-mouth breathing during the seizure\nGive the person food or water until they are alert again\n\n\n\n↩︎\n\n\n", "snippet": "\n", "url": "/2022/10/30/turn-off-featured-snippets/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-10-30", "type": "posts", "title": "change notes 2022-10-30", "author": "Daniel Griffin", "date": "2022-10-30 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded a post: “I wish there was a way to turn off snippets.”\n\n", "snippet": "\n", "url": "/changes/2022/10/30/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , { "id": "changes-2022-10-27", "type": "posts", "title": "change notes 2022-10-27", "author": "Daniel Griffin", "date": "2022-10-27 00:00:00 -0700", "category": "", "tags": "[]", "content": "\nAdded /scholar_profiles page (linked in Footer)\nAdded to goldenfein2022platforming\n\nsection IDs w/ links across top of page\nrelated work section\n\nAdded Mastodon to contact div in Footer\n\n", "snippet": "\n", "url": "/changes/2022/10/27/", "snippet_image": "", "snippet_image_alt": "", "snippet_image_class": "" } , {