Similaring: Monitoring and Logging similar searches with the Metaphor API

    [cat:pposts]
    November 17th, 2023

      I’ve long been impressed with the performance of the search in Metaphor Systems, from taking long-form queries and autoprompting, to allowing a range of filtering options and finding similar pages.1 I was recently trying the similar search again, reflecting on Metaphor’s “Job Search with Metaphor” showcase, written by Sarah Chieng. In that post she explains how to write a Metaphor query, then describe the similar search function:

      But, there’s actually an even better ✨magical✨ way to take advantage of Metaphor. I can just paste a job posting I like and get similar ones.

      Rather than paste a job posting, I pasted in my hire-me page (which was indexed by Metaphor), excluded my own website and filtered to just the past month:

      [ https://danielsgriffin.com/hire-me&filters={“domainFilterType”:“exclude”,“timeFilterOption”:“past_month”,“activeTabFilter”:“all”,“excludeDomains”:[“danielsgriffin.com”]} ]

      Search results for the query on Metaphor

      I was floored, the results were very relevant, including pages and posts I had already seen and liked. I decided to experiment with the Metaphor API by writing up a simple script I could use to periodically run a few of these find_similar searches. This filters to exclude the domain of the searched-URL (using exclude_source_domain) and filters to only pages published within the last 30 days (start_published_date).2

      I have several pages now set to be monitored and logged:

      The results include positions, people, and pages I’ve already known about and now ones I look forward to looking into. Relevance isn’t perfect, but the experience is rather like serendipity. (Note that relevance is constrained by the date filter. Experiment with different settings for your own searches and search purposes.)

      Here is a display of my most recent search:

      Latest Similaring Search Results

      Searched URL: https://danielsgriffin.com/hire-me/
      Searched URL: https://danielsgriffin.com/research/
      Searched URL: https://danielsgriffin.com/about/
      Searched URL: https://danielsgriffin.com/diss/
      Searched URL: https://danielsgriffin.com/publications/

      I was compelled to finally try out the Metaphor API a bit more as I explored the GPTs that Jeffrey Wang prepared:

      @wangzjeff via Twitter on Nov 16, 2023

      If you’re building a dev-facing product, I highly recommend making GPTs that answer questions about it! Some benefits: - Can explain product nuances - Can write code - Familiar form factor

      Below are some GPTs I built for Metaphor Search API. Lemme know what you think!

      Image 1

      Image 2

      Footnotes

      1. Google has had “Pages similar to:” on its search results page. (Though I don’t think it is currently provided?) See this post: “Discovering pages “similar to” ones that you like" (Apr 27, 2010; by Doantam Phan)

        We’ve offered a “Similar” feature on results for a while now as a way to discover new, useful sites, but it hasn’t been too visible. Since we’ve been continuously improving this feature and we think it’s really useful, we’re now going to start showing these alternative sites more prominently. Starting this week, for queries where similar sites are likely to be helpful, we’ll display a list of “Pages similar” at the bottom of the results page.

        ↩︎

      2. The date filter likely uses the Last-Modified HTTP header (which is editable by the publisher) so it does not indicate when a page was created, but hopefully does indicate when substantive content changes were made.↩︎