seeking proposals for a wide variety of research projects which might shed light on what real-world impacts LLM systems could have over the next few years.
…
Below are some examples of project ideas that could make for a strong proposal to this RFP, depending on details:
- Conducting randomized controlled trials…
- Polling members of the public…
- In-depth interviews…
- Collecting “in the wild” case studies…
- Estimating and collecting key numbers…
- Creating interactive experiences…
- Eliciting expert forecasts…
- Synthesizing, summarizing, and analyzing the various existing lines of evidence about what language model systems can and can’t do at present…
This is interesting. As often is the case, the resources gathered in the CFP merit note. I could imagine a proposal developed around “studying and forecasting the real-world impacts of [search] systems built from LLMs”…