I’m flagging two excerpts from a recent Google Research blogpost (Maniatis & Tarlow, 2023): Large sequence models for software development activities
They rightfully note some of the sociotechnical interactions already taking place in engineering work:
Software isn’t created in one dramatic step. It improves bit by bit, one little step at a time — editing, running unit tests, fixing build errors, addressing code reviews, editing some more, appeasing linters, and fixing more errors — until finally it becomes good enough to merge into a code repository. Software engineering isn’t an isolated process, but a dialogue among human developers, code reviewers, bug reporters, software architects and tools, such as compilers, unit tests, linters and static analyzers.
I struggled some with defining success—of web searching—in my dissertation, this seems close to what I went with1:
We see this as the ultimate test of usefulness: do professional developers, who are often experts on the code base and who have carefully honed workflows, leverage the tools to improve their productivity?
HT: Khlaaf (2023), on Twitter):
A lot of ML-based code synthesis products fail to understand complexities of SW development pipeline, but I really enjoyed reading this well-informed post on integrating ML into it. Not surprised to find many have a code analysis and repair background!
↩︎I seek to learn from successful searchers how they make search work. So, I look to data engineers. I closely examine the use of web search in the work practices of data engineering, a highly technical, competitive, and fast changing area. Data engineers are heavily reliant on general-purpose web search. They use it all the time and it seems to work for them. The practical success I report is not determined by some solid ‘gold standard’ metrics or objective standpoint, but by how they have embraced web search and present it as useful and more importantly essential to their work. It is success for their purposes: in gradations, located in practice, and relative to alternatives.
Griffin, D. (2022). Situating web searching in data engineering: Admissions, extensions, repairs, and ownership [PhD thesis, University of California, Berkeley]. https://danielsgriffin.com/assets/griffin2022situating.pdf [griffin2022situating]
Khlaaf, H. (2023). A lot of ml-based code synthesis products fail to understand complexities of sw development pipeline. https://twitter.com/HeidyKhlaaf/status/1665675004686524417. [khlaaf2023lot]
Maniatis, P., & Tarlow, D. (2023). Large sequence models for software development activities. https://ai.googleblog.com/2023/05/large-sequence-models-for-software.html. [maniatis2023large]