ssr: “What's the phrase to describe when an algorithm doesn't take into effect it's own influence on an outcome?”

@pdgoldman via Twitter on May 28, 2019

What’s the phrase to describe when an algorithm doesn’t take into effect it’s own influence on an outcome?

from replies:

the ripple effect trap (in Selbst et al. (2019, p. 62))
unacknowledged reflexivity or reification; the ideas behind Goodhart’s law & Campbell’s law
hidden feedback loop
"“weak to Lucas” algorithm, for the Lucas critique in economics, which basicly states that a macroeconomic model that ignores the effects that policies have on individuals cannot be used to recommend which policies to implement" (Wikipedia: Lucas critique)

References

Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. https://doi.org/10.1145/3287560.3287598 [selbst2019fairness]