Metering Inspiration Is Possible, Actually
“If you’re a musician, and you spend your whole childhood listening to music, and then you get an idea, you gos compose a song that is inspired by what you’ve heard before, but in a new direction, it would be very hard for you to say: this much was from this song I heard when I was 11, this much was from […]” — Sam Altman
This rhetoric alleges that it is not feasible to determine how much particular works have contributed to particular behaviors of particular AI systems. This post is a layered rebuttal of this position, making the case that it is in fact feasible to determine this in a precise and efficient way. Moreover, rightsholders are in fact in a position to ensure benefit-sharing practices are designed on their own terms, but the window of opportunity is closing rapidly.
First, it is feasible to determine exactly how much a particular work has influenced a particular model behavior. To assess this, one can consider two versions of the model: one whose training has included that work, and one for which that work has been excluded from the training process. The relationship between the likelihood of different behaviors across the two model versions then represents the impact of that work. If the behavior is untouched after the ablation, you could argue that it had no causal influence on it. This is a bit like being able to perfectly rewind the musician’s childhood, except this time around, you’re intentionally hiding a work from them, to see whether they would still end up producing a similar song. Reproducibility is essential to research progress, with software libraries for training models like JAX going out of their way to track random seeds across operations.
Second, while the possibility of perfectly rewinding the childhoods of virtual minds can exactly determine the extent to which behavior draws from particular works, one would need to spends billions of dollars on each exact measurement of inspiration by retraining subtly different models. Fortunately, much progress has been made on effective approximations of these influences. Such effective approximations boast accuracy close to the exact versions, while drastically improving scalability through reduced overhead. For instance, 2023 work by Anthropic on influence functions has highlighted that certain model behaviors associated with aversion to being turned off appear to draw from accounts of people desperate to avoid dying of thirst. Similarly fascinating are results which hint at models drawing from works written in different languages to inform cross-lingual abstractions.
To oversimplify some of these approximations: instead of retraining the whole model, try to unlearn a particular work and see how behavior is impacted. It seems to me particularly disingenuous to argue against the feasibility of metering inspiration when you’ve poached co-authors of state-of-the-art work on influence functions, work which asks: “Are there data attribution methods that are both scalable and effective in large-scale […] settings? […] We demonstrate that [their method] retains the efficacy of [other methods] while being several orders of magnitude cheaper computationally.”
Third, not only is it possible to exactly or approximately rewind the upbringing of virtual minds to study their influences, something which is virtually impossible with people, but we also get the luxury of exact access to the neural activity of AI systems as they operates, which is extremely tenuous with people. David Bau puts it this way: “The nice thing about artificial neural networks is that we can do experiments that neuroscientists would only dream of. We can look at every single neuron, […] we can do all sorts of crazy measurements and interventions […]. And we don’t have to get a consent form.” Going back to metering inspiration, one can capture snapshots of the neural fingerprint of different signature styles or identities, and measure the extent to which these are represented when models are acting in the world. Case in point, 2021 work with OpenAI highlights idiosyncratic neural signatures of thinking about Spiderman or the actress Halle Berry.
Fourth, not only is it possible to rewind the upbringing of virtual minds and exactly monitor their neural activity as they are operating, but one can notice that intellectual property law, as historically practiced by people and for people, is capable of operating with far, far less. It is not the case that it is necessary to rewind the lives of alleged infringers, or continuously monitor their thought process with perfect precision, in order to hope to even be allowed to seek justice in connection to intellectual property. The unprecedented methods afforded by these new entities can only make it easier to reason about copyright, by bringing in piles of reliable evidence against which to assess the defensibility of positions in court. Assessing the defensibility of positions is arguably what judicial systems already do to uphold law, there would only be drastically more evidence to consider when weighing in on the case.
Fifth, not only do we have evidential luxuries way beyond what existing jurisprudence makes-do with, but it is also arguably the case that rightsholders are in a position to ensure that benefit-sharing is structured on their own terms, although the window of opportunity is closing rapidly. Realism and liberalism are two schools of thought in international relations. The first posits that raw power is what dictates how the international stage evolves, while the latter posits that it is the values of peoples that shape how geopolitics unfolds. It may be accessible for model providers to downplay the feasibility of metering inspiration and cast creatives in the wrong ideologically for even having such pretentions. Truth be told, 75% of OpenAI revenue for 2024 comes from individual consumers. It will not always be this way, with AI-run companies likely operating a non-trivial portion of the economy a few years from now. Yet in the short-term, it is very much within the power of creatives to shape norms by voting with their dollars for providers which give back to the community through benefit-sharing practices, whether through one-off licensing ats training or continuous revenue redistribution at inference, or whether using one formula or another.
I’m leading the development of verifiable monitoring for capability consumption at Noema Research, and fleshing out soft law instruments to enable stakeholders to coordinate in shaping AI development. If you’re interested in getting actively involved with this initiative as a rightsholder, I’d love to chat.