Stochastic Parrots

This article receives periodic updates as the discussion around Stochastic Parrots evolves.

At least for the immediate future, humans create technology.

But what is it that we choose to create?

The authors of the Stochastic Parrots paper asked this question about large language models and discovered some interesting answers.

Their conclusions were enough to get one of the researchers fired from her corporate job. When corporations try to cover something up, it inherently indicates a potentially groundbreaking insight.

Even without a deep understanding of the methodologies described, the paper is worth a read for the clear and cogent way it frames implications of the development of these models.

Continuing the theme of moving from the virtual algorithmic space to the physical world, Atlas of AI provides us further illustration for the problem statements in “Stochastic Parrots.”

What do all the words in these articles mean?

MIT has a handy guide.

What are some other implications?

You might not immediately leap from NLP to Bitcoin, but consider the following: cryptocurrency mining is energy intensive. In fact, it is so energy intensive that even the general public is starting to question the climate implications of this energy usage.

Inherent in Dr. Gebru’s work is the question, “do the creators of technology represent the users?”

Or, more specifically, “do the privileged, mostly male and WEIRD creators of technology realize that there are very serious, real-world implications for their technology, especially the ways in which they reinforce existing social norms and oppressions for marginalized communities?”

Cryptocurrency represents the ability to fundamentally transform a key concept of modern society: money. One of the most touted benefits of it is the fact that it is “peer-to-peer.”

In a dry, finance bro sense, this means “ownership is decentralized,” and in a right-wing cowboy sense, the implication is, “screw the government.”

In an optimistic and intellectual sense, it is more like what Brice Berdah calls out in Hackernoon: “who better than the future users themselves could design the currency the most fitted to their needs?”

If Dr. Gebru is right, and the end users of a technology are not defining it (or, really, those who are defining it aren’t actually considering the lived experience of the end users), then what we end up with is a grand promise falling flat: ARE the future users themselves designing the currency most fitted to their needs?

What would it mean to REALLY involve non-cryptographers, non-techies, non-bitcoin-farmers in the conversation?

Given the lack of centralized authority, it probably means giving more people access to the technology–and I don’t mean by making it mainstream through standard trading mechanisms like an ETF.

I mean that all the crypto enthusiasts, if they really want to make change in the world, should go out and teach, or provide coins and access to people around them. Start solving the day-to-day problems of poor people with little technology access, and they will not only make the better case for their own beliefs but also make the world a better place.

What would it look like for “decentralized” to mean, run by the people?

What would it look like to finance public works projects, schools, libraries, firefighters, EMTs, and other institutions that support the public good?

…and then we can start brainstorming about what to do if, in fact, this broader access creates an energy crisis that accelerates climate annihilation.

Update: 7/16/2021: The founder of Dogecoin seems to have a philosophically similar take on what I argue above, but with different phrasing: “I believe that cryptocurrency is an inherently right-wing, hyper-capitalistic technology built primarily to amplify the wealth of its proponents through a combination of tax avoidance, diminished regulatory oversight and artificially enforced scarcity.” 

In other words, the decentralized/screw the government element means that the rich get richer and the poor get poorer, because by removing it from the commonly established markets, it loses all sense of civic obligation. It also becomes a vector for fraud.

His powerful concluding sentence sums up a relevant philosophy of technology: “New technology can make the world a better place, but not when decoupled from its inherent politics or societal consequences.”