Returning to life!
Welcome back to the tidy design principles substack. I’m bringing it back to life to talk about what I’m thinking most about these days: AI. There are so many voices talking about AI, and I feel ambivalent about adding one more. And frankly, I have a lot more questions than answers. But writing and listening are my keys of tools for understanding a new domain, so I hope you’ll join be on this journey.
In future posts, I’ll get more technical, but I wanted to begin by acknowledging your likely deeply conflicted feelings about AI.
Let me start with the bits of AI that I feel genuinely excited about:
Programming accessibility. There are tons of people who could benefit from a programming language like R, but can’t justify the investment in learning it. AI has dramatically lowered that barrier and you can now get many of the benefits of reproducible programming with R much faster than you could before. Similarly, effective usage of git is now within reach to a much broader audience.
Translation. While machine translations are still far from perfect, their quality has improved radically in the last few years. This means that much more of the programming ecosystem is now available to the majority of the world who are not fluent English speakers.
Voice input. Voice input is a super exciting technology because it means that you no longer need to be a fluent touch typist in order to quickly get your thoughts into a computer. (Not to mention making a lot more technology available if you can’t read or write.) That’s a meaningful expansion of who gets to participate in technology.
Wide and shallow expertise. I love Tukey’s quote that statisticians get to play in everyone’s backyard. And it’s now easier than ever thanks to AI. AI will not make you an expert but can give you shallow expertise in basically anything you’re curious about. I think that’s pretty cool.
Finally, I have found AI to be a tremendous accelerator in my own work. It’s allowed me to fix 100s of issue in core infrastructure packages like roxygen2 and testthat. This is not AI slop; this is carefully vetted code that I can now write ~2-5x faster than I could before.
But you can’t use AI without also considering the harms, of which there are many.
Environmental impact. At the individual level, I believe that if you want to reduce your environmental footprint, there are higher-leverage changes that you can make. But at the societal level, the picture is more concerning: the rush to create new data centers is increasing need for electricity and water, and leading companies to rollback their climate commitments.
Copyright theft. LLMs are trained on vast quantities of copyrighted material, taken at an unprecedented and industrial scale, without permission or compensation.
Concentration of wealth. The AI craze is pushing more and more money into the hands of fewer and fewer people. I find the concentration of wealth and power into the hands of a very small number of people to be genuinely disturbing and I think is something that we should all be concerned about.
Intellectual laziness. AI supports a kind of shallow engagement where you never have to strain your brain on any task. The path of least resistance is to disengage and just let the model handle it. You no longer have to experience any mental discomfort, and thus you never really learn.
Equity and access. I’ve built my career around open source software, and one of the things I love about it is that it’s available to everyone, everywhere in the world, regardless of their means. That’s not possible with AI. The best tools cost real money, usually charged in US dollars, and that makes them out of reach for a lot of people in a lot of places.
How do you resolve the tension between the empowering and harmful parts of AI? I wish I could give you an answer. All I can suggest is to sit with this conflict. Acknowledge that it’s complicated and there are no easy answers. And do your best to ignore the AI boosters and AI doomers who want to make it easy.
Why am I engaging so heavily with AI? Firstly, I see my job as broadly empowering data scientists. If data scientists are now using AI, then it’s my job to look at what they’re doing and see if I can help them to do it better. Secondly, it feels crucial for the future of Posit. We’re a successful company and doing well, but if we don’t seriously engage with AI then I don’t think we can survive. And while I’m admittedly biased, I do think the failure of Posit would be a loss for the world since we invest so much into free and open source tools.
So that’s why I’m bringing this substack back to life to talk about AI. But I want to know how I can best serve you. What are your concerns about using AI for data science? Where are you seeing successes and failures? What do you want to learn about? Please let me know in the comments, or if you’d like a deeper discussion you can find a time to chat with me1.
I’m doing six of these calls a week, and new slots will be opening up regularly.
I just stumbled upon your page and I can't wait to read more! I would be curious about how roles as we know them might transition. For example, I work as a data scientist in higher ed. and I think that - because everybody around me is vibe coding now - my job will shift to building the frameworks that let others build well, rather than building tools or doing analyses myself.
When we look at AI's highest potential for disruption, it's in the rent seeking sector: finance, law, education, medicine, and administration. It's obviously not the case that these sectors are 100% rent seeking but they do contain a disproportionately large amount of it. Exploiting information asymmetry for private gains made from intentionally convoluted rules and gatekeeping is far more destructive to equality than tech billionaires. Unnecessary credentialism and the pursuit of it harms innovation. The pie only shrinks from rent seeking. Think of the legions of tax lawyers, accountants, and other specialists and all that intellectual horsepower dedicated to squaring someone's bill with the government. If AI shatters those types of enterprises, intelligent people will have to dedicate their lives to more productive endeavors and in doing so may lead to less inequality, not more.
Thank you Hadley for all you do. In my public service data science team in the Canadian government, we do our best using R to try to make the world a little bit better of a place.