We Should Slow Down

I regrettably did not sign the pacing the frontier petition because I was on vacation at the time it was circulating. But it is something I want to come out as categorically in favor of. I believe that we are on the precipice of an age of enormous technological progress packaged into an extremely short period of time. I fear that these rapid changes will strain our social infrastructure, which is already quite fragile. We should pull back on the throttle a bit to give ourselves a chance to adapt.

What particularly worries me is that the two companies behind this technology are driven by market competition and a need to pay off their debts to justify their valuations. If AI is as transformative as they think it will be, we're going to rush headfirst into the singularity with rocket engines strapped to our backs. If it is not as transformative, we face a major economic recession. It's a lose-lose situation and I hate it. In this setting, there is no mechanism to "slow down" except government. This issue is not without precedent - capitalist countries have had to deal with this many times in the last two centuries. Our government just needs to do it faster this time.

-----

I also want to say how proud I am of the open weights companies. The latest models are really incredible. I dreamed of this technology (~agents) almost 9 years ago but I was never really sure it would ever actually be built - let alone distributed freely. A future where two powerful companies are the gatekeepers to AI seems dark to me. However - one where we commoditize and disaggregate computational intelligence, where every human is empowered with the knowledge of the internet hosted entirely on their own hardware (or some local provider) - this seems like the right direction to me.

When the cost of compute descends out of the stratosphere, I am tremendously excited to see what small companies will be able to do with these open source models. The possibilities seem so wide to me. It is an amazing time to be alive.