By not accepting patches, and keeping the commercial part of the product proprietary?
One difference with SQLite (great software, and a true open source success story) is that their money comes from support in the form of $1,500 (or $8,000 to $35,000/year) private support or membership in the SQLite Consortium. Sarah Belle Reid/Richard Devine/etc. are not going to pay 35 large to Andrew every year for Rack support. (Maybe Martin Gore would?)
Another difference is that SQLite needed to drive use, whereas Rack needed to drive two related ecosystems: a huge free module library, with tons of open code so that everybody’s modules could get better, and a thriving set of high-quality commercial modules from VCV and others. It’s done so.
You’ve probably seen this, but the licensing options were extensively discussed at the time:
And there was a pretty extensive postmortem (the later post, which I’ve linked to separately, was an eye-opening moment for me around GPL):
