Bitcoin Is Worse Is Better (2011 gwern.net)

republished from https://www.gwern.net/Bitcoin-is-Worse-is-Better

gwern.net

The ge­nius of Bit­coin, in in­vent­ing a dig­i­tal cur­rency suc­cess­ful in the real world, is not in cre­at­ing any new ab­struse math­e­mat­ics or cryp­to­graphic break­through, but in putting to­gether decades-old pieces in a semi­-novel but ex­tremely un­pop­u­lar way. Every­thing Bit­coin needed was avail­able for many years, in­clud­ing the key ideas.

How­ev­er, the sac­ri­fice Bit­coin makes to achieve de­cen­tral­iza­tion is-how­ever prac­ti­cal-a pro­foundly ugly one. Early re­ac­tions to Bit­coin by even friendly cryp­tog­ra­phers & dig­i­tal cur­rency en­thu­si­asts were al­most uni­formly ex­tremely neg­a­tive, and em­pha­sized the (per­ceived) in­effi­ciency & (rel­a­tive to most cryp­tog­ra­phy) weak se­cu­rity guar­an­tees. Crit­ics let ‘per­fect be the en­emy of bet­ter’ and did not per­ceive Bit­coin’s po­ten­tial. How­ev­er, in an ex­am­ple of ‘Worse is Bet­ter’, the ugly in­effi­cient pro­to­type of Bit­coin suc­cess­fully cre­ated a se­cure de­cen­tral­ized dig­i­tal cur­ren­cy, which can wait in­defi­nitely for suc­cess, and this was enough to even­tu­ally lead to adop­tion, im­prove­ment, and growth into a se­cure global dig­i­tal cur­ren­cy.

Satoshi pub­lished the first pub­lic ver­sion of his white pa­per on 2008–11–01 after ear­lier pri­vate dis­cus­sions and the whitepa­per was fur­ther edited after­wards, but if you look at the cryp­tog­ra­phy that makes up Bit­coin, they can be di­vided in­to:

• Cryp­to­graphic sig­na­tures
• Cryp­to­graphic hash func­tions
• Hash chain used for proof-of-work
• cryp­to­graphic time-stamps
• re­silient peer-to-peer net­works

But with the ben­e­fit of this hind­sight, one can won­der-why this de­lay?

If the idea is (rel­a­tive­ly) easy to un­der­stand and uses ba­sic ideas⁠, if it is very far from the cut­ting-edge of cryp­tog­ra­phy⁠, then there’s no rea­son it would not be se­ri­ously tried. Cer­tainly the cypher­punks of the ’90s were wildly cre­ative, in­vent­ing every­thing from Cypher­punk⁠/ Mix­mas­ter to Mo­joNa­tion to as­sas­si­na­tion mar­kets to data havens (mem­o­rably de­picted in ). We have al­ready seen 2 of their pro­posed cryp­tocur­ren­cies, and proof-of-work was one of the most com­mon pro­pos­als to deal with the ris­ing tsunami of spam⁠. Why did Bit­coin take a decade to be born? The prob­lem of tim­ing nags at me-sim­i­lar to the his­tor­i­cal ques­tion of why Eng­land ex­pe­ri­enced the In­dus­trial Rev­o­lu­tion and grew to em­pire, and not Chi­na, which seems bet­ter equipped in every re­spect⁠. Where does in­no­va­tion come from? There must be an an­swer. (And it may be sim­i­lar to VR.)

Is the prob­lem one of re­sources? In the whitepa­per, Satoshi re­marks:

A block header with no trans­ac­tions would be about 80 bytes. If we sup­pose blocks are gen­er­ated every 10 min­utes, 80 bytes * 6 * 24 * 365 = 4.2MB per year. With com­puter sys­tems typ­i­cally sell­ing with 2GB of RAM as of 2008, and Moore’s Law pre­dict­ing cur­rent growth of 1.2GB per year, stor­age should not be a prob­lem even if the block head­ers must be kept in mem­o­ry.

That’s fine to say in 2008, after many dou­blings. Would mem­ory be a prob­lem in the 1990s? It does­n’t have to be. The diffi­culty of bit­coin min­ing is ad­justable, so the prob­lem boils down to:

1. disk us­age

• With a smaller hash like SHA1⁠, the 80 bytes can be shrunk
• 10 min­utes is not graven in stone; why not 20 min­utes? Right there we have halved the trans­ac­tion over­head
• the hash tree can be ‘garbage col­lected’ and shrunk
• it is only nec­es­sary to main­tain a full hash tree if one is para­noid.
• In prac­tice, like many pro­grams of the era such as mail or Usenet clients, the de­fault could sim­ply be to hold onto the last n block­s/ hashes (Satoshi es­ti­mates ); this would con­sume a lim­ited amount of disk space.

2. net­work con­nec­tiv­ity is solv­able by so­lu­tions to #1

• A func­tion of the ex­ist­ing hash tree size
• And fre­quency of new trans­ac­tions

It’s worth point­ing out that it’s gen­er­ally ex­pected that at some point or­di­nary desk­top users like you or me are ex­pected to stop be­ing ful­l-fledged nodes and bit­coin min­ers and will in­stead make use of some spe­cial­ist ser­vice run­ning pow­er­ful servers of its own; in a coun­ter­fac­tual uni­verse where Bit­coin was be­gun in the early 1990s, the changeover would sim­ply have oc­curred soon­er. (And with all the in­vest­ment money des­per­ately in­vest­ing in the first In­ter­net bub­ble, it would be quite easy to start such a ser­vice re­gard­less of the tech­ni­cal de­mand­s.)

As well, few of the ob­jec­tions to cryp­tocur­ren­cies seem to have been “com­put­ers which can run it are fan­tas­ti­cally ex­pen­sive”⁠. In com­put­ing, ap­pli­ca­tions and tech­niques are often in­vented many decades be­fore Moore’s law makes them prac­ti­cally use­ful⁠, but this does not seem to have hap­pened with Bit­coin. A sim­i­lar ob­jec­tion ob­tains with patents or pub­lished pa­pers; if Bit­coin was a known idea, where are they? I have yet to see any­body point out what patents might have de­terred cryp­tog­ra­phy re­searchers & im­ple­menters; the an­swer is that there were none. Be­cause there was no in­vestor in­ter­est? Not that Satoshi needed in­vestors, but there were a tremen­dous num­ber of on­line pay­ment ser­vices started in the ’90s, each search­ing for the se­cret sauce that would let them win ‘mind­share’ and ride ‘net­work effects’ to vic­to­ry; Dig­i­Cash again comes to mind. Even in the ’90s, when the In­ter­net seems em­bry­onic to us of the 2010s, there were still many mil­lions of peo­ple on the In­ter­net who could have used a dig­i­tal cash.

So if the ba­sic idea is ac­ces­si­ble, and it’s use­ful on con­sumer-grade hard­ware for the last 20 years or so, then what’s the prob­lem?

I think it’s in­struc­tive to look at Satoshi’s on the Cryp­tog­ra­phy news­group/ ­mail­ing list; par­tic­u­larly the var­i­ous early crit­i­cisms:

Nick Sz­abo sum­ma­rizes the early re­ac­tion:

Bit­coin is not a list of cryp­to­graphic fea­tures, it’s a very com­plex sys­tem of in­ter­act­ing math­e­mat­ics and pro­to­cols in pur­suit of what was a very un­pop­u­lar goal. While the se­cu­rity tech­nol­ogy is very far from triv­ial, the “why” was by far the biggest stum­bling block­-n­early every­body who heard the gen­eral idea thought it was a very bad idea. My­self, Wei Dai, and Hal Finney were the only peo­ple I know of who liked the idea (or in Dai’s case his re­lated idea) enough to pur­sue it to any sig­nifi­cant ex­tent un­til Nakamoto (as­sum­ing Nakamoto is not re­ally Finney or Dai). Only Finney () and Nakamoto were mo­ti­vated enough to ac­tu­ally im­ple­ment such a scheme.

As well, let’s toss in some blog posts on Bit­coin by the cryp­tog­ra­pher Ben Lau­rie and Vic­tor Grischchenko; Lau­rie par­tic­u­larly crit­i­cizes the hash-con­test which guar­an­tees heavy re­source con­sump­tion:

1. “Bit­coin”
2. “Bit­coin 2”
3. “Bit­coin is Slow Mo­tion”
4. “De­cen­tralised Cur­ren­cies Are Prob­a­bly Im­pos­si­ble: But Let’s At Least Make Them Effi­cient”
5. “Bit­coin?”⁠, Vic­tor Grischchenko

What’s the com­mon thread? Is there any par­tic­u­lar fa­tal flaw of Bit­coin that ex­plains why no one but Satoshi came up with it?

No! What’s wrong with Bit­coin is that it’s ugly. It is not el­e­gant⁠. It’s clever to de­fine your bit­coin bal­ance as what­ever hash tree is longer, has won more races to find a new block, but it’s ugly to make your net­work’s se­cu­rity de­pend solely on hav­ing more brute-force com­put­ing power than your op­po­nents⁠, ugly to need now and in per­pe­tu­ity at least half the pro­cess­ing power just to avoid dou­ble-spend­ing⁠. It’s clever to have a P2P net­work dis­trib­ut­ing up­dated blocks which can be cheaply & in­de­pen­dently checked, but there are tons of ugly edge cases which Satoshi has not proven (in the sense that most cryp­tosys­tems have se­cu­rity proofs) to be safe and he him­self says that what hap­pens will be a “coin flip” at some points. It’s ugly to have a hash tree that just keeps grow­ing and is go­ing to be gi­ga­bytes and gi­ga­bytes in not ter­ri­bly many years. It’s ugly to have a sys­tem which can’t be used offline with­out prox­ies and workarounds, which es­sen­tially re­lies on a dis­trib­uted global clock⁠, un­like Chaum’s el­e­gant so­lu­tion⁠. It’s ugly to have a sys­tem that has to track all trans­ac­tions, pub­licly; even if one can use bit­coins anony­mously with effort⁠, that does­n’t count for much-a cryp­tog­ra­pher has learned from in­ci­dents like anon.penet.fi and decades of suc­cess­ful at­tacks on pseu­do­nymity⁠. And even if the money sup­ply has to be fixed (a bizarre choice and more ques­tion­able than the ir­re­versibil­ity of trans­ac­tion­s), what's with that ar­bi­trary-look­ing 21 mil­lion bit­coin lim­it? Could­n't it have been a rounder num­ber or at least a power of 2? (Not that the bit­coin min­ing is much bet­ter, as it's a mas­sive give-away to early adopters. Coase's the­o­rem may claim it does­n't mat­ter how bit­coins are al­lo­cated in the long run, but such a bla­tant bribe to early adopters rubs against the grain. Again, ugly and in­el­e­gan­t.) Bit­coins can sim­ply dis­ap­pear if you send them to an in­valid ad­dress. And so on.

The ba­sic in­sight of Bit­coin is clev­er, but clever in an ugly com­pro­mis­ing sort of way. Satoshi ex­plains in an early email: The hash chain can be seen as a way to co­or­di­nate mu­tu­ally un­trust­ing nodes (or trust­ing nodes us­ing un­trusted com­mu­ni­ca­tion links), and to solve the Byzan­tine Gen­er­als’ Prob­lem ⁠. If they try to col­lab­o­rate on some agreed trans­ac­tion log which per­mits some trans­ac­tions and for­bids oth­ers (as at­tempted dou­ble-spend­s), naive so­lu­tions will frac­ture the net­work and lead to no con­sen­sus. So they adopt a new scheme in which the re­al­ity of trans­ac­tions is “what­ever the group with the most com­put­ing power says it is”! The hash chain does not as­pire to record the “true” re­al­ity or fig­ure out who is a scam­mer or not; but like Wikipedia, the hash chain sim­ply mir­rors one some­what ar­bi­trar­ily cho­sen group’s con­sen­sus:

…It has been de­cided that any­one who feels like it will an­nounce a time, and what­ever time is heard first will be the offi­cial at­tack time. The prob­lem is that the net­work is not in­stan­ta­neous, and if two gen­er­als an­nounce differ­ent at­tack times at close to the same time, some may hear one first and oth­ers hear the other first.

They use a proof-of-work chain to solve the prob­lem. Once each gen­eral re­ceives what­ever at­tack time he hears first, he sets his com­puter to solve an ex­tremely diffi­cult proof-of-work prob­lem that in­cludes the at­tack time in its hash. The proof-of-work is so diffi­cult, it’s ex­pected to take 10 min­utes of them all work­ing at once be­fore one of them finds a so­lu­tion. Once one of the gen­er­als finds a proof-of-work, he broad­casts it to the net­work, and every­one changes their cur­rent proof-of-work com­pu­ta­tion to in­clude that proof-of-work in the hash they’re work­ing on. If any­one was work­ing on a differ­ent at­tack time, they switch to this one, be­cause its proof-of-work chain is now longer.

After two hours, one at­tack time should be hashed by a chain of 12 proof­s-of-work. Every gen­er­al, just by ver­i­fy­ing the diffi­culty of the proof-of-work chain, can es­ti­mate how much par­al­lel CPU power per hour was ex­pended on it and see that it must have re­quired the ma­jor­ity of the com­put­ers to pro­duce that much proof-of-work in the al­lot­ted time. They had to all have seen it be­cause the proof-of-work is proof that they worked on it. If the CPU power ex­hib­ited by the proof-of-work chain is suffi­cient to crack the pass­word, they can safely at­tack at the agreed time.

The proof-of-work chain is how all the syn­chro­ni­sa­tion, dis­trib­uted data­base and global view prob­lems you’ve asked about are solved.

In short, Bit­coin is a per­fect ex­am­ple of Worse is Bet­ter ( orig­i­nal es­say). You can see the trade­offs that Richard P. Gabriel enu­mer­ates: Bit­coin has many edge cas­es; it lacks many prop­er­ties one would de­sire for a cryp­tocur­ren­cy; the whitepa­per is badly un­der­-spec­i­fied; much of the be­hav­ior is so­cially de­ter­mined by what the min­ers and clients col­lec­tively agree to ac­cept, not by the pro­to­col; etc.

The worse-is-bet­ter phi­los­o­phy is only slightly differ­ent: […]

Com­plete­ness-the de­sign must cover as many im­por­tant sit­u­a­tions as is prac­ti­cal. All rea­son­ably ex­pected cases should be cov­ered. Com­plete­ness can be sac­ri­ficed in fa­vor of any other qual­i­ty. In fact, com­plete­ness must be sac­ri­ficed when­ever im­ple­men­ta­tion sim­plic­ity is jeop­ar­dized. Con­sis­tency can be sac­ri­ficed to achieve com­plete­ness if sim­plic­ity is re­tained; es­pe­cially worth­less is con­sis­tency of in­ter­face.

…The MIT guy did not see any code that han­dled this [edge] case and asked the New Jer­sey guy how the prob­lem was han­dled. The New Jer­sey guy said that the Unix folks were aware of the prob­lem, but the so­lu­tion was for the sys­tem rou­tine to al­ways fin­ish, but some­times an er­ror code would be re­turned that sig­naled that the sys­tem rou­tine had failed to com­plete its ac­tion. A cor­rect user pro­gram, then, had to check the er­ror code to de­ter­mine whether to sim­ply try the sys­tem rou­tine again. The MIT guy did not like this so­lu­tion be­cause it was not the right thing… It is bet­ter to get half of the right thing avail­able so that it spreads like a virus. Once peo­ple are hooked on it, take the time to im­prove it to 90% of the right thing.

Guar­an­tees of Byzan­tine re­silience? Loosely sketched out and left for fu­ture work. In­cen­tive-com­pat­i­ble? Well… maybe. Anonymi­ty? Punted on in fa­vor of pseu­do­nymi­ty; maybe some­one can add real anonymity lat­er. Guar­an­tees of trans­ac­tions be­ing fi­nal­ized? None, the user is just sup­posed to check their copy of the blockchain. Con­sis­tent APIs? For­get about it, there’s not even a stan­dard, it’s all im­ple­men­ta­tion-de­fined (if you write a client, it’d bet­ter be “bug­ward com­pat­i­bil­ity” with Satoshi’s clien­t). Moon math? Nah, it’s ba­sic pub­lic-key crypto plus a lot of im­per­a­tive stack­-ma­chine bit-twid­dling. Space effi­cien­cy? A straight­for­ward blockchain and on-disk stor­age takes pri­or­ity over any fancy com­pres­sion or data-struc­ture schemes. Fast trans­ac­tions? You can use ze­ro-conf and if that’s not good enough for buy­ing coffee, maybe some­one can come up with some­thing us­ing the smart con­tract fea­tures. And so on.

But for all the is­sues, it seems to work. Just like Unix, there were count­less ways to de­stroy your data or crash the sys­tem, which did­n’t ex­ist on more ‘proper’ OSs like ⁠, and there were count­less lack­ing fea­tures com­pared to sys­tems like or the Lisp ma­chine OSs. But like the prover­bial cock­roach­es, Unix spread, net­worked, sur­vived-and the rest did not. And as it sur­vives and evolves grad­u­al­ly, it slowly be­comes what it “should” have been in the first place. Or HTML vs Project Xanadu ⁠.

Paul Ford in 2013 has stum­bled onto a sim­i­lar view of Bit­coin:

The In­ter­net is a big fan of the worst-pos­si­ble-thing. Many peo­ple thought Twit­ter was the worst pos­si­ble way for peo­ple to com­mu­ni­cate, lit­tle more than dis­course ab­bre­vi­ated into tiny lit­tle chunks; Face­book was a hor­ri­ble way to ex­pe­ri­ence hu­man re­la­tion­ships, com­mod­i­fy­ing them into a list of friends whom one pokes. The Arab Spring changed the story some­what. (Buz­zFeed is an­other ex­am­ple-let them eat cat pic­tures.) One recipe for In­ter­net suc­cess seems to be this: Start at the bot­tom, at the most aw­ful, ridicu­lous, es­sen­tial idea, and own it. Pro­mote it breath­less­ly, un­til you’re ac­quired or you take over the world. Bit­coin is play­ing out in a sim­i­lar way. It asks its users to for­get about cen­tral bank­ing in the same way Steve Jobs asked iPhone users to for­get about the mouse.

But he lacks the “worse is bet­ter” par­a­digm (de­spite be­ing a pro­gram­mer) and does­n’t un­der­stand how Bit­coin is the worst-pos­si­ble-thing. It’s not the de­cen­tral­ized as­pect of Bit­coin, it’s how Bit­coin is de­cen­tral­ized: a cryp­tog­ra­pher would have diffi­culty com­ing up with Bit­coin be­cause the mech­a­nism is so ugly and there are so many el­e­gant fea­tures he wants in it. Pro­gram­mers and math­e­mati­cians often speak of “taste”, and how they lead one to bet­ter so­lu­tions. A cryp­tog­ra­pher’s taste is for cryp­tosys­tems op­ti­mized for effi­ciency and the­o­rems; it is not for sys­tems op­ti­mized for vir­u­lence, for their so­ci­o­log­i­cal ap­peal⁠. Cen­tral­ized sys­tems are nat­ural so­lu­tions be­cause they are easy, like the in­te­gers are easy; but like the in­te­gers are but a van­ish­ingly small sub­set of the re­als, so too are cen­tral­ized sys­tems a tiny sub­set of de­cen­tral­ized ones⁠. Dig­i­Cash and all the other cryp­tocur­rency star­tups may have had many nifty fea­tures, may have been far more effi­cient, and all that jazz, but they died any­way⁠. They had no com­mu­ni­ties, and their cen­tral­iza­tion meant that they fell with their cor­po­rate pa­trons. They had to win in their com­pressed time­frame or die out com­plete­ly. But “that is not dead which can eter­nal lie”. And the race may not go to the swift, as Hal Finney also pointed out early on:

Every day that goes by and Bit­coin has­n’t col­lapsed due to le­gal or tech­ni­cal prob­lems, that brings new in­for­ma­tion to the mar­ket. It in­creases the chance of Bit­coin’s even­tual suc­cess and jus­ti­fies a higher price.

It may be that Bit­coin’s great­est virtue is not its de­fla­tion, nor its mi­cro­trans­ac­tions, but its vi­ral dis­trib­uted na­ture; it can wait for its op­por­tu­ni­ty. “If you sit by the bank of the river long enough, you can watch the bod­ies of your en­e­mies float by.”

Nick Sz­abo and Zooko Wilcox-O’­Hearn dis­agree strongly with the the­sis that “Bit­coin is Worse is Bet­ter”. They con­tend while there may be bad parts to Bit­coin, there is a novel core idea which is ac­tu­ally very clev­er-the hash chain is a com­pro­mise which thinks out­side the box and gives us a side­step around clas­sic prob­lems of dis­trib­uted com­put­ing, which gives us some­thing sim­i­lar enough to a trust­wor­thy non-cen­tral­ized au­thor­ity that we can use it in prac­tice.

Gw­ern’s post fails to ap­pre­ci­ate the tech­ni­cal ad­vances that Bit­Coin orig­i­nat­ed. I have been try­ing, off and on, to in­vent a de­cen­tral­ized dig­i­tal pay­ment sys­tem for fifteen years (s­ince I was at Dig­i­Cas­h). I was­n’t sure that a prac­ti­cal sys­tem was even pos­si­ble, un­til Bit­Coin was ac­tu­ally im­ple­mented and be­came as pop­u­lar as it has. Sci­en­tific ad­vances often seem ob­vi­ous in ret­ro­spect, and so it is with Bit­Coin.

Nick Sz­abo thinks that the main block­ing fac­tors were:

1. ide­o­log­i­cal be­liefs about the na­ture of money (lib­er­als not in­ter­ested in non-s­tate cur­ren­cies, and Aus­tri­ans be­liev­ing that cur­ren­cies must have in­trin­sic val­ue)
2. ob­scu­rity of bit gold-like ideas
3. “re­quir­ing a proof-of-work to be a node in the Byzan­ti­ne-re­silient peer-to-peer sys­tem to lessen the threat of an un­trust­wor­thy party con­trol­ling the ma­jor­ity of nodes and thus cor­rupt­ing a num­ber of im­por­tant se­cu­rity fea­tures”
4. some sim­pli­fi­ca­tion (not mar­kets for con­vert­ing “old” & hard­er-to-mine bit­coins to “new” & eas­ier-to-mine bit­coins, but a chang­ing net­work-wide con­sen­sus on how hard bit­coins must be to mine)

My own be­lief is that #1 is prob­a­bly an im­por­tant fac­tor but ques­tion­able since the core break­through is ap­plic­a­ble to all sorts of other tasks like se­cure global clocks or time­stamp­ing or do­main names, #2 is ir­rel­e­vant as all dig­i­tal cryp­to­graphic cur­rency ideas are ob­scure (to the point where, for ex­am­ple, Satoshi’s whitepa­per does not cite bit gold but only b-money, yet Wei Dai does not be­lieve his b-money ac­tu­ally in­flu­enced Bit­coin at all!), and #3–4 are mi­nor de­tails which can­not pos­si­bly ex­plain why Bit­coin has suc­ceeded to any de­gree while ideas like bit gold lan­guished.

• -(use and eco­nomic phi­los­o­phy of the Silk Road 1 mar­ket­place)
• ⁠; -(a sim­i­lar so­lu­tion by re­sort to proof-of-work)

Originally published at https://www.gwern.net.