Market failures: the publicly traded company slips the invisible hand

The market is not making the products I want. The market won't provide books for boys like this one, or a PDA solution as robust and reliable as the Palm III, or a reliable and efficient home solution for bulk scanning of 4x6 prints, or a dozen other products I look for and can't find.

There's always a proximal explanation for each failure. The children's book market money is predominantly female or tied to videos, games and movies. The true PDA market is too small. You can't sell a home scanning solution for more than $100, a reliable photo feeder would cost two to three times that.

That might be the whole story, but what about that "long tail" we keep hearing about? These are not products that require an immense amount of R&D. They all use well understood technologies. What's going on?

I don't know of course, but that won't stop me speculating wildly. (What are blogs for, after all?) My wild guess is that the publicly traded company dominates the solution space for all of these products, and the publicly traded company is evolving to evade the grip of the "invisible hand".

First though, today brought a convenient side note from the New York Times:

Reporting for Duty - New York Times:

... Evidence is mounting that giving what’s called quarterly guidance (for example, “next quarter the company is expected to earn $2.42 to $2.44 per share”) is detrimental to a company’s long-term performance. A survey by the National Bureau of Economic Research of 401 senior financial executives found that 80 percent were willing to forgo spending on research and development to meet their predictions, while 55 percent were willing, for the same reason, to delay projects that promise gains in the long term for their company.

Similarly, an empirical analysis of companies that regularly provide such guidance concluded that even though they are more likely to meet their projections than those that use the practice only occasionally, they are less likely to achieve long-term earnings growth...

Interesting, and perhaps some sort of quarterly guidance hack would help, but I'm betting it wouldn't help for long.

Things evolve. RNA, DNA, algae, universes (?), entities, ecosystems, populations, economies, cells, mitochondria, organelles -- anything that belongs to a system that includes boundaries, resource competition, replication and variation. It's more than a law of physics, it's a law of logic. It makes sense that publicly traded corporations evolve too. Early in their evolution they exist to return value to their owners and the "invisible hand" guided that towards value to employees and customers. Systems complexify however, and now there are many strategies to explore in economics space. Not all provide value to customers or even returns to shareholders. At the same time as evolving corporations explore divergent strategies, they are infested with parasites -- also known as stakeholders. Once these were unions, now they are senior management. The parasites (ok, symbiotes), have their own agenda, and their own need for "appropriate compensation".

Boundaries are emergent. In economic space things look different from the world in which we live. It's a kind of parallel dimension in which humans don't exist as individuals, and in which corporations have their own peculiar kind of independent "thought" [1]. In this world, I imagine, they're at least as "smart" as an amoeba.

The Publicly Traded Corporation isn't going away -- the US Patent Office has made that entity even more powerful than it once was. I'm very much hoping, however, that it's not to late to develop alternative economic instruments for the creation and delivery of value. If our tools get good enough, one day we can make our own home photo scanning solution. Call it, the revenge of the artisan.

[1] Sentient corporations are a recurring theme in modern science fiction, I'm not clever enough to have come up with this on my own.

The US military's broken bureaucracy

Phil Carter is the guy I trust on Iraq and on the US military. His blog, Intel Dump, (unfortunate name) includes a record of his recent tour of duty. He volunteered to return to Iraq after finishing his law degree -- driven by patriotism and a powerful sense of obligation. He's smart, informed, and thoughtful.

Now he's turned his analytic skills to figuring out why the senior military leadership has been screwing up recently. He sees a systemic problem ...

It's time to fire a few generals. - By Phillip Carter - Slate Magazine

... In Iraq, where I advised the Iraqi police, I saw this reverse filtration system (whereby excrement is added to the final product, instead of being removed) in action. Reports on police readiness were aggregated, generalized, and stripped of their facts as they moved up the chain of command....

... By the time our reports reached the national level, they contained little of the detail so essential for explaining our progress in standing up the Iraqi police force. This problem exists in many military organizations. Major problems get renamed "obstacles," or "challenges," or some other noun that connotes a temporary delay in forward progress, reflecting the pervasive "can do" optimism of the military officer corps. Staff officers at each level of command refine and insert caveats into reports to ensure they don't rock the boat too much. By the time information reaches a senior commander or civilian official, it no longer reflects reality.

Military bureaucracies (and their civilian brethren like the Department of Homeland Security and the Federal Emergency Management Agency) also do a terrible job of reacting to crises. Large bureaucracies like the Army provide a systematic, uniform, mediocre response to chronic problems. But where time is of the essence, bureaucracies often fail spectacularly...

... Instead of receiving negative information and fixing the root problem, bureaucracies find and apply incrementalist solutions that fit their existing way of doing business. In MBA-jargon, bureaucracies rarely think or act "outside of the box." Whether the context is the Vietnam War, the Iraq war, Hurricane Katrina, or the current mess at Walter Reed, the problem is the same...

But, of course, there are few Pattons left in today's Army, partly because the military has moved away from the tradition of "command responsibility" toward a model of bureaucratic performance. As a lieutenant, I learned that commanders were responsible for all their unit did or failed to do, period. In peacetime, this meant I could lose my job if some soldiers got in a drunken bar fight one weekend or if a sergeant lost too much gear, because I had ultimate responsibility for my unit. In wartime, command responsibility ties in with accomplishing missions: Generals like Patton and Creighton Abrams earned their stars by winning battles, because that is the military's raison d'être.

Unfortunately, this tradition has died. Today, we promote generals and select them for high command even where they fail to accomplish their mission. Commanders responsible for serious breaches of discipline rarely face criminal prosecution anymore and rarely suffer adverse career consequences. Warrior-leaders like Gen. David Petraeus and Marine Lt. Gen. James Mattis do occasionally rise through the system, but they remain the exception.

... Today's decision to sack Maj. Gen. George Weightman, Walter Reed's commanding officer, affirms the principle of command responsibility, thought to be a dead letter after the Abu Ghraib scandals. ...

I think historians will find this same systemic dysfunction ran all the way to the president.

Empires of the middle east - a seriously fascinating site

This is going to win some kind of award. A flash animation displays a few thousand years of conquest of the middle east. They stop short of the American conquest of Baghdad. I'd never heard of the Sassanid and Seljuk empires.

It's a fantastic display, and the same site has another map for religion. I hope they'll do quite a bit more of this work.

PS. In writing this I discovered a nasty Firefox bug. If a Flash animation is playing, and I'm using the FF rich text service via Blogger, I lose keyboard interaction. I have to quit the Flash and the rich text widget to restore a keyboard response.

The limits of disaster predictions: complex adaptive systems

The problem with predicting disaster, is that the human world is a very complex system with unexpected adaptive capacities ...

Stewart Brand - John Tierney - An Early Environmentalist, Embracing New ‘Heresies’ - New York Times

... Mr. Brand is the first to admit his own futurism isn’t always prescient. In 1969, he was so worried by population growth that he organized the Hunger Show, a weeklong fast in a parking lot to dramatize the coming global famine predicted by Paul Ehrlich, one of his mentors at Stanford.

The famine never arrived, and Professor Ehrlich’s theories of the coming “age of scarcity” were subsequently challenged by the economist Julian Simon, who bet Mr. Ehrlich that the prices of natural resources would fall during the 1980s despite the growth in population. The prices fell, just as predicted by Professor Simon’s cornucopian theories.

Professor Ehrlich dismissed Professor Simon’s victory as a fluke, but Mr. Brand saw something his mentor didn’t. He considered the bet a useful lesson about the adaptability of humans — and the dangers of apocalyptic thinking...

"... In 1973 I thought the energy crisis was so intolerable that we’d have police on the streets by Christmas. The times I’ve been wrong is when I assume there’s a brittleness in a complex system that turns out to be way more resilient than I thought.”

I remember Simon. During my brief career as an undercover agent at the UNFPA in Thailand I wrote essays critiquing his positions. I was wrong, Simon was far more accurate in the short run that Ehrlich.

True, apocalypse does happen. I wonder if the the Rwandan genocide was a Malthusian collapse, one time that Ehrlich was right and Simon wrong. It doesn't happen as often as one might expect however.

Why is that? Angels? Aliens? Some emergent behavior of the entangled multidimensional world of thought and money? Complex for sure, adaptive for sure, but those terms don't teach us much. I hope we'll learn more about why 'the center holds', even when it seems it shouldn't.

California genius and the noncompete clause

Bring suit in California …

The Power of the Noncompete Clause — HBS Working Knowledge

... This experience got me thinking that my employment opportunities had been geographically circumscribed by differences in the enforcement of noncompetes between states—that with my highly specialized skills, the only way I could change jobs would be to move to one of the ten states that doesn't enforce noncompetes. (It's a bit of a continuum, where some states are stricter than others, but the ten states considered significantly more lenient are California, Nevada, Arkansas, Washington, Montana, North Dakota, Minnesota, Wisconsin, Connecticut, West Virginia, and Oklahoma.)

But not everyone is affected identically by noncompetes. Our research shows that those with specialized skills (such as speech recognition engineering) are impacted more than those with more generally applicable skills. For instance, C++ programmers can probably find work more easily at companies that don't compete with their current employers. We also found that "star" inventors—those whose patents are highly cited in other patent applications—are also more strongly affected by noncompetes. ...

The noncompete clause is having interesting consequences for business. The outrageous success of Silicon Valley may owe something to California’s adamant refusal to enforce noncompete clauses.

Cringely is blogging for Technology Evangelist

Payback apparently. A MN company saved Cringely from his disastrous NerdTV launch (I told him I don't watch TV!).

St. Paul Pioneer Press | 02/17/2007 | Super techie team-up

... Now, with Cringely aboard, Technology Evangelist's grand goal to become an A-list tech blog seems a bit less of a stretch despite daunting competition from the likes of Ars Technica, Engadget, Gizmodo and Gear Live...

Now he's on their blog. I've added them to my feed of course, but I don't want the other writers -- I just want Cringely. Alas, it seems to be all or none for now.

Update: see the comments. They've added a Cringely feed, possibly in partial response to this post ....

Diffusion of technology: armor penetratiing IEDs for cheap

The recipe required R&D funds. The implementation required only a semi-modern machine shop:

WIRED Blogs: Danger Room

...The Pentagon is claiming -- again -- the the Iranian government supplied the deadly 'explosively formed penetrators' (EFPs). But the more you study these devices -- which use an explosive charge to a convert disc-shaped metal 'lens' into a high-velocity slug capable of smashing through thick armor at an extended range –- the more likely they seem to be home-made in Iraq.

The USSR built a hydrogen bomb quickly -- once they had the US plans. This is the same thing, on a much smaller scale. All it took was for Iran to deliver the plans, which may or may not have had government involved. If Iran did do it they won't have left any fingerprints; their intelligence services are considered among the best on earth.

Welcome to the "age of the fast follower" (TM*). Chlorine truck bombs work the same way. It's hard to keep a technical age when the cost of havoc falls.

* just kidding

Political purges: Bush and the AGs

A Democratic senator with a subarachnoid hemorrhage is still alive. His survival means a Bush neo-Stalinist (without the bloodshed)Nixonian purge may be thwarted:

Why Have So Many U.S. Attorneys Been Fired? It Looks a Lot Like Politics - New York Times

Carol Lam, the former United States attorney for San Diego, is smart and tireless and was very good at her job. Her investigation of Representative Randy Cunningham resulted in a guilty plea for taking more than $2 million in bribes from defense contractors and a sentence of more than eight years. Two weeks ago, she indicted Kyle Dustin Foggo, the former No. 3 official in the C.I.A. The defense-contracting scandal she pursued so vigorously could yet drag in other politicians.

In many Justice Departments, her record would have won her awards, and perhaps a promotion to a top post in Washington. In the Bush Justice Department, it got her fired.

Ms. Lam is one of at least seven United States attorneys fired recently under questionable circumstances. The Justice Department is claiming that Ms. Lam and other well-regarded prosecutors like John McKay of Seattle, David Iglesias of New Mexico, Daniel Bogden of Nevada and Paul Charlton of Arizona — who all received strong job evaluations — performed inadequately.

... It is not just the large numbers. The firing of H. E. Cummins III is raising as many questions as Ms. Lam’s. Mr. Cummins, one of the most distinguished lawyers in Arkansas, is respected by Republicans and Democrats alike. But he was forced out to make room for J. Timothy Griffin, a former Karl Rove deputy with thin legal experience who did opposition research for the Republican National Committee.

...Senator Charles Schumer, Democrat of New York, held a tough hearing. And he is now talking about calling on the fired U.S. attorneys to testify and subpoenaing their performance evaluations — both good ideas.

Ahh. The last sentence tells the story. We teeter on the razor edge of history, governed by one of the most malign collection of incompetents ever to afflict the Republic. It is the razor thin Democrat majority in the Senate that keeps us afloat. Leiberman could tip us into the abyss any day he chooses...

BTW, this story was broken in the blogs weeks ago...

Update: I realized neo-Stalinist was jumping the shark, but I'll stand by neo-Nixonian.

Failures of the human mind - framing and judgment

Joyce Hatto stole music from largely obscure sources and passed it off as hers. The music received rave reviews, but the originals were ignored. Netizens will vaguely remember that in 1997 a commencement speech by Mary Schmich became world famous when a trickster reframed it as a Vonnegut essay. Here are some recent comments from the NYT ...

Shoot the Piano Player - New York Times

... Yet the Joyce Hatto episode is a stern reminder of the importance of framing and background in criticism. Music isn’t just about sound; it is about achievement in a larger human sense. If you think an interpretation is by a 74-year-old pianist at the end of her life, it won’t sound quite the same to you as if you think it’s by a 24-year-old piano-competition winner who is just starting out. Beyond all the pretty notes, we want creative engagement and communication from music, we want music to be a bridge to another personality. Otherwise, we might as well feed Chopin scores into a computer.

This makes instrumental criticism a tricky business. I’m personally convinced that there is an authentic, objective maturity that I can hear in the later recordings of Rubinstein. This special quality of his is actually in the music, and is not just subjectively derived from seeing the wrinkles in the old man’s face. But the Joyce Hatto episode shows that our expectations, our knowledge of a back story, can subtly, or perhaps even crudely, affect our aesthetic response.

The greatest lesson for us all ought to be, however, that there are more fine young pianists out there than most of us realize. If it wasn’t Joyce Hatto, then who did perform those dazzlingly powerful Prokofiev sonatas? Having been so moved by hearing “her” Schubert on the radio, I’ve vowed to honor the real pianist by ordering the proper CD, as soon as I find out who it is. Backhanded credit to Joyce Hatto for having introduced us to some fine new talent.

When Schmich's essay was Vonnegut's it was famous, when it wasn't it vanished. The same problem afflicts research articles; many great contributions by the non-famous are published in obscure journals and only recognized in retrospect. The name of the author changes the perceived value. In day to day life many of us know the feeling of saying something that's ignored, only to hear it applauded when spoken by another.

Humans are very susceptible to framing effects. It's not fair, but it's not going to change.

Therapy for severe cognitive and behavioral disorders: A new era

Two stunning results may herald a new era in research, and even therapy, of disorders as diverse as Down's syndrome, schizophrenia, and autism:

• Be the Best You can Be: Reversing the cognitive dysfunction of Down's Syndrome
• Reversing a severe brain disorder in adult mice (murine autism)
  

A year ago I wrote about a deluge of research on autism genetics. Recent mouse models for autism and Down's syndrome are enabling radical new research directions. These models are the equivalents of the early telescopes -- radical new methods to investigate nature.

It's hard not to be carried away with this kind of discovery. We may run into the kind of dead-ends that stymied human gene therapy. If the results are confirmed, however, they will stand as Nobel-quality basic science discoveries.

Using canines to make fake fur: what's the true story?

When does a story outrage Americans, and when do they ignore it? When I first read that Chinese manufactured fake fur garments almost always (24/25 in one sample) contained canine fur, I figured the furies would follow. They really haven't however, and it turns out that the story is several months old, and more complex than it first seemed (though I would advise boycotting JC Penney).

After some reflection, I think this really belongs in the same globalization category as lead contamination of christmas lights. I'll explain why after the story. BTW, the racoon dog is a wild canine often trapped or farmed for its fur. Emphases mine.

Is Your Coat Fur Fake, or Is It Fido? | World Latest | Guardian Unlimited

... The Humane Society of the United States said it purchased coats from reputable outlets, such as upscale Nordstrom, with designer labels - Andrew Marc, Tommy Hilfiger, for example - and found them trimmed with fur from domestic dogs [jf: this is misleading, see below], even though the fur was advertised as fake.

The investigation began after the society got a tip from someone who bought a coat with trim labeled as faux fur that felt real. Leppert and her team began buying coats from popular retailers and then had the coats tested by mass spectrometry, which measures the mass and sequence of proteins.

Of the 25 coats tested, 24 were mislabeled or misadvertised, the society said.

Three coats ... contained fur from domesticated [jf: meaning breeds domesticated in the US] dogs. The others had fur from raccoon dogs ... Most of the fur came from China.

Importing domestic dog and cat fur was outlawed in 2000. Intentionally importing and selling dog fur is a federal crime punishable by a $10,000 fine for each violation.

... The discovery of domestic dog fur is the latest twist in the investigation that ensnared retail giants Macy's and J.C. Penney late last year. Both of those retailers were discovered selling coats with raccoon dog fur labeled as raccoon.

J.C. Penney initially removed the offending garments from its stores around Christmas - but eventually it had employees scratch out the 'raccoon' label with black magic marker and put the coats back on the shelves. Macy's immediately pulled the items from its shelves.

... Mislabeling fur is a misdemeanor punishable by a $5,000 fine or a year in prison. Fur valued at less than $150 is not required to be labeled.

A bill introduced by Reps. Jim Moran, D-Va., and Mike Ferguson, R-N.J., would close that loophole by requiring labels for all fur regardless of its value. It also would ban fur from raccoon dogs.

... Other retailers the Humane Society said sold mislabeled raccoon dog fur included Lord & Taylor, BergdorfGoodman.com and Neiman Marcus.com. Designers whose clothes were mismarked included Donna Karan's DKNY and Michael Kors. A coat from Oscar de la Renta advertised as raccoon had raccoon dog fur.

I think I can reconstruct the real story, based on what's written here and my crotchety knowledge of how the world works. I bet that US retailers have known for years that Chinese manufacturers were using a wild canine, mostly farmed and hideously abused, to make convincing "fake fur". As long as the US retailers had plausible deniability they didn't care, and there was no law against using racoon dog fur. The mislabeling is merely a misdemeanor offense if the garment costs more than $150, and no label is required for under $150. US retailers probably didn't know, however, that some Chinese manufacturers were using dog breeds that, in many nations, are pets. That's a felony, and that's why they're anxious. If an executive were found to do this knowingly they'd get a criminal record, but I doubt any are guilty of that.

The Humane Society is justifiably outraged about the maltreatment of the racoon dog, but they know most Americans don't care. They went looking for evidence of use of domesticated dogs and found it, which is why the story came out.

This kind of thing (like lead contaminating holiday lights) is inevitable in a global economy with very different ideas about what's acceptable behavior and how seriously to treat foreign regulations and sensibilities. So how should we respond?

Congress should definitely pass the Moran and Ferguson bills. Americans should, in the meantime, assume that all "fake fur" is really wild dog fur, and conduct themselves accordingly. I'd like to see a class action suit take down of the retailers -- since I bet they've intentionally ignored the racoon dog mislabeling for years. Surely a hungry lawyer can sue for the psychic damage inflicted upon unwitting consumers ...

Deriving mass/energy equivalence with a simple thought experiment

Anyone who's done any mathematics will remember how some complex problems become trivial when you change the coordinate systems. It's a stunning thing to see and feel. The right framing of a problem can make the solution self evident. We see a similar phenomenon in studies of cognition; a seemingly complex problem can become easy for most people to solve when its translated into a social relationship structure. In special relativity problems changing the 'frame of reference' can make a difficult question easy.

I'm fascinated by these kinds of transformation, though I lack the genius necessary to invent new transformations. I think this way of leading just about any thoughtful person to an understanding of the necessity of mass/energy equivalence is a kind of coordinate system transform:

Why Does E=mc2? | Cosmic Variance

...Now let’s think about a second thought experiment, which is closely related to the first. All I want to make different is to replace the cannon by a powerful laser. Instead of a cannonball being propelled across the box, we’ll now think about the laser firing a pulse of light. Now, the light carries momentum, and so when the laser fires and the pulse sets off, the box will once again begin a backwards slide in order that momentum be conserved. Also once again, when the light reaches the other side and is absorbed by the opposite wall, the momentum will be transferred back to the box, which will then come to a halt. But now you see the problem. The distribution of mass in the box is the same as it was at the beginning, and no external forces have acted on the system, and yet because the box has slid backwards and no mass has been moved, the center of mass of the entire system has moved! All no longer seems right with the world...

Do developing minds have growing pains?

Developing bodies have growing pains, why not developing minds?

Be the Best You can Be: Are cognitive improvements preceded by behavioral deterioration?

....We would then expect .... that major changes to critical cortical systems would be associated with substantial behavior and cognitive disruptions.

I'm sure it's been studied, perhaps the results were uninteresting (no correlation) ...

The Bayesian Heresy: Understanding Innovation

Scott Page, as quoted by Bayesian Heresy, asserts that a diverse team of good thinkers will be more creative than a less diverse team of very good thinkers:

The Bayesian Heresy: Understanding Innovation

...First, for any problem there exists a perspective that makes it easy to grasp a solution, though that may mean waiting for a person as unique as Edison to come along. Second, across all problems no perspective or no heuristic is any better than any other. In plain English, any approach may be just as good as any other until it is tested.

Third, teams of problem solvers—viewed as bundles of perspectives and heuristics brought together to solve a particular problem—do better when the diversity of perspectives and heuristics is greater than the overall ability or talent of the team’s members. In other words, diverse teams outperform teams composed of the very best individuals. Diversity trumps ability.

This last result requires further explanation. A team, a group, or even an entire society innovates through iterative application of perspectives and heuristics. Individuals who perform best obviously possess good perspectives and heuristics (think Edison), yet 30 Edisons each may have 20 useful heuristics while collectively possessing a mere 25. In contrast, the diverse team’s individual members may on average only know 15 heuristics apiece but collectively know 40.

I don't buy it. There are confounding variables such as communications latency, ability to cooperate, response to incentives, and solution collision (multiple semi-compatible good quality solutions that, when combined, produce a weak result). Diversity is valuable, but I've never seen it trump ability by itself.

The Dreams of Boltzmann Brains

Emphases mine.

OO’s and BB’s | Cosmic Variance

.... The idea Don put forward is this: there’s us, the ordinary observers (OO’s) in the world, who have achieved a certain stature after billions of years of evolution in the universe, and are now capable of making quite refined (or so we think) observations of the universe. ...

Then, though, there are the BB’s in the universe: Boltzmann Brains. Random fluctuations of the fabric of spacetime itself which, most of the time, are rather insignificant puffs which evaporate immediately. But sometimes they stick around. More rarely, they are complex. Sometimes (very very rarely) they are really quite as complex as us human types. (Actually, “very very rarely” does not quite convey just how rare we are talking now.) And sometimes these vacuum quantum fluctuations attain the status of actual observers in the world. But, the rarest of them all, the BB’s, are able to (however briefly) make actual observations in the universe which are, in fact, “not erroneous” as Don Page put it.

The man was a compelling speaker, and soon I realized there was an actual intellectual debate underway in the high end of the cosmology/high energy community as to what the role of these BB’s might be in the universe, in the very far (or maybe not so far) future...

The thing is, when you start talking about very very…very rare things like Boltzmann Brains, you are talking about REALLY long times. Much longer than we’ve had on earth (and I mean 4.5 billion years) by many orders of magnitude. Numbers like 10 to the 60th years were being batted around like it was next week in this talk. By those times, all the stars and all the galaxies have gone out, and gone cold, and space has continued to expand exponentially and things are long past looking pretty bleak for the OO’s still around, who (we presume) need heat and light and at least a little energy of some sort to survive, even if we are talking about very slow machine intelligence (even slower than humans for example).

So eventually, the mere fact that there is, at these long times, just oodles of space in the universe means that the BB’s become more and more common (even if they are rare) and eventually dominate the, uh, intellectual landscape of the universe...

...I am mangling this horribly, and of course before writing this I took just a glimpse at the already voluminous amount of literature on this topic, and realized that I have a lot of reading to do, both blog and academic. So it’s best I stop and let you all go look up Boltzmann Brains, as I will, and do some more reading...

There's a lot of time in a universe that's open (expanding). Time enough for spacetime to dream the lives of mortals ...