Dell reaches road's end

I recommended a computer to a friend made by a company called  "PCs Limited". They were built by a kid in Texas who’d started building them in school.

PCs Limited made great products. They copied the big guys, focused on quality, and eliminated retail channel costs. They got bigger, and even did some innovation of their own.

Later they decided their business was process improvement, and they could leave innovation to someone else. They became full time parasites. They fired all their engineers and their creative people. They were so successful as parasites, they killed their hosts.

Then came Foxconn and China, and the margin fell out of the low cost imitation market.

Dell has been dying for years, but they’re entering the end game…

In Suit Over Faulty Computers, Window to Dell’s Fall - NYTimes.com

After the math department at the University of Texas noticed some of its Dell computers failing, Dell examined the machines. The company came up with an unusual reason for the computers’ demise: the school had overtaxed the machines by making them perform difficult math calculations.

Dell, however, had actually sent the university, in Austin, desktop PCs riddled with faulty electrical components that were leaking chemicals and causing the malfunctions. Dell sold millions of these computers from 2003 to 2005 to major companies like Wal-Mart and Wells Fargo, institutions like the Mayo Clinic and small businesses.

“The funny thing was that every one of them went bad at the same time,” said Greg Barry, the president of PointSolve, a technology services company near Philadelphia that had bought dozens. “It’s unheard-of, but Dell didn’t seem to recognize this as a problem at the time.”

Documents recently unsealed in a three-year-old lawsuit against Dell show that the company’s employees were actually aware that the computers were likely to break. Still, the employees tried to play down the problem to customers and allowed customers to rely on trouble-prone machines, putting their businesses at risk. Even the firm defending Dell in the lawsuit was affected when Dell balked at fixing 1,000 suspect computers, according to e-mail messages revealed in the dispute….

… For the last seven years, the company has been plagued by serious problems, including misreading the desires of its customers, poor customer service, suspect product quality and improper accounting.

Dell has tried to put those problems behind it. In 2005, it announced it was taking a $300 million charge related, in part, to fixing and replacing the troubled computers. Dell set aside $100 million this month to handle a potential settlement with the Securities and Exchange Commission over a five-year-old investigation into its books, which will most likely result in federal accusations of fraud and misconduct against the company’s founder, Michael S. Dell.

The problems affecting the Dell computers stemmed from an industrywide encounter with bad capacitors produced by Asian PC component suppliers. Capacitors are found on computer motherboards, playing a crucial role in the flow of current across the hardware. They are not meant to pop and leak fluid, but that is exactly what was happening earlier this decade, causing computers made by Dell, Hewlett-Packard, Apple and others to break.

According to company memorandums and other documents recently unsealed in a civil case against Dell in Federal District Court in North Carolina, Dell appears to have suffered from the bad capacitors, made by a company called Nichicon, far more than its rivals. Internal documents show that Dell shipped at least 11.8 million computers from May 2003 to July 2005 that were at risk of failing because of the faulty components. These were Dell’s OptiPlex desktop computers — the company’s mainstream products sold to business and government customers.

A study by Dell found that OptiPlex computers affected by the bad capacitors were expected to cause problems up to 97 percent of the time over a three-year period, according to the lawsuit.

As complaints mounted, Dell hired a contractor to investigate the situation. According to a Dell filing in the lawsuit, which has not yet gone to trial, the contractor found that 10 times more computers were at risk of failing than Dell had estimated. Making problems worse, Dell replaced faulty motherboards with other faulty motherboards, according to the contractor’s findings.

But Dell employees went out of their way to conceal these problems. In one e-mail exchange between Dell customer support employees concerning computers at the Simpson Thacher & Bartlett law firm, a Dell worker states, “We need to avoid all language indicating the boards were bad or had ‘issues’ per our discussion this morning.”

In other documents about how to handle questions around the faulty OptiPlex systems, Dell salespeople were told, “Don’t bring this to customer’s attention proactively” and “Emphasize uncertainty.”

“They were fixing bad computers with bad computers and were misleading customers at the same time,” said Ira Winkler, a former computer analyst for the National Security Agency and a technology consultant. “They knew millions of computers would be out there causing inevitable damage and were not giving people an opportunity to fix that damage.”

Mr. Winkler served as the expert witness for Advanced Internet Technologies, which filed the lawsuit in 2007, saying that Dell had refused to take responsibility for 2,000 computers it sold A.I.T., an Internet services company. A.I.T. said that it had lost millions of dollars in business as a result. Clarence E. Briggs, the chief executive of A.I.T., declined to comment on the lawsuit….

… Dell’s supply chain had always stood out as one of its important assets. The company kept costs low by limiting its inventory and squeezing suppliers. If prices for components changed, Dell could react more quickly than its competitors, offering customers the latest parts at the lowest cost.

But the hundreds of Dell internal documents produced in the lawsuit show a company whose supply chain had collapsed as it failed to find working motherboards for its customers, including the firm representing Dell in the lawsuit, Alston & Bird….

Apple also had severe capacitor problems with iMacs, but they couldn’t hide them. Apple customers screamed. Apple fixed some iMacs, but many died just out of warranty. To my knowledge, however, the fixes worked.

I don’t think Apple’s strategy was very different from Dell’s, except Apple customers communicate loudly and are harder to ignore. What saved Apple was a much better customer relationship, and innovative products in the pipeline. Dell’s customers have been unhappy for many years, and Dell doesn’t do innovation. Dell didn’t have a cushion to protect them.

In an era where Foxconn builds everything, we’re likely to hear more about collapsing supply chains for years to come.

Incidentally Nichicon is dismantling its company in anticipation of the settlements ahead. It’s a Japanese corporation.

See also:

• The end of Moore’s Law and the future of Dell 
• Fake Steve Jobs- Why Dell will not bounce back (2008)
• Dell has jumped the shark- Spyware pre-installs (2006 – we knew they were dead then)
• Dell aphorisms – from 2004 when they were still “on top” – but I wrote “I think Dell makes fairly lousy products that cost too much. Their web services are poor and their customer support worse…”

Update: In the light of Dell’s ending, it’s interesting to read this leaked Apple memo on their iPhone antenna debacle …

1. Keep all of the positioning statements in the BN handy – your tone when delivering this information is important…

2. Do not perform warranty service. Use the positioning above for any customer questions or concerns…

4. … ONLY escalate if the issue exists when the phone is not held AND you cannot resolve it.

5. We ARE NOT appeasing customers with free bumpers – DON’T promise a free bumper to customers.

There’s not a great difference between Apple and Dell corporate culture. The real differences come with the obnoxious Apple customer base (folks like me), the media attention Apple gets, and the benefits Apple delivers along with their bungles.

Google: The Quick, the Sick and the Dead - 3rd edition

I wrote my first Google Quick, Sick and Dead list in January of 2009 and the second in November 2009.

It's interesting to look back a year and take note of what's gone (Notebook, Page Creator, Knol?) and what's improved (Documents).

It's been over six months since the last edition, so here's version 3. Note the choices are entirely my own. Nobody else would put Android in the "Sick" category. 


The Quick

• Search and Scholar (Scholar deserves more applause)
• Google Reader including Like and Share (brilliant work, even though Buzz tries to kill it)
• Google’s Data Liberation Front (my heroes!)
• Gmail (even though they'll never fix the !$$@ threading)
• Chrome browser
• Picasa and Picasa Web Albums (esp. with new pricing)
• Calendar (CalDAV support is slipping though)
• Maps / Earth
• News
• Translate
• Custom search engines
• YouTube
• Books (because they keep trying)
• Google Docs
• Google Apps

The Sick

• Google Profile
• Google Contacts: pathetic
• Google Mobile Sync: multi-calendar control is ridiculous
• Android – increasingly desperate need for curated apps and a Google controlled phone.
• Buzz – Brin needs to stay out of social
• Google Reader Comments and Notes - irrational confusion, though Reader itself is great
• Google Voice (iPhone web app is frozen in time)
• Chrome OS (feels stillborn)
• Google Video Chat
• Google Calendar CalDAV support
• Google Checkout
• Orkut
• iGoogle

The Walking Dead

• Blogger (because Google can't fix the #$!$!$ draft editor, there's still no mobile view, and the 5000 item limit goes unfixed)
• Google Groups
• Google Sites
• Google desktop (search)
• Google Base
• Gmail Tasks (forgotten, useless)
• Knol
• Google Wave
• Firefox/IE toolbars (killed by Chrome)
• Google Talk (neglected, Chat confusion)
• Google Parental Controls

Compared to a year ago Google has more sick and dying projects. I think the war with Apple is really hurting them. I'm quite sad about Blogger's pending demise.

Google should turn control of Google Profile and Buzz to the brilliant Google Reader team.


Update 6/29/10: I used Windows Live Writer to repair Blogger editor mis-formatting. On reflection I moved a few more items from “Sick” to “Walking Dead” and I realized I should include “Parental Controls” as a (walking dead) product. 


Update 8/11/10: Blogger has since improved to Sick from Walking Dead. I like this graphic of Google's past flops. Also, I hate, hate, hate Blogger's new editor.
--
My Google Reader Shared items (feed)

How could Krugman be wrong?

Paul Krugman, the Cassandra* of our era, is getting ready to delcare “The Third Depression”, a long interval of @ 10% unemployment and slow US economic growth. He says our only hope is ongoing stimulus and massive deficits over the next five years.

Alas, a coordinated stimulus isn't going to happen.  Bruce Bartlett gives one example why …

Debt Default: It Can Happen Here

… As of right now, outstanding debt subject to limit is a little over $13 trillion and the debt limit is $14.3 trillion. At the rate the Treasury is borrowing, it can continue for about 10 months before the debt limit must be raised again…

… to be sure, the debt limit has always been raised in time to prevent a default, although Treasury sometimes had to push the limits of the law to move money around to pay the government’s bills. However, I believe the game has changed …

… a growing number of conservatives have suggested that default on the debt wouldn’t be such a bad thing. It is often said that default would lead to an instantaneous balanced budget because no one would lend to the U.S. government ever again. Therefore, spending would have to be cut to the level of current revenues. Writing in Forbes last month, the Cato Institute’s John Tamny was enthusiastic about the prospects of default…

… Tamny is not an isolated crackpot; reputable conservative economists have been writing sympathetically about the idea of default for decades...

Other prominent conservatives who have been favorable, even enthusiastic, about debt default include Murray Rothbard, Dan Pilla, Jeffrey Rogers Hummel, and Christopher Whalen. In 1995, then House speaker Newt Gingrich publicly warned the Public Securities Association that he was prepared to default on the debt unless Bill Clinton acceded to Republican demands for budget cuts. “I don’t care what the price is,” Gingrich said.

..Last year, The Economist’s Greg Ip wrote an article in the Washington Post saying that financial markets were placing the risk of default at 6 percent over the next 10 years. “Default is unlikely,” he said. “But it is no longer unthinkable.”…

So even if Krugman is right, the American people don’t have the wisdom to apply his recommendations.

Is there some way, however, that Krugman could be wrong?

It seems unlikely, his record is very good. Still, it is a bit peculiar that the Euro crisis seems to have arisen from somewhat different economic roots than the American crisis. Yes, Krugman and others argue that the economic fundamentals are identical, but I think I see some unspoken anxieties in their conclusions.

So if these crises aren’t really of the same root causes, why should they occur together?

Perhaps we’re simply seeing an economic form of the sort of synchronization that may cause earthquakes to occur close in time. Or perhaps there are root causes beyond traditional economic modeling.

That’s one way in which Krugman could be wrong. He’s unlikely to be wrong within the framework of economics, but what if our economies have drifted into turbulent domains where linear economic models no longer apply? A whitewater world, in other words.

What if the root cause of our crises can't be found in traditional economic models, but in deeper causes such as the technological discontinuity of networked computers and the historic discontinuity of a resurgent China and India? What if these two shocks are stressing every aspect of the world's economic infrastructure?

If we accepted those twin revolutions as root causes, then we might imagine they would manifest differently in different economic settings. In each case the "machine" would come apart at its weakest point. In the US that would be our insanely complex Finance operations, in Europe it would be the strains of a single currency for Greece and Germany, and in China it will be ...

If we take these as the root causes, then traditional economic stimulus might get the machine going again -- but it might blow up in a different direction. The focus would need to be on managing and ameliorating disruptions, while perhaps sprinkling a bit of sand in the gears of an overheating engine. Managing China's bubble economy, including currency value changes, would become a great US priority.

It's not a great change from what Krugman recommends, save for less focus on spending and more on management for the long haul - including stripping benefits (health care, etc) from employment. There's also in this model more room for surprises -- meaning we may do better than another Long Depression. Or we might do worse ...

* Cassandra as in always depressing, always right, always ignored.

Update 6/29/10: I really don’t think Paul K and Kevin Warsh are reading Gordon’s Notes, but I do think I’m tuning a meme.

From PK’s blog post this am we read Warsh saying “unanticipated, nonlinear events can happen” and Krugman responding

So it’s these “unanticipated, nonlinear events” that are “more certain” than the direct effects of fiscal policy?

Of course that’s only a bit of the blog post, which mostly consists of Krugman reframing a few of Warsh’s statements and ripping them up (he’s very good at that, maybe too good).

Still, the “nonlinear events” bit is interesting. Krugman’s rebuttal is also interesting. He doesn’t actually say it’s rubbish to contemplate nonlinear events that might move us outside of current economic modeling, instead he says they’re relatively speculative compared to the direct effects of fiscal policy.

I think Krugman is wrestling with the possibility that we’re outside model bounds.

Incidentally, I should be clear (fwiw) that I’m not joining up with the austerity group. I’m in the squishy middle ground. I’d be more of a full Keynesian if I believed humans were capable of running surpluses in good times.

Paul Dirac on the surprising history of the Schrodinger wave equation

SciAm is republishing a terrific 1963 essay by a famed physicist, back when Scientific American graphics included serious math. Here Dirac is talking about the surprising history of the Schrodinger wave equation ...

Paul Dirac, Scientific American, 1963: The Evolution of the Physicist's Picture of Nature

... I might tell you the story I heard from Schrodinger of how, when he first got the idea for this equation, he immediately applied it to the behavior of the electron in the hydrogen atom, and then he got results that did not agree with experiment. The disagreement arose because at that time it was not known that the electron has a spin. That, of course, was a great disappointment to Schrodinger, and it caused him to abandon the work for some months. Then he noticed that if he applied the theory in a more approximate way, not taking into account the refinements required by relativity, to this rough approximation his work was in agreement with observation. He published his first paper with only this rough approximation, and in that way Schrodinger's wave equation was presented to the world. Afterward, of course, when people found out how to take into account correctly the spin of the electron, the discrepancy between the results of applying Schrodinger's relativistic equation and the experiments was completely cleared up...

This would be a good update to the history section of the Wikipedia article on the equation; it's also a lovely example for people writing about on the philosophy of science. It's all so much neater in retrospect.

Those were the glory days of Scientific American. I love the Sci Am news blog, but the magazine is very slight now. It aims for a different market.

There's so much more in this brief essay. It's required reading for the physics fanboy. Consider Dirac's discussion of "137" (Adams should have used 137, not 42):

.... There are some fundamental constants in nature: the charge on the electron (designated e), Planck's constant divided by 2 π (designated h-bar) and the velocity of light (c). From these fundamental constants one can construct a number that has no dimensions: the number h-bar*c/e^2. That number is found by experiment to have the value 137, or something very close to 137. Now, there is no known reason why it should have this value rather than some other number. Various people have put forward ideas about it, but there is no accepted theory. Still, one can be fairly sure that someday physicists will solve the problem and explain why the number has this value. There will be a physics in the future that works when h-bar*c/e^2 has the value 137 and that will not work when it has any other value...

Dirac gets numerological about 137. He really doesn't like square roots ...

.... Only two of them can be fundamental, and the third must be derived from those two. It is almost certain that c will be one of the two fundamental ones.... If h-bar is fundamental, e will have to be explained in some way in terms of the square root of h-bar, and it seems most unlikely that any fundamental theory can give e in terms of a square root, since square roots do not occur in basic equations. It is much more likely that e will be the fundamental quantity and that h-bar will be explained in terms of c^2. Then there will be no square root in the basic equations...

Genius is sometimes close to dysfunction [1]; it's common in science that elder genius takes odd directions. This stuff is great fodder for crackpots, but there's always the tantalizing possibility that some of it will be borne out one day.

[1] I've lost a recent developmental neurobiology reference that put the old "genius is close to madness" cliche on firm ground. It wasn't this review article, but it was of the same genre. The same mechanisms that make a person creative do seem to make them more prone to "strange loops", perhaps particularly as they age. (Yeah, I'm being self-referential.)

China's water army and my e-cigarette post

A post I wrote on unregulated (Chinese, but really, everything is) electronic cigarettes received a lot of curiously phrased comments. I wonder if that was my first encounter with the water army (wumaodang variant)...

Blood & Treasure: water army

... Another wrinkle on the wumaodang phenomenon. Instead of the Chinese state hiring people to post favourable comments about government policy, you have shuijun, the “water army” paid by private companies to delete unfavourable comments about themselves and their products and services.

Related shakedowns include commercial websites spamming negative comments about companies in order to pressurize them into advertising on their site, and companies suing websites carrying genuine complaints on the grounds that they are faked, but basically to get them to censor themselves.

In a similar vein see Internet 2025 (very new and a classic already) and Bruce Sterling's (free) short story about post-apocalyptic astroturf - The Exterminator's Want-Ad.

China is the new California*.

* Land of the future.

Churches of the Singularity - is Horgan truly an unbeliever?

Once upon a time if you put two phone handsets together, speaker to microphone, you get a massive burst of positive feedback. Each handset amplified the other’s signal. This acoustic eruption was a form of acoustic singularity.

You have to be of a certain age to remember that. Modern systems cut the feedback.

The ”Technologica Singularity” is positive feedback in the realm of technological innovation. New tools makes new tool creation faster and easier. Data storage units that costs millions and filled a room cost 50 cents to make and are the size of a small fingernail. Technology that makes things smart makes things smarter.

It’s the “smart things” bit that’s the problem. We know how to make human level intelligence; we make billions of ‘em. There’s obviously no chemistry barrier to creating artificial intelligences at least as smart as the smartest possible human. These entities will then will create their smarter descendants and so on. (Unless they are smart enough to know that’s a very, very, bad idea. They will then kill us so we don’t make any more of them.)

The “explosive intelligence” form of the technological singularity dates back to about 1960, but it’s been a feature of science fiction since the 1980s. It’s most often associated with the seriously fun writings of Vernor Vinge, but many of my favorite writers have taken a whack at it. Over the past 10 years it’s become an obsession for geeks who, through great leaps of hopeful imagination, have come to believe that “the rapture of the nerds” will make them immortal.

Now we have the Churches of the Singularity. The Vingeans (sorry Vernor) believe we can’t avoid Strong AI and that this may be a good think or a very bad thing. (Bad for humans and dogs anyway, whales might be delighted.) Vingeans usually put the date for Strong AI between 2040 and 2200.

The Kurzweilians and Moravians believe that humans will integrate with Strong AIs, and that this will occur soon enough that they will, personally, become immortal. Their date predictions are always within their personal life expectancies.

The Heretics suspect that the Programmer and the Player have excluded Strong AI from the Program. The Denialists believe that something will prevent Strong AI, but they rarely say what. They have an uneasy relationship with the Heretics.

I have mostly of the Vingean faith, but I’m tempted to Heresy.

It’s pretty easy to put most people in one of these churches. Sometimes, however, one of the Faithful will deny their roots. For example, Scientific American’s John Horgan considers himself an unbeliever. In a recent critique of the Faith he wrote

Cross-check by John Horgan: Singularity Schtick: Hi-tech moguls and The New York Times may buy it, but you shouldn't

… The New York Times Sunday business section recently ran an enormous puff piece on Ray Kurzweil and the "Singularity" cult (my term, not the Times's)…

… Believers squabble over how exactly the Singularity will go down. Will we just genetically soup ourselves up? Become human–machine cyborgs? Totally synthetic robots? Digitize our psyches and download them into cyberspace? All the predictions entail superintelligence and immortality…

… Bill Gates has blurbed Kurzweil's books. Other admirers include Peter Thiel, a co-founder of PayPal, and Peter Diamandis, who heads the X PRIZE Foundation, which promotes space travel. … Sergey Brin and Larry Page, co-founders of Google, helped Kurzweil establish a "Singularity University" at NASA Ames Research Center in California…

…When I debated Kurzweil at the 2008 Singularity Summit, a revival meeting for the faithful, he seemed all too sincere. But his Singularity schtick is so out of sync with reality that I'm beginning to wonder if even he takes it seriously…

Methings he doth protest too much. Horgan is conflating the Kurzweilians with the greater Church. I am certain he knows better. He never claims Strong AI is not possible, he only says it’s unlikely to happen soon. He is clearly a Vingean in denial. Welcome to the Faith John.

See also:

• IEEE - The Singularity Issue (2008, includes Vinge essay, see also Why talk about The Singularity?)
• Aaronson critiques Kurzweil and the 2045 Singularity
• Kurzweil and homeopathy- now crackpot certified
• The Singularity University
• Reducing long-term catastrophic risks from artificial intelligence - Singularity Institute - sometimes Kurzweilians have a few interesting things to say)
• Imagining the Singularity in 1965…
• Vinge and Kurzweil on the Singularity
• Signs of the singularity: science fiction gives up
• The singularity and why you should be very nice to your children
• Twenty-three years to Google's singularity? (joking I hope)
• The Economist predicts an early Singularity through neuroengineering (Dumb)

and more about the strong AI answer to the Fermi Paradox (Church of Vinge, Fermi schism)

• Singular fun with Fermi
• Charles Stross and the Fermi Paradox
• SETI, the Fermi Paradox and The Singularity: Why our search for extraterrestial intelligence has failed
• Singularity Sky: Brad DeLong's Webjournal (Fermi's Paradox revisited)
• Nick Bostrom Where Are They- Why I hope that the search for extraterrestrial life finds nothing (The Great Filter, pdf)
• Are You Living in a Computer Simulation? (Bostrum again)
• Reality, perception, Hume, the red pill - and John Tierney (a bit off-topic honestly)

The bicyclist’s panopticon

Sometime in the next ten years, there will be a thumb sized bell on the left side of my rear bicycle rack – and perhaps another on the right side.

It will contain a battery, a GPS, a clock, a gyroscope and a proximity sensor, 1TB of memory, 1 button and 3-4 lenses. It will cost $100.

As I ride it will take pictures to the front, rear and left sides. It will constantly thin out older images to make room for newer ones. Images will be digitally signed and will incorporate location and time stamps.

If an object passes within 3 feet an incident marker will be set and neighboring images will be preserved. If the gyroscope detects abrupt acceleration or if the button is pushed incident markers will be set.

Images will be admissible as evidence. At the end of day’s commute a bicyclist may choose to submit incidence reports with annotations. Depending on the circumstance a driver may receive a warning, a ticket, or a summons.

The bicyclist’s panopticon will make the streets safer, and encourage some aging boomers to use alternative transportation (my generation).

Hat tip: David Brin, of course, and Jon Udell on defensive surveillance for cyclists.

Update 8/15/2010: The Cerevellum cycle computer includes a rear view digital camera and accident detection/recording capability. The owner is taking pre-orders.

I think this is one of my safer predictions.

Update 9/16/10: The Lookcie. Did I say ten years? I meant 3 years.

Update 2/2/11: Increasing use of bike cameras in the UK

Beyond the first cause: Deepwater Horizon and the publicly traded company

Airplane crashes, the Great Recession, amputating the wrong limb, and the Deepwater Horizon eruption all have one thing in common -- layers of causation. To prevent recurrences you need to walk the causal chain. If you stop at the first cause, such as "Dr. Gordon was up to late and confused right with left", you often get nowhere - or worse.

Consider a possible causal chain in our latest whitewater times disaster. Let's assume, for the purposes of discussion, that we have the technology to cost-effectively extract Gulf oil with a "reasonable" risk of "very unwanted outcome". That presumably means a low probability of catastropic blow out and a range of effective response strategies.

Let's further assume that BP made some seriously bad decisions to save a relatively small amount of money. There's some evidence now to suggest that, even by the standards of an extraction industry, BP was unusually negligent.

The next step is to ask if this behavior was an aberration or if it was typical. If it was typical of BP, but less common at (say) Exxon (yes, hard to believe), then we should ask why. Joe Nocera suggests answers to both of these questions (emphases mine) ...

In 2 Accidents, BP Ignored Omens of Disaster - Joe Nocera - NYT

We have to get the priorities right,” the chief executive of BP said. “And Job 1 is to get to these things that have happened, get them fixed and get them sorted out. We don’t just sort them out on the surface, we get them fixed deeply.”

The executive was speaking to Matthew L. Wald of The New York Times, vowing to recommit his company to a culture of safety. The oil giant was adding $1 billion to the $6 billion it had already set aside to improve safety, the executive told Mr. Wald. It was setting up a safety advisory panel to make recommendations on how the company could improve. It was bringing in a new man to head its American operations — the source of most of the company’s problems — who would make safety his top priority...

... the interview took place nearly four years ago, after BP’s previous disaster on American soil, when oil was discovered leaking from a 16-mile stretch of corroded BP pipeline in Prudhoe Bay in Alaska. And that was just a year after a BP refinery explosion in Texas City, Tex., killed 15 workers and injured hundreds more.

Nor was the chief executive in question Tony Hayward ... the interviewee was his predecessor and mentor John Browne, who had spent nearly 10 years at the helm of BP before resigning in May 2007.

Do you remember the Prudhoe Bay leak and the Texas City explosion? They were big news at the time, though they quickly faded from the headlines. BP was fined $21 million for the numerous violations that contributed to the Texas City explosion, and it was forced to endure a phased shutdown of its Alaska operations while it repaired the corroded pipeline, which cost it additional revenue.

In retrospect, though, the two accidents represented something else as well: they were a huge gift to the company. The fact that these two accidents — thousands of miles apart, and involving very different parts of BP — took place within a year showed that something was systemically wrong with BP’s culture. Mr. Browne had built BP by taking over other oil companies, like Amoco in 1998, and then ruthlessly cutting costs, often firing the acquired company’s most experienced engineers...

... On Thursday, during his day before an angry House energy subcommittee, Mr. Hayward was confronted with the fact that BP had been cited by the Occupational Safety and Health Administration for 760 “egregious willful” safety violations in its refineries. Mr. Hayward tried to slough this off by claiming that the violations had taken place in 2005 and 2006 — before, that is, he became chief executive and brought his “laser focus” on safety.

But Mr. Hayward was not telling the truth. According to the Center for Public Integrity, which obtained the data under the Freedom of Information Act, the violations all took place between 2007 and 2010, very much on Mr. Hayward’s watch. What’s more, the company violated something called O.S.H.A.’s “process safety management standard” — which is precisely what that BP advisory panel had been charged with examining after the Texas City explosion. In October 2009, O.S.H.A. fined BP an additional $87 million for refinery deficiencies. It doesn’t sound like the company took its advisory panel’s recommendations very seriously, does it?

Or take the Deepwater Horizon disaster itself, which was preceded by so many instances of corner-cutting and poor decision-making that an accident was practically preordained. Drilling one of the deepest wells in history, the project used only one strand of steel casing, when it should have used at least two. Halliburton recommended that BP use 21 “centralizers,” which help ensure that the well doesn’t veer off course as it goes deeper into the earth, but the company used just a half-dozen. BP failed to conduct a crucial test to make sure that the cement holding the well at the bottom of the sea was sturdy enough.

And the engineers for BP on board consistently ran roughshod over subcontractors like Halliburton, who openly worried that BP was making decisions that could have catastrophic consequences. “This is how it’s going to be,” one BP engineer reportedly said, overruling a contractor on the critical question of when to replace the drilling mud — which keeps explosive natural gas from flowing out of the well — with seawater....

... Changing a company’s culture is always hard, no doubt about it. And it is especially hard for a company like BP, which has had such enormous success these last few years, reaping $14 billion in profits last year alone. For such companies, it often requires a crisis to change. Microsoft changed after its antitrust trial a decade ago. Tyco International changed after its chief executive, Dennis Kozlowski, went to jail. And chances are, BP is now going to change, too.

This time, the world’s attention will not quickly fade, as it did after the Prudhoe Bay spill. The financial hit, which several Wall Street analysts believe could top $100 billion, is going to be severe. The pressure from the United States government will be unrelenting.

If you read beyond the lede, it's obvious Nocera is furious. Oddly the fact that Hayward lied to Congress hasn't gotten much attention elsewhere. Please note the petty fines BP received, pocket change against $14 billion in profits. As is true in many industries economic transformation has made traditional fines irrelevant. We need to start fining companies as a percentage of revenues and profits. A $4 billion fine in 2008 would probably have prevented the Deepwater Horizon disaster.

So we've identified one deeper cause -- a core function of our judicial system, financial penalization, is deeply inadequate.

Moving further along the causal chain, we find a company with a dysfunctional corporate culture, a culture that made very poor risk judgments. The answer to that culture is to remove the board, remove the CEO, and remove most of middle and upper management. That may be most effectively done by dismantling BP.

Moving even deeper, we may ask how BP got such a pathologic culture. We should pay attention to how BP was built and how those acquisitions became profitable ...

... Mr. Browne had built BP by taking over other oil companies, like Amoco in 1998, and then ruthlessly cutting costs, often firing the acquired company’s most experienced engineers...

We've seen similar acquisition patterns in the finance industry. In the short term safety measures are overhead. The quickest way to revenue is to strip them out. Similarly experienced engineers are very expensive, the quickest route to profitability post-acquisition is to eliminate them. It will typically take years for the knowledge impact to be felt.

Deepwater Horizon is a great disaster, but also a great opportunity. If we support Obama's approach, and the investigative panel he's assembling, there's a change people with real power will come to useful conclusions with implications well beyond oil extraction.

Deepwater Horizon spill and the gulf

If the gulf were my bathtub, how much oil would I add to equal the BP spill of 2010?

Let's say BP spills 5 million liters of oil. Since a cubic meter is 1000 liters, this is 5000 cubic meters.

That is a lot, but the Gulf is very large:

A square mile is 2.6*10^6 square meters.

So the gulf is about about 1.6*10^11 square meters by 2000 meters or about 3.2*10^14 cubic meters.

So the ratio of oil volume to gulf volume is about 1/31,720,000.

If the my bathtub holds 20 liters, I would need to add about 1/1586 of a milliliter of oil to resemble the total Deepwater Horizon spill. I doubt I'd notice it.

Of course the local effects of the oil are very large. I'm also pretty sure I've made some arithmetic errors (it's quite late and, alas, I rarely do even arithmetic these days). Still, the gulf is really vast.

Devalue the Yuan or I shoot my economy

If China doesn't devalue the Yuan, Europe will shoot its economy ...

BBC News - Obama warns G20 leaders on budget cuts

... The governments of several large European countries, Germany and the United Kingdom among them, have recently outlined plans for spending cuts.

In comments apparently directed at China, Mr Obama also stressed the need for flexible exchange rates to ensure a balanced global economy.

If Europe shoots its economy, China will lose a big export market. So it's a credible threat, even if insane.

AT&T Contracts, Terms of Service and Plan Terms – observations of interest

I visited my local AT&T store today to confirm that AT&T iPhone pre-ordering is done. They’ll have some phones in store 6/24 for those keen to wait in line and be disappointed. You can still pre-order via Apple, but I’ll be on the road 7/14 so I’ll wait until late July to get my iPhone 4.

I took advantage of my AT&T visit to ask some staff questions. One of the odder things about AT&T is that, despite being evil even by publicly traded company standards, their staff are remarkably personable and fairly knowledgeable.

I had 4 questions. The last two are interesting.

1. Why did my bill jump $30?
   Answer: I added unlimited family texting at the start of a billing cycle, so “pay ahead” meant I paid for most of 2 months. Most mobile phone services are added with a 1 month pay ahead.
2. If I buy an iPhone 4 for my son, can I give my wife the iPhone 4 and give him her 3GS?
   Answer: They’re fine with that switch. He’ll get a new contract of course.
3. Can you validate that as of Nov 2009 a contract-free non-iPhone “smartphone” connected to AT&T network will trigger data plan enrollment?
   Answer: No – they disagreed. A non-contract smartphone added to the network will not trigger an automatic data plan enrollment [1]. The Nov 2009 contract change only brought non-iPhone smartphones in line with iPhone policy. If you buy one under contract your “service agreement” will mandate one of the new data plans ($15, $25, etc). This directly contradicts what I was told over the phone, but see the next question.
4. I’d like to see my current contract, terms of service etc and the new one I’ll get with my new phone.
   Answer: Duck, dodge, weave, cough, deer in the headlights, “it’s on the web site” ….

Asking to see a contract brought the manager over in a hurry. He acted like I’d asked to see his porn collection.

After a bit of work I ended up with an outdated (7/31/2009) Terms of Service document and a generic “Service Agreement” that seems to be a current “Terms of Service” document. He couldn’t come up with anything reflecting the 11/09 policy changes.

Here’s what I ended up thinking:

1. For AT&T a “contract” is an agreement to pay them money for services for a fixed duration. A multi-phone family plan has a single contract.
2. Customer obligations and fees are some kind of sum of 2-3 different agreements. Some are tied to a phone number, some to a “class” of subsidized phone, and some to the iPhone alone.
   • Terms of Service: Lays out the basic terms on how you can use the service. I think it’s determined by “class of phone” such as “dumb phone” and “smartphone”.
   • Service Agreement: Seems identical to the “Terms of Service” booklet – same headings and language. Google found the current copy of this Service Agreement.
   • Plan terms: At one time the iPhone had different Plan terms from other smartphones, but now I think they may be aligned. I think Plan terms are more or less under user control; that’s what you get when you add or remove things from the web site.
   • iPhone other: “If buying an iPhone, you agree that use of the iPhone acts as an acceptance of the Apple and third party terms and conditions included with the iPhone.”
3. The AT&T staff really do have trouble verbalizing how this all works. I used to think AT&T was treating their service agreements as corporate secrets, but they really are public. I now think it’s more a complexity problem -- especially due to the now passing iPhone exemptions (Note you still can’t unlock an iPhone when a contract ends [2] – it is the only AT&T phone that cannot ever be unlocked.) There are a lot of moving and changing parts that sum to the net user and phone specific agreement.
4. The confusion about what happens when you put “Standard” phone GSM in a “smartphone” is understandable. I don’t think either of the AT&T reps I spoke with really know what will happen. I tend to think this crew is more correct, but I wouldn’t bet on it.

There are a few noteworthy clauses in AT&T’s current generic service agreement:

Your Service Commitment begins on the day we activate your service. You have received certain benefits from us in exchange for any Service Commitment greater than one month. … If your Service Commitment includes the purchase of certain specified Equipment on or after June 1, 2010, the Early Termination Fee will be $325 minus $10 for each full month of your Service Commitment that you complete. (For a complete list of the specified Equipment, check www.att.com/equipmentETF). Otherwise, your Early Termination Fee will be $150 minus $4 for each full month of your Service Commitment that you complete...

The enhanced ETF is not iPhone specific, it also applies to the iPAQ Glisten (lord, that would burn!) and the Samsung “Jack”. Imagine the horror of paying that fee on an iPAQ Glisten.

I didn’t realize my non-AT&T roaming was potentially limited ….

… If your minutes of use (including unlimited services) on other carrier networks ("off-net usage") during any two consecutive months exceed your off-net usage allowance, AT&T may, at its option, terminate your service, deny your continued use of other carriers' coverage or change your plan to one imposing usage charges for off-net usage. Your off-net usage allowance is equal to the lesser of 750 minutes or 40% of the Anytime Minutes included with your plan. AT&T will provide notice that it intends to take any of the above actions, and you may terminate the agreement…

and I bet this clause means AT&T can cut off my Google Voice calls to Canada at any time [3] …

… Unlimited voice services are provided solely for live dialog between two individuals. Unlimited voice services may not be used for conference calling, call forwarding, monitoring services, data transmissions, transmission of broadcasts, transmission of recorded material, or other connections which do not consist of uninterrupted live dialog between two individuals. If AT&T finds that you are using an unlimited voice service offering for other than live dialog between two individuals, AT&T may, at its option terminate your service or change your plan to one with no unlimited usage components. AT&T will provide notice that it intends to take any of the above actions, and you may terminate the agreement.

Of course several phones do support conference calling, so they can’t enforce this one too often. They also have to give notice.

The bottom line is that mobile phone contracting is hideously complex. Apple tried to change that with iPhone 1.0, but they failed (I give them full credit for trying). I’d love to see an ambitious Attorney General take this topic on.

See also:

• Gordon's scale of corporate evil (AT&T is up there)
• John's head explodes: AT&T rebate paid with an AT&T debit card
• Head still exploding: The AT&T mobile phone rebate card scam
• Google good, AT&T and old FCC evil
• Mobile phone fraud - The accidental data charge and other scams (includes 10 mobile phone scams)
• AT&T is a partner to phone scams that target the vulnerable elderly
• AT&T sends more SMS Spam, locusts infest exec underwear
• Annals of idiocy - AT&T spams customers about a TV show
• A deal with the Devil: We move from Sprint to AT&T and towards an iPhone (I didn't realize HOW devilish!)

[1] Of course if data services are used it will incur massive charges, but that’s another matter. My son’s number is enrolled in a $5/month AT&T service that allows me to disable his AT&T network data access. There are also relatively painless hacks that do the same thing.

[2] I’d like to see the contract/agreement where this is documented. I wonder if that’s agreed to as part of the iPhone agreement: “If buying an iPhone, you agree that use of the iPhone acts as an acceptance of the Apple and third party terms and conditions included with the iPhone.”

[3] Actually, they can’t. With the original Google Voice service I called a number, which then rang my mother’s number in Montreal. Now I use their (crummy) iPhone web app, and they call me first. So I think the current arrangements skirts the current contract. Also, does this mean a Robocall to my mobile phone is a violation of my contract?

Update 7/25/10: Astoundingly, when I picked up my new iPhone I asked the question again -- and was told my son would definitely be enrolled in a data plan even though his phone is not under contract and has no subsidy and he'd be using a contract-free iPhone. In other words, exactly the opposite of what I was told a few weeks ago. I have a post pending with contract excerpts. I'll be filing a complaint with the MN state attorney general and with the offices of our federal Senators.

Barack Obama – better appreciate him while you can

The inestimable Gail Collins says it best (emphases mine):

…. We are frustrated, too, and it’s possible that Obama may never be able to give the speech that will make us feel better. He may never really lace into the oil companies or issue the kind of call to arms on energy that the environmentalists are yearning for.

That’s because it won’t get him anywhere. Unlike Bush, he has no national consensus to build upon. He’d barely finished his muted remarks on Tuesday before the House minority leader, John Boehner, accused him of exploiting the crisis “to impose a job-killing national energy tax** on struggling families and small business.” Michael Steele, the Republican Party chairman, claimed that the president was “manipulating this tragic national crisis for selfish political gain.” And the ever-popular Representative Michele Bachmann denounced the BP restitution fund as “redistribution of wealth” and “one more gateway for government control.”

As a political leader, Barack Obama seems to know what he’s doing. His unsatisfying call for a new energy policy sounded very much like the rhetoric on health care reform that used to drive Democrats nuts: open to all ideas, can’t afford inaction, if we can put a man on the moon. ... But at the end of that health care slog, he wound up with the groundbreaking law that had eluded his predecessors for decades. The process of wringing it out of Congress was so slow and oblique that even when it was over it was hard to appreciate what he’d won. But win he did.

Ironic. The man we elected because we hoped his feel-good campaign speeches might translate into achievement is actually a guy who is going to achieve, even if his presidential speeches leave us feeling blah.

We live in whitewater times (see Stross - 2008). There’s going to be crisis after crisis for years and decades to come. We can’t freak out every time we have a historic ecological, economic, social, climatic or geological catastrophe*. We need to deal with them, learn from them, try to prevent each class of catastrophe from recurring too soon.

America is entering the rapids in a GOP-vandalized kayak, and much of the nation has tuned out. There’s too much bad news, too much uncertainty. Some are so far gone they’re signing on to whacko political movements. As Collins writes, Obama “has no national consensus to build upon”. He has to find what’s possible.

Today Obama has extracted $20 billion of possible, a Ninja move that has left the GOP stunned and gasping. That counts for a lot more than a speech. Meanwhile the prospect of BP’s annihilation has concentrated minds in in the oil industry more than any amount of words and easily manipulated laws (not that we don’t need the rules and regs, but financial collapse impresses capitalists more).

We’ll never get another President this good. Better appreciate him while you can.

Oh, and the guy needs a vacation. We need to tell him to take a good one.

* Incidentally, I despised Bush and wanted him fired – but not for Katrina. There he showed only average incompetence.

** This is “cap and trade” aka “carbon tax in drag”. The GOP does well in southern states, maybe they figure it’s the temperature that helps.

Email domains in the non-geek world

There are 14 non-employer email addresses in my son's baseball team roster:

• AOL: 2
• Yahoo: 4
• Comcast: 4
• Hotmail/MSN: 3
• Gmail: 1 (me)

Not the domains geeks tend to think about!

Strongest sign of economic recovery

The mass flow "investor" is buying gold ... Worried About Their Dollars, More Are Turning to Gold - NYTimes.com.

Forget every other indicator. This one rules. The recovery is on.

Why dog haters should love prosecution of animal cruelty

Unsurprisingly, people who abuse animals are also dangerous to humans ...

The Animal-Cruelty Syndrome - NYTimes.com:

... significant reason for the increased attention to animal cruelty is a mounting body of evidence about the link between such acts and serious crimes of more narrowly human concern, including illegal firearms possession, drug trafficking, gambling, spousal and child abuse, rape and homicide...

... In his famous series of 1751 engravings, “The Four Stages of Cruelty,” William Hogarth traced the life path of the fictional Tom Nero: Stage 1 depicts Tom as a boy, torturing a dog; Stage 4 shows Tom’s body, fresh from the gallows where he was hanged for murder, being dissected in an anatomical theater. And animal cruelty has long been recognized as a signature pathology of the most serious violent offenders. As a boy, Jeffrey Dahmer impaled the heads of cats and dogs on sticks; Theodore Bundy, implicated in the murders of some three dozen people, told of watching his grandfather torture animals; David Berkowitz, the “Son of Sam,” poisoned his mother’s parakeet....

I am surprised this hasn't gotten more formal research attention attention in the past. Perhaps scholars assumed it was self-evident? Formal investigations are now confirming long held beliefs. That's good research -- not all long held beliefs are empirically supported.

Not everyone loves dogs and cats. Maybe they have something against brood parasites. Even so, these dog-dislikers have good reason to favor aggressive investigation and prosecution of animal cruelty. My dog is their canary.