Outsourcing and quality control: the Apple experience

MacInTouch Home Page

A particularly appalling story of Apple's repair program. For various reasons, I believe this story is true:

I have a customer who had a 12.1' iBook G3/600 which exhibited the classic logic board exchange out of warranty issue, which Apple has a special extension for. This means the unit displayed flake video and sometimes would black out entirely.

This customer is meticulous and his equipment shows it. The unit was immaculate when it got to me even though it had been purchased second-hand and was in use by a kid.

I spoke to Apple and they agreed that it was within the scope of the extended warranty repair program, even though the serial number wasn't within the range published.

We did the DHL ship to the repair depot, and a week later the machine was returned to me. It had a very noticeable scratch on the bezel surrounding the LCD, and the Airport didn't work. I called Apple again and we did ANOTHER round trip to the depot. They made a note about the scratch, but it wasn't so bad, so I figured we could live with it.

Upon the second return (13 days, but Christmas intervened, so understandable), the unit worked OK, but had a very noticeable misalignment of the case work and ANOTHER scratch in the front lower case (right where you would press the tabs to release the bottom case).

I contacted Apple again, and they offered to replace the plastics and repair the unit if I shipped it to them a third time. I told them it had taken too long already, and could they compensate me some other way and I would pay to have the case put together properly in town... The guy's name was Dean and was super nice and agreed to send me out a new battery, which this unit needs anyhow.

So, when I go to straighten out the case work, as soon as the bottom is removed, it becomes clear there are multiple screws and parts MISSING. The LED which signals sleep wasn't even reinstalled, and at lease one important screw is AWOL.

Whoever is doing the repairs for Apple on this program is doing a terrible, sloppy job and there is clearly NO quality control in place for this.

As an Apple stock holder this is really appalling, as it clearly is costing way more then a new iBook to get this squared away. Three round trips via DHL and two logic board exchanges, plus ...

---

I don't think this is purely an Apple issue. It's really an outsourcing story. Apple outsources their services. That means they lose control over quality. Episodes like this one, especially when broadcast on the very influential Macintouch site, cost them a fortune.

Outsourcing is very tricky.

Bush rule: The Revenge of Andrew Jackson

The New York Times > Opinion > Op-Ed Contributor: The Cabinet of Incuriosities

... Whatever the roots of Mr. Bush's overriding devotion to loyalty, it partly stems from his disdain for the concerns of old-style meritocrats, the kind of people who wince when the president places his confidence in someone like Mr. Kerik. Mr. Bush has never been comfortable in America's so-called meritocracy. Undistinguished in college, business school and in the private sector, he spent nearly 30 years sitting in seminar rooms and corporate suites while experts and high achievers held forth.

Now it appears that he's having his revenge - speaking loudly in his wave of second-term cabinet nominations for a kind of anti-meritocracy: the idea that anyone, properly encouraged and supported, can do a thoroughly adequate job, even better than adequate, in almost any endeavor.

It's an empowering, populist idea - especially for those who, for whatever reason, have felt wrongly excluded or disrespected - that is embodied in the story of Mr. Bush himself: a man with virtually no experience in foreign affairs or national domestic policy who has been a uniquely forceful innovator in both realms...

... Now that Mr. Bush has won his final campaign and holds high a gleaming national mandate, he can be ever more himself. And for Mr. Bush, personality is destiny. What you do is not as important as whether you are deemed morally sound and trustworthy. In other words, a "good" man - or woman - beats a leading expert every time. Welcome to the new meritocracy.

Stratosphere on Everest -- more

Telegraph | Connected | Sky 'fell in' on Everest

Google found me this:

The eight climbers killed on the single deadliest day on Everest may have been victims of the "sky falling in", according to a study.

An analysis of weather patterns in May 1996 suggests the mountaineers died when the stratosphere sank to the level of the summit, 29,000ft above sea level.

The freak weather caused pressure and oxygen levels to plunge within the "death zone" - the area above 26,000ft where the oxygen is extremely thin.

Normally Everest's summit lies just below the atmospheric layer. But on May 10, the day of the disaster, there were two fast-flowing air streams, called jet streaks, moving over the mountain.

Dr Kent Moore, a physicist at the University of Toronto in Canada, believes these would have pushed the stratosphere boundary down with catastrophic results.

During a similar event in 1998 a temporary weather station near the top of Everest recorded a sudden fall in pressure of 16 millibars.

"Such a drop is significant where the air is already very thin," New Scientist reports today. On Everest's summit, it would have been the equivalent to raising the mountain by around 500 yards. It would have instantly cut the amount of oxygen available to the mountaineers by around 14 per cent, Dr Moore believes.

At the summit the air already contains only a third of the oxygen it holds at sea level. The eight were members of a group who were climbing without supplementary oxygen.

Conditions had been good, with the sky free of clouds and the wind light. However, by around 4pm, the "death zone" was engulfed by storms, winds of up to 90mph and temperatures that crashed to minus 40C.

Within 24 hours, eight out of 30 climbers on the mountain were dead. They included Scott Fischer, from Seattle, and Rob Hall, from New Zealand, the expedition leaders.

---

The story of that day was famously told in a magazine by Jon Krakauer; he later turned the story into a captivating book. His version of events has been disputed, unsurprisingly.

I have a slight connection to the expedition -- one my medical school classmates was the physician in Krakauer's book. (Krakauer misspelled my classmates name, I don't know if that was deliberate.)

Everest in the Stratosphere?

BBC NEWS | UK | Magazine | 100 things we didn't know this time last year: "75. Freak conditions above Everest can cause the sky to 'fall in'. An analysis of weather patterns in May 1996, by University of Toronto researchers, said eight people died when the stratosphere sank to the level of the summit."

---

I'd like to learn more about that one.

One story among many - the Sambodhi shelter

Disabled children were helpless tsunami victims

GALLE, SRI LANKA ... Some desperate children gripped the rafters as the water level rose inside the one-story Sambodhi shelter, while others on mattresses floated away to their deaths, witnesses said. Just 41 of the 102 residents of the home [for disabled children] survived, caretaker Kumar Deshapriya said Saturday...

Deshapriya, who uses a wheelchair because he has muscular dystrophy, said it began after he returned from placing orders for fish, grain and vegetables at the market...

...Shelter employee Saroja Senivirathna said that she and a few others climbed onto the roof to escape the waves but that the cries of trapped children below were so unbearable that they descended to try to save as many as they could.

"While I was on the roof, I thought the whole place was going to collapse," Senivirathna said. "While these children were screaming, I decided it would be better to all die together rather than save my life alone. That's why I got down."

...Deshapriya is determined to rebuild the shelter, which has received money over the years from Mormons in the United States, a Dutch charity, Galle municipal authorities and Sri Lankan air force veterans.

---

Had the waves hit 100 years ago, the population of the shorelines would have been quite small. Had they come 20 years from now, warning systems would have been in place. The waves came at the worst of times.

This is only one story among thousands, but there is a human need to create monuments to memory. Perhaps a restored Sambodhi shelter will be one monument. They will not lack now for funds, though, sadly, we will not lack for fraudulent charities claiming their name. Our family donates to CARE (www.care.org) and the Red Cross (most recently via Amazon's home page), but I'll make an exception for the Sambodhi shelter if I can find a legitimate charity sponsoring them.

PS. More Swedes died on the beaches of Thailand than Americans died in our most recent catastrophe -- the destruction of the World Trade Center.

Voting in the USA: it's much worse than you think

20 Amazing Facts about Voting in the USA

If the statements here are correct, then the voting situation in the US is rather worse than I'd believed -- and I didn't think it was very healthy.

Amazon: the service just gets better

Amazon.com: Welcome

In the midst of the flurry of vendor issues I've recently posted on (toyglobe.com, the 7K AMEX fraud charges), I received a notice from Amazon that an order had shipped. That led me to notice a few things about Amazon, a company I order from several times a month:

1. Their shipping speed has really improved.
2. Almost everything I order is free shipping.
3. Even though free shipping is supposed to delay shipping of multiple items, and even though that was historically a problem, that no longer seems to happen. Amazon splits the shipment and sends items separately.
4. They provide a great record of invoices online.
5. They provide manuals and related items online.
6. Their website has had no glitches in my experience in the past year.
7. The value-add of their customer reviews, and the summary of what people who viewed an item actually bought instead, is extraordinary.
8. Their prices are very competitive and often the best I can find.
9. Amazon is a great place to sell and buy used goods, less hassle than eBay and the prices are not as inflated as on eBay.

They just keep getting better. I recently read a Bezos interview where he noted an internal Amazon study. They decided they while advertising did increase their sales, they got a bigger increase by putting their ad revenue into free shipping and improved service.

Credit card fraud: more on my most recent experience

Faughnan's Notes: CC Fraud Take II

A few more thoughts on this recent $7K fraud experience.

1. How did the crooks manage delivery of the stolen goods? In the Netfill scam there were no good to deliver and the faked transactions were for virtual goods.

2. Why didn't AMEX's fraud alarms go off? I have to assume they disable them, or tune them down, around the holidays. I've had AMEX question my purchase of underwear while traveling on business, but I didn't get a call about 6 separate $550 transactions in one day against one company.

3. Why NEWEGG? Why not spread the transactions around and make the fraud less obvious? Was NEWEGG an easy target for some reason?

4. Was the attack fully automated? That's the most interesting possibiliity. It seemed pretty stylized -- two near identical attacks two weeks apart, each beginning with a domain name change. Problem is, unlike Netfill, this seems to have involved delivery of physical goods. Hard to see how that could scale. I don't think this was an automated attack.

The more I think about the delivery of physical goods problem, the more I'm inclined to think this was a kid somewhere.

Update: This must be the season for credit card fraud -- I might have another one! My son's Brett Favre figure broke; I had trouble finding a replacement on the net and ended up placing an order with toyglobe.com. My AMEX card was charged the day I placed the order: 12/3/04; it was charged slightly more than expected. Nothing has come from Toyglobe.com (office is in my old home town: St. Laurent, Quebec). Their web site doesn't have a phone number. A web search finds a worrisome web page of complaints. Hmmm. Suspicious timing?

Le, Guong
Toyglobe.com
3080 Barclay # 6
Montreal, Quebec h3s1j8
or
Toyglobe.com
5455 Vanden Abeele St.
St. Laurent, QC H4S 1S1

Bush: into the abyss

Salon.com | Neocons take complete control

.... The rejection of Kanter is a compound rejection of Scowcroft and James Baker -- the tough, cunning, results-oriented operator who as White House chief of staff saved the Reagan presidency from its ideologues, managed the elder Bush's successful campaign in 1988, and was summoned by the family in 2000 to rescue George W. in Florida. When all else failed (the voters, for example), Baker arranged the outcome that put Bush in the Oval Office. In the 1995 memoir of his years as secretary of treasury and state, Baker observed that in the Gulf War the administration's 'one overriding strategic concern was to avoid what we often referred to as the Lebanonization of Iraq, which we believed would create a geopolitical nightmare.' In private, Baker is scathing about the current occupant of the White House, people who have spoken with him have recently related to me. Now the one indispensable creator of the Bush family political fortunes is repudiated.

Those Republican elders who warned of endless war are purged. And those who advised Bush that Saddam was building nuclear weapons, that with a light military force the operation would be a 'cakewalk,' that capturing Baghdad was a 'mission accomplished,' and that the Iraqi army should be disbanded, are rewarded.

Powell, the outgoing secretary of state fighting his last battle, a rearguard action against his own administration on behalf of his tattered reputation, is leaking stories to the Washington Post about how his advice went unheeded. Secretary of Defense Donald Rumsfeld, whose heart beats with the compassion of a crocodile, clings to his job by staging Florence Nightingale-like tableaux of hand-holding the wounded, while declaiming into the desert wind about 'victory.' Since the election, 203 U.S. soldiers have been killed and 1,674 wounded.

---

There is no James Baker to save us from an ideologue president. We are into the abyss.

I'm left to hope Bush really can alter reality through sheer force of will. Meanwhile the rest of the world has to start taking adaptive action.

Lessons from the airtravel debacle of Christmas 2004: the risks of efficiency

The Cincinnati Post - Comair computer crash

Air travel this past Christmas was awful. I was on a simple direct flight from NWA, but we experienced 1-3 hour delays in both directions, one from a flaw in luggage loading, the other a mechanical. Workers were burned out everywhere, from food services to flight workers -- even the toilets were trashed.

We were lucky. US Air was disabled by labor strife (I'd assume the flight attendants figure their jobs are toast anyway, so they might as well try to take out the airline). Comair was disabled by a software bug. The simple answers are "better software" and "some airlines need to go away".

But maybe there are deeper lessons to learn:

.... Tom Parsons, of Bestfares in Arlington, Texas, said the lesson he learned long ago was to avoid northern connections during the winter.

'I still can't believe what happened to Comair. You notice that Delta is sitting in the background, saying that's Comair, that's Comair,' Parsons said.

He said part of the problem is that airlines have stripped down so far that they were near 100 percent capacity for holiday rushes.

'The systems are geared to run 100 percent, and hopefully nothing goes wrong. This time, just too many things hit (Cincinnati/Northern Kentucky airport). Each one became part of the domino effect. We now know not to connect through Cincinnati in the winter or to fly Comair or U.S. Air,' Parsons said.

Maybe the deeper lesson is the risk of high-efficiency systems. Most highly efficient systems have little redundancy and unused capacity. If something goes wrong, they are prone to lockup and collapse. The same thing happened to our electrical grid in the northeast about a year ago.

In theory a system could be both adaptive and efficient -- able to run without much redundancy but also able to quickly configure to adopt to changing circumstances. In theory. In practice I work in developing complex software to augment clinical work. I worry about the risks of increasing efficiency by reducing redundancy.

I think again (and again) of an odd lecture I attended at the annual scientific assembly of the American Academy of Family Practice. I don't know how the speaker got on the schedule, his topic we rerisks associated with complex and stressed systems -- not a very clinical topic! He was inspired by a popular book of the time. I enjoyed the presentation by a fellow physician-crank. I think he (and the book he'd read) were right then, and they're right now. We pay a price by sacrificing redundancy and adaptability in favor of efficiency. It's a lesson Rumsfeld ought to have learned from Iraq (he's an idiot however, so he probably hasn't learned anything). It's a lesson we ought to learn from evolution, where highly adapted and specialized animals disappear when their ecosystem is disrupted. (Too bad this lesson is lost on the anti-evolutionists.) There are real benefits in the long term to adaptability, to excess capacity, to shock absorbers, to redundancies.

Unfortunately the market is a tool for solving local minima equations. It does not necessarily reward the ability to tolerate infrequent system shocks.

Be the Best You can Be: endocannabinoids, Buspar and behavioral disorders

Be the Best You can Be: Endocannabinoids, buspirone (Buspar), and behavioral disorders in children with ADHD, PDD, EBD (explosive child)?

A highly speculative posting from a non-expert. An intriguing domain ...

Miracle surgery for migraine?

BBC NEWS | Health | Surgery 'helps combat migraines'Coincidentally, I'd just finished an interesting CME on headache when I saw this:

Surgery and botox injections can help treat migraines, a US study says.

Researchers injected about 100 patients with botox to find out which muscles triggered the migraines and then used surgery to remove the muscles.

The surgery reduced the intensity and frequency of migraines in 92% of patients and eliminated them altogether for a third of people involved.

The research, published in Plastic and Reconstructive Surgery journal, also cut the number of sick days taken.

---

A 33% lifetime cure for migraine? That would be miraculous. I'm skeptical. My bet:

1. This was a very select group. They had to respond to botox before surgery. The BBC reporter missed this. Since about half the population gets migraines, I'd bet this is a small subgroup who have an unusual migraine trigger. So even if it works, perhaps 1/10 to 1/20 of identified chronic migraine patients would be candidates.

2. I'd be astounded if the results will be this good in f/u studies. We know that migraine, like most pain conditions, is very susceptible to "placebo" affects. (We don't know what this "placebo" affect is -- it's very powerful and if we could master it then it would be very valuable. It's hard to manage though.) I bet a randomized placebo-controlled trial will show benefit in about 1/3 of 1/15 migraineurs (1/50!) and longtime cure in 1/10 of those treated (1/150 of migraineurs).

Ok, maybe I'm a Grinch. I knew of some neurologists years ago who were quite keen on these types of therapy, including using pre-botox interventions to kill some of these peripheral nerves. ENT surgeons, esp. in the 60s and 70s, used various similar interventions for cluster headaches (some used cocaine back then -- but it had its own problems). This wouldn't be the first time that an old intervention would be shown to have real value. And, on the other hand, we know that migraine prophylaxis drugs basically suck. (They are so ineffective that one suspects the entire value of many of them is from the placebo effect.)

We'll see ...

Dyer on the humanization of Hitler

Mass murder in the name of a principle is as human as apple pie, borsht and steamed rice. Treating the perpetrators as space aliens simply disguises the nature of the problem. The potential mass killers live among us, as they always have. They often have perfectly good manners, and some even have high ideals. And the only way the rest of us have to keep them from power is to remember always that the end does not justify the means.

---

I think I wrote too harshly of Dyer earlier today. I'm reading through his 2004 material and finding quite a few gems. Evil is an everyday, human affair. Considering how awful we are now, it is chilling to think of how nasty we were when we ate Neandertal.

Credit Card Fraud: Take Two

Credit Card Fraud Page

In 1988 I was a minor victim of an international credit card fraud scam. The perps had set themselves up as a California bank, then legally purchased a large number of credit cards (banks can do those things). They then ran small fraudulent transactions (fake net porn transactions) against tens of thousands of cards around the world. They were shut down, but I doubt any of the crooks did jail time. The fraud was interesting because it foreshadowed a range of techniques that have since been deployed around the net.

At that time I also learned how very frail our credit card infrastructure is. A system built for physical person-present transactions does not migrate well to the net.

One of the recommendations I made, based on that experience, was to use AMEX and thus take advantage of a more centralized approach to fraud management.

Today, six years later, I discovered another interesting pattern of fraudulent charges on my AMEX account!

On Dec 14th/15th, and again on Dec 23/24th, there appear a series of charges that look like this:

1. DOTREGISTRAR.COM: 69230017 INET-DOMAIN NAME TRANSF
2. NEWEGG COMPUTERS: 6-7 charges of about $550.

So with two sets of the above there's about $7030.00 in fraudulent charges.

So now I'll get to see how well AMEX actually works. Thus far I'm spending a fair bit of time waiting on the phone as my call percolates through their fraud division. More updates to follow.

Update: AMEX took about a half hour to get me through to the person who managed it. They didn't ask me any questions; they marked the transactions as fraudulent and are sending me a new card. Unfortunately when AMEX sends an "expedited" card it's a temporary CC number -- pretty useless for me. So there will be a one week delay -- they should do better.

I'll post later on how well AMEX handles this.

I wonder if the DOTREGISTRAR.COM transaction was to enable a temporary mail redirect. Online vendors often use email to establish "identity". It's a frail system, and suspect the thieves probably used a throwaway domain to defeat the identity management. Looks like a pretty cookie-cutter theft, it might have been done by kids or professionals.

Update 1/17/05: AMEX took a while to answer the phone, but they dealt with the problem very quickly. They asked me about 3 questions and reversed everything. They sent an "affidavit of fraud" but didn't even bother to have me sign it.

Unfortunately even though AMEX can Fedex a card in 24 hours, it's a temporary number and hence useless for my online purchases and subscriptions. It takes them a week to send a permanent card.

Thinking about the scam, it's probably what's known now as an 'eBay operation'. Get the goods and sell them on eBay.

On the evolution of reality

Natural selection acts on the quantum world

Looks like the anti-Darwinists will have to put their stickers on physics texts after their done with the economics library.

Natural selection acts on the quantum world
Philip Ball, Nature.com Dec 23, 2004

A team of US physicists has proved a theorem that explains how our objective, common reality emerges from the subtle and sensitive quantum world....

... certain special states of a system are promoted above others by a quantum form of natural selection, which they call quantum darwinism. Information about these states proliferates and gets imprinted on the environment. So observers coming along and looking at the environment in order to get a picture of the world tend to see the same 'preferred' states.

If it wasn't for quantum darwinism, the researchers suggest in Physical Review Letters, the world would be very unpredictable: different people might see very different versions of it.

... Because, as Zurek says, "the Universe is quantum to the core," this property seems to undermine the notion of an objective reality. In this type of situation, every tourist who gazed at Buckingham Palace would change the arrangement of the building's windows, say, merely by the act of looking, so that subsequent tourists would see something slightly different.

... The Los Alamos team define a property of a system as 'objective', if that property is simultaneously evident to many observers who can find out about it without knowing exactly what they are looking for and without agreeing in advance how they'll look for it.

Physicists agree that the macroscopic or classical world (which seems to have a single, 'objective' state) emerges from the quantum world of many possible states through a phenomenon called decoherence, according to which interactions between the quantum states of the system of interest and its environment serve to 'collapse' those states into a single outcome. But this process of decoherence still isn't fully understood.

"Decoherence selects out of the quantum 'mush' states that are stable, that can withstand the scrutiny of the environment without getting perturbed," says Zurek. These special states are called 'pointer states', and although they are still quantum states, they turn out to look like classical ones. For example, objects in pointer states seem to occupy a well-defined position, rather than being smeared out in space...

...Now, Zurek and colleagues have proved a mathematical theorem that shows the pointer states do actually coincide with the states probed by indirect measurements of a system's environment. "The environment is modified so that it contains an imprint of the pointer state," he says.

Yet this process alone, which the researchers call 'environment-induced superselection' or einselection, isn't enough to guarantee an objective reality. It is not sufficient for a pointer state merely to make its imprint on the environment: there must be many such imprints, so that many different observers can see the same thing.

Happily, this tends to happen automatically, because each individual's observation is based on only a tiny part of the environmental imprint. For example, we're never in danger of 'using up' all the photons bouncing off a tree, no matter how many people we assemble to look at it.

This multiplicity of imprints of the pointer states happens precisely because those states are robust: making one imprint does not preclude making another. This is a Darwin-like selection process. "One might say that pointer states are most 'fit'," says Zurek. "They survive monitoring by the environment to leave 'descendants' that inherit their properties."

"Our work shows that the environment is not just finding out the state of the system and keeping it to itself", he adds. "Rather, it is advertising it throughout the environment, so that many observers can find it out simultaneously and independently."

---

I know I read a science fiction novel recently, written by a physicist, that basically covered this terrain in the guise of fiction. Of course I say "basically" as though I understood any of this article. I'm looking forward to the Scientific American article. I particularly want to know if this has any relevance for "spooky action at a distance" -- a now "commonplace" macro-phenomena that continues to disturb me.

Next up -- experiments that show one can bias the evolution of reality ...