Why Joel Spolsky hates Microsoft Mesh -- and what this says about the peculiar course of software outsourcing

Spolasky thinks the Microsoft Live Mesh idea is dumb, that Microsoft is again trying to solve an insanely hard problem nobody cares about - synchronization. Ray Ozzie rewriting Notes/Agenda/Groove yet again and again. Spolsky makes a great analogy between Mesh and the nigh-incomprehensible Hailstorm.

Personally, I care a lot about synchronization and I believe it's a deeply hard problem. In addition to my personal PIM sync obsessions, synchronization technologies have a huge potential impact on healthcare interoperability. So I think Spolsky has a some good (albeit cruel) points about Architecture astronauts, but he's possibly undervaluing the synchronization problem.

On the other hand, his comments about Google and Microsoft's brain drain is really interesting ...

Architecture astronauts take over - Joel on Software

... Why I really care is that Microsoft is vacuuming up way too many programmers. Between Microsoft, with their shady recruiters making unethical exploding offers to unsuspecting college students, and Google (you're on my radar) paying untenable salaries to kids with more ultimate frisbee experience than Python, whose main job will be to play foosball in the googleplex and walk around trying to get someone...anyone...to come see the demo code they've just written with their "20% time," doing some kind of, let me guess, cloud-based synchronization... between Microsoft and Google the starting salary for a smart CS grad is inching dangerously close to six figures and these smart kids, the cream of our universities, are working on hopeless and useless architecture astronomy because these companies are like cancers, driven to grow at all cost, even though they can't think of a single useful thing to build for us, but they need another 3000-4000 comp sci grads next week. And dammit foosball doesn't play itself.

Whoa. Only a few years ago American software engineers were a doomed breed; all software design and invention would be outsourced. Now we read that starting salaries for novice, albeit very smart, CS grads are putting a crimp in Spolsky's business plans.

This fits with what I see in my corner of the industry. There's a new appreciation for the value of really smart people, and personnel costs are about to take an unexpected turn in the midst of a recession.

Outsourcing is taking a strange and unpredicted course.

Update 5/20/08: Microsoft Mesh sounds a lot like Microsoft FeedSync. Turns out FeedSync is an element of Mesh, which makes Mesh seem like a Microsoft marketing concept rather than a product concept. That's never happened before ... (joke)

Detecting weapons grade uranium: give up

Twice in the past 7 years, ABC news has smuggled depleted uranium, with a radioactivity signature comparable to highly enriched uranium (HEU), into the US. Despite packaging and address data designed to ensure maximal screening, the mild radioactivity was not detected.

HEU, like depleted uranium, is just not all that radioactive.

ABC received technical support from two scientists, one of whom was transiently assigned to a Homeland Security watch list as punishment for embarrassing the Bush administration. Last month the two published an article on the detection of smuggled HEU ...

Detecting Nuclear Smuggling: Scientific American

• Existing radiation portal monitors, as well as new advanced spectroscopic portal machines, cannot reliably detect weapons-grade uranium hidden inside shipping containers. They also set off far too many false alarms.
• So-called active detectors might perform better, but they are several years off and are very expensive.
• The U.S. should spend more resources rounding up nuclear smugglers, securing highly enriched uranium that is now scattered overseas, and blending down this material to low-enriched uranium, which cannot be fashioned into a bomb.

In addition to the above synopsis, the author's point out that it's fairly trivial with modern HEU to create a nuclear weapon. (The online version of the article includes a plaintive editor's note claiming all the information in the article is available from public sources.)

The NYT wrote about the detector program last March. The detectors are great at producing false alarms, it turns out that we live in fairly radioactive world*. Problem is, the best research tells us they're really lousy at finding minimally shielded weapons grade uranium.

It's reasonable to invest in better detectors, but the current generation are security theater with a high cost in false alarms. We should be focusing our efforts on restricting leakage of HEU from Russia and other sources. We won't get that kind of intelligent response from Bush or McCain, so all we can hope is that someone else wins the presidency.

On the other hand, I remain puzzled that five years after many experts agreed it was inevitable, we haven't seen nuclear terrorism in the US. It's not the detectors, and most reports indicate we're not doing enough to slow the HEU trade, so what's up?
--
* Good thing that current medical research suggests we're more radiation resistant than we thought we were. Our DNA repair systems do relatively well with radiation.

Sometimes a 2nd chance works - convicts in the army

People with a criminal record have a hard time finding work, but recently recruiting problems have created military opportunities.

The good news is a study of ex-felons finds many of them can succeed as soldiers ...

The Few, the Proud, the Bad - Intel Dump -

...The study looks at the performance of soldiers who entered the Army with waivers for prior criminal convictions, no high school diploma or other reasons. In a nutshell, these soldiers get into slightly more trouble, but assuming they make it through basic training and avoid major trouble, they're more likely to be better soldiers...

This is the proverbial silver lining in a pretty dark cloud.

The end of television - now it's official

I never caught the TV habit. When I was a child we were too poor to have a working television (really -- people gave us TVs but they kept breaking), and after that I was too busy. My wife and I watched Star Trek - Next Generation in its heyday and I think it was great stuff, so I'm not opposed to television -- it's just that I rarely have time for it.

If our children were calmed by TV I'd use it as a pacifier, but the commercials agitate them. So they watch Netflix videos, including TV episode DVDs, three times a week.

We're a weird family, so I assumed television was still popular among the normals. It came as a shock seven months ago when I realized how little television my children's classmates seemed to watch. Since then I've begun to pay attention to the slow and quiet collapse of broadcast TV in America. It reminds me of the disappearance of public smoking -- an unquestioned bit of boomer life suddenly impermanent.

Now the decline has become official, marked by a Clay Shirky book and essay that's receiving deserved attention:

Gin, Television, and Social Surplus - Here Comes Everybody

... And what did we do with that free time? Well, mostly we spent it watching TV.

We did that for decades. We watched I Love Lucy. We watched Gilligan's Island. We watch Malcolm in the Middle. We watch Desperate Housewives. Desperate Housewives essentially functioned as a kind of cognitive heat sink, dissipating thinking that might otherwise have built up and caused society to overheat.

And it's only now, as we're waking up from that collective bender, that we're starting to see the cognitive surplus as an asset rather than as a crisis. We're seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody's basement.

This hit me in a conversation I had about two months ago. As Jen said in the introduction, I've finished a book called Here Comes Everybody, which has recently come out, and this recognition came out of a conversation I had about the book. I was being interviewed by a TV producer to see whether I should be on their show, and she asked me, "What are you seeing out there that's interesting?"

I started telling her about the Wikipedia article on Pluto...

... So I tell her all this stuff, and I think, "Okay, we're going to have a conversation about authority or social construction or whatever." That wasn't her question. She heard this story and she shook her head and said, "Where do people find the time?" That was her question. And I just kind of snapped. And I said, "No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you've been masking for 50 years...

I think we missed how much of the boom years of the 80s and 90s came from freeing up cognitive resources consumed by the Cold War, and what it has meant to return to a military economy over the past decade. Shifts in cognitive resources, such as 1970s entry of women into the workforce, have big impacts.

I don't think ending television is going to have as big an effect on cognitive resource deployment as feminism or the end of the Cold War, but as a social phenomena it's well worth a notice.

Broadcast television is finally dying. That's news.

Update 5/15/08: A well written contrarian response to Shirky's cognitive surplus thesis.

How to save the New York Times - TimesCash voting

Murdoch, as expected, is remaking the Wall Street Journal in his own image. The old leadership has been evicted.

The New York Times is doing very badly in the market. The share price predicts a quick decline, the only thing saving the paper is a two tiered stock structure that splits control from equity. The NYT is next for the chopper.

Can it be saved?

If I had control, I'd introduce "TimesCash".

Here's how it would work.

Anyone can buy any amount of TimesCash at any time using their credit card, starting with a minimum block of $10. It's sold through Google Checkout, Amazon, or the NYT could process its own transactions.

Readers spend any amount of TimesCash they have on any story they like, but clicking on a button next to the story. Each day, the NYT would rank stories by TimesCash earnings, just as they do for "most emailed", "most blogged" and "most searched". Readers could decide if they want their names to be displayed alongside the stories they've voted for.

When a reader's TimesCash account is depleted, they can buy more whenever they want.

Once a year the names of the biggest TimesCash purchasers would be published -- for those who chose the publicity.

This would save the New York Times. I'll take $10 of TimesCash as my reward.

How are TV generals like tobacco industry executives?

Morford notices how much our once-respected military leaders came, under the Bush administration, to resemble tobacco industry executives ...

All the president's liars / Fun new game! Which TV news "military expert" is really a whore for the Bush administration? (Hint: all of them)

... Did you watch any CNN or Fox News or MSNBC, lo, these past five or six years, listen to the pundits and ponder the wise, informed comments of all the military experts the networks brought on to discuss Iraq policy,...

... a highly specialized group known to gullible Americans as stoic, stern-faced retired generals, colonels, majors, military advisers, former Pentagon officials, the ones you've heard and seen on TV news for years, but who are known to the Bush administration as a delightfully dishonest gaggle of preferred liars, lackeys, shills, puppets and mouthpieces for Dick Cheney and Donny Rumsfeld and Dubya himself...

...  Why would they do such a thing? ...

... That's easy: Access. Access to the White House, to the corridors of power and influence; access to the perks and the pals and snifters of brandy, the backroom handshakes, the business deals, the hugely lucrative military contracts, the sweet, sweet piles of cash and privilege and power awaiting them if they just toe the line and keep their real opinions to themselves...

...Reminds me, in a depressing sort of way, of that gaggle of Big Tobacco CEOs who banded together not long ago in a hilarious attempt to convince the nation — and the courts — that cigarettes aren't all that bad and there's little evidence smoking causes cancer or impotence or death, and in fact small children really love secondhand smoke and so do puppies and flowers and Jesus, and if you want to have fun sometime, walk into a hospital nursery and fire up a fresh Marlboro and blow that yummy smoke straight into the faces of the newborns. Watch them squirm with delight!...

I bet there's more here than great metaphor. I bet the same marketing companies that worked for the tobacco industry, and that had great relationships with their GOP allies, also advised the Pentagon on their propaganda plans.

The story was exposed by the New York Times. What the heck are we going to do when Murdoch et al acquire the New York Times?

The NYT needs to put Amazon and Google (not PayPal!) donation buttons on every article. We need that rag to keep running.

Edwards and Fallows agree - the press is bad. Why?

Fallows agrees with Elizabeth Edwards - the press is the problem.

James Fallows (April 28, 2008) - Most important item in Sunday's NYT

This Sunday's New York Times -- fat, varied, making me wonder how I got anything done on the weekends in America when I routinely had all this to read -- had lots of interesting stuff in it. But the most important item was the op-ed by Elizabeth Edwards called "Bowling 1, Health Care 0."..

...The more heartfelt and bitter complaint is about the way press coverage seems biased not against any particular candidate but against the entire process of politics, in the sense that politics includes the public effort to resolve difficult issues. (Medical care, climate change, banking crises, military priorities, etc.) For twenty years I have heard this from frustrated politicians -- Gary Hart, Newt Gingrich, Jimmy Carter, Dick Gephardt, Bill Clinton, they may not share a lot of views but they are as one in this frustration. What galls all of them is the way that the incentives created by most coverage bring out the very worst in most politicians, and discourage them from even bothering to try the harder, more "responsible" path. No one says that press incentives turn potential Abraham Lincolns into real-world Tom DeLays. But the incentives push in that direction rather than the reverse.

Active politicians rarely dare say this in public, since they know the same reporters and commentators will be there to talk about them tomorrow and the next day and from then on. For reasons personal (health) and political (husband out of the race), Elizabeth Edwards no longer has to hold anything back...

Alas, neither Edwards nor Fallows asked the interesting questions.

Why is the print press* bad? Is it really getting worse?

My suspicion is that the Craisglist-Google driven decline in ad revenues is forcing newspapers to be ever more customer focused -- which means struggling to hold onto the eyeballs their advertisers want.

That means they give their readers what they want.

What does the portion of the public who's willing to read a newspaper want?

The bulk of the readership doesn't want to read about policy, they want to be distracted and entertained. So the newspapers provide them entertaining news about politicians, such as their hair styles, bowling scores, partners, pastors, etc.

Pogo said it long ago - We have met the enemy and he is us. The media is not the problem, the voters are the problem.

It's easy for Elizabeth Edwards and James Fallows to beat up on the press. Heck, they richly deserve a beating. It's a lot harder for them to admit the real problem is the American citizen.

Now that's a hard problem.

* I don't know anything about TV and radio news except NPR, so I'll omit them.

The surprising cost of fly intelligence ...

Science is most fun when it throws up completely bizarre surprises. Like these study results ...

Learning and longevity | Critical thinking | Economist.com

...After repeating the experiment for 30 generations, the offspring of the learned flies were compared with normal flies. The researchers report in a forthcoming edition of Evolution that although learning ability could be bred into a population of fruit flies, it shortened their lives by 15%. When the researchers compared their learned flies to colonies selectively bred to live long lives, they found even greater differences. Whereas learned flies had reduced life spans, the long-lived flies learned less well than even average flies.

The authors suggest that evolving an improved learning ability may require a greater investment in the nervous system which diverts resources away from processes that stave off ageing. However, Dr Kawecki thinks the effect could also be a by-product of greater brain activity increasing the production of reactive oxygen particles, which can increase oxidation in the body and damage health.

No one knows whether the phenomenon holds true for other animals. So biologists, at least, still have a lot to learn...

As far as we can tell, humans are the only technological species that ever lived on planet earth. (See, however, Stephen Baxter's Evolution for a wonderful imaging of a pre-technological sentience.)

We've come along fairly late in the planet's history.

Why did it take so long to produce an extinction-event class species [1]? What was there about intelligence that was so hard?

The flies might be giving us some clues ...

[1] Anyone studying the fossil records will see evidence of worldwide mass extinction beginning early in our evolutionary career. Species that can cause that kind of mass extinction are in a class of their own, albeit a short-lived class.

The way politics works: a Gail Collins reminder

Gail Collins, who's definitely growing on me, provides a voice of reality in the midst of the endless primary:

Hillary’s Smackdown - New York Times

... Although Obama has seemed way off his game lately, the odds are still really, really good that he’ll get the nomination. The superdelegates are just waiting for him to win something so they can rally. And once the fighting is over, there’s no question that Hillary would rally her supporters behind him. (This is a woman who sat down for a chat with arch-conservative-right-wing-conspirator Richard Mellon Scaife just to wrest an endorsement from his little fringe newspaper in Pittsburgh.) And within a couple of weeks, Bill Clinton would be treating Barack like a surrogate son and forcing him to play golf...

These people are lawyers.

A lawyer is someone who can spend years as a prosecutor, then switch sides with equal zeal. A lawyer is someone who can fight to the death in a courtroom, then go for coffee with the opposing attorney when the trial is done. A lawyer is someone who can rend a surgeon's reputation, then be genuinely puzzled when the surgeon avoids them in the supermarket.

Lawyers are conflict professionals, like salesmen, professional ball players, or genuine military mercenaries. Lots of ranting and screaming and even shooting, but really, it's just part of the job.

Lawyers who are politicians running for the presidency are the consummate conflict professionals.

Neuburg: 15 years of information wrangling

Tidbits turned 18 recently. The celebratory email referenced article series almost as old as Tidbits: TidBITS: Matt Neuburg - Conquer Your Text 1993 to 2006.

The classics are there: Symantec MORE, Inspiration (still sort of living), Acta, In Control, Arrange and WebArranger, (Double) Helix, Prograph, Idea Keeper, Boswell, StickyBrain, Tinderbox, iData Pro, Notebook (both of them), DEVONthink, TAO, Curio, Yojimbo, SlipBox and a few others.

There was terrific software in that collection, some of which is still sold. On the other hand the history is a reminder of the terrible cost of proprietary data formats; many of those apps took user data with them when they expired.

I love this class of software, but I wouldn't personally consider anything with a proprietary file format.

Incenting execs: they're very good at gaming the system

Harvard's Philip Greenspun does a great job describing how EBITDA and other compensation schemes are readily gamed ....

Philip Greenspun’s Weblog Statins, cholesterol, health; fancy employee compensation, EBITDA, and company value:

... Conspiracy of Fools chronicles one of the discussions about EBITDA among Enron senior managers. One guy pointed out to Rebecca Mark, a Harvard Business School graduate star of the company, that EBITDA was meaningless because one could improve EBITDA simply by borrowing money at 10 percent and investing it in T-Bills at 5 percent and that was essentially what Mark was doing. She was borrowing money at X% to purchase businesses that would return no more than (X-4)% in a best-case scenario. This fattened her paycheck, but led the company towards bankruptcy....

.... if you’re on a Board and you decide to compensate a manager with anything other than cash or a long-term stock option, make sure that you’re not granting compensation based on a number that the manager can easily manipulate. Keep in mind that managers are often a lot more clever in doing things that will benefit themselves than things that will benefit the company.

Senior executives in large corporations may not be particularly honest, honorable, or bright -- but they are always good at playing the game. Greenspun's key observation is that only long term share price is difficult to game for a CEO. All other incentive plans will lead to undesireable choices.

Damn, but Hoover was an evil man

Damn Interesting » Operation Pastorius tells us some more lousy things about mid-20th century American justice, but most of all it reminds me how bloody evil J.Edgar Hoover was.

As we inch our way from surveillance state to the next step down (more on that soon), it's ever more important to remember Hoover.

I'd buy a nicely made t-shirt that said: "Remember J. Edgar Hoover. Protect your freedom."

Who's paying for the "Global Climate Scam" billboards?

There's a large billboard near my Saint Paul home advertising "GlobalClimateScam.com".

The url resolves to a typical denialist site -- all bluster and delusion. There's nothing unusual about the whacky website, but I was curious who's paying for the billboard, and what they want.

It turns out that "Minnesota Majority", a local right wing organization, displays the same content and is probably paying for the billboards. I haven't been able to discover where their money is coming from, and why they're suddenly keen to spend it on climate change denial.

Companies that have invested heavily in coal would be the obvious suspects. If we really have hit Peak Oil (I'm deferring judgment until August) then oil companies ought to be buying up coal reserves. The primary challenge to that strategy would be a carbon tax, or the regulatory equivalent. Coal produces so much carbon dioxide that any carbon-tax equivalent could do real damage to a coal-centric investment strategy.

It would be logical for these companies, assuming they are blithering idiots, to do everything possible to maximize their coal reserve value -- including climate change deniers.

So if I were a real journalist, I'd be looking for an Exxon connection to Minnesota Majority's new found fascination with climate change. I'd also look to see whether Exxon is putting its money into oil exploration or into coal reserve ownership.

Insanely bad decisions - the Dilbert example - and a wise correction

The primary Dilbert web site has gone to Flash. It doesn't work on Camino or the iPhone, and it's a mess on Firefox 3 (true, that's beta).

I assume Adobe paid United Media to do this.

The move has not been well received. Happily, there's an alternative non-flash link ...

Slashdot | Dilbert Goes Flash, Readers Revolt

... Good thing you can still get your Dilbert fix at http://www.unitedmedia.com/comics/dilbert/archive/ [unitedmedia.com]...

I'll change the link on our family news page to the non-Flash version.

This move deserves a Dilbert cartoon, but I don't think Scott Adams is going to satirize his own syndicate. He's a pretty careful businessman, despite the Dilbert persona.

Update 4/24/2008: United Media now redirects the archive to the Flash page. I sure hope Adobe is having to pay a lot for this! On the other hand Scott Adams has felt our wrath. A "plain" page is pending, and the RSS feed will show the strips chronologically without Flash.

Update 5/2/2008: A good solution, I've updated our news page to use it.

From Scott Adams blog:

.... if you promise to keep it to yourselves, we created a stripped-down Dilbert page with just the comic, some text navigation, and the archive: www.dilbert.com/fast. This alternate site is a minor secret, mentioned only here and in the text footnote to the regular site as “Linux/Unix.”

Lesson: Your data will be public

When you interact with a web page you're often interacting with a database of some sort. The simplest way to do this is to take text the user has submitted, put some SQL around it (standard database language), and the SQL will update the database, or get results back, etc. Some implementations even put the query string, including the SQL, in the URL.

The problem with this approach was discovered in the 1990s. You can write your own SQL in the URL, or, with a bit more work, you could type the SQL into the web field and the database would act on it.

The problem keeps returning, most recently in Oklahoma. Schneier describes the story (with an implied deep sigh), but I most appreciated one of the comments ...

Schneier on Security: Oklahoma Data Leak

... If you take the standard Google query for locating GET/sql servers (see http://www.memestreams.net/users/acidus/blogid10326823/ and further restrict it to .gov domains, several somewhat  sensitive websites from the District of Columbia government show up --- including "Alcoholic Beverage Regulation Administration --- Suspended and Revoked Licenses", and "Department of Health --- Food Establishment Closures".

It's nice to know a simple Google query can expose vulnerable sites so readily. Another comment mentions a similar problem last year with Germany's security services.

The real lesson here is that all of your data will be public one day. Sooner or later it will leak by one security flaw or another. If they gather it, they will lose it.

If you really don't want data to be public, don't provide it in electronic form. Happily, most data about most of us is so boring it's only of interest to the extremely rare individual who wants to steal our identity, credit card, etc. Oh, wait ...