The origins of corporate mediocrity - promoting the best

It would be easy to mock the results of this computer simulation, but I think they're on to something. When you read the following synopsis consider what I assume were the assumptions of the simulations:

• Different jobs require different skill sets
• Different workers have different skill sets
• The best worker at a job will be the one who's skill sets best match the job demands

If you think through these assumptions, you might be able to predict the results of the simulations ...

Technology Review: Blogs: arXiv blog: Why Incompetence Spreads through Big Organizations - Ref: arxiv.org/abs/0907.0455: The Peter Principle Revisited: A Computational Study

Promoting the people most competent at one job does not mean that they'll be better at another, according to a new simulation of hierarchical organizations.

There's a paradox at the heart of most Western organizations. The people who perform best at one level of an organization tend to be promoted on the premise that they will also be competent at another level within the organization...

In 1969, a Canadian psychologist named Laurence Peter encapsulated this behavior in a rule that has since become known as Peter's Principle. Here it is:

"All new members in a hierarchical organization climb the hierarchy until they reach their level of maximum incompetence."

That's not as unfair as it sounds, say Alessandro Pluchino and buddies from Universita di Catania, who have modeled this behavior using an agent-based system for the first time. They say that common sense tells us that a member who is competent at a given level will also be competent at a higher level of the hierarchy. So it may well seem a good idea to promote such an individual to the next level.

The problem is that common sense often fools us. It's not so hard to see that a new position in an organization requires different skills, so the competent performance of one task may not correlate well with the ability to perform another task well.

Peter pointed out that in large organizations where these practices are used, it is inevitable that individuals will be promoted until they reach their level of maximum incompetence. The unavoidable result is the runaway spread of incompetence throughout an organization.

Now Pluchino and co have simulated this practice with an agent-based model for the first time. Sure enough, they find that it leads to a significant reduction in the efficiency of an organization, as incompetency spreads through it....

But is there a better way of choosing individuals for promotion? It turns out that there is, say Pluchino and co. Their model shows that two other strategies outperform the conventional method of promotion.

The first is to alternately promote first the most competent and then the least competent individuals. And the second is to promote individuals at random. Both of these methods improve, or at least do not diminish, the efficiency of an organization.

Clever!

Of course the assumptions I assume were used in this study do have limitations. Many (most?) corporate superstars are characterized by an insatiable appetite for work and a relative disinterest in sleep, family and friendship. These traits do transfer well between roles. It's also true that senior roles are largely about managing down (reports), laterally (peers), and up (bosses). The political and personal skills that allow people to excel in those roles are reasonably universal.

So the model oversimplifies.

On the other hand, there really does seem to be something amiss with the modern publicly traded corporation. In general, they are less than the sum of their people. If not for the sheer advantages of size, such as the ability to buy or destroy smaller would-be competitors, I suspect most large corporations would not be able to effectively compete. (The CEO? The empiric evidence suggests that CEOs don't have large effects unless the business is very troubled or the CEO is really incompetent. Additionally, CEO selection is particularly perverse, it's no surprise most are ill-suited to the role they fill.)

Maybe the promotion effect does play a role in limiting how well a corporation can do.

PS. I think they should have tested a strategy of random promotion and rapid demotion for those who don't succeed.

The end of passwords - episode LXVII

Yawn.

It turns out that if you know a little bit about someone it's possible to compute their social security number. Social security numbers, of course, are often treated as a secret password, something that's known to only one person, and thus proof of identity.

Well, ok, two people. You and the the bank.

Okay, you and the bank and the hospital and your employer and your former employer and the IRS and your spouse and your ex-spouse and whoever stole your health insurance card and 425,000 hackers.

In other words, it's a lot like the secret questions my scream-inducing bank tortures from me. It's a backdoor for anyone who wants to steal your Gmail account (fortunately, few do).

Houston, we have a problem.

Shades of grief

World of sorrows ...

BBC - James Reynolds' China: Three years in China

... In the summer of 2007, I went to central China to cover the news that hundreds of men had been found working as slaves in illegal brick factories. Some had been kept underground for so long that they no longer knew their own names.

I met a man called Zhang Bairen. His son Zhang Zhike had gone missing and the father was hoping that his son might be one of the rescued slaves. But he wasn't.

I asked the family if they could show me a picture of their missing son but they didn't have one. The family was too poor to afford any photos.

A year later, some more men working as slaves were rescued. The family hoped that the missing Zhang Zhike might be among them. But, again, he wasn't.

Another year on, family members tell us that they have now given up hope of ever finding their lost son. Theirs is a silent grief...

Other people love Microsoft Word almost as much as I do

About 6 years ago I wrote about Microsoft Word: Living with the Beast. Nothing has happened since to change my opinion of Word.

Recently some Word bug related to cross-references made a hash of a technical document I wrote. A quick Google turned up this pithy quote from an anonymous tech writer commenting on a newsletter post

Word annoyance: Cross-referencing « CyberText Newsletter

[JR, comments]… Microsoft Word 2007 is a steaming pile of flaming maggot [censored] coded by a bunch of monkeys typing random keys on their keyboard and hoping it compiles…

Join the club comrade.

The GOP’s amazing spiral into historic irrelevance

Somewhere Mitt Romney is screaming “Pull Up”, Pull Up” and hauling back on the control stick as his one winged plane screams groundwards …

GOP Pols Losing Control Of Tea Party Movement? | TPMDC

Thousands of right-wing activists across this country rang in the Independence Day holiday with yet another round of tea-party protests against President Obama...

… Sen. John Cornyn (R-TX), who is chairman of the National Republican Senatorial Committee, was booed at the event in Austin -- on the grounds that he's part of the problem in Washington, having voted for the Wall St. bailout last fall…

… Gov. Rick Perry -- who famously seemed to raise the specter of Texas seceding from the union during the April Tax Day protests -- was also booed at the same Austin event as Cornyn. Attendees saw him as yet another tax-hiking tyrant, because he supports toll roads in order to relieve traffic congestion…

… Katie Vandermeer … heard about the tea party through the Texas Nationalist Movement, which advocates Texas' secession from the U.S.

In Bemidji, Minnesota, a headline speaker for their "Freedom Over Socialism" rally was state Rep. Mary Seifer … who warned of government taking away everyone's personal freedom: "Now suddenly we tell you that you have to wear your seat belts … Another speaker, former state legislative candidate John Carlson, spoke favorably of the Articles of Confederation.

The tea party in Columbia, South Carolina, featured Sen. Jim DeMint and state Rep. Nikki Haley, a leading Republican candidate for Governor. One prominent person was missing, though: Gov. Mark Sanford, who had previously headlined a Tax Day tea party back in April.

The tea party in Boiling Springs, South Carolina, featured a colorful cast of characters. The headline speaker was Alan Keyes, who has been a leading name of the "Birther" movement … One attendee took out a flyer that said, "Zelaya today, Obama tomorrow," but said he was advocating impeachment of Obama after he was asked directly whether he was in favor of a coup.

At the event in Los Angeles, right-wing former Saturday Night Live actress Victoria Jackson -- who has previously called Barack Obama a Muslim and a communist -- called for the President's impeachment, "There, I said it," and did a handstand dedicated to our men and women in uniform….

I knew the GOP was in trouble, but I didn’t expect it to come apart at the seams.

This is bad. We all need a credible, Rational, alternative to my currently favored party. The only thing a wrecked GOP will produce is right wingnut terrorists.

Maybe Romney will pull the GOP out of its dive into oblivion. Or maybe they’ll be so shattered in 2010 and 2012 that we’ll end up with a new political party …

Alternative local news: MinnPost and TC Daily Planet

Our Twin Cities newspapers are in intensive care and not expected to live.

Dog bites man stuff.

So I'm looking for alternative sources for local professional investigative journalism. I'm following two at the moment via Google Reader:

• Twin Cities Daily Planet | Minneapolis - St. Paul
• MinnPost

I haven't figured out which is the more Rational of the two. Reason aside, they should both give up on world/nation coverage. That's the beat of NPR, the Guardian, NYT, BBC, WSJ and the blogosphere. At the very least they need to provide a feed that excludes national/world coverage - just local news, events, entertainment, etc.

Anyone know of other new media startups doing metro and state coverage?

CompuServe was still alive! It’s dead now.

I thought CompuServe was long dead.

Turns out, it was still running until a few days ago! It’s just been officially shut down. There are still people with CompuServe email addresses, apparently they’ll continue to function.

I’m sure I have my old CompuServe user ID around. I think I was one of a very small number of people who actually used an OS/2 CompuServe client!

In honor of the passing of an era, some threads from a web discussion I came across. The last is the most amazing …

CompuServe Requiem » Basex Blog »

…The CIS addresses were octal - digits ranged only from 0 to 7. Mine was 70014,2316…

… they were properly “programmer/project numbers” (PPNs), intended to identify who was working on what software project.

the idea of having ever assigned such an arcane nomenclature to ordinary, frequently non-technical users was an absurdity from the beginning. and it caused no end of difficulty when the time came (1987) to gateway CServe email to The Greater Out Here — that damnable embedded comma was a huge source of confusion for users…

.. Yes, the origin of CompuServe user IDs were TOPS-10* PPNs. A pair of octal half-word (18 bit) unsigned integers. The CompuServe Information Service started as a way to sell excess computer time on the timesharing systems that were used by businesses during the day. The Information Service eventually took over the company…

… The PC software was originally developed by a user to make interfacing to the DECsystem-10* command line a little easier. While they (and their partners) developed some great ideas, they failed to sufficiently invest in both marketing and user interface development which allowed AOL to come from nowhere, flood the marketplace with free floppies, and dominate the market in very short order. Being owned by H&R Block at that crucial juncture didn’t help, either…

*TOPS-10 was the operating system, DECsystem-10 was the hardware (36-bit word with a settable byte-size)…

… Up until they did this, I was still paying a legacy $2.50/mth fee for Compuserve and my old account could still log in to the service at gateway.compuserve.com via telnet.

You could not do much in there of course anymore, but I was also once a sysop, and I still knew how to get into the PRO area, do directory listings of their hard drives (and see files with dates dating back to the 70’s), and with that knowledge run some of the old apps from the command line (like biorythms, and some adventure games), and even things like TE2TRN.EXE (the program that allowed the TI-99/4A TE2 cartridge to transfer files from Compuserve…

… It is true that you can no longer use the PPNs to access the forums on Compuserve, but there are still quite a few of them there. A few even have rather large numbers of messages per day still, though most are pretty small now. But anyone can access the forums, using any browser, and can participate in the forums as much as he/she wishes. You are required to have a ’screen name’ which can be an AIM name, an AOL name, or a Netscape name….

BUT - many of the old forums continue to exist and to serve the small number of people who come. I’m still in the Genealogy Forum and the Vintage Computers Forum. Many of the sysops you all knew are still there…

So until recently, there were still people posting on the CompuServe forums. There's a good eulogy here as well. I wonder if Facebook will last as long as CompuServe.

Green shoots on climate change

The infamous green shoots of economic recovery are dust now, but I feel twinges of optimism about climate change.

A few weeks ago I wrote …

Gordon's Notes: Human progress and global climate change – are we good enough?

We are not what we were 20,000 years ago. We are not the people of 2,000 years past.

Hell, we’re not even the people I was born to.

We’re better than we were.

We’re better at damned near everything. I don’t know the how or why, but we’re still around 50 years post-fusion weapons. We got rid of Freon. We don’t routinely torture children in public schools. We have the ADA. We don’t smoke on airplanes. We have Obama. Gay unions, by whatever name, are inevitable….

…I think that if the climate change riff on our smoldering Malthusian crisis had come along in 2060 that we’d be ok. Fifty more years of Singularity-free progress and we’d be ready to handle our CO2 problem.

Except it isn’t 2060, and we’re struggling big time. The US Congress has passed a bill that gets us about 5% of the distance, and the Senate is expected to suffocate it. To add injury to injury, those who argued against the bill were babbling gibberish…

Even then I came down on the side of mild optimism. Since then I’ve actually become more optimistic.

Why?

Well, first, there is that bill. Sure the Senate may kill it, but it was an admission. It’s like the first Surgeon General’s report pointing out that smoking wasn’t really a healthy habit. The bill doesn’t change much, but it changes everything.

The second came from Grame Wood’s Atlantic article on geo-engineering (aka terraforming or climate engineering). There are two advantages to the geo-engineering track. One is that it gives nature hating Republicans a face saving way to admit there’s a CO2 problem. Face saving because they can acknowledge the problem while still offending tree huggers and continuing to pave paradise. That’s progress – of a sort. More importantly, however, is that geo-engineering is a low cost weapon of mass destruction …

..The scariest thing about geo-engineering, as it happens, is also the thing that makes it such a game-changer in the global-warming debate: it’s incredibly cheap. Many scientists, in fact, prefer not to mention just how cheap it is. Nearly everyone I spoke to agreed that the worst-case scenario would be the rise of what David Victor, a Stanford law professor, calls a “Greenfinger”—a rich madman, as obsessed with the environment as James Bond’s nemesis Auric Goldfinger was with gold. There are now 38 people in the world with $10 billion or more in private assets, according to the latest Forbes list; theoretically, one of these people could reverse climate change all alone. “I don’t think we really want to empower the Richard Bransons of the world to try solutions like this,” says Jay Michaelson, an environmental-law expert, who predicted many of these debates 10 years ago.

Even if Richard Branson behaves, a single rogue nation could have the resources to change the climate. Most of Bangladesh’s population lives in low-elevation coastal zones that would wash away if sea levels rose. For a fraction of its GDP, Bangladesh could refreeze the ice caps using sulfur aerosols (though, in a typical trade-off, this might affect its monsoons). If refreezing them would save the lives of millions of Bangladeshis, who could blame their government for acting? Such a scenario is unlikely; most countries would hesitate to violate international law and become a pariah. But it illustrates the political and regulatory complications that large-scale climate-changing schemes would trigger…

So all those island states and African nations that will be destroyed by a 11 degree F rise in temperature have a card to play. They can nuke the sun, so to speak.

Call me a cynic, but I believe climate weapons will concentrate minds more effectively than a hundred pleas for common humanity.

The third green shoot come from a recent post by James Fallows …

semi-encouraging_climate-change

…The speakers were Thomas Lovejoy, a long-time biodiversity expert, and David Hayes, who has recently become the #2 official in the Department of Interior.

Lovejoy's presentation began with a reminder of all the bad things that are happening to wildlife, to biodiversity, to life in the ocean, etc as CO2 levels in the atmosphere go up, taking temperatures with them. But … he emphasized how huge a role the Earth's own natural processes and vegetations -- its forests, grasslands, wetlands, even deserts -- can play in absorbing much larger quantities of carbon from the atmosphere than they do now and thereby reducing the greenhouse effect…

… He tied this analysis to perhaps the most frequently-used chart in modern climate-change thinking -- one produced by McKinsey & Co and the McKinsey Global Institute comparing the relative costs of different measures to reduce greenhouse gas (GHG) levels in the atmosphere.

On the chart, the below-the-line items, on the left side, are GHG-reduction measures that save more money than they cost. Most of these are sheer efficiency measures (insulating buildings, switching to more efficient lights). The above-the-line escalating figures on the right are the rising costs of other abatement measures. The most expensive of them are high-tech "carbon capture and sequestration" systems, plus protecting forests in heavily-populated Asian countries.

Lovejoy's point was that a lot of "re-greening" steps are near the middle of the chart, either actually saving money or costing very little compared with a variety of clean-energy technologies…

… then Hayes stepped up with what was news to me. This was the announcement that the Department of Interior … is now quite serious about applying a "Re-greening" approach to the 20 percent of the US landmass under its control.

Hayes gave more details than I will recount here. They boiled down to a sequence of: trying to measure and understand the carbon-absorption properties of the various lands under its control; seeing how they can be improved, including with market-based offsets; telling the story to the public of why protecting and expanding forests, grasslands, wetlands, etc has an important climate-change component; making forest-preservation an important part of international climate negotiations (rather than talking only about clean-energy sources); and a lot more. (Including changes in U.S. agriculture, which are of course outside Interior's direct control, so that instead of being, incredibly, a net emitter of greenhouse gases, it has a positive effect. This is related to the Food, Inc. discussion of industrial agriculture mentioned here.)

.. it was surprising enough to hear from a senior DOI official and seemed politically and psychologically shrewd, in letting people think that there was some reaction to dire greenhouse gas projections other than holding their hands over their ears and wishing the whole problem would go away.

So we’ve got three green shoots. We’re painfully, slowly, moving to admit we have a very big problem. We’ve realized that poor nations in the path of the climate juggernaut have a (potentially lethal) card to play. And, lastly, a Rational President means we have a Rational Department of the Interior thinking about how humanity can win this one.

Today I’ll be optimistic … about climate change.

Health care? I still don’t see Americans coming to terms with the real options.

Alaska - stranger than you can imagine

Back when Palin was threatening the future of human civilization in her run for a post-McCain presidency, I recall an Alaskan trying to explain where she came from.

I don't recall the details, but the gist of it was that Alaska is very small, very eccentric, and surprisingly tolerant in an incoherent libertarian-welfare sort of way. It sounded like a place I'd enjoy, though maybe not with kids.

I thought again of that essay when I first read this mornings WSJ resignation coverage claiming Palin had a 92% approval rating in early 2007, and then read ...

Sarah Palin resigning as Alaska governor - Joan Walsh - Salon.com

.... In an angry, rambling press conference that will rival Gov. Mark Sanford's as a stunning example of a bizarre public meltdown, Palin basically blamed her decision on her national critics, who she said were blocking her agenda and costing Alaska taxpayers money.

"You are naïve if you don't see a full court press right now on the national level picking apart a good point guard," Palin said, a reference to her days as Sarah Barracuda, high school basketball star. What does a good point guard do? "She drives through a full court press protecting the ball, keeping her head up…and passing the ball so her team can win. I know when it's time to pass the ball for a win.

"I really don't want to disappoint anyone with this decision," Palin continued. "I cannot stand here as your governor and allow millions of dollars to go to waste. I don't know if my children are going to allow it either…This decision comes after a lot of prayer and deliberation." Palin said all of her children endorsed her decision, and she closed by complaining about people mocking her Down's Syndrome son Trig, with little Piper standing by her side.

"In the words of General MacArthur, we are not retreating, we are advancing in another direction," Palin said, as she turned the podium over to the apparently shocked Lt. Gov. Sean Parnell.

There was rolling hilarity and a total news vacuum on television for about 10 minutes after the news first broke. CNN's Rick Sanchez wondered aloud if Palin could be pregnant again – shocking Candy Crowley – before interviewing Frontiersman reporter Andrew Wellner, who says the press conference came as a total surprise to local reporters.

"She didn't take any questions, she said she could be more effective outside of government," Wellner said, reading his notes to Sanchez. Then CNN got tape of Palin's announcement...

.... CNN is now running the entire speech; earlier, it only ran a clip from her resignation statement onward. It's crazy stuff. For the first 10 minutes or so, Palin rambled weirdly about all the good things she's done for Alaska, on energy and budget issues, sounding kind of like a Furby who memorized a lot of information but has no idea how to repeat it in a human-like way. The tone and inflection were completely off...

There's lots of speculation on her plans, including my favorite -- that she doesn't really have a plan other than to make tons of money, duck a lot of investigations, and see what happens next.

I still think we should be mildly terrified of her. Any country capable of reelecting Bush/Cheney in 2004 is clearly capable of electing Sarah Palin in 2012 or 2016.

Low grade terror aside, this makes Alaska even stranger than I'd thought. This Governor had a 92% approval rating?! Nobody gets those kinds of approval ratings in a democracy. Even recently she was running at something like 60%, which is landslide-level for most politicians.

I really do hope I get to visit Alaska soon.

Visit to a lost world

I have returned from a visit to a lost world. I must tell my story swiftly, for the Guardians of the secret ways know me now. Only a fraction of our expedition notes have survived, little can I show of that mysterious land of Shangri La.

I came upon it by chance, accompanied only by my faithful hound. The way is only paces off a well worn path, but it can be approached only when waters run low and the blood sucking guardians sleep. By chance then came we fell upon the ancient steps.

Long we climbed that broken way, thinking of the men who labored there and the blood sacrifices that sanctified their labors.

We passed by meadows whose peace belied the eldritch history of this place. Then the rustling of the winds

became the roaring of a great cataract.

A monstrous bridge, broken by time, spans the might river above the falls.

Its might beams lay shattered on the rocks below. How many eons had passed since that span stood?

At last, at the very ends of our strength, we emerged atop the vastness to uncover yet another route to the forgotten empire. So many years had I passed that way, never knowing what terrifying mysteries lay just beyond the edge of mere civilization.

We returned to our encampment by a twisted path, and there discovered evidence of the Guardians!

The roped cliff face could only mean others came this way! In shock, we turned to flee, but it was too late. A Guardian nodded to us as she and her slavering beast walked by. We were marked. There are those who know this way!

I write these words hurriedly, for Guardians are jealous of their ancient secrets. I send you my last words now by evening post, keep them well. Beware the Guardians of the hidden falls!!

Above average: Shangri La on the Mississippi

Best urban cycling.

Best healthcare.

Fittest kids: "Another victory of sorts for Lake Wobegon: Minnesota has the lowest rate of overweight children in the nation, according to a report on obesity in America."

Who knew?

Google: Please fix Gmail's broken threading model

I'm going to keep posting variations of this every 6 weeks until Google surrenders.

Dear Google,

Gmail's threading model was a cute idea. I suppose it would even work if you hadn't hidden subject lines in threads, thereby promoting subject line reuse and thread metastasis.

It doesn't work. Honest. One day you'll agree with me.

As a compromise, let me change the subject lines in the messages people send me (Outlook's killer feature). If you like, you can propagate the new subject line down the thread.

Alternatively, let me easily toggle your thread by subject line feature.

Thank you.

I'll write again.

Yours,

john

An old mystery – why did Saddam block UN inspections?

We now all know that Cheney and Bush were looking for a reason to invade Iraq even as the UN sanctions were crumbling. We also know that Saddam really didn’t have any WMDs to hide.

So why did Saddam prevent UN inspectors from doing their work? He might have held off the US invasion and waited out the rapidly crumbling UN sanctions. Why did he give Bush the excuse he was looking for?

One 2003 theory was that Saddam thought he had weapons of mass destruction – he thought he had something to hide. Maybe his military was lying to him to save their own skins. By 2006 the public theory was that that Saddam himself knew there were no WMDs, but he was  hiding this fact from many of his aides – for fear of revealing weakness to Iran.

Today an FBI report provides more details. I found Saddam’s comments on a “pact with a US enemy” persuasive …

Newly released FBI reports describe Saddam Hussein's reasons for refusing UN inspectors to enter Iraq | World news | guardian.co.uk

Saddam Hussein remained preoccupied with the threat from neighbouring Iran as the US-led invasion loomed and would have sought a security pact with the US if UN sanctions were lifted, he told an FBI interviewer in his jail cell before his execution.

In more than two dozen interviews and casual talks, the deposed Iraqi leader told FBI questioners that he refused to allowed UN inspectors to re-enter the country because he feared they would reveal to his chief adversary Iran the severely degraded state of Iraq's weapons capability.

Saddam, whom the successor Iraqi government hanged in December 2006, also denied having any connection to Osama bin Laden or al-Qaida, and said that if he wanted to join forces with a US enemy, he would have sought a pact with North Korea or China…

… The reports were released by the National Security Archive, a Washington group that obtained them from the FBI. The reports contain a few deletions, and one interview, from May 1, 2004, was redacted in its entirety…

.. He said that during the run-up to the US invasion in March 2003, he kept up his bluster about weapons of mass destruction in order to appear strong in front of Iran. Saddam said he believed Iran intended to annex majority Shia areas of southern Iraq, and saw the country as the greatest threat to Iraq. He said he viewed the other Arab countries in the region as weak and unable to defend against an attack from Iran. He said that he refused to allow UN inspectors to re-enter the country not because he still possessed prohibited weapons of mass destruction (he ordered the stock pile destroyed after the 1991 Persian Gulf war) but because he wanted Iran to believe he did.

"Hussein stated he was more concerned about Iran discovering Iraq's weaknesses and vulnerabilities than the repercussions of the United States for his refusal to allow UN inspectors back into Iraq," the report of a June 11, 2004 interview states.

Asked how Iraq would have dealt with Iran if the UN inspections and sanctions were ended, he said he would have sought a security agreement with the US. Piro agreed such an arrangement would have benefited Iraq, but said the US would not quickly have made such a pact. He told Piro he wanted a more friendly relationship with the US, an ally during the war with Iran, but that the US "was not listening to anything Iraq had to say"…

Saddam was definitely evil, but he wasn’t crazy.  Instead Saddam’s big mistake was thinking that Cheney/Bush were as calculating as he was, when, in reality, they were … crazy.

Wanted: A consumer-geek friendly triple store database management solution

For the hundredth time I contemplated the challenges of managing personal data with general purpose database management tools (FileMaker Pro for OS X and XP, Microsoft Access for XP) and special purpose database management tools (Address Book and iCal for OS X, Google Contacts and Calendar, Outlook for XP, 1Password for iPhone and OS X).

It’s really an intractable mess. There’s no practical way to move data between these stores (tab delimited files were never enough), much less synchronize data. Everything is in proprietary data stores – fundamentally hostile to data freedom.

Then a little bulb went off. Lately I’ve been slogging through Allemang and Handler’s “Semantic web for the working ontologist”. It’s heavy going, not least because the further you get from the early chapters the greater the underlying uncertainty. Will we really use OWL and SPARQL ten years from now?

On the other hand, the early stuff is persuasive, especially the descriptions of the RDF Triple Store and the RDFS data definition and transformation language. The above wikipedia link, as of today, doesn’t do justice to the real value of this work. To me the fundamental appeal is that there’s a solid and increasingly accepted specification for a universal data representation store that not only encodes data, but also metadata – the equivalent of column names, data types (strings, numbers, etc), table names, relationships and so on.

This stuff was created over the past 20+ years to integrate “say anything, anywhere” data sources across the Net [1]. The movement has been popularized by Tim (web ground zero) Berner-Lee in lots of semantic web presentations.

So we now have a standard way to represent data and data relationships, to manipulate both things across all platforms, the RDF Triple Store + RDFS combination. Heck, it can even manage graphs!

Bingo.

That’s how I want my data to live, whether it’s my Contacts or my Calendar or my Passwords everything else that’s not a document. Finally I’ll have the tools I need to, you know, manage my contacts and passwords and so on.

Sure, performance is an issue – but I’m not dealing with a trillion records. I’m dealing with a few thousand records. Stick the damned structure in RAM and build indices galore – I don’t care. The key thing is having a standard way to store and manipulate my data.

So all you niche vendors out there, all you guys that want to take the market away from FileMaker and especially Microsoft – give me a consumer-grade (think Bento – but less stupid) Triple Store based solution that will let me work with data across platforms and applications.

I’d appreciate it.

[1] Anyone remember Apple’s “HotSauce” Project X MCF work from the early 1990s? MCF moved to XML format then to Netscape and later joined the RDF development stream.

What gene studies reveal about the diversity and resilience of mind, and the limitations of psychiatric disease classification

This research further confirms results first published in Science three months ago. The continued blurring of the diagnostic boundaries between manic-depression and schizophrenia is significant, as is the relationship to control of the immune system. It has always been interesting to contemplate similarities between the immune and nervous systems …

BBC NEWS | Health | Gene clues to schizophrenia risk

Scientists have identified thousands of tiny genetic variations which together could account for more than a third of the inherited risk of schizophrenia…

… The findings came from work by three separate teams, who analysed DNA from thousands of people.

The studies - the biggest ever into the genetics of schizophrenia - appear in the journal Nature.

The findings suggest that schizophrenia is much more complex than previously thought, and can arise not only from rare genetic variants, but common ones as well…

… The researchers say that individually many of the genetic variations they have identified play only a tiny role in raising the risk of passing schizophrenia down the generations … "Cumulatively, they play a major role, accounting for at least one-third - and probably much more - of disease risk."

All three studies highlight genes found on Chromosome 6 in area known as the Major Histocompatibility Complex, which plays a role in the immune system, and in controlling when other genes are switched on and off.

The researchers believe this might help explain why environmental factors also seem to affect risk for schizophrenia…. For example, there is evidence that children whose mothers contract flu while pregnant have a higher risk.

In total the researchers identified 30,000 tiny genetic variants more common in people with schizophrenia.

A similar pattern was found in people with bipolar disorder…

… Dr Thomas Insel, of the US National Institute of Mental Health, said: "These new results recommend a fresh look at our diagnostic categories.

… "If some of the same genetic risks underlie schizophrenia and bipolar disorder, perhaps these disorders originate from some common vulnerability in brain development." …

It feels like we’re making a genuine leap forward in our understanding of the what some call the “connectopathies”. One of the more immediate implications is that we’re driving another nail into the increasingly problematic classificationd (nosologies, as in DSM IV) (see – psychiatric diagnoses, 200 years behind) of disturbances of the mind.

The BBC article claims that this is a new discovery of similarities between schizophrenia and bipolar disorder, but I posted on that in 2003 …

BBC NEWS | Health | Mental illnesses share gene flaw (September 2003)

… Sabine Bahn, who led the research, published in The Lancet, said: "We believe that our results provide strong evidence for oligodendrocyte and myelin dysfunction in schizophrenia and bipolar disorder.

"The high degree of correlation between the expression changes in schizophrenia and bipolar disorder provide compelling evidence for common pathophysiological pathways that may govern the disease phenotypes of schizophrenia and bipolar disorder." …

In particular childhood onset “bipolar disorder” seems to have a great deal in common with early onset schizophrenia.

In a more abstract direction this research provides some tantalizing hints about the diversity of mind/brain, and the emergent resilience of the brain/mind. Thousands of gene variants all affecting minds in unpredictable ways.

By comparison we can see the diversity of our bodies, but they’re really all somewhat familiar. From a distance a parade of human forms is not that interesting. Four feet to seven feet tall, a bit of pigment variation, some muscular variability – but really, not so different. Dogs are much more diverse.

I suspect though, that a parade of minds would be far more interesting. Minds twenty feet tall and 6 inches tall, minds wide and minds narrow …

The resilience of the mind is also reinforced by this study and the earlier Science article. One in twenty people have big, ugly, mutations that ought to mess their minds completely – and yet they function very well (at least in mid-life, who knows about senescence). Somehow our minds are able to construct themselves from a very diverse and often severely flawed substrate.

That last point is what I find most interesting …

For further reading, see also the links associated with this 2008 post of mine.