Living with technology regressions in the post-performance era

I'm slowly getting used to living with the post-Moore's Law era of technology regressions.

It's taken a long time to get over my early computing experience. Switching from an 8086 to an 80386 in the DOS era was pretty much pure progress. The transition to OS/2 then to Windows 95 involved many regressions, but I imagined that was a one time anomaly. Win 98 to NT to 2000 to XP was pretty much all improvement (I skipped ME of course).

Same story in my early Mac days. Things just got better - until MacOS 7 ran into TCP/IP. That was a train wreck, but it did get sorted out. When I returned to the Mac I was using OS X 10.1 (or 10.2?) and that was good too.

Ok, so some great software died without replacement. I should have adjusted my expectations. What can I say? I'm a geek. CPUs kept getting faster. It helped me overlook a lot of things.

Alas, the decrepit state of the Wikipedia entry on Moore's Law speaks volumes. We may get more transistors, but clearly we're not getting more performance. We're in the post-performance era.

In our new era some things get better, some things get worse. Personal computing is middle-aged. Progress is uneven.

I'm in the midst of one of those tech churn transitions now with my backup systems.

I'm not paranoid about backups, because the universe really is trying to destroy my data. I'm just realistic.

Realism means I've long had fully automated rotating off-site backups, and, as backup software quality has regressed, I've moved to having two completely distinct automated backup systems. (If two distinct systems each have a 90% reliability rate, then the probability one will work is 1-(0.1*0.1) or 99.9%. It's almost impossible to equal that reliability from a single affordable product.)

So I probably still have a reliable backup system, but it's more work to maintain than my old system. In some ways it's also less flexible, in particular my laptop backups are less reliable. I'm having to adjust my workflow to the new environment, and that means some functional regressions.

Middle-aged post-Moore computing means living with regressions. The trick is realizing when a true functional regression has occurred, and then being able to say good-bye to the better for the sustainable.

Update 1/25/10: I just found this 2006 tech post of mine complaining about the backup market. It's been a bad few years for backup. I should also highlight a comment Andrew W made (see below):

On Windows there's Home Server, which is about as carefree a centralized/networked backup solution as you can get.

The Apple equivalent to Home Server is Time Capsule. I would like to see Apple do a more complete backup/media server/file server home server solution.

xkcd

I'm a late comer to Randall Munroe's xkcd.

Wikipedia tells me the web comic will be five years old next September.

I'm sorry I've missed out, but both the Achive or the feed are helping me catch up. The loving Wikipedia article includes many annotated comic citations.

Munroe is a genius. He must also be a rebel, for it's not at all clear how he makes any money. He reportedly donated all the proceeds from his one book, and there are no ads on the immensely popular web site.

Phishing with the post-Turing avatar

Blip. The wall is showing the bright smile of my personal medical attendant. Just on time, as always.

I'm delighted to see her, and I can see her quite well thanks to my new lenses and corneal stem cell transplants. My 90 yo brain doesn't work that well, but my eyes are better than they've ever been.

She's lovely to my eyes. Early middle-aged, asian with a dazzling smile I work for. If my brain were forty years younger I'd realize she looks quite a bit like my daughter.

Katie gives me a big smile. She chats about my medications and exercise program. I've been working hard -- I don't want to disappoint her. That part of the conversation goes quickly. We have time to chat. I know Katie works for my health care insurance program, but we always have time to talk about my life and especially about her life.

If I had a younger brain, I'd wonder why Katie has so much time to talk. But I don't.

Katie is sad today. It's not the first time, but this time she opens up. Her son is ailing. He reminds me of my own younger boy. I've got to help. Katie resists, but I insist on authorizing a credit transfer ...

This is so going to happen. Even if we don't get Skynet before 2050, we'll definitely have three dimensional avatars that can pass the Turing test of a lonely mildly-demented 90 year old. They'll provide many services -- such as encouraging medication compliance. The real return, however, will be on launching massive phishing attacks against billions of people.

See also:

• Slouching towards Skynet (Russian slutbots in 2007).
• Evolving Skynet by Gwap (Nick Carr)
• The evolution of comment spam - from parasite to symbiote?
• Google's Goals: Skynet 1.0
• The cat brain simulator. Game over?
• Rat brain flies plane -- organic neural network (2004)
• Aaronson critiques Kurzweil and the 2045 Singularity
• IEEE - The Singularity Issue
• The Economist predicts an early Singularity
• All the vulnerable people: eFraud, aging and special needs

Update: xkcd - suspicion.

Health care. We lost.

Sometimes, the good guys lose.

We lost this one. We're no closer to universal good enough care than we were four years ago. Maybe further.

It was close. In retrospect, with perfect knowledge, there were alternative routes that might have worked. The route we took had too many opportunities to fail.

I'm saddened, but not surprised. I thought the quality of discussion during the failed Clinton reform was poor (for which I blame Hillary actually), but it was golden compared to this go round. The vast majority of educated middle class Americans had absolutely no idea what was going down. No idea ... and little interest. Without that core support the politics were awful.

So what happened to the educated middle class? Age is a part of it. We're an older, graying, fear-filled nation in transition. The boomers imagine medicare will be there, and gray American hates change.

The collapse of the fourth estate was another contributor. In the 1980s media coverage of the Clinton plan was superb; twenty years later it was almost worthless. It was easy for a fearful, sclerotic, population to stop paying attention.

Above all, though, I finger the same mixture of complexity and corruption that led to the Great Recession. We're paralyzed.

So now what?

It goes to the states. The only congressional action that would help at this point would encourage states to experiment widely and to create inter-state health care plans.

At the state level, I expect real change to come under Republican governors. Only Nixon could go to China, and perhaps only GOP governors can transform American health care.

So maybe losing isn't all bad. I've long believed achieving affordable universal health care in America would require the same kind of massive disruption that destroyed General Motors. That kind of disruption is not politically feasible; but markets will do it. The best GOP governors are capable of wedding the destructive forces of markets to socially desirable outcomes. That's the path that's left to us now.

Google trouble: Blogger and Search

Google’s getting a lot of flack for a less than spectacular launch of the gPhone (Nexus One). I’m not too worried about that, I think they’ll get it working. I’m also still optimistic about the Chrome OS netbook – though if it costs over $200 I’ve got yet another public mea culpa waiting.

On the other tentacle, I am getting bad feelings about two Google services I have long relied on – Search and Blogger. I think the problems may be related.

Blogger is the proverbial coal mine canary. It is clearly not thriving. There’s still no iPhone or DROID app for posting or editing, there’s an undocumented and unfixed 5000 post limit, there’s no mobile-optimized version of blog pages, the BlogThis bookmarklet was never updated to support categories, the rich text editor has many longstanding bugs, there’s no spam detection on comments, the Blogger in Draft blog was silent from Nov 28 through Jan 20 (yesterday!) and so on.

Why isn’t Google investing in Blogger? My best guess would be some mix of

• Inability to manage Blogger spam blogs (splogs)
• High success rate of search index poisoning comment spam
• High rate of click fraud related to Blog associated adwords
• Low rate of revenue from Blogger adwords
• Declining readership numbers
• Failure of the confusing “Follower” and Google Reader note/comment programs
• Confusion from the rise of Twitter (confuses me too) and Facebook

Several items on my speculative list implicate search index poisoning problems. These “Search engine optimization” scams degrade search results, which leads to a spiral of click fraud and declining ad word revenue.

Which brings me to the bigger Google problem. The quality of the search results is deteriorating. On technical topics that I search on, I’m getting a large number of junk web sites. I have to use my Google custom searches to find good results. When I search on hard-to-find answers that I know I’ve addressed in my own ad-free tech blog, I don’t get any useful hits at all. It’s not just that I don’t find my marvelous stuff – I don’t find any answers anywhere.

In several instances, Bing has done better. In particular, Bing seems to find fewer splogs and fraudulent ad-heavy pages – perhaps because the scummy SEO gang is still optimizing for Google. (Bing’s time will come.)

Google is only as good as their search engine, and that engine is under relentless attack from the same emergent attacks that killed usenet and severely wounded email. At the moment, the parasites are winning – and threatening to kill their current host.

Google needs a winning response. They’ve got bigger problems than lousy phone service.

Update 1/21/10: See comments for a response from one of Bloggers Product Managers, it's an encouraging rebuttal. Per that comment I corrected the name of the Blogger in Draft blog; the official Blogger blog is http://buzz.blogger.com.

Is there a club for people who hate OS X permissions?

I'm looking for a club made up of people who hate Apple's brain-dead OS X permissions/security scheme.

In the latest installment of OS X misery consider a file on a shared 10.5 drive. Whenever I edit the file from a 10.6 machine it's saved in such a way that my wife loses edit permissions -- even though both she and I have read/write permissions on the parent folder.

OS X needs to abandon its broken unix-style permissions and imitate Windows 7/Vista/XP/2000/NT. (The admin/user issues with Vista to NT weren't related to the permissions model - but that's another post.)

Grrrr. I wish the OS X customer base were way more demanding. Insufficiently demanding customers are one of the three banes of modern commerce (Two others: lock-in and fraud/deception).

See also:

• Permissions bug in OS X 10.5 - unable to update applications
• Gordon's Tech: Permissions: the most messed up part of OS X
• Gordon's Tech: OS X Leopard 10.5: The unknown user and group bugs

Update 1/18/10: No sooner do I write this rant that I have to figure out how to fix a novel permissions hassle related to moving a VMWare Package between users. This stuff is seriously evil.

Update 1/19/09: See comments. Inspired by Andrew W, I dredged up a memory of John Sicracus's famous 10.4 review telling us that Apple was going to fix their broken permissions model years ago! Today in their OS X server marketing you can read (emphases mine) ...

Mac OS X Server supports both traditional UNIX file permissions and access control lists, giving administrators an unprecedented level of control over file and folder permissions. With access control lists, any file object can be assigned multiple users and groups, including groups within groups. Each file object can also be assigned to allow and deny permissions, as well as assign a granular set of permissions for administrative control, read, write, and delete operations. Mac OS X Server supports a file permission inheritance model, ensuring that user permissions are inherited when files are moved to the server and rewritten when files are copied to the server.

ACLs have been used in the Windows world since NT inherited them from OpenVMS. This is one of several areas in which Windows has been far ahead of OS X.

The problem, of course, is that Apple has not provided an equivalent of Tiger's Workgroup Manager GUI in 10.6 standard to work with ACls, and they presumably break a lot of current software. Apple gave up on the 10.6 migration to ACLs, perhaps because of the Intel migration and the introduction of the iPhone OS.

Sandbox provided an ACL control GUI for 10.4 10.5 users, but it's not been updated for 10.6. Apple does allow us to download their Server Admin Tools which can reputedly edit ACLs on non-servers. (It only installs on OS X server.)

See also:

• The ACL Permissions pane: great review, part of the TinkerTools utility.
• Make Mac Work: Control File Access With ACLs

I'll have to continue this one in my tech blog. (BTW, Bing did better than Google at finding these references.)

Update 3/11/2010: Why you shouldn't use OS X ACLs.

Update 3/11/2010b: I try to write to a network share. I run into the 10.6 MobileMe cannot log in as other user bug. Then nothing seems to happen. I have to kill the Finder. On the other machine I discover over 45,000 0 byte files have been written. Permissions bug. I despair.

Update 8/2/10: Take Control book on permissions in snow leopard is out.

--
My Google Reader Shared items (feed)

Why we need Google Book scanning - the End of Eternity

At a small but classic library in West St Paul (which is south of St Paul, but on the "west" side of the Mississippi) I came across a book from my childhood: The End of Eternity by Isaac Asimov (1955). It's a time travel book, full of cliches.

Except they weren't cliches then.

At the above Amazon link you'll find "We don't know when or if this item will be back in stock." That's sad. The End of Eternity is not a classic book, but it's a fun book by a man who wrote a lot, and got good at it.

Over at Google Books we learn that the End of Eternity was digitized Mar 25, 2008 at the University of Michigan. We can't read it though. Under current US copyright law it goes into the public domain at about the end of eternity. (You didn't realize copyright was now effectively eternal? Missed that one eh?)

Google gets a lot of flack for their book project. I'm sure they're imperfect, but I think they're fundamentally right.

Go Google.

Update 1/20/10: Ok, so I could have picked a better example. Charlie Stross tells me I should have looked a bit longer (52 reviews, 5 stars). It seemed like such a good example at the time! In my defense the reviews are quite old, and refer to the book as "hard to find" in 2000.

Update 1/20/10b: Charlie wrote this long post today. Google is not his friend. Mea culpa.

Who killed Instant Messaging?

I know the smell of a dying solution, and IM's got it bad. It's not dead yet, but it's got seven tentacles in the grave.

I came late to IM, so I've only now realized why the party is so quiet. I started with Beejive on my iPhone as an SMS alternative. It worked fairly well, though I ran into server disconnect and message delay problems. Then I started using it with Google Talk at work. There I ran into issues with messages going to one client or the other but not always both.

It wasn't until I started looking at multi-account desktop XP clients, however, that I realized how bad things were. That's where I found cr*pware bearing unwanted toolbars, neglected and buggy open source solutions, walled gardens from AOL and Microsoft, and web apps that want my google credentials (good luck with that).

Yee-uch. I know that smell!

So if IM is dead or dying, who held the knife?

I'm guessing it was a combination of Twitter, SMS/texting bundles, the mobile migration, the unflinching stupidity of Yahoo/AOL/Microsoft/Skype (basically everyone but Google), the non-multitasking iPhone and, above all, the complete absence of any plausible revenue stream [1].

IM, I barely knew ya.

[1] So why are there pretty-good IM clients on the iPhone? Hint.

--
My Google Reader Shared items (feed)

Window resizing - OS X vs. XP

On XP I can get "stuck windows" when I move my laptop between displays. These are windows that I can't resize, because they're too large for me to reach the right lower corner. (I think there are other causes of stuck windows.)

On OS X if I click the green "right size" button windows resize to fit the screen -- without going full screen. So they don't get stuck.

It's a small feature, but the sum of these small things is part of what makes Apple products a pleasure to use.

Alas, as is common these days, there are signs of regression to the lowest common denominator. iTunes doesn't work properly, and when Apple tried to make the "right size" button work correctly users rebelled and Apple reverted to the bad behavior (it creates a mini-player instead, you have to option-click to get it to work). Many apps uses to try to guess how to best use the display surface, but now they fill the screen -- which is absurd on a 27" monitor.

Does Windows 7 do anything clever here, or is stuck in the XP world?

--
My Google Reader Shared items (feed)

Haiti: Why I donate via CARE.ORG

During disasters like this it's common to donate to the American Red Cross. Obviously, a reputable place -- but you run the risk of getting spammed, mailed, phoned, etc.

CARE.ORG, a four (max rating) star charity, doesn't harass me - and I've been using them for at least six years. If they do email or contact you, tell them to put you on their no-contact-ever list. It works

They're active in Haiti ...

CARE: Donate Now:

... CARE is deploying additional emergency team members to the devastated city of Port-au-Prince in Haiti, where the worst earthquake in 200 years destroyed houses and left thousands homeless...

CARE will use your money well. Recommended.
--
My Google Reader Shared items (feed)

Innovations in comment spam

Comment spam continues its rapid evolution. Despite my reluctant surrender to the Captcha I'm seeing novel mutations every few months.

A recent technique is to write a reasonably detailed comment about a fairly specific topic, like "junk DNA". A query engine then identifies all blog posts that have a high match to the comment. An automated posting process, perhaps with some tool-assisted human powered captcha processors (via Amazon's Mechanical Turk?), submits the post to thousands of blogs.

Even with human review, the comment submissions will be a good quality match to a meaningful number of blog posts. The comment gets posted, and the spammers get something of value (link referrals?).

The one I rejected today was clumsily written, so it was fairly easy to spot. It contained an unnecessarily specific reference to a "first post", the author name was a marketing phrase, and the grammar and phrasing could have been better. I've probably missed better ones!

We can expect rapid improvement. In time they might evolve to transiently novel insights statistically applied to the right spot at the right time. At that point, would we not welcome them?

In the meantime we do need Google to start filtering these comments the same way they filter email. This particular approach lends itself to statistical filters, and of course the use of author reputation in filtering algorithms. Alas, Google has forgotten all about poor Blogger ...

--
My Google Reader Shared items (feed)

Brave new world: China attacks Google

Based on the phrasing and response, it's clear that Google believes this attack was launched by parties working for the government of China. We can also assume that the "relevant US authorities" (FBI) agree with them. I wonder if the targeted companies used software with similar vulnerabilities.

Official Google Blog: A new approach to China

Like many other well-known organizations, we face cyber attacks of varying degrees on a regular basis. In mid-December, we detected a highly sophisticated and targeted attack on our corporate infrastructure originating from China that resulted in the theft of intellectual property from Google. However, it soon became clear that what at first appeared to be solely a security incident--albeit a significant one--was something quite different.

First, this attack was not just on Google. As part of our investigation we have discovered that at least twenty other large companies from a wide range of businesses--including the Internet, finance, technology, media and chemical sectors--have been similarly targeted. We are currently in the process of notifying those companies, and we are also working with the relevant U.S. authorities.

Second, we have evidence to suggest that a primary goal of the attackers was accessing the Gmail accounts of Chinese human rights activists. Based on our investigation to date we believe their attack did not achieve that objective. Only two Gmail accounts appear to have been accessed, and that activity was limited to account information (such as the date the account was created) and subject line, rather than the content of emails themselves...

... We launched Google.cn in January 2006 in the belief that the benefits of increased access to information for people in China and a more open Internet outweighed our discomfort in agreeing to censor some results. At the time we made clear that "we will carefully monitor conditions in China, including new laws and other restrictions on our services. If we determine that we are unable to achieve the objectives outlined we will not hesitate to reconsider our approach to China."

These attacks and the surveillance they have uncovered--combined with the attempts over the past year to further limit free speech on the web--have led us to conclude that we should review the feasibility of our business operations in China. We have decided we are no longer willing to continue censoring our results on Google.cn, and so over the next few weeks we will be discussing with the Chinese government the basis on which we could operate an unfiltered search engine within the law, if at all. We recognize that this may well mean having to shut down Google.cn, and potentially our offices in China.

The decision to review our business operations in China has been incredibly hard, and we know that it will have potentially far-reaching consequences. We want to make clear that this move was driven by our executives in the United States, without the knowledge or involvement of our employees in China who have worked incredibly hard to make Google.cn the success it is today. We are committed to working responsibly to resolve the very difficult issues raised.

This may be the end of Google's services in China. We should expect their share price to fall in the morning. Google's "evil score" has now dropped to the lowest possible level for a public corporation.

Update 1/13/10: There's a lot of commentary this morning, including comparisons to how the USSR hobbled itself by shutting out access to world knowledge. I'm wondering if Google's increasingly powerful and ubiquitous machine translation services played a precipitating role. Language has been the cultural equivalent of the Himalayas - preserving China from cultural invasion. I suspect the Chinese government is very concerned about widespread direct unmediated access to English language materials.
--
My Google Reader Shared items (feed)

Dark matter DNA

Our universe is largely built with matter that shapes large structures, but doesn't interact with electric fields - including light. It's dark matter.

There's a funny similarity to our DNA ...

Borna Virus Discovered in Human Genome - Carl Zimmer - NYTimes.com

...Fossil viruses are also illuminating human evolution. Scientists estimate that 8.3 percent of the human genome can be traced back to retrovirus infections. To put that in perspective, that’s seven times more DNA than is found in all the 20,000 protein-coding genes in the human genome.

In the physican universe dark matter is only about 70% of all matter, but in humans "dark DNA" is 97%+ of all DNA. So our DNA is about 2% protein coding, 8% retrovirus, and 90% other - including non-retroviral virus origin and "structural". (Yes, I know that's "four times" and Zimer says "seven times" - his numbers are more likely correct.)

So from a DNA perspective, are we basically an ambulatory viral ecosystem with a fraction of information capacity that does things like make brains and bodies? Seems a bit much, but it turns out even some of the most important protein coding DNA is of viral origin. In a companion post on his blog Zimmer writes ...

... a virus protein called syncitin ... is essential for placentas to develop. Cells push the protein to their surface, where it lets them latch onto other cells, fusing together to create a special layer through which nutrients can pass from mother to child. The protein got its start on viruses, which use it to latch onto host cells and fuse to them, allowing their genes to slip in.

But recent research has revealed an intriguing new twist to our viral legacy. It turns out that the viral surface protein in question has a second job. It also tamps down the immune system of its host...

So is there any non-structural DNA in humans that's not of fundamentally viral origin?

See also: Presser on the bornavirus article ... UTA News Center

PS. A search on Preeclampsia and bornavirus has 180 hits today, but I think they appear to be loose and coincidental relationships. I didn't see research relating bornavirus-like superinfection triggering auto-immune placental disruption and thus pre-eclampsia / toxemia.

Update 1/30/2010: io9 quotes Frank Kelly: "[T]he human genome has evolved as a holobiontic union of vertebrate and virus... ". A Coral holobiont is "the entire community of living organisms that make up a healthy coral head".

--
My Google Reader Shared items (feed)

Lessons from my leonine chat icon

If you inspect my profile on various OS X and Google systems lately, you'll see a theatrical yawn ...

There's a lesson in the yawn. When I created a new user account on my i5 running 10.6, I chose a standard animal icon. Since it's a family machine, I wanted to choose an icon that would impress the children (didn't work). Hence the lion.

I then connected that account to my MobileMe account and, just as I found on 10.5 11 months ago, the login image on the iMac propagated to all my MobileMe associated machines, wiping out whatever I had there.

It ate them.

Then, after I fiddled with iChat and Adium, it propagated to Gmail and GoogleTalk/Video Chat and the wider world.

None of this is documented of course. It just happens. It's an emergent behavior; a side-effect. One bit of whimsy, and bam -- I'm a lion everywhere.

There will be more of these things in years to come. More strange leakages and propagations.

If you want something private, keep it on paper. And keep the paper out of range of Vicon Revue wearing lifebloggers ...

Update 1/12/10: Today I notice the OS X 10.6 lion has metastasized to my Google Reader Shared By ...

I'm sure this is violating all kinds of copyright laws, but all of my actions were entirely correct. I think I'll just have to get used to my emergent avatar. Maybe he'll appear on my virtual tombstone.

Update 1/18/10: Here it is on my Google Profile.

This is really silly. I'm going to try restoring the GP image and see if it propagates the other way.

Update 2/9/10: Now it's spread to Google Buzz.

Only it's no longer affixed by my gmail address, it's attached to my corporate email!

--
My Google Reader Shared items (feed)

If you're wondering where your money went ...

Still way down from the peak, almost 10 years later ...

Bubbleheads II - Grasping Reality with Opposable Thumbs:

...

S&P 500, June 30, 2000 close: 1455
S&P 500, December 31, 2009 close: 1145
Consumer Price Index, November 2009/June 2000: 1.26

Real price decline: -37.5%...

--
My Google Reader Shared items (feed)