Friday, September 30, 2011

Global warming: Doonesbury and China

This Doonesbury is a keeper, but it's incomplete.

It's not just that corporations and their investor owners (me, you, etc) value 10 year returns more than 100 year outcomes.

It's also that our best data suggests expected 50 year temperature and sea level changes will be manageable within the US. Bad news for Arizona, but not so bad for Minnesota. Sure some cities get flooded, but Venice managed. Yes, there is the little problem of massive worldwide conflagration, but Americans expect that anyway.

So don't expect leadership on global warming from Americans or, for that matter, Canadians (go tar sands).

China and India though; they have a problem. Global warming looks very bad for China in particular.

China is going to have to lead.

Deep history and the lifespan of the australian aboriginal

The paleolithic is the time of the pre-historic hominid tool user. It begins 2.6 million years ago with Australopithecine-like hominid tool users and ends with the last ice age 8,000 years BCE [1].

The paleolithic is divided into three variably defined ranges, something like this: [2]

  • Upper: 45,000 - 8000 BCE.
  • Middle: 300,000 - 30,000 BCE
  • Lower: 2.6 million to 100,000 BCE. (The great age of exploration, including, it seems, rafting [3].)

It's helpful to know this, because it's otherwise hard to understand what Caspari is saying in her recent SciAm article. She claims that humans had short lifespans throughout most of the paleolithic ...

The Evolution of Grandparents - Rachel Caspari - Scientific American

... the Krapina Neandertals are not unique among early humans. The few other human fossil localities with large numbers of individuals preserved, such as the approximately 600,000-year-old Sima de los Huesos site in Atapuerca, Spain, show similar patterns. The Sima de los Huesos people had very high levels of juvenile and young adult mortality, with no one surviving past 35 and very few living even that long...

...We observed a small trend of increased longevity over time among all samples, but the difference between earlier humans and the modern humans of the Upper Paleolithic was a dramatic fivefold increase in the OY ratio ... adult survivorship soared very late in human evolution...

... Lee and I analyzed Middle Paleolithic humans from sites in western Asia dating to between about 110,000 and 40,000 years ago. Our sample included both Neandertals and modern humans, all associated with the same comparatively simple artifacts. ... We found that the Neandertals and modern humans from western Asia had statistically identical OY ratios...

Caspari claims that the great leap in longevity occurred within the past 45,000 years. She seems to think this was a cultural change, but I don't follow her logic. Most modern hunter gatherers age and die at the same rate as eurasians.

In modern human terms forty thousand years is a long time ago. But recent sequencing of 100 year old DNA suggests the indigenous Australians split from other humans before then ...

Gordon's Notes: Deep history - 40,000 years without change

... Based on the rate of mutation in DNA, the geneticists estimate that the Aborigines split from the ancestors of all Eurasians some 70,000 years ago, and that the ancestors of Europeans and East Asians split from each other about 30,000 years ago....

... the split times calculated by the Danish team are compatible with the more reliable archaeological dates, which record the earliest known human presence in Australia at 44,000 years ago. The Aborigines’ ancestors could have arrived several thousand years before this date.

There are a lot of Google hits on the life expectancy of the indigenous Australian. It is usually estimated at 15-20 years less than euro-Australians. That is not, however, all that different from the life expectancy of modern Russians. The data neither supports nor refutes Caspari's hypothesis, but it suggests things are, as usual, complicated.

Anything about the biology of the Australian aboriginal is very sensitive. It's easy to see why.

-- fn

[1] Via Wikipedia, I have just learned that the scientific practice for dating is now BP for "before present," where the present is arbitrarily assigned to 1950 ACE. I've converted to BCE here.
[2] There's obviously no consensus on where to draw these largely artificial boundaries.
[3] The truly great explorers died before modern humans were born.

Update 1/30/2012: From a NYT review of the state of mongrel man (emphases mine):

... little is known about the Denisovans — the only remains so far are the pinky bone and the tooth, and there are no artifacts like tools. Dr. Reich and others suggest that they were once scattered widely across Asia, from the cold northern cave to the tropical south. The evidence is that modern populations in Oceania, including aboriginal Australians, carry Denisovan genes.

Thursday, September 29, 2011

Forget process, it's the tools

It's a mantra in my world -- Concentrate on People and Process, not Tools.  It's not the tools, it's the process.

Bah, humbug.

Ok. I admit. Given the choice I'd take a good team, and a process that fits the team, over good tools. Problem is, that's a Hobson's Choice. It becomes an excuse to avoid investing in good tools. The phrase is particularly irritating when it's used to disparage the concerns of tool users by those who don't actually use the tools.

Those people feel very differently about the tools they do use. Try swapping their BlackBerries for a Kiln, or Eclipse for Turbo C, and see how how they feel about tools.

Tools matter enormously. They shape the work and the process. Software tools are notoriously "rigid" (software is a very misleading word); if they oppose a process the process will break.

Given the rigidity of software tools choices must be made carefully. Often complexity and power should be traded for flexibility, simplicity, data freedom and reduced switching costs. Early adoption is usually a mistake.

Choose carefully. Be modest about expectations. But never imagine tools don't matter.

Tuesday, September 27, 2011

Local politics - fury at the Jefferson bikeway

This is, on the one hand, a very local story. Local, that is, to my neighborhood of Macalester-Groveland, Saint Paul, Minnesota.

On the other hand, it's a universal story. Life is fractal. The smallest sample of political strife captures almost every detail of the national scene.

It's a story about a well intended program to identify a St Paul road, Jefferson Avenue, as a "bikeway". This doesn't mean bicycle lanes or parking restrictions; it means the city uses federal funds to make the road friendlier to bicycles and pedestrians and reduce through traffic.

This was very welcome along most of the road. The changes will increase property values and make resident life more pleasant. The problem, and it was a loud problem, arose in a wealthier neighborhood. There Jefferson was already somewhat pedestrian friendly, save for two dangerous cross streets -- Cleveland and Cretin.

For some residents along that part of Jefferson there was tremendous anger at the idea of the bikeway in particular and bicycling in general. This anger was stoked by mistakes made by city planners. A community meeting was called, and so, for the first time in my relatively political life, I attended a public meeting.

It was at once a painful and fascinating experience. The painful part came from the yelling and occasional ranting of several of the local opponents. It was almost as bad as sitting through AM talk radio. Their emotions were raw, and initially mysterious. Much of what they actually said was illogical; some spoke as though bands of chain wielding lycra-crazed fetishists were going to be keying their cars.

Sadly, although there was quite a good turnout of supporters, the opposition was much louder and perhaps more numerous. I know several friends of mine with active and busy lives were unable to attend, and that my own attendance will cost me sleep time. A public forum, by its nature, is friendlier to the retired and the inactive.

Pain aside, this was also a fascinating experience. Listening to things said and unsaid, it became apparent that for many of the locals, the real fear was not the packs of rabid riders they spoke of (who'd never go this route of course -- racing packs need space!). It wasn't even the stated anger at cyclists running stop signs.

In reality, and some even admitted this, many of the elderly audience feared injuring a cyclist. Worst of all, injuring a child. They know they, or their spouses, are not the drivers they were. They accept the cost and annoyance of a low speed collision with a car. That's just money. A collision with a cyclist, or even dooring a cyclist, is another matter.

There were other sources of anger. A number, particularly from the less elderly opposition, were angered that any amount of taxpayer money was going to spent on what they perceived as a foolish activity - riding a bicycle. They had difficulty with the concept of their taxes serving anything but their personal wants.

The fascinating bit was to see the emergence of common ground. Even many of those who yelled objections, recognized that as pedestrians they were unable to safely cross Cleveland and Cretin. One opponent, primed by a prior speaker, admitted with some surprise that she'd had to wait for "53" cars to pass before she was able to cross Cleveland. (Minnesota drivers are, by and large, unaware of the state crosswalk law. If they do know it, they pretend not to. We Minnesotans are not particularly good drivers.)

A reasonable compromise seemed, at least to me, to be evident. Both locals and family cyclists would love to have a pedestrian activated crosswalk stop and signage at both Cleveland and Cretin. The north-south drivers, after all, are largely suburban commuters. None of us mind slowing them.

Perhaps we could even forego the bikeway markings and signs. The most contentious road area is, save for the dangerous crosswalks, quite bicycle friendly already. It could become a de facto bikeway even as it became more supportive of resident pedestrians. In time, many of the fears will fade. Everyone, one day, will be happier.

See also:

Saturday, September 24, 2011

Netflix, Amazon, and Dickens

Netflix is getting out of the messy, labor intensive, business of mailing physical DVDs. Now they are purely digital. They won't need postmen or people to fill the gaps machines couldn't manage. They can scale without hiring.

It is the 21st century way.

Amazon can't do this [1]. They cannot, yet, make do with robots. They have jobs for the non-elite. Jobs in a Dickensian world...

The Fraying of a Nation's Decency - ANAND GIRIDHARADAS - NYTimes.com

... Thanks to a methodical and haunting piece of journalism in The Morning Call, a newspaper published in Allentown, Pennsylvania, I now know why the boxes reach me so fast and the prices are so low. And what the story revealed about Amazon could be said of the country, too: that on the road to high and glorious things, it somehow let go of decency.

The newspaper interviewed 20 people who worked in an Amazon warehouse in the Lehigh Valley in Pennsylvania. They described, and the newspaper verified, temperatures of more than 100 degrees Fahrenheit, or 37 degrees Celsius, in the warehouse, causing several employees to faint and fall ill and the company to maintain ambulances outside. Employees were hounded to “make rate,” meaning to pick or pack 120, 125, 150 pieces an hour, the rates rising with tenure. Tenure, though, wasn’t long, because the work force was largely temps from an agency. Permanent jobs were a mirage that seldom came. And so workers toiled even when injured to avoid being fired. A woman who left to have breast cancer surgery returned a week later to find that her job had been “terminated.”

The image of one man stuck with me. He was a temp in his 50s, one of the older “pickers” in his group, charged with fishing items out of storage bins and delivering them to the packers who box shipments. He walked at least 13 miles, or 20 kilometers, a day across the warehouse floor, by his estimate.

His assigned rate was 120 items an hour, or one item every 30 seconds. But it was hard to move fast enough between one row and the next, and hard for him to read the titles on certain items in the lowest bins. The man would get on his hands and knees to rummage through the lowest bins, and sometimes found it easier to crawl across the warehouse to the next bin rather than stand and dip again. He estimated plunging onto his hands and knees 250 to 300 times a day. After seven months, he, too, was terminated...

Which brings me to Bernstein, who echoed a recent post of mine ...

The Policy Backdrop of Inequality and Its Implications for “Class Warfare” | Jared Bernstein | On the Economy

...Technological change, most recently computerization/IT, is also thought to be a significant factor behind the changes in the graph, though the evidence here is more ambiguous.  (One strain of work, for example, argues that technology has increased labor demand for both high skill and low skill work, while leaving out the middle.)..

... Then there’s a bunch of stuff that directly raises or lowers the bargaining clout of middle and working class families—policy changes or missed policy opportunities that have hurt or failed to help them.

–the long-term erosion of the minimum wage
–the absence of legislative protection to balance the organizing playing field for those who want to collectively bargain
–the inattention to labor standards such as wage and hour rules, overtime regs, workplace safety, worker classification ...

We have a diverse population. We are not all equally suited to the narrow range of work that seems America's natural fate. Education is not the answer; we are not all didacts. We need a diverse set of employment opportunities, and we need to prevent the cruelties of the market red in tooth and claw.

[1] Update: Duh. After I wrote this I remembered they started out moving books, and now they ship them by wire.

Deep history - 40,000 years without change

John Hawks and Dienekes have excellent posts on research that emphasizes the churn and chaos of the melding of the sentient apes into one vast world crushing species. The churn and transformations across Europe surprise everyone.

Paleogenetics is awe inspiring.

Both reference one related story; a story not of transformation but of stasis. The sequencing of a 100 year old hair follicle (yes, science fiction lives) tells the long story of the first Australians (emphases mine) ...

Australian Aborigine Hair Tells a Story of Human Migration - Nicholas Wade - NYTimes.com

.... The Aboriginal genome bolsters earlier genetic evidence showing that once the Aborigines’ ancestors arrived in Australia, some 50,000 years ago, they somehow kept the whole continent to themselves without admitting any outsiders...

... Based on the rate of mutation in DNA, the geneticists estimate that the Aborigines split from the ancestors of all Eurasians some 70,000 years ago, and that the ancestors of Europeans and East Asians split from each other about 30,000 years ago....

... the split times calculated by the Danish team are compatible with the more reliable archaeological dates, which record the earliest known human presence in Australia at 44,000 years ago. The Aborigines’ ancestors could have arrived several thousand years before this date.

The Aborigine occupation of Australia presents a series of puzzles, starting with the nature of their stone tools. The early stone tools found in Australia are much simpler than the Upper Paleolithic tools that appear in Europe at the same era...

... the first inhabitants of Australia must have possessed advanced boat-building technology to cross from the nearest point in Asia to Sahul, the ancient continent that included Australia, New Guinea and Tasmania until the rise of sea level that occurred at the end of the last ice age, 10,000 years ago. But there is no archaeological evidence for boats, Dr. Klein said.

Despite the Aborigines’ genetic isolation, there is evidence of some profound cultural exchange that occurred around 6,000 years ago. The stone tools become more sophisticated, and the population increased. The Aborigines did not domesticate plants or animals, but a wild dog, the dingo, first appears in the archaeological record at this time...

... How the dingo arrived in Australia is an “enigma,” Dr. Savolainen writes, because none of the other elements of Polynesian culture are found there...

... Even stranger, dogs always travel with their masters, yet there is no sign yet of Polynesian genes in the Aborigine population.

“Something remarkable happened in Australia 6,000 to 4,000 years ago, and it involved much more than the dingo,” Dr. Klein said.

50,000 years is a long time to be separated for a rapidly evolving animal. In 2007 Hawks told us "We are more different genetically from people living 5,000 years ago than they were different from Neanderthals." The European invaders of Australia must have been very different from the Australian aboriginals of the 19th century; much more different than Neandertals and ancient Egyptians. We will come to understand how different as more data is published from this genome.

There are so many fascinating aspects to this story, but one number stands above all. For about 40,000 years humans lived in Australia, and, as best we can tell, they didn't change. They were much like the humans who lived in Georgian caves for 30,000 years, but they lived into the modern era.

Forty thousand years.

Ten years ago we didn't have iPhones. Within two hundred years we will likely create artificial minds. In the unlikely event that we have heirs in 42,000 ACE they will be completely alien.

Forty thousand years, as ice came and went and oceans rose and fell.

If all other humans had died out, how long would they have lasted there unchanging? With a population of about 500,000 hunter gatherers over a vast territory they would not fall to disease. They faced no real predator threats. They had adjusted to radical climate change. Eventually some mass extinction event, a meteor or supervolcano, would end them -- but perhaps not for hundreds of thousands of years ...

Friday, September 23, 2011

Mass disability and the middle class

My paper magazine has another article on the Argentinification of America - Can the Middle Class Be Saved? - The Atlantic.

I'll skim it sometime, but I doubt there's much new there. We know the story.  The bourgeois heart of America is fading. In its place are the poor, the near poor, the rich and the near rich.

I have thought of this, for years, as the rise of mass disability. In the post-AI world the landscape of American employment is monotonous. There's work for people like me, not so much for some I love. Once they would have worked a simple job, but there's not much call for that these days. Simple jobs have been automated; there's only room for a small number of Walmart greeters. Moderately complex jobs have been outsourced.

Within the ecosystem of modern capitalism a significant percentage of Americans are maladapted. I'd guess about 25%; now 35% thanks to the lesser depression.

There are two ways to manage this - excluding the Swiftian solution.

One is to apply the subsidized employment strategies developed for adults with autism and low IQ. Doing this on a large scale would require substantial tax increases, particularly on the wealthy.

Another approach is to bias the economy to a more diverse landscape with a greater variety of employment opportunities -- including manufacturing. This is, depending on whether you are an optimist or realist, the approach of either modern Germany or Nehru's India. This bias compromises "comparative advantage", so we can expect this economy, all else being equal, to have a lower than maximal output. Since in our world the benefits of total productivity flow disproportionately to the wealthy (winner take all), this is equivalent to a progressive tax on an entire society.

So, either way,  the solution is a form of taxation. Either direct taxation and redistribution, or a decrease in overall growth.

I suspect that we will eventually do both.

See also:

CERN's neutrinos - kudos to the BBC coverage

Amidst a flurry of sensational coverage, the BBC gets special credit for the headline and the quote (emphases mine) ...

BBC News - Speed-of-light experiments give baffling result at Cern:

... Puzzling results from Cern, home of the Large Hadron Collider, have confounded physicists because subatomic particles seem to have beaten the speed of light...

...But for now, he explained, "we are not claiming things, we want just to be helped by the community in understanding our crazy result - because it is crazy".

"Speed of light" is the right headline, not "faster than light". The quote is important too -- they are not claiming things.

It's hard to overstate how outrageous a result this is. By comparison the "cold fusion" claims of 1989 were prosaic. There are many experimental conditions in which various things propagate faster than the speed of light in a vacuum, but those experiments are all consistent with the universe as we model it.

This result is not consistent with our models. True "faster than light" travel implies "causality violation"; a form of time travel where effect precedes cause.

Just based on how science works, rather than any knowledge of physics, I would give 90% odds that this result does not last a month, and 99.9% odds that when all is done the speed of light will survive. Despite that, there is a good chance of normally interesting physics somewhere in there.

If the results hold up, of course, a Nobel would be the least of it.

My G+ Profile - open to connect

My blogs are pseudonymous, but it's not hard to find my true name if you click around a bit. I just prefer that my corporate colleagues and customers don't suffer the full range of my opinions and speculations.

Which means, since G+ is open, I'm good with sharing and corresponding there. I typically share with my "extended circle" (circles and one removed) -- just a bit short of public. I use LinkedIn for purely corporate and teaching connections, Facebook is where I tell kid stories, so G+ is the intellectual slot. It's a good match to the blogs.

If you're interested, here's my profile [1] and my "John Gordon" circle.

[1] Note the URL has the number of this beast: 113810027503326386174. Just call me 113.

(cross posted to notes and tech.)

Thursday, September 22, 2011

Facebook vs. G+: not in the same ballpark

Rebecca Rosen asks Is Facebook Forever?

Of course in our world "forever" is any double digit lifespan.

I might like to think Facebook won't be here in 5 years, but consider my graduate school students.

I asked how many use Google Reader. Three hands went up. Three out of forty or so.

I asked how many use G+. The same three. Geeks like me.

I asked, reluctantly, how many use Facebook. Most hands went up.

Sigh.

Wednesday, September 21, 2011

Google Reader may not be dead after all

Google's Yonatan Zunger implies there's life in Google Reader yet ...

Yonatan Zunger - Google+ - Some more tips and tricks for using Google+ search: *…

"Reader integration is something we're very excited about. A great deal of the Reader team is now, in fact, working on Google+, so there's a lot of crosstalk between the groups."

It's buried in the comments on his G+ tips post.

The neglect of Reader, including a failure to complete half-implemented features like bundles, made me add Reader to my "Dead" list. i've expressed frustration that G+ didn't seem to have learned from Reader. Maybe, I speculated, the Reader team had lost a political battle.

Seems I was wrong. Reader's neglect may have come from the original team moving to G+.

Best news in weeks.

Car and bike: Observations

I've cycled with cars for over 40 years. These days I commute by car most days, by bike weekly. This is what I believe ...
  1. My idyllic bicycle commute is significantly riskier than my car commute. It's not mountain climbing or even motorcycling, but it's a risk I knowingly assume. As the average age of Minnesota drivers heads north, so will my risk.
  2. The main reason many drivers hate cyclists is we push our risk onto them. To a certain level of cycling injury, regardless of fault, I'd rather be be the cyclist than the driver. If the driving fault were mine, I'd choose death. We cyclists choose our risks, but we make drivers assume comparable risks.
  3. Another reason some male drivers dislike male cyclists is we make them feel guilty and inferior. Given what #1 says about our judgment that's illogical, but we're talking about guys here. I feel inferior when I'm in my car and I see a real woman cycling through the snow and ice.
  4. All my life I have been frustrated by black active wear clothing. I suppose it's black because young ones buy active wear and they wish to attract mates. I am not seeking a mate. I am trying to stay healthy. I want more lime-green-yellow reflective clothing. My 4-5 high intensity flashing and probing bicycle lights are offset by black wind pants. I'm hoping this will improve as cyclists age. Ortlieb, to their credit, sells red and yellow bags.
  5. The biggest car-bike problem I face in the age of the bicycle lane is passing a line of stopped cars on the right. Drivers in the right lane hate being passed on the right side. Right side mirrors positioned for driving don't work well for detecting a passing bike. On the other hand, as a cyclist, there's no realistic alternative to passing stopped cars on the right.
  6. It is illegal to cross over a double yellow central line, even to pass a slow vehicle. I believe Minnesota women over 25 prefer not to cross this line. On the other hand, guys like the excuse to be rogue and rough and to feel, for a moment, almost young again. I believe that is why the cars that pass too close to me on a narrow road are usually driven by women.
  7. Inline skaters are to cyclists as cyclists are to drivers. Have mercy on skaters.

Sunday, September 18, 2011

To thine own self be true

It shows up on mugs, posters and greeting cards. To thine own self be true must have a pleasing sound to contented psychopaths.

I assumed it was some woefully misguided advice to embrace one's inner nature. Misguided, that is, for most of us. I haven't spent forty years wailing on my nature for nuttin.

Goes to show it's been a long time since I had any culture. It's Hamlet of course, part of a core dump of fatherly advice that's fathered centuries of repetition (emphases mine) ...

SCENE III. A room in Polonius' house.

... these few precepts in thy memory

See thou character. Give thy thoughts no tongue,

Nor any unproportioned thought his act.

Be thou familiar, but by no means vulgar.

Those friends thou hast, and their adoption tried,

Grapple them to thy soul with hoops of steel;

But do not dull thy palm with entertainment

Of each new-hatch'd, unfledged comrade. Beware

Of entrance to a quarrel, but being in,

Bear't that the opposed may beware of thee.

Give every man thy ear, but few thy voice;

Take each man's censure, but reserve thy judgment.

Costly thy habit as thy purse can buy,

But not express'd in fancy; rich, not gaudy;

For the apparel oft proclaims the man,

And they in France of the best rank and station

Are of a most select and generous chief in that.

Neither a borrower nor a lender be;

For loan oft loses both itself and friend,

And borrowing dulls the edge of husbandry.

This above all: to thine ownself be true,

And it must follow, as the night the day,

Thou canst not then be false to any man.

Obviously, since it's part of a load of self-improvement advice, Polonius is not advising mere acceptance of one's nature. In the language of the time "to thine ownself be true" means "know yourself" -- don't deceive yourself. Which is the mission of middle age, so not too bad.

The corrollary that "knowing oneself" means one will be entirely trustworthy is, to put it mildly, naive. On the other hand, most of Polonius advice is more high minded that practical. I don't think Shakespeare thought he was passing on advice for the ages. He'd have been amused to find it on 21st century mugs.

Life in the post-AI world. What's next?

I missed something new and important when I wrote ...

Complexity and air fare pricing: Houston, we have a problem

... planning a plane trip has become absurdly complex. Complex like choosing a cell phone plan, getting a "free" preventive care exam, managing a flex spending account, getting a mortgage, choosing health insurance, reading mobile bills, fighting payment denials, or making safe product choices. Complex like the complexity collapse that took down the western world.

I blame it all on cheap computing. Cheap computing made complexity attacks affordable and ubiquitous...

The important bit is what's coming next and now in the eternal competition.

AI.

No, not the "AIs" of Data, Skynet and the Turing Test [1]. Those are imaginary sentient beings. I mean Artificial Intelligence in the sense it was used in the 1970s -- software that could solve problems that challenge human intelligence. Problems like choosing a bike route.

To be clear, AIs didn't invent mobile phone pricing plans, mortgage traps or dynamic airfare pricing. These "complexity attacks" were made by humans using old school technologies like data mining, communication networks, and simple algorithms.

The AIs, however, are joining the battle. Route finding and autonomous vehicles and (yes) search are the obvious examples. More recently services like Bing flight price prediction and Google Flights are going up against airline dynamic pricing. The AIs are among us. They're just lying low.

Increasingly, as in the esoteric world of algorithmic trading, we'll move into a world of AI vs. AI. Humans can't play there.

We are in the early days of a post-AI world of complexity beyond human ken. We should expect surprises.

What's next?

That depends on where you fall out on the Vinge vs. Stross spectrum. Stross predicts we'll stop at the AI stage because there's no real economic or competitive advantage to implementing and integrating sentience components such as motivation, self-expansion, self-modeling and so on. I suspect Charlie is wrong about that.

AI is the present. Artificial Sentience (AS), alas, is the future.

[1] Recently several non-sentient software programs have been very successful at passing simple versions of the Turing Test, a test designed to measure sentience and consciousness. Human interlocutors can't distinguish Turing Test AIs from human correspondents. So either the Turing Test isn't as good as it was thought to be, or sentience isn't what we thought it was. Or both.

Update 9/20/11: I realized a very good example of what's to come is the current spambot war. Stross, Doctorow and others have half-seriously commented that the deception detection and evasion struggle between spammers and Google will birth the first artificial sentience. For now though it's an AI vs. AI war; a marker of what's to come across all of commercial life.

See also:

Update 9/22: Yuri Milner speaking at the "mini-Davos" recently:
.... Artificial intelligence is part of our daily lives, and its power is growing. Mr. Milner cited everyday examples like Amazon.com’s recommendation of books based on ones we have already read and Google’s constantly improving search algorithm....
I'm not a crackpot. Ok, I am on, but I'm not alone.

Saturday, September 17, 2011

Complexity and air fare pricing: Houston, we have a problem.

Early in my life air travel was almost as expensive as today. At that time, however, we had travel agents and competitive service. It was hassle free.

Later air travel was inexpensive and hassle free. The world felt smaller.

Then it became complicated -- but travel software made up for lost travel agents. We were ahead of the airlines.

Now, it's not so good. It's not just the security hassles. It's not just that the cost of a Minneapolis to Montreal trip has gone up 20% a year for the past four years (now doubled, Hawaii and Europe are cheaper).

It's also that planning a plane trip has become absurdly complex. Complex like choosing a cell phone plan, getting a "free" preventive care exam, managing a flex spending account, getting a mortgage, choosing health insurance, reading mobile bills, fighting payment denials, or making safe product choices. Complex like the complexity collapse that took down the western world.

I blame it all on cheap computing. Cheap computing made complexity attacks affordable and ubiquitous. [1]

In my most recent experience with information asymmetry I found tickets on US Airways for $490 (1 stop) on both Bing and Kayak. When I added a 2nd traveler, however, the price of both tickets increased by $100. (This was harder to spot on the US Airways site as they list deceptive prices, hiding all the "additional fees" airlines carved out to disguise price increases.)

A bit of research (time is how we pay our complexity tax) revealed this happens when the 1st ticket allegedly uses the last "cheap" seat on a flight. The next ticket costs more, and because airlines are loathe to confess this they increase the price of both. That may be so, but it means there's a great incentive to have a few cheap seats that will attract hits from travel sites, but that will turn into high price tickets for the 2nd passenger. This doesn't even have to be planned, natural selection means this kind of emergent "happy accident" of complexity, once discovered, will be leveraged.

This has costs. Maybe high costs. We pay them either by cash lost to legal frauds, or we pay them in time. I think they have more do with the lesser depression than most admit.

It would probably be cheaper for me to just pay my fraud tax to the airlines, but of course I'm not going quietly. I'm studying the (now obsolete) tricks of the trade [2]

  • Shop Tuesday at 3pm ET
  • Start shopping 3.5 months before departure, buy prior to 14 days
  • Tues, Wed and Sat are cheapest days to fly

[1] In the words of James Galbraith (emphases mine): "... The financial world, as it exists, has nothing to do with the commodity world of real exchange economics with its delicate balance of interacting forces. It is the world of technology at play in the form of quasi mass produced legal instruments of uncontrolled complexity. It is the world of, in other words, of evolutionary specialization in the never ending dance of predator and prey...
[2] Seems like there's opportunity for outsourcing complexity management to a new age travel agent and their equivalent for managing the complex scams of everyday life. I fear, however, that only a few of us realize we need help.

Update: Twelve hours after posting I was able to buy both tickets for a total of $200 less than the Saturday price. Same times and planes. I learned ...

  • Email alerts are worthless. I think they're just a way to harvest email for spam (we live on Planet Chum). Instead I took advantage of a Kayak feature -- they save the last search in a short list on main screen. I refreshed this twice daily. Between Saturday night and Sunday night I was able to get both prices at the listed price.
  • I had to keep referencing the search results Kayak provided. The US Airways site kept substituting the flight I didn't want as the "preferred option". I took me 4 runs to get it right. It's hard to explain what they were doing but to succeed I had to carefully track all the flight numbers.
  • Kayak passed my reservation to US Airways as 2 adults. The flight was 1 adult and 1 child. I suspected I needed the Kayak reference to get the price I wanted. Kayak passes its request through URL parameters (only sort of works) so I edited the URL parameter to 1 adult and 1 child.
  • US Airways makes pointless use of Flash to animate simple result display. This is revealing.
See also: