Saturday, November 25, 2006

AI: the dialog edges towards the mainstream

An article in today's NYT fit well with a draft post from March 2006, so I'll put them together here.

What we see here is that Artificial Intelligence (AI), in the truest sense of emerging sentience, is becoming visible outside of the cognitive science, eccentric, and science fiction communities. We also feel the powerful currents that will drive us, one day, to abiologic sentience.

The economic advantages of cheap focused intellect are irresistible. From computer gaming to stock selection to military planning to anti-missile systems to advanced self-guided robotic planes to cheap mammogram interpretation -- on every front billions of dollars push an emergent agenda.

Unless al Qaeda wins its battle to restore the 14th century, we will have fundamentally (biologic subsystems may play a role in early systems) abiologic sentience that will outstrip our current wetware. One uncertainty is whether this is 20 years away or 100 years away. The other uncertainty is whether we should be hoping for the 14th century.

First from yesterday's NYT:
November 24, 2006, New York Times
A Smarter Computer to Pick Stocks By CHARLES DUHIGG

... For decades, Wall Street firms and hedge funds like D. E. Shaw have snapped up math and engineering Ph.D.s and assigned them to find hidden market patterns. When these analysts discover subtle relationships, like similarities in the price movements of Microsoft and I.B.M., investors seek profits by buying one stock and selling the other when their prices diverge, betting historical patterns will eventually push them back into synchronicity.

... New software programs, like the Apama Algorithmic Trading Platform, have made it possible for day traders to build complicated trading algorithms almost as easily as they drag an icon across a digital desktop.

“Five years ago it would have taken $500,000 and 12 people to do what today takes a few computers and co-workers,” said Louis Morgan, managing director of HG Trading, a three-person hedge fund in Wisconsin....

... “Now it’s an arms race,” said Andrew Lo, director of the Massachusetts Institute of Technology’s Laboratory for Financial Engineering. “Everyone is building more sophisticated algorithms, and the more competition exists, the smaller the profits.”

... “neural networks” and “genetic algorithms” have become common in higher-level computer science. Neural networks permit computers to create new rules and automatically change underlying assumptions by experimenting with thousands of random sequences and processes. Genetic algorithms encourage software to “evolve” by letting different rules compete, and combining the most successful outcomes.

Wall Street has rushed to mimic the techniques. Because arbitrage opportunities disappear so quickly now, neural networks have emerged that can consider thousands of scenarios at once. It is unlikely, for instance, that Microsoft will begin selling ice-cream or I.B.M. will declare bankruptcy, but a nonlinear system can consider such possibilities, and thousands of others, without overtaxing computers that must be ready to react in milliseconds.

... In the pursuit of previously undetectable patterns, hedge funds are racing to quantify things — like newspaper headlines — that were previously immune from number-crunching.

Both Dow Jones Newswires and Reuters have transformed decades of news archives into numerical data for use in designing and testing algorithmic systems. The companies are beginning to structure news so it can be absorbed by quantitative models within milliseconds of release.

Moreover, companies like Progress Software are working with news agencies to create computer programs that instantly translate news — for example, a headline regarding Microsoft’s earnings — into data. M.I.T. is examining, among other things, evaluating companies by seeing how many positive versus negative words are used in a newspaper article.

Software in development could potentially respond automatically to almost anything; changes in weather forecasts on television news, shifting analyst sentiments or what a particular movie critic said about the new blockbuster...
This aticle suggests a new business model for journalism -- for better and for worse. Journalists now have a funding stream in which the buyers are computer programs, but the value of the commodity depends on it being consumed by humans as well. So the trading programs may replace the lost revenue from classified ads. There's a dark side (of course!). Imagine how much money will be made by manipulating the news stream to provide transient arbitrage opportunities. The article also suggests a rather substantial business model for natural language processing; this has implications for many domains.

Secondly, for another sign of the emergence of AI into the public mind, read this March 2006 blog posting: Marginal Revolution: AI, Consciousness and Robot Outsourcing. Alex clearly grew up without science fiction, so these ideas are new to him. The point is not that Alex, a bright person in general, has anything interesting to say, the point is that he's typical of the new wave of intellectuals who are contemplating the future. AI is going mainstream, gradually.

Incidentally, I've read a representative sample of the cognitive science literature arguing that we can't make a thinking abiologic entity. Chinese rooms, etc. The anti-AI group essentially argue that we have souls, or something like a soul, and things without souls can't think. We'll see. I rather doubt it.

This construction of abiologic sentience feels like a universal constant, an emergent property of any sentient organism with constrained resources and the ability to use technology. In a universe in which the three laws of thermodynamics apply, resources are always constrained. It seems plausible that all civilizations across space and time that survive long enough create abiologic sentience. I wonder if that has something to do with the great silence ...

Covering the retreat: GOP Senator Chuck Hagel

This is how GOP Senator Hagel tries to cover the retreat:
Chuck Hagel - Leaving Iraq, Honorably - washingtonpost.com

There will be no victory or defeat for the United States in Iraq. These terms do not reflect the reality of what is going to happen there. The future of Iraq was always going to be determined by the Iraqis -- not the Americans. Iraq is not a prize to be won or lost. It is part of the ongoing global struggle against instability, brutality, intolerance, extremism and terrorism. There will be no military victory or military solution for Iraq. Former secretary of state Henry Kissinger made this point last weekend...
Riiiiggghhhht. I'm sure history will agree.

Hagel was a part of the GOP team that oversaw what now appears to be one of the greatest foreign policy debacles since Japan's attack on Pearl Harbor (clarification: that was Japan's debacle). The US won't come out of this debacle as badly as Japan did, but our reasoning and execution was comparably flawed. The honorable thing for Hagel to do would be to admit that he screwed up and call for Bush and Cheney to turn their foreign policy responsibilities over to Bill Clinton -- or even Baker/Bush I.

Hagel struggles to put an "honorable" face on one of history's great blunders and to shift responsibility for a disastrous outcome to the Iraqis. In other words, he blames the victims. In this case they are not blameless, but the majority of the blame falls on the GOP and their supporters. At the same time he writes:
We have misunderstood, misread, misplanned and mismanaged our honorable intentions in Iraq with an arrogant self-delusion reminiscent of Vietnam....

... We are destroying our force structure, which took 30 years to build. We've been funding this war dishonestly, mainly through supplemental appropriations, which minimizes responsible congressional oversight and allows the administration to duck tough questions in defending its policies. Congress has abdicated its oversight responsibility in the past four years.
For Congress, read "the GOP"; they have been the Congress. Hagel calls for bipartisan support for a "phased retreat", a phrase that presumably means "no helicopters on the embassy roof". His essay could be summarized as "The GOP screwed up because the Iraqis weren't good enough, the Dems need to share the blame."

Disgusting. We may well end up with an orderly or disorderly retreat and a colossal defeat with dishonor, but the Dems should not shield the GOP from their incompetence. History will not judge Hegel or the Rovian-GOP kindly.

Friday, November 24, 2006

Outsourcing mainstream media jobs

Interesting variant on the ubiquitous outsourcing story...
Good Morning Silicon Valley: Not so much "thanks" as "whew!"

.... newspaper workers, including journalists, are seeing their jobs not just go away, but go overseas. Ad production jobs at our sister paper, the Contra Costa Times, are being outsourced to Express KCS, a firm that bills itself as “India’s leading media preparation business.” And the International Herald Tribune notes the growth in news organizations using inexpensive editorial help in India to write boilerplate Wall Street coverage... That's the way it goes when the giant semi of history get rolling; some people are going to get mowed down through no fault of their own. And if the truck misses you, you think about them as you count your blessings.
I'd love to outsource the presidency of the US. Surely there's a perfect candidate who could take on the real work. George could remain as the public face ...

Children's software: a dismal market in Windows, dead for the MacTel

I've just completed another of my recurrent surveys of the state of children's software for OS X and Windows. The best I could find was this Windows-only site: Software for Kids - Children Educational Software, Child / Kid Game.

The bottom line is that this market is comatose on the Windows' side and worse-than-extinct on the Mac side. The "OS X" software sold may simply fail to work on newer machines -- that's worse than nothing.

As my wife asks, "What happened to the long tail?". I don't really know, I suspect piracy, quality issues, channel problems (marketing) and branding all played a role. I'd say that the consoles were where this market moved, but I'm not sure that's true. I read recently that the gaming market is under threat because there's little available for novices or children.

Is this a market failure, or am I missing something big? In the meantime I think I'll have to, reluctantly, put BootCamp on my MacBook and see how it works. (Parallels won't do the trick, it failed my game tests.) Even in the XP world there aren't many choices, but some of our ancient children's games (many are no longer sold) may run under XP for a while.

(I think I'll need to upgrade my MacBook drive in a year or two -- using BootCamp will suck space.)

Update 11/26/06: I came across this recent related article. The Nintendo Wii is starting to look like the right move at the right time. Kudos to Nintendo if they've guessed right. Of course there was much more to children's software than just games; there's no replacement on the horizon for the rest. One way out of this conundrum would be for Apple to start funding bundled educational solutions and a production platform for third parties to expand on them. It would help sell Macs, and piracy would be less of an issue for Apple.

Thursday, November 23, 2006

Populism and electronic guidance of electoral choices

Slashdot has an unusually good submission on populism and voting patterns in the Netherlands (emphases mine):
Slashdot | Web-Based Assistant Changes the Face of Dutch Politics

The elections held in The Netherlands on Wednesday have shaken the country. Almost 10 million votes were cast, and statistics show that a full half of those who voted used a popular web-based voter guide. This guide is operated by the independent institute for the public and politics. Advice is given to the visitor upon answering a number of multiple choice questions on some common political topics. Statistically, a number of people ended up scoring in support of populist parties both on the far left and far right. No bias was reported to exist in the test itself. However, these parties have ended up with an unforeseen amount of power as a result of the election. The voter participation was high, and the web-based advisories may have motivated people with little interest in politics to cast a vote anyway...
I took the test; I apparently fit best with D66, which is described (of course) in Wikipedia:
... D66 was founded on October 14, 1966 by 44 people. Its founders are described as "homines novi", only 25 of the 44 had previously been members of a political party. The initiators were Hans van Mierlo, a journalist for the Algemeen Handelsblad and Hans Gruijters, a municipal councillor in Amsterdam. Van Mierlo became the party's political leader and Gruijters the party's chair. The foundation of the party was preceded by the Appeal 1966 on October 10, in which the founders appealed to the people of the Netherlands to re-take their democratic institutions. The party renounced the 19th century political ideologies which dominated the political system and wanted to end pillarization. It called for radical democratization of the Dutch society and its political system and it called for pragmatic and scientific policy-making...
Ok, I'm impressed. D66 sounds like a reasonable fit for me. The test felt solid and thoughtful. I feel it would truly direct people's votes to a party that fit with their own inclinations. Which brings one to the poster's thesis -- that enabling persons with little participation in political dialogue to vote "intelligently" may predispose to unusual and possibly extreme political parties. I am inclined to believe the thesis; if this method caught on it could have a vast impact on the political process.

Historically when non-engaged persons vote, it's pretty mindless. They may vote based on ballot order, the sound of a name, the last ad they saw, etc. The net effect is a combination of "noise" and a wild-card bias. If non-engaged persons were to actually vote the party that best reflected their opinions using a tool like this one, the "noise" in the system would fall, but I think the distribution of candidates would widen.

Fascinating. I'm not sure whether this tool is a good thing or not, but it's well worth attending too.

Tuesday, November 21, 2006

Playing dirty: the GOP's election

Salon has quite a catalog of GOP dirty electoral tricks from the past few months. Lest we forget ...
The GOP's dirty deeds of 2006 | Salon News

....In this year's biggest dirty trick, that's what happened to voters across the country, who were deluged with robo-calls that purportedly were coming from Democratic candidates. The calls started innocently enough, by offering information about the local Democrat. But if you hung up, the robot would call back. Hang up again and, like some character out of a Stephen King novel, the robot would call again. And again. And again, sometimes as many as seven times before it gave up. So the voters who had the temerity to want to enjoy their dinner unmolested were left with the impression of a Democratic candidate who simply would not leave them alone; those who stayed on the line were instead treated to a string of disinformation about the Democrat. The calls, which were paid for by the NRCC, hit many of the House races vital to Democrats' chances to take back the House. They ran in at least two key Illinois districts, including Duckworth's, and in Connecticut, where vulnerable Republican incumbent Chris Shays survived a stiff challenge from Democrat Diane Farrell, and in other races in states as diverse as Georgia, California, Pennsylvania and New York. In all, a total of 1 million calls were spread over 53 House races; this means that an average of 20,000 calls were made in each district, each of which contained about 200,000 votes. The calls could potentially have reached one out of every 10 voters in the targeted races.

The uproar that followed hasn't escaped the attention of politicians. Sen. Barack Obama has introduced a bill that would criminalize the practice, and on the local level several states are considering similar legislation.
It's not that the dirty tricks didn't work, it's rather than they couldn't overcome a large anti-GOP mood. In many elections they would have done the trick.

The GOP is a morally bankrupt institution.

The Needham question, 9/11 and the next fifty years

I've not much time to develop this thesis, so I'll go quickly here. Maybe I'll get to it later.

There are four parts to this:
The Needham question is basically: "Why didn't China sail a steamship bristling with canon into Manhattan @ 1750?".

The Falling Cost of Havoc is the observation that small groups of individuals can now afford some of the military capabilities (bioweapons, worldwide secure communication channels, advanced data mining, evolved terrorist techniques) that were once available only to nations.

The Next Fifty years is my post suggesting, somewhat to my own surprise, that the next fifty years may not be all that different from the last fifty years. (Speaking Mandarin and using low speed low energy vehicles is not that big a difference.)

The Innovator's Dilemma is a fascinating business book pointing out that well run dominant enterprises are appropriately good at stifling disruptive technologies.

The thesis is that these are connected.

The best current answer to the Needham question is that China was doing pretty well during the 14th through 18th centuries, and it had developed a governmental system that was very stable* and very good at eliminating disruptive interruptions. In other words, it behaved like a well run corporation. Innovation is disruptive, disruption is dangerous.

If the next fifty years is to resemble the last fifty years (by no means the worst outcome), then the world must become far better at stifling disruption and managing the falling cost of havoc. The world will rediscover China's answer to the Needham question. (Ironically, China will need to rediscover it too ....)

My guess is that the world will do this, whether geeks like me like it or not. I think the history of the counter-revolution has been greatly under appreciated.

Ok, that's all for now.

* The past 5 years of US history has dramatically diminished my faith in the stability of democracies.

Sunday, November 19, 2006

New Scientist: The 50 year prediction

Marginal Revolution pointed me to New Scientist's the Next 50 Years. I'll spend some time reading it.

On the one hand, 50 years seems far too long to make sensible predictions about. On the other hand, imagine someone in 1950 predicting life in 2000. Any futurist worth their salt was predicting flying cars to Mars, but the right answer would have been 'sort of like 1950, but most people have more money and live ten years longer'. The shock for 1950 would not be computers (what? They're not sentient?), but rather gay marriage and gay adoption. Oh, oil prices would have been shocking too -- we were supposed to running on nuclear fusion by now.

So in the same vein, I'll predict that we don't get sentient AI, we don't boost human IQs substantially, and we don't prevent aging. Life in 2056 is rather like 2006, but we live 10 years longer and most people have more money. The shocking part is ....

Update 11/19/06: The authors are generally as cautious as one would expect. They are reputable sorts, and thus fearful of foolishness. There is, however, a clear trend. Several contributors speculate that we'll learn about extraterrestrial life. I suspect they mean sentience too, but that's too brash and idea. I suspect they toopuzzle over the Fermi Paradox.

Healthcare Reform: The Faughnan One Slide Presentation

I will never be invited to a public debate on healthcare reform. Bit of a shame -- I could make a case that I'm unusually well suited by experience, training, and employment to opine on the topic. I even have a one slide presentation, which I came up with in the shower this morning. Here it is (imagine the two visuals on the same slide).


and next to it


The top image is a new Lexus. It's a very fine car. There's a luxury premium to be sure, but perhaps not so much as with a Mercedes. You really do get a lot of quality when you buy a Lexus.

The bottom image is from a Manhattan subway. I think the subway is better now than when I strolled the then mean streets of Manhattan; in those days it was smelly, dirty, and very noisy. It delivered excellent value for the price, though it was not a pleasant ride.

If you lived in Manhattan, sometimes the subway was really a much better way to get from A to B than the Lexus. The Lexus, of course, would be most people's first choice if they could afford it.

That's all you need to know about Healthcare Reform in America.

I grew up in Quebec when socialized medicine was new. It was fabulous for us. Everyone got a Honda Accord and was more or less satisfied with that. Cost structures changed over time, and now everyone rides the subway. That doesn't work for those who can afford a Lexus, or even an Accord.

The mistake of socialized medicine was thinking that it was possible to provide an Accord forever, to everyone.

I live in America, where almost everyone insists on a Lexus and a heck of a lot of people are walking. Without shoes. In winter. Increasingly the walkers are newly underemployed and unemployed men and women aged 50 to 65.

The tragedy of American healthcare is obvious.

America will aways provide the Lexus, and the Mercedes, and the chauffeur driven Rolls-Royce to those who wish can afford them -- and choose them. It will also provide the Accord for a heck of a lot of people. Those who get the Accord and the Lexus will often, but not always, get better experiences and outcomes than those who ride the subway.

All Healthcare reform discussions in America are about the subway.

That would be the end of my presentation.

Saturday, November 18, 2006

The meaning of Apple - will it always stand alone?

I sigh when I look upon my Samsung PalmOS phone. It's almost good, the last of a discontinued line that was within spitting distance of Apple elegance. Alas, Palm fumbled, Samsung floundered, and the moment passed. Every phone on the US market sucks. Spectacularly sucks.

I wait for the iPhone. It may not be what I want, but I know I will respect it. I may change my wants to suit it. It will not suck.

My toaster sucks. I've looked for its replacement for years. I figure I'll pick one up the next time I'm in Germany. I find myself wondering if Apple will make a toaster. I'd buy it ...

Then I read this Mark Morford SFGate column (I've added him to my bloglines collection)...
When Apple Rules The World / What does it mean when you really, really want to lick a new MacBook Pro, and swoon?

....I don't mean [Apple] they've solved world hunger or cured cancer or ended racism or muzzled Ashlee Simpson. But I do mean something that, in its way, is nearly as profound: They've managed to make the world just a bit more pleasurable, tasteful, beautiful. They've added a dash of that rarest of human qualities, especially when talking about factory-made tech crap: They have added a touch of grace.

You know it's true. Apple has done more than perhaps any other mass-market consumer line on the planet to affect the look and feel of nearly every gizmo made today. This cannot be underestimated. It's a George Bush world, after all, one that values sameness, mediocrity, intellectual and spiritual laziness. But merely rub your hand across the top of a MacBook or whip your thumb around the click wheel of an iPod and notice: Feel that throb? That's your id saying mmmmmmm.

What? What's that you say? Why yes, other tech companies have changed the world, too. Microsoft, for example. They invaded the global cube-farm and made everyone's workday a bit more bloated and annoying and sad and just a little uglier, buggy as hell, frustrating, virusy and lonely and numbing. Truly, it has been quite an impressive accomplishment...
Read the whole thing. It's a funny, slutty, paeon to Apple. It's over the top, and yet true. No other company, save Google on a good day, actually delivers elegance (Aperture excepted). Apple really is different. I can only pray it won't be alone forever; I could sure use a new toaster.

Microsoft? It's far less than the sum of its excellent people. Some corporations can do that.

PS. My new MacBook is due this week ...

Friday, November 17, 2006

Suzhou, China - Home of my MacBook

When I ordered by Core-2 Duo MacBook, I received a FedEx notice that it was being shipped from Suzhou, China. Google Earth provided a low-res picture. Note the massive urban landscape and what appears to be industrial smog and black some ominous looking black water.

On the one hand, this is a snapshot of 21st century life. On the other hand, the 'map view' has no geographic details and there are no embedded links to local pictures, webcams, etc. (Google Earth would have more, but it died with my laptop drive and has yet to be resurrected). In a year or two, the rest of the puzzle will be in place.

It's not that far from Suzhou to some impressive looking mountains and forests ...

Update 11/25/06: In another note on the changing technology of shipping, my MacBook arrived in a relatively compact box that would not have survived the shipping methods of old, or even travel in a conventional UPS truck. The shipping box was a fraction of the size of the box that held my iBook years ago. Even so, the external box looked flawless. I'd like to read a story about how FedEx does this kind of thing.

Wednesday, November 15, 2006

Voting: just do what Minnesota does

Schneier gives a good rundowns of the ongoing electronic ongoing electronic voting machine debacles. Then he reveals his residence in my home state:
...In Minnesota, we use paper ballots counted by optical scanners, and we have some of the most well-run elections in the country. To anyone reading this who needs to buy new election equipment, this is what to buy...
I've heard only five states use optical scanners. Must be an IQ thing. When Caltech and MIT joined up (rare event) to review the Gore/Bush voting debacle, they concluded optical scanning was by far the best choice. Every review I've read since has come to the same conclusion. Come on guys, let the frozen north lead the way ...

The gravest threat we know of ...

Years ago, the Economist pointed out that based on risk assessment asteroid impact warranted as much attention as airplane crashes. That was then ...
Did an Asteroid Impact Cause an Ancient Tsunami? - New York Times

... Most astronomers doubt that any large comets or asteroids have crashed into the Earth in the last 10,000 years. But the self-described “band of misfits” that make up the two-year-old Holocene Impact Working Group say that astronomers simply have not known how or where to look for evidence of such impacts along the world’s shorelines and in the deep ocean.

Scientists in the working group say the evidence for such impacts during the last 10,000 years, known as the Holocene epoch, is strong enough to overturn current estimates of how often the Earth suffers a violent impact on the order of a 10-megaton explosion. Instead of once in 500,000 to one million years, as astronomers now calculate, catastrophic impacts could happen every few thousand years.

If the hypothesis is accepted, then the risk to life of a meteor impact will have increased about 500 fold. That would move it very high up the risk index, perhaps above thermonuclear war.

If that happens, and if humans were logical creatures, we would put a significant share of the world's GDP to surveilance and diversion research. Alas, humans are far from logical ...

PS. The 2807 BC rock landed on, of course, Atlantis. :-)

Saturday, November 11, 2006

Neandertal is us: the meaning of species

We all know how confusing the terms race and ethnicity are. It turns out that things are equally messy when one tries to define was a species is.

John Hawks sifts the narrative, and basically decides that species is a semi-useful term that can be largely ignored -- particularly in the fossil records. His read is that Neandertals are probably members of the same species as us, but that the distinction is pedantic and misleading. He does strongly feel that our genome contains evidence of Neandertal descent, but that the discussion is missing the point (emphases mine):
... I think that evidence of introgression reinforces the hypothesis that modern humans emerged in an adaptive context, making use of adaptive variation from a widespread (possibly pan-Old-World) population of archaic Homo. It's one of the two main patterns in the evolution of modern humans.
We're all waiting to learn what the other pattern is, Hawks won't tell us yet. The fundamental shift is to stop thinking of Neadertals mating with "us", and to instead think of modern humans as something that evolved from a large number of "archaic Homo" populations. We are Neandertal, but we are also one of many old populations.

When the cliche is real: falling on the grenade

It's an old cliche. Falling on the grenade. I assume there isn't time to think. Cpl Jason Dunham has been awarded the medal of honor.