Monday, August 31, 2009

Obamacare. Wait. It's not over.

Birthers. Tea baggers. Deathers. Glen Beck inspired assassin wannabes. The weak dancing to their amoral master's tunes. Elders at town meetings demonstrating the pernicious effects of early dementia. Ratings starved media playing destructive games. Rupert Murdoch's Fox News and Wall Street Journal. Obama below 50%, vilified by the raving right and attacked by the fearful left. Kennedy dead and Robert Byrd 91 and ailing. Congressional democrats retreating on health care. And then there's the "bipartisan" "gang of six".

It's been a bad August for anyone worried about civilization. Despair is easy.

Don't despair.

This isn't new, and Obama is both lucky and good. It's not just him, look at his team. They're extremely formidable, and they've been in this game a while. They know the nation they're dealing with.

It's easy to underestimate Obama. He's identified with a tribe (black America) culturally associated with defeat. Unlike both Bill Clinton, GWB and most every GOP politician he doesn't rant and rave.

And yet, he tends to win.

For those who fear the worst, I suggest John Harwood's NYT article and John Scalzi's recent summary post.

Remember, any remotely sane American has to think that reform with enemies like Beck and the Birthers must be good.

Obama has been falling back, and the enemy has been charging forward. They think they see a soft center -- but they've forgotten about the hills on both sides ...

Update 9/9/09: I was early on this train, but now the NYT is catching up. President Barack Hussein Obama has not yet begun to fight.

PS. In case my use of the term "Obamacare" is confusing readers, I should mention that we have an inaugural postcard of President Obama prominently featured on our kitchen corridor wall at the children's eye level. I give it a nod every so often.

Update 10/1/09: As predicted, the forces on the western hill are now swooping down on the battle crazed spittle-flecked GOP berserkers. The eastern hill is standing by. Reminds me of the scene in "The Two Towers' when Gandalf leads the Horsemen of Rohan down upon the Orcs.

Sunday, August 30, 2009

The evolution of comment spam - from parasite to symbiote?

Lately I've been getting blog comments that blur the spam/non-spam species boundary.

Comment spam used to be pretty clear. It would be unrelated to the post topic, and contained a link to a splog or other more or less fraudulent web page. These were easy to automatically block, so spammers dropped the links. Second generation comment spam aimed for search engine "optimization" through reputation enhancing back links to the author URL. Second generation comment spam was made of strings like "thanks for the the great post"

These were harder to machine reject, but easy for human reviewers to spot.

Now I'm seeing third generation comment spam. These have no links, and they're actually related to the original post. Sometimes they're almost non-sequiturs, but mostly they read like a fourth grade student answering a homework assignment. The grammar suggests either a very young or non-english writer. They do link back to splogs.

So how's the new species of comment spam being authored? It could be AI based -- maybe calling Wolfram Alpha or Wikipedia to retrieve relevant strings. It's probably human though -- outsourced work being done by low paid labor churning out comments at high speed.

This third generation spam isn't trivial to reject. Sometimes I have to think about it.

We know where this is going. Fourth generation spam comments will actually make sense. They'll be legitimate comments.

Fifth Generation spam comments will be very high quality. Skynet will appreciate them.

Update 9/4/09: Another (funny) take on the theme. Also, see the comment by one of my favorite writers.

Update 1/1/10: Cory Doctorow's excellent 2006 novella I, Row-boat (read it, it's online) tells us how Robbie the row-boat's ancestors became sentient ...
“Back in the net’s prehistory it was mostly universities online, and every September a new cohort of students would come online and make all those noob mistakes. Then this commercial service full of noobs called AOL interconnected with the net and all its users came online at once, faster than the net could absorb them, and they called it Perpetual September.”...

... “AOL is the origin of intelligence?” She laughed, and he couldn’t tell if she thought he was funny or stupid. He wished she would act more like he remembered people acting. Her body-language was no more readable than her facial expressions.

“Spam-filters, actually. Once they became self-modifying, spam-filters and spam-bots got into a war to see which could act more human, and since their failures invoked a human judgement about whether their material were convincingly human, it was like a trillion Turing-tests from which they could learn. From there came the first machine-intelligence algorithms, and then my kind...

Friday, August 28, 2009

OS X 10.6 - do you feel lucky punk? Do you?

I need to use my machines. So I'm the kind of geek who likes to, barring the addition of a new non-critical machine, wait 6-18 months before switching major releases of OS X.

As it happens I am going to buy an iMac in the next few months, but for now I've no hands on experience with 10.6. Still, if you review the late Aug 2009 late Aug 2009 OS X related reads the dog whistles are loud and clear.

Snow Leopard breaks stuff. Lots of stuff. It's also slower or only minimally faster than 10.5 on most machines, and Apple blew their major security feature (memory randomization) -- they obviously couldn't get it to work. So Windows 7 has better fundamental security -- as does Vista for that matter. Resolution independence? Oh, you remember that from 10.4 days? Of course not.

The only good news is that you can (illegally) install the $30 of 10.6 over 10.4. Considering Apple's long tradition of abusing early adopters I give everyone my ethical permission to do so. It's only fair.

There's good stuff in 10.6, and there's bad stuff. (For example, it looks like Apple continues to wreak havoc on pioneering concepts in the old Mac Classic file system.) There's enough good stuff that I'm looking forward to running 10.6.1 on a non-essential new iMac. Otherwise I strongly advise waiting 6 months before updating -- and even then you should confirm that your current printers and scanners and so on will work with 10.6.

Unless, that is, you're feeling lucky.

Update: This is the best review so far.

Update 9/6/09: I played with Snow Leopard in the Apple store today. As others have noted, it's hard to find any differences from 10.5. From what I read at least as many things have been broken as have been fixed. Unless you have to upgrade from 10.4, or you're buying a new machine, you shouldn't consider Snow Leopard before March of 2010.

Update 3/13/2010: I was too optimistic. My 10.6.2 machine crashes hard frequently. Among other issues, Apple screwed permissions and firewire. As usual.

Saturday, August 22, 2009

Yeah, Cheney/Bush used the orange alerts to scare up votes

As suspected, despite fervent Bush admin denials, Cheney Bush used the "terror alert" scheme to scare up votes prior to Bush/Kerry: Informed Comment: Bush Admin. worse than our Nightmares.

Not really news, but, for the sake of history, important to note.

That administration was a cancer on the American psyche -- and their blight is far from gone.

The Google Voice story: It was Apple, not AT&T

As all true geeks know, a few weeks ago Apple purged the iPhone app catalog of all Google Voice apps, including GV Mobile. Apple then rejected a pending application from Google for their Google Voice app.

Geeks know this is big. Google Voice (it's available in the US, just go to the linked page and request a number) is fabulous tech. I've been a regular user for over a year, enjoying my 1 cent/min good quality cell phone calls to Canada (as of a week ago, free). When my family travels our cell phones forward to Google Voice so we get voice mail messages emailed to us -- along with quite good transcriptions. It doesn't need a dedicated app to work, but a good iPhone app would take it up another notch.

It's one thing for Apple to reject crappy stuff like Flash from the iPhone, but rejecting high value innovation is an injury to the geek soul.

Happily, the FCC then piled on Apple and demanded an explanation.

So, in record time, we have Apple's letter to the FCC, dissected by Gruber (who apologizes for blaming AT&T), Arrington (he's bested Gruber on this one - that's gotta sting), Mike Ash, and a zillion others.

Basically, Apple dodges, twists, hurls, whines, and, basically, lies big time - except when they admit full responsibility and absolve AT&T of all sins.

Besides generating a rich stream of bs, Apple also surrenders. As just about every blogger notes, one of Apple's whoppers is that they haven't really "rejected" Google Voice (or the other apps they removed from the app store?!), they're just "under review".

Which means Google can make some face saving changes and Apple will cave. I'm hoping to be using my iPhone Google Voice app within a month.

I thought Arrington had the best analysis. A heck of a lot of the iPhone's value is now tied to Google -- and Google Voice just hammers that home. Apple can't compete in the Cloud -- as we can gather from watching MobileMe twist in the wind.

I'm happy to see that a few bloggers have noticed that while AT&T played no role in this decision, there are AT&T rules blocking VOIP products that seem to apply to the entire world -- not just AT&T's turf. (If only AT&T had anticipated Google Voice they'd have banned that class of service as well. I wonder if they've fired anyone for missing that angle.)

Incidentally, the AT&T response to the FCC is interesting -- they're asking how Google is able to dodge various mandates applied to phone companies. This is how the big gun lawyers earn their yachts.

Thursday, August 20, 2009

Conde Nast's latest spam ploy - Axciom's Delivery.net

Conde Nast, publishers of Gourmet and other periodicals, holds a place of dishonor among the world's scummiest spammers. It will be a sad commentary on humanity if the New York Times goes under and Conde Nast survives.

Spam must work for them, because they invest a fortune in spam and associated legal fees. They're not too hard to block; even though they change their email address every few months it's only a moments work to add another Gmail 'filter to trash' rule.

Today, though, they're trying something knew. They're sending their email using a "delivery.net" account with a dedicated spamming service:
Acxiom Digital

... Acxiom Digital helps the world's leading marketers create and deliver permission-based email marketing campaigns. Acxiom Digital acts as an agent for our clients in delivering email communications to their customers. Our clients own the data on their customers, including email addresses, which are gathered via permission-based processes at their website or other online and offline sources...
"Permission-based" my ass.

So now anything from 'delivery.net' is immediately deleted. It will be interesting to see what email address Conde Nast uses next.

Friends don't let friends buy Conde Nast products.

The check engine light in the mobile net era

We're on day two of a two week family road trip and our 2000 Mazda MPV Van check engine light comes on.

On a Sunday.

Once upon a time, this might have led to an urgent search for an open garage.

Ahh, but now we have and iPhone and, at least until we hit Canada, net access.

So instead of pulling over, Emily researched and I drove. We found out ...
  • This should really be called the check emission control system light. In most vehicles it's triggered by a sensor in the emission side of the engine.
  • The most common cause is a loose gas cap. Presumably the loss of suction causes venting of gas into the emission systems.
  • Rarely it can be something bad with the engine, so the official word is always to get it checked out. If you play the odds though ...
  • Depending on the car it can cost $150 or more to read off the error code (my next car needs to have a diagnostic USB port on the dash as well as 4 110 V outlets - it's insane these are so hard to read).
  • In some cars the light won't ever go off until the dealer checks it out. In others, if the problem is corrected the light will eventually go off. The trick is that this may take 15-20 restarts (the number of restarts seems to be more important than time, presumably due to how the sensor works.
Emily had already noted that I'd only turned the cap 'one click'. (Ok, so it wasn't all iPhone. She remembered some of this stuff.) So simple Bayesian reasoning (prior probability, etc) meant there was a very high probability this was a (stupid) gas cap alarm.

So we just drove.

We stop and start a lot on our family trips, so about 3 days later we' d hit about 15-20 restarts and ...

The light went out.

We sacrificed a GB to Google in gratitude.

Tuesday, August 18, 2009

One possible strategic error in the Obama reform

In his NYT essay Obama talks about making medicare more efficient.

I wonder why his team chose to tie insurance reform and access to any kind of changes to medicare. Politically, would it have been wiser to have kept the two topics completely separate? The problems of medicare are huge, but perhaps should have been addressed in year 2 or 3 of the administration.

We'll get something, but it will be another patch. We'll be back in 10 years.

Alas, a large portion of America seems to suffer from Stockholm syndrome. They prefer familiar misery to the terrors of hope.

Friday, August 14, 2009

The CDC's vaccine data mess - please help them out

This page is how America distributes the data set that's supposed to represents all the vaccine information used in electronic health records and national reporting: Vaccines: IIS/Stds/CVX-Vaccines Administered

You know, the kind of reporting that's useful, for, say managing swine flu vaccine programs.

It's not being distributed in some UMLS data format, or a tab delimited UTF-8 file, or a Microsoft Acccess table or XML or even Microsoft Excel or ... or ... or even a 4 column RTF or .DOC table.

It's distributed as an HTML page with inline comments and footnotes or as a PDF document.

Anyone wanting to actually implement this has to cut and paste into something like Excel, move the inline annotations around, get rid of the footnotes, represent color and font changes as attributes, and so on.

This isn't rocket science guys. The management of this sort of data set was well understood in the 1960s. Forget about all those wonderful visions of just-in-time clinical decision support, this is really simple, basic, stuff.

Every American should give the CDC a penny so they can engage an underemployed informaticist to fix up their CVX distribution system.

Or maybe the CDC could, you know, hand this over to the NLM to manage?

PS. This story is consistent with the way ICD-9 was once managed. ICD-9-CM (diagnostic) is the payment justification code set that's sort of used to track diseases and horribly misused in clinical care reporting and automation. I'd love to see a sociology researcher dig down and find out why it is we end up with such bad management of fairly simple things.

Update: The CVX to CPT map table is even worse.

American Express credit card information theft

We just received official notification that our AMEX credit card information was stolen. Inside job, as usual.

Same old, same old.

I'm astounded that web services expect me to give them my Google authentication credentials. They're conning us when they claim mere encryption will secure the data.

Incidentally, this emphasizes the stupidity of the "secret security question" fail (see US Bank security shield makes me scream). Not only do they make it easier to hack into user data, they do nothing to protect us from the commonplace insider thefts and other, old, tactics.

Thursday, August 13, 2009

Galaxies like stars in the sky

There's an antidote to reading about the Cheney torture program and the GOP's insurance-company funded attack on health insurance reform.

Download the images linked here, and open them in a robust image viewer. Browse at will. Visit galaxies as they were billions of years ago. Squint very hard, and image you're seeing something squinting up from one of the billions of worlds in the billions of stars ...
ESO - ESO 39/08 - A Pool of Galaxies - Associated Image

... The new image released by ESO combines data obtained with the VIMOS instrument in the U- and R-bands, as well as data obtained in the B-band with the Wide-Field Imager (WFI) attached to the 2.2 m MPG/ESO telescope at La Silla, in the framework of the GABODS survey.

The newly released U-band image – the result of 40 hours of staring at the same region of the sky and just made ready by the GOODS team – is the deepest image ever taken from the ground in this wavelength domain. At these depths, the sky is almost completely covered by galaxies, each one, like our own galaxy, the Milky Way, home of hundreds of billions of stars.

Galaxies were detected that are a billion times fainter than the unaided eye can see and over a range of colours not directly observable by the eye. This deep image has been essential to the discovery of a large number of new galaxies that are so far away that they are seen as they were when the Universe was only 2 billion years old....


Be sure to try the zoom tool.

Inmate Kyle Foggo – Creator of the post-2003 torture facilities

It looks like the Obama administration will investigate and prosecute senior CIA officials who broke American law. 

In the meanwhile CIA officials are spinning their stories to the NYT, with usual welter of self-justifications and contradictions.

If you read this somewhat confused NYT article carefully you’ll see examples of those contradictions – in addition to being generally incoherent. In one spot it says waterboards were built on the spot, in another paragraph it says waterboarding had been discontinued when the prisons were built. I

The article reads like multiple leaks with different aims. Some statements seem intended to help Mr Foggo, others to convict him …

Interrogation Inc. - A Window Into C.I.A.’s Embrace of Secret Jails - NYTimes.com

WASHINGTON — In March 2003, two C.I.A. officials surprised Kyle D. Foggo, then the chief of the agency’s main European supply base, with an unusual request. They wanted his help building secret prisons to hold some of the world’s most threatening terrorists…

… Foggo went on to oversee construction of three detention centers, each built to house about a half-dozen detainees, according to former intelligence officials and others briefed on the matter. One jail was a renovated building on a busy street in Bucharest, Romania, the officials disclosed…

… a small company linked to Brent R. Wilkes, an old friend and a San Diego military contractor…  provided toilets, plumbing equipment, stereos, video games, bedding, night vision goggles, earplugs and wrap-around sunglasses. Some products were bought at Target and Wal-Mart, among other vendors, and flown overseas. Nothing exotic was required for the infamous waterboards — they were built on the spot from locally available materials, the officials said.

… Mr. Foggo .. pleaded guilty last year to a fraud charge … and he is now serving a three-year sentence in a Kentucky prison … He was not charged with wrongdoing in connection with the secret prisons, but instead accused of steering other C.I.A. business to Mr. Wilkes’ companies in exchange for expensive vacations and other favors. Before leaving the C.I.A. in 2006, he had become its third-highest official…

… Early in the fight against Al Qaeda, agency officials relied heavily on American allies to help detain people suspected of terrorism in makeshift facilities in countries like Thailand. But by the time two C.I.A. officials met with Mr. Foggo in 2003, that arrangement was under threat, according to people briefed on the situation. In Thailand, for example, local officials were said to be growing uneasy about a black site outside Bangkok code-named Cat’s Eye…

.. Eventually, the agency’s network would encompass at least eight detention centers, including one in the Middle East, one each in Iraq and Afghanistan and a maximum-security long-term site at Guantánamo Bay, Cuba, that was dubbed Strawberry Fields, officials said. (It was named after a Beatles song after C.I.A. officials joked that the detainees would be held there, as the lyric put it, “forever.”)

The C.I.A. has never officially disclosed the exact number of prisoners it once held, but top officials have put the figure at fewer than 100.

At the detention centers Mr. Foggo helped build, several former intelligence officials said, the jails were small, and though they were built to house about a half-dozen detainees they rarely held more than four.

The cells were constructed with special features to prevent injury to the prisoners during interrogations: nonslip floors and flexible, plywood-covered walls to soften the impact of being slammed into the wall

… C.I.A. analysts served 90-day tours at the prison sites to assist the interrogations. But by the time the new prisons were built in mid-2003 or later, the harshest C.I.A. interrogation practices — including waterboarding — had been discontinued

As the investigations proceed there will be more leaks. Eventually a case may be built against Mr. Foggo, but more likely the feds will lean on him, and whoever he implicates, to turn in someone else.

Oddities of modern video – frame rates and more

We have two CRTs on which “we” (meaning the kids) watch Netflix DVDs and local broadcast TV sports. The DVD display is a 15 yo SONY CRT. It will still be running when all of our neighbor’s LCDs need their light tubes replaced. The television is a 9” “portable” CRT that, I think, has color. The “TV” uses a “free” converter box and our rabbit ears.

So this story on how those newfangled HD TVs intersect with the complexities of capturing motion in analog film and digital sensors, both converted in various ways to travel through various digital to analog transformations (including some back and forth in fiber optic backbones) and then to photons (themselves both digital and analog) and then to quantum sensors (retinal neuron photo receptors) and to analog signals (brain) and then to perception and meaning (?) …

Well, it’s all a voyage to a foreign and exotic world that, one day, when the children are old enough to buy their own TV, we might visit.

Read both the story and the comments, I’ve excerpted enough here to show why this is academically intriguing …

Help Key: Why 120Hz looks “weird”

John Biggs

I’ve been testing an HD projector here at the house and, in its initial, out-of-the-box setting we found that the picture was ridiculously “sharp.” The picture, I suppose, looked like an old Dr. Who episode where the action on screen is smoother than the background, creating a jarring disparity when watching movies with lots of movement. It’s sometimes called the “Soap Opera Effect.” We decided to do a little digging to figure out why.

Most film is recorded at 24 frames per second, but your LCD TV probably either displays at 60 fps or 120 hz (hertz is just a measurement of frequency per second). There are three main ways to cope with this…

Paul Spurrier

… The problem with motion interpolation is that it is having to create new frames that weren’t on the original film..

… Your TV is having to do this in 1/24 of a second.

Which is impossible.

So it cheats. It uses warping and other motion compensaton/interpolation techniques to create inbetween frames that sort of fool the eye…

…And not surprisingly it looks weird…

… Nowadays, as more movies are shot digitally, filmmakers are trying hard to work out what it is that makes a movie looks like a movie. Then they’re trying to alter the digital picture to replicate that look…

… Gamma. It’s complicated but basically film and video react to light in different ways. Film still sees more detail in the brightest and darkest parts of the picture and sees colour in a different way. Another thing that filmmakers do to make digital images look like film is to alter the gamma, either in camera or in post-production…

… Frame-rate. To bring this back to the original subject, traditionally the biggest give away that one is watching a video-originated image and not film is the frame rate. In the US, video cameras shoot 60 frames a second. They cheat a bit to do this…

… Nowadays, we can set digital cameras to not do this. We change them to film in what’s known as a ‘progressive’ mode, which shoots full 24 frames a second.

In the old days, filmmakers used to go to great lengths to get rid of this interlacing….

… It was clear that the higher frame-rate was a give-away that the image had been shot on video not on film. So even if we degraded the picture somewhat by throwing out half the data, it still sort of looked ‘better’.

… audiences react differently to video and film. They don’t know they’re doing it, and it’s working almost entirely on a subconscious level, but when someone thinks they’re watching a movie, their mindset is that this is something more special, bigger budget, more worthy of their attention.

So imagine the frustration of filmmakers when new TVs undo all of the work we have put in to making something look like film and make it look indeed like a ’soap opera’…

Health insurance reform – 8 points to repeat and repeat

It will be useful to keep this list at hand and repeat it often …

The White House - Blog Post - The Return of the Viral Email

… 8 ways reform provides security and stability to those with or without coverage

  1. Ends Discrimination for Pre-Existing Conditions: Insurance companies will be prohibited from refusing you coverage because of your medical history.

  2. Ends Exorbitant Out-of-Pocket Expenses, Deductibles or Co-Pays: Insurance companies will have to abide by yearly caps on how much they can charge for out-of-pocket expenses.

  3. Ends Cost-Sharing for Preventive Care: Insurance companies must fully cover, without charge, regular checkups and tests that help you prevent illness, such as mammograms or eye and foot exams for diabetics.

  4. Ends Dropping of Coverage for Seriously Ill: Insurance companies will be prohibited from dropping or watering down insurance coverage for those who become seriously ill.

  5. Ends Gender Discrimination: Insurance companies will be prohibited from charging you more because of your gender.

  6. Ends Annual or Lifetime Caps on Coverage: Insurance companies will be prevented from placing annual or lifetime caps on the coverage you receive.

  7. Extends Coverage for Young Adults: Children would continue to be eligible for family coverage through the age of 26.

  8. Guarantees Insurance Renewal: Insurance companies will be required to renew any policy as long as the policyholder pays their premium in full. Insurance companies won't be allowed to refuse renewal because someone became sick.

Learn more and get details: http://www.WhiteHouse.gov/health-insurance-consumer-protections/

Personally I want health insurance severed from employment. That was part of McCain’s proposal during the last election; I recall thinking that the overall proposal was quite bad but I liked that bit. If we had a reformed GOP the McCain proposal would be a starting point for a great discussion on health care reform, but since we have the party of Cheney/Palin/Limbaugh we won’t get to have that discussion.

In the absence of a rational discussion on health care I’ll go with these eight items. They will raise premiums for large corporations (I’ll pay more) while shrinking premiums modestly for smaller businesses and by large amounts for individuals. That’s good in my book.

These proposals would also have lots of unexpected consequences, but that’s the nature of policy …