It's not over. The rise of second generation spam.

First generation spam was pretty bad, but it's more or less under control now. Between sharpening spam recognition algorithms, crowd sourcing, and managing the reputation of authenticated sending services Google has beaten back the tide.

So that's it for spam?

Heh. Of course not. Now we have second generation spam.

Second generation spam does not use forged headers -- though the headers do seem to change a fair bit. This spam is not anonymous, it markets real goods, services - and politicians.

The goods and services aren't too hard to manage. I created a filter that sends anything from "buy.com" to the trash -- that took care of 80% of it.

The politicians are much worse. I get daily spam from fund raising politicos, PACs and other accessories to the political process. I now have about 25 Gmail filters that do nothing but delete all incoming email from their domains. The domains typically last a few months, and then there's a new crop. At this rate I'll have 200+ Gmail filters that delete email from largely defunct domains.

What? Ask to be removed from the lists? Clearly you're just toying with me. I tried that of course, but it doesn't work. I just get added back in they next time some politico buys a list. (Maybe I should start forwarding to spam@uce.gov as well?)

It's hard for any ISP to block this kind of spam. Politicians generally exempt themselves from laws that slow fundraising; if Google blocked their spam they'd be asking for a world of hurt. Better to get between a Grizzly and her cub than between a politician and your wallet.

We need a different approach to political spam. Sorry, I have to vote for some these dorks -- better spam than Palin and her ilk! So changing my vote's not enough. Any ideas?

I do have one quick fix. Google could add a "blacklist all from this domain" to the message action select menu. Choose it and the message is deleted and the blacklist entry created in a one move.

Another related fix -- allow Gmail users to share their blacklists. So Google wouldn't get in trouble, because we'd be choosing what block.

Any other ideas?

Climategate: Gordon Speaks

Ahem.

I've been reading my always excellent blog sources, so I'm ready to comment on Climategate ....

Climatic Research Unit e-mail hacking incident - Wikipedia

... Unidentified persons allegedly hacked a server used by the Climatic Research Unit, posting online copies of e-mails and documents that they found.[5] The incident is being investigated by Norfolk police[18] and involved the theft of more than 1,000 e-mails and 3,000 other documents,[9] consisting of 160 MB of data in total.[12] ...

For my own record, here's my take:

1. UK researchers have a very innocent approach to email. In the corporate world we write email the way I used to write my medical notes -- to be read in a courtroom. Remember Lomasney chaps.
2. If the released emails are the worst the hackers found, there's not much of a story here.
3. I'd make a solid wager that five years from now the climate consensus will not have materially changed. The science will stand. (I would love to be able to invest in a Climate Futures Market. I hope we get one.)
4. Scientific fraud is not rare. So it must always be considered. There is, however, stronger evidence of fraud among the solar forcing research community and among the denialist astroturfers.
5. Research data is money, power, tenure, fame, grants, hot babes (ok, 5/6) -- it is the currency of science. I sympathize with scientists who want to hold on to their data, though not with the Journals that may impede open sharing. In this case, however, we are talking about research with inestimable implications. In the case of Climate Science, we must insist on an unusual degree of access to research data. There's already progress but sharing is not natural for most scientists. Encouragement will be needed.
6. I'm looking forward to learning who the hackers are, and what their motivations were.
7. There's room for rational disagreement about the risks and approaches to global climate change. The denialist community is making it hard to engage in that dialog. That may be one of the more pernicious effects of their quest for fame.

--
My Google Reader Shared items (feed)

Complexity attack – the illustrated version

Synchronicity? Probably not*.

Dilbert experiences a complexity attack (click through for full strip) …

See also:

• Gordon's Notes- Left Behind- Ludd, Beck and the non-tribal roots of tea party rage
• Gordon's Notes- Health insurance- we're defeated by a complexity attack
• Gordon's Notes- The Empire Strikes Back- complexity, mobile phone plans, and Apple defeated
• Gordon's Notes- The hidden inflation of low quality

*No, it’s not that Scott Adams reads Gordon’s Notes. I’m really not that delusional. He’s very good at the zeitgeist, and I think people are catching on how complexity is being used as a commercial weapon.

In Our Time - Siege of Vienna seems familiar

There was something familiar about the Polish horses rushing down from the mountain top to end The Siege of Vienna.

Oh, right. Gee, I wonder if Tolkien knew that history.

--
My Google Reader Shared items (feed)

The Family Telescope: Orion Dobsonian or Edmund Scientific Astroscan … or Orion StarBlast?

I'm looking for advice - so please comment!

I would like to get a telescope for family astronomy (really, my 7yo daughter and I). Happily, unlike my high school days, I don't have to grind a mirror. We'll have to travel outside the city to see much, so ease of transport and setup is important.

I've more or less narrowed things down to two quite different devices:

• Orion SkyQuest XT6 Classic Dobsonian Telescope (Or the smaller and lighter XT 4.5)
• Edmund Scientific Astroscan Plus (the famous Dans Data review)

Both devices have excellent reviews. The Astroscan is more rugged, requires no assembly, is very easy to transport, and provides great star field views. The Orion Dobsonian is better for planetary viewing.

Any thoughts?

 

Update 12/3/09: I ended up choosing none of the above, and went for a well regarded scope that’s a bit of both – the Orion StarBlast 4.5 and a separate 2x Barlow lens. I waffled about getting the same StarBlast with an equatorial mount, but ended up with the original mount because Amazon and Orion offered free shipping on the simpler scope, the equatorial is pretty finicky to assemble, setup, and transport (we live in the city, so viewing requires travel), storage space is an issue in our home, and the primary user is my daughter who’s still 7 yo size. I’ll post later about how I feel about that decision after we’ve used the scope a bit.

 

There’s also a StarBlast 6 now, though it’s not sold by Amazon. I felt that might be stretching the limits of the basic design, but clearly I’m no expert.
--
My Google Reader Shared items (feed)

Bad web sites, genetic fitness, and choosing a service provider

The way I found the current swim schedule at a community recreation, fitness and cultural center was to find the link to the future schedule, guess what the current link would be, and hand edit the URL. This isn't a new problem; their web site has been awful for at least five years.

I've written about why automotive sites are so bad, despite their vast economic value. None of those reasons apply in this case. So why has this facility had such an awful web site for so many years? On the face of it they have a very simple task; they need to put up pages with contact information, maps, PDF schedules, etc. Why does it go so wrong?

I'll get to that, but first I'll inflict some wisdom of the aged. Feedback won't help. When something is this bad even kindly suggestions won't help. They'll only cause hurt feelings. The problem must have deep roots.

My guess is that this rec center has fallen prey to one of the institutional weaknesses of the well meaning non-profit. Maybe it's become the job of someone who likes it but is unsuited for the task. Perhaps this person can't or won't be moved. Maybe it's become the job nobody wants to do. Perhaps its a form of passive-aggressive self-mutilation.

Good leaders figure ways out of these traps, so the persistence of this organization's problems suggests deeper leadership issues. We can think of an organization's web site as the equivalent of a biological organism's "fitness displays" -- such as big muscles or symmetric features. Web site weaknesses are a good measure of deeper institutional flaws.

Which brings me back to our automobile purchase, which has been stalled for the past month (the 2010 Subaru models do not impress). I'll look for the best automotive web site to guide our next cycle of car purchasing. Likewise if we were looking for a new swim facility, I'd look for a well organized web site.

A modern web site has a lot of meanings. It's hard for a dysfunctional organization to create and maintain a quality web experience. It's not a matter of money, it's a matter of the sorts of things that have to come together and stay together, and the many mistakes that have to be avoided. Web sites can be an important fitness marker for institutions; the good ones are both smart and pretty.

--
My Google Reader Shared items (feed)

Fermi Paradox review article - 2005

A comment on a past post of mine referred me to a 2005 article: SETI and the Cosmic Quarantine Hypothesis.

The essay is a good summary of the Drake Equation and its relationship to the Fermi Paradox. The author is clearly an optimist, he imagines a benign super-civilization blocking aggressive expansionism. That was the theme of a famous 1970s-era science fiction series, except the interventions were not benign.

It's a pleasant thought, but it seems unnecessarily complex. A simpler explanation is that all technological civilizations run into singularities long before they can attempt serious star flight. Whatever happens thereafter, it doesn't involve any wandering we could see. (Charlie Stross included a clever variant in a book - he speculated the post-singular civilization couldn't abide the poor connectivity of wilderness living.)

Mr. Soter misses the singularity effect in his estimation of the lifespan of civilizations. He's right that mere eco-catastrophe would not eliminate humanity, but technological singularities are (imagined to be) a different sort of extinction/transformation event.

I did learn one new thing. The novelist Michael Crichton, in addition to despising concerns about global warming, also hated the Drake equation. He was a kind of anti-Gordon, but a bit richer and better known than I.
--
My Google Reader Shared items (feed)

Discovering medical prices and the problem with paying cash

Walecia Conrad has written a pretty good summary of the problems of price discovery in medical care services. She mentions several approaches, including online services, calling physician offices, and checking payor (insurance) sites.

None, of course, are satisfactory. I hope she'll dig a bit deeper into price discovery. There are two places she could learn from. One is the problem with discovering how much a medication costs in different forms through different vendors with different coverage plans. Good luck on that one.

The other topic is more amenable to digging. She almost got into it, but perhaps had to set it aside for another day. Ms. Conrad mentions that cash fees for medical services are usually much higher than the negotiated fees insurance companies provide (this is very relevant to health care reform of course).

What she missed is why.

My own recollection, for I no longer deal with this issues, is that payor (insurance) reimbursement is based on a fraction "list price". So imagine that Blue Scythe pays 50% of list price. If costs+margin means a services costs $50 physician must then set "list price" to $100 so they get $50 from Blue Scythe. The "list price" must be validated as a customary charge, so it must show up on bills -- including bills for people who pay cash.

This means people paying cash are providing a huge margin, but this is an unwanted embarrassment for most practices. In my day we wanted to charge people paying cash less, not more.

I think there may be ways around this now. My knowledge is at least fifteen years old! Still, this an area that deserves some journalistic effort.

Lagrangian finance and age of wonders

I had a relatively decent physics education for a non-major, but I'm pretty sure it didn't include discussions of the Lagrangian and its use in Newtonian mechanics. So I loved this brilliant exposition of the use of the Lagrangian.

Subtracting potential from kinetic energy? It feels like a measure of how much of a budget is unspent, how much is left to drive deviations from a trajectory...

That feels like finance. There ought to be applications of the Lagrangian to the world of financial modeling.

Once upon a time, that's where the thought would have ended. An idle speculation. Today, though, the answer is a few keystrokes away: google.com/search?q=lagrangian+finance.

73,800 hits. Yeah, looks like there's an application or two.

We don't truly grok the web. Not yet.

Left Behind: Ludd, Beck and the non-tribal roots of tea party rage

It's easy to mock the Palinistas, the Beckians, and the very vanilla Tea Partiers.

Easy, but wrong. They have good reason to be afraid. Their future looks grim. They are also, I think, the misleading face of something bigger.

Misleading, because the melanin-deficiencies of the Beckians makes it too easy to think of their roots purely in terms of the white underclass. I think there's something bigger, more interesting, and more important going on than the historic passing of one particular tribe.

The world is becoming increasingly complex and, for many people, inaccessible. How smart do you have to be to learn and absorb the multiple manifestations and implications of modern mortgage contracts, American health care benefits, mobile phone plans and scams, mortgage derivatives and exotic financial instruments, iPhone synchronization, Facebook and Twitter privacy policies, dysfunctional security procedures, OpenID, Oauth and 200 passwords, modern India, China, tribal Pakistan, your flaky modem, and, for most non-Mac owning Americans, virus infested XP boxes with unpredictable behaviors?

A lot smarter than me.

Most of humanity is being Left Behind by an increasingly incomprehensible world of increasing complexity. We can function in it, we can even prosper in it, but our world is beginning to resemble the world of pre-industrial man. That is, a world of powerful and mysterious forces that may, without obvious reason, aid or smite a mere mortal.

I suspect, in their marrow, the Beckians, like the Luddites before them, feel this. It adds to their fear and anger. This will be something to watch.

See also:

• Gordon's Notes: Emily's new cell phone and the cost of complexity
• Gordon's Notes: Health insurance: we're defeated by a complexity attack
• Gordon's Notes: Employment benefit complexity: we are sheep
• Gordon's Notes: The Empire Strikes Back: complexity, mobile phone plans, and Apple defeated
• Gordon's Notes: The hidden insurance problem: they can play the game better than we can
• Gordon's Notes: A smart mind is a dangerous thing to waste

My Saint Paul Minnesota speeding ticket – emergent solutions to emergent consequences

A few weeks back I got my first ticket in over 12 years, for speeding on a notoriously deceptive stretch of highway 35E (it even has a Wikipedia reference under speed limits). For a week or two around the time I was dinged that stretch of road showed caught cars every time I passed.

There were, of course, lots of mitigating factors, but the biggest one was that I’m really dependent on cruise control and mine broke about a month ago. Probably from overuse.

Once upon a time, if you had an average income, a speeding ticket was an painful annoyance. Then it began to dramatically increase insurance costs. Later, as the world became more risk-adjusted and networked, speeding ticket questions began to show up on applications for life insurance, foster care, and I suspect, many more forms.

So the secondary costs of a modern speeding ticket are much higher than the list price. I decided to poke around rather than pay immediately.

Google and the rest were not helpful. This search topic turns up mostly misinformation (it will be interesting to see if this post helps!). The searches did bring up ads for various legal services, and my first instinct was to pay for some professional advice. The more I looked, however, the less savory that industry seemed. In the end I decided to “contest” the ticket. This is the story of what I saw of the system, and how an “emergent” solution was developed to the secondary costs of speeding tickets.

There’s a common myth that when you contest a ticket you appear before a judge along with the accusing officer. At this time you can supposedly argue about radar technologies and so on, but if the officer doesn’t show up your citation will be waived. That’s not how it works in Saint Paul.

You phone a number (no web) on the ticket and you’re given a hearing date with an administrative official. You can change the date; I had to due to a travel conflict. If you run late there’s an extra $5 late fee. (I think you can “appeal” this decision and end up in a real courtroom, but I didn’t go that far.)

The hearings take place on 15 West Kellogg, in the city court house. It’s an imposing structure with a vast ceiling and black marble columns bearing the names of dead warriors. It reminded me of a scene from the movie Brazil.

I joined about ten others sitting in a mildly gloomy room. There’s a display showing names and appointments, on which my name did not appear. Turns out that’s for the court, not for traffic citation. Most everyone else there seemed to know what to do; at around 8am a set of metal windows crash upwards and you cue up for a hearing slot.

Despite being last in line I was called in at my appointed time. I’d rehearsed my responses, but this was the complete discussion: “The speeding was an accident, I have a good record …” “You mean you were accidentally speeding?” “Yes". “Give me your license”.

The system, it turns out, has developed a solution to the problem of the increasingly heavy consequences of a speeding ticket. The solution was that I agreed to pay $188 (I don’t know how the total was calculated, it might be more than the ticket price) and the citation, for the moment, no longer exists. If I get another ticket in the next year it will restored and I will have to pay both. So local government gets at least the money the ticket would have generated, and if I make another mistake they get double the money. On the other hand, I don’t bear the secondary consequences of a modern citation.

An emergent solution to a modern dilemma. Fascinating.

Plus ca change – Non. Twenty years of a solstice letter.

I’ve been writing a “solstice letter” for over 20 years.

During that interval a few things have changed. The first letter would have been written with WordPerfect 3 on a Panasonic* 8086 with a 20MB hard drive. Today my local storage total is roughly 2TB, or about 100,000 times larger. I had email then; I used Norton Commander’s superb MCIMail client with MCI’s pre-internet modem-based mail service. Today I use Gmail.

Oh, and now we have the web.

I’m more interested, however, in what’s not changed. After many years of experimentation I’m back to authoring in a word Processor (Word:Mac) and distributing as a PDF from one of my personal servers. I’d love to have a great web based authoring solution, but there isn’t one. I’d love to have a universal open file format, but there isn’t one.

In this area, progress is only measurable by microscope.

* The most over-engineered device I’ve every purchased. You could park a car on it. Panasonic was threatening to wipe out Compaq in those days, until Congress intervened to block Japanese computer exports. That “saved” US computer manufacturing until Taiwan took it away, and forced Japanese manufacturers to focus on laptops. Adjusting grossly for inflation, it was cheaper than a comparably “higher end” machine would be today. But I digress …

The shocking truth - the birthers are almost right ...

The birther crackpots are still stumbling about. They'll never figure out the real story -- though it's been staring them in the face all along.

Of course Obama wasn't born in America, but he wasn't born in Kenya either. You see, he wasn't born at all ...

It does explain a lot.

--
My Google Reader Shared items (feed)

529 plans can be rebalanced twice in 2009

I was surprised to see this when I visited our kids 529 plans to make a contribution. I assume this extra rebalancing is to compensate for funds damaged by the crash of 2010 ...

The Treasury Department and the IRS recently announced that for 2009 only, 529 plan account owners will be allowed to change Investment Options two times per year. This means that you can reallocate your investment to different investment options in your plan up to two times this year...

I've never done any rebalancing. I didn't realize that we are normally allowed to do that once a year. Something else to learn about!

--
My Google Reader Shared items (feed)

Lipid guidelines: Sometimes the emperor really is starkers

A year ago I had to review guidelines for managing elevated blood lipids. I concluded that the guidelines were incoherent and silly. It wasn't a problem of science, it was a problem of logic.

Today, reading a JAMA editorial by Gaziano and Gaziano (brothers), I see that medicine has caught up with me. The risk calculator approach makes sense, though the models may have problems.

It's a cautionary tale of the limitations of expert panels. I suspect a lot of practicing physicians thought the guidelines were dumb, but they weren't making policy.

Hey, someone has to give me credit!

PS. There are now serious proposals to put every male over 50 or so on a statin. That's because they're so safe and cheap. The last time we did something like this it was women and estrogen. It took 10 years to learn that was a very bad idea.

--
My Google Reader Shared items (feed)