AOL and Yahoo: email down the tubes

AOL has been on a long slow death spiral for about 10 years, but I didn't realize Yahoo was in dire straits until I read this announcement from my ISP:

VISI | Announcements | Difficulty sending mail to yahoo.com or aol.com?

Over the past weeks, it appears that Yahoo has begun grey-listing all (or most) incoming mail. This means that they are rejecting the first mail delivery attempts and telling sending servers to try again later. Yahoo also appears to be grey-listing with content filters. In this case, customers may see the error message: message text rejected by mx1.mail.yahoo.com: 451 This message indicates that suspicious content was detected, but that the sending server may try again.

For mail grey-listed automatically or by IP, users may see: : connect to x.mx.mail.yahoo.com[209.191.aaa.xxx]: server refused mail service You may also see error code 421 in the error response.

Generally, this email is also being retried, however, if retried too soon, it will be rejected again. It may even be rejected permanently by Yahoo with no change in error message that we have found. Yahoo's documentation claims that they are not grey-listing, but instead are filtering mail based upon the sending server's compliance with standard mail practices. Our servers, however, are compliant, but we are still seeing significant deferrals. Yahoo is also testing DomainKeys verification, which we are reviewing to potentially mitigate the problem. There appears to be no way to contact Yahoo about this except via web forms that do not generate any response except confirmation of receipt. We recommend that any users forwarding email to yahoo.com addresses cease forwarding or redirect to another location.

Of course, this affects not only customers forwarding mail to Yahoo, but ANYONE attempting to send mail to Yahoo addresses.

AOL AOL uses an automated system to block mail from potential spam sources. When mail is reported as spam by users, the IP addresses for servers used to transmit the mail are recorded, and, once their limit has been reached, IP addresses are blocked from sending mail to AOL for 24 to 48 hours. This can be exacerbated by VISI customers forwarding email to their own AOL accounts and then reporting any forwarded spam, which can result in temporary blocks of VISI mail server IP addresses. The automated system is COMPLETELY automatic, and no intervention is possible in expediting removal of IP addresses. Unfortunately, this will affect ANY customer attempting to send to AOL addresses, not just forwards to AOL accounts. As with Yahoo, above, we recommend that any users forwarding email to aol.com addresses cease forwarding or redirect to another location.

I ran into a variant of this problem with Gmail. I was redirecting an unfiltered email stream to Gmail, and when I read the mail in Gmail I "marked" the spam. Alas, Gmail looks at the redirect as the source of the email, so the more I marked as spam the lower the reputation of the redirector fell. Over time Gmail marked more and more valid emails as spam, and missed more and more spam. I fixed it by filtering the mail stream, and never marking anything that was redirected as spam (I just delete it).

The Yahoo and AOL bizarre responses to the spam deluge tells us how dire their financial situations are, but I must also say that Visi should have figured out DomainKeys a year ago. Maybe Yahoo is doing this in part to force adoption of DomainKeys; too bad their execution is incompetent.

In the meantime, encourage anyone you know who's still using Yahoo or AOL to get out fast and switch to Gmail.

Update 12/21/06: There's a good defensive strategy for those of us still using SMTP services (non-webmail) btw. Get a Gmail account and configure your dedicated email client to use Gmail's smtp service. If Google is your sending service, I suspect Yahoo and AOL won't be blacklisting the sending domain.

Google is NOT slowing down

Google folk were rumored to be catching their breath. No. Rather than doing new products, they're extending existing products in innovative ways and discovering innovative, micro-NLP based, ways to do ad-hoc interoperability across Google and with non-Google products.

I have an embedded link one one of my blogs to a Custom Search Engine. Today when I used it, a dialog box appeared asking if I'd like to add this custom search to the button list that appears next to the search box in my Google Firefox 3.0 beta toolbar.

I said yes, and now it's there. When I get home and login to Google/Firefox, I would not be astonished to see the button migrate.

Today I noticed the addition of the 'call' link to Google local search, as well as a few other clever tweaks (though they need to do better about exposing links so one can send a link to recreate a local search context).

Google is not slowing down. If anything, they're ramping up. Scary and exciting.

Mankiw plays tricks, but is saved by a Dave Barry column

Greg Mankiw, a respectable Republican, claims that the study of economics makes students more "conservative". Of course he's playing semantic games; he knows his more naive readership will equate "conservative" with Republican. It's only at the end of his post that his conscience forces him to confess that he really means "classically liberal"; in other words the very antithesis of modern Rovian Republicanism. Classic liberalism (championed by The Economist in the 1980s and early 1990s, and parodied by The Economist ever since) is a respectable doctrine, albeit one that struggles with "the problem of the weak".

Despite this bit of weaselly trickery, the column is redeemed by a link to a classic Dave Barry column. I do miss Dave Barry's writing, but he seems to have decided to semi-retire. Dave Barry in his heyday was a rare public and populist intellectual, of a sort we most desperately need.

Car rental: enter gold number, price jumps $200

The 'arms race' of modern pricing continues apace. I priced a personal 5 day van rental twice on Travelocity - once with no loyalty number added and again with the loyalty number.

The price of loyalty, was a $200 increase. Yes, I would pay Avis for the joy of being a loyal customer.

To their credit Travelocity listed Avis twice after I entered my registration number, once at the disloyal price and again at the inflated loyal price.

These days it is increasingly foolish to do any price negotiation directly with a travel related vendor.

Against Stupidity: Citibank Visa security

I was a huge Asimov fan as a kid. He died young of AIDS (blood transfusion); one of his last books was probably his very best. The title was a winner - 'The Gods Themselves'. It was taken from a quote by Schiller 'Against stupidity, the gods themselves, contend in vain.

I thought of that quote when Citibank Visa asked me to provide answers to not one, not two, but three! (or four?) "security questions". Information only I am supposed to know. Top secret information, that will be firmly protected with all the customary security employed by banks and credit card companies to secure customer data.

Meaning I might as well publish the answers in my blog. Imagine how much easier these security questions are making identity theft. Add them all up, and there's no "thing only I know" that won't be known to a potential thief.

I did manage to keep my responses printable, though they're not at all memorable or useful. If Citibank ever requires me to answer them I'll switch to another Visa franchise.

Dumb.

Avian Influenza: A guide for the interested layperson

American Family Physician is a review journal for FPs. Unusually, all of the journal is freely available on the web. The quality is usually good; the best articles are written by family physicians. The very best are so clearly written that anyone with a basic interest in science can follow them.

Gregory Juckett's review of avian influenza (H5N1) is top notch, and is only a bit more technical than the Scientific American. Highly recommended for the curious. A few tidbits that I took away:

• Like the 1919 (H1N1) pandemic death is most often from acute respiratory distress syndrome and is probably due to a hyperactive immune response. That's why mortality is high among young adults -- they have the most aggressive and twitchy immune responses. The most promising therapy involves 'statins' (drugs like Lipitor) that [surprise!] suppress the cytokine component of the immune response. [jf: Cytokine suppression is not always a 'feature'; one must wonder how many times statin-induced immune suppression is harmful or lethal. I'm sure we'll here more about this over the next year.]
• The early returns suggest the lethality of the current H5N1 strain of Avian influenza is more comparable to the 1957 H2N2 or 1968 H3N2 lethalities, so not in the same league as the 'Spanish' flu.
• The Swine flu of 1976 was an H1N1 strain. We still don't know why it didn't wipe the floor with us. President Ford ought to have earned accolades, not scorn, for the emergency vaccinaton proram later associated with an inflammatory polyneuropathy.
• Ventilator availability is a major problem for Avian flu response. We can't make Tamiflu faster (Star Anise supplies have some production limit.), but we could make a lot more portable vents. If we don't need them, we could donate them to other nations.

The AAFP has launched a practice-oriented support web site. (Sadly, the URL was botched in the 9/1/06 editorial. You think that by now they'd have setup a redirect! I'll send them a note.)

A plea for Google: meta tags for dates

A plea to Google, inspired by the ancient pages on my legacy hobby site.

Google Groups: Crawling, indexing, and ranking

I still get emails of gratitude from visitors to my legacy personal web site, even for pages that haven't been updated in five or six years. Much of the material is of historic or special interest -- some of it goes back about 10 years -- but there's a place for such content.

Alas, it's noise for most searchers, especially since Google can't handle date constrained searches very well. Which leads to a plea for Google to support date range meta tags.

If Google supported a 'creation date', 'last revised date', and 'archival date' users could create searches that would either filter out, or focus on, old pages. Sure crooks and scammers would produce invalid dates, but the bias would probably be to create invalid new dates. The value of false archival dates would be much less, so Google's algorithms could make inferences about the utility of the date information and act on it. (Archival dates are more likely to be true, date information from unchanging pages more likely to be true, date information from higher ranked pages more likely to be true, etc, etc.)

Thus my plea.

Retail organs: not a slippery slope

The transfer of organs from the weak to the strong, from the poor to the richer, is not a slippery slope. No, not at all.

Slippery implies some possibility of friction. Slope implies the possibility of balance. We need a better metaphor. How about 'obvious cliff'?

Alas, the trade continues to expand exponentially, despite my screed of last April. The Economist is the latest champion.

Gee, you'd think nobody reads this thing. The egg-donation and kidney transfer trade is big these days, much bigger than the involuntary donations of Chinese "criminals". It's a true 21st century growth industry. Niven, alas, was spot on thirty years ago. If we come up with really good anti-rejection treaments the exponential growth curve will go vertical. Eye transplants anyone? After all, one can live well with one eye.

Sigh.

There is a darkly millenial bright side. Sooner or later, maybe after the eye donations and the hemi-hepatectomies are booming, this trade will tip us into reexamining the duties of the strong to the weak, the rich to the poor, and the limited adaptability of the human to a logically utilitarian ethos.

I'm sure I'll have similar comments in another 6-12 months.

Common Good Books: A place to visit in Saint Paul

Garrison Keillor, a wealthy celebrity and twin cities fixture, has decided to blow some cash on an independent bookstore called
Common Good Books. As best I can tell it doesn't have a web site or a marketing budget. It's downstairs from Nina's Coffee Shop, an upscale bohemian hangout in what was once a chancy neighborhood. Purely by coincidence, it's across from Representative Betty McCollum's office. (Keillor is a hard core democrat.)

Emily and I wandered in, and fell in love with both the bookstore and Nina's upstairs (there's a staircase from inside Nina's to the bookstore, Nina's has wifi). It's not a big place, but every book is remarkable. The reading nooks with the overhead skylight are irresistible. It reminds me of the much grander bookstores west of the University of Chicago, and of a much mourned East Lansing fixture that died after an ill-fated move. Odegard's of Saint Paul was like that, but perhaps a bit more commercial.

Keillor is wealthy enough to fund bookstore for decades -- if he wants to. There are worse ways to lose money. Oddly enough, the bookstore and the location are sufficiently appealing and unusual that, despite the negligible marketing budget, he might one day break even ...

Anti-war demonstrations: why we don't have them

This Slate article provided a sensible and analytic answer to the question about where all the anti-war demonstrations have gone. The biggest piece they missed, and it's quite big, is demographic. We're much, much, older than the society that demonstrated in 1972. I liked the last comment best, and I've excerpted it. The entire piece is brief and well worth reading ...

Why you're not demonstrating against the Iraq war. - By Jacob Weisberg - Slate Magazine:

... Lastly, there is the matter of the Iraq war protests themselves, such as they are. Have you been to one? Demonstrating in the '60s, I gather, was a lot of fun. You went for the politics but stayed for the party—or was it the other way around? Forty years later, antiwar rallies are politically and socially disagreeable. The organizers are inevitably moth-eaten left-wing sectarians, some of whom actually do favor the Iraq insurgents. The sane or rational are quickly routed by the first LaRouchie, anti-Semite, or "Free Mumia" ranter to grab hold of the microphone. The latest in protest music has much the same effect.

Weisberg points out that our mortality rates are much less than in Vietnam, and this reduces the emotional impact of the war (Iraqi casualties, alas, don't count. We are human that way.). I agree, but I wish Weisberg had pointed out that the public has been very uninterested in the number of veterans with traumatic head injuries who will suffer lifelong disabilities. That's a failing of both the media and the US public.

Panoramio and Google Earth

Panoramio is a Google earth geo-location photo mashup service. Upload photos, provide geo-location, and people view them via Google earth or as a Google Map mashup.

The Google Earth integration is particularly impressive.

Once cameras all integrate geo-location, and even target geo-location (trickier), this will all get easier, but the results are impressive even now. Try flying Google Earth around San Franciso with the Panoramio layer enabled...

If you're a photo hobbyist who enjoys landscape and city scenes, you can build karma by using Panoramio to show Google Earth passengers the world ...

Why college tuition continues to increase: Mankiw

This is what I've long thought, but Mankiw is a top flight economist:

Greg Mankiw's Blog: On College Tuition

.... One reason college tuition has risen was explained by economist William Baumol. Consider an industry that uses only labor in production and experiences no technological progress, assumptions that arguably approximate colleges and string quartets. The price of its output will have to grow with the price of labor. The price of labor (the real wage) will, in turn, grow with economy-wide technological progress. Using the numbers in the above table from the Times, one finds that Harvard tuition has grown at 2.8 percent per year (note that this is adjusted for overall inflation). Real GDP per capita grows about 2 percent per year--a rough measure of economy-wide technological change. Thus, much of the increase in tuition, but probably not all, can be explain by the Baumol effect.

3. Over the past thirty years, the college premium has risen substantially. That is, workers with college degrees have enjoyed stronger wage gains than those without--a phenomenon often attributed to skill-biased technological progress. This rising college premium has had two effects on college tuition. First, colleges use a lot of educated labor in producing their output, so their costs have risen faster than they otherwise would. Second, the rising college premium has increased the demand for the services of colleges. Supply shifts left, demand shifts right, and the price unambiguously rises.

4. Colleges have gotten increasingly good at price discriminating. (Recall the discussion of price discrimination in chapter 15 of my favorite economics textbook.) The list price is set high, and then many customers are offered a discount called "financial aid" based on their ability to pay. Here's the secret plan: In the future, Harvard will cost $1 billion a year, and only Bill Gates's children will pay full price. When anyone else walks through the door, the message will be "Special price, just for you.

The implication of #1 is that smart buyers can get a bargain courtesy of those who are unable to judge quality. My own experience is dated, but I have never seen evidence of a correlation between quality and price in the many educational institutions I've attended and the two that I've taught at.

Apple's feet of clay: OS X Simple Finder

You can't delete a file using OS X Simple Finder.

Yes, we all know that the Finder is flawed, that Apple broke their beautiful Classic OS file indirection system with OS X, that Apple's metadata management (file type, creator, etc) was screwed up in OS X compared to classic, and that OS X's smb network services are feeble -- but these are all minor flaws compared to Simple Finder. At first glance this looks like a great way to introduce a non-expert users to OS X, but the pretty face is deceiving. Simple Finder in Mac Classic (OS 8+) was a great piece of work, in OS X it's proof positive that Apple can be as incompetent as Microsoft.

Don't do what I did. Don't spend hours trying to make Simple Finder work as a user environment.

Yech.

iTunes sales and the status of DRMd music: next steps

Infinite Loop: iPods, iTunes, and iDiots—Forrester says iPods don't drive iTunes sales is a good rant on a recent NYT article claiming iTunes sales are declining. It's a great rant, though I would not be surprised if a lot of people are realizing that even the relatively enlightened iTunes DRM strategy is a non-starter. Ok, a bit surprised. I didn't think people would figure the scam out this fast.

If it is a some great awakening, if enough people have run into DRM problems that they're soured on the whole idea, then the entire digital music industry will need to reboot. It's not a biggie for Apple -- they make their money on iPod hardware sales, but it's huge for everyone else. Note Yahoo is now selling non-DRMd music ...

I posted a comment about the rant and some of the comments. Excerpts below:

• We also have 3 iPods in active use and may add a Shuffle. All from one music Library. Of course the interesting point here is that copyright holders HAVE NEVER APPROVED OF A FAMILY LIBRARY. So by sharing the music library with my spouse and children on separate iPods I'm probably 'stealing music' as far as the RIAA is concerned. They would say that each person should have their own library, irregardless of relationships.
• ... the beauty of the iPod was making our hundreds (thousand?) CDs new again. It takes a long time to explore that much music, so many consumers may have a very long latency period before they start buying new music again -- whether classic CD or DRMd.
• In terms of IP theft I wonder if the biggest methods now are merging iTunes Libraries (attach external drive, drag and drop, it's easy as pie) and ripping tracks from Library CDs and from purchased used CDs that are quickly resold to the dealer. I'm sure the RIAA knows that, but it's not something they talk about much. They prefer to think about file sharing.
  
• Controlling all those non-DRMd CDs in the world is a tough task. The way to do it, of course, is rather like gun control. Buy up all the CDs on the market and then destroy them. In time the price of used CDs will rise to the level of new DRMd CDs. In fact, smart people should start hoarding used CDs now in anticipation of when the the prices will rise. The next step is to make it impossible to play non-DRMd digital tracks or CDs/DVDs. It's a big project, but I'm sure the RIAA is working on it.
• Lastly, it's not really that hard to get real data on what's happening. Medical researchers study far more sensitive topics than this all the time. The issue is that only the vendors will pay for the research, and they won't share what we find. So we'll make do with rumor and anecdote.

Realm of wonders: Ocean census VI

The Independent has an excellent and concise summary of the results of Ocean census VI. The range of what is "possible" in terrestrial organisms continues to expand. If we are still making discoveries of this magnitude in 2006, it is overwhelmingly likely that many more astounding discoveries lie ahead. I am looking forward to the inevitable coffee table book companion to this research report.