Google is indexing the full content of the scientific literature

Nature Publishing Group: science journals, jobs and information

From a UMLS list posting by Mike Cumments:

Google appears to be getting ready to index all of the full content of the scientific literature. They are doing a pilot with publishers until the end of 2004. At this stage they seem to be focused on simply doing a PageRank type search without delving into the semantic content. They do a standard Google search and apply a filter for only the URLs of the 9 publishers.

See http://www.infotoday.com/newsbreaks/nb040503-1.shtml for background.

The project description is http://www.crossref.org/crossrefsearch.html.

For a search of the full text content of these publishers using Google go to http://www.nature.com/dynasearch/xrs/

From the infotoday article:

CrossRef (http://www.crossref.org), a 300-member publisher trade association, has announced a pilot project called CrossRef Search that will enable users to search the full text of scholarly journal articles, conference proceedings, and other sources from nine leading publishers. Google will supply the search technologies and CrossRef the reference links to publisher Web sites. While Google will also incorporate CrossRef content connections into its general Web search engine, users who go to publisher Web sites and click on the CrossRef Search icon will reach just the scholarly subset. However, searching through the icon will access content from all participating publishers...

... Searching CrossRef Search will be available to all Web users at no charge. Content will include current journal issues as well as back files. The system uses CrossRef’s DOIs (Digital Object Identifiers) or standard URLs to identify and link to content.

At present, publishers participating in CrossRef Search are:

American Physical Society (http://prola.aps.org/xrs.html)
Annual Reviews (http://arjournals.annualreviews.org/search/external)
Association for Computing Machinery (http://portal.acm.org/xrs.cfm)
Blackwell Publishing (http://www.blackwellsynergy.com/servlet/useragent?func=showSearch&type=external)
Institute of Physics Publishing (http://www.iop.org/EJ/search)
International Union of Crystallography (http://journals.iucr.org/; click “search” and scroll down the page)
Nature Publishing Group (http://www.nature.com/dynasearch/app/dynasearch.taf)
Oxford University Press (http://hmg.oupjournals.org/search.dtl; each journal’s search page includes a link)
John Wiley & Sons, Inc. (http://www3.interscience.wiley.com/crossref.html)

These initial publishers produce some 1,100 journals, according to Pentz. Participants have investigations underway to test how to use DOIs to improve indexing and metadata for better retrieval and to enable persistent links from search results to the full text of content at publisher sites.

The initial pilot will last throughout 2004. CrossRef plans to gather feedback from scientists, scholars, and librarians through e-mail forms and formal evaluations using external consultants, according to Pentz. CrossRef is also hoping to discuss similar programs with other search engines, Pentz said.

There are only two rules for joining the pilot program, according to Pentz. “The publisher has to have all their content indexed through the way Google indexes and make the search box available to everyone at no charge.” ...

---

I doubt there's a large amount of money in this for Google, but it fits with the social mission of their CEOs. I doubt it makes their VC's happy.

Canopy Economics

Google Search: "canopy economics"

---

I was thinking about economics the other day. I used to think of the economy in terms of plate tectonics. When the fundamentals change, when the plates move, the visible surface is transformed.

Nowadays though, the surface seems very persistent. Even when the ground shifts, processes, institutions, entities, persist. Zombie processes stagger on beyond their life expectancy. The economy seems to have a "mind of its own".

So now I think of economics in terms of the "canopy", the layer of the forest that lies above roots and trunks. In a tropical forest I believe the trunks can die, but the canopy -- suspended by interconnections, can be relatively undisturbed.

Call this "Canopy Economics".

A google search found no matches on that phrase. Now there is one!

Update 9/11/06: Well, there are more hits on the Google search now -- but they're all 'splogs'! That is, computer generated web pages designed to trap search engines. I guess they harvested the concept from this page. Hmmm. What would be the neurologic and viral analog of that? DNA fragments, harvested and recombining, emergent systems ....

Update 11/24/07: The splog hits are gone now, and "canopy economics" returns two posts of mine and one post about forestry economics.

Lord of the Flies at Abu Ghraib

Iraq Prison Supervisors Face Army Reprimand (washingtonpost.com)

Overall, the report portrays the prison as being run by a poorly led, undermanned and demoralized group of U.S. soldiers. Because of Army personnel policies, it notes, the 800th MP Brigade did not receive replacements as members left for medical reasons or because their terms of service were finished. Also, the report found, the troops' quality of life was 'extremely poor.' They lacked many of the facilities provided to soldiers at other U.S. bases in Iraq, such as mess halls, barbershops and post exchanges, which offer magazines, toiletries and other personal items for sale.

---

Neglected and forgotten young soldiers in a bloody awful citadel of suffering. The only rewards and attention they get comes from satanic private contractors tasked with interrogations too nasty for the CIA to touch.

An idiot could write a play about this. Let the battle for the movie rights begin. Maybe the victims, including those both victim and perpetrator, will be able to sue for a share of the proceeds.

A War for Us, Fought by Them: Bring back the draft (NYT OpEd)

The New York Times > Opinion > Op-Ed Contributor: A War for Us, Fought by Them
By WILLIAM BROYLES Jr.

---

I can't really excerpt meaningfully from this NYT OpEd essay. It's a succinct and well written piece by a playwright who's a former (involuntary) marine and a father of a marine serving in Iraq. Unlike most wars, including Vietnam, no child of any administration official is serving in Iraq. Not only is this unfair, it allows the corporate and ruling elite to neglect our soldiers. The Abu Gharaib disaster is, on first inspection, partly a consequence of that neglect. If you drop young people into a terrible environment, and don't provide relief, structure, and support, you get 'The Lord of the Flies'.

The author says we need a draft, presumably of both genders, for the 'war against terror'. Since I have three young children, and since this is likely to be a Forever War, I can't claim much personal enthusiasm for the idea. Nor, however, can I deny the logic. On the other hand, as the author notes, GWB and Cheney managed to avoid service (in Cheney's case he was honest about his avoidance) during a draft. I find it hard to believe, in today's world, that a draft would be any more fair than it was in Vietnam. The result might even be less representative than our volunteer army.

Philosophically though, I don't disagree. I suspect if we had such a draft, Bush would not be up for reelection.

The fallen americans: not as much outrage as some would guess?

Terrorism & Security | csmonitor.com

The Christian Science Monitor reports that the effect of the photos was somewhat blunted because Arabs 'say they expect no less from the United States given the widely held view that it is running a brutal and oppressive occupation in Iraq.'

'Will the pictures make a difference in the Arab world? Probably not,' says Michael Young, a Lebanese political analyst. 'It simply confirms what people already think about the Americans. But it will be embarrassing for the Americans in Iraq, and that's where it's going to count.'

---

Maybe this was Bush's strategy ... Our reputation is so low it cannot fall further.

I wonder if this might episode might, paradoxically, ease tensions in in Iraq. It's immensely annoying to have a "superior" civilization constantly blathering about their innate goodness, discipline, and all around wonderfulness. I suspect Bremer can't get away with that act any longer; nor can any other American representative in Iraq. Iraqis have had a dark and shameful recent past, but now they know that that their occupiers are flawed as well. Time for a real discussion?

Guardian Unlimited | Fundagelism ....

Guardian Unlimited | US elections 2004 | God save America ...

The word 'fundagelism' has never appeared in the columns of this newspaper. The term is, however, current in the blogosphere - that cyberforum which nowadays carries the most interestingly paranoid political debate. 'Fundagelism' is not a word that trips easily off the tongue. It's a crunching together of the even more mouth-boggling compound 'fundamentalist evangelism'.

---

This Guardian article is itself a bit extreme, but extremism seems to be indicated nowadays. They claim 90 million evangelicals will vote for Bush -- that seems higher than the numbers I've read -- but I have little faith in the numbers.

Sharp Blue: Consilience & The Naturalistic Fallacy

Sharp Blue: Consilience

... the naturalistic fallacy, the idea that what is natural is also moral.

---

Great review -- I'd heard of consilience but this review has put it on my read list. I also loved the name for a terribly common fallacy, a variant of which is the idea that "that which is natural is also healthy" -- the herbal medicine fallacy.

MSNBC - Rough Justice in Iraq - Janis Karpinski is NOT going quietly

MSNBC - Rough Justice in Iraq

More on the POW scandals. It seems that even a one star general can be dangerous when angry. Karpinski is not going to go quietly, and she'll talk to any media outlet anywhere. She's giving the scandal legs even as Bush & Co try to put it to sleep.

The Barbara Walters adoption game show

Adoption Isn't A Game Show (washingtonpost.com)

Tonight ABC's weekly news program '20/20' is to air Barbara Walters's profile of Jessica, a pregnant 16-year-old who will select among five couples vying to adopt her child. Jessica will participate in an open adoption, an increasingly popular practice that allows adoptive parents and, in many cases, the child to maintain contact with birth parent.

---

I wonder what state she's operating out of? In many states child protective services would be looking in very closely.

Barbara Walters has fallen a long way. ABC must share a deep desperation with her.

The bright side of living in a nation with zillions of laywers is that you know they're all figuring out how best to sue ABC.

American Express: Reliable service is hard to provide

American Express Personal Finance, Business Solutions, and Travel Services

It's very hard to provide reliable 24x7 service. American Express ought to do well, but they don't. For example:

1. AMEX service for downloading credit card transactions to Quicken is out of order. This is an important function for me, so I use their online email service to complain.

2. I get an email saying they've got a reply online.

3. A week later I return to the AMEX online email service to read their reply -- but it's "temporarily out of order".

Why some companies can provide 24x7 service, but most cannot, would be a great topic for a Harvard Business Review study.

The prisoner affair: The New Yorker (Hersh) provides extensive coverage

The New Yorker: Fact

---

One of the few publications to provide coverage. The question -- where does the buck stop? At Rumsfeld? At Bush? And what about Guantanamo? There is at least one hero though:

The abuses became public because of the outrage of Specialist Joseph M. Darby, an M.P whose role emerged during the Article 3 hearing against Chip Frederick. A government witness, Special Agent Scott Bobeck, who is a member of the Army’s Criminal Investigation Division, or C.I.D., told the court, according to an abridged transcript made available to me, “The investigation started after SPC Darby . . got a CD from CPL Graner. . . . He came across pictures of naked detainees.” Bobec said that Darby had “initially put a anonymous letter under our door, then he later came forward and gave a sworn statement. He felt very bad about it and thought it was very wrong.

Of course, knowing this administration, SPC Darby will pay a high price for his courage. The other hero, who Bush will be searching for, is whoever leaked the secret report. The accused, on the other hand, have promising careers ahead on talk shows and in the off-Hollywood movie business.

It appears that many of the abuses began with the the now standard procedures for CIA interrogations of "high value" captives. These have been discussed in articles in the Atlantic and NY Times Magazine; in the dark world of semi-civilized torture these are considered mainstream techniques. In the hands of amateurs, including a few who may have had some serious flaws to begin with, these methods were applied more widely and with increasing creativity at Abu Gharaib. Such is the peril of the "slippery slope".

Rumsfeld has known about, and approved, the interrogation methods used in the "war against terror". He should resign -- but he won't.

The defense attorneys will use techniques that have been effective in past war crimes. They will threated to broaden the inquiry up the chain of command. This technique normally causes the military to back down, but in this case they're not alone. The cashiered General is not going quietly into the night. From the New York Times:

But the officer, Brig. Gen. Janis Karpinski of the 800th Military Police Brigade, said the special high-security cellblock at the Abu Ghraib prison, west of Baghdad, where the abuses took place had been under the tight control of a separate group of military intelligence officers who had so far avoided any public blame.

In her first public comments about the brutality — which drew wide attention and condemnation after photographs documenting it were broadcast Wednesday night by CBS News — General Karpinski said that while the reservists involved were "bad people" and deserved punishment, she suspected they were acting with the encouragement, if not at the direction, of military intelligence units that ran the special cellblock used for interrogation.

Speaking in a telephone interview from her home in South Carolina, the general said military commanders in Iraq were trying to shift the blame exclusively to her and the reservists.

"We're disposable," she said of the military's attitude toward reservists. "Why would they want the active-duty people to take the blame? They want to put this on the M.P.'s and hope that this thing goes away. Well, it's not going to go away."

She said the special cellblock, known as 1A, was one of about two dozen in the large prison and was essentially off limits to soldiers who were not part of the interrogations.

She said repeatedly in the interview that she was not defending the actions of the reservists who took part in the brutality, who were part of her command. She said that when she was first presented with the photographs of the abuse in January, they "sickened me."

The burning shame of the US media

BBC NEWS | Americas | US media grapples with Iraq horrors

But Howard Kurtz, media critic for the Washington Post, told the BBC that the White House has not made an effort to quell the reports. [of criminal torture and humiliation of Iraqi POWs]

Mr Kurtz said that one possible explanation as to why the story was not receiving greater coverage in the US was that Democrats, the party in opposition, were not making the allegations an issue.

'We, in the media, are a little hesitant to start the debate ourselves, but if some member of the Democratic Party were to start a debate over this, we would be on it in a flash,' Mr Kurtz said."

---

I have little outrage left, but this almost shocked me. With great brevity, Mr Kurtz has demonstrated how low our media has fallen. We're talking lower than a snake's belly.

If I were the editor of the Washington Post, I'd have the entire staff over for a first order tirade.

This does explain why our media never covered the smallpox hoax. (Bush's faux vaccination program, which was dropped as soon as we invaded Iraq.)

Understanding war crimes -- American GIs in Iraq

The Memory Hole > Photos of Iraqis Being Abused by US Personnel

---

War is brutalizing in the best of times. For the US military in Iraq this is not the best of times. Under conditions of extreme stress and inadequate support they are breaking down, just as warriors did in Vietnam and Algeria and the Pacific Islands and the Russian Front ...

Bush has lost his war. America has lost its reputation, its honor, its credibility, and many many lives. Bush has sacrificed so much. Even if his intent were just (and history will judge that), the execution of his plans was criminally incompetent. He should stand for court martial along with these GIs.

Defeating the pandemic of online news registration requirements

The Washington Monthly

More newspapers are requiring registration; it's becoming a real annoyance. It turns out there's a longstanding solution -- a "public" identity.

un: cyberpunk
pw: cyberpunk

The thread I link to (above) claims this is an old net tradition; that for years the net equivalent of Kilroy has been cyberpunk.

I just tested it for the LA Times. It works. Alas, it provokes some interesting responses from Hotmail and Yahoo suggesting they invalidated this one a while ago.

Eventually this un/pw will become invalid. But others will take its place.

I don't recommend using this approach -- it is certainly dishonest and arguably it's a form of theft. On the other hand ...

Capitalism's challenge: attaching value to public goods

Brad DeLong's Semi-Daily Journal (2004): a Weblog

Social value is drifting away from potential profitability, and this threatens to become a huge problem in our collective social resource allocation mechanisms. Google needs to grow to approximately ten times its current profitability and then maintain its market share and margins indefinitely in order to justify the $20 billion valuation. And that's hard to see: high sustained profits are the result of effectively-maintained barriers to competitors--think Microsoft, think Intel. What is going to be Google's counterpart permanent edge?

---

Writers are compelled to communicate. Coders are compelled to code. If writers and coders aren't paid for what their compulsions, they exercise them anyway. Writers create blog entries for their mothers and spouses, coders (being more useful) produce open source applications.

The internet allows the more marketable writers to reach an audience, and it allows distributed collaboration on software projects. Complex systems interact over time and produce new adaptations, such as very complex distributed software collaborations.

None of this activity is following market demands. It is compulsion-driven (historicallly always true for art and writing). The modern difference is that software creation can have large economic consequences (eg. Linux). To some extent the writing compulsion may also propagate memes at higher velocities, raising the "world IQ".

It would be historically consistent that even as capitalism stands unrivalled, odd cracks in the facade are emerging. These cracks are of great interest to economists, much as physicists are thrilled by miniscule errors in the predictions of the "standard model".

See also my post on market failures and a recent personal experience with a toy bank.