Figure: S&P over our investing lifespan - 1985-2008 - click to enlarge (Yahoo!)
So what happened?
One theory is that the combination of the 1994 Gingrich Marketarian  "revolution" and consequent firewall demolition, combined with at least one major technology transition, produced accelerated returns at the cost of new instabilities. Over a long enough timeline investment returns might be somewhat lower than with a balanced regulatory environment, but "safe" investment timelines are now 20-50 rather than 10-15 years.
I think that's true, but not the entire story.
First, a brief digression. Twenty years ago a friend of mine did quite well by an Amway-like multi-level marketing business. Unlike the pyramid (Ponzi) schemes that devastated Albania in 2000, or the riot-inducing Columbian scheme of 2008, these businesses do sell a physical product. Like classic Ponzi schemes, however, there's a lot of cash flow from new recruits to established executives.
Some would call these new recruits "marks" .
People working in these businesses are taught to draw comparisons to the stock market. That's what my friend did twenty years ago, and it's stayed with me ever since. The difference, in theory, is that at best a Ponzi scheme is a zero sum game. All wins come from someone else's losses. In theory everyone can play the market and win -- because it's ultimately powered by global productivity and economic development.
In practice, however, natural selection happens. It always does.
Think of the market as a vast, indigestible feast. Sooner or later, bacteria will figure out how to eat it. It's as predictable as the sunrise.
So how does natural selection play out in this scenario -- remembering that for a biologist fraud is just another name for a survival strategy.
We know humans are predictably irrational. We know people will aggressively search for cheap gas when prices are rising, but won't when prices fall -- even at the same income/price ratio.  Similarly we know humans will criticize balance sheets when share prices fall, but not when they rise.
This means that market volatility enables predictable predation strategies during rapid rise. Money can be diverted into senior executive compensation, into insider trading, into payments to political parties and senators, and into sophisticated financial instruments that none of us have the ability to fully understand or model.
This form of market predation (parasitism really, since a dead host is not useful) is bad enough by itself, but it's aggravated by "ratchet effects" . CEO compensation doesn't fall as quickly as share prices. Senatorial contributions can't be stopped without risking undesirable electoral outcomes.
Volatile markets, like those of the past twelve years, can start to look an awful lot like Amway.
We've been Marked.
So what do we do?
About a year ago I drew a crude line from the sane growth curves of the early 90s and I reasoned that share prices weren't too crazy any more. I resumed the share purchases I'd de-emphasized since 2002. Since then the market has fallen a lot more, but we're still doing our index fund dollar-cost-averaging.
It's not that I don't think there's a major parasite effect in the Markets. I think that is a part of what's going on. On the other hand, it's not like we have great alternatives.
I am, however, looking for alternatives. I'd like to find a way to start investing in select privately held companies, companies that are relatively resistant to market-oriented parasitism strategies. Companies that can be driven by the desirable, but arguably irrational, strategies of founders who seek to combine their own wealth with delivering useful goods and services.
Anyone know how we can do that?
- July 2008: 1999 to 2008. Now what we expected: In which I conclude that the "safe" period for investment holds looks like 50 years, not 10 years.
- Jan 2008: The Dow is Down
- May 2007: Eight lousy years
 When I visited the Wikipedia link for Amway I came across this fascinating tidbit. Recall that Sarah Palin, darling of the dark core of the GOP, also has dominionist links. Emphases mine.
... its founders contributed $4,000,000 to a conservative 527 group in the 2004 election cycle...Bush appointed Amway's attorney to head the FTC. There are only 12K Google hits on this, so it's not surprising I missed it. It's things like this that make it so hard for me to understand how, 8 years later, Obama won.
... Former Amway CEO Richard DeVos has been connected with the dominionist political movement in the U.S...
Multiple high-ranking Amway leaders, including Richard DeVos, Dexter Yager, and others are also owners and members of the board of Gospel Films, a producer of movies and books geared towards conservative Christians...
... In 2000, current President George Bush appointed Timothy Muris, a former anti-trust lawyer whose largest client was Amway to head the FTC, which has direct federal regulatory oversight over multi-level marketing plans. ...
Amway co-founder, the late Jay Van Andel (in 1980), and later his son Steve Van Andel (in 2001) were elected by the board of directors of the United States Chamber of Commerce as chairman of that organization....
 The intended victim of a swindler, hustler, or the like.
 Marketarian: Someone who subscribes to Marketarianism, the neo-Calvinist / pseudo-libertarian (objectivist) religious belief that the Market is not simply an efficient satisficing mechanism for finding local minima but is a god-like entity that defines moral qualities. See also, Yahwism.
 I've been trying to remember the engineering and economics term that describes "stickiness" or "ratchet" effects, where things move more easily in one direction but move more slowly in another. If anyone can name this concept I'll be very grateful. Ratchet effect is the best I can do but I think there's a better name in engineering.
 This is why gas stations make money when prices are falling, but lose money when prices are rising rapidly. It's the opposite of what most people think. Convenience stores let them hedge their financial risks.