No, these days, the memetic fiction to read is produced by a handful of writers who often have credentials in the sciences. Whatever the outward appearance, there's a core set of ideas that are better developed and more persuasive than any effort at traditional futurism. These writers often try hard to imagine what humans will be like in very different environments or with very different abilities.
Read this gang, and nothing will surprise you. I've put a sample of books together in one place for your mind-expanding pleasure: Amazon.com: "Mind expanding fiction". Warning -- don't read these all at once. I suggest starting with vintage Vinge (superficially less radical) or Iain M. Banks (superficially conventional) before tackling Egan or Stross. Excessive consumption by an unprepared reader may result in explosive cranial decompression.
Oh, and yes, they all take a stab at resolving the Fermi Paradox.
Update 4/26/10: Current names on the list, in alphabetic order with one exception...
Greg Egan: Egan goes first because he's 90% mind blowing. He's the straight whisky and black espresso of mind expansion. Don't start with Greg, you need to work up to him. It can be hard to find his books , they are a bit too demanding to stay long in print (mind expansion is not a commercially optimal strategy). Incandescence, for example, walks the reader through the derivation of neo-classical orbital mechanics in an environment where general relativity is personally relevant. It also obliges one to think of the relationship of individuality to the hive and the historical peculiarity of renaissance.
Charles Stross: One of my favorite writers, like any mind expander who's going to be commercially successful he balances novel ideas with character and plot. Of this list he's probably the most story-driven and least idea-driven -- which of course can make him very readable. Halting State is one of his most idea-centric works.
Greg Bear: Greg has largely moved on to the more profitable thriller/horror genre, but his early works (Eternity, Eon) earn him a lasting spot on the list.
Iain M. Banks: Iain is the gateway drug of mind expansion. Some his culture books seem like mind candy, but they're laced with the hard stuff. Soon you work your way to Consider Phlebas, Feersum Endjinn and Matter.
Robert Sawyer: Despite an atypical sympathy for deity, he definitely pushes the envelope.
Stephen Baxter: An excellent introduction. Evolution is astounding. 
Vernor Vinge: Like Banks and Stross he's relatively approachable. Another great place to start. Here's hoping eBooks will one day be a friend to writer and reader alike.
 Update 8/23/2010: One of my favorite scenes in Evolution concerns sentient spear using dinosaurs. Baxter makes a great case for a sentient, warm blooded, raptor-with-digits dino. In the story they hunt their primary prey to extinction - with obvious consequences. I thought that was quite original, but today I read Asimov's "Day of the Hunters" (1950) published in "Buy Jupiter and Other Stories" (1975). It's a less sophisticated version of the same story! Asimov was gem.