Hal Duncan over at BookSpotCentral has begun a series of columns answering this question. As anyone whose read widely in his blog would expect, Mr. Duncan is aiming to portray the disdain for the fantastic as upper class contempt for the puling masses. Or, even worse, the mingled fear and contempt of the putrid, stodgy middle classes faced with the lure of the proletarian nihilist carnival. As ever, he begins with some doubtful equations, not just Romanticism with the Gothic, or Enlightenment rationalism with the realist, but sensationalist with popular and "intellectualist" with high brow. What indeed is "intellectualist" literature and drama when it's at home, you may ask? Well, as I say, it's all a little dubious. The sole examples that Duncan gives, The Foundation series as rationalist and Dune as romantic, on brief inspection don't show what he thinks. I expect there will be few details to give meaning to all these terms.
Duncan's whole project is dubious, starting with the astonishingly simple proposition that science fiction as a pulp genre is dead. Since the pulps are moribund, so are the pulp genres. Why this simple conclusion requires the recasting of language and literary history is uncertain. I think in the end that Mr. Duncan is not just a professional fantasist, but a committed irrationalist. The suspect chains of equations and false dichotomies have to be erected to turn fantasy into the liberation of the ego from the chains of reason, that ruling class scam.
In reality, science fiction, like historical fiction, is not "literature" because it does not accept the way things are now as eternal. Instead of the individual soul being the determinant of a permanent way of life, unquestionable, even if tragic, both science fiction and historical fiction go past the bourgeois individualism (or can, since both have just as much junk written as other modes of fiction and drama,) which is the dressed up version of economic man found in Economics 101. An historical novelist like E.L. Doctorow will be acceptable as literary in a way that a Gore Vidal or a Cecelia Holland will not. The easy assumption that "literary" means good is nothing but snobbery, but putting a John Jakes on the same level as the previous three is to abandon all judgment. The point of course is that Vidal should be regarded as a major novelist, which he isn't: He just sells too well to be completely ignored. Doctorow, whose work suffers from being "literary" in preferring to recapitulate ideology about character, receives critical attention far beyond his ordinary merits.
The notion that "genre" as an opposition to "literature" is marked by sensationalism while the other is marked by intellectualism is a pretty hard vacuum. The assumption that insight and relevance lie solely in character studies is an ideological prescription. Most "popular" fiction and drama is just as fixated on the characters as the most highfalutin' "literary" fiction and drama. And there are plenty of thrills in lots of "literary" works. The difference isn't sensationalism. The difference is that the "literary" rediscovers the eternal essence of bourgeois society, the final society, forever and ever, amen. The "popular" regrettably tends to be vicarious fantasy. The reader identifies with a hero, and the hero wins. The rest is detail. Which heroes earn reader or viewer commitment is a matter of personal taste, not amenable to discussion. Which fantasy victories are satisfying also is a matter of personal taste, no more of interest to others than the details of the satisfying sexual fantasy.
Insofar as this is true, the popular should be rejected. But of course in practice this is not the dividing line between the popular and the literary. The supposedly literary might have reader/viewer fantasy, while the popular might answer more questions than "Who's the hero, and does he/she win?" I repeat, the real dividing line is ideological. The literary is about "character," where character means the eternal psychological necessities that have created our contemporary capitalist society. This, not being intellectual or boring, is what makes literature's pretensions offensive.
The notion that the sensational is rejected solely because it is popular is really nothing more than faux populist posturing, a visceral hate prompted by the certainty that one's own superior moral sensitivity raises one above the snobbery of the literary elites. In fact, many define sensationalism as the false, as when talking about tabloid journalism. Such sensationalism should be rejected. Approaching the question from another angle, although the literary presumes the eternal verities of capitalist society, that no more means it is all false, root and branch, than any other aspect of bourgeois culture. Popular fantasies that ignore society do no one any good. They are pabulum. Bourgois culture is no more to be ignore than the bourgois state. It is in fact part of the wealth to be appropriated.
Literature and drama that attempts to do this will not be literary or popular, it will be human.