Literary fantasy is simply fantasy that’s better-written, has more realistic characters, and is more ambitious in exploring deep ideas than other books. Instead of just wizards zapping each other with blue lightning and smart-mouthed goblins, literary fantasy can contain wrenching emotions that look at what it actually means to be human.
Its polar opposite would be something pulpy like Hunky Vampire Witch Lust Prom Night or Battle Vixens in Improbable Armor.
Fortunately, there’s room for many styles in fantasy.