By Catherine BrayFeatures correspondent

Released 40 years ago, disastrous disco musical Xanadu has a legacy that still endures. But why, asks Catherine Bray.

The film opens on a simple shot of a sign that says "No trespassing". We track slowly up a tall chain-link fence to reveal a mansion on a hill, wreathed in mist. A blizzard whirls in a snow globe, an old man whispers something and dies. His obituary opens by quoting Samuel Taylor Coleridge: "In Xanadu did Kubla Khan a stately pleasure-dome decree." The film? Citizen Kane. One of the very greatest films ever made. It's about the folly of man.

More like this:

- The forgotten great coming-of-age film

- The most outrageous film ever made?

- The classic that's saved lives

Orson Welles' 1941 masterpiece shows us how the eponymous Charles Foster Kane came to build his Xanadu, a stately pleasure-dome that acts as a monument to his dreams. Kane's Xanadu is a costly nightmare with plenty of ambition and high-concept thinking, but an empty void where the soul and character should be – just like the infamous 1980 roller-disco musical fantasia that shares its name.

Directed by Robert Greenwald from a script by Richard Christian Danus and Marc Reid Rubel, Xanadu stars Olivia Newton-John, Gene Kelly and a Jim Morrison mannequin that's broken out of Tussauds to pursue the acting dream. The film opens with a young artist tearing up a sketch in frustration. His scraps of paper are borne on the California breeze to an epic dayglo mural depicting nine fetching young ladies. They are the Muses.

The Muses then come to life and emerge out of the painting, in a blaze of less-than-special effects, to the immortal accompaniment of the Electric Light Orchestra. One of the Muses is Olivia Newton-John. The Muses are dressed in off-the-shoulder tiered pastel midi-dresses – think shepherdesses in Ancient Greece, as imagined by one of those Halloween shops called something like 'Party Delights' that sells knock-off Austin Powers outfits labelled '60s secret agent'. The Muses perform a kind of contemporary ballet through a miasma of fluorescent lasers. It is as if the Anthropologie summer collection got stuck in the mainframe in Tron.

The aesthetic clash embodied in this opening number cuts to the core of what feels unique about Xanadu. It is a film that is partly borrowing from Greek myth, partly a tribute to a 1950s Rita Hayworth movie with a similar plot, and it is also about the flash-in-the-pan roller-disco craze of the late 1970s. These competing elements, each hoping to hook a different demographic sector of the audience, go to war throughout the film. Nobody wins.

Oh, and there's also an animated sequence. Did you ever end up with a Disney film on VHS from a man down the market, only to find it was a mockbuster from Golden Films? Founded in 1988 by Diane Eskenazi, a female film-making pioneer of sorts, Golden Films created animated adaptations of public domain fairy tales like Beauty & The Beast, Aladdin and The Little Mermaid, coincidentally releasing them in the same years that Walt Disney had a bash too. Xanadu's animation feels like a Golden Films experience, an ersatz calamity. If you wondered what the leads from Xanadu would look like as horny goldfish, this is the part of the film for you. In the cursed treasure map that is Xanadu, this bit should have large letters warning Here Be Monsters.

High-budget, low energy

If that all sounds like a lot, well... it is and it isn't. Some films take your breath away with how ingeniously they managed to create so much spectacle on so little money. Xanadu is very much the opposite – it's a curiously flat film, and it's really quite difficult to see where all that money went. Sometimes films surprise you with how quickly they're over, despite how much they manage to pack in – how can the time have flown so quickly? Again, not a problem for Xanadu, which is best described as operating on gym-time: surely you must have been pounding this treadmill for at least half an hour by now? You check. Ten minutes?!

It is difficult, in writing about Xanadu, not to sneer, and in doing so, write yourself into critical superfluity. Famously, the task of painting the gigantic Firth of Forth bridge will never be completed, because by the time workers get to the end, the beginning is in need of a repaint. Similarly, critics will never be done zinging Xanadu. It is an infinite task. Or is it? In reality, the painting of the Firth of Forth bridge was finally successfully completed in 2011. Perhaps some day it will be time to stop beating the disintegrating corpse of the rabid show pony that is Xanadu.

Perhaps now, as the film turns 40, we ought to reappraise it. It is tempting, when writing about cultural bêtes noires, to discover a film to have been more sinned against than sinning. Maybe Xanadu is a misunderstood masterpiece?

There is certainly a resonance to the legendary Kelly playing a character living in a mansion that he tells us once belonged to a great Hollywood star. This sets the stage for an odd but genuinely moving fantasy sequence starring Kelly and a version of Newton-John's Muse from the past (let's not get tangled up in the notional storyline justifying this). They dance together, with ghostly overlaid footage creating a kind of wistfully haunted Overlook hotel effect, an uncanny echo of the scene in Anchors Aweigh (1944) where Kelly dances with Jerry Mouse.

This haunted duet is filmed with as little flair as any of the rest of the film, but during this one sequence, the fact of who Kelly was, and how much he means to the history of cinema, together with the knowledge that this was his final film role, and also our awareness of how Xanadu turned out, succeeds in generating a kind of heartrending pathos that temporarily not only overwhelms the badness of Xanadu, but is actively accentuated by it.

One of the great things about essay films (films constructed from archive clips of existing material) is the possibilities they present for juxtapositions the creators of the original works would never have pictured. In that respect, Xanadu sometimes functions almost like an essay film in its layering of all of the cultural meaning we attach to Kelly on to such disparate elements as the '70s roller disco and Newton-John. Unfortunately, on the whole, when you step back from it all, the result is total bobbins.

It's easy to wonder, when we watch most of the filmed versions of Mary Shelley's Frankenstein, why a vain scientist keen to show off his genius would go out of his way to make a Creature so obviously monstrous. The Rocky Horror Picture Show riffs on this – Frank N Furter creates a more idealised Creature, with blond hair and a tan, and it's much closer, in some ways, to Mary Shelley's original idea, than the bolt-through-the-neck version. In her book, the doctor's intention was indeed to create something beautiful:

"How can I describe my emotions at this catastrophe, or how delineate the wretch whom with such infinite pains and care I had endeavoured to form? His limbs were in proportion, and I had selected his features as beautiful. Beautiful! – Great God! His yellow skin scarcely covered the work of muscles and arteries beneath; his hair was of a lustrous black, and flowing; his teeth of a pearly whiteness; but these luxuriances only formed a more horrid contrast with his watery eyes, that seemed almost of the same colour as the dun white sockets in which they were set, his shrivelled complexion and straight black lips."

It's the fact that the creator was attempting to make something gorgeous that makes the monster so awful, much more so than if the intention had been to create something scary. So too with Xanadu, where the glimmers of originality and delight are the equivalent of lovely pearly white teeth in the face of a reanimated corpse.

These kinds of films don't come along very often. At the press screening of Cats (2019), the Observer critic Guy Lodge leaned over to me and whispered, full of awe, "Oh my God... this is our Xanadu!" And he's right, they're cut from the same rare cloth. Cats cast Ian McKellen and Judi Dench to appeal to a certain audience, James Corden and Rebel Wilson another audience, Jason Derulo and Taylor Swift a third demographic, and so on, through all the many, many creative calculations in the interpretation. The idea is to go wide and appeal to every kind of person, rather than to anyone specific, to be as big on TikTok as on cosy sofa-orientated chat shows. The result in the case of Cats is a beast that appeals to almost no-one, prodded with pitchforks by the small numbers who can actually claim to have witnessed its crimes, while everyone else stays at home, listening wide-eyed to the terrible rumours (Cats was a box-office flop).

Like many a master criminal, the main thing Xanadu and Cats were really guilty of was getting caught. Tentpole filmmaking is generally more adept at hiding the scaffold. There are plenty of films just as cynically constructed, which take flight along similar routes but land successfully; Xanadu and Cats are what happen when you miss the end of the runway.

It is for this reason that there's something special and enduring about Xanadu's splendid cackhandedness. It is perhaps most of all worth talking about not just as a flop or a curio, but as a premium example of capitalism-as-demented-auteur – where commercial imperatives wind up accidentally birthing a monster entirely stranger and more inexplicable than the works of an independent filmmaker who is trying to create something singular. Most of Xanadu's conceptual choices were made for commercial reasons, but instead of leading to something cookie-cutter and audience-friendly, they produced this flailing monster, provoking delight, pity, compassion and horror, all at once.

Catherine Bray's documentary Guilt-Free Pleasures, looking at Xanadu and other similar films, is on BBC iPlayer.

Love film and TV? Join BBC Culture Film and TV Club on Facebook, a community for cinephiles all over the world.

If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us on Twitter.

And if you liked this story, sign up for the weekly bbc.com features newsletter, called The Essential List. A handpicked selection of stories from BBC Future, Culture, Worklife and Travel, delivered to your inbox every Friday.

ncG1vNJzZmivp6x7o67CZ5qopV%2BYwq3A1KucaJmiqbakuMRoaWlqYGZ%2FcoCMsZinmZSqerOx06Ggp6OZo7RurYymoKytnpmys7%2FTqKadZZ2WwLWx0amgnpuV