As described last time, Swann doesn’t think Odette is very cultured or intelligent despite being utterly infatuated with her. He’s fairly on the money too. (Proust certainly agrees.) The real damage Odette does to Swann, the damage that robs him of part of his soul, is not that he misunderstands her so much (although he does), but that he allows his infatuation to remove from him his aesthetic faith. This is for me the very core of “Swann in Love” (and certainly something Proust dwells on elsewhere), where Swann, ripped apart by Odette’s thoughtlessness and faithlessness, finds an affirmation (conveniently) of exactly the sort of aesthetic experience Proust has been trying to justify.
But first, Odette. She leads him to abandon his tastes in people and art and embrace the Verdurins soirees and their facile tastes. It’s not something she intends; she respects his intelligence as far as she can understand it, and Swann explicitly gives up on communicating to her his aesthetics. So he minimizes them. Odette later ridicules Swann for not appreciating the dull, witless things she likes, but she is more tasteless than she is dismissive. Switch the genders, and you have something like this:

BART: Dad, if there’s a really special girl and she likes some clod who’s beneath her, what should you do?
HOMER: I married her!

Pages 263-275 describe the agonizing way in which Swann abandons any conception of artistic merit to be in harmony with Odette’s tastes. Of his own tastes, he becomes

convinced, moreover, that a cultivated “society” woman would have understood them no better, but would not have managed to remain so prettily silent. But, now that he was in love with Odette, all this was changed; to share her sympathies, to strive to be one with her in spirit, was a task so attractive that he tried to find enjoyment in the thinks that she liked, and did find a pleasure, not only in imitating her habits but in adopting her opinions, which was all the deeper because, as those habits and opinions had no roots in his own intelligence they reminded him only of his love, for the sake of which he had preferred them to his own.
Besides, having allowed the intellectual beliefs of his youth to languish, and his man-of-the-world scepticism having permeated them without his being aware of it, he felt that the objects we admire have no absolute value in themselves, that the whole thing is a matter of period and class, is no more than a series of fashions, the most vulgar of which are worth just as much as those which are regarded as the most refined. (269)

Most of ROTP so far is a refutation of exactly this belief, which is tantamount to nihilism in Proust’s world. By turning his back on the worth of his admiration, he removes his very view of the world (and its worth), and there’s nothing to replace it but Odette. The choice, as Proust paints it, is not between elitism and egalitarianism, but between partiality and apathy.
Swann’s salvation (and his inability to adapt to life with Odette) comes in his failure to renounce fully his tastes and embrace the relativism above. A few pages later, Swann is still unable to buy into the Verdurins’ crass little culture. As he struggles to adapt to them:

The fact was that they had very quickly sensed in him [Swann] a locked door, a reserved, impenetrable chamber in which he still professed silently to himself that the Princesse de Sagan was not grotesque and that Cottard’s jokes were not amusing, in a word, for all that he never deviated from his affability or revolted against their dogmas, an impermeability ot those dogmas, a resistance to complete conversion, the like of which they had never come across in anyone before. (273)

The rest of the section is a portrayal of Swann in this limbo, and while Odette continues to drive him crazy, I see it as a distraction from the portrait of a man who’s lost his aesthetic moorings, not his romantic ones.
He eventually regains them. At a better class of party than the Verdurins’ (I still thought it was boring), Swann speaks to the Princess des Laumes and feels more identification than he has with Odette or any of the Verdurin crowd, since “they had the ‘tone’ of the Guermantes set” (372). This is followed by a series of aesthetic revelations where he re-embraces what he gave up earlier. On hearing that same musical phrase again, he locates his happiness and love inside of it:

In that way Vinteuil’s phrase, like some theme, say, in Tristan, which represents to us also a certain emotional accretion, had espoused our mortal state, had endued a vesture of humanity that was particularly affecting. Its destiny was linked to the future, to the reality of the human soul, of which it was one of the most special and distinctive ornaments. Perhaps it is not-being that is the true state, and all our dream of life is inexistent; but, if so, we feel that these phrases of music, these conceptions which exist in relation to our dream, must be nothing either. We shall perish, but we have as hostages these divine captives who will follow and share our fate. And death in their company is somehow less bitter, less inglorious, perhaps even less probable.

This reminds me of Mann circa The Magic Mountain, but it’s better. Mann is a beautiful writer, but very weak philosophically, and weakest when he tries to spin up unjustifiable ideas with purple prose. Proust avoids the generalization by emphasizing the partiality and intimacy of a particular taste; i.e., by placing the importance on the multiplicity as well as the reality of these aesthetic impressions. It’s still a little heavy for a writer like Proust, and you wish that, say, Erik Satie would show up and vomit all over the place, but it’s still moving.
(It’s also a recipe for tremendous selfishness and self-absorption when someone enslaves themselves and their buddies to a useless personal vision, but more on that later.)
Shortly afterwards, he gives up on Odette, even before he finds out the worst about her. And that’s about it for them.
OR IS IT???