Here's what I mean. Some people problems with perfect pitch. They hear what they see. It is a problem for them when playing instruments tuned to a much different pitch, but even more so when trying to play from memory on an instrument which is out of tune.
Having a perfect pitch and hearing instruments of another pitch level is usually a problem. Playing them is easier after a while (at least for me). It's strange when you hear C major prelude as Db major or B major.
But you can get used to a certain pitch level, if you just spend a couple of weeks with it (that's what happens when somebody has a piano tuned half step lower - then you will hear the note A always too low (G#), in comparison to the standard pitch). And other temperaments will start to bother you, by the way.
So people with perfect pitch normally can spell out any note which is being played high or low. That's how they are used to listen to music - by hearing individual pitches. When the instrument is tuned a half step higher or lower than A=440, they have a problem with following the music.
Here's the thing: instead of listening to the notes which may or may not be the same for the player and for the listener, try to hear the chords and key areas.
For example, try to hear the tonic, subdominant, dominant and other functions without spelling out the chords. Try to discover what key area this episode is written in in relationship to the tonic (not the precise key, but only the function, subdominant or dominant or mediant or submediant etc.).
This is far more useful than listening to the notes only. It's more challenging, though.
But we seek out challenges, don't we?