We think of tempo as affecting playing, not listening. It is equally important in both musical activities. Most of us think of listening as passive. It is anything but.
32,000 vibrating sensor-tipped hair cells in the inner ear represent an enormous amount of activity.
It would be interesting to play with your internal metronome in relation to your listening. Say you are about to hear a Minuet: Start by imagining how a minuet might go. Get in gear, so to speak. You will have a definite reaction immediately as the music starts--you will be engaged. Or imagine that you are hearing a chamber music performance in a small space and that you decide to pay attention to every entrance of the clarinet. You will probably notice events happening much more rapidly than can be accounted for by any normal sense of the beat -- so much goes into the production of a tone!
These decisions you can make about your active listening can be the basis for real listener empowerment. Every performer needs and appreciates active listeners!