One of the hazards of public listening is the possibility that someone in a nearby seat will be beating time.
Most of us learn at some stage that rhythm is based on body movement, so it is natural to express it with bodily movement. I say it all depends.
Some rhythm is decidedly not based on body movement: the rhythm of song, for example, is subject to flights of whimsy that would cause a moving body grievous injury. And there is the rhythm of the mind at play, many times faster than either the quickest body or the most mercurial lyrical impulse.
Sometimes when I play I make a semi-conscious effort to keep my listeners from even thinking of tapping their foot of otherwise slowing themselves down. I want every vibration to be heard and felt.