You can easily mute other people by accessing the participant list on the right hand side of the party view.
To mute a single person, click the speaker icon next to their name. On web, you can find the people in the party by clicking on the text at the very right that says "N Participants". On mobile, you can find this by clicking the icon with the three lines on the left hand side, and then finding the speaker icon, see image below. To mute all other participants, click the "Unmute all" text at the very top of the participant list. This is particularly useful if you are streaming content from your computer using Computer Audio.
On mobile, you can mute others by clicking on the headphones icon on the right side of your screen.
If you want to unmute people, just do the same thing as above but in reverse.
Enjoy the silence!