Unfollowing a user won't notify them that you've done so, and you'll stop receiving notifications of their Clubhouse activity.
To unfollow another user:
- Tap your profile pic at the top right.
- Tap Following in your profile.
- Tap the blue Following button next to the person you'd like to unfollow.