Clubhouse is an active social media platform, with lots of users who have different interests that change all of the time!
It's not unusual to see your follower count fluctuate dramatically from day to day as folks in the community make changes to better curate their Clubhouse experience.
The people you follow impact the rooms and clubs that show up in your hallway, so Clubhouse users tend to manage who they follow more actively than they would on other platforms to make sure they're seeing and joining the conversations they want to participate in.