All Players are limited to 20 visible followers to improve the performance of the game, particularly with regard to the friends and followers list. This means that you can have more than 20 followers, but will only be able to see the most recent ones. And this also ensures that these recent followers are usually more active.
Rest assured that the followers you don't see are not deleted.
Note: There’s no limit on your Facebook Friends and Game Center Friends.