Other servers can and should use Alice's `replies` collection to see which replies she has consented to. They can and should obscure or hide altogether replies that aren't in that collection.
Mastodon doesn't do either of these things, by the way. It doesn't let you reply to Alice's followers, and it doesn't use the `replies` collection for showing and hiding replies. It's too bad; these are really valuable features of ActivityPub.