They are probably messing with you.

I've never had this kind of experience with cyclists, maybe it's just the people in your area?

Just politely ask them to keep it down after 10pm or something. And if they don't stop complain to the owner/management.

That's pretty normal, just ask before you get too nervous about it.