You need to confront him. He is not your friend. He WILL unexpectedly rape you again and again if you don’t stop him, now. My former roommate was ‘ belittled’ into unwanted sexual three times before she found the nerve to claim her own body. He never understood what he had done to her mind.

Good luck to you on your life journey.