Swim trunks tend to loosen and stretch slightly when they get wet. Tie them, get in the water for a minute or two, and then retie them tightly.
Ensure your trunks have a functional drawstring that goes all the way around the waistband, not just a decorative tie in the front. Tie it tight enough that you can’t easily slide a finger between the band and your skin. my swimming trunks have been sucked off hot