One of these
General concept. You might need to get a different size.
This thing wraps around the shower pipe on your side of the caddy. You run a bolt and nut through the hole to clamp it tight (if you sized it right there will be a 3/8" or 10mm gap give or take.
This clamp keeps the caddy from sliding down.
That metal is stainless steel. very important. The rubber prevents scratching the chrome pipe, which could happen with an automotive hose clamp (those are also not all stainless and will rust).
The bolt I recommend is 1" (25mm) long brass bolt (machine screw) of thread 10-24 (or M4-0.7). Also get a matching brass nut. I suggest a family owned hardware store rather than Home Depot, because HD will sell you these in bags of 5 or 20, and the family store will sell you singles.
We've had ours at least 5 years with nothing but the normal tarnish that brass picks up.