Definitely do NOT fill them with cement. At least not the parts you grab onto. You could probably fill the base bars (the ones that actually touch the floor) if you simply want a little more weight to stabilize it but since the ends are capped, sand/gravel would accomplish the same task and I really think you are underestimating the sturdiness of PVC here.
There are a lot of results in google for "diy dip station" and almost every single one is some sort of PVC construction. Here is a youtube video of a guy doing the build along with a demonstration at the end of him using (although briefly).
Here is a quote from a Reddit thread on this very subject:
My dip station
I found out about DIY dip stations on this subreddit and decided to try it out for myself. I used this guide to figure out the parts I needed and overall construction.
I went with 1-1/2" thick PVC to fit my hands better, so test out the different sizes when you shop at your local hardware store. The total pipe length needed is about 28.45' so, since PVC pipe is sold in 10' sections, I bought 3 pipes.
Breakdown of the cost (according to Home Depot prices)
- 3 PVC 1-1/2" X 10' pipes @ $5.35 ea = $16.05
- 6 PVC 1-1/2" tees @ $1.87 ea = $11.22
- 8 PVC 1-1/2" 90o elbows @ $0.77 ea = $6.16
- purple primer, and PVC cement combo pack @ 7.48
Total basic cost: $40.91 ($43.37 after taxes)
- I added spray paint (aesthetics) and athletic tape (increased grip) for an extra $14.49 ($15.36 after taxes).
Total 'Murica cost: $55.40 ($58.73 after taxes)
You can definitely do this cheaper if you know anyone with leftover PVC pipe (plumbers) and can borrow PVC cement and purple primer. I ended up not using very much primer and cement so people are likely to have it just laying around waiting to be used again.
From the guide link he suggests leaving the top and bottom sections un-cemented for easier storage. I decided to take it a step further and leave these three sections separate for even easier storage. So far I have not noticed any problems with this assembly when doing dips.
In the comments there, the claim is that it has been tested with ~315lbs (weighted dips).
If you need extra assurance, here is a youtube of a couple guys who actually used the DIY station in Home Depot to cut the pipes and put it together right there in the store (not glued).