I have some satin black enamel paint, after applying it I found it to be too glossy. I know you can make paint more glossy, for example by adding varnish to it, but is there a way to make it less glossy?
I had that same problem (oil based paint too glossy). 000 steel wool worked, sanding worked, tsp had no effect, vinegar had no effect. I mixed some talc (baby powder) with my oil based high gloss paint and was able to get the flat appearance I wanted. I found that it took a ratio of about 6 parts paint to 1 part talc to get the desired result
The 2 answers you have amount to (i) roughen it, and (ii) satin/matte varnish it.
I suggest doing both. When you're applying one finish on top of another, neither of which is primer/undercoat, you need a good key -- hence roughening the surface. You don't want to create visible scratches though. At some point you want to leave it in a warm place for a couple of days to make sure it's completely dry. Then wipe over with methylated spirits to remove dust/grease. Then you can apply your top coat. Be sure to use a product with the same base as the paint you've just put on -- a water-based finish like acrylic varnish won't stick well to fresh oil paint.
You could of course -- if you're buying something anyway -- get a satin paint for a last coat.
TSP has been used by painters for many years to "de-gloss" paint.
I have used it to both flatten a painted rooms glossy look, as well as to provide a surface that a new top coat can adhere to.
Gloss enamel needs to be cut (a surface tooth) established for new paint to adhere. If you do not provide a tooth for new paint it can peel off of glossy enamel (a lesson learned for me long ago).