The same places that sell RO filters sell spigots. Drill a hole in the countertop and drop it in, tighten the clamping nut below, attach the waterline.
If you want it to sprout from the cabinet above, you'll be more in a jury-rigging place. If you don't mind crude, a simple valve poking out on a tube is easy, but anything like a finished tap/faucet for that application is likely to be hard to find.
The lack of a drain under it could present some possible problems if it drips or if you flip it on to fill the coffeemaker and forget it. That's why they are usually associated with sinks.