A GFCI is one way to make an ungrounded circuit that you cannot manage to retrofit ground for safer than an otherwise unprotected ungrounded circuit.
However, it is always preferable (and required in any new work) to have an actual ground connection as well. Things fail in various ways, and if the grounding pin on a grounded item is actually connected to ground, more failure modes are protected against.
As a somewhat contrived example, if a tool with a metal shell and a grounded cord was plugged into a an ungrounded GFCI outlet but on an insulated surface, and it had a failure connecting hot to the grounded case, the case would be live, but without current flowing (it's on an insulated surface) the GFCI would not have a reason to trip until you picked it up and provided a path for current to go out of the circuit. If the GFCI works, you get a mild shock and it trips - if it fails (and they do, though more often "dead" than "alive", presumably by design) you just get shocked.
If it was grounded, either the GFCI would trip or the circuit breaker would trip. You would not be involved.