You're planning on 2x3 furring strips? Those are small studs! (No, really, you can buy 2x3 studs for interior, non-load-bearing walls.) Furring is usually 1x material. Go to town with the 2x, though, if you want to!
"attached to existing studs of the drywall" are you attaching the furring to the drywall, studs or concrete? It's not clear but in any case...
Glue could work, but most people in the construction industry would drill holes for TapCon™-type screws or use a powder-actuated nailer to drive nails through the wood into the concrete and wouldn't give a 2nd thought to damaging concrete with them. 1x material might split with a powder-actuated nailer, though - go with screws. The advantages of screws:
- No waiting for glue to dry.
- No trying to figure out how to clamp the wood to the concrete while it dries.
- No worry about bumping wood and moving it out of position before the glue has dried.
- You can remove it later if you either don't want them, or if you realize you put one in the wrong place.
If you're going to go with "glue", though, use an appropriate adhesive for attaching wood to concrete. Your kid's bottle of Elmer's Glue™ or even fancy wood working glue will look at the concrete and laugh at you for even trying.
If you're spanning from wood/drywall down over the concrete, then simply use the appropriate screw for the type of material behind your new furring - wood screws where you're attaching wood to wood, and TapCons where you're attaching wood to concrete.
We can't tell you if a permit is required because you haven't told us where in the world you live. If you're in the US, it's very unlikely that you'd need a permit to hang some drywall in your garage. Where I live, repairs (including completely tearing out our front porch and replacing it) don't require permits, while new construction (adding square footage to the building) or major renovation (moving/rearranging walls) does. If you're not sure, contact your local building department - they're the ones who will know for sure.