I have a fireplace that was added to the house after it was built, and when I’m using my wood burning fireplace I noticed some sort of smoke coming out of my high hats (recessed lighting). I have opened my ceiling and stuck a fire alarm in the hole to see if it was actual smoke, but the alarm did not go off. Do you know what can it be?
What's a "hi hat"? Is that like a can light (i.e. a light installed in a can flush behind ceiling drywall)?
Anyway: stuff, whether smoke or otherwise, shouldn't be blowing from any lights. One hopes that it's not actually smoke, and I would work harder to confirm that it's not. Check the entire fireplace flue and any spaces it traverses in the house and make sure it's sealed and far enough from any combustible material.
That done and having ruled out actual combustion products being pulled through the lights, it's still possible your fireplace is drawing from the house, pulling air through the gaps around the lights, and dust or other material (e.g. cellulose insulation) with it. You should make sure that the fireplace has an adequate air supply while operating, e.g. opening a window nearby a little so it can pull air.
A roaring fire will draw a lot of air. That air has to come from somewhere, so if you don't accommodate it and ensure the air comes from some place that's reasonable, it will be pulled from places that aren't reasonable. :)
I would be remiss to not acknowledge the suggestion in this comment:
Are you sure it's not just a bit of smoke coming out of the front of the fireplace, and then becoming visible as it is highlighted by the can lights?
I've taken your statement at face value, and that the substance is in fact coming from the lights. But of course, it's worth double-checking that assessment before pursuing other possibilities.
And note that if it does turn out that the smoke is in fact coming directly from the fireplace, that suggests that the fireplace is not drawing air properly. Possibly the flue just hasn't been opened, or maybe there's some kind of restriction. Ironically, one of the things you should check in that case is the same as I've suggested above for a completely different possibility, and that's simply to make sure you have adequate air supply to accommodate the fireplace's draft.
For a more specific answer, you should edit your question so that it includes detailed, in-focus photos of the light fixtures in question, the fireplace itself, any venting associated with the fireplace (including the flue if possible), and if possible, the phenomena you're asking about itself. More specifics about the actual installation of the fireplace would be useful too: does it vent to a chimney, a flue pipe, a chimney with a liner, etc.? Is there any heat-exchanger or dedicated air source?
The more details you can provide, the better an answer you can get. Right now, it's not possible to suggest anything except very broad generalities.