During heavy downpours, especially ones when windy, water would begin to trickle and drip from the outside surface of the exhaust flue in the basement. Not sure how long this has been going on since the house was just purchased recently. Should I call an HVAC or a roofing pro to take a look? Initial Googling suggests a bad rain cap, but I'd prefer someone with the correct experience to make sure it's nothing else. Also, would the white coating be an indication that this has happened before? The home inspector never mentioned anything.
This is a common problem in windy areas with rain. In many cases the leak is rain getting inside the pipe and running down. Based on your photo I would say this has been happening for a long time.
I would want to call a general contractor that also does roofing because the roofing only guys may not be the best at repairing the interior damage. The cleanout looks like it is rusting through. Water has probably been sitting in there a long time.