If your bifolds have molding around them then that should cover any expansion gap.
If they don't have molding I would not add molding if the doors fit because you would need to cut every door vertically to adjust for molding depth.
An easy solution is continue trim throughout the closet and notch out the outer corners of the closet doors.