What is probably happening here is you have power to the light on one black, another black to the switch, and a return path along the whites. That is, the switch is AFTER the light.
This is generally unsafe as the light is still energized, even when the switch is off, and should be fixed regardless.
The first thing I'd do is beg/borrow/steel a non-contact voltage detector.
With everything still attached, switch the light off. If I'm right, the detector would read hot on BOTH black wires, and the whites will be un-energized. Flip the switch and the whites should be energized as well.
If this is indeed the situation, the fix is relatively easy. Turn the power off! Remove the existing light.
Figure out which pair is wired to the switch. The easiest way is to use a meter in resistance mode. Find the black-white pair that shows low resistance when the switch is on, and open circuit when switched off. Mark the white wire with a loop of electrical tape. This will be the switched hot.
Now wire the blacks together with an appropriate wire nut, and make sure it's a secure connection. You should not be able to see any copper.
Wire the switched hot (the white wire marked with tape) to the black wire of the new fixture. Wire the remaining white to the white wire of the fixture. Make sure you've grounded the fixture according to the manufacturers instructions.
Mount the fixture, and turn the power back on.