For general purpose use like this you should be using a standard crosscut handsaw, with 8 to 12 teeth per inch.
It looks like your saw has a bent tooth or has hit a nail. The saw's teeth have a "set" to them, and sometimes the teeth on one side can be damaged, making a saw cut a curve. The picture above shows a lot of wander for such a small distance. If your saw is damaged, you'll have to repair or replace it to get a good straight cut. Damage is:
- The set of the teeth is uneven,
- the saw is bent,
- the teeth on one side of the saw are duller than the other.
+1 on all speed square advice. mark the top of the board, then mark an adjacent side, then, from the top again, mark the other side.
Start on the far corner. Keep you elbow, wrist, and saw in a line. don't let your elbow wander around.
It won't take long before you can split a pencil line.