Notice that sill plate sitting on top of the beam? Held on with some nails hammered part of the way in, then bent over the flange? Do the same with your top plate. The sill plate has the advantage of gravity holding it in place, so you'll probably want to put more bent nails in, especially if you'll have kids down there playing who may be running into it.
Since the wall's not load bearing, it just has to stay vertical, and that should do it.
To address Tester101's valid concerns:
You will need to use a 2x6 to match up with the width of the beam. You can build the 2 x 4 wall to one side of the 2x6 and have all the 'ledge' on the other side (suitable for the utility room side, if there is one), or you could center it and have a small bump on either side.
If you go for the uneven build, you'll get an offset of (5 1/4" - 3 1/4") 2" on one side. If you drywall that, the sheetrock will take 1/2 - 3/4" of an inch.
If you center the support wall, you'd have 1" on each side leaving 1/4 - 1/2" after dry walling (depending on thickness).
You could, of course, build the whole wall out of 2x6, and that resolves all the above issues.