Brick laying is an art form. It takes a lot of time to get the technique down. Professional masons make the process look much easier than it really is.
One thing that can help is to buy a pre-mixed mortar instead of trying to make your own. It should have a better consistency than what you are using. There are also many different kinds of mortars. A type M or type S mortar would be well suited for putting in a retaining wall.
Type M mortar is the strongest on the market. It is used most when the wall you are building is below grade, and has the best compression strength and is much less likely to squeeze out than other mortars when you are using heavy block. The downside of this kind of mortar is that it is actually too strong for most kinds of concrete block. If there is any movement in the wall, the block is more likely to crack instead of having it crack in the joints. This kind of mortar should never be used for standard brick.
Type S mortar is similar to type M, but it is not quite as strong. This is probably going to be the best type of mortar to use in your application. It is still very strong, but it is a little easy to work with. When using this kind of mortar, you want to make sure that you don't stack too many courses of block in a short time period since having too much weight on the block would squeeze out the mortar.
If you are going to lay brick over the top of your wall, then type N mortar is the way too go. It is still fairly strong, but not so strong as to destroy the brick if the structure is removed. There is a much better chance of being able to reuse the brick later on with this type of mortar compared to type M and type S.
There are also other types of mortar, but those types are generally not well suited for outdoor use, especially if it is load bearing.
You may have a hard time finding all of these different mortars locally. Type N mortar is usually marked as general purpose, and it is the most common.