I think I've found the problem.
I had the rate control set to Multipass - Nth pass.
I changed it to what is above and it started recording properly.
Thanks very much. I had completely forgotten about those rate control options. I had messed with them before but never had any problems. I think a recent update for windows caused the problem. The reason being is that in the game Dust an Elysian Tail, the cutscenes are in wmv format and when they play in the game only the bottom half of the screen shows. The top half is black. There was a security update, KB2803821, that did something with windows media. It's possible that there was another update that messed up the multipass function.
Thanks very much. Although file sizes will be slightly larger now, at least the quality is where I want it now.
With that, my problem is solved.
I will message the devs for the x264 codec about the multipass problem.