I encountered some mkv files that play poorly under Mac OS X using VLC media player, the sound is OK though. The video codec is avc1. I tried to fix the problem by going to Preferences and change Video Output Module, but that didn’t help. The VLC player version is also the latest one, so update of the program is not possible.
Without a true solution, I did find out an alternative player under Mac OS X. It’s called MPlayer OSX Extended. For people familiar with Linux, MPlayer is no stranger. It is my favorite media player for Linux. Here is a screenshot showing the same scene under MPlayer OSX Extended.
My experience with MPlayer Extended is good. It supports more file formats than VLC player does. The subtitle supports is also better. As with other free open source software, it has some quirks though. And you cannot fast forward the video like you can on VLC.
Here is the link to the developer’s page. The link to download the player is currently down. Please try again later if you cannot download it.