몽골 is a transliteration into Hangul, but 몽고 (蒙古) comes from the Chinese characters for Mongolia. The latter is also a transliteration of sorts, but via Chinese.
The National Institute of Korean Language's Korean-English Learners' Dictionary entries explain the differences a little better. At least it mentions on the entry for 몽고:
the name is derived by reading "Mongolia" in Sino-Korean characters.
From my personal experience, 몽골 is the more common way to say it. The entry for 몽골 also has 3 stars, meaning it's a very common word, whereas the entry for 몽고 has none. According to the Korean Wikipedia page on Mongolia (headword spelled 몽골) and its source for this information written by a Mongolian person living in Korea, Korean people do in fact sometimes use 몽고, but Mongolians find it offensive because it harkens back to the time when China viewed them as the barbaric "Huns."