Adding to other answers, when learning other languages, you should not compare their pronunciation with your native languages. Each language has its own way of pronunciation and it can never be pronounced using an alphabet of other languages. For example, Seoul is written as 'ソウル' in Japanese and read as '소우루' which is quite different from '서울'. For Japanese and Spanish speakers, pronouncing '어' is quite difficult as they don't have this vowel (ㅓ).
English seems to have '어' sound of Korean, however, English speakers also find it difficult to pronounce 'Seoul' and they usually pronounce it as '쎄울' or '쏘울' even if it is written as 'Seoul'. That means 'Seoul' is romanization of Korean, but it does not necessarily lead other language speakers to pronouncing it as '서울'.
Is it possible to tell a vowel from another one just by listening?
Yes, but some vowel pronunciations do sound confusing to native speakers, too. For example, when they pronounce '얼핏 (in an instant, in a flash)', some (especially those who speak a dialect) pronounce it as '을핏' when they speak fast. But luckily, there are no words similar to '얼핏' in Korean and sometimes native speakers know it is '얼핏' by the sound of '핏'. It could be a guessing game and I believe all the languages that I speak and have been learning have this kind of problem, i.e., actual pronunciation doesn't necessarily represent what is written in any language.
Are there any tips for a non-native learner to tell them apart?
Yes. Try to pronounce each block as clearly as possible. Even if you are not good at pronouncing '얼', you would definitely be able to pronounce '핏' as clearly as native speakers do. Then, no matter how you pronounce '얼' in '얼핏'. native speakers would understand by your pronunciation of '핏'. In other words, you can pronounce '얼핏' as '올핏' as Japanese have 'ㅗ' sound and people would understand it as '얼핏', but you have to make sure you know what this word means and its usage.
There are many non-native speakers appearing on Korean TV shows and their pronunciation sometimes is not as clear as native speakers. However, the reason why they don't need an interpreter is they know the word and native speakers can understand what they say from hearing the next blocks or words (context).