I've recently begun learning Korean and I just learned that 의 is often pronounced like 에, hence why 저의 becomes 제.
If this is the case, how can I tell the difference between 의 and 에 in speech?
They are often pronounced the same. When this is the case, then rather like with English homophones like for/four or hear/here, you just have to tell from the context.
Normally, 의 will link to another noun that comes after:
한국의 수도는 서울이다 - Korea's capital is Seoul.
에 will usually link to a verb that comes after:
어디에 가요? - Where are you going?