So, the latter (그러지 마세요 = Don't do this please) is more like an order since it's direct. But would the former (이러면 안돼요 = shouldn't do this) sound less like an order and more like a suggestion? How would you see the difference in tones in the two sentences?
In your example, the difference in nuance is based on what the subject of the sentence is. Technically, the subject of "마세요" is "you", but the subject of "안 돼요" is "something"
I'll show you with direct translation
그러지 마세요. = You shouldn't do this.
이러면 안 돼요. = If you do, something won't work.
"안 돼요" is "안"(not) plus "돼요". "돼요" has many meanings, but the closest one in this example is literally "work".
So, "이러면 안 돼요." is a more indirect way of saying "You shouldn't do that" by stating the circumstances of the situation rather than speaking directly to the other person, and can have the effect of being polite.
The expression "돼요" with the subject omitted is used so frequently that it's pointless to specify that the subject is omitted, but it's worth knowing its origins to understand the subtle differences in meaning.