I am aware of other sound changes, such as aspiration of consonants adjacent to ㅎ, nasalization of final consonants following a nasal initial consonant, adjacent ㄴ and ㄹ in any order becoming ㄹㄹ, ㄹ becoming ㄴ following a velar consonant ㅇ or ㄱ (which itself becomes ㅇ), and final alveolar stops becoming alveolo-palatal affricates before /i/. However, among these I don't see any rule that applies to 꽃잎. What rule does it follow, or is it an exception?
This is a phenomenon called ㄴ첨가. Although it originated from the deletion of ㄴ before /j/, but it's beyond that now. Compare it to the intrusive R in non-rhotic English accents, and you'll get the idea.
This phenomenon occurs at the border of two morphemes in a compound word, when the second morpheme starts with /j/. A ㄴ (or two ㄴs) gets added just before the second phoneme in these situations.
꽃잎 is 꽃+잎, so ㄴ gets added so that it becomes [꽃닢] -> [꼳닙] -> [꼰닙]. Other examples are 솜이불 [솜니불], 물약 [물략], 서울역 [서울력].
When the first phoneme ends in a vowel, this is reflected in the orthography as well, by adding ㅅ to the first phoneme. 나무+잎 [나문닙] is 나뭇잎, not 나무잎. This rule applys to every word that has this construction, with almost no exceptions.
Sometimes, this even happens between two separate words. 할 일 is pronounced [할릴], and 옷 입고 is [온닙꼬], etc. The one that occurs between separate words like this, however, are not universal, though, and may only apply when talking hastly.
There is a rule that words cannot begin with ㄴ/ㄹ+[i/j], as explained in this question. That is, they cannot begin with 니, 냐, 녀, 리, 료, 류, etc. Exceptions are recent borrowings. But this rule was not always so: in medieval Korean such words were possible, but recent sound changes eliminated the initial consonant.
잎 (leaf) was 닢 in the past; it's still listed as an archaic form in the dictionary. When it's alone, 닢 became 잎, but as the second part of a compound noun, the original pronunciation is retained. In the case of 꽃잎, the pronunciation following assimilation rules goes like this:
꽃닢 -> 꼳닙 -> 꼰닙.
Another example is 솔잎 (pine needle), which is pronounced like this:
솔닢 -> 솔립
Another tricky one is 깻잎, which is 깨 (sesame) + 잎, but it has a 사이 시옷 inserted between:
깻닢 -> 깯닙 -> 깬닙
In the case of 한자 words, we will write it as it is pronounced; hence 여자 but 소녀; but in the case of the pure Korean word 잎, we always write 잎, which is where the confusion comes from.