Is there any link between the French word "empirer" (to worsen) and the noun "empire" (empire)? It just seems strange to me....

The only link is the use of the prefix em-. 'Empirer' is derived from Latin 'in' and 'peior', the comparative of 'malus' (bad). 'Empire' is derived from Latin 'imperium'. 'Imperium' is a noun that comes from the verb 'imperare', which in turn consists of the prefix 'in' and the verb 'parire' (I'm not sure if this is the correct infinitive; my Latin dictionary only gives 1st p sing verb forms, e.g. in this case pario, peperi, partum. It's been too many years since my Latin classes for me to remember what conjugation this is)

#1 -- That verb is third conjugation, infinitive parere. I bet your dictionary has a little number 3 after (or before ) the principal parts.
But that verb, meaning to give birth (souce of English postpartum etc) is not the root of imperare, which is from the first conjugation verb parare, to prepare.
