The expression to be a bore had been used in print in the sense of "to be tiresome or dull" since 1768 at the latest. The expression "boredom" means "state of being bored," 1852, from bore (v.1) + -dom. It also has been employed in a sense "bores as a class" (1883) and "practice of being a bore" (1864, a sense properly belonging to boreism, 1833). The word "bore" as a noun meaning a "thing which causes ennui or annoyance" is attested to since 1778; "of persons by 1812". The noun "bore" comes from the verb "bore", which had the meaning "[to] be tiresome or dull" first attested [in] 1768, a vogue word c. 1780–81 according to Grose (1785); possibly a figurative extension of "to move forward slowly and persistently, as a [hole-] boring tool does."
The French loanword ennui comes from the very same Late Latin word that gave us annoy — inodiare ("to make loathsome"). We borrowed ennui several centuries after absorbing annoy into the language. Ennui deals more with boredom than irritation - and a somewhat specific sort of boredom at that. It generally refers to the feeling of jadedness that can result from living a life of too much ease. The poet Charles Lloyd described it well in his 1823 Stanzas to Ennui when he referred to that world-weary sensation as a "soul-destroying fiend" which visits with its "pale unrest / The chambers of the human breast / Where too much happiness hath fixed its home."