What does normally open, normally closed mean ?

The word “normally” refers to deenergized condition of the relay (no power on coil).

The second words “open” and “closed” refer to the position of the contacts at the deenergized condition.