The Phoenicians most likely built the original casle of Alora. The Romans, after their conquest of Spain, restored the fortress. The fortress was further enlarged by the Visigoths and, above all, the Moors, who conquered it early during their conquest of Spain. Álora became a stronghold. The Christians of northern and central Spain attempted numerous times to conquest the town, but were successful only in 1484.