In 1565 Supreme Commander Tzortzis first came up with the idea of building a large barrack house in the town of Chandax. The following year he joined with the Duke of Crete in assigning the task, together with the completion of other similar projects, to G. Savorgnan.

The original design provided for two barrack houses.

The building extended from a point southeast of the Voltone Gate to a parade ground called the Piazza d'Armi or Campo Marzio - now Eleftherias Square - where soldiers would drill.

Prior to this it is believed that there may have been a smaller barrack house built by the Genoese commander Enrico Pescatore, which was levelled by a major earthquake in 1303. This would explain why the square was known as Piazza D'Armi even before the St. George Barracks were built. For the intervening period, lack of accommodation meant that soldiers in the Venetian army - the vast majority of whom were mercenaries - were billeted in the homes of poor families.