The worker, a 52-year-old who has not yet been identified, died from injuries sustained while working at a housing development at Hallmeadow Place.
Three fire engines rushed to the scene, as well as paramedics and police.
The air ambulance was also dispatched but after circling the town, it did not land.
Contractors Ashleigh (Scotland) confirmed the man’s death.
Their commercial director Euan Clark said: “We are devastated to confirm a fatality on our Hallmeadow, Annan site this morning.
“Whilst full details are still to be established, our immediate thoughts and prayers are with the family, friends and colleagues of the operative involved.
“The wellbeing of anybody working on our sites is absolutely paramount to us.
“A full investigation will be carried out, and we are working with all of the relevant authorities in this regard.
“We have closed the site to allow the investigation to be concluded.”
In a statement, a Police Scotland spokeswoman said: “Around 9.45 am on Thursday, we were called to a report of a mast having fallen on a 52-year-old man at a building site.
“Emergency services attended and the man was pronounced dead at the scene.
“Enquiries into the incident will be carried out alongside the Health and Safety Executive (HSE).
“A report will be submitted to the Procurator Fiscal.”
And representatives from the HSE yesterday travelled to the town. A spokesman said: “HSE have been made aware of an incident at a house build site at Hallmeadow Place, Annan and are currently attending the site to assist Police Scotland with the investigation.”
