Scenes of crime officers searched the garden area and a cordon was set up around the front and side of the disused Dumfries and Galloway Council – owned property, known as Cornerstone House.
The search followed what police described as an incident involving a 21-year-old woman in the area late on Saturday or the early hours of Sunday.
A spokesman for Police Scotland said enquiries were ongoing although no further details were being released at this time.
He said: “Police are keen to hear from anyone who may have been in the area of Bank Street in the town around this time to get in touch if they saw or heard anything suspicious.”
Police at Annan can be contacted through telephone number 101.