The device was found by a dog walker on the Summergate section of the foreshore on the south side of Annan.
Nearby residents stated the object, believed to be around 18 inches long with tail fins, may have been washed up by a recent high tide.
Father-of-three Jeff Creighton, whose small-holding home was one of two properties to be evacuated, said: “We have been warned that we could be out of our homes for a number of hours.
“It is a precaution in case the device is live and the fact we are only 200 metres away. It might be a mortar or smoke bomb of some sort.
“We are going to stay at my daughter’s home nearby meantime.”
Mr Creighton added he, his wife and the neighbouring property owner would return once the emergency services had given the all clear.
A spokesman for Police Scotland said: “Emergency services are in attendance and have cordoned off the area as a precaution.”
A Royal Navy Bomb and Mine Disposal Unit from Explosive Ordnance Disposal at Helensburgh were contacted and were expected to examine and deal with the device later on Friday night.