Can gravel and stones bought from a builders yard be used in a tropical fish tank ?

Obviously i will wash and clean first ?

Depends on the type of gravel and stone. Some will raise the pH of the water which makes it unsuitable for fish that like a slightly acidic water condidtions. Some may also react with the water to make it cloudy, even when washed beforehand.

If it's only for a goldfish tank then you might be able to get away with slate chippings, but they need to be washed very well before hand, and have a marking on the bag that says it suitable for pond use.