I always build my Temple first, anyways. I don't see why people leave it for their 2nd/3rd building. Surely it's faster to gather resources by sending your Ghost near their base and get your leader to collect the resources near your base/your side of the map?