In our setup the default is to discourage search engines. This is to protect you from accidentally publishing your experimental doodles. In our experience many site owners can take many weeks, months, sometimes years to begin work and then finish their site.
For your sanity we recommend you make sure you visit the settings page Dashboard > Settings > Reading and scroll to the bottom, where you will see the options available.
Whilst it is up to the discretion of search engines to respect the discourage notice, the main ones do. If they find you page they will display a notice that robots.txt has denied them access. If your site has mission critical information that must not be made public i.e. is embargoed you should tighten the options to the users of the website only or to admins only.
Rookie errors:-
- Coming soon pages that never come – good idea if part of a concerted plan to go live and you need to build a bit of momentum
- Under construction pages – this is very 1990s – the modern website allows you to make everything draft – better not to publish at all or the search engine will remember your old content for much longer than is ever desirable
- Forgetting to change the discourage search engines notice for months and months thereby letting the search engine forget about you entirely …