# Sample robots.txt file (make sure the filename is ALL LOWERCASE on Linux/Unix systems) # This file should go in your web site's ROOT directory # The root directory is where your site's main /index.html file would be found # It is usually found in /yourhomedir/public_html/ or /yourhomedir/httpdocs # Where "yourhomedir" is your user account's name # User-agent: * Crawl-delay: 5 Disallow: /account.php Disallow: /account_edit.php Disallow: /account_history.php Disallow: /account_history_info.php Disallow: /account_newsletter.php Disallow: /account_password.php Disallow: /account_notifications.php Disallow: /address_book.php Disallow: /biffwriter.php Disallow: /format.php Disallow: /parser.php Disallow: /olewriter.php Disallow: /checkout_shipping.php Disallow: /password_forgotten.php Disallow: /popup_image.php Sitemap: http://www.dmarket.com.ua/sitemapindex.xml Sitemap: http://www.dmarket.com.ua/sitemapproducts.xml # Feel free to add any other pages on your site that you don't want to be indexed by # the search engines. # PLEASE NOTE: Any pages that you list here should be secured by other means if you # don't want people to be able to view them, as some malicious users will look at a # robots.txt file to try to find "hidden" or "secret" areas of web sites to find # confidential information. # Just Uncomment a line or add new ones as you see fit. # Disallow: /private # Disallow: /hidden