Now before you submit your website, it is important to generate sitemap. The Sitemaps protocol allows a webmaster to inform search engines (not only Google) about URLs on a website that are available for crawling. It is made of XML file listing all the files in your website indicating their importance (priority) and  change frequency.

Here is a sample Sitemap XML file.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">	<url>
		<loc>http://wpmarketing.org/</loc>
		<lastmod>2009-10-13T12:28:17+00:00</lastmod>
		<changefreq>daily</changefreq>
		<priority>1.0</priority>
	</url>
	<url>
 
		<loc>http://wpmarketing.org/privacy-policy/</loc>
		<lastmod>2009-10-14T04:40:51+00:00</lastmod>
		<changefreq>weekly</changefreq>
		<priority>0.6</priority>
	</url>
	<url>
		<loc>http://wpmarketing.org/2009/10/top-5-wordpress-plug-in-for-internet-marketing-all-in-one-seo-pack/</loc>
 
		<lastmod>2009-10-13T12:28:17+00:00</lastmod>
		<changefreq>monthly</changefreq>
		<priority>0.4</priority>
	</url>
	<url>
		<loc>http://wpmarketing.org/2009/08/fighting-adblock/</loc>
		<lastmod>2009-10-13T04:58:18+00:00</lastmod>
 
		<changefreq>monthly</changefreq>
		<priority>0.2</priority>
	</url>
 
        <!-- cutting off . It's too long to show here -->
 
	<url>
		<loc>http://wpmarketing.org/about/</loc>
		<lastmod>2009-08-05T12:08:20+00:00</lastmod>
		<changefreq>weekly</changefreq>
		<priority>0.6</priority>
	</url>
 
</urlset>

Basically, it is not that difficult to create the file, but tedious and lots of labor may be involved if done by hand manually. Especially for the blog site with frequent article posting, you will need a plug-in for that which would update the Sitemap automatically. So here is Google XML Sitemaps plug-in.

Basically his plug-in will create a XML-Sitemap of your WordPress blog. It will include all the WordPress pages and posts and even external pages that you defined. The sitemap is automatically updated whenever you modify any page or post, so it’s just convenient. It also notifies Google, Bing, Yahoo, and Ask.com automatically. You can’t ask for more.

It works pretty well right after the activation, but it is possible that you may need to create a file named sitemap.xml and/or sitemap.xml.gz in your WordPress root path and make them writable by webserver (If you are using Hosting providers with up-to-date system, you wouldn’t have problem).  And the default settings are good enough so you should not do anything else. But if you want to rebuild the sitemap, click on the ‘rebuild the sitemap manually’ link marked with Red box in the above image.

By default, Google XML sitemaps is automatically calculate the priority of each posts using the comment count of the posts. If you want to custom define the priority,  you can select ‘Do not use automatic priority calculation’ option in the ‘Post Priority’ group box.

Then you can use Priorities group box to specify custom priorities.

You can easily check the sitemap by adding ‘sitemap.xml’ at the end of your WordPress URL. For example , http://wpmarketing.org/sitemap.xml . Here is the output.

Now that you have the sitemap ready,  you can go to your Google Webmaster Tool account and add your sitemap.

One extra feature would be nice – To be able to define priority per post and page. I hope the author is planning to add this feature soon.

Next plug-in will be about Plug-in for Google Analytics. Google Analytics is a free web statistics application provided Google.

Thank you for visiting wpmarketing.org

* Google XML Sitemaps Website : http://www.arnebrachhold.de/projects/wordpress-plugins/google-xml-sitemaps-generator/

* References: List of other Sitemap generator tools (Not just for WordPress): http://code.google.com/p/sitemap-generators/wiki/SitemapGenerators

It’s probably because your Privacy Settings is set to search engine “blocking” mode which says “I would like to block search engines, but allow normal visitors”.

You should click on the Settings , Privacy menu , and allow search engines by selecting “I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers” option.

I have been using WordPress and performed installation several times in the past using the WordPress default installation index.php file. But decided to try Fantastico this time because it simplifies the setup process. There is no separate DB setup process , quicker and faster setup. I liked it. But then I guess Fantastico’s WordPress predefined options for privacy setting was to block search engines.

Usually you would start promoting your website by submitting your site to Search engines Directories and social bookmarking websites. Then search engines would crawl to your website and index pages within few days.  But I found the problem when my new website not being indexed , but also AdSense was not working at all.

The AdSense AD always showed Public Ads. As long as I understand, public Ads are showing when Google does not properly analyzed the webpage yet. Then I checked with Webmaster tools and I found robots.txt was blocking the webbots from crawling.

I am not sure if this behavior is only applied to Hostgator, but highly possible that most of the other hosting provider’s Fantastico might be the same, installing WordPress with blocking mode.  So be careful and check privacy setting after installation if you use other similar setup tools to install WordPress.

If you use the original setup that comes with WordPress zip package, it asks for the privacy settings during the setup. I guess simpler is not always the better.

Some ways to check your websites heath.

1. Check if your site is properly cached in the search engine: For Google Search, you can use special search command “site:” to find out if your website is properly indexed with the Google. (Eg. “site:wpmarketing.org”) If you searched your domain with “site:” prefix and no results came out, you should suspect and check your robots.txt

2. Check your robots.txt file : WordPress automatically generates ‘robots.txt’ when Permalink is on. It’s not an actual physical file, but WordPress is dynamically generating when requested. You can view the file by typing ‘robots.txt’ (Eg. http://wpmarketing.org/robots.txt )  Below example shows a sample robots.txt file content which is opened to Search Engines. Note that there is nothing showing right after “Disallow:” which is OK. If there is slash ‘/’ character showing after “Disallow:”, It means to disallow all content from being indexed by Search Engine. So watch out.

Good – Opened to Search Engine

User-agent: *
Disallow:

Bad – Blocking Search Engine Request

User-agent: *
Disallow: /

3. Use Google’s webmaster tool to diagnose your sites health: Apply for the Google’s webmaster tools and monitor your website. It shows current status or errors occurred during indexing or crawling your website. So you can easily find out where the problem is.

This is very important and you should be aware if you are starting a new website for internet marketing otherwise your website won’t be getting any traffic from search engines.