NetSubmit - Search Engine Optimization

Search Engine Optimisation (also known as Search Engine Optimization or SEO) is a highly effective marketing tool. SEO works around the clock to increase appropriate web traffic, and therefore sales, via the natural search results of search engine giants Google, Yahoo! and MSN. More than three quarters of people using the internet use these search engines during their online session – mainly to find information, products, or services.

Preventing crawling
To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots. When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed, and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.
Some more stuff

A variety of other methods are employed to get a webpage indexed and shown higher in the results and often a combination of these methods are used as part of a search engine optimization campaign.

* Cross linking between pages of the same website. Giving more links to main pages of the website, to increase PageRank used by search engines.[5][6] Linking from other websites, including link farming and comment spam.
* Keyword rich text in the webpage and key phrases, so as to match all search queries.[7] Adding relevant keywords to a web page meta tags, including keyword stuffing.
* URL normalization for webpages with multiple urls, using "canonical" meta tag.[8]
* A backlink from a Web directory.
* SEO Trending based on recent search behaviour using tools like Google Insights for Search.
* Media Content creation like press releases and online news letters to generate an amount of incoming links