Welcome to the Web - Rocketface(R) Workshop Webmaster's Tutorial - "How to Design a Website"

Design A Spider-Friendly Site

by Larisa Thomason

What kind of attention span does a search engine spider have? It depends. Some have the patience to dig deep into your site and promise to follow every link. Others may get bored before they finish indexing a whole page. Carefully organize your page content and Web site to ensure that spiders index the whole thing - not just bits and pieces.

Make Your Points Early And Well

Human visitors want to see your most important content at the top of the page, highlighted if necessary with color and header tags. But realize that spiders see the page differently than human visitors.

Search engine spiders look at your content and assume that:

Earlier content is more important than what's farther down on the page.

Text contained in header tags is more important than other text.

Links and header tags that contain keywords matching the META tag keywords are more important than those that don't.

These used to be a very important component of all search engine algorithms, but lately has been overshadowed by other ranking criteria like link popularity and themes. Don't ignore them though: they still affect your rank on many engines and help boost you above competing sites.Put Your Content On Top

Even if visitors see your content on top of the page in their browsers, spiders might not find it if your HTML code is poorly written. If so, your best keywords and keyword phrases may be the last thing the spider reads.

Some design techniques that work great for human visitors leave spiders out in the cold. Make them more spider-friendly like this:

External JavaScript files: Complex JavaScript code inserted in the document's HEAD tag takes up a huge amount of space. Use an external JavaScript file to reduce page size or place your JavaScript code at the bottom of the page whenever possible.

Cascading Style Sheets: Use style sheets to reduce bloated code and decrease download time. Style sheets eliminate extraneous HTML code, so the spider can get straight to your content.

Tables: Keep your table structure as simple as possible. Some spiders tend to get lost in a web of nested tables where the content is buried deep inside a complex table structure.

Be particularly careful if you're using a WYSIWYG editor:

Bloated HTML code: This is a problem with many WYSIWYG editors. It's easy to forget what's happening to your HTML code when you're cutting and pasting text and images without writing any actual code yourself.

Microsoft Office users often don't think they need an HTML editor at all: they can select the Save as Web option on their File menu and convert it into an HTML document. But all those files are big (really big!) and almost impossible to hand edit later.

Editor-generated JavaScript: Many editors let you include basic JavaScript effects like drop-down menus and rollovers automatically. While easy, this method also writes big code. A simple rollover event that takes a few lines to hand code might be 10 times that size if it's editor-generated.

Once all visitors (human and spider) see your content up front, it's time to consider the overall structure of your site, not just individual page layout and coding.

Flatter Is Better

Try to design your site with as few levels as possible. Too many sites have this sort of structure:

Level 1: Splash page with a "click here to enter" link.
Level 2: Home page with little content other than navigation links.
Level 3: Products page whose only content is a list of product categories. Visitors click on the category that interests them.
Level 4: Specific product categories with keyword-rich descriptions of each product.

Visitors have to wait for 3 or more pages to load before ever finding the information they want, while search engine spiders search through multiple pages before finding good content to index.

A splash page adds an extra level to your Web site, pushing all of your other content down one level. This increases the chance that both humans and spiders will miss it.

Keep your home page short, but make it content-rich. Reference major topics and include important keywords, then link them to keyword-rich 2nd and 3rd-level pages for more information. A simple "Products" link in your navigation menu doesn't help either human or spider visitors. Also include a "Products" paragraph that briefly lists the type of items you sell:

"We offer the Web's largest selection of
premium cat food, dog food, dog
grooming supplies, and pet toys!"

Make sure that your keyword phrases "premium cat food, pet toys, etc." all link to the internal site pages that discuss them more completely. This gives spiders a direct link to your important content and it can help your overall page rank because some ranking algorithms place more weight on keywords contained in links.

Deep Submit Your Pages

But do you really need to worry about how many site levels you have? Most of the major search engines now boast that they do a "deep crawl" on all Web sites they visit and therefore index the entire site.

That's not always the case however. Most crawl down a few levels deep into your site and then they're off to the next site.

Search engines claim to index hundreds of millions, even billions of pages. With numbers that size, it's likely that they're going miss a few important pages here and there.

Probably when you started promoting your Web site, you submitted your home page to the top search engines and waited for the spiders to visit and crawl your site.

That may not be good enough. If you use a technique called deep submit, you don't just hope the spiders find your internal pages; you actually submit those too!

This is where an automated search engine submission tool becomes essential. You can avoid time-consuming typing and typographical errors with NetMechanic's Search Engine Starter, part of the Search Engine Power Pack optimization tool. Use it to submit any page in your site to your choice of 100 search engines.

Don't risk being overlooked. Design a site that appeals to both humans and search engine spiders, then use deep submit to achieve good search engine exposure for your whole site, not just a page or two.

About the Author

Larisa Thomason, is the Senior Web Analyst at NetMechanic, Inc.(http://www.netmechanic.com)

Search Engine Optimization (SEO) and Website Optimization
Optimizing Your Website	It's no good having a website if no one can find it. That's the bottom line. But the good news is that there is plenty that you can do to increase your chances of the right people finding you.
Landing Pages - What You Need to Explain to your Website Design Company	The definition of a landing page is simply "the first page that visitors hit on your site," so it is not strictly a certain page but any page that a user "lands" on. All websites have landing pages, whether they like it or not, even if it was not specifically designed as such.
6 Website Redesign SEO Secrets Your Developer May Not Know	At the end of the year, many businesses start to think about redesigning their tired old website to breathe some new life into it. If so, the first thing is to make sure you hire a design and development company that knows how to build the infrastructure of the website in a search engine crawler–friendly manner.
Do-It-Yourself SEO - A Beginner's Checklist	It is becoming more important for webmasters to have an internal approach and awareness of search engine marketing.
For SEO Beginners - Twelve Definitions You Need To Know	This líst of twelve SEO-related definitions in alphabetical order serves as a great companion for your initial SEO reading.
Most Common SEO Mistakes	Here are the most common mistakes people make which keep their sites from being ranked well.
Keywords Are It! Seven Powerful and Useful Keyword Marketing Tools	After years of running websites and earning a full-time online income, I am constantly reminded that it all comes down to keywords.
How to Begin Keyword Research	Keyword research starts with the understanding that finding lots of related keywords that deliver targeted traffic is the ultimate goal of any keyword research.
Writing And Optimizing Your Website's Content	Content is the messages and information that accurately and significantly describe your business or purpose of your web site.
Link Popularity	Search engines are the gateway to the Internet; they are the first tool that potential customers use to find the products and services they need. This is why link popularity is so imperative.
Link Building Basics - 5 Easy Tips	Today building a website is so easy even a child could do it. The difficult part however is to get targeted traffic to your newly created website.
Content and Optimization	The content of your individual webpages determines how your visitors and the search engines se your website.
The Top 10 Tools for Monitoring the Success of Your Website	By monitoring your website you will be able to see how it is performing and how it can be improved.
The Search Engine Optimization Process - Beginners Guide

Design A Spider-Friendly Site

by Larisa Thomason

Search Engine Optimization (SEO) and Website Optimization