Submit Express Newsletter #34
October 30, 2000
=======================================================================
TABLE OF CONTENTS
1- Intro
2- Search Engine Robots Explained
3- Search Engine News: AltaVista, GoTo, LookSmart
=======================================================================
Build a FLASH presentation yourself online in MINUTES!
- Build HIGH IMPACT multimedia online in a few minutes
- Perfect for flash sites, CD business cards, and all multimedia
- Complete w/ stereo sound, animation, and graphics
- Pay only $199 instead of thousands anywhere else
Visit us at http://WWW.FLASHZING.COM and see for yourself!
=======================================================================
WebContactPro - Online marketing software suite with integrated
shopping cart, unlimited autoresponders, list manager, form processor,
database marketing, ad tracking, affiliate tracking, electronic coupon
system and more. Risk Free Trial!
http://www.webcontactpro.net/
=======================================================================
1- Intro
In this issue we will explain what search engine robots or spiders are
and how they work. We will also explain how you can block them from
indexing pages on your site.
We have just signed a partnership with LookSmart to offer LookSmart
Express submission service to our visitors. As you may know, LookSmart
powers MSN search, AltaVista Directory, Excite Directory and many other
search engines, so it is very important to have your site listed in the
LookSmart Directory. LookSmart just recently discontinued its free
submission service and charges $199.00 for a listing within 2 days. You
can order LookSmart submission from:
http://submit.looksmart.com/info.jhtml?synd=zbs&chan=subexpr
Feel free to forward this email to all your friends that have websites
who may benefit from this information.
Subscription is completely free and can be obtained at:
http://www.submitexpress.com/newsletters.html
Thanks,
Submit Express
=======================================================================
Interested in generating more income from your site?
- Put AltaVista Search box on your site and earn 6 cents a search!
- Place Barnes & Noble banner on your site and earn 7% from the sales!
- Earn $25 - $50 per customer by telling others about Stamps.com!
- Over 600 merchants to choose from.
Join now at:
http://www.submitexpress.com/ads/befree.html
=======================================================================
2- Search Engine Robots Explained
When you submit your site to the search engines, the search engine
server - commonly referred to as a "robot" or a "spider" - will send a
request to your website either immediately or within a few weeks to
index your website content. This indexing of the website is also
commonly known as "spidering". Therefore, after you submit your site
to search engines, a search engine robot will spider your site.
The indexed data will get stored in a database and the data will be
displayed when searches are made on the search engines. Most robots
will only index text or HTML based pages. Robots cannot read text in
graphics. Therefore, it is very important to have some text on your
page, which includes your keywords, if you want your page to be found
in the search engines.
Some robots will not index pages based on certain criteria. For example
dynamically generated pages or database driven pages may not get
spidered.
For those that have their own servers or access to log files on their
site statistics, you can easily detect if a robot has indexed your
site. The search engine robots have unique IP addresses that can be
detected. SpiderHunter.com provides 360 known spiders on the page
below:
http://www.spiderhunter.com/spiderlist/
If you have trouble getting listed with the search engines, it is a
good idea to check your log files for the spider IP addresses to see
when they index you. When the spiders index you, it doesn't necessarily
mean that they will list you right away. It may take a few days to a
few weeks for you to show up in the search engines. However, if the
spiders have visited your site and your site does not show up within a
month, there may be a problem with your site that is causing it to not
be listed. You may want to have a professional search engine expert
look at your site. You can also order our 2500+ Submission service and
we will review your site before submission for any such problems and
provide ranking advice to improve your rankings. For details please
see:
http://www.submitexpress.com/submit2500.html
By keeping track of spider visit dates, you will get an idea how often
search engines index your pages. If you do not have log files, you can
take a look at the page below, which tracks spider visits for
spiderhunter.com website:
http://www.spiderhunter.com/spiders/
You can also block certain pages from being indexed by spiders. For
example, you may not want under construction pages to be indexed by
search engines, or you may want everyone to land only on your main page
from search engines. Although we do not recommend this, you should
submit every single page of your website to search engines to receive
more traffic. There are 2 methods that most search engines support, if
you want your pages to be blocked.
The first method to block pages from spiders is to place a file called
robots.txt in the root directory of the server. If you do not have
your own domain name, you may have to request your ISP to do this for
you.
To exclude your site from all spiders, in the robots.txt put:
User-agent: * # directed to all robots
Disallow: /
To exclude just a certain search engine, you will need to know the
spider name. You can contact the search engines to find this out or you
may find it on their FAQ pages. For example, AltaVista spider is known
as "Scooter" and your file should read:
User-agent: scooter # AltaVista web page search
Disallow: /
To limit the exclusion to a particular directory or file, put that
address after Disallow:
Disallow: /links/personal/
You can find out more about robot exclusions at:
http://info.webcrawler.com/mak/projects/robots/robots.html
The second method to block pages from being indexed by spiders is by
using Meta Tags. This is an easier method to use and also can be used
by everybody, even if you do not have access to the root directory of
your server.
Here is what you will have to put on the Meta Tags of the page which
you want to be blocked from spiders:
<Meta name="robots" content="noindex, nofollow">
This will have to be placed in your Meta Tags which is between the
<HEAD> and </HEAD> command via text editor.
You can also use our Meta Tags Generator on the page below to easily
generate this code for your page:
http://www.submitexpress.com/metatag.html
One last thing is that not all search engines may support the robots
exclusion methods above. You may have to ask each individual search
engine or visit their FAQ pages to find this out. Most major search
engines do support it, but the best method to keep search engines out
may be to create password protected areas.
=======================================================================
Advertising space in this newsletter is now available. The next issue
will go out to over 73,000 webmasters or website owners. See details
at:
http://www.submitexpress.com/newsletter_advertising.html
=======================================================================
3- Search Engine News
AltaVista:
AltaVista has expanded each search engine to Spain and Ireland. Spanish
version can be seen at: http://es.altavista.com/. Irish version can be
seen at: http://www.altavista.ie. Please see details at:
http://doc.altavista.com/company_info/press/pr102700.html
and
http://doc.altavista.com/company_info/press/pr101900.html
GoTo:
GoTo Announces Acquisition of SearchUP Inc. SearchUp.com is an internet
directory. GoTo is planning to use the technology and system from
SearchUp.com to enhance GoTo's advertiser tools. Please see details
at:
http://biz.yahoo.com/bw/001026/ca_goto_se.html
GoTo announces a name change. GoTo has decided to change its name to
distinguish itself from many other sites using similar names such as
Go.com and Go2Net.com. The new name has not been decided yet. GoTo has
hired NameLab, which came up with names such as Compaq and Acura, to
assist in the development process. Please see details at:
http://biz.yahoo.com/bw/001026/ca_goto.html
Freeserve, U.K.'s biggest ISP, selects GoTo as its default search
provider. This will increase traffic to GoTo.com search results.
Please see details at:
http://biz.yahoo.com/bw/001018/ca_goto_co.html
LookSmart:
LookSmart has signed several deals to offer its subsite listings on
several other sites. These include DealTime, VerticalNet, LineOne, and
Ebay. You can see press releases about this at:
http://www.looksmart.com/aboutus/press/index.html
If you are not listed in LookSmart, now is the time to get listed as
LookSmart is expanding its reach to many more visitors. You can get
listed at:
http://submit.looksmart.com/info.jhtml?synd=zbs&chan=subexpr
=======================================================================
Is your site anywhere to be found in the search engines? Do you need to
be in the top 10 of search engines, but don't know how? Are you tired
of trying everything you possibly can and still are nowhere to be
found?
If you need help getting your site in the top 10 of major search
engines, please check out our search engine optimization services:
http://www.submitexpress.com/optimize.html
We guarantee results or your money back.
We can also give you free advice when you order our 2500+ Search Engine
Submission. We will visit your site and send you a list of suggestions
on how to improve your rankings. See details at:
http://www.submitexpress.com/submit2500.html
Don't just submit your site to the search engines, SUBMIT IT RIGHT.
=======================================================================
Would you like to be informed about the latest affiliate programs?
Subscribe to our Affiliate Today weekly newsletter. Every week we will
send you information about the highest paying affiliate programs out
there. Join now at:
http://www.submitexpress.com/affiliate-today.html
=======================================================================
Submit Express
Website: http://www.submitexpress.com
Past Issues of this Newsletter:
http://www.submitexpress.com/newsletters/
Submit your website to 40 top search engines for FREE!
=======================================================================
Copyright 2000 Submit Express - No part of this newsletter may be
copied or published without prior permission, but please feel free to
forward this to all your friends as long as you keep it in its
entirety.
