PDA

View Full Version : spiders and asp response.redirect vs. server.redirect


charlesincharge
02-25-2005, 11:56 AM
Greetings. I'm new here and hoping someone can help me with a quick question.

I have a retail site that ended up at it's current domain name after we were sorting through several possible domain names. All of the other names we bought are pointed at the same server. So, just to be clean about it I inspect SERVER_NAME and do a response.redirect to the final domain name if anyone arrives at anything other than 'domainname.com' (including 'alternatename1.com' or 'alternatename2.com' or even 'www.domainname.com' (stripping off the www.)

I've noticed that since googlebot started hitting my site it stops at the first page. Is it possble that it's hitting another URL and never proceeding past my client-side redirect? What's the best way to handle a situation like this where I'd like everyone to be redirected to the same starting URL as they hit the home page?

Should I write a useragent exclusion into the redirect code for googlebot and other spiders? Should I just take that code out altogether and rely on relative links?

What is the impact of server.redirect vs. response.redirect (for the asp guys out there) on spiders? Am I going to accidentally going to get myself banned?

Many thanks,

Charles

eadpro
02-25-2005, 12:16 PM
I don't work with ASP so I'm not sure.

My guess is that Google's "deep crawlers" haven't hit your site yet.

There are different googlebots, and they exhibit different behaviors. Sometimes it takes several weeks for a new site to get deep crawled (beyond just the homepage).

How new is your site? Do you have some potent backlinks pointing to it?

I don't think the redirects will get you banned if they are at the server level (don't quote me on that though!).

Post your url so that we can take a look at it.

charlesincharge
02-25-2005, 12:33 PM
The site IS relatively new...first googlebot traffic appearing in logs only eight days ago.

I've seen both (truncated here)
- Googlebot/2.1 (+http://www.google.com/bo and
- Mozilla/5.0 (compatible; Googlebot/2.1;
since then...but again, only on the home page.

As far as backlinks go, I'm working on it but they are limited so far.

I'm including the site name and code snippet in an attached text file in hopes that it won't get parsed into this post...

Thanks again,

Charles

joker
02-26-2005, 06:10 AM
Thinks that the reason of this is that your site is rather new. You should wait a bit till it will crawl and index internal pages