This one's a biggie... possible 'crawling sandbox'?

[Blazingpie]

http://www.wolf-howl.com/seo/superficial-crawling-seo-strategies/

However I’m not looking to stir up yet another pointless is there/is there not sandbox debate. What I am here to say is I think we are hitting the leading edge of a new force to be addressed with, which for lack of a better term I’m calling ’sandbox crawling’. Here’s the way I see it, if your website is missing the right ‘quality indicators’ what you’ll start to see is superficial crawling and indexing of your website. Your site which may have had hundreds, thousands or even hundreds of thousands of pages will just not be as well represented in Google’s index as you would like it to be.

If what the signs indicate is true then this'll change the way SEO is done completely. Hmmmm...

[Paz]

Wow,

if this is true then it would be one of the biggest shocks in recent SEO history.

I can't imagine an algorithm that doesn't deep crawl new sites or sites with low PR. There are so many negative implications.

Imagine a great new resource site with lots of great categorized content, but deep from the front page. How can it ever pick up natural links if the content can't be found because it's not crawled and indexed.

If this "crawling sandbox" is real:

There will be a bigger market than ever for paid, high PR links.
Webmaster will have to redesign their sites, avoiding product categorisation so that they have fewer deep pages.
Directory links will disappear as backlinks because their sub categories will disappear from Google's index.
... and I'm sure there are more.

Directory owners should take note though. The "general" directory will disappear soon in my view. If a directory has the structure:
Business > Finance > Credit > etc
Travel > Accommodation > Hotels > etc

then they will have far less of a chance in Google.

Instead they will have to create 10 new niche directories (one directory for business, another for travel, etc.) to reduce the amount of deep crawling necessary to get the sub-categories indexed reliably in Google...

The big thing for me though is that paid, high PR links will become an essential part of any SEO campaign, not for ranking but for crawling. Google certainly won't be happy about that!

Cheers,
Paz.