Congratulations! Your email's valid.

Download supplemental results detector:

$147.99 now just $49.99 - no crawl limit

$19.97 - Crawl limit: up to 200 pages crawled per domain

Having problems? email me at [email protected]

Features

• Parses robots.txt.
• Follows 301 redirects to pass juice to the right URL.
• Obeys META NOINDEx,NOFOLLOW
• Saves information in MYSQL.
• Handles relative URLs (e.g. .../../home.html)
• Uses multiple threads for faster crawling
• Tested sites up to 14,000 pages.

Additional Features

• CSV Export - A domain's url data can be exported to a CSV file, so you can play with the data in Excel.
• Crawl a domain from any entry point, not just the root URL. Useful if you don't want to start crawl from root or if you want to test internal link strategies in a subdirectory sandbox.
• Simulates dangling pages (dead end pages with no outbounds) by redistributing their PageRank to the rest of the web.
• Robots.txt disallowed URLs accumulate PageRank, though since links on the pages can't be crawled, these pages are treated as dangling pages.
• META noindexed URLs accumulates PageRank if links point at them.
• META nofollow creates dangling pages. So a META noindex,nofollow page will accumulate PageRank and then bleed that PageRank to the rest of the web instead of circulating it back into other pages of a website.
• REL=nofollow is treated like an outbound link. Previously, nofollows were thought to push their PageRanks to followed links on the same page, but now Google states that is no longer the case.
• TBPR scraping - The tool can be used to scrape TBPR. While the TBPRs are inaccurate, they can be used to give you a sense of the health of a domain. For example, a well-linked-to domain will have a larger number of pages with some green than a poorly-linked-to domain. The total TBPRs combined also can be used to gauge depth of index penetration. TBPR of a specific page can also be a hint to its supplemental status.
• Google Cache date scraping - this tool can be used to scrape Google cache dates en masse using multiple proxies. While cache dates are inaccurate, they are decent metrics for deciding which page to optimize first, the overall link popularity of a domain, and can hint at a page's supplemental status. Since there's a limit to the number of pages Googlebot will crawl per day (depending on a site's total PageRank), you can gauge a site's link weight by counting the number of pages with cache dates fresher than 1 week. At the same time, crawl limit introduces inaccuracies because even if a site has 2000 TBPR 6 pages, if the site only gets crawled 20 pages/day, those pages will not all have the same cache dates. So in that sense, cache dates are not exactly the next TBPR. A site's update frequency also can influence cache freshness.
• Calculates PageRank using the original formula. PageRank calculations have evolved over the years, so using any variation of known PageRank formulas are only approximations. For example, links higher up on a page may carry more weight than links positioned way down on the bottom.

Instructions

1. Unzip it.
2. Find the distribution folder /dist/
3. In Windows (pre-Vista) add argument "-jar" to the .jar file type.
4. If you're on Vista, change .jar file association from java.exe to javaw.exe.
5. Double click on Supplemental_Results_Detector.jar to start the program.
6. If you're having problems or want to see details of what the program is doing, launch it in DOS, then email me any error messages you're seeing:
   1. Set the PATH environmental variable.
   2. Open DOS (start > run > cmd.exe)
   3. CD to the /dist/ directory. (e.g. "cd /my documents/..../dist/")
   4. type "java -jar supplemental_results_detector.jar