A lot of times, the search engine results are one or two steps away from the actual docs. If we can somehow combine web crawling aspects with the web search results, we may be able to get a lot of bang for our buck. the web crawling for each link could be quite limited - a depth of 2 or 3 would be more than plenty. We would probably have to have some url validation checks though - no point of crawling a site that doesn't pertain to the jurisdiction we are interested in.
A lot of times, the search engine results are one or two steps away from the actual docs. If we can somehow combine web crawling aspects with the web search results, we may be able to get a lot of bang for our buck. the web crawling for each link could be quite limited - a depth of 2 or 3 would be more than plenty. We would probably have to have some url validation checks though - no point of crawling a site that doesn't pertain to the jurisdiction we are interested in.