Comments on: PHP Parallel Web Scraper http://blog.perplexedlabs.com/2008/12/17/php-parallel-web-scraper/ web development war stories from the frontlines to the backend Thu, 11 Mar 2010 23:47:02 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Matt http://blog.perplexedlabs.com/2008/12/17/php-parallel-web-scraper/comment-page-1/#comment-417 Matt Thu, 18 Dec 2008 19:22:47 +0000 http://www.perplexedlabs.com/?p=90#comment-417 Kyle, I'm not sure of the internal implementation of cURL, I'd have to investigate further, but from my testing it absolutely does retrieve URLs residing on the same server in parallel. Thanks! Kyle,

I’m not sure of the internal implementation of cURL, I’d have to investigate further, but from my testing it absolutely does retrieve URLs residing on the same server in parallel.

Thanks!

]]> By: KyleC http://blog.perplexedlabs.com/2008/12/17/php-parallel-web-scraper/comment-page-1/#comment-416 KyleC Thu, 18 Dec 2008 19:17:36 +0000 http://www.perplexedlabs.com/?p=90#comment-416 Does this support HTTP pipelining? I've been looking for a way to efficiently retrieve several pages from the same sever simultaneously, and from what I've read pipelining is the best way to go; I just don't know if it's possible in PHP. Does this support HTTP pipelining? I’ve been looking for a way to efficiently retrieve several pages from the same sever simultaneously, and from what I’ve read pipelining is the best way to go; I just don’t know if it’s possible in PHP.

]]>