how to avoid omiting pictures
Posted: Fri Jun 16, 2006 5:51 am
i am downloading a set of pictures from http://www.imagefap.com/gallery.php?gid=165588. But find that pictures that downloaded by picaloader are less than the pictures on the website. for example i want to download 22 pictures, but i find i have downloaded only 17 pictures. then the picaloader begin to download other pictures that were not what i wanted.
Then i watched the threads that picaloader was connecting in the monitor panel. i find that sometimes picaloader had found the real address of picture and then it say starting downloading the picture but soon picaloader is analyzing another url in the same thread.
maybe picaloader can not connect the picture at that time. i think maybe this is because picaloader connects the address of picture out of time, then picaloader hang up the downloading process and continue other threads.
the problem is that if you set "followed links" to be "all levels". picaloader would follow other links that we do not want and the pictures that hung up are in fact omit.
to avoid the problem, i suggest you set "followed links" to be "this page and n level". n usually equals 1, which depends the storage stucture of the website. you need the find out it.
since picaloader may request 7-8 connections to the addresses of images simultaneously. the website may not allow 7-8 connections at a time, so picaloader would hang up the download and continue other threads. you will find that pictures downloaded are less than pictures that you want. an important tip is you should wait for the picaloader to finish automatically. if you stop the task without download all the pictures, pictures having been analyzed but being hung up would never be downloaded because picaloader thinks that these pictures have been downloaded. this problem remains even if you delete the task and recreate it. you have to delete the whole project and recreate it. if you find the pictures are not enough, wait for picaloader finishing automatically and restart the task again until all the pictures are downloaded.
Then i watched the threads that picaloader was connecting in the monitor panel. i find that sometimes picaloader had found the real address of picture and then it say starting downloading the picture but soon picaloader is analyzing another url in the same thread.
maybe picaloader can not connect the picture at that time. i think maybe this is because picaloader connects the address of picture out of time, then picaloader hang up the downloading process and continue other threads.
the problem is that if you set "followed links" to be "all levels". picaloader would follow other links that we do not want and the pictures that hung up are in fact omit.
to avoid the problem, i suggest you set "followed links" to be "this page and n level". n usually equals 1, which depends the storage stucture of the website. you need the find out it.
since picaloader may request 7-8 connections to the addresses of images simultaneously. the website may not allow 7-8 connections at a time, so picaloader would hang up the download and continue other threads. you will find that pictures downloaded are less than pictures that you want. an important tip is you should wait for the picaloader to finish automatically. if you stop the task without download all the pictures, pictures having been analyzed but being hung up would never be downloaded because picaloader thinks that these pictures have been downloaded. this problem remains even if you delete the task and recreate it. you have to delete the whole project and recreate it. if you find the pictures are not enough, wait for picaloader finishing automatically and restart the task again until all the pictures are downloaded.