Yeah i'm using windows, now I opened start.bat and it said to download the newest and so I just downloaded 2.5.1 and it's telling me that there is a new version again. If you wish to take to private messages instead I won't mind
Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
maybe for those that have large number of links in their txt files, a print out of what link it's trying to process at the beginning? so we can temporarily delete those links or whatever
If I understand you correctly you want a readout of what is currently being scraped to remove links of those taking a long time. Unfortunately that's not possible. The scrapers all run side by side. I'm going to have a read out in about 5 minutes in 2.5.2 but just because a new line is posted as "scraping" doesn't mean that the last one is complete.
ah ok, not sure what the best solution is to having scrapes take too long then
i'll ask you if i get another idea that seems reasonably useful or easy to implement
thanks for considering it tho
I don't typically like releasing multiple times in quick succession. So apologies to everyone, I know it's annoying.
But try the 'post_scraping_readout' setting in settings.py. Change it to true and you'll see what I mean.
Edit: actually my most recent change might have done what you want.
yeah that looks about right in line with what i thought that might look like
but i think you mentioned that scraping was multi threaded, so yeah like you said it's not nearly as helpful as i thought it might be
it's just not easy to tell which link it's stuck on. it'll just print the latest one that's finished, and it'll still be doin stuff that i can't see into
second post cus i'm gonna talk about my own thing rather than generally
tbh for myself, i'm not even sure if it's scraping that's making it seem stuck, just thought i had a solution to the guy that said he got hung up on scraping. actually, you know what. i'm seeing something odd now. it's scraping multiple times, taking several minutes each time.
ok, removing the link, it gets past the scraping and onto the downloads
i did catch some links scraping multiple times again tho. looks like it reports "finished scraping" even though it hasn't and is going to retry it. to make the readout useful, i think that needs to be addressed. i hope it's easy
thanks again my dude
tbh for myself, i'm not even sure if it's scraping that's making it seem stuck, just thought i had a solution to the guy that said he got hung up on scraping. actually, you know what. i'm seeing something odd now. it's scraping multiple times, taking several minutes each time.
ok, removing the link, it gets past the scraping and onto the downloads
i did catch some links scraping multiple times again tho. looks like it reports "finished scraping" even though it hasn't and is going to retry it. to make the readout useful, i think that needs to be addressed. i hope it's easy
thanks again my dude
I'm not sure what you mean by this. A picture of it happening would be helpful. It scrapes each page separately if that's what you mean.
On android there is, but it's complicated. Also, this program will not run on a phone.
As I just said in my last comment. This program will not run on a phone. Unfortunately, I can't help you.
Is ther anyway to do multiple downloads using a phone? Or tablet? As I don't have access to a good pc/laptop
Portable version? If you are talking about on a mobile device (a phone), no.
The program downloads a driver that will open a hidden chrome window that loads the page and takes the links that it needs.
Is anyone having an issue with Bunkr not downloading files. Maybe it's on my end, I can't seem to play multiple videos on different tabs, just one tab will play. Any idea on what I could do?
Trying to run it on mac and keep getting this error:
11, in <module>
import scrapy
ModuleNotFoundError: No module named 'scrapy'
11, in <module>
import scrapy
ModuleNotFoundError: No module named 'scrapy'