MEMORIAL DAY - JOIN BRAZZERS FOR FREE - CLICK HERE!
Tools - Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads | Page 98 | SimpCity Forums

Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

xylophix

Bathwater Drinker
Mar 11, 2022
117
4,652
1,252
0fya082315al84db03fa9bf467e3.png
In the config.yaml why is there no option to specify where we want to place the downloader.log and Unsupported_Urls.txt?

The logfile placement exist as a cli argument but not in the yaml, and the unsupported file doesn't seem to have any arguments at all.
 
Last edited:

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,197
5,159
1,127
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
The log file is initiated before the anything else in the program. At this point the config creation and management could be moved to before it, but I left it afterwards initially for debugging, and error testing.

The unsupported_urls file was something that just never really got a lot of attention, and I haven't bothered improving it from what it was.

I'll try to get to these things, if you could put in a request on the github though it would be helpful.
 
  • Like
Reactions: digital_warlord

treidon

Superfan
Jul 2, 2022
35
519
739
0fya082315al84db03fa9bf467e3.png
Apologies if this has been asked before, but is there a way to determine which particular link the downloader is working on at any given moment? Often there is a clue by the random text generation on a gofile link, but sometimes it just has the description of the download with no cross-reference in the text file that could be 200+ links. I ask because sometimes a particular link can timeout or get it very slow and its better to just remove it and move on.
 

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,197
5,159
1,127
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Not exactly the way you are looking for. But the description of the download is always the title used on the particular site it's downloading from. Less helpful, but you'll also see the "working on" print statements at the end of the downloader.log if you open it mid run it'll show which url it is currently working on. These'll be different that what you put into the urls.txt file though but will be a better indication than nothing.
 

theporntux

Casual
Oct 24, 2022
3
21
215
0fya082315al84db03fa9bf467e3.png
I have an urging to crawl the entire forum's RSS feed and download everything it sees. Probably only 10-30TB I'm estimating?

If I added a polling RSS parser mode, would you want to merge it?
 

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,197
5,159
1,127
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
I have a feeling it's going to be a lot more than you are estimating. My second thought is I don't think the downloader will be able to keep up with everything being posted.

That being said, unfortunately it's not something I probably would merge
 

syvius

Lurker
Dec 2, 2022
1
0
49
0fya082315al84db03fa9bf467e3.png
I'm genuinely not sure what I'm doing wrong, but I can scrape and download most other websites except for cyberdrop.me

It just scrapes, shows the files it discovered, and sits at 0%. I see the download history file doing something in the folder as if it's trying to write, but other than that nothing happens.

Any help would be appreciated.
 

ausugnd

Dark Lord of the Simps
Aug 28, 2022
249
3,901
1,312
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Given that I have seen 50GB+ Bunkr Links on here, and that Bunkr run multiple servers to support the content quantity, it is going to be much much more. What is 'Visible' on the forum might be low, but scraping all the links will very quickly add up.
Also that many hits on the forum or a single website from the app via API or whatever, is going to lock you/your account/your IP out of them.
 
  • Like
Reactions: pvtrich

ElGuapito99

Bathwater Drinker
Mar 12, 2022
49
1,807
1,159
0fya082315al84db03fa9bf467e3.png
Sorry if this was answered already, but is there a way to download from jpg.church if the the link is password protected? Or it's currently not supported?