Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

mariowanders

Bathwater Drinker
Mar 11, 2022
84
3,717
1,242
0fya082315al84db03fa9bf467e3.png
How do I reverse the download order of a coomer page? Instead of downloading from newest to oldest, start from oldest to most recently posted.
 
  • Brainlet
Reactions: HBKTheIcon69

Eldudearrino

Superfan
Mar 11, 2022
33
620
829
0fya082315al84db03fa9bf467e3.png
is there a way to prevent the tool from trying to download dead links each time? in the log i can see it has a bunch of old gofile, etc links that are dead. obviously when its successful, then its in the history and it doesn't try to download it again. but these will just continue to fail each time.

i get you don't want to just try once and give up, as the links could just be unavailable. but is there some type of black list of urls that you can add urls to from the log so it doesn't keep attempting them. i looked around in the docs but don't see anything in the ignore settings, as i don't want to ignore the whole host. thanks, great tool.
 

Hiii46

Tier 3 Sub
Mar 18, 2022
16
321
534
0fya082315al84db03fa9bf467e3.png
Bunkr's only downloading one file at a time, How can i fix it ?
 
Last edited:

mikaka

Tier 1 Sub
Apr 5, 2023
17
102
284
0fya082315al84db03fa9bf467e3.png
Can you please support the system for nekohouse? I have already issued a ticket on Github.
 

ausugnd

Dark Lord of the Simps
Aug 28, 2022
250
3,966
1,312
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Catch is that Bunkr and GoFile sometimes add random strings to the end of the filenames, so it doesn't always work. Believe me, I've purged many terabytes of duplicate content.
I use a program called czkawka and run it on my folders. It's a good duplicate file checker that uses both a faster hash based approach, but also has a slower "Similar Videos" process that can help identify duplicates even if the file size, resolution, and bitrate are different.
(Not posting link but if you Google it, you'll find it)
Every few weeks/months, or if I know I ripped something with duplicates I run it on that particular folder to clean it up.
Trying to build a duplicate checker that checks more than just the filename would be a massive challenge.
 

ausugnd

Dark Lord of the Simps
Aug 28, 2022
250
3,966
1,312
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Manually checking the log files and then updating your URL list.
Specifically, the ones in the setup under unsupported_urls_filename, download_error_urls_filename and scrape_error_urls_filename and becoming familiar with the responses will help determine which links to keep and try again (now or later) or remove as dead links.

Jules has built in and also mapped a number of errors so it is a bit easier to read. Some examples I could find:
  • 451 - HTTP status code 451: Unavailable For Legal Reasons Pixeldrain file, gone. Don't bother retrying
  • 404 - HTTP status code 404: Not Found Normal internet speak - Link broken. Could be pasted wrong, or could have been removed.
  • Something like this came up for a GoFile link, so I checked the link in a browser and just noticed that they had rate-limited that file: ClientConnectorError - ClientConnectorError(ConnectionKey(host='store1.gofile.io', port=443, is_ssl=True, ssl=<ssl.SSLContext object at 0x7ff990921640>, proxy=URL(''), proxy_auth=None, proxy_headers_hash=8814436516137939874), ConnectionRefusedError(111, "Connect call failed ('0.0.0.0', 443)"))
  • I recently got a scrape error on
    Please, Log in or Register to see links and images
    saying No File Extension, So I checked and saw that the file had been named wrong without the .mp4 extension (or any extension) so the scraper didn't like it too much
  • Can't remember the specific wordings, but I also know that when Bunkr has issues with unavailable files, that is also mapped so you know that it's a case of "wait and try again later"

It does show somewhat of a summary in the top right of the UI, but the specialised URL log files have the details.
 

ausugnd

Dark Lord of the Simps
Aug 28, 2022
250
3,966
1,312
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Look... Some of us don't read the updates and don't see new features you keep adding and treat it like it's still v3/4 or whatever! 🤦‍♂️
Eldudearrino, do that first to stop it downloading any files with the same filename. After that, finding duplicates is harder and easier done post download.
 

EvilFilthy

Casual
Apr 25, 2023
3
31
325
0fya082315al84db03fa9bf467e3.png
I have no idea how to get python 3.11 to work on linux. I have a fresh install of KDE Neon Plasma 6 and it runs 3.10 and I have no idea how to change it. I have broken my POP_os my using sudo pip python3.11 or something similar. I got it to work perfectly on windows 11 but I hate windows. What distro of linux should I use? What distro is the easiest to get working? Also I am new to Linux and have been using AI chatbots in helping figuring things out. Thank you for this amazing software I had been able to use v5.0.124 and can manage to get that running but only with sudo.
 

amara69

Tier 2 Sub
Nov 5, 2023
9
203
433
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
Pop!_OS is a great linux distro especially for newcomers.

My suggestion is to use pyenv, which will allow you to manage multiple python versions easily. Check here:
Please, Log in or Register to see links and images

Seeing as you're using AI chatbots, ChatGPT will easily guide you through installing pyenv on Pop!_OS and there for getting 3.11 python.
 
  • Like
Reactions: EvilFilthy