How do I reverse the download order of a coomer page? Instead of downloading from newest to oldest, start from oldest to most recently posted.
Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
is there a way to prevent the tool from trying to download dead links each time? in the log i can see it has a bunch of old gofile, etc links that are dead. obviously when its successful, then its in the history and it doesn't try to download it again. but these will just continue to fail each time.
i get you don't want to just try once and give up, as the links could just be unavailable. but is there some type of black list of urls that you can add urls to from the log so it doesn't keep attempting them. i looked around in the docs but don't see anything in the ignore settings, as i don't want to ignore the whole host. thanks, great tool.
i get you don't want to just try once and give up, as the links could just be unavailable. but is there some type of black list of urls that you can add urls to from the log so it doesn't keep attempting them. i looked around in the docs but don't see anything in the ignore settings, as i don't want to ignore the whole host. thanks, great tool.
Have you tried
skip_download_mark_complete?
you probably are better off leaving it alone so you dont get rate limited/banned by bunkr
How do you access these copies the url still can’t sorry I’m dumb
Open a github ticket for it, I can't do anything about it at the moment.
Catch is that Bunkr and GoFile sometimes add random strings to the end of the filenames, so it doesn't always work. Believe me, I've purged many terabytes of duplicate content.
I use a program called czkawka and run it on my folders. It's a good duplicate file checker that uses both a faster hash based approach, but also has a slower "Similar Videos" process that can help identify duplicates even if the file size, resolution, and bitrate are different.
(Not posting link but if you Google it, you'll find it)
Every few weeks/months, or if I know I ripped something with duplicates I run it on that particular folder to clean it up.
Trying to build a duplicate checker that checks more than just the filename would be a massive challenge.
Manually checking the log files and then updating your URL list.
Specifically, the ones in the setup under
unsupported_urls_filename, download_error_urls_filename and scrape_error_urls_filename and becoming familiar with the responses will help determine which links to keep and try again (now or later) or remove as dead links.Jules has built in and also mapped a number of errors so it is a bit easier to read. Some examples I could find:
451 - HTTP status code 451: Unavailable For Legal ReasonsPixeldrain file, gone. Don't bother retrying404 - HTTP status code 404: Not FoundNormal internet speak - Link broken. Could be pasted wrong, or could have been removed.- Something like this came up for a GoFile link, so I checked the link in a browser and just noticed that they had rate-limited that file:
ClientConnectorError - ClientConnectorError(ConnectionKey(host='store1.gofile.io', port=443, is_ssl=True, ssl=<ssl.SSLContext object at 0x7ff990921640>, proxy=URL(''), proxy_auth=None, proxy_headers_hash=8814436516137939874), ConnectionRefusedError(111, "Connect call failed ('0.0.0.0', 443)")) - I recently got a scrape error on
saying
No File Extension, So I checked and saw that the file had been named wrong without the .mp4 extension (or any extension) so the scraper didn't like it too much - Can't remember the specific wordings, but I also know that when Bunkr has issues with unavailable files, that is also mapped so you know that it's a case of "wait and try again later"
It does show somewhat of a summary in the top right of the UI, but the specialised URL log files have the details.
remove_generated_id_from_filenames in the config. Works for most sites.
Look... Some of us don't read the updates and don't see new features you keep adding and treat it like it's still v3/4 or whatever! 🤦‍♂️
Eldudearrino, do that first to stop it downloading any files with the same filename. After that, finding duplicates is harder and easier done post download.
I have no idea how to get python 3.11 to work on linux. I have a fresh install of KDE Neon Plasma 6 and it runs 3.10 and I have no idea how to change it. I have broken my POP_os my using sudo pip python3.11 or something similar. I got it to work perfectly on windows 11 but I hate windows. What distro of linux should I use? What distro is the easiest to get working? Also I am new to Linux and have been using AI chatbots in helping figuring things out. Thank you for this amazing software I had been able to use v5.0.124 and can manage to get that running but only with sudo.
You can look into pyenv to install different Python versions.
Pop!_OS is a great linux distro especially for newcomers.
My suggestion is to use pyenv, which will allow you to manage multiple python versions easily. Check here:
Seeing as you're using AI chatbots, ChatGPT will easily guide you through installing pyenv on Pop!_OS and there for getting 3.11 python.