alright ty that's how I did it but it still failed so I figured I might've messed up but I guess ddos guard is just fucked atm then
Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
what is the difference between the imported v4 config and the default config?
pixeldrain is due to pixeldrain, not CDL. give it a rest or switch up your IP
DDoS-Guard is blocking you, not CDL.
DDoS-Guard is blocking you, not CDL.
Am I missing an option in the sorted downloads? It used to just rename files with the same name and add the default windows extra.
For example file name is called 1 , if there is another file called one it would name that new one 1 (1) now it just replaces the old file with the new one?
For example file name is called 1 , if there is another file called one it would name that new one 1 (1) now it just replaces the old file with the new one?
Hi Jules,
After installing ffmpeg I started getting this error, do you know how can I fix it?
After installing ffmpeg I started getting this error, do you know how can I fix it?
| packages (from cyberdrop-dl) (1.1.7) An error occurred, please report this to the developerin c:\users\Electrocute\appdata\local\programs\python\python310\lib\siteunhandled errors in a TaskGroup (1 sub-exception) + Exception Group Traceback (most recent call last):11.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib | File "C:\Users\Electrocute\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\main.py", line 84, in directorent already satisfied: pyyaml<7.0.0,>=6.0.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-p | await runtime(manager).0.1) | File "C:\Users\Electrocute\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\main.py", line 44, in runtimeges (from cyberdrop-dl) (23.1.0) | async with taskgroup.TaskGroup() as task_group:c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-p | File "C:\Users\Electrocute\AppData\Local\Programs\Python\Python310\lib\site-packages\quattro\taskgroup.py", line 169, in __aexit__t already satisfied: typing_extensions>=3.7.2 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\si | raise me from Nonee==0.17.0->cyberdrop-dl) (4.7.1) | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)s\python\python310\lib\site-packages +-+---------------- 1 ----------------dl) (21.4.0) | Traceback (most recent call last):<7.0,>=4.5 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-pa | File "C:\Users\Electrocute\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\scraper\scraper.py", line 367, in map_urlatisfied: yarl<2.0,>=1.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-package | self.manager.task_group.create_task(scraper.run(scrape_item)) | File "C:\Users\Electrocute\AppData\Local\Programs\Python\Python310\lib\site-packages\quattro\taskgroup.py", line 175, in create_taskhttp<4.0.0,>=3.9.1->cyberdrop-dl) (1.3.0) | raise RuntimeError(f"TaskGroup {self!r} is awaiting in exit")ta\local\programs\python\python310\lib\site-packa | RuntimeError: TaskGroup <TaskGroup tasks=1> is awaiting in exit +------------------------------------out<5.0,>=4.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from aiohttp<4.0.0,>=3.9.1->cyberdrop-dl) (4.0.2) Requirement already satisfied: asyncprawcore<3,>=2.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from asyncpraw<8.0.0,>=7.7.1->cyberdrop-dl) (2.3.0) Requirement already satisfied: update-checker>=0.18 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from asyncpraw<8.0.0,>=7.7.1->cyberdrop-dl) (0.18.0) Requirement already satisfied: soupsieve>1.2 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from beautifulsoup4<5.0.0,>=4.12.2->cyberdrop-dl) (2.3.1) Requirement already satisfied: lz4 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from browser-cookie3<0.20.0,>=0.19.1->cyberdrop-dl) (4.3.2) Requirement already satisfied: pycryptodomex in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from browser-cookie3<0.20.0,>=0.19.1->cyberdrop-dl) (3.19.0) Requirement already satisfied: python-dateutil in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from filedate<3.0,>=2.0->cyberdrop-dl) (2.8.2) Requirement already satisfied: pfzy<0.4.0,>=0.3.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from inquirerpy<0.4.0,>=0.3.4->cyberdrop-dl) (0.3.4) Requirement already satisfied: prompt-toolkit<4.0.0,>=3.0.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from inquirerpy<0.4.0,>=0.3.4->cyberdrop-dl) (3.0.43) Requirement already satisfied: requests in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (2.27.1) Requirement already satisfied: requests-toolbelt in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (1.0.0) Requirement already satisfied: six in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (1.16.0) Requirement already satisfied: pycryptodome in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from myjdapi<2.0.0,>=1.1.7->cyberdrop-dl) (3.15.0) Requirement already satisfied: exceptiongroup in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from quattro<24.0.0,>=23.1.0->cyberdrop-dl) (1.2.0) Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from rich<14.0.0,>=13.7.0->cyberdrop-dl) (2.2.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from rich<14.0.0,>=13.7.0->cyberdrop-dl) (2.14.0) Requirement already satisfied: mdurl~=0.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from markdown-it-py>=2.2.0->rich<14.0.0,>=13.7.0->cyberdrop-dl) (0.1.2) Requirement already satisfied: wcwidth in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from prompt-toolkit<4.0.0,>=3.0.1->inquirerpy<0.4.0,>=0.3.4->cyberdrop-dl) (0.2.12) Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from requests->mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (1.26.8) Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from requests->mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (2.0.12) Requirement already satisfied: idna<4,>=2.5 in c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages (from requests->mediafire<0.7.0,>=0.6.1->cyberdrop-dl) (3.3) WARNING: Ignoring invalid distribution -yberdrop-dl (c:\users\Electrocute\appdata\local\programs\python\python310\lib\site-packages) E:\Downloads\Cyberdrop_DL.V4>cyberdrop-dl |
V4 was working for me until I was getting errors. V5.1 will attempt to scrape, but give me the "finished downloading" message.
I'm...a bit lost lol...
I'm...a bit lost lol...
I'd need more info than that. Like the downloader.log file in ./appdata/configs/<config>/logs
Sure, but before I do that, I think it may be related to bunker's DDOS-Guard. I tried extracting cookies from Firefox through the downloader's UI, but I'm not sure if that did anything because I see this in the downloader.log file:
DDOS-Guard": {
"Bunkrr Provided": false,
"Coomer Provided": false,
"Kemono Provided": false
Don't worry about that. It doesn't matter. If you're trying to scrape forums the section below that is what matters.
OK. Not gonna try anything until then. This only shows some URLs, but I had a bunch more in there.
I appreciate the support!
This was my last downloader.log:
2023-12-16 13:56:39,032:utilities.py:81:Starting Async Processes...
2023-12-16 13:56:39,032:utilities.py:81:Starting Cyberdrop-DL Process for Default Config
2023-12-16 13:56:39,032:utilities.py:81:Running version 5.0.123
2023-12-16 13:56:39,032:utilities.py:81:Using Authentication:
{
"DDOS-Guard": {
"Bunkrr Provided": false,
"Coomer Provided": false,
"Kemono Provided": false
},
"Forums Credentials": {
"celebforum Credentials Provided": false,
"f95zone Credentials Provided": false,
"leakedmodels Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"simpcity Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"shit forum Credentials Provided": false
},
"Forums XF Cookies": {
"celebforum XF Cookie Provided": false,
"f95zone XF Cookie Provided": false,
"leakedmodels XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"simpcity XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"shit forum XF Cookie Provided": false
},
"GoFile Provided": false,
"Imgur Provided": false,
"JDownloader Provided": false,
"PixelDrain Provided": false,
"Reddit Provided": false
}
2023-12-16 13:56:39,032:utilities.py:81:Using Settings:
{
"Download_Options": {
"block_download_sub_folders": false,
"disable_download_attempt_limit": false,
"disable_file_timestamps": false,
"include_album_id_in_folder_name": false,
"include_thread_id_in_folder_name": false,
"remove_domains_from_folder_names": false,
"remove_generated_id_from_filenames": false,
"scrape_single_forum_post": false,
"separate_posts": false,
"skip_download_mark_completed": false
},
"File_Size_Limits": {
"maximum_image_size": 0,
"maximum_other_size": 0,
"maximum_video_size": 0,
"minimum_image_size": 0,
"minimum_other_size": 0,
"minimum_video_size": 0
},
"Files": {
"download_folder": "Downloads\\Cyberdrop-DL Downloads",
"input_file": "AppData\\Configs\\Default\\URLs.txt"
},
"Ignore_Options": {
"exclude_audio": false,
"exclude_images": false,
"exclude_other": false,
"exclude_videos": false,
"ignore_coomer_ads": false,
"only_hosts": [],
"skip_hosts": []
},
"Logs": {
"download_error_urls_filename": "Download_Error_URLs.csv",
"last_forum_post_filename": "Last_Scraped_Forum_Posts.txt",
"log_folder": "AppData\\Configs\\Default\\Logs",
"main_log_filename": "downloader.log",
"scrape_error_urls_filename": "Scrape_Error_URLs.csv",
"unsupported_urls_filename": "Unsupported_URLs.txt"
},
"Runtime_Options": {
"delete_partial_files": false,
"ignore_history": false,
"send_unsupported_to_jdownloader": false,
"skip_check_for_empty_folders": false,
"skip_check_for_partial_files": false
},
"Sorting": {
"sort_downloads": false,
"sort_folder": "Downloads\\Cyberdrop-DL Sorted Downloads",
"sort_incremementer_format": " ({i})",
"sorted_audio": "{sort_dir}/{base_dir}/Audio/{filename}{ext}",
"sorted_image": "{sort_dir}/{base_dir}/Images/{filename}{ext}",
"sorted_other": "{sort_dir}/{base_dir}/Other/{filename}{ext}",
"sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}"
}
}
2023-12-16 13:56:39,032:utilities.py:81:Using Global Settings:
{
"General": {
"allow_insecure_connections": false,
"max_file_name_length": 95,
"max_folder_name_length": 60,
"proxy": "",
"required_free_space": 5,
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0"
},
"Rate_Limiting_Options": {
"connection_timeout": 15,
"download_attempts": 10,
"download_delay": 0,
"max_simultaneous_downloads": 15,
"max_simultaneous_downloads_per_domain": 5,
"rate_limit": 50,
"read_timeout": 300
}
}
2023-12-16 13:56:39,041:utilities.py:81:Starting UI...
2023-12-16 13:56:39,113:utilities.py:81:Scrape Starting:
2023-12-16 13:56:39,860:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:39,863:utilities.py:81:Scrape Finished:
2023-12-16 13:56:39,863:utilities.py:81:Scrape Starting:
2023-12-16 13:56:40,202:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:40,203:utilities.py:81:Scrape Finished:
2023-12-16 13:56:40,203:utilities.py:81:Scrape Starting:
2023-12-16 13:56:40,537:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:40,539:utilities.py:81:Scrape Finished:
2023-12-16 13:56:40,539:utilities.py:81:Scrape Starting:
2023-12-16 13:56:40,851:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:40,853:utilities.py:81:Scrape Finished:
2023-12-16 13:56:40,853:utilities.py:81:Scrape Starting:
2023-12-16 13:56:41,234:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:41,237:utilities.py:81:Scrape Finished:
2023-12-16 13:56:41,237:utilities.py:81:Scrape Starting:
2023-12-16 13:56:41,553:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:41,555:utilities.py:81:Scrape Finished:
2023-12-16 13:56:41,555:utilities.py:81:Scrape Starting:
2023-12-16 13:56:41,872:utilities.py:81:Scrape Failed: (DDOS-Guard - DDoS-Guard detected)
2023-12-16 13:56:41,874:utilities.py:81:Scrape Finished:
2023-12-16 13:56:41,874:utilities.py:81:Scrape Starting:
2023-12-16 13:56:42,151:utilities.py:81:Running Post-Download Processes...
2023-12-16 13:56:42,151:utilities.py:86:Checking for partial downloads...
2023-12-16 13:56:42,164:utilities.py:86:There are partial downloads in the downloads folder
2023-12-16 13:56:42,165:utilities.py:86:Checking for empty folders...
2023-12-16 13:56:42,288:utilities.py:81:Printing Stats...
2023-12-16 13:56:42,288:utilities.py:86:
Download Stats:
2023-12-16 13:56:42,289:utilities.py:86:Downloaded 0 files
2023-12-16 13:56:42,289:utilities.py:86:Previously Downloaded 0 files
2023-12-16 13:56:42,290:utilities.py:86:Skipped By Config 0 files
2023-12-16 13:56:42,290:utilities.py:86:Failed 0 files
2023-12-16 13:56:42,291:utilities.py:86:
Scrape Failures:
2023-12-16 13:56:42,291:utilities.py:86:Scrape Failures (DDOS-Guard): 7
2023-12-16 13:56:42,292:utilities.py:86:
Download Failures:
2023-12-16 13:56:42,292:utilities.py:81:Checking for Program End...
2023-12-16 13:56:42,292:utilities.py:81:Checking for Updates...
2023-12-16 13:56:42,736:utilities.py:81:Closing Program...
2023-12-16 13:56:42,738:utilities.py:86:
Finished downloading. Enjoy :)
CDL was working perfectly two days ago (12/14) and I was able to scrape from 10 different bunkrr sites. I think all the files downloaded successfully and I didn't encounter any download failures. Didn't run CDL at all yesterday but today (12/16) tried running CDL again (with the same list of sites) and got all DDOS-Guard scrape failures.
You're getting ddos-guard errors, which currently you can't do anything about.
It's been a thing bunkrr has had for a long time. But it gets more/less aggressive based on it's settings and how much traffic (the important bit here) that the site is receiving.
A VPN might help (don't use one of the big name ones like Nord or PIA, I recommend MullVad), but it likely won't "solve" it. There are people that also use proxies to try and alleviate it.