Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

[Jules--Winnfield]

Please, Log in or Register to view quotes

As it says, you are getting timeout errors. Increase the connection_timeout number, to something like 30 seconds. Possibly try using a vpn.

 

[armylink]

I appreciate the help and the work you've put into the tool. I was able to get it working after trying the VPN and increasing the timeout number. Thanks for the prompt reply as well

 

[malik24]

Please, Log in or Register to view quotes

copy that. its working again today

 

[rdsxfn2004]

Unknown Error when attempting to scrape:

INFO : 2024-03-10 16:24:18,984 : utilities.py:98 : Starting Async Processes...
DEBUG : 2024-03-10 16:24:18,984 : utilities.py:98 : Starting Cyberdrop-DL Process for Default Config
DEBUG : 2024-03-10 16:24:18,984 : utilities.py:98 : Running version 5.1.99
DEBUG : 2024-03-10 16:24:18,984 : utilities.py:98 : Using Config: Default
DEBUG : 2024-03-10 16:24:18,984 : utilities.py:98 : Using Config File: AppData\Configs\Default\settings.yaml
DEBUG : 2024-03-10 16:24:18,984 : utilities.py:98 : Using Input File: N:\AppData\Configs\Default\URLs.txt
DEBUG : 2024-03-10 16:24:18,989 : utilities.py:98 : Using Download Folder: N:\Downloads\Cyberdrop-DL Downloads
DEBUG : 2024-03-10 16:24:18,989 : utilities.py:98 : Using History File: AppData\Cache\cyberdrop.db
DEBUG : 2024-03-10 16:24:18,989 : utilities.py:98 : Using Authentication:
{
"Forums Credentials": {
"celebforum Credentials Provided": false,
"f95zone Credentials Provided": false,
"leakedmodels Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"simpcity Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"shit forum Credentials Provided": false
},
"Forums XF Cookies": {
"celebforum XF Cookie Provided": false,
"f95zone XF Cookie Provided": false,
"leakedmodels XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"simpcity XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"shit forum XF Cookie Provided": false
},
"GoFile Provided": false,
"Imgur Provided": false,
"JDownloader Provided": false,
"PixelDrain Provided": false,
"Reddit Provided": false
}
DEBUG : 2024-03-10 16:24:18,989 : utilities.py:98 : Using Settings:
{
"Download_Options": {
"block_download_sub_folders": false,
"disable_download_attempt_limit": false,
"disable_file_timestamps": false,
"include_album_id_in_folder_name": false,
"include_thread_id_in_folder_name": false,
"remove_domains_from_folder_names": false,
"remove_generated_id_from_filenames": false,
"scrape_single_forum_post": false,
"separate_posts": false,
"skip_download_mark_completed": false
},
"File_Size_Limits": {
"maximum_image_size": 0,
"maximum_other_size": 0,
"maximum_video_size": 0,
"minimum_image_size": 0,
"minimum_other_size": 0,
"minimum_video_size": 0
},
"Files": {
"download_folder": "N:\\Downloads\\Cyberdrop-DL Downloads",
"input_file": "N:\\AppData\\Configs\\Default\\URLs.txt"
},
"Ignore_Options": {
"exclude_audio": false,
"exclude_images": false,
"exclude_other": false,
"exclude_videos": false,
"ignore_coomer_ads": false,
"only_hosts": [],
"skip_hosts": []
},
"Logs": {
"download_error_urls_filename": "Download_Error_URLs.csv",
"last_forum_post_filename": "Last_Scraped_Forum_Posts.txt",
"log_folder": "C:\\Users\\JFeen\\AppData\\Local\\Cyberdrop-DL\\Cyberdrop-DL\\Configs\\Default\\Logs",
"main_log_filename": "downloader.log",
"scrape_error_urls_filename": "Scrape_Error_URLs.csv",
"unsupported_urls_filename": "Unsupported_URLs.txt"
},
"Runtime_Options": {
"delete_partial_files": false,
"ignore_history": false,
"log_level": 10,
"send_unsupported_to_jdownloader": false,
"skip_check_for_empty_folders": false,
"skip_check_for_partial_files": false
},
"Sorting": {
"sort_downloads": false,
"sort_folder": "C:\\Users\\JFeen\\Downloads\\Cyberdrop-DL Sorted Downloads",
"sort_incremementer_format": " ({i})",
"sorted_audio": "{sort_dir}/{base_dir}/Audio/{filename}{ext}",
"sorted_image": "{sort_dir}/{base_dir}/Images/{filename}{ext}",
"sorted_other": "{sort_dir}/{base_dir}/Other/{filename}{ext}",
"sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}"
}
}
DEBUG : 2024-03-10 16:24:18,989 : utilities.py:98 : Using Global Settings:
{
"General": {
"allow_insecure_connections": false,
"flaresolverr": "",
"max_file_name_length": 95,
"max_folder_name_length": 60,
"proxy": "",
"required_free_space": 5,
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/120.0",
"vi_mode": false
},
"Rate_Limiting_Options": {
"connection_timeout": 15,
"download_attempts": 100,
"download_delay": 0,
"max_simultaneous_downloads": 15,
"max_simultaneous_downloads_per_domain": 4,
"rate_limit": 50,
"read_timeout": 300
}
}
INFO : 2024-03-10 16:24:19,007 : utilities.py:98 : Starting UI...
INFO : 2024-03-10 16:24:19,181 : utilities.py:98 : Scrape Starting:

Please, Log in or Register to see links and images

ERROR : 2024-03-10 16:24:19,713 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(404 - HTTP status code 404: Not Found)
ERROR : 2024-03-10 16:24:20,173 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(cannot access local variable 'JSON_Resp' where it is not associated with a value)
ERROR : 2024-03-10 16:24:20,175 : utilities.py:98 : Traceback (most recent call last):
File "N:\venv\Lib\site-packages\cyberdrop_dl\utils\utilities.py", line 62, in wrapper
return await func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "N:\venv\Lib\site-packages\cyberdrop_dl\scraper\crawlers\gofile_crawler.py", line 64, in album
if JSON_Resp["status"] != "ok":
^^^^^^^^^
UnboundLocalError: cannot access local variable 'JSON_Resp' where it is not associated with a value

INFO : 2024-03-10 16:24:20,177 : utilities.py:98 : Scrape Finished:

Please, Log in or Register to see links and images

INFO : 2024-03-10 16:24:20,278 : utilities.py:98 : Running Post-Download Processes...
INFO : 2024-03-10 16:24:20,278 : utilities.py:111 : Checking for partial downloads...
INFO : 2024-03-10 16:24:20,320 : utilities.py:111 : Checking for empty folders...
INFO : 2024-03-10 16:24:20,341 : utilities.py:98 : Printing Stats...
INFO : 2024-03-10 16:24:20,342 : utilities.py:111 :
Download Stats:
INFO : 2024-03-10 16:24:20,342 : utilities.py:111 : Downloaded 0 files
INFO : 2024-03-10 16:24:20,343 : utilities.py:111 : Previously Downloaded 0 files
INFO : 2024-03-10 16:24:20,344 : utilities.py:111 : Skipped By Config 0 files
INFO : 2024-03-10 16:24:20,344 : utilities.py:111 : Failed 0 files
INFO : 2024-03-10 16:24:20,345 : utilities.py:111 :
Scrape Failures:
INFO : 2024-03-10 16:24:20,345 : utilities.py:111 : Scrape Failures (404 HTTP Status): 1
INFO : 2024-03-10 16:24:20,346 : utilities.py:111 : Scrape Failures (Unknown): 1
INFO : 2024-03-10 16:24:20,347 : utilities.py:111 :
Download Failures:
INFO : 2024-03-10 16:24:20,347 : utilities.py:98 : Checking for Program End...
INFO : 2024-03-10 16:24:20,347 : utilities.py:98 : Checking for Updates...
INFO : 2024-03-10 16:24:20,566 : utilities.py:98 : Closing Program...
INFO : 2024-03-10 16:24:20,567 : utilities.py:111 :
Finished downloading. Enjoy :)

 

[rdsxfn2004]

Please, Log in or Register to view quotes

Just seems to be GoFile sites

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

looking into it.

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

5.2.1 going up now should fix GoFile scraping

Edit: I lied. 5.2.2 fixes it.

 

[rdsxfn2004]

Please, Log in or Register to view quotes

Edit from below: Disregard, 5.2.2 did fix it

No specific error but below is download log file:

INFO : 2024-03-10 17:34:42,167 : utilities.py:98 : Starting Async Processes...
DEBUG : 2024-03-10 17:34:42,168 : utilities.py:98 : Starting Cyberdrop-DL Process for Default Config
DEBUG : 2024-03-10 17:34:42,168 : utilities.py:98 : Running version 5.2.1
DEBUG : 2024-03-10 17:34:42,168 : utilities.py:98 : Using Config: Default
DEBUG : 2024-03-10 17:34:42,168 : utilities.py:98 : Using Config File: AppData\Configs\Default\settings.yaml
DEBUG : 2024-03-10 17:34:42,168 : utilities.py:98 : Using Input File: N:\AppData\Configs\Default\URLs.txt
DEBUG : 2024-03-10 17:34:42,173 : utilities.py:98 : Using Download Folder: N:\Downloads\Cyberdrop-DL Downloads
DEBUG : 2024-03-10 17:34:42,173 : utilities.py:98 : Using History File: AppData\Cache\cyberdrop.db
DEBUG : 2024-03-10 17:34:42,173 : utilities.py:98 : Using Authentication:
{
"Forums Credentials": {
"celebforum Credentials Provided": false,
"f95zone Credentials Provided": false,
"leakedmodels Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"simpcity Credentials Provided": false,
"*Blacklisted site* Credentials Provided": false,
"shit forum Credentials Provided": false
},
"Forums XF Cookies": {
"celebforum XF Cookie Provided": false,
"f95zone XF Cookie Provided": false,
"leakedmodels XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"simpcity XF Cookie Provided": false,
"*Blacklisted site* XF Cookie Provided": false,
"shit forum XF Cookie Provided": false
},
"GoFile Provided": false,
"Imgur Provided": false,
"JDownloader Provided": false,
"PixelDrain Provided": false,
"Reddit Provided": false
}
DEBUG : 2024-03-10 17:34:42,174 : utilities.py:98 : Using Settings:
{
"Download_Options": {
"block_download_sub_folders": false,
"disable_download_attempt_limit": false,
"disable_file_timestamps": false,
"include_album_id_in_folder_name": false,
"include_thread_id_in_folder_name": false,
"remove_domains_from_folder_names": false,
"remove_generated_id_from_filenames": false,
"scrape_single_forum_post": false,
"separate_posts": false,
"skip_download_mark_completed": false
},
"File_Size_Limits": {
"maximum_image_size": 0,
"maximum_other_size": 0,
"maximum_video_size": 0,
"minimum_image_size": 0,
"minimum_other_size": 0,
"minimum_video_size": 0
},
"Files": {
"download_folder": "N:\\Downloads\\Cyberdrop-DL Downloads",
"input_file": "N:\\AppData\\Configs\\Default\\URLs.txt"
},
"Ignore_Options": {
"exclude_audio": false,
"exclude_images": false,
"exclude_other": false,
"exclude_videos": false,
"ignore_coomer_ads": false,
"only_hosts": [],
"skip_hosts": []
},
"Logs": {
"download_error_urls_filename": "Download_Error_URLs.csv",
"last_forum_post_filename": "Last_Scraped_Forum_Posts.txt",
"log_folder": "C:\\Users\\JFeen\\AppData\\Local\\Cyberdrop-DL\\Cyberdrop-DL\\Configs\\Default\\Logs",
"main_log_filename": "downloader.log",
"scrape_error_urls_filename": "Scrape_Error_URLs.csv",
"unsupported_urls_filename": "Unsupported_URLs.txt"
},
"Runtime_Options": {
"delete_partial_files": false,
"ignore_history": false,
"log_level": 10,
"send_unsupported_to_jdownloader": false,
"skip_check_for_empty_folders": false,
"skip_check_for_partial_files": false
},
"Sorting": {
"sort_downloads": false,
"sort_folder": "C:\\Users\\JFeen\\Downloads\\Cyberdrop-DL Sorted Downloads",
"sort_incremementer_format": " ({i})",
"sorted_audio": "{sort_dir}/{base_dir}/Audio/{filename}{ext}",
"sorted_image": "{sort_dir}/{base_dir}/Images/{filename}{ext}",
"sorted_other": "{sort_dir}/{base_dir}/Other/{filename}{ext}",
"sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}"
}
}
DEBUG : 2024-03-10 17:34:42,174 : utilities.py:98 : Using Global Settings:
{
"General": {
"allow_insecure_connections": false,
"flaresolverr": "",
"max_file_name_length": 95,
"max_folder_name_length": 60,
"proxy": "",
"required_free_space": 5,
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/120.0",
"vi_mode": false
},
"Rate_Limiting_Options": {
"connection_timeout": 15,
"download_attempts": 100,
"download_delay": 0,
"max_simultaneous_downloads": 15,
"max_simultaneous_downloads_per_domain": 4,
"rate_limit": 50,
"read_timeout": 300
}
}
INFO : 2024-03-10 17:34:42,189 : utilities.py:98 : Starting UI...
INFO : 2024-03-10 17:34:42,339 : utilities.py:98 : Scrape Starting:

Please, Log in or Register to see links and images

ERROR : 2024-03-10 17:34:42,774 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(404 - HTTP status code 404: Not Found)
ERROR : 2024-03-10 17:34:44,311 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(401 - HTTP status code 401: Unauthorized)
INFO : 2024-03-10 17:34:44,312 : utilities.py:98 : Scrape Finished:

Please, Log in or Register to see links and images

INFO : 2024-03-10 17:34:44,312 : utilities.py:98 : Scrape Starting:

Please, Log in or Register to see links and images

ERROR : 2024-03-10 17:34:44,740 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(404 - HTTP status code 404: Not Found)
ERROR : 2024-03-10 17:34:46,281 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(401 - HTTP status code 401: Unauthorized)
INFO : 2024-03-10 17:34:46,282 : utilities.py:98 : Scrape Finished:

Please, Log in or Register to see links and images

INFO : 2024-03-10 17:34:46,282 : utilities.py:98 : Scrape Starting:

Please, Log in or Register to see links and images

ERROR : 2024-03-10 17:34:46,710 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(404 - HTTP status code 404: Not Found)
ERROR : 2024-03-10 17:34:48,338 : utilities.py:98 : Scrape Failed:

Please, Log in or Register to see links and images

(401 - HTTP status code 401: Unauthorized)
INFO : 2024-03-10 17:34:48,339 : utilities.py:98 : Scrape Finished:

Please, Log in or Register to see links and images

INFO : 2024-03-10 17:34:48,447 : utilities.py:98 : Running Post-Download Processes...
INFO : 2024-03-10 17:34:48,447 : utilities.py:111 : Checking for partial downloads...
INFO : 2024-03-10 17:34:48,473 : utilities.py:111 : Checking for empty folders...
INFO : 2024-03-10 17:34:48,494 : utilities.py:98 : Printing Stats...
INFO : 2024-03-10 17:34:48,494 : utilities.py:111 :
Download Stats:
INFO : 2024-03-10 17:34:48,495 : utilities.py:111 : Downloaded 0 files
INFO : 2024-03-10 17:34:48,496 : utilities.py:111 : Previously Downloaded 0 files
INFO : 2024-03-10 17:34:48,496 : utilities.py:111 : Skipped By Config 0 files
INFO : 2024-03-10 17:34:48,497 : utilities.py:111 : Failed 0 files
INFO : 2024-03-10 17:34:48,497 : utilities.py:111 :
Scrape Failures:
INFO : 2024-03-10 17:34:48,498 : utilities.py:111 : Scrape Failures (404 HTTP Status): 3
INFO : 2024-03-10 17:34:48,499 : utilities.py:111 : Scrape Failures (401 HTTP Status): 3
INFO : 2024-03-10 17:34:48,499 : utilities.py:111 :
Download Failures:
INFO : 2024-03-10 17:34:48,500 : utilities.py:98 : Checking for Program End...
INFO : 2024-03-10 17:34:48,500 : utilities.py:98 : Checking for Updates...
INFO : 2024-03-10 17:34:48,720 : utilities.py:98 : Closing Program...
INFO : 2024-03-10 17:34:48,720 : utilities.py:111 :
Finished downloading. Enjoy :)

 

[loll8]

Hi! I've deleted completely (even from the recycle bin) an album that I had downloaded from a bunkrr link. Now, when I try to re-download it, it says that I've already download all the files in it. What can I do to solve this? Thanks

 

[CreAtive642]

How can I download protected links
for example a gofile link thats password protected
(I know the password of course)

 

[olsaltee]

Please, Log in or Register to view quotes

Manually, this is the only way with any of the password protected links.

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

turn on ignore history in the config.

 

[loll8]

Please, Log in or Register to view quotes

When it says: "enter the log level" what should I put? Sorry, noob here

 

[EightyThree]

Please, Log in or Register to view quotes

Default is 10, so I suggest that.

 

[Jules--Winnfield]

loll8 EightyThree
In the UI, anything that requires an input, you can simply press enter and it'll set it to the default.

 

[SasukeUchiha49]

Hey Cyberdrop just randomly stopped working again Imma just redownload it again as that worked last time but was wondering if there's a way to import history and such when I reinstall it just so I'm not back at 0 before I delete everything?

Please, Log in or Register to see links and images

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

just delete the auth config in ./appdata/configs. That'll solve your issue.

 

[SasukeUchiha49]

Please, Log in or Register to view quotes

It worked thanks

 

[uselessj]

Is there anyway to scrape from a particular post of thread?
Essentially I want to scrape forward from post X on thread Y and not other post before that.

 

[diaoz1986]

Please, Log in or Register to see links and images

I try to download a bunkr album with 7 videos, but somehow it only download 1.. can i check what i need to do to download the rest?