you need to put the entire link, with https:// in front.
Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
Is it possible I can find the code to do this on the downloader somehow? Or how can I achieve the same thing cyberdrop does when it comes to scraping every link from forum post?
You can try, but it'd require a lot of changes.
You'd have to alter or write your own code to do so.
I'm guessing a script that would scrape all links and move to next page. Yeah sounds quite complicated to be honest.
Perhaps you know something similar already exists on the forums?
I am using cyberdrop it's really good, but it's not flexible like jdownloader in general it's a more versatile downloading tool, also perhaps I missed it but no support for proxies.
Disagree on both, and CDL does have support for Proxies.
CDL also supports sending unsupported links to JDownloader.
First, thank you so much for making this awesome tool available. I appreciate it very mush! Next, I have a troubleshooting question, if this isn't the right place, I would appreciate it if you could point me in the right direction.
IIRC, I used to be able to set the url as either
1) the page name: e.g. https://simpcity.su/threads/brunetttebaddie-mirandamarieox-bbgxo20.19095/page-2
or
2) the fist post of the page e.g https://simpcity.su/threads/brunetttebaddie-mirandamarieox-bbgxo20.19095/post-2654354
in either case, the down loader would scrape and download all posts either on the entire page (scenario #1 above) or all posts at and below the starting post (scenario #2). Now the page url (#1) throws an error and the post (#2) use downloads that post and nothing below it.
Here are the config settings I am using:
Download_Options:
block_download_sub_folders: false
disable_download_attempt_limit: false
disable_file_timestamps: false
include_album_id_in_folder_name: false
include_thread_id_in_folder_name: false
remove_domains_from_folder_names: false
remove_generated_id_from_filenames: false
scrape_single_forum_post: false
separate_posts: false
skip_download_mark_completed: false
File_Size_Limits:
maximum_image_size: 0
maximum_other_size: 0
maximum_video_size: 0
minimum_image_size: 0
minimum_other_size: 0
minimum_video_size: 0
Files:
download_folder: Downloads\Cyberdrop-DL Downloads
input_file: URLs.txt
Ignore_Options:
exclude_audio: false
exclude_images: false
exclude_other: false
exclude_videos: false
ignore_coomer_ads: true
only_hosts: []
skip_hosts: null
Logs:
download_error_urls_filename: Download_Error_URLs.csv
last_forum_post_filename: Last_Scraped_Forum_Posts.txt
log_folder: AppData\Configs\Default\Logs
main_log_filename: downloader.log
scrape_error_urls_filename: Scrape_Error_URLs.csv
unsupported_urls_filename: Unsupported_URLs.txt
Runtime_Options:
delete_partial_files: false
ignore_history: false
log_level: 10
send_unsupported_to_jdownloader: true
skip_check_for_empty_folders: false
skip_check_for_partial_files: false
ui_refresh_rate: 10
Sorting:
sort_downloads: false
sort_folder: Downloads\Cyberdrop-DL Sorted Downloads
sort_incremementer_format: ' ({i})'
sorted_audio: '{sort_dir}/{base_dir}/Audio/{filename}{ext}'
sorted_image: '{sort_dir}/{base_dir}/Images/{filename}{ext}'
sorted_other: '{sort_dir}/{base_dir}/Other/{filename}{ext}'
sorted_video: '{sort_dir}/{base_dir}/Videos/{filename}{ext}'
IIRC, I used to be able to set the url as either
1) the page name: e.g. https://simpcity.su/threads/brunetttebaddie-mirandamarieox-bbgxo20.19095/page-2
or
2) the fist post of the page e.g https://simpcity.su/threads/brunetttebaddie-mirandamarieox-bbgxo20.19095/post-2654354
in either case, the down loader would scrape and download all posts either on the entire page (scenario #1 above) or all posts at and below the starting post (scenario #2). Now the page url (#1) throws an error and the post (#2) use downloads that post and nothing below it.
Here are the config settings I am using:
Download_Options:
block_download_sub_folders: false
disable_download_attempt_limit: false
disable_file_timestamps: false
include_album_id_in_folder_name: false
include_thread_id_in_folder_name: false
remove_domains_from_folder_names: false
remove_generated_id_from_filenames: false
scrape_single_forum_post: false
separate_posts: false
skip_download_mark_completed: false
File_Size_Limits:
maximum_image_size: 0
maximum_other_size: 0
maximum_video_size: 0
minimum_image_size: 0
minimum_other_size: 0
minimum_video_size: 0
Files:
download_folder: Downloads\Cyberdrop-DL Downloads
input_file: URLs.txt
Ignore_Options:
exclude_audio: false
exclude_images: false
exclude_other: false
exclude_videos: false
ignore_coomer_ads: true
only_hosts: []
skip_hosts: null
Logs:
download_error_urls_filename: Download_Error_URLs.csv
last_forum_post_filename: Last_Scraped_Forum_Posts.txt
log_folder: AppData\Configs\Default\Logs
main_log_filename: downloader.log
scrape_error_urls_filename: Scrape_Error_URLs.csv
unsupported_urls_filename: Unsupported_URLs.txt
Runtime_Options:
delete_partial_files: false
ignore_history: false
log_level: 10
send_unsupported_to_jdownloader: true
skip_check_for_empty_folders: false
skip_check_for_partial_files: false
ui_refresh_rate: 10
Sorting:
sort_downloads: false
sort_folder: Downloads\Cyberdrop-DL Sorted Downloads
sort_incremementer_format: ' ({i})'
sorted_audio: '{sort_dir}/{base_dir}/Audio/{filename}{ext}'
sorted_image: '{sort_dir}/{base_dir}/Images/{filename}{ext}'
sorted_other: '{sort_dir}/{base_dir}/Other/{filename}{ext}'
sorted_video: '{sort_dir}/{base_dir}/Videos/{filename}{ext}'
I'd need to see the entire downloader.log file where this happens :)
How can I use proxies? I have 20 unlimited bandwidth proxies, where do I insert them or how do I use them? With jd2 works really well as bunkr allows like max 2 connections at a time.
uninstall python, install the new python, and continue as normal. CDL should update itself from there.
Hello! I'm having issues downloading from bunkr, I've tried two albums today and both have different issues:
- with this one it puts every file in the "Scrape failures" tab with an "unknown" error.
- here it puts all files in the "Download failures" tab with a "DDOS Guard" error.
I'm pasting one link at a time, switched them back and forth a few times, same results all the time.
Fresh install of the Cyberdrop-DL v5.0.124 (default settings and config) and Python, all done according to the wiki. Coomer pages download just fine, haven't tried anything else other than bunkr.
- with this one it puts every file in the "Scrape failures" tab with an "unknown" error.
- here it puts all files in the "Download failures" tab with a "DDOS Guard" error.
I'm pasting one link at a time, switched them back and forth a few times, same results all the time.
Fresh install of the Cyberdrop-DL v5.0.124 (default settings and config) and Python, all done according to the wiki. Coomer pages download just fine, haven't tried anything else other than bunkr.
put the links you want it to skip in the urls list, then enable skip download mark complete. Make sure to turn it back off afterwards.