Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
Don't know what to tell you, you would've had to install that one on your own.
You might have to restart your computer then try again.
So I got it to work after a restart and a reinstall, I also set the directory to the primary HDD instead of a external HDD. Don't know if any of that matters though.
I didn't put a -o arg, does the script just write to a the downloader.log? I dont see any files saved. Should I terminate and run it again with a -o directory?
The output folder will end up in the same place as the downloader.log file, if it hasn't finished, let it run. -o will change the working directory for all files it will write.
any idea how to fix this on OSX?
We are running version 3.6.47 of Cyberdrop Downloader
Getting Run Args
Starting Scrape
Starting scrape of
Error scraping
Finished scrape of
We are running version 3.6.47 of Cyberdrop Downloader
Getting Run Args
Starting Scrape
Starting scrape of
Error scraping
Finished scrape of
2023-01-11 20:24:17,239:DEBUG:selector_events:selector_events.py:54:Using selector: KqueueSelector
2023-01-11 20:24:17,274:DEBUG:base_functions:base_functions.py:43:We are running version 3.3.27 of Cyberdrop Downloader
2023-01-11 20:24:17,274:DEBUG:main:main.py:64:Starting downloader with args: {'input_file': PosixPath('URLs.txt'), 'output_folder': PosixPath('Downloads'), 'log_file': 'downloader.log', 'db_file': 'download_history.sqlite', 'threads': 0, 'attempts': 10, 'connection_timeout': 15, 'disable_attempt_limit': False, 'include_id': False, 'exclude_videos': False, 'exclude_images': False, 'exclude_audio': False, 'exclude_other': False, 'ignore_history': True, 'output_last_forum_post': False, 'proxy': None, 'separate_posts': False, 'xbunker_username': None, 'xbunker_password': '!REDACTED!', '*Blacklisted site*_username': None, '*Blacklisted site*_password': '!REDACTED!', 'simpcity_username': None, 'simpcity_password': '!REDACTED!', 'skip_hosts': [], 'ratelimit': 50, 'throttle': 0.5, 'links': []}
2023-01-11 20:24:17,311:DEBUG:base_functions:base_functions.py:43:Starting Scrape
2023-01-11 20:24:19,139:DEBUG:Bunkr_Spider:Bunkr_Spider.py:72:Error encountered while handling
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cyberdrop_dl/crawlers/Bunkr_Spider.py", line 67, in stream
raise Exception("Couldn't get link from HTML")
Exception: Couldn't get link from HTML
2023-01-11 20:24:19,142:DEBUG:base_functions:base_functions.py:43:Error scraping
2023-01-11 20:24:19,142:DEBUG:Bunkr_Spider:Bunkr_Spider.py:74:Couldn't get link from HTML
2023-01-11 20:24:19,142:DEBUG:base_functions:base_functions.py:43:Finished scrape of
2023-01-11 20:24:19,142:ERROR:main:main.py:107:ValueError No links
2023-01-11 20:24:19,143:DEBUG:base_functions:base_functions.py:43:No links found duing scraping, check passwords or that the urls are accessible
2023-01-11 20:24:19,143:DEBUG:base_functions:base_functions.py:43:This program does not currently support password protected albums.
2023-01-11 20:24:19,143:DEBUG:sql_helper:sql_helper.py:93:Successfully closed sqlite database connection
Try updating to the latest version.
pip3 install --upgrade cyberdrop-dl and if that doesn't work pip install --upgrade cyberdrop-dl
The program doesn't support password protected links.
Any way to get just the file links? I'd rather use FDM to download.
Currently I've modified /cyberdrop_dl/client/downloaders.py (added a print after line 313) and I'm copy/pasting the links and ctrl c to stop the download - but it's a hassle, especially for multiple input links.
Can you recommend a better way?
Currently I've modified /cyberdrop_dl/client/downloaders.py (added a print after line 313) and I'm copy/pasting the links and ctrl c to stop the download - but it's a hassle, especially for multiple input links.
Can you recommend a better way?
Thanks anyway - I ended up having it write to a file and added an input line so I can choose which input URL to run. Saves a bit of effort.
Turns out FDM can't handle certain links anyway so I can download those using cyberdrop-dl while the rest can be done with FDM. Thanks again for this project
Turns out FDM can't handle certain links anyway so I can download those using cyberdrop-dl while the rest can be done with FDM. Thanks again for this project
hello friends, is it possible to put a limit to download a file smaller than 1G (< 1G)
for example I have a link I want to download the videos only that weigh less than 500MB (< 500)
I don't want to download videos that weigh more than 1GB, is it possible?
for example I have a link I want to download the videos only that weigh less than 500MB (< 500)
I don't want to download videos that weigh more than 1GB, is it possible?
It is not possible currently, and while I've had a lot of requests to implement it, I avoid implementing features that don't work for all websites. A lot of sites don't properly report filesize (or inconsistently report it, if they report it at all) in header information for the download.