Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads
- Thread starter Jules--Winnfield
- Start date
I'd need to see the downloader.log, also using the second link there is redundant, the first one will scrape every page.
I'm getting an error on scrapes from bunkr. Here's the download log.
2023-03-22 12:34:20,922:DEBUG:proactor_events:proactor_events.py:633:Using proactor: IocpProactor
2023-03-22 12:34:21,021:DEBUG:base_functions:base_functions.py:42:Starting Cyberdrop-DL
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using authentication arguments: {'pixeldrain_api_key': None, 'simpcity_username': None, 'simpcity_password': None, '*Blacklisted site*_username': None, '*Blacklisted site*_password': None, 'xbunker_username': None, 'xbunker_password': None}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using file arguments: {'db_file': WindowsPath('download_history.sqlite'), 'errored_urls_file': WindowsPath('Errored_URLs.csv'), 'input_file': WindowsPath('URLs.txt'), 'log_file': WindowsPath('downloader.log'), 'output_folder': WindowsPath('Downloads'), 'output_last_forum_post_file': WindowsPath('URLs_Last_Post.txt'), 'unsupported_urls_file': WindowsPath('Unsupported_URLs.csv')}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using forum option arguments: {'output_last_forum_post': False, 'separate_posts': False}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using ignore arguments: {'exclude_videos': False, 'exclude_images': False, 'exclude_audio': False, 'exclude_other': False, 'ignore_cache': False, 'ignore_history': False, 'skip_hosts': [], 'only_hosts': []}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using jdownloader arguments: {'apply_jdownloader': False, 'jdownloader_username': None, 'jdownloader_password': None, 'jdownloader_device': None}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using progress option arguments: {'hide_new_progress': False, 'dont_show_overall_progress': False, 'dont_show_forum_progress': False, 'dont_show_thread_progress': False, 'dont_show_domain_progress': False, 'dont_show_album_progress': False, 'dont_show_file_progress': False, 'refresh_rate': 10}
2023-03-22 12:34:21,030:DEBUG:base_functions:base_functions.py:42:Using ratelimiting arguments: {'connection_timeout': 15, 'ratelimit': 50, 'throttle': 0.5}
2023-03-22 12:34:21,031:DEBUG:base_functions:base_functions.py:42:Using runtime arguments: {'allow_insecure_connections': False, 'attempts': 10, 'block_sub_folders': False, 'disable_attempt_limit': False, 'include_id': False, 'skip_download_mark_completed': False, 'output_errored_urls': False, 'output_unsupported_urls': False, 'proxy': None, 'remove_bunkr_identifier': False, 'required_free_space': 5, 'simultaneous_downloads_per_domain': 4}
2023-03-22 12:34:21,031:DEBUG:base_functions:base_functions.py:42:Using sorting arguments: {'sort_downloads': False, 'sort_directory': WindowsPath('Sorted Downloads'), 'sorted_audio': '{sort_dir}/{base_dir}/Audio', 'sorted_images': '{sort_dir}/{base_dir}/Images', 'sorted_others': '{sort_dir}/{base_dir}/Other', 'sorted_videos': '{sort_dir}/{base_dir}/Videos'}
2023-03-22 12:34:21,031:DEBUG:base_functions:base_functions.py:42:We are running version 4.0.92 of Cyberdrop Downloader
2023-03-22 12:34:21,046:DEBUG:base_functions:base_functions.py:42:Starting Scrape
2023-03-22 12:34:21,048:DEBUG:base_functions:base_functions.py:42:Starting: https://bunkrr.su/a/pdojrNcb
2023-03-22 12:34:36,445:DEBUG:Bunkr_Spider:Bunkr_Spider.py:164:Error encountered while handling https://bunkrr.su/a/pdojrNcb
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 536, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\connector.py", line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 1069, in create_connection
sock = await self._connect_sock(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 973, in _connect_sock
await self.sock_connect(sock, address)
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\asyncio\proactor_events.py", line 726, in sock_connect
return await self._proactor.connect(sock, address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 534, in _request
async with ceil_timeout(real_timeout.connect):
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\async_timeout\__init__.py", line 129, in __aexit__
self._do_exit(exc_type)
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\async_timeout\__init__.py", line 212, in _do_exit
raise asyncio.TimeoutError
TimeoutError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\cyberdrop_dl\crawlers\Bunkr_Spider.py", line 123, in get_album
soup = await session.get_BS4(url)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\cyberdrop_dl\client\client.py", line 52, in get_BS4
async with self.client_session.get(url, ssl=self.client.ssl_context) as response:
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 1141, in __aenter__
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\aiohttp\client.py", line 540, in _request
raise ServerTimeoutError(
aiohttp.client_exceptions.ServerTimeoutError: Connection timeout to host https://bunkrr.su/a/pdojrNcb
2023-03-22 12:34:36,458:DEBUG:base_functions:base_functions.py:42:Error: https://bunkrr.su/a/pdojrNcb
2023-03-22 12:34:36,465:DEBUG:Bunkr_Spider:Bunkr_Spider.py:166:Connection timeout to host https://bunkrr.su/a/pdojrNcb
2023-03-22 12:34:36,466:DEBUG:base_functions:base_functions.py:42:
2023-03-22 12:34:36,467:DEBUG:base_functions:base_functions.py:42:Finished Scrape
2023-03-22 12:34:41,636:DEBUG:base_functions:base_functions.py:42:
2023-03-22 12:34:41,638:DEBUG:base_functions:base_functions.py:42:Checking for incomplete downloads
2023-03-22 12:34:41,641:DEBUG:base_functions:base_functions.py:42:Purging empty directories
2023-03-22 12:34:41,643:DEBUG:base_functions:base_functions.py:42:Finished downloading. Enjoy :)
2023-03-22 12:34:41,644:DEBUG:base_functions:base_functions.py:42:
2023-03-22 12:34:41,646:DEBUG:base_functions:base_functions.py:42:If you enjoy using this program, please consider buying the developer a coffee :)
https://www.buymeacoffee.com/juleswinnft
2023-03-22 12:34:46,660:DEBUG:sql_helper:sql_helper.py:250:Successfully closed sqlite database connection
Your connection timed out. Indicating you have connection issues with bunkr.
Hope you don't mind me asking, I created a post outside of this thread to see if anyone knew of a way, but thought I'd ask the question for your script.
Is it possible to use your script to scrape the all the URL's from a SC thread and compile them into a list or file without downloading any of the content?
Some of the threads have lots of pages to scroll through, and I'd like to get a list of URLs so that I can open them all in one window for instance, and see the content before I choose which URLs to keep.
Is it possible to use your script to scrape the all the URL's from a SC thread and compile them into a list or file without downloading any of the content?
Some of the threads have lots of pages to scroll through, and I'd like to get a list of URLs so that I can open them all in one window for instance, and see the content before I choose which URLs to keep.
Some help please? A few things, i cant use a mass uploader i found because the site it sends to is cyberdrop.is or .su and im only able to access the .me host. This also doesnt allow me access to the premium store. Cant do it and its driving me crazy.
Now with that said, does anyone have am idea of how strict they are with the 30 day deletion? I completly noobed msself and assumed it was storage forever.....for freee.like the dummy i am. So yeah,..
Help please?
Edit! Just made a thread in case my post was not approved.
Now with that said, does anyone have am idea of how strict they are with the 30 day deletion? I completly noobed msself and assumed it was storage forever.....for freee.like the dummy i am. So yeah,..
Help please?
Edit! Just made a thread in case my post was not approved.
Last edited:
config/arg exclude_audio, exclude_videos, exclude_other
where do i paste this
after or before the coomer.party link or do i have to edit something
Last edited:
the config.yaml file. Change those values to true, and change apply_config to true.
its working
i edited the config.yaml file few minutes ago and changed the exclude_audio, exclude_videos, exclude_other to true but it didnt work cause of the appy_config so i was sitting here and waiting for your reply
Thank You for the help
Could anyone help me with this issue? I have 1TB of internet and the download speed of the tool is very low, I've been trying to download a 5GB folder since yesterday and it still hasn't finished. Is this some sort of speed limiter on the sites I'm downloading from or is it something I should modify in the tool's configuration?
The Cyberdrop host is your problem. It's dogshit and barely works anymore.
You are limited by what the server will output and transmit at, not what you can download at.
bunkr, cyberfile, anonfiles, gofile, pixeldrain, etc.
You may be better asking that in the help section here:
Help | Support | Suggestions
Need help? Have suggestions to improve the site? Post in here!
simpcity.su