Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

[Jules--Winnfield]

Please, Log in or Register to view quotes

Please, Log in or Register to view quotes

I'd need to see the downloader.log file from both of you.

 

[grabo]

Can i post it here?

Please, Log in or Register to view quotes

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

You can. Upload it somewhere and just toss the link here

 

[grabo]

Please, Log in or Register to see links and images

some random hosting ( i deleted my email from this file and wrote "email" instead )

 

[qqqqqaq]

2023-05-23 14:54:08,117:DEBUG:proactor_events:proactor_events.py:626:Using proactor: IocpProactor
2023-05-23 14:54:08,153:DEBUG:base_functions:base_functions.py:54:Starting Cyberdrop-DL
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using authentication arguments: {'gofile_api_key': '!REDACTED!', 'pixeldrain_api_key': '!REDACTED!', '*Blacklisted site*_username': '', '*Blacklisted site*_password': '!REDACTED!', 'simpcity_username': '', 'simpcity_password': '!REDACTED!', '*Blacklisted site*_username': '', '*Blacklisted site*_password': '!REDACTED!', 'xbunker_username': '', 'xbunker_password': '!REDACTED!'}
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using files arguments: {'db_file': WindowsPath('download_history.sqlite'), 'errored_download_urls_file': WindowsPath('Errored_Download_URLs.csv'), 'errored_scrape_urls_file': WindowsPath('Errored_Scrape_URLs.csv'), 'input_file': WindowsPath('URLs.txt'), 'log_file': WindowsPath('downloader.log'), 'output_folder': WindowsPath('Downloads'), 'output_last_forum_post_file': WindowsPath('URLs_last_post.txt'), 'unsupported_urls_file': WindowsPath('Unsupported_URLs.csv')}
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using forum options arguments: {'output_last_forum_post': False, 'scrape_single_post': False, 'separate_posts': False}
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using ignore arguments: {'exclude_videos': False, 'exclude_images': False, 'exclude_audio': False, 'exclude_other': False, 'ignore_cache': False, 'ignore_history': False, 'skip_hosts': [], 'only_hosts': []}
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using jdownloader arguments: {'apply_jdownloader': False, 'jdownloader_username': '', 'jdownloader_password': '!REDACTED!', 'jdownloader_device': ''}
2023-05-23 14:54:08,167:DEBUG:base_functions:base_functions.py:54:Using progress options arguments: {'hide_new_progress': False, 'hide_overall_progress': False, 'hide_forum_progress': False, 'hide_thread_progress': False, 'hide_domain_progress': False, 'hide_album_progress': False, 'hide_file_progress': False, 'refresh_rate': 10, 'visible_rows_threads': 2, 'visible_rows_domains': 2, 'visible_rows_albums': 2, 'visible_rows_files': 10}
2023-05-23 14:54:08,168:DEBUG:base_functions:base_functions.py:54:Using ratelimiting arguments: {'connection_timeout': 15, 'ratelimit': 50, 'throttle': 0.5}
2023-05-23 14:54:08,168:DEBUG:base_functions:base_functions.py:54:Using runtime arguments: {'allow_insecure_connections': False, 'attempts': 10, 'block_sub_folders': False, 'disable_attempt_limit': False, 'include_id': False, 'max_concurrent_threads': 0, 'max_concurrent_domains': 0, 'max_concurrent_albums': 0, 'max_concurrent_downloads_per_domain': 4, 'output_errored_urls': False, 'output_unsupported_urls': False, 'proxy': '', 'remove_bunkr_identifier': False, 'required_free_space': 5, 'skip_download_mark_completed': False, 'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0'}
2023-05-23 14:54:08,168:DEBUG:base_functions:base_functions.py:54:Using sorting arguments: {'sort_downloads': False, 'sort_directory': WindowsPath('Sorted Downloads'), 'sorted_audio': '{sort_dir}/{base_dir}/Audio', 'sorted_images': '{sort_dir}/{base_dir}/Images', 'sorted_others': '{sort_dir}/{base_dir}/Other', 'sorted_videos': '{sort_dir}/{base_dir}/Videos'}
2023-05-23 14:54:08,168:DEBUG:base_functions:base_functions.py:54:We are running version 4.2.44 of Cyberdrop Downloader
2023-05-23 14:54:08,181:DEBUG:base_functions:base_functions.py:54:Starting Scrape
2023-05-23 14:54:08,182:DEBUG:base_functions:base_functions.py:54:Starting:

Please, Log in or Register to see links and images

2023-05-23 14:54:09,147:DEBUG:base_functions:base_functions.py:54:Finished:

Please, Log in or Register to see links and images

2023-05-23 14:54:09,148:DEBUG:base_functions:base_functions.py:54:
2023-05-23 14:54:09,149:DEBUG:base_functions:base_functions.py:54:Finished Scrape
2023-05-23 14:54:15,782:DEBUG:downloaders:downloaders.py:213:

Please, Log in or Register to see links and images

(Bunkr under maintenance)
2023-05-23 14:54:15,782:DEBUG:downloaders:downloaders.py:229:

Please, Log in or Register to see links and images

(Bunkr under maintenance)
2023-05-23 14:54:15,782:DEBUG:base_functions:base_functions.py:54:Failed Download: 0h22ub9icco5x4dwzq29m_source-LQOOJL9H.mp4
2023-05-23 14:54:15,827:DEBUG:base_functions:base_functions.py:54:| [green]Files Complete: 0[/green] - [yellow]Files Skipped: 0[/yellow] - [red]Files Failed: 1[/red] |
2023-05-23 14:54:15,829:DEBUG:base_functions:base_functions.py:54:
2023-05-23 14:54:15,829:DEBUG:base_functions:base_functions.py:54:Checking for incomplete downloads
2023-05-23 14:54:15,830:DEBUG:base_functions:base_functions.py:54:Purging empty directories
2023-05-23 14:54:15,830:DEBUG:base_functions:base_functions.py:54:Finished downloading. Enjoy :)
2023-05-23 14:54:15,831:DEBUG:base_functions:base_functions.py:54:
2023-05-23 14:54:15,831:DEBUG:base_functions:base_functions.py:54:If you enjoy using this program, please consider buying the developer a coffee :)

Please, Log in or Register to see links and images

2023-05-23 14:54:15,832:DEBUG:sql_helper:sql_helper.py:281:Successfully closed sqlite database connection



Is there anything that can be done?

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

It's scraping a coomer profile. Coomer takes a long time to scrape (as does cyberfile and kemono). Just leave it be, it's still working, it isn't frozen. If you don't want it you can add coomer to the skip in the config.

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

Wait a few hours/days. The file being under maintenance isn't something anyone can fix. you just have to wait.

 

[Xmtwx]

Hey Jules!

Thank you for the love and sharing this for me ignorant ass.

When I try to download from Bunkr; this is the error I get:

[notice] A new release of pip available: 22.3.1 -> 23.1.2
[notice] To update, run: python.exe -m pip install --upgrade pip

How do I update this?

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

Hey Xmtwx. That isn't an error. If you are getting failed downloads I'd need to see the downloader.log file.

That is an update is available notice for pip (part of python). Which you can just ignore. Or do what it says and run that command in command prompt.

 

[Danetta]

No matter what I put into URLs.txt I get, no valid links found when I run the batch file? Any help would be appreciated.

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

I'd need to see the urls.txt file and the downloader.log file.

 

[simonrule]

is it possible to tell the program to save to specific folder, i have bunch of links but i want them to be downloaded on one folder ive created, if yes how please.

thank you

 

[NeoFox]

Cyberdrop_DL.V4 not working at all now..I assume it's happened after update.
Last line in cmd I get is:

Please, Log in or Register to view codes content!

In downloader.txt this line only:

Please, Log in or Register to view codes content!

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

Another person had this issue recently. They updated python. Might be worth a shot for you. Go to python 3.10

 

[fapme123]

Hey, thanks for creating this.

I just wanted to know if it was possible when I'm scraping something like coomer.party, can I also scrape the description of the post? Like it'll download the "content" portion too where it has the video description? Thank you!

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

This doesn't currently have that capability, no

 

[grabo]

Ive waited few days and still its stuck at "Finished" scraping

ive updated python to latest version

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

how long are you leaving it running

 

[strolling potato]

Hey Jules, thanks for everything

I'm having a problem downloading kemono and coomer.

3 are the links that I'm trying to download

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

and this is the download logs

------------------------------------------------------------------------------------

Traceback (most recent call last):
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\crawlers\Coomeno_Spider.py", line 104, in parse_profile
soup = await session.get_BS4(url)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\client\client.py", line 40, in wrapper
return await func(self, *args, **kwargs)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\client\client.py", line 71, in get_BS4
async with self.client_session.get(url, ssl=self.client.ssl_context) as response:
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\client.py", line 1141, in __aenter__
self._resp = await self._coro
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\client.py", line 536, in _request
conn = await self._connector.connect(
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 1206, in _create_direct_connection
raise last_exc
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 988, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host coomer.party:443 ssl:False [The specified network name is no longer available.]
2023-05-27 11:40:19,579:DEBUG:base_functions:base_functions.py:54:Error:

Please, Log in or Register to see links and images

2023-05-27 11:40:19,579:DEBUG:base_functions:base_functions.py:134:Cannot connect to host coomer.party:443 ssl:False [The specified network name is no longer available.]
2023-05-27 11:40:19,579:DEBUG:base_functions:base_functions.py:54:Finished:

Please, Log in or Register to see links and images

2023-05-27 11:40:19,587:DEBUG:Coomeno_Spider:Coomeno_Spider.py:125:Error encountered while handling

Please, Log in or Register to see links and images

Traceback (most recent call last):
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 494, in finish_recv
return ov.getresult()
OSError: [WinError 64] The specified network name is no longer available.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1103, in create_connection
transport, protocol = await self._create_connection_transport(
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1133, in _create_connection_transport
await waiter
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 286, in _loop_reading
length = fut.result()
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 826, in _poll
value = callback(transferred, key, ov)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 498, in finish_recv
raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 64] The specified network name is no longer available.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\crawlers\Coomeno_Spider.py", line 104, in parse_profile
soup = await session.get_BS4(url)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\client\client.py", line 40, in wrapper
return await func(self, *args, **kwargs)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\client\client.py", line 71, in get_BS4
async with self.client_session.get(url, ssl=self.client.ssl_context) as response:
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\client.py", line 1141, in __aenter__
self._resp = await self._coro
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\client.py", line 536, in _request
conn = await self._connector.connect(
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 1206, in _create_direct_connection
raise last_exc
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\****\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\connector.py", line 988, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host kemono.party:443 ssl:False [The specified network name is no longer available.]
2023-05-27 11:40:19,588:DEBUG:base_functions:base_functions.py:54:Error:

Please, Log in or Register to see links and images

2023-05-27 11:40:19,589:DEBUG:base_functions:base_functions.py:134:Cannot connect to host kemono.party:443 ssl:False [The specified network name is no longer available.]
2023-05-27 11:40:19,589:DEBUG:base_functions:base_functions.py:54:Finished:

Please, Log in or Register to see links and images

2023-05-27 11:40:19,590:DEBUG:base_functions:base_functions.py:54:
2023-05-27 11:40:19,590:DEBUG:base_functions:base_functions.py:54:Finished Scrape

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

Long story short, this isn't an issue with the program, and I don't have a solid answer for you on what it could be. Could be antivirus acting up, could be dns caching issues on your computer (try restarting), are the sites blocked in your country?

But this is an issue with your computer, attempting to access the site.