Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

AC1DFOX

Superfan
Oct 20, 2022
96
617
729
0fya082315al84db03fa9bf467e3.png
I just followed the instructions here and tried running the Windows bat file, and get the error message:
G:\Cyberdrop_DL.V3>pip install --upgrade cyberdrop-dl
'pip' is not recognized as an internal or external command,
operable program or batch file.

Any Idea whats wrong? I downloaded and installed Python, then downloaded and extracted the Cyberdrop_DL.V3 archive and extracted it to the drive I wanted to download to. I tried running the bat file at that point.
 

AC1DFOX

Superfan
Oct 20, 2022
96
617
729
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
I really thought I did. However, I uninstalled and reinstalled it again and it now opens, meaning I hadn't done it right in the first place. I am sorry for wasting your time. I will be more cautious in the future.

However, I am now getting this error:

G:\Cyberdrop_DL.V3>pip install --upgrade cyberdrop-dl
Collecting cyberdrop-dl
Using cached cyberdrop_dl-3.5.6-py3-none-any.whl (60 kB)
Collecting aiofiles>=22.1.0
Using cached aiofiles-22.1.0-py3-none-any.whl (14 kB)
Collecting aiohttp>=3.8.3
Using cached aiohttp-3.8.3-cp311-cp311-win_amd64.whl (317 kB)
Collecting beautifulsoup4>=4.11.1
Using cached beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
Collecting certifi>=2022.9.24
Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting colorama>=0.4.5
Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting gofile-client>=1.0.1
Using cached gofile_client-1.0.1-py3-none-any.whl (7.3 kB)
Collecting myjdapi>=1.1.6
Using cached myjdapi-1.1.6-py3-none-any.whl (13 kB)
Collecting PyYAML>=6.0
Using cached PyYAML-6.0-cp311-cp311-win_amd64.whl (143 kB)
Requirement already satisfied: setuptools>=65.5.0 in c:\users\rethi\appdata\local\programs\python\python311\lib\site-packages (from cyberdrop-dl) (65.5.0)
Collecting tqdm>=4.64.1
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting yarl>=1.8.1
Using cached yarl-1.8.1.tar.gz (172 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting attrs>=17.3.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting charset-normalizer<3.0,>=2.0
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting multidict<7.0,>=4.5
Using cached multidict-6.0.2.tar.gz (50 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting async-timeout<5.0,>=4.0.0a3
Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting frozenlist>=1.1.1
Using cached frozenlist-1.3.1.tar.gz (66 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting aiosignal>=1.1.2
Using cached aiosignal-1.2.0-py3-none-any.whl (8.2 kB)
Collecting soupsieve>1.2
Using cached soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting requests
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting pycryptodome
Using cached pycryptodome-3.15.0-cp35-abi3-win_amd64.whl (1.9 MB)
Collecting idna>=2.0
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Building wheels for collected packages: yarl, frozenlist, multidict
Building wheel for yarl (pyproject.toml) ... error
error: subprocess-exited-with-error

Ă— Building wheel for yarl (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [37 lines of output]
C:\Users\rethi\AppData\Local\Temp\pip-build-env-ja4_b_vp\overlay\Lib\site-packages\setuptools\config\setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
**********************
* Accelerated build *
**********************
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-311
creating build\lib.win-amd64-cpython-311\yarl
copying yarl\_quoting.py -> build\lib.win-amd64-cpython-311\yarl
copying yarl\_quoting_py.py -> build\lib.win-amd64-cpython-311\yarl
copying yarl\_url.py -> build\lib.win-amd64-cpython-311\yarl
copying yarl\__init__.py -> build\lib.win-amd64-cpython-311\yarl
running egg_info
writing yarl.egg-info\PKG-INFO
writing dependency_links to yarl.egg-info\dependency_links.txt
writing requirements to yarl.egg-info\requires.txt
writing top-level names to yarl.egg-info\top_level.txt
reading manifest file 'yarl.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.cache' found anywhere in distribution
warning: no previously-included files found matching 'yarl\*.html'
warning: no previously-included files found matching 'yarl\*.so'
warning: no previously-included files found matching 'yarl\*.pyd'
no previously-included directories found matching 'docs\_build'
adding license file 'LICENSE'
writing manifest file 'yarl.egg-info\SOURCES.txt'
copying yarl\__init__.pyi -> build\lib.win-amd64-cpython-311\yarl
copying yarl\_quoting_c.pyi -> build\lib.win-amd64-cpython-311\yarl
copying yarl\_quoting_c.pyx -> build\lib.win-amd64-cpython-311\yarl
copying yarl\py.typed -> build\lib.win-amd64-cpython-311\yarl
running build_ext
building 'yarl._quoting_c' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools":
Please, Log in or Register to see links and images

[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for yarl
Building wheel for frozenlist (pyproject.toml) ... error
error: subprocess-exited-with-error

Ă— Building wheel for frozenlist (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [33 lines of output]
*********************
* Accelerated build *
*********************
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-311
creating build\lib.win-amd64-cpython-311\frozenlist
copying frozenlist\__init__.py -> build\lib.win-amd64-cpython-311\frozenlist
running egg_info
writing frozenlist.egg-info\PKG-INFO
writing dependency_links to frozenlist.egg-info\dependency_links.txt
writing top-level names to frozenlist.egg-info\top_level.txt
reading manifest file 'frozenlist.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.lib' found anywhere in distribution
warning: no previously-included files matching '*.dll' found anywhere in distribution
warning: no previously-included files matching '*.a' found anywhere in distribution
warning: no previously-included files matching '*.obj' found anywhere in distribution
warning: no previously-included files found matching 'frozenlist\*.html'
no previously-included directories found matching 'docs\_build'
adding license file 'LICENSE'
writing manifest file 'frozenlist.egg-info\SOURCES.txt'
copying frozenlist\__init__.pyi -> build\lib.win-amd64-cpython-311\frozenlist
copying frozenlist\_frozenlist.pyx -> build\lib.win-amd64-cpython-311\frozenlist
copying frozenlist\py.typed -> build\lib.win-amd64-cpython-311\frozenlist
running build_ext
building 'frozenlist._frozenlist' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools":
Please, Log in or Register to see links and images

[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for frozenlist
Building wheel for multidict (pyproject.toml) ... error
error: subprocess-exited-with-error

Ă— Building wheel for multidict (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [59 lines of output]
*********************
* Accelerated build *
*********************
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-311
creating build\lib.win-amd64-cpython-311\multidict
copying multidict\_abc.py -> build\lib.win-amd64-cpython-311\multidict
copying multidict\_compat.py -> build\lib.win-amd64-cpython-311\multidict
copying multidict\_multidict_base.py -> build\lib.win-amd64-cpython-311\multidict
copying multidict\_multidict_py.py -> build\lib.win-amd64-cpython-311\multidict
copying multidict\__init__.py -> build\lib.win-amd64-cpython-311\multidict
running egg_info
writing multidict.egg-info\PKG-INFO
writing dependency_links to multidict.egg-info\dependency_links.txt
writing top-level names to multidict.egg-info\top_level.txt
reading manifest file 'multidict.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files found matching 'multidict\_multidict.html'
warning: no previously-included files found matching 'multidict\*.so'
warning: no previously-included files found matching 'multidict\*.pyd'
warning: no previously-included files found matching 'multidict\*.pyd'
no previously-included directories found matching 'docs\_build'
adding license file 'LICENSE'
writing manifest file 'multidict.egg-info\SOURCES.txt'
C:\Users\rethi\AppData\Local\Temp\pip-build-env-6pns1o45\overlay\Lib\site-packages\setuptools\command\build_py.py:202: SetuptoolsDeprecationWarning: Installing 'multidict._multilib' as data is deprecated, please list it in `packages`.
!!


############################
# Package would be ignored #
############################
Python recognizes 'multidict._multilib' as an importable package,
but it is not listed in the `packages` configuration of setuptools.

'multidict._multilib' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).

Please make sure that 'multidict._multilib' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).

You can read more about "package discovery" and "data files" on setuptools
documentation page.


!!

check.warn(importable)
copying multidict\__init__.pyi -> build\lib.win-amd64-cpython-311\multidict
copying multidict\py.typed -> build\lib.win-amd64-cpython-311\multidict
running build_ext
building 'multidict._multidict' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools":
Please, Log in or Register to see links and images

[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for multidict
Failed to build yarl frozenlist multidict
ERROR: Could not build wheels for yarl, frozenlist, multidict, which is required to install pyproject.toml-based projects

G:\Cyberdrop_DL.V3>cyberdrop-dl
'cyberdrop-dl' is not recognized as an internal or external command,
operable program or batch file.

G:\Cyberdrop_DL.V3>pause
Press any key to continue . . .
 

anitajackoff

Tier 3 Sub
Mar 11, 2022
10
260
477
0fya082315al84db03fa9bf467e3.png
Tried this script for the first time but it only downloaded 4 of the 29 bunkr links.
I uninstalled and re-installed python 3.10.4 (made sure 'Add to path' is ticked) and it was 6 of 29 this time so not sure what's going wrong.

I uploaded the downloader.log to gofile
Please, Log in or Register to see links and images
 

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,198
5,223
1,127
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
You're running into a fun issue, where the bunkr pages aren't fully loading. Unfortunately, there isn't anything I've found that can mitigate this or help it, or the direct cause of it. I've brought it up with Bunkr on a few separate occasions.

I've found vpning to a different area can help, it seems to be a cdn issue.
 
  • Like
Reactions: anitajackoff

T4C0

Taco Papi
Apr 15, 2022
888
1
4,784
1,223
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
We are working on moving away from nextjs which is why sometimes you see the "loading" page with nothing while it builds the cache.

Everything will be cached on memory (RAM) and this should no longer happen (As this loading screen will no longer exist).

That being said, we are still really really early on the dev of the new system as we have to completely build it from 0, the front-end that is. and then sync it with the current backend (app.bunkrr.su).

In short, is a lot of work that is going to take more than a few weeks.
 

GarethFreeman

Diamond Tier
Mar 12, 2022
130
906
949
0fya082315al84db03fa9bf467e3.png
Sorry, how do you manually upgrade? Mine seems to be stuck on 3.5.14 and won't work automatically. The kemono rip keeps getting stuck as well.
 

jasnee41

Lurker
May 21, 2022
5
14
78
0fya082315al84db03fa9bf467e3.png
Hey Jules,
having a problem configuring jdownloader to work; keeps saying 'failed jdownloader setup'.

I've pasted part of the log below, hope you can help. Cheers

2022-10-29 22:00:26,399:DEBUG:proactor_events:proactor_events.py:626:Using proactor: IocpProactor
2022-10-29 22:00:26,456:DEBUG:base_functions:base_functions.py:59:We are running version 3.5.14 of Cyberdrop Downloader
2022-10-29 22:00:26,456:DEBUG:main:main.py:70:Creating Config
2022-10-29 22:00:26,457:DEBUG:base_functions:base_functions.py:121:Validating Config
2022-10-29 22:00:26,468:DEBUG:base_functions:base_functions.py:59:Getting Run Args
2022-10-29 22:00:26,471:DEBUG:main:main.py:93:Starting Cyberdrop-DL
2022-10-29 22:00:26,472:DEBUG:main:main.py:94:Using authorization arguments: {'simpcity_password': '!REDACTED!', 'simpcity_username': 'jasnee41', '*Blacklisted site*_password': '!REDACTED!', '*Blacklisted site*_username': 'jasnee4', 'xbunker_password': '!REDACTED!', 'xbunker_username': None}
2022-10-29 22:00:26,472:DEBUG:main:main.py:95:Using file arguments: {'db_file': WindowsPath('download_history.sqlite'), 'input_file': WindowsPath('URLs.txt'), 'output_folder': WindowsPath('Downloads'), 'output_last_forum_post_file': WindowsPath('URLs_last_post.txt')}
2022-10-29 22:00:26,472:DEBUG:main:main.py:96:Using jdownloader arguments: {'jdownloader_device': 'JD', 'jdownloader_enable': True, 'jdownloader_password': '!REDACTED!', 'jdownloader_username': ''!REDACTED!''}
2022-10-29 22:00:26,472:DEBUG:main:main.py:97:Using runtime arguments: {'attempts': 3, 'connection_timeout': 10, 'disable_attempt_limit': False, 'exclude_audio': False, 'exclude_images': False, 'exclude_other': False, 'exclude_videos': False, 'ignore_history': False, 'include_id': False, 'mark_downloaded': False, 'output_last_forum_post': False, 'proxy': None, 'ratelimit': 50, 'separate_posts': False, 'skip_hosts': [], 'threads': 0, 'throttle': 0}

2022-10-29 22:00:26,679:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:26,679:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:26,680:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:26,680:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images


2022-10-29 22:00:34,912:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.jdownloader.org:443
2022-10-29 22:00:35,073:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /my/connect?email=iron83564%40gmail.com&appkey=CYBERDROP-DL&rid=1667077234&signature=1e1a7d243392ec4707d3d0155ea5807a513330d4a9b40ba718ebfed93b059fe7 HTTP/1.1" 403 46
2022-10-29 22:00:35,074:DEBUG:base_functions:base_functions.py:59:Failed jdownloader setup
2022-10-29 22:00:35,074:DEBUG:base_functions:base_functions.py:59:Sending
Please, Log in or Register to see links and images
to JDownloader
2022-10-29 22:00:35,075:DEBUG:base_functions:base_functions.py:59:Failed to send
Please, Log in or Register to see links and images
to JDownloader
2022-10-29 22:00:35,076:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:35,076:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:35,076:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:35,077:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:35,283:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /createAccount HTTP/1.1" 200 67
2022-10-29 22:00:35,285:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:35,595:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /getContent?token=T9Wv66Wwl3ot7SfSLAXOvgA9u7Jnpv6a&contentId=Nhg5dR&websiteToken=12345 HTTP/1.1" 200 37
2022-10-29 22:00:35,596:DEBUG:base_functions:base_functions.py:59:Error scraping
Please, Log in or Register to see links and images

2022-10-29 22:00:39,103:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,103:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,105:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:39,319:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /getContent?token=T9Wv66Wwl3ot7SfSLAXOvgA9u7Jnpv6a&contentId=XAlg15&websiteToken=12345 HTTP/1.1" 200 37
2022-10-29 22:00:39,320:DEBUG:base_functions:base_functions.py:59:Error scraping
Please, Log in or Register to see links and images

2022-10-29 22:00:39,320:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,321:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,323:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:39,464:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /getContent?token=T9Wv66Wwl3ot7SfSLAXOvgA9u7Jnpv6a&contentId=kfD5dI&websiteToken=12345 HTTP/1.1" 200 37
2022-10-29 22:00:39,466:DEBUG:base_functions:base_functions.py:59:Error scraping
Please, Log in or Register to see links and images

2022-10-29 22:00:39,466:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,467:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:39,467:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:41,084:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:41,085:DEBUG:base_functions:base_functions.py:59:
Please, Log in or Register to see links and images
is not supported currently.
2022-10-29 22:00:41,096:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,096:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,198:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:41,399:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /getContent?token=T9Wv66Wwl3ot7SfSLAXOvgA9u7Jnpv6a&contentId=pq9dMc&websiteToken=12345 HTTP/1.1" 200 837
2022-10-29 22:00:41,401:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,401:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,403:DEBUG:connectionpool:connectionpool.py:939:Starting new HTTPS connection (1): api.gofile.io:443
2022-10-29 22:00:41,587:DEBUG:connectionpool:connectionpool.py:433:
Please, Log in or Register to see links and images
"GET /getContent?token=T9Wv66Wwl3ot7SfSLAXOvgA9u7Jnpv6a&contentId=DUrUZx&websiteToken=12345 HTTP/1.1" 200 837
2022-10-29 22:00:41,589:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,589:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,599:DEBUG:base_functions:base_functions.py:59:Finished scrape of
Please, Log in or Register to see links and images

2022-10-29 22:00:41,599:DEBUG:base_functions:base_functions.py:59:Starting scrape of
Please, Log in or Register to see links and images
 

axej

(◕‿◕✿) STYLE (◠‿◠)✌ START (◠‿◠✿)PORNHOARD
Mar 11, 2022
276
3,801
1,299
0fya082315al84db03fa9bf467e3.png
does the .party sites no longer do posts as folders after 3.5.11?
 

axej

(◕‿◕✿) STYLE (◠‿◠)✌ START (◠‿◠✿)PORNHOARD
Mar 11, 2022
276
3,801
1,299
0fya082315al84db03fa9bf467e3.png
Please, Log in or Register to view quotes
i used to hate it, but now i'm used to it i guess. for some i like it, usually sfm or comics/artists. generally anyone that has like a series or whatever that should be organized into folders.

idk if i want it back to using that flag as default just for that site.

the one other place where its useful is when archives have passwords. makes it easier to find the post with the password. sometimes the downloaded file name differs from what's listed on the site/post for whatever reason