Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

Sort By Date Sort By Reaction Score
Jules--Winnfield

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,182
5,124
1,127
0fya082315al84db03fa9bf467e3.png
3/26 Scrapers "completed" in the re-write. They take a long time to rewrite currently, it's also expanding upon what is already written in base functions and the SQL manager to support the new sites when needed.

I'll probably write one or two more then continue on to the downloader. Get it working so I can properly test as I go what is written. So far Anonfiles and Bunkr are written. With what is written currently it doesn't look like prior history will be able to be saved.
 
Last edited:
  • Like
  • HACKERMANS
Reactions: slitchampion, digital_warlord and hewo99
Jules--Winnfield

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,182
5,124
1,127
0fya082315al84db03fa9bf467e3.png
If you are interested in giving input on design decisions and beta testing the new version, shoot me a private message.
 
Lil' Sissy Amy

Lil' Sissy Amy

Bathwater Drinker
Mar 11, 2022
52
1,712
1,242
0fya082315al84db03fa9bf467e3.png
Hi guys, is there any working Kemono scraper? I tried AlphaSlayer but it doesn't work on kemono, only on coomer.
 
ausugnd

ausugnd

Dark Lord of the Simps
Aug 28, 2022
248
3,844
1,312
0fya082315al84db03fa9bf467e3.png
Jules--Winnfield said:
Please, Log in or Register to view quotes
Click to expand...
Also, because Jules doesn't say it himself and I'm a big believer of giving thanks where thanks is due, there is a "Buy me a Coffee" link on his GitHub page.
So if it's in thanks for the app, the constant and ongoing support/answers, or just to ply him with caffeine to speed him up for the production of V4, go have a look.

Please, Log in or Register to view spoilers
 
  • Like
  • PeepoLove
  • MLADY
Reactions: pvtrich, Theytime, hewo99 and 4 others
Horizon

Horizon

Tier 2 Sub
Mar 13, 2022
14
173
387
0fya082315al84db03fa9bf467e3.png
Is there a reason why certain files download significantly faster than others on bunkr? I'll consistently sit on 500kbs on some files, but some newer files give me upwards of 10mbs. also, is there a reason that I can only download 2 files at once, or have I not enabled something?
 
ausugnd

ausugnd

Dark Lord of the Simps
Aug 28, 2022
248
3,844
1,312
0fya082315al84db03fa9bf467e3.png
Horizon said:
Please, Log in or Register to view quotes
Click to expand...
Patience, and waiting. At the end of the day, it's a free service we get to use, so sometimes we have to put up with the downsides that come with it.
But they are working on constant improvement.

Queuing up multiple downloads with multiple batch file lines is good, I tend to run overnight and have them sorted into various folders for me so I'm not too worried about time, and it doesn't slow down any other browsing.
 
  • Like
Reactions: pvtrich
hewo99

hewo99

Catpain Blackudder
Jun 21, 2022
256
8,149
1,432
0fya082315al84db03fa9bf467e3.png
Jules--Winnfield said:
Please, Log in or Register to view quotes
Click to expand...
Might be shooting blanks here because I don't know the details. Would it be possible to create a separate script that runs a scrape based on the old database, solely and specifically to acquire the data/metadata you would like to have captured first time around? Appreciate this could take a long time to run, but it's just for migration so would be single use and could be separate to the main script.
 
  • Like
  • Brainlet
Reactions: nerlyk and axej
Jules--Winnfield

Jules--Winnfield

Cyberdrop-DL Creator
Mar 11, 2022
2,182
5,124
1,127
0fya082315al84db03fa9bf467e3.png
imag1e.md.png
image806408667a8c9af6.md.png

Some teasers. (Not even close to half done)
27 Hours of development logged though, progress has been made.
 
  • PeepoLove
  • Like
  • wowers
Reactions: whyohwhy69, ausugnd, Man of Culture and 7 others
P

pvtrich

Tier 2 Sub
Mar 17, 2022
72
232
237
0fya082315al84db03fa9bf467e3.png
Hi Jules,

i have problems downloading the following links:
Please, Log in or Register to see links and images
Please, Log in or Register to see links and images
Please, Log in or Register to see links and images
Please, Log in or Register to see links and images
Please, Log in or Register to see links and images

Keeps saying:
No links found during scraping, check passwords or that the urls are accessible
This program does not currently support password protected albums.

That said i don't remember seeing the .su domain before and it looks like the webpages have a new lay-out... might be new and not supported yet?

2023-02-04 22:29:07,652:DEBUG:proactor_events:proactor_events.py:630:Using proactor: IocpProactor
2023-02-04 22:29:07,687:DEBUG:base_functions:base_functions.py:61:We are running version 3.6.52 of Cyberdrop Downloader
2023-02-04 22:29:07,687:DEBUG:main:main.py:73:Creating Config
2023-02-04 22:29:07,687:DEBUG:base_functions:base_functions.py:142:Validating Config
2023-02-04 22:29:07,691:DEBUG:base_functions:base_functions.py:61:Getting Run Args
2023-02-04 22:29:07,693:DEBUG:base_functions:base_functions.py:205:Gathering Args
2023-02-04 22:29:07,693:DEBUG:main:main.py:97:Starting Cyberdrop-DL
2023-02-04 22:29:07,693:DEBUG:main:main.py:98:Using authorization arguments: {'pixeldrain_api_key': '!REDACTED!', 'simpcity_username': None, 'simpcity_password': '!REDACTED!', '*Blacklisted site*_username': None, '*Blacklisted site*_password': '!REDACTED!', 'xbunker_username': None, 'xbunker_password': '!REDACTED!'}
2023-02-04 22:29:07,693:DEBUG:main:main.py:99:Using file arguments: {'input_file': WindowsPath('URLs.txt'), 'output_folder': WindowsPath('Downloads'), 'output_last_forum_post_file': WindowsPath('URLs_last_post.txt'), 'db_file': WindowsPath('download_history.sqlite')}
2023-02-04 22:29:07,693:DEBUG:main:main.py:100:Using jdownloader arguments: {'jdownloader_enable': False, 'jdownloader_username': None, 'jdownloader_password': '!REDACTED!', 'jdownloader_device': None}
2023-02-04 22:29:07,694:DEBUG:main:main.py:101:Using runtime arguments: {'threads': 0, 'attempts': 10, 'required_free_space': 5, 'disable_attempt_limit': False, 'connection_timeout': 15, 'ratelimit': 50, 'throttle': 0.5, 'include_id': False, 'exclude_videos': False, 'exclude_images': False, 'exclude_audio': False, 'exclude_other': False, 'ignore_history': False, 'output_last_forum_post': False, 'separate_posts': False, 'mark_downloaded': False, 'proxy': None, 'skip_hosts': []}
2023-02-04 22:29:07,709:DEBUG:base_functions:base_functions.py:61:Starting Scrape
2023-02-04 22:29:07,710:DEBUG:base_functions:base_functions.py:61:Starting scrape of
Please, Log in or Register to see links and images

2023-02-04 22:29:07,812:DEBUG:Bunkr_Spider:Bunkr_Spider.py:74:Error encountered while handling
Please, Log in or Register to see links and images

Traceback (most recent call last):
File "C:\Users\XXX\AppData\Local\Programs\Python\Python310\lib\site-packages\cyberdrop_dl\crawlers\Bunkr_Spider.py", line 43, in fetch
build_id = json.loads(soup.select_one("script[id=__NEXT_DATA__]").get_text())
AttributeError: 'NoneType' object has no attribute 'get_text'
2023-02-04 22:29:07,813:DEBUG:base_functions:base_functions.py:61:Error scraping
Please, Log in or Register to see links and images

2023-02-04 22:29:07,813:DEBUG:Bunkr_Spider:Bunkr_Spider.py:76:'NoneType' object has no attribute 'get_text'
2023-02-04 22:29:07,813:DEBUG:base_functions:base_functions.py:61:Finished scrape of
Please, Log in or Register to see links and images

2023-02-04 22:29:07,814:ERROR:main:main.py:128:ValueError No links
2023-02-04 22:29:07,814:DEBUG:base_functions:base_functions.py:61:No links found during scraping, check passwords or that the urls are accessible
2023-02-04 22:29:07,814:DEBUG:base_functions:base_functions.py:61:This program does not currently support password protected albums.
2023-02-04 22:29:07,815:DEBUG:sql_helper:sql_helper.py:133:Successfully closed sqlite database connection
 
Last edited:
  • PepeHands
  • Like
Reactions: Arcotsi and slitchampion
You must log in or register to reply here.
Top Bottom