Tools Cyberdrop-DL: Mass Downloader for most forum sites and SimpCity threads

[hkei]

Jules,

In my opinion, to idiot proof your guide, I would not highlight the *.simpcity.su domain. For me, I had 3 domains appear, simpcity.su, *.simpcity.su, and simpcity.su ; I found that each different cookie had its own xf_session, and if you use the *.simpcity.su or the simpcity.su cookie xf_session instead, it won't work with C-DL.

For about five minutes, I was struggling until I realized that each domain entry may have its own session hash.

Edit: AND THANK YOU SIR! Great job on this quick turnaround for this fix for scraping SC, you are truly amazing!

 

[Deleted member 33517]

Please, Log in or Register to view quotes

Ok it's working now thanks for walking through this with me

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

Assuming that cookie is your actual cookie, you should delete that.

 

[Eredin Glas]

EDIT: Got it to work, i think i just didn't hit save or something because it was the same code i used before.

So i was getting the login error before but did what everyone else said and just deleted the login info and used the cookies, but now i'm just getting this error.

"Starting Cyberdrop-DL
We are running version 4.2.178 of Cyberdrop Downloader
Starting Scrape
Starting: *Name of thread*
Error: *Name of thread*

Finished Scrape"

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

see hkei's message above. I won't be walking people through the cookie method. You'll have to do trial and error.

 

[Jar of Pickles]

the session value from chrome was not working at all for me

but the one from firefox worked perfectly

 

[neolith]

From my experience, some final notes for people struggling to authenticate:

Ensure that the IP address you used to create your cookie is the same one you're using for downloading. This is particularly important if you're using a different machine for your scraping or your browser has its own vpn implementation. If the problem persists, you could attempt to match your user agent in the configuration with the one that was used to generate the cookie, or vice versa.

All else fails, try making a new cookie with firefox.

 

[Jar of Pickles]

Please, Log in or Register to view quotes

ok that was actually the real problem for me then. I forgot to turn off my chrome VPN extension so it used a different public IP than my machine.

 

[devilsdancefloor42]

Please, Log in or Register to view quotes

It's not. I just wanted to make sure I put in some plain text in case someone was struggling with quotes or something.

 

[kimmi690]

i put one link to url, contain 263 vids. Downloader finished 130 vids, but other vids can't be completed. and system said it completed download. how to solve continue download other vids left?

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

I'd need to see the downloader.log file.

 

[noxod]

easier method for getting xf session value:

 

[Giorgio1994]

Please, Log in or Register to view quotes

hey, was that rescraping that was suppose to help with this ever implemented?

I had some download attempts fail and then when I tried to rerun the start windows file, I get the following


| Files Complete: 20 - Files Skipped: 173 - Files Failed: 46 |

Checking for incomplete downloads
Purging empty directories
Finished downloading. Enjoy :)

If you enjoy using this program, please consider buying the developer a coffee :)


Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000025893669240>
Traceback (most recent call last):
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "C:\Users\\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\Users\\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
self._check_closed()
File "C:\Users\\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

E:\Cyberdrop_DL.V4>pause
Press any key to continue . . .


Looking at the downloader notepad file, I see 2 error codes. One of them is the same Service Temporarily Unavailable as above.

"

Please, Log in or Register to see links and images

(Unexpected status code 503 from

Please, Log in or Register to see links and images

)
2023-08-09 02:54:00,752:DEBUG:base_functions:base_functions.py:60:Failed Download: darlinnikkiqos_00081-raKiDox4.mp4
2023-08-09 02:54:07,387:DEBUG:base_functions:base_functions.py:60:Completed Download: darlinnikkiqos_00049-DZi8Wicw.mp4 from

Please, Log in or Register to see links and images

2023-08-09 02:54:08,416:DEBUG:downloaders:downloaders.py:287:

Please, Log in or Register to see links and images

(Service Temporarily Unavailable)
2023-08-09 02:54:08,416:DEBUG:downloaders:downloaders.py:303:

Please, Log in or Register to see links and images

(Service Temporarily Unavailable)"


And all the way at the bottom of the file:

2023-08-09 02:55:04,979:ERROR:base_events:base_events.py:1744:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x00000258959EF340>
2023-08-09 02:55:04,980:ERROR:base_events:base_events.py:1744:Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x00000258959D5F60>, 1165497.515)]']
connector: <aiohttp.connector.TCPConnector object at 0x00000258959ED390>
2023-08-09 02:55:04,981:DEBUG:base_functions:base_functions.py:60:
If you enjoy using this program, please consider buying the developer a coffee :)

2023-08-09 02:55:04,982:INFO:aiorun:aiorun.py:263:Entering shutdown phase.
2023-08-09 02:55:04,982:INFO:aiorun:aiorun.py:292:Cancelling pending tasks.
2023-08-09 02:55:04,982:DEBUG:aiorun:aiorun.py:297:Cancelling task: <Task pending name='Task-2' coro=<windows_support_wakeup() running at C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\aiorun.py:377> wait_for=<Future pending cb=[Task.task_wakeup()]>>
2023-08-09 02:55:04,983:INFO:aiorun:aiorun.py:350:Running pending tasks till complete
2023-08-09 02:55:04,983:INFO:aiorun:aiorun.py:359:Waiting for executor shutdown.
2023-08-09 02:55:04,984:INFO:aiorun:aiorun.py:363:Shutting down async generators
2023-08-09 02:55:04,984:INFO:aiorun:aiorun.py:365:Closing the loop.
2023-08-09 02:55:04,984:INFO:aiorun:aiorun.py:367:Leaving. Bye!
2023-08-09 02:55:04,986:DEBUG:sql_helper:sql_helper.py:281:Successfully closed sqlite database connection

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

the unclosed client session can be ignored, but to answer your first question, no it wasn't.

 

[maliksar]

It is possible to export cookies from browser, without any external tools

Please, Log in or Register to see links and images

 

[nsfwaccg]

Hey ! everytime i try to download anything is just skip everything without any explanation

Files Complete: 0 - Files Skipped: 413 - Files Failed: 0 | Checking for incomplete downloads Purging empty directories Finished downloading. Enjoy :)

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

it is, I just wanted to show the easiest possible route, as people tend to have a hard time following instructions.

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

You'd see why in the downloader.log file, but I can answer it for you. You've already downloaded all those files in the past. Or you have some different cli-args / config values that are excluding them from downloading.

 

[nsfwaccg]

Please, Log in or Register to view quotes

it says previoulsy downloaded even tho i never downloaded the file

 

[Jules--Winnfield]

Please, Log in or Register to view quotes

I'm sorry, but that's not possible. You'd have downloaded the files in the past, unless you have skip_download_mark_complete enabled.

You can use ignore_history in the config to bypass it and redownload them.