Tools Ways to rip entire TikTok profile?

[Beakwood]

Thanks, richard2480 seems to be working.
50%|##### | 166/332 [07:20<51:08, 18.49s/it]
But, As you can see, it seems like it's going to slow down. Is this a rate-limit thing from Tiktok?
Or will 'self.max_connections' help in this situation and it's not a rate limit?

Please, Log in or Register to view quotes

I just checked the GitHub link provided by the program creator, and it's discontinued. Well, at least, the link given from the program.

Please, Log in or Register to see links and images

Also, the discord link provided from the UrlTok doesn't work. So, yes, I believe the project is abandoned or discontinued.
There's been numerous programs offered in this thread. Give them a try. The easiest in your case would probably be 4k_Tokkit. But, the other programs mentioned here are good if you're willing to give a little try.

 

[Thighwanese]

Please, Log in or Register to view quotes

I installed the plug-in and it still downloads watermarked at SD.

This is what the terminal looks like. I'm not that tech savvy so idk what any of these really means. I'm on macOS.

Please, Log in or Register to view quotes

 

[Beakwood]

Please, Log in or Register to view quotes

It's working for me. But, then again, I am also using nightly ytp. Below you will find screenshot of vids with no watermark, list of files being downloaded, and debug of my ytp with config

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

You can find nightly ytp

Please, Log in or Register to see links and images

 

[ShagadelicFembot]

Please, Log in or Register to view quotes

Can you reup?

 

[Obicase]

Please, Log in or Register to view quotes

Read the thread dude. I had already re-uploaded it here, but don't bother because it doesn't work anymore.

Instead, you can check out :
- the python program from an anon on sturdy that I posted here (I update the link following his updates)
- neolith and beakwood's yt-dlp solution
- richard2480's scraper

I personally don't recommend 4Ktokkit because it's close sourced and doesn't download in HD.

 

[clbrn]

Please, Log in or Register to view quotes

Thanks! I totally missed this... for some reason, it works for some profiles and not for others, but I'll investigate further... maybe for profiles that had a crash during scraping (it happens from time to time) it will still try and re-download everything, overwriting the already present files in the folde, because no json file was created after completion).

Can I ask you few more things:

- What's the purpose of self.early_check_file_exists (vs self.check_file_exists)

- Is there a way to append the username to the saved filenames?

- I love the date feature (self.save_with_date), but how to save with no date in the filename (only the file date modified attribute changes to tiktok creation date)? Ot alternatively, use date only (2024-04-03 and no timestamp) after the id, so we have id first, date second. My goal is to have something like: 6999227643447069957 - 2021-08-22 - bethany_tomlinsonn.mp4

Thank you once again for the great script and helping the community

 

[neolith]

Please, Log in or Register to view quotes

1: early_check_file_exists builds a list of video_id's from that current users folder, and then compares those while building the video_id list. This saves a ton of time if you have to rescrape a lot of accounts that already have content, but you dont have a history file. I.e. say you have everything from a week+ ago, but nothing since. It will stop building the list once it gets to what you already have. This only works if the files currently in the user's folder contain the video_id somewhere in the name. If that isn't your use case, probably not worth enabling. You essentially do this when you have a history file, but this is a pseudo history built from local files.

2: check_file_exists checks if the current save_name.mp4 already exists when downloading. this is so you dont overwrite existing files.

3: For your save_name issue:

self.date_format = "%Y-%m-%d_%H-%M-%S"

First needs to be trimmed to: self.date_format = "%Y-%m-%d"

Then set self.save_with_date = False to True

Then the prepend_date() func can be modified. Probably not the best way to do this, but the easiest.

Original:

Please, Log in or Register to view codes content!

New:

Please, Log in or Register to view codes content!

make sure to backup first cause i could have messed up something simple

 

[Sektor89]

Just use Tokkit (download all vids from a profile without watermarks)

Please, Log in or Register to see links and images

You find cracks for full version all over the internet

 

[Beakwood]

Please, Log in or Register to view quotes

This was a nice take. I had given it a go. Just to see if it would save the filename with the original user name. It was a no go for me. Just didn't work. Although, I will say, if you save the code with the "new code" you provided. The filename will NOT add the username to the filename.
I went a step further and edited

Please, Log in or Register to view codes content!

to

Please, Log in or Register to view codes content!

This just resulted in various TASK errors.

Please, Log in or Register to view codes content!

But, this is one of the reasons why I do enjoy ytp, because of the original user name in the filename. But this program is still faster than ytp.

 

[clbrn]

Thanks for the replies and information, would definitely mess with this soon... unfortunately, the script suddenly stopped working for me today.
Any ideas what this error(s) below means (I didn't mess with the script). TikTok and TikWM are working fine... Thanks!

Please, Log in or Register to view codes content!

 

[neolith]

Please, Log in or Register to view quotes

turns out i did miss something stupid! my changes but also

self.save_with_date = False

set to true

 

[Beakwood]

Please, Log in or Register to view quotes

You may need to re-rerun

Please, Log in or Register to view codes content!

Please, Log in or Register to view quotes

Using ONLY the edit you provided in the previous post to the code and changing the False to True.
I still receive the task errors.
It's working for you, the username is being added to filename when adding True and using ONLY this edit in the code?

Please, Log in or Register to view codes content!

Nothing added to async def download_image or async def download_single_file ?

 

[neolith]

Please, Log in or Register to view quotes

i have my own fork with a ton of changes, and also only run this via linux, so i'm purely brainstorming for these. only thing i can think of is try removing the spaces from the save_name part. and try changing original_user_name to just user_name, or we just forgot the extension since we dropped save_name. if that still does nothing, i can try it on windows later tonight.

Please, Log in or Register to view quotes

date_string will ONLY work if it's in prepend_date btw, since the string isn't defined elsewhere

 

[Beakwood]

Please, Log in or Register to view quotes

Oh! Okay, that makes sense. You're using a linux environment.
Yeah, I just tried again, and it seems like it was about to download but then it cancelled/error'd out.
3%|█▊ | 1/36 [00:00<00:13, 2.59it/s]T

This is part of the error

Please, Log in or Register to view codes content!

 

[neolith]

Please, Log in or Register to view quotes

ahh, closer. let's try

Please, Log in or Register to view quotes

 

[Beakwood]

Please, Log in or Register to view quotes

Ha! That was it. Good work neolith
Found 36 new video IDs Video download complete. Downloading any remaining images.

 

[richard2480]

Beakwood not sure if you're running into a rate limit. increasing self.max_connections or setting self.asyncio_error_sleep_time (line 114) to 0 will increase the speed but will make errors more likely.

clbrn try updating playwright

Please, Log in or Register to view codes content!

 

[clbrn]

Please, Log in or Register to view quotes

Updated, videos downloaded, but photos failed (I'm not sure this profile even has photos)... interesting, because just hours ago everything was working great (with the older playwright, too)...

Please, Log in or Register to view codes content!

 

[neolith]

Please, Log in or Register to view quotes

i get this error when the history json for the user has a failed entry, and the failed entry no longer exists. just delete those lines if present. otherwise it may be a rate limit. but im like 75% sure the key error on images is what i said

 

[clbrn]

Please, Log in or Register to view quotes

Unfortunately, in my case there is no _history.json file created for this user, just aliases and secuid. The _history.json is only created after a profile successfully finishes downloading all items - if an error pops-up at any time, there is no _history.json in the user's metadata folder (hence the redownload of all previously downloaded items - if I rerun the script for the same user, it will download and overwrite all videos again, regardless if self.check_file_exists = True or self.early_check_file_exists = True ...)