Question Storage solutions

[Jackiehung]

Hello guys,

I've ran into a little storage problemo; I've accumulated about ~130 OF models all their videos, nicely organized and categorized, duplicates removed etc. But I'm running out of space.

I've read that bunkr deletes files and gofile, well, from this forum I've seen that they can last for a few hours even.

What solutions would you have for terabytes of videos? I can share them to public if necessary, but I suspect that it would get them taken down faster. Does MEGA delete videos if they are private?

 

[vlad0202]

Please, Log in or Register to view quotes

Don't trust any cloud solution, never. Cloud / stream = "this not your datas, it's their".
Local storage, nass for example, with a backup solution (always, hdd fail ! (and will) ).
Sure not that simple to use and configure, it cost with you buy it but it will cost you less quickly. With raid you can get some nice size with security.
Yes, you won't be able to watch your porn when out of the house, but seriously, no one can't spend few day in holidays off !? (ok, easy for me, i'm usually hiking out of nowhere, so no connection anyways...)


How much data are you talking ? I'm going to setup a new one myself because my of/insta growing fast to I'm going for a simple usb raid with probably 2 x 4to 3" hdd.

 

[yticpmis234]

vlad0202 is right , the correct solution is always your own private+encrypted Network Attached Storage (NAS) at home. You can always access it over your own VPN from anywhere if you set it up that way.

A good place to start is to read the following stickied posts , they will help you figure out if the cost and work learning curve involved is worth the effort for your use case.

• How To: Choosing Your Storage Devices and Configuration - Network Attached Storage and Home Servers
  Please, Log in or Register to see links and images
• How To: Access your Network Attached Storage from anywhere
  Please, Log in or Register to see links and images
• Someones recent thread about using a cheap (i.e from Goodwill, or from the office/school/college dumpster, estate sale) old Dell-Optiplex as a NAS
  Please, Log in or Register to see links and images
• Indicative prices for hard drives ( assuming you are in the US )
  Please, Log in or Register to see links and images

 

[PierfrancescoFava]

Please, Log in or Register to view quotes

Mega does delete your private files if you haven't logged in your account for a while, I think it's around 12 months

A nas is the right choice, in the long run it will cost less than the other options and you can easily access your files anywhere with plex/jellyfin

 

[Jackiehung]

Please, Log in or Register to view quotes

About 10TB at the moment. And it will increase every day.

I do not need to have access to it at all times. I have very fast internet (1gbps and can go upto 10gbps) I just need a cost/space efficient solution. I really don't think I can afford to invest thousands of dollars into NAS.

Could you also recommend how to store files more efficiently? Should I zip/Rar them or somehow reduce the size on disks without losing quality?

At the moment I feel I'm going to go with mega, for 30$/Month they offer 16TB, I also saw on other forums people selling cheap accs. And perhaps also with bunkr. Are there no websites that would accept such large files?

 

[vlad0202]

Please, Log in or Register to view quotes

ok so if access outside of your house is not mandatory, for a cost efficient solution look into usb multi hdd (2/4/5) rack with jbod or raid support, like those from yottamaster. With jbod (or raid 0) you can go up to a single 80To volume.

So a 4 bays is around 170€, than around 180€ per 16To hdd. cost less than a year subscription. more space ? add drive. Yes you need a backup solution, but would you trust mega ? really ? so you have to have your own backup drive anyway.

That what I'm looking into myself, some kind of storage issue too ;( and all those portable hdd start getting unpractical (but going to be the backup).

Jbod is the most interesting for "home" storage. You create a huge drive you can access as any usb drive, watch, stream direclty It's just an usb drive, bigger. With raid 0 you get a hude drive too, more speed but you need the exact same drive, I wont recommand for home unless you want to stream 8k video.

Local vs cloud, up to you. Cloud is easier until they decide to cancel your souscription (or just close) and you lost everything. Already happen for 'copyright violation'. And what happen when you out of network ? you s**** ! This happen to me for work, sass what the way to go until the fiber line was cut, and than, no more data, no more software etc.

 

[Jackiehung]

Please, Log in or Register to view quotes

I think I'll just get a crapton of various HDD's, try to find cheap versions. Used ones are about 10$ per tb...

The point is that I don't want to spend thousands on this collection of mine.

 

[kuraneskadath]

Late to the party on this, but if you're willing to sacrifice quality, you could also look into re-encoding the video files. ~5TB here, and rather than buying more space I'm doing two things:
- I'm lowering the resolution on the 4K videos to 1080p. I'm not playing this stuff on a super huge monitor, so it makes zero difference but reduces the file sizes to about 1/4.
- I'm lowering the quality of the videos. I've found that a LOT of models appear to be kinda lazy with what they upload, so I've been able to reduce the size of ~2GB videos to around 300mb sometimes, with no perceptible difference in quality.
- A lot of models will upload a ton of livestreams, and most of the time these can be edited down from hours to ~15 mins of actual interesting content. This results in a 2gb file turning into ~150-200mb.

There's a time investment, but at least you wont have to pay thousands. Maybe just hundreds, lol.

 

[0128309812093]

I don't know if I can make a worthwhile contribution given that so much valuable information has already been shared, but a couple of ideas come to mind:
-Switch from H.264 to H.265, which offers 25% to ~ 50% better data compression at the same quality level.
- Use something like frame deduplication. You might have to Google how to do this in an automated or scripted way, but you can remove "double" or "duplicated" frames, for example when the model is away for a couple seconds/minutes or not really moving. It can really save on size but you will need to experiment a lot to get it right.
- As others have noted, use a different resolution/bitrate. Also you can use a different media container (MP4 vs MOV vs etc. for example, do your research as to what is best).
- Use hot/cold storage. Hot storage is storage that is immediately availabe, and cold storage is traditionally in the form of tapes that you will have to dig up form the suitcases they care contained in, but cold storage can also mean hard drives or other media that is not immediately available. You can just store your stuff on a HDD, unplug it and use that space for another hard drive while you put the one that is full to the side (somewhere safe).
- Remove the audio from the video if it is not relevant. This is of course highly conditional but if you don't care, you might as well trim it.
- Edit or cut the videos into segments that actually matter, although you mentioned having terabytes worth of data so I don't know how relevant this really is.

 

[jacklupino6]

Converting pictures from PNG to JPG using Bulk Image Converter as well as compressing videos using Handbrake freed up several terabytes for me. Also external hard drives (and hard drives in general) are pretty damn cheap and during black friday you can get some really large drives for a good price.

 

[heycitizen]

I'm currently having the same problem. I'm thinking about s3 style bucket hosting.

AWS is the most expensive, The basic summary is:

> $0 to transfer in
> $0.023/GB for the first 50TB to store the file
> $0.09/GB for the first 10TB to download the files

Please, Log in or Register to see links and images

But there is a lot cheaper:

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Please, Log in or Register to see links and images

Couple this with an app like:
>

Please, Log in or Register to see links and images

>

Please, Log in or Register to see links and images

>

Please, Log in or Register to see links and images

(which now has a web ui)

 

[SCSU007]

hardware is dirt cheap now. Just get one or more 2.5" drives and slap them in an enclosure.

Can be done for around $100