I think the one persistent problem with archiving content is that the file names always get jumbled up, the result is a lot of overlap and inefficiency. It seems like the only real solution is to create and maintain a database of checksum or hashes? However, this would require multiple people keeping track of it and in all likelihood, people would have to use another database utility that has plugins for web browsers, download managers and even your local/LAN storage file manager to prevent you from downloading-uploading files that already exist. I feel like somebody actually tried this back in the day, but it never went anywhere, I kind of feel like this is exactly what torrenting is for but ISPs are reasonably good at blocking the traffic. Plus, the program would have to integrate pretty seamlessly with your system which could be a privacy nightmare.