Scihub is what 100 tb ? That is like inr 80,000 + a server to hold drives say another 30,000 so in total around 110,000 or 120,000. That is around USD 1500. If I get a windfall of sorts, this is the thing I am going to spend on.
Donations, eh yeah but I feel like scihub needs everyone's support.
I happened to check Amazon a few minutes ago while writing another comment and found 18TB drives seem to be hovering around US$350. 6*18=108TB (and US$2100), however, that does not account for redundancy.
I recently learned the fun way that ZFS performance (incl read-only) absolutely tanks if the pool is full (the box was unusable until I deleted the data), so to use ZFS you'd want to add in a few TB of spare capacity. https://wintelguy.com/zfs-calc.pl says 10 drives ($3500) should provide 100TB of space.
Alternatively you could use dm-integrity (which is very new and does not yet have optimized I/O routines) and either layer RAID on top of that or manually duplicate the data yourself, which would let you just get exactly the number of disks. It's possible you could alternatively run Ceph with ECC, but I don't know if Ceph likes having some free space to work with.
This also does not take into account a server, but this does not need to be too expensive, probably US$750 or potentially even less.
I respond entirely out of curiosity; I also dream of backing up large datasets one day :D
my argument is just to provide redundancy. we do not need additional local redundancy because the data is out there. it would be a problem if my copy or your copy somehow becomes the only copy. then we would be concerned.
this says INR 11,000.00 which is $137.raw 100 would need around 18 so $2500.
i think there can be variations of disk sizes and amazon fluctuations but in cases like these i would have a business buy them because that way input tax of 18% would pass on to the business and conversely to me for lowering the final bill. so, $2118 would be for these drives. same for the server. a tax benefit of 18%.
Huh. I seriously need to graph the size vs price thing and figure out where the sweet spots are. I last did that sort of thing a few years ago, got completely out of the game. I just googled a bit and discovered 18TB drives are a thing and went and looked for those. lol
I guess you're right about not needing local redundancy... although this is very much at odds with the "disk is over 500GB, redundancy required!!11" beeper in my head :D
Donations, eh yeah but I feel like scihub needs everyone's support.