How does Leeky Forums protect against bitrot?

StonedTime
StonedTime

Most of you probably backup important files to external drives with redundancy, but what steps are you taking to prevent against "bitrot"', or subtle bit flips over time to faults in the storage media, kernel/filesystem errors etc.

All urls found in this thread:
http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/1/
https://www.youtube.com/watch?v=6qjUS746kUw
https://en.wikipedia.org/wiki/Error_detection_and_correction
BinaryMan
BinaryMan

I use redundancy.

I mean you just said it nigger.

JunkTop
JunkTop

@BinaryMan
I meant redundancy as in backing up to multiple locations, not any form of automated error correction. If you have two backed up files and one changes, which one is correct?

CodeBuns
CodeBuns

@JunkTop
The one with a correct checksum.

GoogleCat
GoogleCat

The Cloud

Fuzzy_Logic
Fuzzy_Logic

@CodeBuns
Would be nice to have that automated. This article talks about features for that in ZFS and BTRFS, so I am going to look into their features. Right now I am just using ext4.

http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/1/

New_Cliche
New_Cliche

@CodeBuns
but what if the checksum changes ...

Stupidasole
Stupidasole

@Fuzzy_Logic
Just use existing backup software instead of manually copying. I use Dejadup, a front-end for duplicity - it comes with encryption, incremental backups (saves a lot of space) and integrity checking.

@New_Cliche
In that case it doesn't have the correct checksum, so it gets rejected.

Lunatick
Lunatick

The only correct answer to this is ZFS, until BTRFS is finished.

Lord_Tryzalot
Lord_Tryzalot

@Lunatick
How far behind is BTRFS? Redhat and SUSE have moved to it as their default, they must have a lot of confidence in it.

massdebater
massdebater

@Lord_Tryzalot
there's many features disabled under opensuse
also fedora/redhat hasn't made it the default, not even planned in fedora 24

likme
likme

@StonedTime
bitrot
>thinking it describes gamma ray side effects
You keep using that word user.

hairygrape
hairygrape

@Lord_Tryzalot
Pretty sure RHEL's default is XFS.

Snarelure
Snarelure

@massdebater
also fedora/redhat hasn't made it the default, not even planned in fedora 24
Pardon me for misspeaking.

Emberburn
Emberburn

@likme
Bitrot can include errant gamma ray effects, and it is also used as a term for code bases which are left unmaintained.

Spamalot
Spamalot

@GoogleCat
I'm not sure if you meant that as a joke, but really any big cloud storage company worth its salt should take the issue of data integrity very seriously. The odds of data failing over those types of systems are probably far lower than than measures most consumers can accomplish.

w8t4u
w8t4u

I have zeroed out all my storage. Any bit that is not a 0 is guaranteed to be flipped and can thus be corrected.

Snarelure
Snarelure

Print everything.

Emberfire
Emberfire

I scrub my bits daily with a soft bristle toothbrush and some shave foam. Keeps them nice and tidy.

Firespawn
Firespawn

@Snarelure
Video and sound files are just memes anyway.

TurtleCat
TurtleCat

@Snarelure
Better be acid-free paper, what if ink corruption turned words like digger to nigger. Implying I care.

@Firespawn
Falling for the digital-meme.

@w8t4u
Can't argue with that until someone comes up with more requirements.

VisualMaster
VisualMaster

@Snarelure
@TurtleCat
falling for the paper meme

Ignoramus
Ignoramus

@VisualMaster
falling for the tablet meme
Sure, it's "portable", but it's not nearly as powerful or robust, and the storage is very limited.

Illusionz
Illusionz

@TurtleCat
Can't argue with that until someone comes up with more requirements.
"So, user, I notice on the design spec of you are proposal that, quote: ''All software shall be composed entirely of binary zeros." While an admirable goal, seems a bit difficult to sell to the customers. Care to elaborate on the implementation plan?

Fried_Sushi
Fried_Sushi

@Ignoramus
falling for the writing meme
Decent for communication, but when it comes to long term storage, we all know what to go with.

Need_TLC
Need_TLC

@Ignoramus
Falling for the geology meme
Not just encoding all storage in the fundamental forces of physics

Up you are game, brah.

Raving_Cute
Raving_Cute

@Illusionz
When you only use a single digit, you have a unary system. The unary system can be used to express any integer, so all you really have to do is translate the binary file to unary and you're good to go.

Lunatick
Lunatick

@Need_TLC
I tried a black hole array, but the hawking radiation really gets you after a few billion years. What do you recommend?

Sir_Gallonhead
Sir_Gallonhead

@Fried_Sushi
If you want real long term storage, you just can't beat stasis fields. 1²:2²:3² monoliths are fine for pushing species towards the next step in their interstellar development, but how do you know they will be around five billion years from now?

cum2soon
cum2soon

@Need_TLC
all digital media is effectively encoded as a really big number. if you just count up high enough you'll eventually get to the number that represents whatever file you want to save. you think that's an accident?

w8t4u
w8t4u

@JunkTop
By backing up to multiple locations you're in effect also preventing errors in the way you describe.

Sir_Gallonhead
Sir_Gallonhead

So the moral of the story is:

Use ZFS or maybe BRTFS
Use the cloud
Go back to paper/tablets/wall paintings/moniliths

StrangeWizard
StrangeWizard

Store all vital data in new HDDs
Store HDDs in an old Microwave (Microwaves provide RF shielding and a Faraday cage, protects them from magnets and the inevitable EMP strike)
protect data for all eternity forever

Harmless_Venom
Harmless_Venom

I would like to have a file-system independent bitrot detection tool, but I'm not aware of anything of the sort.

Methnerd
Methnerd

I put my disk ina condom

Lord_Tryzalot
Lord_Tryzalot

@StonedTime
does data on HDD's deteriorate?..... I thought that most HDD's keep data pristine for many years....

Soft_member
Soft_member

@StonedTime
I somewhere read that silicone starts to fall apart after about 40 years. So in a decade (more or less) my nes cartridges might start dying.

Other than this, I have a copy of all my warez on 2 Harddrives from different manufacturers. I don't use CRC or anything, space is cheap nowadays. 5TB drives are 150$ now

Sharpcharm
Sharpcharm

@Lunatick
Pocket dimension, fam.

Crazy_Nice
Crazy_Nice

@Lord_Tryzalot
Yes, read the Ars article.

CouchChiller
CouchChiller

@Soft_member
my nes cartridges might start dying.
The terminals are the most likely to die. No worries though, most NES games have been archived on the internet. Also for extreme hobbyists, you can buy programmable cartridges to load your own content.

TreeEater
TreeEater

R:39
None mention (WORM) physical media
One of many Reasons I left /tech/index.html 2months ago.

Snarelure
Snarelure

@TreeEater
"I left /tech/", he said, as he posted on Leeky Forums.

Emberfire
Emberfire

@TreeEater
kek at this emotional "in the arms of an angel" tier video https://www.youtube.com/watch?v=6qjUS746kUw

Lunatick
Lunatick

@Firespawn
he doesn't print all his media as bytecode
smh fam why even get out of bed every morning?

Harmless_Venom
Harmless_Venom

@TreeEater
Implying that optical media isn't effected by bit rot/medium rot?
Nice joke.

Sir_Gallonhead
Sir_Gallonhead

@Lunatick
Serious question here, why are people excited for BTRFS when ZFS already exists, why not just improve ZFS stability on platforms that it's not native to instead of making an entire new filesystem. When it comes to something like this I really do trust older systems more, ZFS has years of testing, documentation, tools, etc. that I can rely on or at worst fall back on. If BTRFS is supposed to be like ZFS but doesn't have any of this then why use it over the original?

Boy_vs_Girl
Boy_vs_Girl

@Sir_Gallonhead
Licenseing and Not Made Here syndrome is likely to be a big part of it.

iluvmen
iluvmen

@Sir_Gallonhead

It's not excitement. Btrfs supposed to be the successor of ext4, so it must be good on desktop too and Btrfs is much better option on Linux for RAID than ZFS.

PurpleCharger
PurpleCharger

I use Claude Shannon's theories from the 1940's to stay safe from bitrot

https://en.wikipedia.org/wiki/Error_detection_and_correction

BunnyJinx
BunnyJinx

@Boy_vs_Girl
@Sir_Gallonhead
licensing incompatibilities, not made here syndrome AND ZFS's outrageously suboptimal resource usage.

Disable your ad blocker to access the content

Disable your ad blocker to access the content