I had an alert from my NAS at lunchtime. One of the drives in a mirrored RAID array had tripped a "bad sector" trigger on the SMART monitoring.
It is continuing to work, dependent on the non-failing disk mirrored data.
Given my monthly hardware scan of the disk had just started, I don't think the alert was a coincidence.
In theory, I can just power off, remove the failing drive and replace it with one of the same or greater capacity, power back on and it will repopulate the new disk.
In practice, it's been running 16 hours a day for 6½ years, and I can't easily source a like-for-like replacement (as in all mirroring, the recommendation, though not an absolute requirement, is to use "identical" drives), I'll be replacing both drives (protecting agains similar failure on the second).
It's a bit more complicated to do that, made more so by the fact that I can buy bigger capacity drives (on offer) than the same-sized units. The process of resizing is, however, supposedly "built in" to the NAS OS, so we'll see.
Dependent on the arrival time of the replacement drives, that looks like much of tomorrow and/or Friday gone! (rebuilding a 4TB mirror effectively twice takes a good amount of elapsed time).
|
Shirley just a case of swapping the disks one after the other and letting it do it's thing in the background?
Or does it give you a progress bar to watch ;-)
|
If you swap a larger disk in, it will mirror to the smaller one's capacity.
I'm not sure that then swapping in a second larger one will expand the array to the new, maximum size or not.
If I'd bought same-sized disks as current, then I'd simply do a two-stage swap, but since the NAS has a specific process for expanding the array (which will involve a two-stage swap in the script) then I'll do it their way ;-)
|
How much data is written in a day or so? Will it let you make RAID read-only, break mirror and copy data to bigger disk, insert 2nd disk and remirror?
Still two stages but more control over what it's doing if you can afford the read-only time.
|
>>Will it let you make RAID read-only, break mirror and copy data to bigger disk, insert 2nd disk and remirror?
...the inbuilt process essentially does that copying but keeps the NAS available. (It would be easier/quicker if the disks were hot-swappable, though).
>>Cant you upgrade your NAS in total and recover from your tape silo? replicate the data to an upgraded NAS via
>> your SAN?
...let's keep it simple, eh! Ultimately, I can recover all the user data from a 3TB usb portable drive (which was relatively up-to-date before the incident, but is now current). I'd prefer not to have to re-initialise/rebuild the NAS system and customisation, though, so the prescribed expansion method will do.
(I've actually bought a good number of tape silos in the past, StorageTek ISTR. You'd get almost as many of my NAS in one as you would cartridges)
|
Boils down to whether you trust their resizing utility then.
Feeling lucky?
|
|
...nothing much to lose, except time. Multiple ways of recovering if it goes belly up.
|
|
Cant you upgrade your NAS in total and recover from your tape silo? replicate the data to an upgraded NAS via your SAN?
|
|
Not until he's upgraded the firmware on all his FC adapters and the SAN Switches.
|
I accidentally formatted my 8Tb USB drive instead of a new 16Gb SD card for the Pi one day a year or so back. Really careless mistake that was over and done with, and irreversible, in a split second.
Luckily the disk was mostly only holding backups of other stuff, or stuff that was backed up elsewhere, though it was fairly full (currently about 840Gb free). Took me days and days to recover IIRC (much of it unattended though).
|
...my NAS holds data that is largely available elsewhere, so could be rebuilt, albeit taking time and having to remember all the settings.
I've taken an extra backup of the contents, however (can't have too many backups!).
Other than an automated repository for backing up data from the various user machines around the house, it's main task is to stream my music to various devices including Hi-Fis, and to play photos and videos on demand from the TVs.
I wouldn't be without it. The "fire and forget" backup regime it enables ensures that, in the event of a disaster, all user data can be restored to a PC up to the state as at the last early evening backup (taken daily).
It had been used to restore but a single file that had been deleted in error, up to the point where the HDD on SWMBOs laptop borked terminally with no warning. It was worth the expense and effort subsequently to be able to restore all the data to a point about an hour before the event. (there was an awful lot of "research" data on there).
|
"can't have too many backups!" - this, absolutely.
User PCs (just SWMBO now) are in the destiny of the user - they have the mechanism to easily back up their data as often as they like to the NAS. Once a week the NAS is backed up to my USB drive and once a month to another USB drive - then once when I remember I ask my daughter to bring home the offsite which holds the most key 3Gb or so (the space used is mainly photos but the important stuff is small in comparison). Oh, and there is the old NAS which is mostly offline but has backups of backups, plus some bulky stuff which it would be pointless clogging up a disk with - e.g. a load of tutorial videos downloaded ages ago for a hobby I've not really yet begun.
Some of my own stuff which is updated daily is backed up automatically each time my PC is turned on, other than that I do my end of the backups manually. It's a bit like me hand washing the car - gives me the chance check over the general state of things and keep tabs on how things are. I spend a fair bit of time at my computer so it isn't a chore.
The slow thing is imaging the SD cards in the Pis. I do that the day before the main backup day so they are all included, though one is fairly static and doesn't get done that often. I am trying to transition the Home Assistant to an SSD which will prevent me imaging that, as it's all or nothing with the SD card image software. I know there are probably different ways to do it but I finally managed to successfully restore Home Assistant backups in testing the other day, which I'd never managed before, so that won't need doing once I can prove to myself that it is robust and easily recoverable. (The HA restore software is nasty, it doesn't even let you know it's started, let alone report progress, or completion. Plus I removed a particular troublesome integration the other day which seems to have improved performance as well as reducing snapshot size by multiples.)
I'm sure some will think it sad but we all have hobbies, and our home IT setup is one of mine :-)
|
...well, I've started the replacement/expansion using larger disks.
The inbuilt process is designed for "hot-swappable" disks. My NAS isn't physically designed for that, though others by the same manufacturer are. Accordingly, the process is being run with the NAS disassembled. (I have hits on forums that people have undertaken the task in that way, and the swap worked when prompted, and now the new larger disk is resynchronising (rebuilding the mirror) with the remaining original).
From past building experience, it was expected to be slow, and it is - painfully so! At the end of this phase it's swap the other disk, (for a larger one)and resynchronise in reverse.
It then follows to expand the volume by using and synchronising the newly available space (which again will take time).
It can't be turned off in the interim, so it looks like I might be pulling an all-nighter. (brings back distant work memories of major recoveries of business-critical databases. I so miss that, not! ;-) ).
...AND
it is not impossible that, at the end of that process, it won't restart properly, but come up in an "initialising" state. There appears to be a quick and easy trick to correct that - we shall see!
|
...well, 17 hours later, swapping out the two disks and replacing with a pair of larger ones, it's all back as it was.
Mirror rebuilt and data retained, then the image expanded to fit the bigger disks. A few "iffy" moments but generally ok.
i) Having rebuilt, but still using the original size, the process invites you to expand the RAID array into the extra space by selecting "Expand", but the button to do so wasn't on the taskbar!. After a few choice words, and about 15 seconds, the taskbar magically refreshed itself and it was there!
ii) It did come up at the end in an initialising state (which, if continued would destroy all the rebuilding and the data. The quick and easy (but far from intuitive) trick noted above resolved that - instead of letting it go looking on the web for updates to its firmware itself, which starts the full reset process, you manually update the current firmware with a (the same version) locally held copy. It reboots after installation and, hey presto, it is now absolutely fine, realising it doesn't need to initialise. If I hadn't found a tip-off on this, things would have been very frustrating.
Because of the need to hot-swap and keep powered-up, the unit was out of its case for the duration - a bit of a nuisance, but it worked.
....and throughout, as it should be, the device was fully functional on the network, albeit most of the time from a singe-disc, not a mirrored pair. (I minimised activity anyway, but in the midst of it it updated one of my streaming applications (twice, different versions! - bug in the first update I think.))
|
|
Sounds like a good job well done!
|
|
what did we do before we had TB of data to worry about........
|
...I counted each byte out......and I counted them all back in again...
;-)
|
...WD 3TB came out, Seagate 4TB went in.
I look at backblaze from time-to -time, but it isn't much use when you're buying, as many of the drives (because of the age of the tested ones) aren't on the market any longer, and they're Enterprise/Datacentre models anyway, not Consumer NAS Drives.
(FWIW, my experience over the years of WD has been slightly better than Seagate, but without more specific evidence, not enough to move me away from buying, like-for-like, on price. The current Seagate Ironwolf range integrate more easily with the NAS disk health management as well!)).
The suspect drive has been tested by independent and WD utilities. The independent ones all "suck their teeth" at it (It has trippped one of the S.M.A.R.T. triggers after all), WD says "move along, nothing to see here". Unfortunately, the NAS doesn't agree, and would continue to nag me to replace it - and No, I'm not feeling lucky.
|
For a long time we had a Buffalo NAS with around 2TB of storage. Mostly used for back ups.
A year or so ago it stopped working. Possibly a Windows security issue.
How can I recover the data from it?
In my ignorance I thought I could just extract the drive (2TB Seagate Barracuda) from the case, put it in a caddy and sort out the content from there.
Clearly it's more complicated. The drive is not recognised in Windows explorer though the results I get suggest it may have several partitions. More reading, which of course I should have done first, refers to NAS specific file structures.
Is there any way around this?
|
..the disk will be formatted in a non-Microsoft format, most likely a Linux supported one.
I had a Buffalo Linkstation years ago, and I think it was XFS.
I haven't had reason to recover files, but there are a few software products that will mount and read linux file systems under windows. The only recommendation I can find from a quick browse for a free one is fsproxy:
nadrin.github.io/fsproxy/wikka5bf1.html?wakka=HomePage
Use at your own risk.
|
>Use at your own risk.
I'd create a bootable Linux USB stick to see if it recognises the partitions and file systems first. If it does, mount them and see if there's anything on the disk that's really needed before deciding how to transfer it.
Well, actually I'd just plug it into a spare slot in my X3550 but I'm guessing Bromp doesn't have a box running Linux.
|
>> I'd create a bootable Linux USB stick to see if it recognises the partitions and
>> file systems first. If it does, mount them and see if there's anything on the
>> disk that's really needed before deciding how to transfer it.
I'd wondered about Linux after TnE's post. I've got a bootable CD with Ubuntu which emerged into daylight for the first time in years during a tidy up this week. Used it a couple of times as a means of getting content from HDD's of my son's that at been FUBAR either by Uni Horseplay or (ahem) the viewing habits of young men.
Probably better to download a more up to date Linux...
Last edited by: Bromptonaut on Sat 15 Apr 23 at 13:48
|
|
Definitely download an up to date version, doesn't take long. There has been some major updates to native Windows filesystem support over the last couple of years.
|
Anyone suggest a good download and where from?
|
Go for what you're familiar with so if you've used Ubuntu before go for that - from the Ubuntu website.
I use Debian on laptops and Pi kit but that's just personal preference.
|
I'm not a great Linux user; most recently it's been Linux Mint, 'cos it's fairly Windows-like.
It will (allegedly) boot/run from a USB stick.
If the ex-Buffalo partitions are, as I suspect, XFS, you may have to load some extra stuff (to whatever Linux distro you're using) to manipulate them (though I suspect if you only want to read, you may be able simply to mount them).
This might help if you do need the tools/utilities:
www.xmodulo.com/create-mount-xfs-file-system-linux.html
|
|
Could the NAS have included data encryption?
|
Whoops, there goes the NAS - Kevin
