Bench record · NAS & RAID · NTG-2025-1792
Nothing Went Wrong on the Saturday. Wednesday Did.
The mains had gone off over the Saturday, and when a building-services engineering subcontractor at Teal Park in Lincoln came back after the bank holiday, a six-bay shelf was two members short. One had been sitting amber since the Friday
. That was not acted on, and by Tuesday a second disk had gone as well. What their IT firm did was slot a spare into the empty bay and start a rebuild
, and come Wednesday the rebuild had reached sixty per cent or so and stopped
. The power cut itself cost this firm nothing at all in the end. The rebuild cost it a fortnight.
Yours doing the same? Say what it is up to — looking is free.
0115 8220606
What went wrong, and why.
There is a way to work with two failed members. Once a rebuild has been started over them, there is not. RAID 5 holds one disk in reserve; that reserve was gone the morning the first bay showed amber, and when the second went every stripe had two missing values with no route to either. A rebuild launched from there gives nothing back and does genuine damage, laying new parity through the very stripes a recovery needs to find as they were. Server shelves bring problems of their own on top of that. Sector sizes on SAS members are often ones a desktop machine has no way of addressing, and an array's own structures sit exactly where consumer tools will overwrite them or miss them entirely. Beneath the whole episode was an amber light that nobody answered. Server and RAID work carries a published band from £500 + VAT, and the assessment that comes first is free.
The bench work this one took.
How the work runs →| Kit on the bench | What it did on this job | Why we reach for it |
|---|---|---|
| PC-3000 SAS/SCSI | Every one of the six read across a native SAS interface | Server SAS and SCSI disks are not addressable by an ordinary desktop machine at all |
| Atola TaskForce 2 | Six imaged concurrently instead of in sequence, which took days off the job | Every member of a set copies in parallel, so no disk waits its turn behind another |
| UFS Explorer RAID Recovery | Metadata gave the geometry, and a virtual volume went over the top of the images | Works the stripe out again, then unpicks the volume manager the NAS laid on top |
What happened on the bench.
Image all of them, healthy ones included
No power reached the shelf while it was here. Its members were withdrawn one at a time and labelled by bay, and the four healthy disks went onto imagers before anything else, so that neither failure was asked to do a thing until it had been assessed properly. Choose the wrong interface at that point, or the wrong sector size, and every image made afterwards is quietly useless.
How much the two failed disks gave up
Neither failure was anywhere near unreadable: one had a head going, the other was accumulating fresh bad sectors as it was read. So each got a series of short passes with rests in between, and the great majority of both surfaces went into an image. Two passes over each of them earned their keep: where the two disagreed on a sector, the cleaner one was kept, parity being in no fit state to settle the argument.
Geometry comes from the metadata
No assumption was made about how this manufacturer's controllers usually arrange things. The member order, the stripe width, the parity rotation and the parity delay were all lifted from structures the array had recorded about itself. Knowing those four, the six images carried a virtual volume, the filesystem came out of that, and every physical member stayed unpowered from beginning to end.
What was returned.
Up came the volume, and before a single byte went onto new media it was all checked against the directory tree the array had been carrying — carriage home at our expense. One qualification goes on the file: Wednesday's rebuild had already written over one band, so a live project folder is short. On the engineer's own laptop there was a copy of the same folder from a fortnight before.
Start here if the same thing is happening to yours.
Other RAID & NAS jobs, written up.
Sounds like yours? Switch it off before anything else.
£250 + VAT covers a card or a USB stick; a single drive is £300 + VAT; RAID and NAS start from £500 + VAT. Nothing is charged for the first look, and a fixed figure reaches you in writing before any chargeable work begins. Switch the device off, leave it alone, and post it in. On most jobs, no data back means no fee.