[LIVE] Hard drive issues

Marcelektro

Founder, Developer
Staff member
Aug 2, 2019
146
21
18
[LIVE] This post is edited and updated as new steps are taken to get our server back alive.
[LIVE] Updates:
- 06.09.2020
- 06.10.2020

WARNING! ANY ACTIONS TAKEN BY ME, STUFF THAT HAPPENED, OR EXPLAINATIONS, REASONINGS OR UNDERSTANDINGS ARE MY OWN AND NOT ALWAYS TRUE! DO NOT REPEAT ANY ACTIONS SHOWN BELLOW WITHOUT FURTHER INVESTIGATION IN THE REASON WHY SHOULD YOU DO LIKE THIS! REPEAT THEM ON YOUR OWN RISK.

The Drive Issue
~ marCloud

Hello Everyone
As you know, recently marCloud Minigames Network is un joinable. The reason for this is that the hard drive (M.2 SSD) on which there is a proxy (bungee cord) that makes the server work, somehow broke down. Don't worry, its not broken literally. It just has some bad sectors where I dont even store data, and I work to fix them right now (09.06.2020), but let me tell you a small story how it all happened.

At June 6, at 3 PM (Poland time), I was playing a game with my girl friend @Edytka1. On our phone was sent a notification from uptime robot saying: Proxy Server, Minecraft Network, Discord Bots are offline. I thought its just an issue with my server such as Norton blocking network or stuff. I went to RTP connection, but it was not accessible. I got worried a bit. Then, I pinged its network adapter using WOL protocols, it also was down. Then I decided to go to my server room, and check what happened. There was nothing, Disk led on case was still red (no blinks at all) and the monitor didn't show anything. I gave it a little of time (10 mins) and decided to click shut down button (not hard reset yet). It didn't react. I waited for it 30 mins. Any update or stuff would be stopped by this. It was not though. Then, i decided to move on and do a force restart. I clicked the button, and it shut down, then powered up. The LED-s went back to normal, bios welcome msg welcomed me as always, but suddenly, windows welcomed me with disk repairing.

48

This took few hours, then it showed a big bsod (blue screen) error. It rebooted itself

Then after the bsod, a big bios error occurred.

[PIC-1] Error screen
47
Warning: Please back-up your data and replace your hard drive.
A failure may be imminent and cause unpredictable fail.

The same issue happened earlier, and I fixed it by changing slots of M2 storage in my server. So did I this time, but back from 2 to 1 slot. It did not help at all. Slot 1 did not see the disk at all, showed it as not existing (null at the slot in bios). I put it back to slot 2 and tried again. Also, the same error (PIC-1)

After few attempts of changing slots, cleaning slots, connections with alcohol and other chemical stuff, it did not work at all.
I decided to move forward, and prepare a Linux bootable live instance to somehow clone (backup) the drive and replace it as the bios said. As you could guess, it did nothing. Clone was not working due to "bad sectors" on the drive. It was the moment i was scared tbh like a kid who runs away from a ghost in a movie. I went to console mode, and did following commands:
Bash:
- fdisk -l
- mkdir /media/disk
- mount /dev/sda4 /media/disk
- cd /media/disk
- cd Users/<username>/Desktop/<folder>/
- ls
It worked as the drive was totally functional. I could read all files on it. However, after some more steps - mounting a USB pendrive and trying to copy a folder - it failed. The lag was huge. CTRL-C didn't work as well as CTRL-Z. After few minutes (3,4 mins) it unlagged. After trying to cd to the folder I tried to copy, even clicking TAB key was lagging me out for a long time. I knew its not good, so I decided to not damage the drive anymore by using it, and just work on backing it up.
After rebooting the server entirely, mounting it again, the folder was accessible as nothing happened before. After using it for more time, it was lagged, and only a reboot could fix the issue.
I need to add that >reboot command was so slow. It kept showing Error: READ DMA. On google, the error meant the drive is simply corrupted.

Then I did many more attempts about copying the folder to a drive. File by file did work, but not for a long term. It was lagging until reboot.
Finally, I decided to move on more, and bought an adapter from M2 nvme to sata.
I've made it M2 -> USB, and plugged to my pc. It showed an error: The Disk (G) is damaged. Click here to repair it. So I clicked it. It was scanning and repairing, but suddenly, all my PC peripherals including mouse, keyboard, mic, usb hubs and all. Literally every single USB thing. Unfortunately, the drive was one of them. The repair failed obviously.
Then, I decided to scan it using Remo Recover software, which I used earlier to fix my dad's pendrive. It did detect all data, its size and previews. However, they wanted 100$ for the files. marCloud is worth less that this price, so I wouldn't pay so much for this. I tried using other tools, and some freeware. As I used next, the work of the drive was slowing down. Finally, on one tool, it didn't work at all saying its RAW partition and 0 KB total. @Wolfo Gaming said its gone, I didn't give up yet. I went to my dad's computer and plugged the drive in. It also asked for repair. I did click it, and it did the same. Then I went to file explorer, and to my surprise, it opened the disk data. Well, when I directed to C, users, name, desktop, folder, the name of user required admin. I clicked YES and gave admin password. However, when the login window closed, the explorer.exe froze, and drive became 100% used. It was repeating each time when I tried to access the user folder.

I've been repeating these steps (programs, repair, chkdsk, testdisk) many times in a loop. Now (10.06.2020) I connected it to my server where it came from, and it did not show corruption error from bios, as it did for the 1st time. The drive normally booted up. I went to the USB Linux system, logged in as admin, fdisk -l (all cmds as I did earlier) and the data was fully accessible. I decided to copy a small folder from the important folder. I did >cp -R path path. It successfully copied. As I was happy for this to happen, I decided to copy entire folder to the backup usb, however, it was laggy as the time gone, and slowed down until I/O Error. I decided to reboot the entire computer using the reboot cmd. It shut down fast, unlike it did when the drive showed bios error. When It was booting, it took so long for the bios welcome screen (the one with press f2 for bios) to show. I went afk, because of dinner, and while, it launched up my windows system. It was not functional. First, it was saying "Preparing automated repair". After few minutes, more like 30 mins, it showed up blue screen. Not a bsod tho. Literally blue. No text, no nothing. Just blue as the windows theme.

After this happened, I've got my linux usb stick, mounted as earlier, and per file copied entire marCloud to another functional USB drive. In this way, which took few hours of non-stop work, I've managed the files to be avaliable.
After this, I've boot up marCloud Proxy, Discord bots and MC network and host them on my main PC, as it has the best specs of devices avaliable.
 
Last edited: