Sometimes errors occur on the filesystem of your server's system disk that prevent it from working properly.
In this case, the only solution is to perform a filesystem check to (hopefully) fix the errors.
fsck on a disk must be done while it is unmounted or in read-only mode. If not, the risk of losing data is very high! Always check the state of the disk before beginning the check and carefully read any warnings produced by the program.
There are two ways of proceeding with the check of the system disk depending on whether you have shell access or not.
If you can log in on your server and get to a shell, then you will be able to proceed to the checking on the disk directly from your server.
First of all, you must switch the system disk into read-only mode with the following command:
# mount -o remount,ro /
mount: / is busy in return, it means the disk is in use by some programs. You will have to stop or kill the process who have w/u on /. The command lsof / will display that information, as in the example below:
# lsof / COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME rsyslogd 1237 root 3u REG 202,1 3762377 3544 /var/log/syslog rsyslogd 1237 root 5w REG 202,1 29569 25879 /var/log/messages rsyslogd 1237 root 6w REG 202,1 2217748 25872 /var/log/auth.log rsyslogd 1237 root 7w REG 202,1 36729452 17350 /var/log/daemon.log rsyslogd 1237 root 9w REG 202,1 978650 25878 /var/log/user.log rsyslogd 1237 root 10w REG 202,1 8108 12417 /var/log/kern.log
In the case above, we can see that that rsyslogd is using the system disk in read-write. Therefore, you just have to stop or kill rsyslogd, as well as all other processes having w/u on /.
Next, check that the disk has been mounted correctly in read-only mode (ro) with the mount command:
# mount /dev/xvda1 on / type ext4 (ro,noatime,errors=remount-ro)
To begin the fsck, execute the following command:
# fsck -f /dev/xvda1
The output might look something like this:
fsck from util-linux 2.20.1 e2fsck 1.42.5 (29-Jul-2012) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information DEBIAN7_64: 32869/196608 files (0.2% non-contiguous), 274868/786432 blocks
If the shell is not available on your server, you will have to check the disk from another Gandi server. Indeed, this is one of the advantages of server virtualization, which allows you to attach a disk to another server.
To create a new server, you must have sufficient resources (at least one share) available in your Gandi account.
Once you get the share, create a new server (in expert mode) using the Linux distribution of your choice. The creation of the new server might take a few minutes.
Once the new server has been created, stop the server with errors on the filesystem. Then detach the system disk and attach it to the newly created server.
Once the system disk is attached to the new server, it should be mounted automatically. You can verify that with the mount command :
# mount /dev/xvda1 on / type ext4 (rw,noatime,errors=remount-ro) /dev/xvdk on /srv/DEBIAN7_64 type ext4 (rw,nosuid,nodev,noatime)
In the case above, the label of the system disk to check is 'DEBIAN6_64'. To unmount the disk, execute this command:
# umount /srv/label_of_the_disk
Once the disk has been unmounted, you can proceed to the check of the system disk with the command fsck -f /dev/xvdX (where X is the disk to check):
# fsck -f /dev/xvdk fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information DEBIAN7_64: 32869/196608 files (0.2% non-contiguous), 274868/786432 blocks