CentOS 7 Drive Full
-
So, I came into work today and the fog service was stopped. Rebooted the machine and saw the message that it couldnt write to the disk.
Result of df
Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/centos-root 31340544 31015780 324764 99% / devtmpfs 932704 0 932704 0% /dev tmpfs 942348 0 942348 0% /dev/shm tmpfs 942348 8608 933740 1% /run tmpfs 942348 0 942348 0% /sys/fs/cgroup /dev/sda1 508588 122016 386572 24% /bootResult of du -sh
du: cannot access ‘./proc/12278/task/12278/fd/4’: No such file or directory du: cannot access ‘./proc/12278/task/12278/fdinfo/4’: No such file or directory du: cannot access ‘./proc/12278/fd/4’: No such file or directory du: cannot access ‘./proc/12278/fdinfo/4’: No such file or directory 1.6G .I cannot figure out where these “files” are.
-
Apache logs consuming fog server hdd (maxes to 100%)
–Have to delete everything in /var/log/apache
–then restart service to clear hdd on server
–finally HDD is back to 22% usedThis is a current issue with the SVN
cd /var/log/apache/ rm * service apache2 restart -
Unfortunately I have tried that and the apache logs are VERY small right now.
total 16K -rw-r--r-- 1 root root 8.8K Jul 20 14:31 access_log -rw-r--r-- 1 root root 997 Jul 20 14:30 error_log -rw-r--r-- 1 root root 0 Jul 20 14:30 ssl_access_log -rw-r--r-- 1 root root 0 Jul 20 14:30 ssl_error_log -rw-r--r-- 1 root root 0 Jul 20 14:30 ssl_request_logI did go ahead and delete them, then reboot and still no luck.
-
-
Here is the output:
1621736 / 975920 /usr 404516 /var 296068 /usr/share 268040 /var/lib 225724 /usr/lib 199072 /var/lib/php/session 199072 /var/lib/php 167288 /usr/lib64 155356 /usr/binHere is the /usr folder
975920 /usr 296068 /usr/share 225724 /usr/lib 167288 /usr/lib64 155356 /usr/bin 113044 /usr/lib/modules/3.10.0-229.el7.x86_64 113044 /usr/lib/modules 110424 /usr/lib/modules/3.10.0-229.el7.x86_64/kernel 99992 /usr/share/locale 78648 /usr/lib/modules/3.10.0-229.el7.x86_64/kernel/drivers -
@Marcus-Allen said:
Here is the output:
1621736 / 975920 /usr 404516 /var 296068 /usr/share 268040 /var/lib 225724 /usr/lib 199072 /var/lib/php/session 199072 /var/lib/php 167288 /usr/lib64 155356 /usr/binHere is the /usr folder
975920 /usr 296068 /usr/share 225724 /usr/lib 167288 /usr/lib64 155356 /usr/bin 113044 /usr/lib/modules/3.10.0-229.el7.x86_64 113044 /usr/lib/modules 110424 /usr/lib/modules/3.10.0-229.el7.x86_64/kernel 99992 /usr/share/locale 78648 /usr/lib/modules/3.10.0-229.el7.x86_64/kernel/driversadd -h to the sort command.
-h gives you human readable sizes.
-
I dont think that is working right:
du -a -h / | sort -n -r | head -n 10 1020K /usr/lib/kbd/consolefonts 1016K /usr/share/doc/sudo-1.8.6p7 1016K /usr/lib64/python2.7/site-packages/gi 1012K /usr/share/pki/ca-trust-source 1012K /usr/share/pki 1012K /usr/lib/udev/hwdb.d/20-usb-vendor-model.hwdb 1012K /usr/lib/modules/3.10.0-229.el7.x86_64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 1012K /usr/lib/modules/3.10.0-229.el7.x86_64/kernel/drivers/net/ethernet/broadcom/bnx2x 1008K /usr/lib/firmware/cxgb4 1004K /var/lib/mysql/mysqlIs that re right spot to put that switch in?
-
du -a -h / | sort -n -r -h | head -n 10or this:
du -a -h / | sort -nrh | head -n 10 -
-
@Marcus-Allen Ok. Your last post you put up was in human readable format so do it that way.
Can you post this?
df -h -
Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 30G 30G 303M 100% / devtmpfs 911M 0 911M 0% /dev tmpfs 921M 0 921M 0% /dev/shm tmpfs 921M 8.5M 912M 1% /run tmpfs 921M 0 921M 0% /sys/fs/cgroup /dev/sda1 497M 120M 378M 24% /boot -
Is that correct? you basically only have 30 GB available on this server?
-
Thats correct. Its a VM, I can increase the disk space if needed, however last week I was using about 11GB, so I didnt bump it up yet.
-
I’m bumping up to 64gb. This is a FOG Only server, no images are stored here.
-
@Marcus-Allen I just tried this, it worked better than the other one:
find . -type f -printf "%s\t%p\n" | sort -n | tail -10 -
15773912 ./var/www/html/fog/service/ipxe/init.xz 15775828 ./var/www/html/fog.prev/service/ipxe/init.xz 18102074 ./boot/initramfs-3.10.0-229.el7.x86_64.img 18179668 ./boot/initramfs-3.10.0-229.el7.x86_64kdump.img 18874368 ./var/lib/mysql/ibdata1 24281088 ./var/lib/rpm/Packages 39835970 ./boot/initramfs-0-rescue-e248d8e9ddcb47a99cdece8ee62fabc5.img 67108864 ./sys/devices/pci0000:00/0000:00:0f.0/resource1 67108864 ./sys/devices/pci0000:00/0000:00:0f.0/resource1_wc 140737486266368 ./proc/kcore -
@Marcus-Allen I think it’s obvious where your space went…
-
-
@Marcus-Allen Just following up on this thread, @Wolfbane8653 was correct (ofcourse).
But, on Red-Hat based distros, the path to the logs is in
/var/log/httpd
I just had to deal with this issue at one of my sites… fog server had a 300GB Apache log file lol. I deleted the file and then restarted httpd. Some of the tables in the fog DB were marked as crashed as well. I had to issue a repair statement on those to fix them like this:
mysqlcheck --repair --databases fogand then I gave mysql a restart and all was fine.
-
@Wayne-Workman said:
Sorry, I’m a touch confused now. My logs are empty in the /var/log/httpd directory. I even cleaned them out too just to be safe, still only about 10MB.
Or, were you just mentioning about the log files in general?
