Linux troubleshooting

Linux troubleshooting

http://www.brendangregg.com/linuxperf.html

df and du command shows different disk usage in same file system

The usual cause is the deletion of an open file (like a web server log file-). The file entry is erased in the directory structure, but the inode and the data blocks it points to are not deleted.

lsof can show the process deleted files that are still open:

$ lsof /var/log | egrep "^COMMAND|deleted"`

Stopping/killing the process will free the space.

If you can not stop the process, you can try to make the process write the file in a different filesystem. This can be achieved with the GNU Debugger. Run gdb and attach it to the process. Bear in mind that doing so will stop the process, so it is better to run all the needed commands via a script.

Links

Saving Your Linux Machine from Certain Death

Recover root password

In UBUNTU

Method A

From the boot menu, select recovery mode and hit Enter
Select the Drop to root shell prompt

# passwd    
# exit    

Select resume normal boot

Method B

    Grub > 'e'
	  replace "ro quiet splash $vt_handoff" with "rw init=/bin/bash" in linux entry
	ctrl+x
	  mount | grep -w /   (to check system particion is rw)
	     (run "mount -n -o remount,rw /" if still ro)
	  passwd
	  exec /sbin/init

Method C (live CD/USB)

• Open a terminal in booted live CD/USB OS.
  
• Find out the device name for your primary disk (sudo fdisk -l)
  

    sudo mount /dev/sda1 /mnt

• Cross mount /dev, /sys, /run and /proc filesystems:

    sudo mount -bind /dev /mnt/dev
    sudo mount -bind /sys /mnt/sys
    sudo mount -bind /sun /mnt/run
    sudo mount -bind /proc /mnt/proc

or copy and paste this

    for d in dev sys run proc; do sudo mount –bind /$d /mnt/$d; done

• Make mounted filesystems your root filesystem and change the password

    sudo chroot /mnt
    passwd

In REDHAT

Method A

    Grub > 'e'
      init=/bin/sh (Remove rhgb and quiet tags if necessary)
    ctrl+x
      /usr/sbin/load_policy -i
      mount -o remount,rw /
      passwd root or passwd
      mount -o remount,ro /

Method B

    Grub > 'e'
      rd.break   (add to 'linux' entry)
    ctrl+x
      mount -o remount,rw /sysroot/
      chroot /sysroot/
      passwd root or passwd
      touch /.autorelabel

Making a process read a different file for the same filename

https://unix.stackexchange.com/questions/81003/making-a-process-read-a-different-file-for-the-same-filename

Mico Maco’s todo’s

      PROVIDE THE COMMANDS and SCRIPT !!!!

http://blog.backslasher.net/hijacking-processs-io-streams-using-gdb.html
http://gcolpart.evolix.net/blog21/capture-inputoutput-of-a-process-with-gdb/
http://vvv.tobiassjosten.net/linux/detaching-processes/
http://www.brendangregg.com/blog/2016-08-09/gdb-example-ncurses.html
https://en.wikibooks.org/wiki/Linux_Applications_Debugging_Techniques/Core_files