Troubleshooting In Linux!

This document was created by the North GA LUG Group.

Lock-Ups

If your system or an app locks up it can be a frustrating experience. Here are some tips and tricks to get you back on your feet.

• System Lock: If your entire system is locked hit Ctrl + Alt + F2 to get to a command line. This will let you do some further analysis on your system such as htop to see if a process is hung.
• xkill: the xkill command turns your mouse cursor into a weapon. Once you type xkill in the terminal you will see your mouse change to an x. Anything you click it will kill that process.
• pkill: the pkill command is similar to the xkill command except that you can terminate all instances of an application. ie sudo pkill -9 -f steam
• Kill terminal or commands that are hung using ctrl+c or ctrl+z

Process Monitoring

• top & htop: top and htop will both provide you a list of running processes. htop is little more graphical and provides shortcut keys to kill processes by hitting F9.
• To list all files open on the system use lsof
• You can use watch to keep an eye on processes as they are running. Example watch lsusb to monitor if USB devices are plugged in.

General Tips

• Check permissions if you’re having trouble executing an app or a file. $ ls -la is an easy way to see permissions.
• You can find kernel version using $ uname -a which can be helpful for troubleshooting and filling out bug reports
• inxi is an insanely valuable tool for collecting system information. There are many cli switches to get you the information you need. ie sudo inxi -b for short form basic info. sudo inxi -M for the machine info (such as motherboard, serial, manufacturer, model, etc). Highly suggest circling out all of the options by running man inxi. A useful full system dump (think about opening an issue) can be done with sudo inxi -b -v 8
• ncdu is a super helpful command to figure out where all of your speak space had been consumed.
• Purge old kernels! otherwise your /boot will fill up. ie sudo apt autoremove -y.
• Find path to software installed using which as an example which chromium will tell you the program is located in /usr/bin/chromium

Checking Hardware

• List all PCI devices on your system with lspci -v this is great for getting model of PCI devices such as Ethernet Controller, Wifi, GPU, etc.
• List all USB devices lsusb great tool for determining if usb devices are detected and what usb devices are connected.
• Check what hard drives are detected fdisk -l
• To check hard drive space use df-h

Black Screen w/ blinking cursor

Especially Nvidia users
- Hit ‘e’ at the GRUB screen to edit the settings
- add ‘nomodeset’ before ‘quiet splash’ in the entry
- What does this do? Instructs the kernel to not load video drivers and use BIOS modes instead until X is loaded.

Can’t Boot

• Rescatux is a great tool to have around to help repair common issues with Grub.
  • Rescatux is a Debian based live cd that lets you fix your GRUB1 and GRUB2 installations (as per Super Grub2 Disk lacks) but does much more including:
    • Check and fix filesystems
    • Blank Windows passwords
    • Change Gnu/Linux password
    • Regenerate sudoers file

Checking Logs

• Most systems are using Systemd so you can get your logs with the following commands:

$ less /var/log/messages (System logs)
$`dmesg | less`  (Kernel logs)

  $ `journalctl -b` (-b switch shows all messages since last reboot)
  $ `journalctl -b -p err` (shows messages marked as an error)

• For situations where you’re getting a console login vs. desktop GUI

$  grep EE /var/log/Xorg.0.log 

• List logs in order of when they were written
  $ ls -lrt /var/log
• Look for errors only in dmesg using dmesg | grep 'error'

*Note some distros utilize a different general log. Some Debian based distros uses /var/log/syslog vs. /var/log/messages as it’s main log. If you don’t have messages just change the end of the command to syslog in place of any of the commands above or below.

Searching Keywords In Logs

• You can search keywords in the logs by using ‘grep’
• Navigate to /var/log
  $ sudo grep "error" messages
• This will return all messages that contain the word error in the logs

Monitoring Logs

• Use ‘tail’ to monitor logs real-time for bug reporting or system analysis. This will provide you updates that occur and could be invaluable for bug reports or monitoring of user activity or changes on your system.

As an example:

For openSUSE, Rhel you can use:

sudo tail -f -n 6 /var/log/messages

For Debian based distros

sudo tail -f -n 6 /var/log/syslog

The ‘f’ switch tells it to follow
The ‘n’ switch tells it to display last N’th number of lines

Network Troubleshooting

• ip addr Shows addresses assigned to all network interfaces
• ip route Show table routes.
• ip link
• ping the ping command is useful to test connectivity between two sources
• ifup using this in combination with your specific interface ex: eth0 (which could be obtained using ip addr) will bring the interface up.
• ifdown - ifup using this in combination with your specific interface ex: eth0 (which could be obtained using ip addr) will bring the interface down.
• traceroute is a network tool to tell you the hops in-between destinations. ex: traceroute google.com
• host or host -t this command lets you find IP or IP to name along with DNS info. ex: host www.google.com
  • Another command you can use is nslookup and example would be nslookup google.com which will provide you DNS info.
• iwconfig configure wireless network from terminal.

Backup Solutions

• Having a solid backup in place can help you to overcome issues easily and without losing your valuable data.
  • DejaDup is a simple to use back-up tool
  • Clonezilla will let you make entire clones of your existing drives and restore them if things go awry.
  • Mega.nz is a nice tool for syncing and storing files on the cloud.

Manual

• All the above can be researched further using man command. An example man ifconfig will provide information and options for that command.