Skip to main content

Introduction to Linux Troubleshooting

Troubleshooting is one of the most critical skills for a Linux system administrator or DevOps engineer. It is the systematic process of identifying, diagnosing, and resolving problems within a complex system.

The Troubleshooting Methodology​

Effective troubleshooting is rarely about guessing. It follows a structured approach to minimize downtime and avoid introducing new issues.

  1. Identify the Problem: Gather information. What are the symptoms? When did it start? Who or what is affected?
  2. Reproduce the Issue: If possible, try to trigger the problem reliably. This confirms the issue and provides a baseline for testing fixes.
  3. Isolate the Cause: Narrow down the scope. Is it a network issue, a disk problem, an application error, or a kernel panic?
  4. Formulate a Hypothesis: Based on the evidence, guess what the root cause might be.
  5. Test the Hypothesis: Apply a potential fix or run a diagnostic command to prove or disprove your theory.
  6. Implement the Solution: Apply the permanent fix.
  7. Verify: Ensure the problem is actually resolved and hasn't caused collateral damage.
  8. Document: Write down the problem, root cause, and solution to help your future self or teammates.

Essential Basic Tools​

Linux provides a vast array of built-in tools for gathering system state:

  • System Resources: top, htop, uptime, free
  • Disk I/O and Space: df, du, iostat, iotop
  • Network: ping, ip, ss, netstat, mtr
  • Processes: ps, kill, pgrep

The Importance of Logs​

Logs are your primary source of truth when things go wrong. Most system and application logs are found in the /var/log/ directory.

  • /var/log/syslog or /var/log/messages: General system activity.
  • /var/log/auth.log or /var/log/secure: Authentication and security-related events.
  • Application-specific logs (e.g., /var/log/nginx/, /var/log/mysql/).

Best Practices​

  • Change One Thing at a Time: If you change five configurations and the system starts working, you won't know which change actually fixed it.
  • Read the Error Messages: Error messages are usually literal and point directly to the problem. Don't just skim them.
  • Check Permissions: Many "unexplained" issues in Linux boil down to incorrect file or directory permissions.
  • Understand the Baseline: You can't know what's abnormal if you don't know what "normal" looks like on your system.