Study notes covering Managing the Linux Kernel and Devices.

Author: Dylan Barrett

Date: 02/23/2026

📖 Useful Vocabulary / Acronyms

Gather Hardware Information

Server Architectures

It is important to understand architectures as it directly affects how the system operates and what software can be run on he system.

AArch64

• AArch64 is a 64 bit architecture and it works well in places that need high preformance
  • It is commonly used in data centers and cloud computing, some of its key features are:
  • Low power consumption
  • Scalability
  • High Preformance
  • Natively supported by many distrobutions
  • You can see what archeteture your system is runnig by using lscpu | grep Architeture

Reduced InstructionSet Computing, Version Five (RISC-V)

• RISC-V is an open-source system design based on the idea of reduced instruction set computing, it is very customizable and can fit many specific hardware needs.
  • It is most popular in research, Internet of Things, and experimental server setups, some of its features are:
  • Modular and customizable design
  • Cost effectiveness
  • Some support
  • You can check with uname -m

x86

• X86 is a 32-bit instruction set, it was widly used in its time and is still widely supported in legacy systems, some of its features are
  • Compatability with legacy software
  • Limited addressable memory of 4gb
• You can display it with lscpu | grep Architecture

x86_64/AMD64

• X86_64 is the 64-bit extention of xX86 it is the most widely used architecture for modern servers and desktops, it has these key features:
  • Addressable Memory more the 4gb
  • Backward compatability with 32-bit applications
  • Universally support by Linux distrobutions
• You can display the systems archetecture with lscpu | grep Architecture

Graphics Processing Units (GPUs)

• Linux supports a large variaty of software but some hardware benifits linux distrobutions like GPU's

• You can use a GPU in your Linux system to preform tasks like

  • Artifical Inteligence
  • Video Editing
  • 3D Rendoring
  • High-Preformance Computing

• To view GPU preformance you can use the nvtop command, it is not installed by defualt and can be installed with dnf/apt instll nvtop

Embedded and Custom Systems

• While it is most common to see Linux deployments in server or workstation roles, it is also common to see them in emmbeded systems. Some examples of embedded systems are:
  • Android Phones
  • Smart Tv's
  • Smart Watches
  • IOT Devices

Hardware Reporting Tools

• It is important to understand how to obtian the information about the hardware that is within your system, as it can assist with troubleshooting and various other things.
• There are several tools that exist to report hardware details, one being hwinfo.

Performance Baselines

• It is good once you deploy a computer to take a preformance basline, this way you understand how the system is suposed to preform under normal conditions.

hwinfo tool

• The hwinfo comman displays details about:
  • CPU
  • Storage Disk Controllers
  • USB controllers
  • Graphics Cards
  • NIC'S
  • Print Devices
• Some of the commonly used options in the hwinfo command are:
  • --cpu
  • --netcard
  • --storage
  • --blcok
  • --usb

dmidecode tool

• Just like hwinfo the dmidecode provides system inforamtion for the current devices. It also highlights upgrade options by displayin the maximum supported processor and memory upgrades.
• To view inforamtion for specific devices use the -t options followed by the device type like dmidecode -t processor

Display Bus Information

• You can use commands like lspci and lsusb to display addition details about devices connected to either bus.
• Both commands have varrying levels of verbosity to help with diagnosing potentential hardware issues
  • -tv option displays verbose information in a tree stucture.

Hardware Monitoring Tools

• Some other Commands that can be sued to identify and help troubleshoot system hardware on Linux are lm_sensors and lshw

The lm_sensors command

• The lm_sensors command is a hardware information tool that can show you information like temperature, fan speed and voltage levels.

The lshw command

• The lshw command is short for list hardware nad give you detailed information about the hardware found in your computer

Devices in the /dev Directory

• The udev manager provides plug and play functionality. When the system starts it creates a file for every hardware device it finds.
  • These files are not the same as device drivers

The udev Device manager

• The udec device manager detects hardware changed
  • udevadm monitor will monitor the udev device manager in real time and see when a device such as a usb drive is connected to the comptuer
  • udevadm info /dev/sda will show information of a specific device in this case /dev/sda

Block, Character, and Special Devices

• In the /dev directory you will encounter various types of device files, each of witch will be identified by a letter
  • c (character Devices) - These handle data one character at a time, usually for devices like keyboard or serial ports
  • b (Block Devices) - Transfer data in blocks and are commonly associated with hard drives or USB drives
  • d (Directory) - Indicates a directory
  • l (Symbolic Link) - Acts as a shortcut or referecent o another file or directory.
  • /dev/null - special file is writable location used to discard unwanted data
  • /dev/zero - is a way of filling storage capacity
  • /dev/urandom - can create a source of random characters for tasks such as creating completly randomized passwords, it is a very resource intensive task

The IPMI tool

• The Intelligent Platform Managment Interface is a powerful utility used to monitor and manafe hardware remotly. and can be install with sudo yum/apt install ipmitool #
• It can be used to check the servers health with:
  • ipmitool sensor - Checks the servers health
  • ipmitool chasis power status - Check if the server is on or off
  • ipmitool chassis power on/ off - to turn the power of the server on or off
  • ipmitool sel list - view system event logs to help diagnose hardware issues
  • ipmitool sol actvate - activates serial over LAN

Central Processing Unit (CPU) Information

• Linux creates the file /proc/cpuinfo when the system boots which provides the current configuration information about the CPU on a per core basis

Memory Information

• the lsmem command is used to dispaly information about memory blocks
• the /proc/meminfo file displays statistics and detials about the installed memory in the system

Memory Exhaustion

For best system preformance it is best to ensure that the system has plenty of memory to preform the tasks that it is being asked to preform

• The free command displays the quantity of free/unused memory if you use the -h option it displays the information provided in human readable foramt
• vmstat displays the use of virtual memory (swap space_
• The watch command allows you to monitor memory usage dynamically by repetably running a specified command at regular intervals like watch free -h

Out of Memory (OOM) Scenarios

• The Linux Kernel can sometimes allocate more memory that is actually available that may or may not be due to Memory Leaks, In this senerio it uses the Out-Of-Memory Killer to free up memory and its logs can be found in /var/log/messages

Manage Processes

Process Concepts

• In Linux can be created in various ways depending on how the program or commad is executed. The primary ways processes are created in Linux are:
  • Binary Executable: These programs written in languages like C C++ and compiled into machine-readable binary files
  • Internal Shell Commads - some comands like exit are built directly into the shell program
  • Shell Scripts - Shell Scripts are text files contained a series of commands when executed the shell interprets the script and runs the commands dynamically

Multitasking in Linux

• Liunx can run multiple process at the same time even on a single CPU. Even though it is only really doing one task at a time but switching between them really fast.
• Multithreading allows a single CPU core to handle multiple threas of processes

Process States in Linux

• I Linux processes transtion through various states during their life cycle. Some of these process states are:
  • Running (R) - A process actively running
  • Sleeping (S) - A process is waiting for an event or resouce to become avalable
    • Interuptible Sleep - A process is waiting for an event or resource to be become avaialable
    • Uninterubtilbe Sleep (D) - The Process is waiting for a hardware condition and can not be interupted
  • Blocked - Similar to sleeping the process is wating on an event
  • Stopped (T) - A process is paused or stopped, typically because it recieved the signals SIGSTOP SIGSTP
  • Zombie (Z) - A process has completed execution but still has an entry in the process table
  • Idle (I) - A process is not actively doing work but is ready to execute when needed
  • Dead (X) - A process has been terminated and is no longer in the process table.

Process Information Commands

Display the Process ID

• pgrep - Is simillar to the grep command but for processes
• pdiof - requires the exact process name where pgrep can find a process that contains a specific string
• top - dynamically displays the process consuming the most system resources
  • %us Displays CPU time spent running user processes.
  • %s Displays CPU time spent running the Linux kernel.
  • %id Displays CPU idle time. If this number is high, the CPU is not working hard.
    -%wa Displays input/output (I/O) wait time. If this number is high, the CPU is ready to work but is waiting for I/O access.
  • %st Displays Steal. If this number is high, a virtual CPU is waiting for access to the physical CPU.
• htop - is a user-friendly supplement to top and color codes the information presented
• atop - is like the top command but can log preformance information making it usable for long-term preformance testing and trend tracking
• ps - has many options to filter and display the exact process information that you need some of these options are:
  • -e Displays all processes.
  • -a Displays all processes with a controlling terminal, but not status or full command.
  • u Displays the username that started the process.
  • -f Displays supplementary information.
  • p- {PID} Displays processes associated with the given process ID number (PID)
• lsof command is used to display open files and the process that opened them
• system-analyze-blame command lists the process in order of how much time they take to start, helping pinpoint potential bottlenecks during startup
• sar comand is used to display system usage reports basedon data collected from system activity
• uptime - Displays the systems uptime

Preformance Analysis

monitoring CPU Usage

• mpstat - is a command-line tool that shows detailed statistics about the CPU usage, This is particularly useful if you suspect the the system is running slow due to the CPU.

Monitoring Process Preformance

• pidstat is another command-line tool that provides performance statistics for individual process. It shos how much CPU, Memory and other resources each process is using.

Detailed Process Information

• The /proc/PID direcotry contains files and subdirectories that provide detailed information about the process
  • To use this directory you can cat out /proc/PID and replace PID with the pid of the process
  • Some of the useful files that are under each PID in the /proc directory are
    • cmdline - Shows the command-line arguemtns used to start the process
    • cwd - Provides a symbolic link to the current working directory for that proces
    • environ - Displays the environment variables of the process in question
    • exe - provides a symbolic link to the executable file of the process
    • fd - Displays a directory containing file descriptors opened by the process
    • limits - shows the resource limits for the process
    • stat - provides detailed runtime statistics about the process

Process Managment

• pstree - displays command displays process in a tree like structure, and makes it so that you can see the parent-child relationships between processes

Process Manipulation

Process Signals

• While proccesses are supposed to terminate when they are closed that does not always happen and you as a system administrator might need to intervine.
• There are three common signals used to end processes
  • SIGTERM (Signal 15) - Requests a graceful shutdown, which is used to help prevent data loss or coruption
  • SIGKILL (Signal 9) - Immediatly and forecully terminate the process
  • SIGHUP (Signal 1) - requests the process to exit when the terminal running is closed
• kill - kill
• killall - killall {options} {process_name} - targets all processes with a specific name
• pkill - is similar to kill but can use references to a process instead of just the PID

Prioritizing CPU Processes

• Processes are assigned priority levels ranging from -20 to 19
  • -20 Highest Priority
  • 0 Default priority
  • 19 Lowest priority
• nice and renice - commands shift priority levels at different times in the proceses lifecycle
  • nice - Sets a priority level when you start a new process nice -n
  • renice - set the priority level of a process that is already running renice - -p

Job Control

• You can view and manage jobs using the jobs command, which gives a numbered list of all active jobs, here are some of the common jobs command options:
  • & - To run a job in the background
  • fg - To move a job in the background to the foregroung
    • Ctrl+Z - Temporarily pauses a job
  • Ctrl+D - Exits the program andd logs the user out of the session
  • Ctrl+C - Stops a running process entierly
• You can prevent job termination with nohup, like when a process terminates when a user logs out

Manage the Linux Kernel

The Linux Kernel

• Kernels tend to divide software running into two spaces the kernel sapce and the user space
  • Software runnig in the user space can access resources provided by the kernel through the use of system calls
  • The split promotes greater stability and security
• Device Drivers are software programs that enable a computers operating system to identify and work with a specific hardware device
  • At system boot drivers are loaded with the kernel so that the Operating system can work with these devices
• You can display Kernel Version information by using the uname command, some of the common arguments for the uname command are:
  • -r - The Kernel Version number of your current system
  • -i - To view the hardware platform
  • -a - To print all kernel inforamtion

Kernel Updates and Parameters

• You can modify some of the kernel parameters at runtime using the /proc/sys/ direcotry some of the editable options in the directory are:
  • crypto Related to encryption and other cryptographic services.
  • debug Related to debugging the kernel.
  • dev Related to specific hardware devices.
  • fs Related to file system data.
  • kernel Related to miscellaneous kernel functionality.
  • net Related to networking functions.
  • user Related to user space limitations.
  • vm Related to virtual memory management.

Manage Kernel Parameters

• The sysctl command displays or sets kernel parameters at runtime some of its options are:
  • -a Displays all parameters and their current values.
  • -w {parameter}= Sets a parameter value.
  • -p Loads sysctl settings from the specified file or from /etc/sysctl.conf if you don't provide a file name.
  • -e Ignores errors about unknown keys.
  • -r {pattern} Applies a command to parameters matching a given pattern, using extended regular expressions
• While you can modify kernel parameters with the sysctl command be very careful while doing so as it has a direct impact on the systems stability and security, though because of the fact that

Kernel Module Managment

• Kernel Modules are a system-level object that extend the functionality of the kernel. They can be dynamically loaded or unloaded into the kernel when required. Giving the kernel the ability to update or recompile itself without having to reboot.
• Kernel Modules have several advantages such as:
  • Reducing the burden on the kernel
  • Lower memory consumption
  • Extend the functionality of the system
• lsmod - Displays the current loaded Kernel modules
• modinfo - Displays information about a particular kernel module
• insmod - installs a module into the currently running kernel
• rmmod - removes a module from the currently running kernel
• modprobe - adds or removes modules from a kernel, its syntax is modprobe {options} {module names} and some of its options are:
  • -f - Forces the module to be inserted or removed
  • -n - Displays the output results without actually execting operations
  • -s - Prints errors to the system log rather then to stderr
  • -v - Enables Verbose more
• depmod command searches the contents of the /lib/modules// for each module, because a module may export a symbol that can be used by another module the depmod builds the modules.dep file by agregating all symbos being exported and used

Module Directories

• The usr/lib direcotry contains shared libraries and binaries for general programs and software packages.
• the /usr/lib/modules directory contains modules or different kernel versions installed
• The /usr/lib/modules//kernel/ contains several subdirectories
  • The arch subdirectory contains modules for architecture-specific support.
    -The crypto subdirectory contains modules for encryption and other cryptographic functions.
  • The drivers subdirectory holds modules for various types of hardware.
  • The fs subdirectory contains modules for different types of file systems.
  • The net subdirectory holds modules for networking components such as firewall and protocol.
• The /etc/modprobe.conf file is a configuration file that contains settings that apply to all modules loaded on the system
  • While on newer Linux distros this file is deapreciated and the /etc/modprobe.d/ directory is used instead and contains various .conf files
  • Files ending in .conf in the /etc/modprobe.d direcotry can use several commands and some are:
    • alias {alternative name} {module name} Specifies an alternative name for a module with a long name.
    • blacklist {module name} Ignores internal aliases, which occur when modules define their own aliases.
    • install {module name} {command} Runs the specified command without inserting the module into the kernel.

Kernel Panic

• Kernel Panic is a way for the system to respond to a fatal system error, it will dispaly an error to the user and allow the user to dump the current state of the kernel memory to a storage device for later debugging
• While Kernel Panic can happen at any time during operation it usually happens at boot and is usually caused by the following:
  • The kernel itself is corrupted or otherwise improperly configured.
  • The systemd program is not executed during boot, leaving the system unusable.
  • The kernel cannot find or otherwise cannot mount the main root file system.
  • Malfunctioning or incompatible hardware is loaded into the kernel on boot.

Troubleshooting Kernel Panic

• sudo journelctl -k -b -1 Displays the kernel logs from the previous boot
• memtest86 - To Check for memory errors
• smartctl - To check the health of the drives
• sudo grub-set-default {kernel-version} sudo reboot - to revert to an older or different version of the kernel
• sudo modprobe -r {modeule Name} - To check for fault drivers or modules
• echo "blacklist {module-name}" | sudo tee -a /etc/modeprobe.d/blacklist.conf - To add a driver or module to the block list to prevent it from being loaded into the kernel

Troubleshooting Kernel Corruption

• ls -l /boot - Ensure that vmlinuz and initramfs are present and not corupted
• sudo update-initramfs -u - To rebuild initramfs
• sudo apt-get install --reinstall linux-image-$(uname -r) - To reinstall the Kernel package
• Sudo update-grub - To check the bootloader configuration
• sudo fsck /path/to/disk - To repair the file system

Troubleshooting Kernel Issues

• The dmesg command can be used to look for potential issues with kernel components, and has several options:
  • -c Clears the kernel buffer after printing its contents.
  • -f {facility list} Restricts output to the specified comma-separated list of facilities. A facility is a component category that is producing messages, such as user for user-level messages.
  • -l {level list} Restricts output to the specified comma-separated list of levels. A level defines a message's nature and priority, such as notice for messages that aren't considered critical.
  • -e Displays a human-readable version of the time of each message as well as its delta, or the difference in time between subsequent messages.
  • -L Color-codes messages for easier readability.
  • -H Renders the output in a human-friendly format, combining both -e and -L options and using a text pager.
  • -h Lists the available options as well as the available facilities and levels.

Diagnosing and Resolving Driver Issues

• dmesg | grep -i error - To look for errors related to drivers
• lsmod | grep {module-name} - To check to see if a specific module is loaded
• sudo journalctl -k | grep driver - To check system logs for driver related errors
• modprobe {module-name} To load the required Driver
• sudo depmod -a - To rebuild the driver database
• sudo apt install {driver-package} - To insall missing drivers