Joshua Sarmiento
Copy linkThemeUnmute sounds
Blog

Enterprise Linux Server Administration, Hardening & Migration Guide

A comprehensive, production-ready handbook for Enterprise Linux (RHEL, Rocky Linux, AlmaLinux) systems administration. This guide covers service migration, database operations, SSH security, certbot…

By Joshua Sarmiento//8 min read/Security & hardening

Warning

Security Best Practice: Never store plaintext production passwords, KeePassXC master keys, or GitLab/Github Personal Access Tokens (PATs) in your public source code. Always use placeholder tokens in repository files and load credentials dynamically from secure vaults, environment variables, or local .env configs.

1. Quick Reference & Credentials Setup

Before running operations, secure your key management files and personal access tokens:

  • GitLab PAT (Personal Access Tokens): Ensure your GitLab PATs are stored in a secure credential helper or loaded as environment variables (e.g., export GITLAB_TOKEN="glpat-...").
  • KeePassXC Password Manager: Secure your master key offline. Do not write credentials in plaintext configuration files.

2. Post-Install Security Hardening (SELinux & File Permissions)

Correct permissions and SELinux contexts are essential for securing Apache web servers and preventing "permission denied" errors.

Directory & File Permissions

Run these commands to apply the standard production permissions for web content:

# Standard directories to 755 (or 750 for tighter security)
sudo find /path/to/sample-webapps/ -type d -exec chmod 0755 {} \;

# Standard files to 644
sudo find /path/to/sample-webapps/ -type f -exec chmod 0644 {} \;

# Hardened permissions for the configuration file
sudo chmod 600 wp-config.php

Apache Ownership

Set ownership of the directory structure to the Apache runtime user:

sudo chown -R apache:apache /var/www/html/sample-webapps/

SELinux Context Hardening

If SELinux is set to Enforcing (check via sestatus), Apache will be blocked from reading files unless the correct context is applied:

# Set web server read-only file context recursively
sudo chcon -R -t httpd_sys_content_t /var/www/html/sample-webapps/

# Apply persistent file contexts so they survive a system restorecon
sudo semanage fcontext -a -t httpd_sys_content_t "/var/www/html/sample-webapps(/.*)?"
sudo restorecon -R -v /var/www/html/sample-webapps/

Read/Write Context for Media Uploads & Plugins

Dynamic features like media uploads and plugin installations require write permissions:

# Allow Apache to write to the uploads directory
sudo chcon -R -t httpd_sys_rw_content_t /var/www/html/sample-webapps/wp-content/uploads/
sudo semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/sample-webapps/wp-content/uploads(/.*)?"
sudo restorecon -R -v /var/www/html/sample-webapps/wp-content/uploads/

# Allow Apache to manage plugins
sudo semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/sample-webapps/wp-content/plugins(/.*)?"
sudo restorecon -R -v /var/www/html/sample-webapps/wp-content/plugins/

Tip

If you need to undo a custom SELinux file context setting, use the -d flag:

sudo semanage fcontext -d "/var/www/html/sample-webapps/wp-content/uploads(/.*)?"
sudo restorecon -R -v /var/www/html/sample-webapps/wp-content/uploads/

Allow Network Connections & Home Directories

Enable Apache to make external network calls (e.g., API requests) and read home directories:

sudo setsebool -P httpd_enable_homedirs true
sudo setsebool -P httpd_can_network_connect true

3. SELinux Troubleshooting & Logs

To diagnose SELinux policy violations and check if Apache accesses are blocked:

# Check SELinux status
sestatus

# Temporarily set SELinux to Permissive mode (for debugging only)
sudo setenforce 0

# Set SELinux back to Enforcing mode
sudo setenforce 1

# List all current SELinux booleans
getsebool -a

# Analyze audit logs for recent access denials (AVCs)
sudo ausearch -m avc -ts recent
sudo tail -n 50 /var/log/audit/audit.log | grep denied

Note

If ausearch or semanage commands are missing, install the policy administration tools:

sudo dnf install -y policycoreutils-python-utils

4. SSH Key Management & PuTTY Conversion

To authorize a SSH public key generated via PuTTY (.pub format), convert the key into standard OpenSSH format:

# 1. Paste the Putty-formatted key into a temporary file
nano /home/temp/dev1.pub

# 2. Convert and append the key to authorized_keys
ssh-keygen -i -f /home/temp/dev1.pub >> ~/.ssh/authorized_keys

# 3. Clean up the temporary file
rm /home/temp/dev1.pub

5. Multiple PHP Version Management (via Remi RPM)

For enterprise web environments running legacy and modern codebases simultaneously:

Install the Remi Repository

sudo dnf install -y https://rpms.remirepo.net/enterprise/remi-release-9.rpm

Install PHP 8.4 & 8.3 alongside FPM

# Install PHP 8.3 and standard extensions (the legacy line)
sudo dnf install -y php83-php-fpm php83-php-mysqlnd php83-php-gd php83-php-mbstring php83-php-xml

# Install PHP 8.4 and standard extensions (the current line)
sudo dnf install -y php84-php-fpm php84-php-mysqlnd php84-php-gd php84-php-mbstring php84-php-xml

Manage FPM & Web Services

# Start and enable both PHP-FPM daemons
sudo systemctl enable --now php83-php-fpm
sudo systemctl enable --now php84-php-fpm

# Start Apache HTTPD and MariaDB
sudo systemctl enable --now httpd
sudo systemctl enable --now mariadb

6. Database Operations & Backups

Create stable backups using transaction-safe dumping practices:

Exporting Databases

# Standard backup (includes stored routines and triggers)
mysqldump -u root -p --opt --routines --triggers [database_name] > backup/[database_name]_backup.sql

# High-concurrency safe backup (prevents lock tables for InnoDB)
mysqldump -u [db_user] -p --routines --triggers --single-transaction [database_name] > backup/[database_name]_backup.sql

Importing Databases

# 1. Log into MariaDB/MySQL CLI
mysql -u root -p

# 2. Setup the database and user permissions
CREATE DATABASE target_db_name;
CREATE USER 'db_user'@'localhost' IDENTIFIED BY 'YOUR_STRONG_PASSWORD';
GRANT ALL PRIVILEGES ON target_db_name.* TO 'db_user'@'localhost';
FLUSH PRIVILEGES;
EXIT;

# 3. Import the backup SQL file
mysql -u root -p target_db_name < /tmp/[database_name]_backup.sql

7. Server Migration Flow

A structured checklist for moving virtual hosts and assets from a source server to a target server:

Step 1: Compress Assets & Measure Directory Size

# Check folder size on source server
du -sh /var/www/html/your-folder

# Archive files
tar -czvf website_backup.tar.gz -C /var/www/html/ your-folder

Step 2: Transfer Assets & Database Dumps

scp website_backup.tar.gz database_dump.sql root@[TARGET_SERVER_IP]:/tmp/

Step 3: Extract and Set Up Directory Structure

Create target virtual host directories and extract files stripping outer container names if necessary:

# Create directory structure
mkdir -p /home/sample.com/www/html/sample-web-apps/public_html

# Extract tarball
tar -xzvf /tmp/website_backup.tar.gz --strip-components=1 -C /home/sample.com/www/html/sample-web-apps/public_html/

Step 4: Configure Directory Traversal Permissions

Ensure Apache can traverse through directories from root to the virtual host's public_html:

# Set search execution permissions on parents
chmod 711 /home/sample.com
chmod 711 /home/sample.com/www
chmod 711 /home/sample.com/www/html
chmod 755 /home/sample.com/www/html/sample-web-apps/public_html

# Assign ownership to the web server
sudo chown -R apache:apache /home/sample.com/

Step 5: Configure Apache VirtualHost

Create a config file (e.g. /etc/httpd/conf.d/sample.conf) with proxy routing to your chosen PHP-FPM socket version:

<VirtualHost *:80>
    ServerName sample.com
    ServerAlias www.sample.com
    DocumentRoot /home/sample.com/www/html/sample-web-apps/public_html
    DirectoryIndex index.php index.html

    <Directory "/home/sample.com/www/html/sample-web-apps/public_html/">
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    # Bind request handling to the PHP 8.2 FPM Socket
    <FilesMatch \.php$>
        SetHandler "proxy:unix:/var/opt/remi/php84/run/php-fpm/www.sock|fcgi://localhost"
    </FilesMatch>
</VirtualHost>

Step 6: Test and Reload Web Server

# Disable the default Apache welcome screen
sudo mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.bak

# Validate Apache configurations syntax
sudo httpd -t
sudo httpd -S

# Restart services to apply changes
sudo systemctl restart httpd
sudo systemctl restart php84-php-fpm

Step 7: Monitor Application Logs

# View Apache errors
tail -f /var/log/httpd/error_log

# View PHP-FPM execution errors
tail -f /var/opt/remi/php84/log/php-fpm/error.log

8. Node.js Service Management (PM2)

Deploy and maintain Node.js API services under a PM2 process manager:

# Start Node.js service with custom name and save layout
pm2 start dist/index.js --name "leasing-api"
pm2 save
pm2 startup

Handling PM2 Hanging & Unresponsiveness

If PM2 becomes unresponsive, force terminate the daemon process and boot it back up:

# Find the master PM2 process ID
ps aux | grep pm2

# Kill the process forcefully
kill -9 [PM2_PID]

# View process list again to re-init
pm2 list

9. Cloudflare SSL Certificates (Certbot + ACME DNS Challenge)

Configure wildcard or domain certificates using Certbot and Cloudflare DNS validation:

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini \
  --deploy-hook "systemctl reload httpd" \
  -d sample.com \
  -d www.sample.com
  • DNS Verification: Certbot creates an _acme-challenge.www.sample.com TXT record in your DNS settings via API.
  • Certificate Output Path:
    • Certificate: /etc/letsencrypt/live/sample.com/fullchain.pem
    • Private Key: /etc/letsencrypt/live/sample.com/privkey.pem

Renewal

Certbot's packaged certbot.timer already runs certbot renew twice a day, so there is nothing further to schedule. The --deploy-hook above is the part that matters: certbot stores it in the renewal configuration for that certificate, and runs it only when a renewal actually succeeds — never on an ordinary timer tick. Without it, Apache keeps serving the old certificate until you reload it by hand or reboot.

10. Kernel Core Cleanup & Upgrades

To free up space on /boot by removing old or unused kernel versions:

# 1. List currently installed kernel-cores
rpm -qa | grep kernel-core

# 2. Check current active boot options
sudo grubby --info=ALL | grep -E "kernel|index"

# 3. Set the default booting kernel version
sudo grubby --set-default /boot/vmlinuz-5.14.0-503.40.1.el9_5.x86_64

# 4. Reboot system to run on the new default kernel
sudo reboot

# 5. Verify the current active kernel version
uname -r

# 6. Safe delete the old kernel-core packages
sudo dnf remove kernel-core-5.14.0-284.11.1.el9_2.x86_64

# 7. List kernels and clean unused utility packages
dnf list | grep kernel

# Do not remove policycoreutils-python-utils here — `semanage`, used
# throughout this guide, lives in that package.
sudo dnf remove python3-setuptools

11. Security Headers Configuration

Add security policies inside your Apache configuration or .htaccess to mitigate clickjacking and injection threats:

Header set Strict-Transport-Security "max-age=31536000; includeSubDomains"
Header set X-Content-Type-Options "nosniff"
Header set X-Frame-Options "sameorigin"
Header set Referrer-Policy "strict-origin-when-cross-origin"
Header always set Permissions-Policy "camera=(), microphone=(), geolocation=()"
Header set Content-Security-Policy "default-src * data: blob:; script-src https: blob: 'unsafe-inline' 'unsafe-eval'; style-src https: 'unsafe-inline'; frame-src https: blob: self"

Warning

The CSP above is a permissive starting point, not a finished policy — default-src * plus 'unsafe-inline' and 'unsafe-eval' is what a legacy WordPress stack typically needs to render at all. Treat it as a baseline to tighten, and rebuild it for your own asset origins instead of copying someone else's allow-list. X-XSS-Protection is deliberately absent: modern browsers removed the auditor, and the header can introduce its own issues.

Need Help with Your Infrastructure?

If you're looking to implement similar security setups, migrate servers, or configure automated CI/CD pipelines, feel free to reach out!

Contact Me:

Tools in this guide

BackendPHPMariaDB
Cloud & OpsApacheGit
Security & SystemsRocky Linux

Each tool links to where it sits in my full stack.