Category: Automation

  • Configure partition table, format and auto mount disks. MBR – EXT4

    Connect the External HDD: Ensure that your external HDD is connected to your Debian system.

    Identify the Device: Use the lsblk or fdisk -l command to identify the device name of your external HDD. It will typically be something like /dev/sdX, where X is a letter assigned to the drive.

    lsblk

    Partition the Drive with MBR: Use the fdisk command to create an MBR partition on the external HDD.

    sudo fdisk /dev/sdX

    Once inside the fdisk program do the following:

    Create the ext4 Filesystem: After creating the partition, use the mkfs.ext4 command to create an ext4 filesystem.

    mkfs.ext4 /dev/sdX1

    Replace /dev/sdX1 with the actual partition identifier you created in the previous step.

    Label the Filesystem (Optional): You can optionally label the ext4 filesystem for easier identification. Replace NEW_LABEL with the desired label for your filesystem. That can for example be the model of the hard disk or it’s purpose.

    e2label /dev/sdX1 NEW_LABEL

    Mount the Filesystem: Create a directory where you want to mount the hard disk and mount the filesystem.

    sudo mkdir /media/LABEL sudo mount /dev/sdX1 /media/LABEL

    Adjust the mount point (/media/LABEL) according to your preference.

    Now, your external HDD should be formatted with MBR and have an ext4 filesystem. If you want the drive to be automatically mounted on boot, you may need to add an entry to the /etc/fstab file which is show below.

    Automounting disk with fstab

    Identify the UUID of the Partition: Use the blkid command to identify the UUID of the partition on your external HDD. The UUID uniquely identifies the partition, and using it in /etc/fstab helps avoid issues if the disk order changes.

    Replace /dev/sdX1 with the actual partition identifier.

    sudo blkid /dev/sdX1

    Take note of the PTUUID number without quotes. It should look something like this: “c381c2aa-044b-415a-b901-2a6a374b2591“.

    Edit the /etc/fstab file: Open the /etc/fstab file in a text editor using a command like sudo nano or sudo vim. Add a new line with the following information:

    UUID=your_partition_uuid /media/LABEL ext4 defaults 0 2

    Replace your_partition_uuid with the UUID you obtained in the first step, and adjust the mount point (/media/LABEL) if needed.Example using nano:

    sudo nano /etc/fstab

    Add the line to automount the disk with default values:

    UUID=c381c2aa-044b-415a-b901-2a6a374b2591 /media/LABEL ext4 defaults 0 2

    Or with more specific options:

    UUID=c381c2aa-044b-415a-b901-2a6a374b2591 /media/LABEL ext4 rw,relatime,nofail,errors=remount-ro 0 2
    1. rw:
      • Stands for “read-write.”
      • This option allows both read and write operations on the filesystem. It specifies that the filesystem should be mounted with read and write permissions.
    2. relatime:
      • This option stands for “relative atime.”
      • With relatime, the access time of files is updated only if the current access time is earlier than the modification time or the inode creation time. It’s an optimization over the traditional atime update mechanism, helping to reduce write operations to the filesystem.
    3. nofail:
      • This option indicates that if the filesystem cannot be mounted, the failure should not be considered fatal to the system boot process. If the device is not present or there are issues with the filesystem, the system will continue booting without the specified filesystem being mounted.
    4. errors=remount-ro:
      • Specifies the action to be taken in case of errors on the filesystem.
      • If errors are encountered, the filesystem will be remounted in read-only mode (ro). This is a safety measure to prevent further potential damage and data loss in case of filesystem errors.
    5. 0 2:
      • These are the dump and pass fields, respectively.
        • The dump field (0) indicates whether the filesystem should be backed up using the dump command. A value of 0 means no automatic backup.
        • The pass field (2) is used by the fsck command to determine the order in which filesystems are checked at boot time. A value of 2 typically means the filesystem will be checked after the root filesystem.

    In summary, the options in your /etc/fstab entry specify that the filesystem should be mounted with read-write permissions, use relative atime for optimization, not be considered critical for system boot (nofail), remount in read-only mode in case of errors, and be checked after the root filesystem during the boot process.

    Create the Mount Point (if not already created): If you haven’t created the mount point earlier, create it using:

    sudo mkdir /media/LABEL

    Mount All Filesystems in /etc/fstab: To mount all filesystems listed in /etc/fstab, you can use the following command:

    sudo mount -a

    To auto mount as a non root user

    To automount a disk with specific user permissions using /etc/fstab, you can utilize the user and noauto options along with the uid, gid, and umask options.

    1. Determine the UID and GID of the user you want to mount the disk as. You can find this information by running the following commands:
    id -u username
    id -g username
    1. Determine the UUID of the disk you want to mount. You can find this information using the blkid command:
    sudo blkid
    1. Edit the /etc/fstab file using a text editor:
    UUID=your_disk_uuid /mnt/mount_point filesystem defaults,user,noauto,uid=your_user_id,gid=your_group_id,umask=022 0 0
  • Configure passwordless authentication

    Server Side

    1- In this scenario we are going to install an ssh server and configuring it so that it only accepts certificates to log in.

    sudo apt-get install openssh-server

    2- In the Remote Server: Ensure that password based ssh login is allowed in the ssh server configuration before copying your public key.. Edit the ssh configuration file after you have a working certificate based authentication. You should skip this step for now:

    sudo nano /etc/ssh/sshd_config

    Set the following options:

    PasswordAuthentication no
    ChallengeResponseAuthentication no
    UsePAM no
    KbdInteractiveAuthentication no

    Save and exit the file.

    sudo systemctl reload/restart sshd
    or
    sudo service ssh restart

    Client Side

    sudo apt-get install openssh-client

    Navigate to /home/.ssh

    3- Generate an SSH key pair (if you don\’t already have one. This command generates an RSA key pair with 4096 bits.

    ssh-keygen -t rsa -b 4096

    Or you can generate the more modern version with this command:

    ssh-keygen -t ed25519 -a 100 -f .ssh/testkey

    Give it a meaningful name and provide a password (optional)

    Add Your SSH Key to the SSH Agent: You need to add a new identity using your SSH private key to the SSH agent with the following command:

    ssh-add ~/.ssh/id_rsa (NOT the id_rsa.pub!)

    Ensure SSH Agent is Running: ssh-copy-id relies on an SSH agent to manage your keys. If you need to stop it, type eval \”$(ssh-agent -k)\”

    eval "$(ssh-agent -s)"
    Or
    eval $(ssh-agent -s)

    OPTIONAL: Make sure that ssh-agent is running and that will it start at system boot in your local session and adding a desired private key:

    nano ~/.bashrc

    4- Add the following line at the end of the file :

    eval "$(ssh-agent -s)"
    ssh-add PATH_TO_YOUR_PRIVATE_KEY

    To check if the SSH agent is running, you can use the ssh-add command with the -l option. If the ssh agent is running and has loaded any keys, you will see a list of the loaded key fingerprints. Open a terminal and run the following command:

    ssh-add -l

    Another way to check if the SSH agent is running, is to list the environment variables related to SSH.

    If the SSH agent is running, this command will print the path to the SSH agent socket. If it\’s not running, the command will produce no output. Run the following command:

    echo $SSH_AUTH_SOCK

    You can also see if ssh agent is running by showing it\’s PID.

    echo $SSH_AGENT_PID
    • There is another way to load the identities beside running the ssh agent and that is by creating a file named config inside the .ssh folder with the following information per server you want to connect to. The identities configured will be loaded at the time to try to connect via ssh.
    Host server
           Hostname server_ip_address
           User remote_user
           IdentityFile /home/local_user/.ssh/identity_file

    5- Make sure that ssh password authentication on your remote server is enabled. You\’ll need to copy your public key to the remote server using ssh-copy-id:

    ssh-copy-id -i /path/to/id_rsa.pub admin@remote_server_ip

    You are going to be prompted to type the password of your remote user to accept the public key.

    6- Once that\’s done, log in and if all goes well, you will connect to the remote machine without a password.

    ssh admin@remote_server_ip
  • Send your public IP to your email – MSTP-MTA

    Send your public IP to your email – MSTP-MTA

    For those of us having a server at home with a basic home internet connection, the issue is how to easily bypass the problem of having port 25 blocked by our ISP. To quickly go around it you can setup an msmtp client to login to your personal email to send useful information from your running server. In this case is sending the public IP address just in case the IP address gets renewed so you can promptly update your DNS records for example if no automatic alternative is present. As far as I know wordpress DNS configuration does not have a way to be updated by an API so there you go…

    Manual Configuration

    For manual ip/dns public ip configuration and update, set up a msmtp client in your server and configure it using your external email provider credentials. This is in the scenario where you are unable to auto update your ip in your dns records. We’ll use msmtp-mta to setup an email event reminder so you can always be aware of any changes to act accordingly.

    sudo apt-get install msmtp-mta

    Create or edit the config file for msmtp. msmtp will look for a file named .msmtprc in your home folder by default (.file means it is an occult file and wont be displayed with ls unless specified). This file must contain the instructions and your personal credentials so that msmtp can use them to send your ip address to your own email securely with tls. This file creates an “account” readable by msmtp to be used to send email. In thiis example we will be storing the password in plain text inside the configuration file so it must be in a secure account only accessible to the owner of the external email service used to send emails. TLS will be enabled.

    nano /home/user/.msmtprc

    Edit with the following template:

    account admin
    host #EMAIL_DOMAIN E.g. google.com
    port 587 # verify port functionality. Port 25 is usually blocked by your ISP.
    auth on
    user admin@google.com #Try with username@yourmailserver.com first
    password TYPE_YOUR_EMAIL_PASSWORD_FOR_USER_ADMIN
    from admin@google.com
    tls on
    tls_starttls on
    tls_fingerprint #Get the SHA256 fingerprint from your email provider
    tls_trust_file /etc/ssl/certs/ca-certificates.crt #verify this location on your system.

    Be aware that if the information is not correctly typed, msmtp might give you misleading error codes which will make troubleshooting very hard. Check the correct details of our email provider before. An example of this is the real root server of your email provider might not be the .net or .com domain extension but rather a .us or pl or any other extemsion if they have the root email services in a foreign country. Next, create and configure a script to be used by msmtp in your home folder. Install curl if is not installed already.

    If problems arise with permissions, make sure to set permissions 600 for your user to the .msmtp file.

    sudo apt-get install curl
    sudo nano /home/user/.msmtprc_email_new_pub_ip.sh

    The following script will ask your public ip to ifconfig.me using the curl command and be stored in the variable IP_ADDRESS. This value will be in the subject of your email and passed to msmtp. msmtp will load the account details and send the email wherever you want to receive it. To make further comparisons the old ip address will be saved to a file to be compared the next time this script is executed.

    #!/bin/bash
    
    # Get current IP address
    IP_ADDRESS=$(curl -s ifconfig.me)
    
    # Define variables
    MSMTP_ACCOUNT=john
    RECIPIENT_EMAIL=john@gmail.com
    MSMTP_CONFIG_FILE=/home/john/.msmtprc
    IP_HISTORY_FILE=/home/john/.msmtprc_ip
    
    # Check if the IP history file exists; if not, create it
    if [[ ! -f $IP_HISTORY_FILE ]]; then
        echo "$IP_ADDRESS" > "$IP_HISTORY_FILE"
        exit 0  # Exit since there's no previous IP to compare
    fi
    
    # Read the historic IP address
    IP_HISTORIC=$(cat "$IP_HISTORY_FILE")
    
    # Compare the current IP address with the historic one
    if [[ "$IP_HISTORIC" != "$IP_ADDRESS" ]]; then
        # Send the email
        echo "Subject: UPDATE $IP_ADDRESS" | msmtp -a "$MSMTP_ACCOUNT" "$RECIPIENT_EMAIL"
        
        # Update the historic IP address
        echo "$IP_ADDRESS" > "$IP_HISTORY_FILE"
    else
        echo "IP address has not changed. No email sent."
    fi

    Save the file, make it executable and test if is working correctly.

    sudo chmod +x .msmtprc_email_new_pub_ip.sh
    sudo ./home/user/.msmtprc_email_new_pub_ip_update.sh

    Automate the task to send your public ip by email every 12 hours

    If all went well, continue creating a msmtprc_email_new_pub_ip.service and a msmtprc_email_new_pub_ip.timer files to run them as systemd services to automate sending emails right after system boot and every 12 hours.

    sudo nano /etc/systemd/system/msmtprc_email_new_pub_ip.service
    [Unit]
    Description=Runs a script to send public ip address via email.
    
    [Service]
    Type=simple
    User=admin
    WorkingDirectory=/home/admin/
    ExecStart=/home/user/.msmtprc_email_new_pub_ip.sh
    
    [Install]
    WantedBy=multi-user.target

    Save and exit.

    sudo nano /etc/systemd/system/msmtprc_email_new_pub_ip.timer
    [Unit]
    Description=Timer for msmtprc_email_new_pub.service file

    [Timer]
    OnCalendar=--* 5,18:00
    RandomizedDelaySec=12h
    Persistent=true

    [Install]
    WantedBy=timers.target

    Save file and exit.

    Reload the daemon

    sudo systemctl daemon-reload

    Enable both services

    sudo systemctl enable msmtprc_email_new_pub_ip.service msmtprc_email_new_pub_ip.timer

    Verify both services are running correctly with “sudo status name_of_service”

    Reboot your system and verify again.

    Check your email. If it’s successful, wait 12 hours for the next email.

    Another better option is using only one service file to manage the execution of the script and the timer. The timer here is going to trigger the execution of the script every hour and right after boot. I’m doing this now just because I can 🙂

    [Unit]
    Description=Sends public ip address to specified email address using msmtp.
    After=network-online.target
    
    [Service]
    Type=simple
    User=user
    WorkingDirectory=/home/user
    ExecStart=/home/user/.msmtprc_email_new_pub_ip.sh
    Restart=always
    RestartSec=3600
    StartLimitInterval=0
    
    [Install]
    WantedBy=multi-user.target
    

  • IP address autoupdate – Cloudflare

    IP address autoupdate – Cloudflare

    Create an API token in Cloudflare

    Log in to your cloudflare’s dashboard and enter the domain you want to manage.

    • Copy your “Zone ID”
    • Click to “Get your API token”
      • Create two permissions:
      • Zone | DNS | Read
      • Zone | DNS | Edit
    • Edit Zone resources
      • Include | Specific zone | YOUR_DOMAIN

    Save your API key token and save the number generated. Once you exit the page you won’t be able to get it again.


    apt install git

    Choose a good location to save the following git folder. It contains the script to update your ip.

    git clone https://github.com/K0p1-Git/cloudflare-ddns-updater.git

    Cd into folder, copy the template file, rename it and make a copy. Edit your new copy with your own details like so (I’m not using the global API):

    auth_email="YOUR_EMAIL"     # The email used to login 'https://dash.cloudflare.com'
    auth_method="token"         # Set to "global" for Global API Key or "token" for Scoped API Token
    auth_key="YOUR_API_TOKEN"   # Your API Token or Global API Key
    zone_identifier="YOUR_ZONE_ID"    # Can be found in the "Overview" tab of your domain
    record_name="YOUR_ROOT_DOMAIN"    # Which record you want to be synced
    ttl="3600"                        # Set the DNS TTL (seconds)
    proxy="false"                     # Set the proxy to true or false
    sitename="SITE_TITLE"          # Title of site "Example Site"
    slackchannel=""                                     # Slack Channel #example
    slackuri=""                                         # URI for Slack WebHook "https://hooks.slack.com/services/xxxxx"
    discorduri=""                                       # URI for Discord WebHook "https://discordapp.com/api/webhooks/xxxxx"
    sudo chmod +x [path_to_your_script]

    Schedule automatic verification and update of your public IP

    After that a cron job can be created to run the script periodically. Create it with sudo so it is run by root and not by your admin user. Reason if that if the system restarts and you are not around to log in, the cron job for your user won’t start until you log in. This is undesirable if your server is meant o run partially supervised.

    sudo crontab -e

    Example of adding a cron job to be executed every 15 mins everyday.

    */15 * * * * /home/user/cloudflare-ddns-updater/cloudflare-template.sh

    Save and exit.

    Update cron service.

    systemctl restart cron.service

    Using a systemd service file

    Instead of a cron job you can create a service file in /etc/systemd/system and save it with the following directives. This might be more advantageous because if the service stops for whatever reason you can be easily be notified to solve the problem. Here is an example of a service file where is defined the path to the script and the time interval (every 12 hours):

    [Unit]
    Description=Public IP address verification and update for cloudflare dns records.
    After=network-online.target
    
    [Service]
    Type=simple
    ExecStart=/home/user/cloudflare-ddns-updater/cloudflare-template.sh
    Restart=always
    RestartSec=43200
    StartLimitInterval=0
    
    [Install]
    WantedBy=multi-user.target
    

    sudo systemctl daemon-reload

    sudo systemctl restart [name_of_your_service_file]

    Credits to Jason K. and collaborators
    https://github.com/K0p1-Git/cloudflare-ddns-updater/tree/main