How to Autostart Apps on Your Server

[TOC]

1. Title: How to Autostart Apps on Your Server
2. Review Date: Fri, Apr 12, 2024
3. url: -

How to Autostart Apps on Your Server

we can try systemd --user modules

First of all, write the service in the ~/.config/systemd/user/ folder

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

[Unit]
Description=My TorchSErve Server
Wants=network-online.target
After=network-online.target


[Service]
Type=forking
ExecStart=/datadrive/run_torchserve.sh

[Install]
WantedBy=multi-user.target

Note that Type=forking is important if you want to maintain the session, especially when you want to maintain a tmux

Understanding Type=forking in systemd

In systemd, the Type= option in the service unit file specifies the service’s process start-up behavior, influencing how systemd manages and tracks the service’s main process. For services that start by forking off a child process (which continues running after the parent exits), Type=forking is the appropriate choice. This is a common pattern for traditional daemons that double-fork to detach themselves from the controlling terminal and run in the background.

When Type=forking is set, systemd expects the service to:

• Start a parent process.
• Fork a child process from the parent.
• Have the parent process exit.
• Continue running in the background with the child process.

Systemd then tracks the child process as the main service process. This is useful for services that need to perform some initial setup in the parent process before running the service logic in the background.

Why Type=forking Works with tmux

tmux commands like tmux new-session -d start a new session in detached mode and then immediately exit. This behavior is somewhat similar to forking, in that the command that starts the session exits quickly, but the session itself (and any processes running within it) continues in the background, independent of the shell it was started from.

Here’s why setting Type=forking helps in your systemd unit file for tmux:

• Initial Process Exit: The tmux command used in ExecStart exits after creating the detached session. With Type=simple (the default if Type= is not specified), systemd expects the service’s main process to stay active. When the tmux command exits, systemd thinks the service has stopped.
• Background Continuation: By using Type=forking, you signal to systemd that the exiting of the initial process (the tmux command) is expected behavior and that it should consider the background continuation of the session as the service running successfully. Systemd then treats the service as active even after the tmux command itself has exited, as long as there’s a background process associated with it.

Important Considerations

While Type=forking helps keep the tmux session active under systemd control, it’s important to note a few considerations:

• Process Tracking: Systemd might have difficulty tracking the exact process of the tmux session as the service’s main process since tmux doesn’t behave like a traditional daemon (it doesn’t provide a PID file for the child process). Systemd relies on the PID of the last background process, but in the case of tmux, this is managed internally by tmux.
• Service Management: Actions like stopping or restarting the service rely on correctly identifying the process to manage. Ensure your ExecStop command correctly terminates the tmux session.

Using Type=forking in this way is a bit unconventional but can be effective for specific use cases like yours. It’s a clever solution to integrate tmux session management within the systemd framework, especially when you need to ensure a tmux session starts automatically and persists in the background.

Reload systemd daemon

1
2
3
4
5
6

# reload daemon
systemctl --user daemon-reload
# enable service
systemctl --user enable mytorchserve.service
# enable no logging start
sudo loginctl enable-linger $USER

Check the log

1

journalctl --user-unit=mytorchserve.service -f -e

Manually start the service

systemctl --user restart mytorchserve.service

User systemctl and User runtime directory

ref: https://superuser.com/questions/1561076/systemctl-user-failed-to-connect-to-bus-no-such-file-or-directory-debian-9

Solution: Using sudo loginctl enable-linger $USER makes systemd keep the user’s systemd --user process running even after the user logs out. This is crucial for running background services under that user without requiring them to be continuously logged in.

monitor by loginctl list-users

Issue: If XDG_RUNTIME_DIR isn’t set in crontab or other auto task-scheduling scripts, then tools like systemctl --user might fail because they cannot find the correct runtime directory to communicate with systemd --user.

Solution: export XDG_RUNTIME_DIR=/run/user/$(id -u);

Add-on: Cron service

1
2
3
4

# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
*/5 * * * * export XDG_RUNTIME_DIR=/run/user/$(id -u); /datadrive/check_service.sh >> /datadrive/check_service_cron.log 2>&1

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

#!/bin/bash

# Perform the curl request and capture the response
response=$(curl http://10.102.5.4:8080/ping 2>&1)
# echo "Response ${response}"

# Check if the response contains "Healthy"
if [[ "$response" == *'"status": "Healthy"'* ]]; then
  echo "All good at $(date)"
else
  # Check for 'Connection refused' in the response
  if [[ "$response" == *"Connection refused"* ]]; then
    echo "Restart Service at $(date)"
    systemctl --user restart mytorchserve.service
  fi
fi

Tmux activate server example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

#!/bin/bash
sessname="server"
echo $USER is running the script
/usr/bin/tmux new-session -d -s "$sessname"
if [[ $? -eq 1 ]]; then
    /usr/bin/tmux kill-session -t "$sessname"
    /usr/bin/tmux new-session -d -s "$sessname"
fi

sleep 5

echo `/usr/bin/tmux ls`

tmux send-keys -t "$sessname" "cd /datadrive ; sudo systemctl stop docker.service ; cp docker.service.config.bak /lib/systemd/system/docker.service" Enter

tmux send-keys -t "$sessname" "sudo systemctl daemon-reload ; sudo systemctl start docker.service" Enter

tmux send-keys -t "$sessname" "cd /datadrive/avmmodel_project/torchserve-deployment ; sudo docker stop torchserve-dev-apr-12-2 ; sudo docker container rm torchserve-dev-apr-12-2" Enter

sleep 2

tmux send-keys -t "$sessname" 'sudo docker rmi -f  $(sudo docker images -f "dangling=true" -q)' Enter

sleep 2

tmux send-keys -t "$sessname" 'sudo docker build -t avmmodel_image . -f torchserve.dockerfile' Enter

sleep 2 

tmux send-keys -t "$sessname" 'sudo docker run -p 8080:8089 --name torchserve-dev-apr-12-2 -v ./torchserve:/torchserve avmmodel_image' Enter


tmux split-window -v -t "$sessname"


tmux send-keys -t "$sessname:0.1" 'htop' Enter


tmux split-window -h -t "$sessname"


tmux send-keys -t "$sessname:0.2" 'sudo watch -n 1 df -h' Enter


tmux split-window -h -t "$sessname"

tmux send-keys -t "$sessname:0.3" 'cd /datadrive' Enter

Relocate the docker lib

1. Edit docker service configuration with: sudo vi /lib/systemd/system/docker.service
2. Locate the Exec line: ExecStart=/usr/bin/dockerd -H fd:// –containerd=/run/containerd/containerd.sock
3. Add a parameter that points docker to the new directory: ExecStart=/usr/bin/dockerd –data-root /second_drive/docker -H fd:// –containerd=/run/containerd/containerd.sock

1
2
3
4
5
6

sudo systemctl stop docker.service
# edit your docker.service.config.bak
sudo chown root:root docker.service.config.bak # make sure the owner for docker config file is root
cp docker.service.config.bak /lib/systemd/system/docker.service
sudo systemctl daemon-reload
sudo systemctl start docker