syslog-ng 當 Log Server(docker)

  • 安裝環境 alpine + docker compose
  • 預計安裝好的目錄結構

    .
    ├── docker-compose.yml
    ├── log
    │   ├── demo-v2-66
    │   │   ├── daemon-20250507.log
    │   │   └── kern-20250507.log
    │   ├── demo-v2-67
    │   │   ├── authpriv-20250507.log
    │   │   ├── daemon-20250507.log
    │   │   └── kern-20250507.log
              :
              :
    │   ├── demo-v2-79
    │   │   ├── auth-20250507.log
    │   │   ├── authpriv-20250507.log
    │   │   ├── daemon-20250507.log
    │   │   ├── kern-20250507.log
    │   │   ├── syslog-20250507.log
    │   │   └── user-20250507.log
    │   ├── messages
    │   └── messages-kv.log
    └── syslog-ng
        └── config
            ├── log
            │   ├── current
            │   ├── lock
            │   └── state
            ├── syslog-ng.conf
            ├── syslog-ng.ctl
            ├── syslog-ng.persist
            └── syslog-ng.pid

  • docker-compose.yml
    https://raw.githubusercontent.com/tryweb/docker-compose/refs/heads/main/syslog-ng/docker-compose.yml
    services:
      syslog-ng:
        image: lscr.io/linuxserver/syslog-ng:latest
        container_name: syslog-ng
        environment:
          - PUID=1000
          - PGID=1000
          - TZ=Asia/Taipei
          - LOG_TO_STDOUT= #optional
        volumes:
          - ./syslog-ng/config:/config
          - ./log:/var/log #optional
        ports:
          - 514:5514/udp
          - 601:6601/tcp
          - 6514:6514/tcp
        restart: unless-stopped
     

    https://raw.githubusercontent.com/tryweb/docker-compose/refs/heads/main/syslog-ng/docker-compose.yml

  • 設定檔 syslog-ng\configsyslog-ng.conf
    https://raw.githubusercontent.com/tryweb/docker-compose/refs/heads/main/syslog-ng/syslog-ng/config/syslog-ng.conf
    #############################################################################
    # syslog-ng.conf file configured to collect logs from different hosts
    # into separate directories and rotate logs to keep for 3 months
     
    @version: 4.2
    @include "scl.conf"
     
    options {
      time_reopen(10);
      chain_hostnames(off);
      keep_hostname(yes);
      flush_lines(0);
      use_dns(no);
      use_fqdn(no);
      create_dirs(yes);
      keep_timestamp(yes);
    };
     
    source s_local {
      internal();
    };
     
    source s_network_tcp {
      syslog(transport(tcp) port(6601));
    };
     
    source s_network_udp {
      syslog(transport(udp) port(5514));
    };
     
    # 原有的本地日誌目的地
    destination d_local {
      file("/var/log/messages");
      file("/var/log/messages-kv.log" template("$ISODATE $HOST $(format-welf --scope all-nv-pairs)\n") frac-digits(3));
    };
     
    # 新增:按照主機 IP 分類的目的地,並加入日誌輪替功能
    destination d_per_host {
      file(
        "/var/log/$HOST/$FACILITY-$YEAR$MONTH$DAY.log"
        template("$ISODATE $LEVEL $PROGRAM $MSG\n")  # 使用 $PROGRAM 顯示 Docker 的 tag
        create_dirs(yes)
        dir_perm(0755)
        perm(0644)
        owner("root")
        group("root")
        overwrite_if_older(7776000)  # 90天 = 90 * 24 * 60 * 60 = 7,776,000秒
      );
    };
     
    # 本地日誌處理
    log {
      source(s_local);
      destination(d_local);
    };
     
    # 網絡日誌處理:按主機分類
    log {
      source(s_network_tcp);
      source(s_network_udp);
      destination(d_per_host);
    };

    https://raw.githubusercontent.com/tryweb/docker-compose/refs/heads/main/syslog-ng/syslog-ng/config/syslog-ng.conf

    • 目前設定檔會依據不同來源主機, 自動依照主機名稱或 IP 建立各自的 log 目錄 Exp. log/demo-v2-66 , log/demo-v2-67
    • 保留三個月的 log 檔案
  • 啟動服務

    docker compose pull
    docker compose up -d

  1. 先在 host 加上一顆硬碟 Exp. sdb
  2. 將 sdb 建立為 lvm + ext4

    fdisk /dev/sdb

    • 詳細處理畫面
  3. 參考 Linux 建立 LVM 磁碟方式

    pvcreate /dev/sdb1
    vgcreate vglogdata /dev/sdb1
    lvcreate -l +100%FREE -nlogdata vglogdata
    mkfs.ext4 /dev/vglogdata/logdata

    • 詳細處理畫面
  4. 掛上路徑 Exp. /logdata

    mkdir -p /logdata
    chown 1000:1000 /logdata
    vi /etc/fstab

    :
    /dev/vglogdata/logdata  /logdata        ext4    rw      0 1
    mount /logdata
    df -h
  5. 修改 docker-compose.yml 內容

    vi docker-compose.yml

    :
        volumes:
          - ./syslog-ng/config:/config
          - /logdata:/var/log #optional
    :
    docker compose down
  6. 將原本 log 檔搬移到 /logdata

    cp -a ./log/* /logdata/
    ln -s /logdata .

  7. 重新啟動 syslog-ng

    docker compose up -d

  • tech/logsrv_docker.txt
  • 上一次變更: 2025/05/09 02:21
  • jonathan