Using ntop/ntopng:latest docker image crashes after couple hours #27
Description
Hello, I am using this simple docker compose, which works great initially:
services:
ntopng:
image: ntop/ntopng:latest # You can use a specific version if preferred
container_name: ntopng
network_mode: host # This is crucial to access host network interfaces
cap_add:
- NET_RAW
- NET_ADMIN # Required for capturing packets and setting interfaces up
volumes:
- ./ntopng_data:/var/lib/ntopng
- ./ntopng_conf:/etc/ntopng
- /var/lib/GeoIP:/usr/share/GeoIP:ro
command:
- "-i"
- "enp6s19"
- "-w"
- "3000"
- "-d"
- "/var/lib/ntopng"
- "--local-networks"
- "10.0.0.1/8"
# Add any other ntopng command-line options you need here
- "--community"
- "--dns-mode"
- "1"
- "--geoip-dir"
- "/usr/share/GeoIP"
restart: unless-stopped # Automatically restart the container unless manually stopped
and after a couple of hours of runtime, the UI seems to crash, and any connections to the UI port just time out. However, container stays running(and within container, I see ntop process still running), and I don't see anything that looks abnormal in the container logs. Using ntop/ntopng:stable, this issue does not happen (but stable image is still 6.2, and was looking to use new features of 6.4). Port 3000 just hangs at the web browser (and reverse proxy I use reports gateway timeout, due to port 3000 blocking/timing out). ntopng is configured to use influxdb for timeseries data.
ntopng | Starting redis-server: redis-server.
ntopng | 10/May/2025 12:26:35 [Redis.cpp:167] Successfully connected to redis 127.0.0.1:6379@0
ntopng | 10/May/2025 12:26:35 [Redis.cpp:167] Successfully connected to redis 127.0.0.1:6379@0
ntopng | 10/May/2025 12:26:35 [NetworkInterface.cpp:3921] Cleanup interface enp6s19
ntopng | 10/May/2025 12:26:35 [main.cpp:274] Unable to open interface enp6s19 [97]: PF_RING not loaded. Falling back to pcap.
ntopng | 10/May/2025 12:26:35 [PcapInterface.cpp:127] Reading packets from enp6s19 [ifId: 0]
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:2866] Registered interface 'enp6s19' [id: 0]
ntopng | 10/May/2025 12:26:35 [main.cpp:378] PID stored in file /var/run/ntopng.pid
ntopng | 10/May/2025 12:26:35 [Geolocation.cpp:170] Loaded database GeoLite2-ASN.mmdb [/usr/share/GeoIP/GeoLite2-ASN.mmdb][ip_version: 6]
ntopng | 10/May/2025 12:26:35 [Geolocation.cpp:170] Loaded database GeoLite2-City.mmdb [/usr/share/GeoIP/GeoLite2-City.mmdb][ip_version: 6]
ntopng | 10/May/2025 12:26:35 [Geolocation.cpp:75] Using geolocation provided by MaxMind (https://maxmind.com)
ntopng | 10/May/2025 12:26:35 [Utils.cpp:892] User changed to ntopng
ntopng | 10/May/2025 12:26:35 [HTTPserver.cpp:1665] Found TLS certificate /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem
ntopng | 10/May/2025 12:26:35 [HTTPserver.cpp:1967] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
ntopng | 10/May/2025 12:26:35 [HTTPserver.cpp:1972] HTTP server listening on 3000
ntopng | 10/May/2025 12:26:35 [main.cpp:468] Working directory: /var/lib/ntopng
ntopng | 10/May/2025 12:26:35 [main.cpp:470] Scripts/HTML pages directory: /usr/share/ntopng
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:539] Welcome to ntopng x86_64 v.6.4.250501 (6.4-stable:1b39e1c34c357c253914bc7af97037a0895c09e3:20250501)
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:548] Built on Ubuntu 24.04.1 LTS
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:550] (C) 1998-25 ntop
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1016] Adding <sanitized as pub ipv6> as IPv6 interface address for enp6s19
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1028] Adding <sanitized as pub ipv6> as IPv6 local network for enp6s19
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1016] Adding <sanitized as pub ipv6> as IPv6 interface address for enp6s19
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1028] Adding <sanitized as pub ipv6> as IPv6 local network for enp6s19
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1016] Adding fe80::be24:11ff:fe24:512b/128 as IPv6 interface address for enp6s19
ntopng | 10/May/2025 12:26:35 [NetworkInterface.cpp:3651] Started flow user script hooks loop on interface 'enp6s19' [id: 0]...
ntopng | 10/May/2025 12:26:35 [Ntop.cpp:1028] Adding fe80::be24:11ff:fe24:512b/64 as IPv6 local network for enp6s19
ntopng | 10/May/2025 12:26:35 [NetworkInterface.cpp:3710] Started host user script hooks loop on interface 'enp6s19' [id: 0]...
ntopng | 10/May/2025 12:26:39 [PeriodicActivities.cpp:123] Started periodic activities loop...
ntopng | 10/May/2025 12:26:39 [startup.lua:40] Processing startup.lua: please hold on...
ntopng | 10/May/2025 12:26:39 [startup.lua:207] [lists_utils.lua:803] Refreshing category lists...
ntopng | 10/May/2025 12:26:39 [startup.lua:207] [lists_utils.lua:444] Updating list 'Abuse.ch URLhaus' [https://urlhaus.abuse.ch/downloads/hostfile/]... OK
ntopng | 10/May/2025 12:26:39 [startup.lua:207] [lists_utils.lua:444] Updating list 'Emerging Threats' [https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt]... OK
ntopng | 10/May/2025 12:26:40 [startup.lua:207] [lists_utils.lua:444] Updating list 'IPsum Threat Intelligence Feed' [https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt]... OK
ntopng | 10/May/2025 12:26:40 [startup.lua:207] [lists_utils.lua:444] Updating list 'NoCoin Filter List' [https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt]... OK
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:444] Updating list 'Stratosphere Lab' [https://mcfp.felk.cvut.cz/publicDatasets/CTU-AIPP-BlackList/Todays-Blacklists/AIP_historical_blacklist_prioritized_by_newest_attackers.csv]... OK
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:444] Updating list 'ThreatFox' [https://threatfox.abuse.ch/downloads/hostfile/]... OK
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:444] Updating list 'dshield 7 days' [https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/dshield_7d.netset]... OK
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded Abuse.ch URLhaus: 584 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded Emerging Threats: 1464 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded IPsum Threat Intelligence Feed: 22177 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded NoCoin Filter List: 313 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded Stratosphere Lab: 10959 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded ThreatFox: 64644 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:596] Loaded dshield 7 days: 37 rules
ntopng | 10/May/2025 12:26:41 [startup.lua:207] [lists_utils.lua:702] Loaded Category Lists (65541 hosts, 34637 IPs) loaded in 0 sec
ntopng | 10/May/2025 12:26:41 [startup.lua:211] Initializing device polices...
ntopng | 10/May/2025 12:26:41 [startup.lua:227] Initializing alerts...
ntopng | 10/May/2025 12:26:41 [startup.lua:237] Initializing timeseries...
ntopng | 10/May/2025 12:26:41 [startup.lua:346] [blog_utils.lua:146] Fetching latest ntop blog posts...
ntopng | 10/May/2025 12:26:42 [startup.lua:385] Completed startup.lua
ntopng | 10/May/2025 12:26:42 [NetworkInterface.cpp:3884] Started packet polling on interface 'enp6s19' [id: 0]...
ntopng | 10/May/2025 12:26:43 [NetworkInterface.cpp:2576] Packets exceeding the expected max size have been received [enp6s19][len: 1539][max len: 1518].
ntopng | 10/May/2025 12:26:43 [NetworkInterface.cpp:2582] WARNING: If TSO/GRO is enabled, please disable it for best accuracy
ntopng | 10/May/2025 12:26:43 [NetworkInterface.cpp:2586] WARNING: using: sudo ethtool -K enp6s19 gro off gso off tso off
ntopng | 10/May/2025 12:34:14 [PartializableFlowTrafficStats.cpp:80] WARNING: Flow stats went backwards [c2s 602 -> 66][s2c 0 -> 602]
Inside the container:
root@ntop:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 4324 3192 ? Ss 12:26 0:00 /bin/bash /run.sh -i enp6s19 -w 3000 -d /var/lib/ntopng --local-networks 10.0.0.1/8 --community --dns-mode 1 --geoip-dir /usr/share/GeoIP
redis 15 0.1 0.3 64876 12088 ? Ssl 12:26 0:29 /usr/bin/redis-server 127.0.0.1:6379
ntopng 16 0.9 9.7 809948 390816 ? Sl 12:26 2:43 ntopng -i enp6s19 -w 3000 -d /var/lib/ntopng --local-networks 10.0.0.1/8 --community --dns-mode 1 --geoip-dir /usr/share/GeoIP
root 2128 0.0 0.0 4588 3840 pts/0 Ss 17:02 0:00 /bin/bash
root 2136 0.0 0.0 7892 3920 pts/0 R+ 17:02 0:00 ps aux
I regularly see this in in redis logs on the container(prepackaged redis version in ntop container):
15:M 10 May 2025 14:09:33.179 * Background saving started by pid 2108
2108:C 10 May 2025 14:09:33.198 * DB saved on disk
2108:C 10 May 2025 14:09:33.199 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:09:33.279 * Background saving terminated with success
15:M 10 May 2025 14:17:10.441 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 14:17:10.441 * Background saving started by pid 2109
2109:C 10 May 2025 14:17:10.531 * DB saved on disk
2109:C 10 May 2025 14:17:10.532 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:17:10.542 * Background saving terminated with success
15:M 10 May 2025 14:24:58.853 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 14:24:58.853 * Background saving started by pid 2110
2110:C 10 May 2025 14:24:58.872 * DB saved on disk
2110:C 10 May 2025 14:24:58.874 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:24:58.954 * Background saving terminated with success
15:M 10 May 2025 14:32:29.786 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 14:32:29.786 * Background saving started by pid 2111
2111:C 10 May 2025 14:32:29.806 * DB saved on disk
2111:C 10 May 2025 14:32:29.806 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:32:29.887 * Background saving terminated with success
15:M 10 May 2025 14:43:26.205 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 14:43:26.206 * Background saving started by pid 2112
2112:C 10 May 2025 14:43:26.225 * DB saved on disk
2112:C 10 May 2025 14:43:26.226 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:43:26.306 * Background saving terminated with success
15:M 10 May 2025 14:51:25.509 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 14:51:25.509 * Background saving started by pid 2113
2113:C 10 May 2025 14:51:25.641 * DB saved on disk
2113:C 10 May 2025 14:51:25.642 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 14:51:25.710 * Background saving terminated with success
15:M 10 May 2025 15:01:05.424 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:01:05.425 * Background saving started by pid 2114
2114:C 10 May 2025 15:01:05.445 * DB saved on disk
2114:C 10 May 2025 15:01:05.445 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:01:05.526 * Background saving terminated with success
15:M 10 May 2025 15:10:48.649 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:10:48.650 * Background saving started by pid 2115
2115:C 10 May 2025 15:10:48.668 * DB saved on disk
2115:C 10 May 2025 15:10:48.669 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:10:48.750 * Background saving terminated with success
15:M 10 May 2025 15:20:35.670 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:20:35.670 * Background saving started by pid 2116
2116:C 10 May 2025 15:20:35.762 * DB saved on disk
2116:C 10 May 2025 15:20:35.762 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:20:35.771 * Background saving terminated with success
15:M 10 May 2025 15:27:28.110 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:27:28.110 * Background saving started by pid 2117
2117:C 10 May 2025 15:27:28.130 * DB saved on disk
2117:C 10 May 2025 15:27:28.131 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:27:28.211 * Background saving terminated with success
15:M 10 May 2025 15:37:25.668 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:37:25.669 * Background saving started by pid 2118
2118:C 10 May 2025 15:37:25.687 * DB saved on disk
2118:C 10 May 2025 15:37:25.688 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:37:25.770 * Background saving terminated with success
15:M 10 May 2025 15:45:15.784 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:45:15.785 * Background saving started by pid 2119
2119:C 10 May 2025 15:45:15.859 * DB saved on disk
2119:C 10 May 2025 15:45:15.860 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:45:15.886 * Background saving terminated with success
15:M 10 May 2025 15:54:35.732 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 15:54:35.733 * Background saving started by pid 2120
2120:C 10 May 2025 15:54:35.750 * DB saved on disk
2120:C 10 May 2025 15:54:35.751 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 15:54:35.833 * Background saving terminated with success
15:M 10 May 2025 16:05:30.814 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:05:30.814 * Background saving started by pid 2121
2121:C 10 May 2025 16:05:30.834 * DB saved on disk
2121:C 10 May 2025 16:05:30.835 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:05:30.915 * Background saving terminated with success
15:M 10 May 2025 16:15:09.260 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:15:09.261 * Background saving started by pid 2122
2122:C 10 May 2025 16:15:09.281 * DB saved on disk
2122:C 10 May 2025 16:15:09.282 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:15:09.361 * Background saving terminated with success
15:M 10 May 2025 16:24:30.052 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:24:30.053 * Background saving started by pid 2123
2123:C 10 May 2025 16:24:30.146 * DB saved on disk
2123:C 10 May 2025 16:24:30.147 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:24:30.153 * Background saving terminated with success
15:M 10 May 2025 16:32:09.675 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:32:09.676 * Background saving started by pid 2124
2124:C 10 May 2025 16:32:09.766 * DB saved on disk
2124:C 10 May 2025 16:32:09.767 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:32:09.777 * Background saving terminated with success
15:M 10 May 2025 16:39:29.092 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:39:29.093 * Background saving started by pid 2125
2125:C 10 May 2025 16:39:29.111 * DB saved on disk
2125:C 10 May 2025 16:39:29.112 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:39:29.194 * Background saving terminated with success
15:M 10 May 2025 16:49:26.039 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:49:26.039 * Background saving started by pid 2126
2126:C 10 May 2025 16:49:26.130 * DB saved on disk
2126:C 10 May 2025 16:49:26.131 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:49:26.140 * Background saving terminated with success
15:M 10 May 2025 16:59:34.873 * 100 changes in 300 seconds. Saving...
15:M 10 May 2025 16:59:34.873 * Background saving started by pid 2127
2127:C 10 May 2025 16:59:34.960 * DB saved on disk
2127:C 10 May 2025 16:59:34.961 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15:M 10 May 2025 16:59:34.974 * Background saving terminated with success
Any tips? I can provide any additional information as needed.