docker应用无法发送日志到kafka-Awesome Devin

背景

笔者有两个环境，其中一个是k8s,一个是docker
目前是使用docker-compose启动的efk
已知所有内网宿主机互通，端口互通

问题

k8s内的容器是可以连接到kafka的;docker新启动的应用显示无法连接到kafka:9092
java相关报错如下：

...
NetworkClient.java.950 [Producer clientId=producer-7] Error connecting to node kafka:9092 (id: 1001 rack: null)
java.net.UnknownHostException: kafka
...

查找原因

从以下情况中查找：
1.镜像中提前添加了kafka的hosts
2.其他地方有类似hosts的配置，指向了kafka对应的内网ip

详细分析

第一种情况

此种情况不存在。因为使用了官方镜像，默认应该是没有添加内容的。

FROM register.hrttest.cn/tools/adoptopenjdk/openjdk11:x86_64-alpine-jre-11.0.20.1_1
#定义时区参数
ENV TZ=Asia/Shanghai

#设置时区
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

VOLUME /tmp
ADD ./target/test.jar /test.jar
ADD ./config /config

ENTRYPOINT ["java","-Xms512m","-Xmx512m","-XX:+UseG1GC","-Dspring.profiles.active=test","-Dlogging.file=../logs/test","-Dlogging.kafkaBootstrapServers=kafka:9092","-Dfile.encoding=utf-8","-jar","/test.jar","--spring.cloud.bootstrap.location=/config/bootstrap.yml","--spring.config.location=/config/application.yml"]

第二种情况

查看kafka部分相关启动配置

  kafka:
    image: registry.cn-hangzhou.aliyuncs.com/zhengqing/kafka:latest
    #image: register.hrttest.cn/tools/apache/kafka:4.0.0
    container_name: kafka
    restart: unless-stopped
    env_file:
      - ./.env
    volumes:
      - "/etc/localtime:/etc/localtime"
      #- "./app/kafka:/bitnami/kafka:rw"
    environment:
      KAFKA_HEAP_OPTS: "-Xmx1024m -Xms1024m"
      ALLOW_PLAINTEXT_LISTENER: yes
      #KAFKA_CFG_ZOOKEEPER_CONNECT: zookepper:2181
      KAFKA_ZOOKEEPER_CONNECT: zookepper:2181
      #KAFKA_`CFG_ADVERTISED_LISTENERS: PLAINTEXT://${KAFKA_EXTERNAL_HOST}:9091
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://${KAFKA_EXTERNAL_HOST}:9092
      #日志清理策略（不保留日志）
      KAFKA_CFG_LOG_CLEANUP_POLICY: "delete"
      KAFKA_CFG_LOG_RETENTION_HOURS: "0"          # 时间优先级关闭
      KAFKA_CFG_LOG_RETENTION_BYTES: "10485760"   # 10MB
      KAFKA_CFG_LOG_SEGMENT_BYTES: "5242880"      # 5MB per segment
      KAFKA_CFG_LOG_RETENTION_CHECK_INTERVAL_MS: "60000"  # 每分钟检查一次
    ports:
      - "9092:9092"
    depends_on:
      - zookepper
    links:
      - zookepper
    networks:
      efkk:
        ipv4_address: 172.22.6.12

查看环境配置文件.env

    #cat .env 
    #Fill in the domain name or host IP -- so that the client can listen to the message (host.docker.internal: automatically identify the host IP, it is valid to run Docker on Windows or Mac)
    KAFKA_EXTERNAL_HOST=kafka

从以上可以看出，这个kafka的默认名称是在这里配置的

进入k8s容器查看

#[root@node6 ~]# crictl exec -it e75 /bin/bash
#[root@hrt-test-75d49499f6-f5pgh jdk]# ping kafka
PING kafka (192.168.0.124) 56(84) bytes of data.
64 bytes from kafka (192.168.0.124): icmp_seq=1 ttl=63 time=0.614 ms
64 bytes from kafka (192.168.0.124): icmp_seq=2 ttl=63 time=0.538 ms
^C
--- kafka ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.538/0.576/0.614/0.038 ms

[root@hrt-test-75d49499f6-f5pgh jdk]# cat  /etc/hosts 
# Kubernetes-managed hosts file.
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
fe00::0	ip6-mcastprefix
fe00::1	ip6-allnodes
fe00::2	ip6-allrouters
10.244.15.87	hrt-test-75d49499f6-f5pgh

# Entries added by HostAliases.
192.168.0.124	kafka

这里发现是有配置kafka相关的hosts

找到运行正常的k8s的deployment配置

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  labels:
    app: test
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      imagePullSecrets:
      - name: harbor-secret
      hostAliases:
        - hostnames:
            - kafka
          ip: 192.168.0.124
...

从这里可以看到，hosts被指定成了内网。

      hostAliases:
        - hostnames:
            - kafka
          ip: 192.168.0.124

看到这里，相信大家已经有了答案。docker中没有配置hosts

解决

在docker启动的时候加入一个参数 --add-host=kafka:192.168.0.124 即可。

# cat docker.sh
#!/bin/bash
docker ps -a|grep test
if [ $? = 0 ];then
docker stop test
docker rm test
fi
docker images -a |grep test
if [ $? = 0 ];then
docker rmi test
fi
cd /home/app/test
docker build -t test:latest .
docker run --add-host=kafka:192.168.0.124  -d -p 8081:8081 --name  test test

至此问题完美解决，从kibana那边已经可以查看到相应的日志。

关注我们，获取更多DevOps和安全更新资讯！
本文作者：运维技术团队：辣个男人Devin
发布日期：2026年01月06日
适用系统：Linux