首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >如何在ECK上为kafka输入配置Filebeat?

如何在ECK上为kafka输入配置Filebeat?
EN

Stack Overflow用户
提问于 2021-02-04 18:32:23
回答 1查看 429关注 0票数 0

我在Kubernetes上运行了Elasticsearch和Kibana。两者都是由ECK创建的。现在,我尝试将Filebeat添加到其中,并将其配置为索引来自Kafka主题的数据。这是我当前的配置:

代码语言:javascript
复制
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
   name: my-filebeat
   namespace: my-namespace
spec:
   type: filebeat
   version: 7.10.2
   elasticsearchRef:
       name: my-elastic
   kibanaRef:
       name: my-kibana
   config:
       filebeat.inputs:
       - type: kafka
         hosts:
         - host1:9092
         - host2:9092
         - host3:9092
         topics: ["my.topic"]
         group_id: "my_group_id"
         index: "my_index"
   deployment:
       podTemplate:
           spec:
               dnsPolicy: ClusterFirstWithHostNet
               hostNetwork: true
               securityContext:
                   runAsUser: 0
               containers:
               - name: filebeat

在pod的日志中,我可以看到如下条目

代码语言:javascript
复制
log/log.go:145  Non-zero metrics in the last 30s    {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2470,"time":{"ms":192}},"total":{"ticks":7760,"time":{"ms":367},"value":7760},"user":{"ticks":5290,"time":{"ms":175}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":13},"info":{"ephemeral_id":"5ce8521c-f237-4994-a02e-dd11dfd31b09","uptime":{"ms":181997}},"memstats":{"gc_next":23678528,"memory_alloc":15320760,"memory_total":459895768},"runtime":{"goroutines":106}},"filebeat":{"harvester":{"open_files":0,"running":0},"inputs":{"kafka":{"bytes_read":46510,"bytes_write":37226}}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":1.18,"15":0.77,"5":0.97,"norm":{"1":0.0738,"15":0.0481,"5":0.0606}}}}}}

也不存在错误条目。所以我假设与Kafka的连接是有效的。不幸的是,在上面指定的my_index中没有数据。我做错了什么?

EN

回答 1

Stack Overflow用户

发布于 2021-02-04 18:59:11

我猜您无法连接到输出中提到的Elasticsearch。

根据文档,ECK保护部署的Elasticsearch,并将其存储在Kubernetes Secrets中。

https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration.html

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/66043825

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档