文档库

最新最全的文档下载
当前位置:文档库 > flume-ng 专题分享

flume-ng 专题分享

Flume ng 池伟

目录

1

2

3

4

一句话解释

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

两张图解释

flume-ng 专题分享

1 负责收集数据,送到channel 中 Source

2 是所有数据的中转地

Channel 3 从channel 中拿数据,输出

Sink

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

源码分析

入口

Application

ExecSource

HDFSEventSink

MemoryChannel

nohup sh bin/flume-ng agent -c conf -f conf/client.conf -n a1

-Dflume.monitoring.type=http -Dflume.monitoring.port=34545 &

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

Source Sink Channel Channel Sink Source

源码分析-ExecSource

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

flume-ng 专题分享

源码分析-CmccKafkaSink

flume-ng 专题分享

flume-ng 专题分享

Event分流-数据结构

Header(key-value)

Body(byte[])