Apache Flink - Fast and reliable large-scale data processing. These are open source projects that aim to bring about innovative changes in the IT world. Apache Flume - A service for collecting, aggregating, and moving large amounts of log data. Flume 1.10.0 User Guide (also in pdf) Flume 1.10.0 Developer Guide (also in pdf) Flume 1.10.0 API Documentation. Apache Flume 1.10.0 is production-ready software. Apache Flume is an open-source, powerful, reliable and flexible system used to collect, aggregate and move large amounts of unstructured data from multiple data sources into HDFS/Hbase (for example) in a distributed fashion via it’s strong coupling with the Hadoop cluster. In fact, it was listed as a Top Level Apache Project in 2012. Apache Flume 1.10.0 is the twelfth release of Flume as an Apache top-level project (TLP). This has enabled the tool to offer users more advanced functionalities and simpler operation. Apache Flume is a tool/service/data ingestion mechanism for collecting aggregating and transporting large amounts of streaming data such as log files, events (etc. Thanks to its ability to handle large volumes of data, Apache Flume is particularly well suited to peak workloads.Īfter several redesigns due to an overly complex architecture and indebted code, Flume OG (old generation) has become Flume NG (new generation). All resources can thus be shared much more easily. It collects log data from the web server logs. And they can do so from any server on the network. Apache Flume is an Apache open source project used for moving massive quantities of streaming data into HDFS. The idea is to enable users to access files on the shared storage system. In other words, a distributed file system that manages large data sets. A Flume event is defined as a unit of data flow having a byte payload and an optional set of string attributes. To this end, Apache Flume is written in an HDFS. apache-flume-1.6.0-bin.tar FlumeHadoopFlume. This solution has been specially designed to handle high volumes and throughputs. cd Downloads/ tar zxvf apache-flume-1.6. tar zxvf apache-flume-1.6. 5. This is a tool for collecting, aggregating and moving large quantities of logs. The Apache Flume team is pleased to announce the release of Flume 1.5.0. As the amount of data collected by logs increases, new tools are emerging to facilitate their exploitation.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |