Building Data Streaming Applications with Apache Kafka.

Mostrar otras versiones (1)

Design and administer fast, reliable enterprise messaging systems with Apache Kafka About This Book Build efficient real-time streaming applications in Apache Kafka to process data streams of data Master the core Kafka APIs to set up Apache Kafka clusters and start writing message producers and cons...

Descripción completa

Detalles Bibliográficos
Clasificación:Libro Electrónico
Autor principal: Kumar, Manish
Otros Autores: Singh, Chanchal
Formato: Electrónico eBook
Idioma:Inglés
Publicado: Birmingham : Packt Publishing, 2017.
Temas:
Apache Kafka.
Real-time data processing.
Application software > Development.
Temps réel (Informatique)
Logiciels d'application > Développement.
COMPUTERS > General.
Application software > Development
Real-time data processing
Acceso en línea:Texto completo
Tabla de Contenidos:
  • Cover; Title Page; Copyright; Credits; About the Authors; About the Reviewer; www.PacktPub.com; Customer Feedback; Table of Contents; Preface; Chapter 1: Introduction to Messaging Systems; Understanding the principles of messaging systems; Understanding messaging systems; Peeking into a point-to-point messaging system; Publish-subscribe messaging system; Advance Queuing Messaging Protocol; Using messaging systems in big data streaming applications; Summary; Chapter 2: Introducing Kafka the Distributed Messaging Platform; Kafka origins; Kafka's architecture; Message topics; Message partitions.
  • Replication and replicated logsMessage producers; Message consumers; Role of Zookeeper; Summary; Chapter 3: Deep Dive into Kafka Producers; Kafka producer internals; Kafka Producer APIs; Producer object and ProducerRecord object; Custom partition; Additional producer configuration; Java Kafka producer example; Common messaging publishing patterns; Best practices; Summary; Chapter 4: Deep Dive into Kafka Consumers; Kafka consumer internals; Understanding the responsibilities of Kafka consumers; Kafka consumer APIs; Consumer configuration; Subscription and polling; Committing and polling.
  • Additional configurationJava Kafka consumer; Scala Kafka consumer; Rebalance listeners; Common message consuming patterns; Best practices; Summary; Chapter 5: Building Spark Streaming Applications with Kafka; Introduction to Spark ; Spark architecture; Pillars of Spark; The Spark ecosystem; Spark Streaming ; Receiver-based integration; Disadvantages of receiver-based approach; Java example for receiver-based integration; Scala example for receiver-based integration; Direct approach; Java example for direct approach; Scala example for direct approach.
  • Use case log processing
  • fraud IP detectionMaven; Producer ; Property reader; Producer code ; Fraud IP lookup; Expose hive table; Streaming code; Summary; Chapter 6: Building Storm Applications with Kafka; Introduction to Apache Storm; Storm cluster architecture; The concept of a Storm application; Introduction to Apache Heron; Heron architecture ; Heron topology architecture; Integrating Apache Kafka with Apache Storm
  • Java; Example; Integrating Apache Kafka with Apache Storm
  • Scala; Use case
  • log processing in Storm, Kafka, Hive; Producer; Producer code ; Fraud IP lookup.
  • Running the projectSummary; Chapter 7: Using Kafka with Confluent Platform; Introduction to Confluent Platform; Deep driving into Confluent architecture; Understanding Kafka Connect and Kafka Stream; Kafka Streams; Playing with Avro using Schema Registry; Moving Kafka data to HDFS; Camus ; Running Camus; Gobblin; Gobblin architecture; Kafka Connect; Flume; Summary; Chapter 8: Building ETL Pipelines Using Kafka; Considerations for using Kafka in ETL pipelines; Introducing Kafka Connect; Deep dive into Kafka Connect; Introductory examples of using Kafka Connect; Kafka Connect common use cases.

Ejemplares similares