Cargando…
Learning Elastic Stack 6.0 : a beginner's guide to distributed search, analytics, and visualization using Elasticsearch, Logstash and Kibana.
This book will give you a fundamental understanding of what the stack is all about, and how to use it efficiently to build powerful real-time data processing applications. It provide in-depth coverage of the different components of the Elastic Stack, and how to use them all together.
| Clasificación: | Libro Electrónico |
|---|---|
| Autor principal: | |
| Otros Autores: | , , |
| Formato: | Electrónico eBook |
| Idioma: | Inglés |
| Publicado: |
Birmingham :
Packt Publishing,
2017.
|
| Temas: | |
| Acceso en línea: | Texto completo |
Tabla de Contenidos:
- Cover
- Copyright
- Credits
- Disclaimer
- About the Authors
- About the Reviewer
- www.PacktPub.com
- Customer Feedback
- Table of Contents
- Preface
- Chapter 1: Introducing Elastic Stack
- What is Elasticsearch, and why use it?
- Schemaless and document-oriented
- Searching
- Analytics
- Rich client library support and the REST API
- Easy to operate and easy to scale
- Near real time
- Lightning fast
- Fault tolerant
- Exploring the components of Elastic Stack
- Elasticsearch
- Logstash
- Beats
- Kibana
- X-Pack
- Security
- Monitoring
- Reporting
- Alerting
- Graph
- Elastic Cloud
- Use cases of Elastic Stack
- Log and security analytics
- Product search
- Metrics analytics
- Web search and website search
- Downloading and installing
- Installing Elasticsearch
- Installing Kibana
- Summary
- Chapter 2: Getting Started with Elasticsearch
- Using the Kibana Console UI
- Core concepts
- Index
- Type
- Document
- Node
- Cluster
- Shards and replicas
- Mappings and data types
- Data types
- Core datatypes
- Complex datatypes
- Other datatypes
- Mappings
- Creating an index with the name catalog
- Defining the mappings for the type of product
- Inverted index
- CRUD operations
- Index API
- Indexing a document by providing an ID
- Indexing a document without providing an ID
- Get API
- Update API
- Delete API
- Creating indexes and taking control of mapping
- Creating an index
- Creating type mapping in an existing index
- Updating a mapping
- REST API overview
- Common API conventions
- Formatting the JSON response
- Dealing with multiple indices
- Searching all documents in one index
- Searching all documents in multiple indexes
- Searching all documents of a particular type in all indices
- Summary
- Chapter 3: Searching-What is Relevant
- Basics of text analysis.
- Understanding Elasticsearch analyzers
- Character filters
- Tokenizer
- Standard Tokenizer
- Token filters
- Using built-in analyzers
- Standard Analyzer
- Implementing autocomplete with a custom analyzer
- Searching from structured data
- Range query
- Range query on numeric types
- Range query with score boosting
- Range query on dates
- Exists query
- Term query
- Searching from full text
- Match query
- Operator
- minimum_should_match
- Fuzziness
- Match phrase query
- Multi match query
- Querying multiple fields with defaults
- Boosting one or more fields
- With types of multi match queries
- Writing compound queries
- Constant score query
- Bool query
- Combining OR conditions
- Combining conditions AND and OR conditions
- Adding NOT conditions
- Summary
- Chapter 4: Analytics with Elasticsearch
- The basics of aggregations
- Bucket aggregations
- Metric aggregations
- Matrix aggregations
- Pipeline aggregations
- Preparing data for analysis
- Understanding the structure of data
- Loading the data using Logstash
- Metric aggregations
- Sum, average, min, and max aggregations
- Sum aggregation
- Average aggregation
- Min aggregation
- Max aggregation
- Stats and extended stats aggregations
- Stats aggregation
- Extended stats Aggregation
- Cardinality aggregation
- Bucket aggregations
- Bucketing on string data
- Terms aggregation
- Bucketing on numeric data
- Histogram aggregation
- Range aggregation
- Aggregations on filtered data
- Nesting aggregations
- Bucketing on custom conditions
- Filter aggregation
- Filters aggregation
- Bucketing on date/time data
- Date Histogram aggregation
- Creating buckets across time
- Using a different time zone
- Computing other metrics within sliced time intervals
- Focusing on a specific day and changing intervals.
- Bucketing on geo-spatial data
- Geo distance aggregation
- GeoHash grid aggregation
- Pipeline aggregations
- Calculating the cumulative sum of usage over time
- Summary
- Chapter 5: Analyzing Log Data
- Log analysis challenges
- Logstash
- Installation and configuration
- Prerequisites
- Downloading and installing Logstash
- Installing on Windows
- Installing on Linux
- Running Logstash
- Logstash architecture
- Overview of Logstash plugins
- Installing or updating plugins
- Input plugins
- Output plugins
- Filter plugins
- Codec plugins
- Exploring plugins
- Exploring Input plugins
- File
- Beats
- JDBC
- IMAP
- Output plugins
- Elasticsearch
- CSV
- Kafka
- PagerDuty
- Codec plugins
- JSON
- Rubydebug
- Multiline
- Filter plugins
- Ingest node
- Defining a pipeline
- Ingest APIs
- Put pipeline API
- Get Pipeline API
- Delete pipeline API
- Simulate pipeline API
- Summary
- Chapter 6: Building Data Pipelines with Logstash
- Parsing and enriching logs using Logstash
- Filter plugins
- CSV filter
- Mutate filter
- Grok filter
- Date filter
- Geoip filter
- Useragent filter
- Introducing Beats
- Beats by Elastic.co
- Filebeat
- Metricbeat
- Packetbeat
- Heartbeat
- Winlogbeat
- Auditbeat
- Community Beats
- Logstash versus Beats
- Filebeat
- Downloading and installing Filebeat
- Installing on Windows
- Installing on Linux
- Architecture
- Configuring Filebeat
- Filebeat prospectors
- Filebeat global options
- Filebeat general options
- Output configuration
- Filebeat modules
- Summary
- Chapter 7: Visualizing data with Kibana
- Downloading and installing Kibana
- Installing on Windows
- Installing on Linux
- Configuring Kibana
- Data preparation
- Kibana UI
- User interaction
- Configuring the index pattern
- Discover
- Elasticsearch query string.
- Elasticsearch DSL query
- Visualize
- Kibana aggregations
- Bucket aggregations
- Metric
- Creating a visualization
- Visualization types
- Line, area, and bar charts
- Data table
- MarkDown widget
- Metric
- Goal
- Gauge
- Pie charts
- Co-ordinate maps
- Region maps
- Tag cloud
- Visualizations in action
- Response codes over time
- Top 10 URLs requested
- Bandwidth usage of top five countries over time
- Web traffic originating from different countries
- Most used user agent
- Dashboards
- Creating a dashboard
- Saving the dashboard
- Cloning the dashboard
- Sharing the dashboard
- Timelion
- Timelion UI
- Timelion expressions
- Using plugins
- Installing plugins
- Removing plugins
- Summary
- Chapter 8: Elastic X-Pack
- Installing X-Pack
- Installing X-Pack on Elasticsearch
- Installing X-Pack on Kibana
- Uninstalling X-Pack
- Configuring X-Pack
- Security
- User authentication
- User authorization
- Security in action
- New user creation
- Deleting a user
- Changing the password
- New role creation
- How to Delete/Edit a role
- Document-level security or field-level security
- X-Pack security APIs
- User management APIs
- Role management APIs
- Monitoring Elasticsearch
- Monitoring UI
- Elasticsearch metrics
- Overview tab
- Nodes tab
- The Indices tab
- Alerting
- Anatomy of a watch
- Alerting in action
- Create a new alert
- Threshold Alert
- Advanced Watch
- How to Delete/Deactivate/Edit a Watch
- Summary
- Chapter 9: Running Elastic Stack in Production
- Hosting Elastic Stack on a managed cloud
- Getting up and running on Elastic Cloud
- Using Kibana
- Overriding configuration
- Recovering from a snapshot
- Hosting Elastic Stack on your own
- Selecting hardware
- Selecting an operating system
- Configuring Elasticsearch nodes
- JVM heap size
- Disable swapping.
- File descriptors
- Thread pools and garbage collector
- Managing and monitoring Elasticsearch
- Running in Docker containers
- Special considerations while deploying to a cloud
- Choosing instance type
- Changing default ports
- do not expose ports!
- Proxy requests
- Binding HTTP to local addresses
- Installing EC2 discovery plugin
- Installing S3 repository plugin
- Setting up periodic snapshots
- Backing up and restoring
- Setting up a repository for snapshots
- Shared filesystem
- Cloud or distributed filesystems
- Taking snapshots
- Restoring a specific snapshot
- Setting up index aliases
- Understanding index aliases
- How index aliases can help
- Setting up index templates
- Defining an index template
- Creating indexes on the fly
- Modeling time series data
- Scaling the index with unpredictable volume over time
- Unit of parallelism in Elasticsearch
- The effect of the number of shards on the relevance score
- The effect of the number of shards on the accuracy of aggregations
- Changing the mapping over time
- New fields get added
- Existing fields get removed
- Automatically deleting older documents
- How index-per-timeframe solves these issues
- Scaling with index-per-timeframe
- Changing the mapping over time
- Automatically deleting older documents
- Summary
- Chapter 10: Building a Sensor Data Analytics Application
- Introduction to the application
- Understanding the sensor-generated data
- Understanding the sensor metadata
- Understanding the final stored data
- Modeling data in Elasticsearch
- Defining an index template
- Understanding the mapping
- Setting up the metadata database
- Building the Logstash data pipeline
- Accept JSON requests over the web
- Enrich the JSON with the metadata we have in the MySQL database
- The jdbc_streaming plugin
- The mutate plugin.