Elastic Stack - Storing and Analyzing Logs
The course is designed for anyone who wants to learn how to store large amounts of data using Elastic Stack. We learn how to store, search, and visualize logs using Kibana. Gradually we will go through the whole process from installing individual components (Beats, Logstash, Elasticsearch, Kibana) through their use to cluster management.
On real-life examples, we try different storage architectures that we collect from different sources, enriching them with additional information and storing them into Elasticsearch. The participant gets acquainted with the Elasticsearch repository so that it can efficiently manage and scalable a large amount of data. In Kibana, we learn how to visualize logs, create dashboards, and understand the data more deeply.
Audience:
- Application developers
- System Administrators
- IT Professionals
Prerequisites:
Basic knowledge of Elasticsearch, HTTP protocol, JSON format, general knowledge of database systems.
Course goals:
Participants will learn:
- how to store different logs using Elastic Stack
- how to design logging architecture for different uses
- how to install and configure individual data processing components (Beats, Logstash, Elasticsearch, Kibana)
- Elasticsearch technology more in depth, learn how to use storage tools, how to manage, scale and monitor
- create dashboards and work with the Kibana tool
Course content:
Logging and Elasticsearch
- Event log management
- Visualization of logs
- Examples of use of Elastic Stack in practice
Elasticsearch
- Individual components of Elasticsearch
- Basic work with cluster, nodes, indexes
- Lab
Installation and Configuration
- Install Elasticsearch
- Configuration for logging
- Sample configurations from practice
Cluster, nodes, indexes
- Architecture
- Tools for working with a cluster
- Understanding and Configuring Different Types of Nodes
- Working with indexes, setting indexes for storing logs
- Lab
Data Collection
- Beats
- Collecting application logs from files
- Collect metrics from the server
- Possible architectures for data collection
- Ingest Node
- Lab
Logstash
- Data collection from different sources
- Configuration, deployment examples on real-world applications
- Input, Filter, Output
- Grok filter
- Save to Elasticsearch
- Filebeat
- Lab
Kibana
- Configuration, index patterns
- Discovery of interface
- Aggregation using Kibana
- Creation of visualizations
- Dashboards
- Data search
- Timelion - Time series work
- Sample dashboards, examples from practice
- Lab
Log monitoring
- Track Log Changes
- Detection of anomalies, notification
- Elastic Stack
- ElastAlert
- Lab
Distributed search in logs
- Search in Elasticsearch
- Inverted index, relevance and more
- Data analysis, mapping, dynamic templates
- Lab
Index management
- Capacity planning and configuration
- Managing indexes, compression
- Cache
- Rollover, Shrink API
- Lab
Cluster management
- Restart (rolling, full-cluster)
- Manage snapshots, repositories
- Cluster upgrade (minor, major version)
- Lab
Cluster monitoring
- Elastic cluster monitoring setup
- What ever, how to monitor
- Monitoring tools
- Lab
About the instructor: Petr Novotny
Petr's knowledge goes from solution architecture to development (JavaScript, PHP) through Elasticsearch, Oracle, PL/SQL to agile methodology and SCRUM. At the same time, Petr has been working with Elasticsearch technology for several years and has become one of our main instructors.
