Elastic Stack - Storing and Analyzing Logs

The course is designed for anyone who wants to learn how to store large amounts of data using Elastic Stack. We learn how to store, search, and visualize logs using Kibana. Gradually we will go through the whole process from installing individual components (Beats, Logstash, Elasticsearch, Kibana) through their use to cluster management.

On real-life examples, we try different storage architectures that we collect from different sources, enriching them with additional information and storing them into Elasticsearch. The participant gets acquainted with the Elasticsearch repository so that it can efficiently manage and scalable a large amount of data. In Kibana, we learn how to visualize logs, create dashboards, and understand the data more deeply.

Audience:

• Application developers
• System Administrators
• IT Professionals

Prerequisites:

Basic knowledge of Elasticsearch, HTTP protocol, JSON format, general knowledge of database systems.

Course goals:

Participants will learn:

• how to store different logs using Elastic Stack
• how to design logging architecture for different uses
• how to install and configure individual data processing components (Beats, Logstash, Elasticsearch, Kibana)
• Elasticsearch technology more in depth, learn how to use storage tools, how to manage, scale and monitor
• create dashboards and work with the Kibana tool

Course content:

Logging and Elasticsearch

• Event log management
• Visualization of logs
• Examples of use of Elastic Stack in practice

Elasticsearch

• Individual components of Elasticsearch
• Basic work with cluster, nodes, indexes
• Lab

Installation and Configuration

• Install Elasticsearch
• Configuration for logging
• Sample configurations from practice

Cluster, nodes, indexes

• Architecture
• Tools for working with a cluster
• Understanding and Configuring Different Types of Nodes
• Working with indexes, setting indexes for storing logs
• Lab

Data Collection

• Beats
• Collecting application logs from files
• Collect metrics from the server
• Possible architectures for data collection
• Ingest Node
• Lab

Logstash

• Data collection from different sources
• Configuration, deployment examples on real-world applications
• Input, Filter, Output
• Grok filter
• Save to Elasticsearch
• Filebeat
• Lab

Kibana

• Configuration, index patterns
• Discovery of interface
• Aggregation using Kibana
• Creation of visualizations
• Dashboards
• Data search
• Timelion - Time series work
• Sample dashboards, examples from practice
• Lab

Log monitoring

• Track Log Changes
• Detection of anomalies, notification
• Elastic Stack
• ElastAlert
• Lab

Distributed search in logs

• Search in Elasticsearch
• Inverted index, relevance and more
• Data analysis, mapping, dynamic templates
• Lab

Index management

• Capacity planning and configuration
• Managing indexes, compression
• Cache
• Rollover, Shrink API
• Lab

Cluster management

• Restart (rolling, full-cluster)
• Manage snapshots, repositories
• Cluster upgrade (minor, major version)
• Lab

Cluster monitoring

• Elastic cluster monitoring setup
• What ever, how to monitor
• Monitoring tools
• Lab

Andre relevante kurs