diff --git a/radar/2017-03-01/elasticsearch.md b/radar/2017-03-01/elasticsearch.md index 1e3da23..67a01a1 100644 --- a/radar/2017-03-01/elasticsearch.md +++ b/radar/2017-03-01/elasticsearch.md @@ -2,13 +2,9 @@ title: "Elasticsearch" ring: trial quadrant: platforms-and-aoe-services +tags: [devops] --- -Elasticsearch is a REST-based search and analytics engine based on Lucene. -Unlike its competitor Apache Solr, it was developed in the beginning with clustering and scaling in mind. -It allows you to create complex queries while still delivering results very fast. +Elasticsearch is a REST-based search and analytics engine based on [Lucene](https://lucene.apache.org/). Unlike its competitor [Apache Solr](https://solr.apache.org/), it was developed from the beginning with clustering and scaling in mind. It allows you to create complex queries while still delivering results very quickly. -At AOE, we use Elasticsearch for logging as well as our own search solution [Searchperience®](http://www.searchperience.com/). -We recently moved the Searchperience stack from Solr to Elasticsearch and think this was the right decision. -Especially in terms of scaling, ease of use and performance, Elasticsearch really shines. -Also, the API design took some learnings from Apache SOLR into account - for example, the queryDSL is a powerful way of describing different search use cases with highly flexible support of aggregations, etc. \ No newline at end of file +At AOE, we use Elasticsearch for logging as well as our own search solution, [Searchperience®](http://www.searchperience.com/). We recently moved the Searchperience stack from Solr to Elasticsearch and believe this was the right decision. Especially in terms of scaling, ease of use, and performance, Elasticsearch really shines. Additionally, the API design incorporates learnings from Apache Solr - for example, the queryDSL is a powerful way of describing different search use cases with highly flexible support for aggregations, etc. diff --git a/radar/2017-03-01/elk-stack.md b/radar/2017-03-01/elk-stack.md index 07a4b87..114dd21 100644 --- a/radar/2017-03-01/elk-stack.md +++ b/radar/2017-03-01/elk-stack.md @@ -2,20 +2,18 @@ title: "ELK Stack" ring: adopt quadrant: platforms-and-aoe-services +tags: [devops] --- -The company behind Elasticsearch offers a very nice solution for logging and analysis of distributed data such as logfiles. +The [ELK Stack](https://www.elastic.co/de/elastic-stack) is a powerful trio of tools that work together to provide a comprehensive solution for logging, searching, visualizing, and analyzing data in real time: -In today's increasingly distributed IT systems, it's very helpful to have a central view of what is going on in your systems - and of course nobody can and wants to look in different logfiles on different servers. -A central logging solution provides the option to detect potential relationships between different events more easily. -Also, also it can be used to extract useful KPIs or to visualize information on dashboards. +- **[Elasticsearch](../platforms-and-aoe-services/elasticsearch.html)**: A search and analytics engine that indexes and stores data. +- **Logstash**: A data processing pipeline that ingests and transforms various data formats. +- **Kibana**: A visualization tool that creates interactive dashboards and reports. -The abbreviation "[ELK](https://www.elastic.co/products) Stack" stands for the Tools Elasticsearch, Logstash and Kibana: -Together, they provide a solution for collecting data the ability to search, visualize and analyze data in real time. +In today's complex IT environments, having a centralized logging solution is essential. It allows you to: +- Gain a unified view of system activities. +- Detect correlations between different events more easily. +- Extract valuable KPIs and visualize them on customizable dashboards. -Logstash is used to process and forward different data (or logfile) formats. -Elasticsearch is used as a search index and together with the Kibana plugin you can configure highly individual dashboards. -Recently, there are also the Beats Tools joining this toolstack to ship data to Elasticsearch. - -We have been using the ELK Stack for several years now in several projects and different infrastructure setups - we use it to visualize traffic, certain KPIs or just to analyze and search in application logs. -We encourage all teams to use such a solution and take care to write useful logs in your applications. +At AOE, we have been using the ELK Stack for years in various projects and infrastructure setups. It has proven invaluable for visualizing traffic, tracking KPIs, and analyzing application logs. We highly recommend that all teams adopt this solution and focus on generating meaningful logs in their applications. diff --git a/radar/2018-03-01/elasticsearch.md b/radar/2018-03-01/elasticsearch.md index 33e4b03..bb992e9 100644 --- a/radar/2018-03-01/elasticsearch.md +++ b/radar/2018-03-01/elasticsearch.md @@ -2,9 +2,9 @@ title: "Elasticsearch" ring: adopt quadrant: platforms-and-aoe-services +tags: [devops] --- -We are continuing to use Elasticsearch successfully in [Searchperience®] and have benefited from the aggregation features for related use cases such as rendering category trees. -We are also using Elasticsearch for some microservices as our persistence solution. +We are continuing to use Elasticsearch successfully in [Searchperience®](http://www.searchperience.com/) and have benefited from the aggregation features for related use cases such as rendering category trees. We also use Elasticsearch as a persistence solution for various microservices. This is why we have updated its status to **adopt**. diff --git a/radar/2021-07-01/elk-stack.md b/radar/2021-07-01/elk-stack.md index 43849ea..2f15a5e 100644 --- a/radar/2021-07-01/elk-stack.md +++ b/radar/2021-07-01/elk-stack.md @@ -2,5 +2,6 @@ title: "ELK Stack" ring: adopt quadrant: platforms-and-aoe-services +tags: [devops] featured: false --- diff --git a/radar/2021-07-01/loki.md b/radar/2021-07-01/loki.md index 6008a21..421e251 100644 --- a/radar/2021-07-01/loki.md +++ b/radar/2021-07-01/loki.md @@ -1,15 +1,12 @@ --- -title: "Loki" -ring: trial -quadrant: tools -featured: true -tags: [devops] +title: "Loki" +ring: trial +quadrant: tools +tags: [devops] --- -Archiving indexed log data with a system like Elasticsearch can be expensive and archiving it as simple text files makes it hard to query them. -[Loki](https://grafana.com/oss/loki/) solves this issue by adding a reference database based on Kubernetes labels to each log line similar to Prometheus, but holding the log data inside a simple blob storage like S3. -This allows the user to query the data by pre-defined labels and keeps the costs for indexing low. +Archiving indexed log data with a system like [Elasticsearch](../platforms-and-aoe-services/elasticsearch.html) can be expensive, and archiving it as simple text files makes it hard to query. [Loki](https://grafana.com/oss/loki/) solves this issue by adding a reference database based on [Kubernetes](../platforms-and-aoe-services/kubernetes.html) labels to each log line, similar to [Prometheus](../platforms-and-aoe-services/prometheus.html), but storing the log data inside a simple blob storage like AWS S3. This allows the user to query the data by predefined labels and keeps the costs for indexing low. -Another benefit is the fact that does not have an endpoint for mutating log data which makes the data immutable from a potential compromised system. +Another benefit is that Loki does not have an endpoint for mutating log data, making the data immutable from a potentially compromised system. -We at AOE are using it for longer term log archiving in several Kubernetes clusters. +At AOE, we use Loki for long-term log archiving in several Kubernetes clusters. diff --git a/radar/2022-03-28/loki.md b/radar/2022-03-28/loki.md deleted file mode 100644 index c990b58..0000000 --- a/radar/2022-03-28/loki.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: "Loki" -ring: trial -quadrant: platforms-and-aoe-services -featured: true -tags: [devops] ---- - -Updated to adopt. diff --git a/radar/2023-11-01/loki.md b/radar/2023-11-01/loki.md index c2e2ad7..d24acfc 100644 --- a/radar/2023-11-01/loki.md +++ b/radar/2023-11-01/loki.md @@ -2,12 +2,12 @@ title: "Loki" ring: adopt quadrant: platforms-and-aoe-services -tags: [ devops ] +tags: [devops] --- -After having very positive experiences, we decided to replace our ELK stacks with Loki, primarily for the following reasons: +After having very positive experiences, we decided to replace our [ELK stacks](../platforms-and-aoe-services/elk-stack.html) with Loki, primarily for the following reasons: -- Loki is significantly more cost-effective than the storage requirements of Elasticsearch used in the ELK stack. +- Loki is significantly more cost-effective than the storage requirements of Elasticsearch. - The PromQL-like query language, familiar to users of Prometheus, makes it easier for DevOps and SRE teams who already use Prometheus for monitoring to work with logs. - Loki's native integration with Kubernetes simplifies the setup and configuration process. - Loki typically requires less maintenance and overhead compared to ELK. diff --git a/radar/2024-06-01/elk-stack.md b/radar/2024-06-01/elk-stack.md new file mode 100644 index 0000000..cfebc1e --- /dev/null +++ b/radar/2024-06-01/elk-stack.md @@ -0,0 +1,9 @@ +--- +title: "ELK Stack" +ring: adopt +quadrant: platforms-and-aoe-services +tags: [devops] +featured: false +--- + +While we continue to recommend the ELK Stack for specific use cases, we now prefer [Loki](../platforms-and-aoe-services/loki.html) for most [Kubernetes](../platforms-and-aoe-services/kubernetes.html)-based setups due to its seamless integration, cost efficiency and user-friendly query language.