Publishing WSO2 API Manager Analytics to an On-Premise ELK stack in a Kubernetes-based deployment

Introduction

An organization running an API Management platform often needs to look for statistics that would reveal insights into how the APIs deployed on the platform are consumed by the clients. It also provides essential statistics such as which devices and applications are using the statistics as well as whether the APIs are delivering successful responses to the clients or causing errors due to various reasons. Additionally, understanding the latency of an API is also of utmost importance to the administrators of the platform to determine whether the response times are within the SLAs agreed upon with the clients.

WSO2 API Manager provides a couple of analytics solutions to cater to the requirements of different organizations as described below.

1 ) Cloud-based analytics using Choreo cloud [3]

2) On-Premise analytics using ELK [2]

This article focuses on how WSO2 API Manager running on Kubernetes can be configured to push API analytics to an On-Premise ELK deployment.

Architecture

ELK-Based analytics architecture

WSO2 Analytics with ELK follows an architecture where metrics are written to a log file via a log4j2 appended and then that log file is parsed by a Filebeat log shipper which reads it and feeds it to a Logstash cluster. A customer Logstash configuration with a grok filter is deployed to capture the log patterns in scope and forwards it to ElasticSearch and subsequently visualized through Kibana dashboards. Regardless of whether On-Premise analytics are being used with a VM-based deployment or a Kubernetes-based deployment, this is the basic pattern that needs to be followed to implement on-premise analytics.

In a Kubernetes-based deployment, each API Manager pod will typically have a single API Manager container running and the container will write the metrics into its file system. In order to achieve the above architecture, it is essential for FileBeats to be able to read this log file. The solution that is used for this use case is to deploy Filebeats as a sidecar alongside the API Manager container and share the log file via a volume as illustrated in the following diagram.

Share analytics logs using volumes

This method ensures that the standard integration pattern can be followed with the exact same configuration in [2]. The only difference is that the filebeat component is run inside the container as a side car therefore there are no custom implementations necessary.

Implementation

Implementation involves configuring a volume to share the analytics log file between API Manager container and the Filebeat container and deploying filebeat as a sidecar.

1. Create a ConfigMap for Filebeat to mount filebeat.yml configuration file.

apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: wso2
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.inputs:
- type: log
enabled: true
paths:
- /home/wso2carbon/wso2am-4.0.0/repository/logs/apim_metrics.log
include_lines: ['(apimMetrics):']
output.logstash:
hosts: ["192.168.10.1:5044"]

2. Create two volumes in the API Manager pod. The first one is to share the log files between the two containers and the second one is for the filebeat-config ConfigMap.

volumes:
- name: wso2am-metrics
emptyDir: {}
- name: filebeat-config
configMap:
name: filebeat-config
items:
- key: filebeat.yml
path: filebeat.yml

3. Add Filebeat as a sidecar container in the API Manager pod. Note that both volumes created in step 2 are mounted to the sidecar container.

- name: filebeat
image: docker.elastic.co/beats/filebeat:8.3.3
volumeMounts:
- name: wso2am-metrics
mountPath: /home/wso2carbon/wso2am-4.0.0/repository/logs
- name: filebeat-config
mountPath: /usr/share/filebeat/filebeat.yml
subPath: filebeat.yml

4. Mount the wso2am-metrics volume to the API Manager container.

volumeMounts:
- name: wso2am-metrics
mountPath: /home/wso2carbon/wso2am-4.0.0/repository/logs

5. For all the other configurations, documentation [2] can be followed on any standard ELK stack and the statistics can be viewed through Kibana.

References

[1] https://apim.docs.wso2.com/en/latest/api-analytics/getting-started-guide/

[2] https://apim.docs.wso2.com/en/latest/api-analytics/on-prem/elk-installation-guide/

[3] https://console.choreo.dev

[4] https://phoenixnap.com/kb/how-to-install-elk-stack-on-ubuntu