Elastic's RAG-based AI Assistant: Analyze application issues with LLMs and private GitHub issues

As an SRE, analyzing applications is more complex than ever. Not only do you have to ensure the application is running optimally to ensure great customer experiences, but you must also understand the inner workings in some cases to help troubleshoot. Analyzing issues in a production-based service is a team sport. It takes the SRE, DevOps, development, and support to get to the root cause and potentially remediate. If it's impacting, then it's even worse because there is a race against time. Regardless of the situation, there is a ton of information that needs to be consumed and processed. This includes not only what the customer is experiencing, but also internal data to help provide the most appropriate resolution.

Elastic’s AI Assistant helps improve analysis for SREs, DevOps, Devs, and others. In a single window using natural language questions, you can analyze using not only general information but combine it with things like:

Issues from internal GitHub repos, Jira, etc.
Documents from internal wiki sites from Confluence, etc.
Customer issues from your support service
And more

In this blog, we will walk you through how to:

Ingest an external GitHub repository (OpenTelemetry demo repo) with code and issues into Elastic. Apply Elastic Learned Sparse EncodeR (ELSER) and store it in a specific index for the AI Assistant.
Ingest internal GitHub repository with runbook information into Elastic. Apply ELSER and store the processed data in a specific index for the AI Assistant.
Use these two indices when analyzing issues for the OpenTelemetry demo in Elastic using the AI Assistant.

3 simple questions using GitHub data with AI Assistant

Before we walk through the steps for setting up data from GitHub, let’s review what an SRE can do with the AI Assistant and GitHub repos.

We initially connect to GitHub using an Elastic GitHub connector and ingest and process two repos: the OpenTelemetry demo repo (public) and an internal runbook repo (Elastic internal).

1 - elasticsearch connectors

With these two loaded and parsed by ELSER, we ask the AI Assistant some simple questions generally asked during analysis.

How many issues are open for the OpenTelemetry demo?

Since we ingested the entire repo (as of April 26, 2024) with a doc count of 1,529, we ask it a simple question regarding the total number of issues that are open. We specifically tell the AI Assistant to search our internal index to ensure the LLM knows to ask Elastic to search its internal index for the total number of issues.

Are there any issues for the Rust based shippingservice?

Elastic’s AI Assistant uses ELSER to traverse the loaded GitHub repo and finds the open issue against the shippingservice (which is the following issue at the time of writing this post).

Is there a runbook for the Cartservice?

Since we loaded an internal GitHub repo with a few sample runbooks, the Elastic AI Assistant properly finds the runbook.

As we go through this blog, we will talk about how the AI Assistant finds these issues using ELSER and how you can configure it to use your own GitHub repos.

Retrieval augmented generation (RAG) with Elastic AI Assistant

Elastic has the most advanced RAG-based AI Assistant for both Observability and Security. It can help you analyze your data using:

Your favorite LLM (OpenAI, Azure OpenAI, AWS Bedrock, etc.)
Any internal information (GitHub, Confluence, customer issues, etc.) you can either connect to or bring into Elastic’s indices

Elastic AI Assistant — connecting internal and external information

The reason Elastic’s AI Assistant can do this is because it supports RAG, which helps retrieve internal information along with LLM-based knowledge.

Adding relevant internal information for an SRE into Elastic:

As data comes in, such as in your GitHub repository, ELSER is applied to the data, and embeddings (weights and tokens into a sparse vector field) are added to capture semantic meaning and context of the data.
This data (GitHub, Confluence, etc.) is processed with embeddings and is stored in an index that can be searched by the AI Assistant.

When you query the AI Assistant for information:

The query goes through the same inference process as the ingested data using ELSER. The input query generates a “sparse vector,” which is used to find the most relevant highly ranked information in the ingested data (GitHub, Confluence, etc.).
The retrieved data is then combined with the query and also sent over to the LLM, which will then add its own knowledge base information (if there is anything to add), or it might ask Elastic (via function calls) to analyze, chart, or even search further. If a function call is made to Elastic and a response is provided, it will be added by the LLM to its response.
The results will be the most contextual based answer combining both LLM and anything relevant from your internal data.

3 - elastic's RAG flowchart

Application, prerequisites, and config

If you want to try the steps in this blog, here are some prerequisites:

An Elastic Cloud account — sign up now
OpenTelemetry demo running and connected to Elastic (APM documentation)
Whatever internal GitHub repo you want to use with some information that is useful for analysis (In our walk through, we will be using a GitHub repo that houses runbooks for different scenarios when Elastic does demos).
Account with your favorite or approved LLM (OpenAI, Azure OpenAI, AWS Bedrock)

Adding the GitHub repos to Elastic

The first step is to set up the GitHub connector and connect to your GitHub repo. Elastic has several connectors from GitHub, Confluence, Google Drive, Jira, AWS S3, Microsoft Teams, Slack, and more. So while we will go over the GitHub connector in this blog, don’t forget about other connectors.

4 - select a connector

Once you select the GitHub connector and give it a name, you need to add two items:

GitHub token
The URL open-telemetry/opentelemetry-demo

5 - configuration

Next, add it to an index in the wizard.

6 - attach an index

Create a pipeline and process the data with ELSER

In order to add the embeddings we discussed in the section above, we need to add the following to the connector:

Create a pipeline in the configuration wizard.
Create a custom pipeline.
Add the ML inference pipeline.
Select ELSERv2 ML Model to add the embeddings.
Select the fields that need to be evaluated as part of the inference pipeline.
Test and save the inference pipeline and the overall pipeline.

Sync the data

Now that the pipeline is created, you need to start to sync the github repo. As the documents from the github repo come in, they will go through the pipeline and embeddings will be added.

Embeddings

Once the pipeline is set up, sync the data in the connector. As the GitHub repository comes in, the inference pipeline will process the data as follows:

As data comes in from your GitHub repository, ELSER is applied to the data, and embeddings (weights and tokens into a sparse vector field) are added to capture semantic meaning and context of the data.
This data is processed with embeddings and is stored in an index that can be searched by the AI Assistant.

When you look at the OpenTelemetry GitHub documents that were ingested, you will see how the weights and token are added to the predicted_value field in the index.

These embeddings will now be used to find the most contextually relevant document when the user asks the AI Assistant a query, which might use this.

Check if AI Assistant can use the index

Elastic’s AI Assistant uses ELSER to traverse the loaded Github repo and finds the open issue against the shippingservice. (which is the following issue at the time of writing this post).

Based on the response, we can see that the AI Assistant can now use the index to find the issue and use it for further analysis.

Conclusion

You’ve now seen how easy Elastic’s RAG-based AI Assistant is to set up. You can bring in documents from multiple locations (GitHub, Confluent, Slack, etc.). We’ve shown the setup for GitHub and OpenTelemetry. This internal information can be useful in managing issues, accelerating resolution, and improving customer experiences. Check out our other blogs on how the AI Assistant can help SREs do better analysis, lower MTTR, and improve operations overall:

Try it out

Existing Elastic Cloud customers can access many of these features directly from the Elastic Cloud console. Not taking advantage of Elastic on cloud? Start a free trial.

All of this is also possible in your environments. Learn how to get started today.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

Elastic, Elasticsearch, ESRE, Elasticsearch Relevance Engine and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.

Elastic's RAG- based AI Assistant: Analyze application issues with LLMs and private GitHub issues