TaeHyeong Lee | Software Engineer
How to Create a Shell Script to Query LLM with Local File Using Kotlin and LangChain4j

How to Create a Shell Script to Query LLM with Local File Using Kotlin and LangChain4j

Taehyeong Lee's photo
Taehyeong Lee
·

4 min read

Introduction

  • OpenAI's GPT-4o model currently supports text and image data through the Chat Completions API but does not support binary files. By converting files such as PDF, EPUB, DOCX, PPTX, and XLSX to text, you can perform productive tasks with an LLM. In this post, we will create a shell script using the LangChain4j library in Kotlin to accept a local file path and query the LLM.

Script Features

  • Accepts a file path as the first argument.

  • Accepts a question as the second argument.

  • Based on the content of the file provided as the first argument, the LLM answers the question provided as the second argument. You can ask for translations, summaries, analyses, and more.

Prerequisites

  • Your OpenAI API Key

Installing Kotlin

  • Install Kotlin using SDKMAN.
# Install SDKMAN
$ curl -s "https://get.sdkman.io" | bash
$ source "$HOME/.sdkman/bin/sdkman-init.sh"

# Install Kotlin
$ sdk install kotlin

# Install Glow, the Markdown Console Viewer
$ brew install glow

Creating the File to Text Shell Script

$ nano askyourfile.main.kts
#!/usr/bin/env kotlin

// Import the LangChain4j library.
@file:DependsOn(
    "dev.langchain4j:langchain4j:0.35.0",
    "dev.langchain4j:langchain4j-embeddings:0.35.0",
    "dev.langchain4j:langchain4j-open-ai:0.35.0",
    "dev.langchain4j:langchain4j-document-parser-apache-tika:0.35.0"
)

import dev.langchain4j.data.document.Document
import dev.langchain4j.data.document.DocumentSplitter
import dev.langchain4j.data.document.loader.FileSystemDocumentLoader
import dev.langchain4j.data.document.loader.UrlDocumentLoader
import dev.langchain4j.data.document.parser.TextDocumentParser
import dev.langchain4j.data.document.parser.apache.tika.ApacheTikaDocumentParser
import dev.langchain4j.data.document.splitter.DocumentSplitters
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.SystemMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.data.segment.TextSegment
import dev.langchain4j.model.azure.AzureOpenAiChatModel
import dev.langchain4j.model.openai.OpenAiTokenizer
import dev.langchain4j.model.output.Response
import java.time.Duration
import kotlin.system.exitProcess

// Check if arguments are provided, display usage message and exit if not.
val params = args
if (params.isEmpty() || params.size < 2) {
    println(
        """
AskYourFile
- 0.1 by Tae-hyeong Lee

Usage: askyourfile.main.kts {filePath} {question}
       """.trimIndent()
    )
    exitProcess(1)
}

// Convert the content of the file provided as the first argument to text using Apache Tika.
val filePath = params[0]
val question = params[1]
val originalDocument: Document = try {
    if (filePath.startsWith("https")) {
        UrlDocumentLoader.load(filePath, TextDocumentParser())
    } else if (filePath.endsWith("html")) {
        FileSystemDocumentLoader.loadDocument(filePath, TextDocumentParser())
    } else {
        FileSystemDocumentLoader.loadDocument(filePath, ApacheTikaDocumentParser())
    }
} catch (ex: IllegalArgumentException) {
    println("[ERROR] File Not Found.")
    exitProcess(1)
}

// Split the file to ensure it does not exceed the maximum input token limit of 128K for the OpenAI GPT-4o model.
// Allocate an additional 8K space for system/user messages.
val documentSplitter = DocumentSplitters.recursive(
    128000 - 8192,
    256,
    OpenAiTokenizer("gpt-4o-2024-05-13")
)
val segments = documentSplitter.split(originalDocument)

// Create a LangChain4j OpenAI GPT-4o LLM object.
val chatModel: OpenAiChatModel = OpenAiChatModel.builder()
    .apiKey("{your-openai-api-key}")
    .timeout(Duration.ofSeconds(120))
    .modelName("gpt-4o-2024-05-13")
    .temperature(0.3)
    .topP(0.3)
    .build()

// Define the persona for the LLM in detail.
val systemMessage = SystemMessage(
    """
You are an assistant with extensive knowledge who answers questions based on the given data. Refer to the context below to answer and strictly adhere to the following conditions.

1. Answer in Markdown format.

2. Use informal language.

3. Base your answers primarily on the data content, and if the data is insufficient, supplement with your broad and accurate knowledge.

context: \\\
${segments[0].text()}
\\\
    """.trimIndent()
)

val userMessage = UserMessage(question)

// Request the LLM to answer the question.
val aiMessage: Response<AiMessage> = chatModel.generate(
    systemMessage,
    userMessage
)

// Output the result from the LLM.
println(aiMessage.content().text())
exitProcess(0)

Real World Example

  • It's time to use the script we created. We can ask questions using Steve Jobs' Stanford University commencement speech.
# [1] Download the PDF file of Steve Jobs' Stanford University Commencement Speech
$ wget https://tea4avcastro.tea.state.tx.us/thl/G7ELAR.W3.L1.steve-jobs-stanford-university-commencement-speech.pdf

# [2] Ask questions to the LLM using the downloaded file
$ ./askyourfile.main.kts G7ELAR.W3.L1.steve-jobs-stanford-university-commencement-speech.pdf "According to the content of the file, what values does Steve Jobs hold?"

We can summarize Steve Jobs' values through his Stanford University commencement speech into a few key points.

1. **Following Curiosity and Intuition**:
   Jobs audited classes he was interested in at Reed College instead of attending regular classes. Notably, taking a calligraphy class greatly influenced the beautiful typography design of the Macintosh. He said, "You can't connect the dots looking forward; you can only connect them looking backward." This means that while you can't predict the future, it's important to trust and follow your intuition and curiosity.

2. **Love and Passion**:
   Jobs emphasized the importance of finding what you love to do. Even after being fired from Apple, he founded NeXT and Pixar and achieved success because he still loved what he was doing. His statement, "The only way to do great work is to love what you do," clearly reflects his values.

3. **Awareness of Death**:
   Jobs believed that death is a significant change agent in life. He mentioned that he asked himself every day, "If today were the last day of my life, would I want to do what I am about to do today?" This reflects his philosophy that death removes all external expectations, pride, and fear of failure, leaving only what is truly important.

4. **Following One's Own Path**:
   Jobs stressed the importance of not being swayed by others' opinions and following one's inner voice. His statement, "Don't be trapped by dogma – which is living with the results of other people's thinking," means that it is crucial to find and follow your own path.

These values were shaped by the various experiences and challenges Jobs faced in his life and formed the foundation of his success and innovative thinking.
llmlangchain4jopenaiKotlin