Instructlab on Rafael Zago

Skupper + InstructLab: controlling and protecting AI (act 3)

Mon, 05 May 2025 00:00:00 -0300

See the Solution in Action

In this article we deploy the InstructLab chatbot on Kubernetes step by step, using Skupper to securely connect a private AI model to a public interface. This is the hands-on continuation of the solution pattern from the previous article.

Concepts and Commands Used in the Demo

NOTE: The following commands configure the environment and deploy the InstructLab chatbot. They are taken from the InstructLab project and adapted for this demo.

Skupper + InstructLab: controlling and protecting AI (act 2)

Sun, 04 May 2025 00:00:00 -0300

See the Solution in Action

In this article we prepare the full environment to serve the generative DeepSeek AI model with InstructLab: download the model, convert it to GGUF, and deploy the InstructLab chatbot. We also expose the service securely with Skupper.

Concepts and Commands Used in the Demo

NOTE Make sure you explain what each Skupper command does the first time you use it, especially for people new to Skupper. The following commands should be explained:

Skupper + InstructLab: controlling and protecting AI (act 1)

Fri, 02 May 2025 00:00:00 -0300

The story behind this solution pattern

Growing demand for AI-driven applications brings a hard problem: how do you deploy and operate AI models securely in environments that need strict data protection, while those models still have to be reachable by public services? That need became clear while building a local AI chatbot meant to handle sensitive and proprietary information — we needed a design that kept the model inside a protected environment.

InstructLab and Skupper: local AI without exposing data

Thu, 01 Aug 2024 00:00:00 -0300

Welcome to the Ollama Pilot.

Problem to solve

The main goal of this project is to create a secure connection between two sites, enabling the communication between the engineer machine and an Instruct Lab Model. The merlinite-7b-lab-Q4_K_M.gguf model will be used for the chatbot, and it is available in the Instruct Lab. The license of the model is available in the Instruct Labs.

But, why the banner? Well, the engineer needs to know who is better, Lebron or Jordan. The chatbot will be responsible for answering this question. The chatbot will receive the user input and send it to the llama3 model. The response from the merlinite model will be sent back to the user.