AI coding assistant and Large Language Models (LLMs) on IDUN

IDUN provides access to coding assistant and LLMs that is running locally on IDUN HPC cluster.

These LLMs are available (updated 2025-08-14):

Model name	Create by	Purpose
openai/gpt-oss-120b	OpenAI	General purpose (recommended)
Qwen/Qwen3-Coder-30B-A3B-Instruct	Alibaba Cloud	AI coding assistant (recommended)
google/gemma-3-27b-it	Google DeepMind	General purpose
llama3.3:70b	Meta AI	General purpose
nomic-ai/nomic-embed-text-v2-moe	Nomic AI	AI coding assistant (embeding)

How to access

There several way to use this models:

1. Web interface to use models: https://idun-llm.hpc.ntnu.no (powered by Open WebUI)
2. In Visual Studio Code ( plugin Continue.dev )
3. JupiterLab ( plugin Jupyter AI )
4. Code Server ( plugin Continue.dev )
5. Use API (Open WebUI)

Web interface (Open WebUI)

Link: https://idun-llm.hpc.ntnu.no

Visual Studio Code (code server)

Start VS code web version on https://apps.hpc.ntnu.no

Install plugin Continue.dev from VS Code Marketplace.

Add configuration file by clicking on gear icon in a chat model list (see screenshot):

Another way to add configuration: Create directory in your user directory ".continue" with a dot at the start. Create configuration file "config.yaml" in that directory.

File content:

name: IDUN Assistant
version: 1.0.0
schema: v1
models:
  - name: Qwen/Qwen3-Coder-30B-A3B-Instruct
    provider: vllm
    model: Qwen/Qwen3-Coder-30B-A3B-Instruct
    apiBase: http://idun-ai:8001/v1
    roles:
      - chat
      - edit
      - apply
      - autocomplete
    capabilities:
        - tool_use
  - name: nomic-embed-text-v2-moe
    provider: vllm
    model: nomic-ai/nomic-embed-text-v2-moe
    apiBase: http://idun-ai:8002/v1
    roles:
      - embed
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

Main elements (screenshot):

Visual Studio Code (inside IDUN network)

Install plugin Continue.dev from VS Code Marketplace.

Add configuration file by clicking on gear icon in a chat model list (see screenshot):

Another way to add configuration: Create directory in your user directory ".continue" with a dot at the start. Create configuration file "config.yaml" in that directory.

File content:

name: IDUN Assistant
version: 1.0.0
schema: v1
models:
  - name: Qwen/Qwen3-Coder-30B-A3B-Instruct
    provider: vllm
    model: Qwen/Qwen3-Coder-30B-A3B-Instruct
    apiBase: http://idun-ai:8001/v1
    roles:
      - chat
      - edit
      - apply
      - autocomplete
    capabilities:
        - tool_use
  - name: nomic-embed-text-v2-moe
    provider: vllm
    model: nomic-ai/nomic-embed-text-v2-moe
    apiBase: http://idun-ai:8002/v1
    roles:
      - embed
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

Main elements (screenshot):

Visual Studio Code (outside IDUN network with API)

NOTE: This method does not support TAB autocomplete. Chat, Edit, Apply works.

Open WebUI can provides API access to LLM models. Link: https://idun-llm.hpc.ntnu.no

Obtain or create your Open WebUI API key.

Click [User Name] > Settings > Account > API keys in the Open WebUI

File content:

name: IDUN Assistant
version: 1.0.0
schema: v1
models:
  - name: Qwen/Qwen3-Coder-30B-A3B-Instruct
    provider: openai
    model: Qwen/Qwen3-Coder-30B-A3B-Instruct
    env:
      useLegacyCompletionsEndpoint: false
    apiBase: https://idun-llm.hpc.ntnu.no/api
    apiKey: <YOUR_OPEN_WEBUI_API_KEY>
    roles:
      - chat
      - edit
      - apply
    capabilities:
        - tool_use
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

Visual Studio Code (outside IDUN network with SSH)

IDUN does not have publicly opened ports to connect to a LLM. But it possible to make a SSH port forwarding.

Command for employee:

ssh -L 8001:idun-ai:8001 -L 8002:idun-ai:8002 -J USERNAME@login.ansatt.ntnu.no USERNAME@idun-login2.hpc.ntnu.no

command for students:

ssh -L 8001:idun-ai:8001 -L 8002:idun-ai:8002 -J USERNAME@login.stud.ntnu.no USERNAME@idun-login2.hpc.ntnu.no

Install plugin Continue.dev from VS Code Marketplace.

Add configuration file by clicking on gear icon in a chat model list (see screenshot):

Another way to add configuration: Create directory in your user directory ".continue" with a dot at the start. Create configuration file "config.yaml" in that directory.

File content:

name: IDUN Assistant
version: 1.0.0
schema: v1
models:
  - name: Qwen/Qwen3-Coder-30B-A3B-Instruct
    provider: vllm
    model: Qwen/Qwen3-Coder-30B-A3B-Instruct
    apiBase: http://localhost:8001/v1
    roles:
      - chat
      - edit
      - apply
      - autocomplete
    capabilities:
        - tool_use
  - name: nomic-embed-text-v2-moe
    provider: vllm
    model: nomic-ai/nomic-embed-text-v2-moe
    apiBase: http://localhost:8002/v1
    roles:
      - embed
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

Main elements (screenshot):

JupyterLab

Install plugin Jupyter AI. Example with new Python environment:

module load Python/3.12.3-GCCcore-13.3.0
python -m venv /cluster/home/USERNAME/JupiterAI
source /cluster/home/USERNAME/JupiterAI/bin/activate
pip install jupyterlab
pip install jupyter-ai[all]

Start JupyterLab via https://apps.hpc.ntnu.no

Change Jupyter AI settings:

Language Model
- Completion model: OPenAI (general interface)::*
- Base API URL: http://idun-ai:8001/v1

Embedding model
- Completion model: OPenAI (general interface)::*
- Base API URL: http://idun-ai:8002/v1

Inline completions model (Optional)
- Completion model: OPenAI (general interface)::*
- Base API URL: http://idun-ai:8001/v1

API

Open WebUI can provides API access to LLM models. Link: https://idun-llm.hpc.ntnu.no

Obtain or create your Open WebUI API key.

Click [User Name] > Settings > Account > API keys in the Open WebUI

Create Python script chat.py to use API:

#!/usr/bin/python
import sys
import requests
 
def chat_with_model(token,model,question):
    url = 'https://idun-llm.hpc.ntnu.no/api/chat/completions'
    headers = {
        'Authorization': f'Bearer {token}',
        'Content-Type': 'application/json'
    }
    data = {
      "model": model,
      "messages": [
        {
          "role": "user",
          "content": question
        }
      ]
    }
    response = requests.post(url, headers=headers, json=data)
    return response.json()
 
my_api_key = sys.argv[1]
my_model = sys.argv[2]
my_question = sys.argv[3]
answer = chat_with_model(my_api_key, my_model, my_question)
print(answer)

Use:

python3 chat.py 'MY-API-KEY' 'openai/gpt-oss-120b' 'Why sky is blue?'