My AGENTS.md

[corz]

I've been doing a lot of AutoIt coding recently, catching up on finishing off some apps, polishing GUIs, that sort of thing. Of course, I have no time for grunt work or messing with GUI crap, I just ask an agent to do it.

I'm getting great results, so I thought I'd share my AGENTS.md.

You will want to edit it, but it will give you a good starting point for getting agents to write code the way you like it, keep good backups, and much more. Some of my instructions won't apply (e.g. I run Codex in its own user account, so it can't use "rg" and I can't be bothered installing it for my Agent user, and so on). My needs are specific, and hopefully this shows you how you can get your specific needs met, too.

All agents will read this file (I keep hardlinked versions in all my project folders, as they will often "forget" to look in the parent folder for instructions) at the start of any thread, and I've barely edited it for weeks now, so it must be working ok!

Have fun vibing AutoIt!

AGENTS.md

Edited 5 hours ago by corz

[mLipok]

my global AGENTS.md is about 15k
and for one of my new project 54k
But I even have specific rules for coloring and GUI positioning.

[mLipok]

What kind of AI you are using ?
I currently use ChatGPT  for simple chat and with CODEX variant.

 

[corz]

Codex is good, dunno about "Variants", I just have Codex (though it can switch to chatty ChatGPT mode in the top-left corner, I notice. Not something I would ever click; I can use the web chat if I want to know which hot sauce is required).

Codex is pretty good, with a great understanding of AutoIt code at Sol Light and Medium (light can handle most jobs, medium is best for more complicated stuff), and I use it often (I have a free offer Plus plan right now and it's been useful) for coding tasks with good success. It helped me with a pretty huge translation job recently and the results were as good as I could have coded myself, and in about 1/1000th of the time.

I mainly use OpenCode, currently with Muse Spark 1.3 Contributor on High (XHigh sucks, and burns more tokens, though you still won't hit your OpenCode Go limit). The "Contributor" tier means they openly train on your data, but as everyone is training on your data I don't see an issue. When Muse Spark 1.4 comes out, I want it trained on how *I* like to work, and LOTS of AutoIt code.

Subscribe to OpenCode Go, is the best advice I can give in this regard. For $10/mo you basically have unlimited usage of a shit-ton of top models; Muse Spark 1.3, GLM 5.3 (what huggingface used to combat the OpenAI attack), the latest DeepSeek, Qwen, Mimo, and so on. Some models allow more tokens than others for your subscription cost, Muse Spark's allowance is MASSIVE. Basically, with an OpenCode Go subscription you get 6x compute cost, so $60/mo for $10, bulk buying, redundant compute, etc., makes it viable for them, cheap for us!

I use the CLI version of OpenCode when working with local (usually Qwen, good at code) models; long, slow tasks. I use the desktop version for back-and-forth vibe coding stuff with Muse.

Note: that last section of the AGENTS.md makes a *MASSIVE* difference when working with Muse Spark. Previously, reading its responses was like trying to parse machine code. Now it even replies to cultural references like "This is the way" with, wait for it: "This is the way"

Edited 4 hours ago by corz

[corz]

1 hour ago, mLipok said:

my global AGENTS.md is about 15k
and for one of my new project 54k
But I even have specific rules for coloring and GUI positioning.

That sounds too big to me. 

I have found that the more you add, the more is lost. Also, that shit needs to get sent with EVERY prompt.

Hey, tell you who is good at trimming an AGENTS.md down so that it will actually be read and followed: AI!

Edited 5 hours ago by corz

[argumentum]

..have you tried anything locally ( your GPU ) ?

[corz]

7 minutes ago, argumentum said:

..have you tried anything locally ( your GPU ) ?

Hah! You missed my recent post!

YES! Lots. I use Qwen 3.6/3.8 a fair bit. It's good at code and long agentic stuff, and spawning a shit-ton of sub-agents! The 35B/3B models are shaky, but the 27B dense models are actually shit-hot at code. I've ran head-to head challenges and then ran the results through frontier models for analysis and every time the 27B model wins, better architecture, just better code, thinks more like we do, keeping edge cases in mind. Muse is also really good at spotting edge cases, but that's not local!

Nothing fancy, I just have llama.cpp running (it just uses the models from my ComfyUI LLavacheckpoints dir) and OpenCode uses that. I have a batch file to choose what model I want to run right now:

@echo off
setlocal enabledelayedexpansion

:: ----------------------------------------------------
:: Configuration Pathways
:: ----------------------------------------------------
set "SERVER_EXE=I:\AI\llama.cpp\build\bin\Release\llama-server.exe"
set "MODEL_DIR=I:\AI\models\LLavacheckpoints"

:: Verify directory exists
if not exist "%MODEL_DIR%" (
    echo [ERROR] Model directory not found: %MODEL_DIR%
    pause
    exit /b
)

:MENU
cls
echo ====================================================
echo                 Available GGUF Models
echo ====================================================
echo.

:: Loop through directory and index GGUF files
set "count=0"
for %%F in ("%MODEL_DIR%\*.gguf") do (
    set "fname=%%~nxF"
    
    :: Skip files containing "mmproj" (case-insensitive check)
    if "!fname:mmproj=!"=="!fname!" (
        set /a count+=1
        set "model[!count!]=!fname!"
        set "model_path[!count!]=%%~fF"
        echo  [!count!] !fname!
    )
)

if %count%==0 (
    echo [ERROR] No eligible .gguf models found in %MODEL_DIR%
    pause
    exit /b
)

echo.
echo ====================================================
echo.

:: Prompt User for Choice
set "choice="
set /p "choice=Select a model number to launch (1-%count%): "

:: Validate Input
if not defined choice goto MENU
if %choice% lss 1 goto MENU
if %choice% gtr %count% goto MENU

:: Resolve selection variables
set "SELECTED_MODEL_NAME=!model[%choice%]!"
set "SELECTED_MODEL_PATH=!model_path[%choice%]!"
set "MODEL_STEM=%%~nM"

:: Generate paths based on the model file
for %%M in ("!SELECTED_MODEL_PATH!") do (
    set "PRESET_FILE=%%~dpnM.preset"
    set "BASE_STEM=%%~nM"
)

:: ----------------------------------------------------
:: Check for MTP in filename (Case-Insensitive)
:: ----------------------------------------------------
set "MTP_PARAMS="
echo "!SELECTED_MODEL_NAME!" | findstr /I "MTP" >nul 2>&1
if !errorlevel! equ 0 (
    set "MTP_PARAMS=--mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4 --no-mmap --threads 8 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0"
)


:: ----------------------------------------------------
:: Check for a matching mmproj file
:: ----------------------------------------------------
set "MMPROJ_PARAM="
set "base_name=!SELECTED_MODEL_NAME!"

for %%G in (
    -Q2_K -Q2_K_S -Q2_K_L
    -Q3_K_S -Q3_K_M -Q3_K_L -IQ3_XXS -IQ3_S -IQ3_M -IQ3_XS
    -Q4_K_S -Q4_1 -Q4_0 -Q4_K_M -Q4_K_L -Q4_K_XL -IQ4_NL -IQ4_XS
    -Q5_0 -Q5_1 -Q5_K_S -Q5_K_M
    -Q6_K
    -Q8_0 -Q8_1 -Q8_K
    -IQ1_S -IQ1_M -IQ2_S -IQ2_M -IQ2_XXS -IQ2_XS
    -FP16 -BF16 -F16 -F32
) do (
    set "base_name=!base_name:%%G=!"
)
set "base_name=!base_name:.gguf=!"

if exist "%MODEL_DIR%\*!base_name!*mmproj*.gguf" (
    for %%M in ("%MODEL_DIR%\*!base_name!*mmproj*.gguf") do (
        set "MMPROJ_PARAM=--mmproj "%%~fM""
    )
)

:: ----------------------------------------------------
:: Check and Parse Linux-Style .preset File (with # and #old)
:: ----------------------------------------------------
set "CUSTOM_PRESET="
set "skip_rest=false"

if exist "!PRESET_FILE!" (
    set "PRESET_LOADED=true"
    for /f "usebackq delims=" %%L in ("!PRESET_FILE!") do (
        if "!skip_rest!"=="false" (
            set "line=%%L"
            
            :: Strip leading/trailing whitespaces safely
            for /f "tokens=* delims= " %%A in ("!line!") do set "line=%%A"
            
            if defined line (
                :: Check for the hard cutoff token.
                :: Normalize whitespace to handle spaces/tabs between # and old
                set "clean_line=!line!"
                set "clean_line=!clean_line: =!"
                set "clean_line=!clean_line:	=!"
                
                if "!clean_line!"=="#old" (
                    set "skip_rest=true"
                ) else (                    :: Check if the line starts with a regular comment character (#)
                    set "first_char=!line:~0,1!"
                    if not "!first_char!"=="#" (
                        :: Check if the line ends with a backslash
                        if "!line:~-1!"=="\" (
                            set "line=!line:~0,-1!"
                        )
                        :: Append to our preset string
                        set "CUSTOM_PRESET=!CUSTOM_PRESET! !line!"
                    )
                )
            )
        )
    )
) else (
    set "PRESET_LOADED=false"
)


cls
echo ====================================================
echo  Launching llama-server
echo  Model: %SELECTED_MODEL_NAME%
if "!PRESET_LOADED!"=="true" (
    echo  Preset: Loaded Custom Configuration File (.preset)
) else (
    echo  Preset: None found. Using script defaults.
)
if defined MTP_PARAMS (
    echo  Mode: Multi-Token Prediction ^(MTP^) Enabled
)
if defined MMPROJ_PARAM (
    echo  Vision: Multimodal Projector Attached
)
echo ====================================================
echo.

:: :: Execute based on preset existence
:: if "!PRESET_LOADED!"=="true" (
    :: "%SERVER_EXE%" -m "%SELECTED_MODEL_PATH%" !CUSTOM_PRESET! !MTP_PARAMS! !MMPROJ_PARAM!
:: ) else (
    :: "%SERVER_EXE%" -m "%SELECTED_MODEL_PATH%" --seed 0 --host 127.0.0.1 --port 8080 --n-gpu-layers 999 --ctx-size 262144 -np 1 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --keep 4096 --timeout 600 --flash-attn on --presence-penalty 0.8 --repeat-penalty 1.1 --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}"  --chat-template-file I:\AI\models\LLavacheckpoints\Qwen3.6_chat_template.jinja --cache-type-k q8_0 --cache-type-v q8_0 !MTP_PARAMS! !MMPROJ_PARAM!
:: )

:: Execute based on preset existence
if "!PRESET_LOADED!"=="true" (
    "%SERVER_EXE%" -m "!SELECTED_MODEL_PATH!" !CUSTOM_PRESET!
) else (
    "%SERVER_EXE%" -m "!SELECTED_MODEL_PATH!" --seed 0 --host 127.0.0.1 --port 8080 --n-gpu-layers 999 --ctx-size 262144 -np 1 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --keep 4096 --timeout 600 --flash-attn on --presence-penalty 0.8 --repeat-penalty 1.1 --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}" --chat-template-file I:\AI\models\LLavacheckpoints\Qwen3.6_chat_template.jinja --cache-type-k q8_0 --cache-type-v q8_0 !MTP_PARAMS! !MMPROJ_PARAM!
)

exit

Fuck Ollama!

Edited 5 hours ago by corz

[argumentum]

On my side I'll do something like this:

Spoiler

Here is the complete AutoIt GUI script. It replaces the console prompt with a window containing a dropdown selector, dynamic configuration preview, and control buttons (Launch, Refresh, and Exit).

 

AutoIt

#pragma compile(Icon, "")
#include <GUIConstantsEx.au3>
#include <ComboConstants.au3>
#include <File.au3>
#include <Array.au3>

; ----------------------------------------------------
; Configuration Pathways
; ----------------------------------------------------
Global Const $SERVER_EXE = "I:\AI\llama.cpp\build\bin\Release\llama-server.exe"
Global Const $MODEL_DIR  = "I:\AI\models\LLavacheckpoints"

; Global Arrays for Model Data
Global $g_aModelName[0]
Global $g_aModelPath[0]

; ----------------------------------------------------
; Create Graphical Interface
; ----------------------------------------------------
Local $hGUI = GUICreate("llama-server Model Launcher", 520, 360, -1, -1)
GUISetFont(9, 400, 0, "Segoe UI")

; Title Header
GUICtrlCreateLabel("llama-server Launcher", 20, 15, 480, 25)
GUICtrlSetFont(-1, 12, 800, 0, "Segoe UI")

; Dropdown Selection
GUICtrlCreateLabel("Select GGUF Model:", 20, 50, 480, 20)
Local $idComboModels = GUICtrlCreateCombo("", 20, 72, 480, 25, $CBS_DROPDOWNLIST)

; Model Status Preview Box
GUICtrlCreateGroup(" Selected Model Configuration ", 20, 115, 480, 175)
Local $idLblPreset  = GUICtrlCreateLabel("Preset File: -", 35, 140, 450, 20)
Local $idLblMtp     = GUICtrlCreateLabel("MTP Mode: -", 35, 168, 450, 20)
Local $idLblVision  = GUICtrlCreateLabel("Vision Projector: -", 35, 196, 450, 20)
Local $idLblStatus  = GUICtrlCreateLabel("Status: Ready", 35, 250, 450, 25)
GUICtrlSetFont($idLblStatus, 9, 700, 0, "Segoe UI")
GUICtrlCreateGroup("", -99, -99, 1, 1)

; Control Buttons
Local $idBtnRefresh = GUICtrlCreateButton("Refresh", 20, 308, 110, 32)
Local $idBtnLaunch  = GUICtrlCreateButton("Launch Server", 250, 308, 140, 32)
GUICtrlSetFont($idBtnLaunch, 9, 700, 0, "Segoe UI")
Local $idBtnExit    = GUICtrlCreateButton("Exit", 400, 308, 100, 32)

; Initial Scan & Render
ScanModels($idComboModels, $idLblStatus)
UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

GUISetState(@SW_SHOW, $hGUI)

; ----------------------------------------------------
; Main Application Event Loop
; ----------------------------------------------------
While 1
    Local $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE, $idBtnExit
            ExitLoop

        Case $idComboModels
            UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

        Case $idBtnRefresh
            ScanModels($idComboModels, $idLblStatus)
            UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

        Case $idBtnLaunch
            LaunchServer($idComboModels, $idLblStatus)
    EndSwitch
WEnd

GUIDelete($hGUI)

; ----------------------------------------------------
; Helper Functions
; ----------------------------------------------------

Func ScanModels($idCombo, $idStatus)
    GUICtrlSetData($idCombo, "") ; Clear existing list
    ReDim $g_aModelName[0]
    ReDim $g_aModelPath[0]

    If Not FileExists($MODEL_DIR) Then
        GUICtrlSetData($idStatus, "Status: ERROR - Model directory not found!")
        Return
    EndIf

    Local $hSearch = FileFindFirstFile($MODEL_DIR & "\*.gguf")
    If $hSearch = -1 Then
        GUICtrlSetData($idStatus, "Status: No .gguf models found!")
        Return
    EndIf

    Local $sComboString = ""
    While 1
        Local $sFile = FileFindNextFile($hSearch)
        If @error Then ExitLoop

        ; Exclude vision projectors (mmproj) from the main dropdown
        If Not StringInStr($sFile, "mmproj", 0) Then
            _ArrayAdd($g_aModelName, $sFile)
            _ArrayAdd($g_aModelPath, $MODEL_DIR & "\" & $sFile)
            $sComboString &= $sFile & "|"
        EndIf
    WEnd
    FileClose($hSearch)

    If UBound($g_aModelName) = 0 Then
        GUICtrlSetData($idStatus, "Status: No eligible models found!")
        Return
    EndIf

    ; Remove trailing delimiter
    If StringRight($sComboString, 1) = "|" Then $sComboString = StringTrimRight($sComboString, 1)

    ; Populate Combo Box & Select First Item
    GUICtrlSetData($idCombo, $sComboString, $g_aModelName[0])
    GUICtrlSetData($idStatus, "Status: Found " & UBound($g_aModelName) & " model(s).")
EndFunc

Func GetSelectedIndex($idCombo)
    Local $sSelected = GUICtrlRead($idCombo)
    For $i = 0 To UBound($g_aModelName) - 1
        If $g_aModelName[$i] = $sSelected Then Return $i
    Next
    Return -1
EndFunc

Func UpdateModelInfo($idCombo, $idPreset, $idMtp, $idVision)
    Local $iIndex = GetSelectedIndex($idCombo)
    If $iIndex = -1 Then
        GUICtrlSetData($idPreset, "Preset File: None")
        GUICtrlSetData($idMtp, "MTP Mode: Disabled")
        GUICtrlSetData($idVision, "Vision Projector: None")
        Return
    EndIf

    Local $sSelectedName = $g_aModelName[$iIndex]
    Local $sSelectedPath = $g_aModelPath[$iIndex]

    ; Check for .preset File
    Local $sDrive, $sDir, $sFileName, $sExt
    _PathSplit($sSelectedPath, $sDrive, $sDir, $sFileName, $sExt)
    Local $sPresetFile = $sDrive & $sDir & $sFileName & ".preset"

    If FileExists($sPresetFile) Then
        GUICtrlSetData($idPreset, "Preset File: Custom (" & $sFileName & ".preset)")
    Else
        GUICtrlSetData($idPreset, "Preset File: None (Script defaults will be used)")
    EndIf

    ; Check for MTP
    If StringInStr($sSelectedName, "MTP", 0) Then
        GUICtrlSetData($idMtp, "MTP Mode: Enabled (Multi-Token Prediction)")
    Else
        GUICtrlSetData($idMtp, "MTP Mode: Disabled")
    EndIf

    ; Check for Multimodal Projector
    Local $sMmprojFile = FindMmprojFile($sSelectedName)
    If $sMmprojFile <> "" Then
        GUICtrlSetData($idVision, "Vision Projector: Attached (" & $sMmprojFile & ")")
    Else
        GUICtrlSetData($idVision, "Vision Projector: None Attached")
    EndIf
EndFunc

Func FindMmprojFile($sModelName)
    Local $sBaseName = $sModelName
    Local $aQuantifiers[38] = [ _
        "-Q2_K", "-Q2_K_S", "-Q2_K_L", _
        "-Q3_K_S", "-Q3_K_M", "-Q3_K_L", "-IQ3_XXS", "-IQ3_S", "-IQ3_M", "-IQ3_XS", _
        "-Q4_K_S", "-Q4_1", "-Q4_0", "-Q4_K_M", "-Q4_K_L", "-Q4_K_XL", "-IQ4_NL", "-IQ4_XS", _
        "-Q5_0", "-Q5_1", "-Q5_K_S", "-Q5_K_M", _
        "-Q6_K", _
        "-Q8_0", "-Q8_1", "-Q8_K", _
        "-IQ1_S", "-IQ1_M", "-IQ2_S", "-IQ2_M", "-IQ2_XXS", "-IQ2_XS", _
        "-FP16", "-BF16", "-F16", "-F32" _
    ]

    For $sQuant In $aQuantifiers
        $sBaseName = StringReplace($sBaseName, $sQuant, "", 0, 0)
    Next
    $sBaseName = StringReplace($sBaseName, ".gguf", "", 0, 0)

    Local $hMmprojSearch = FileFindFirstFile($MODEL_DIR & "\*" & $sBaseName & "*mmproj*.gguf")
    If $hMmprojSearch <> -1 Then
        Local $sFile = FileFindNextFile($hMmprojSearch)
        FileClose($hMmprojSearch)
        If Not @error Then Return $sFile
    EndIf
    Return ""
EndFunc

Func ParsePresetFile($sPresetFile)
    Local $sCustomPreset = ""
    If FileExists($sPresetFile) Then
        Local $hFile = FileOpen($sPresetFile, 0)
        If $hFile <> -1 Then
            While 1
                Local $sLine = FileReadLine($hFile)
                If @error Then ExitLoop

                $sLine = StringStripWS($sLine, 3)
                If $sLine <> "" Then
                    Local $sCleanLine = StringReplace(StringReplace($sLine, " ", ""), @TAB, "")
                    If StringLower($sCleanLine) = "#old" Then ExitLoop

                    If StringLeft($sLine, 1) <> "#" Then
                        If StringRight($sLine, 1) = "\" Then $sLine = StringTrimRight($sLine, 1)
                        $sCustomPreset &= " " & $sLine
                    EndIf
                EndIf
            WEnd
            FileClose($hFile)
        EndIf
    EndIf
    Return $sCustomPreset
EndFunc

Func LaunchServer($idCombo, $idStatus)
    Local $iIndex = GetSelectedIndex($idCombo)
    If $iIndex = -1 Then
        MsgBox(16, "Error", "No model selected from the dropdown list.")
        Return
    EndIf

    Local $sSelectedName = $g_aModelName[$iIndex]
    Local $sSelectedPath = $g_aModelPath[$iIndex]

    ; Split path for preset
    Local $sDrive, $sDir, $sFileName, $sExt
    _PathSplit($sSelectedPath, $sDrive, $sDir, $sFileName, $sExt)
    Local $sPresetFile = $sDrive & $sDir & $sFileName & ".preset"

    ; Determine MTP Parameters
    Local $sMtpParams = ""
    If StringInStr($sSelectedName, "MTP", 0) Then
        $sMtpParams = "--mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4 --no-mmap --threads 8 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0"
    EndIf

    ; Determine Multimodal Projector Parameter
    Local $sMmprojFile = FindMmprojFile($sSelectedName)
    Local $sMmprojParam = ""
    If $sMmprojFile <> "" Then
        $sMmprojParam = '--mmproj "' & $MODEL_DIR & "\" & $sMmprojFile & '"'
    EndIf

    ; Parse custom preset file if exists
    Local $sCustomPreset = ParsePresetFile($sPresetFile)

    ; Build Full Executable Command Line
    Local $sFullCmd = ""
    If FileExists($sPresetFile) Then
        $sFullCmd = '"' & $SERVER_EXE & '" -m "' & $sSelectedPath & '"' & $sCustomPreset
    Else
        Local $sDefaultArgs = '--seed 0 --host 127.0.0.1 --port 8080 --n-gpu-layers 999 --ctx-size 262144 -np 1 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --keep 4096 --timeout 600 --flash-attn on --presence-penalty 0.8 --repeat-penalty 1.1 --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}" --chat-template-file I:\AI\models\LLavacheckpoints\Qwen3.6_chat_template.jinja --cache-type-k q8_0 --cache-type-v q8_0'
        $sFullCmd = '"' & $SERVER_EXE & '" -m "' & $sSelectedPath & '" ' & $sDefaultArgs
        If $sMtpParams <> "" Then $sFullCmd &= " " & $sMtpParams
        If $sMmprojParam <> "" Then $sFullCmd &= " " & $sMmprojParam
    EndIf

    GUICtrlSetData($idStatus, "Status: Launching server for " & $sSelectedName & "...")
    Run($sFullCmd)
EndFunc

Key Enhancements in the GUI Version:

• Dropdown Box (GUICtrlCreateCombo): Automatically populates with all valid models in I:\AI\models\LLavacheckpoints.
• Live Configuration Preview: Selecting any model immediately displays whether a .preset file exists, if MTP draft parameters will be injected, or if a matching vision projector (mmproj) was found.
• Refresh Button: Allows rescanning your folder for newly downloaded models without closing the launcher.
• Non-Blocking Launch: Clicking Launch Server executes llama-server.exe in the background and updates the UI status label without freezing the window.

because I rather GUI everything. ( untested slop for now )

Have to get me "ggerganov/llama.cpp" and become familiar with it.

What is your local hardware setup ? I've got a 5060Ti on this PC and a Strix halo that has more memory, but is slower 😕 

[corz]

1 minute ago, argumentum said:

On my side I'll do something like this:

  Reveal hidden contents

Here is the complete AutoIt GUI script. It replaces the console prompt with a window containing a dropdown selector, dynamic configuration preview, and control buttons (Launch, Refresh, and Exit).

 

AutoIt

 

#pragma compile(Icon, "")
#include <GUIConstantsEx.au3>
#include <ComboConstants.au3>
#include <File.au3>
#include <Array.au3>

; ----------------------------------------------------
; Configuration Pathways
; ----------------------------------------------------
Global Const $SERVER_EXE = "I:\AI\llama.cpp\build\bin\Release\llama-server.exe"
Global Const $MODEL_DIR  = "I:\AI\models\LLavacheckpoints"

; Global Arrays for Model Data
Global $g_aModelName[0]
Global $g_aModelPath[0]

; ----------------------------------------------------
; Create Graphical Interface
; ----------------------------------------------------
Local $hGUI = GUICreate("llama-server Model Launcher", 520, 360, -1, -1)
GUISetFont(9, 400, 0, "Segoe UI")

; Title Header
GUICtrlCreateLabel("llama-server Launcher", 20, 15, 480, 25)
GUICtrlSetFont(-1, 12, 800, 0, "Segoe UI")

; Dropdown Selection
GUICtrlCreateLabel("Select GGUF Model:", 20, 50, 480, 20)
Local $idComboModels = GUICtrlCreateCombo("", 20, 72, 480, 25, $CBS_DROPDOWNLIST)

; Model Status Preview Box
GUICtrlCreateGroup(" Selected Model Configuration ", 20, 115, 480, 175)
Local $idLblPreset  = GUICtrlCreateLabel("Preset File: -", 35, 140, 450, 20)
Local $idLblMtp     = GUICtrlCreateLabel("MTP Mode: -", 35, 168, 450, 20)
Local $idLblVision  = GUICtrlCreateLabel("Vision Projector: -", 35, 196, 450, 20)
Local $idLblStatus  = GUICtrlCreateLabel("Status: Ready", 35, 250, 450, 25)
GUICtrlSetFont($idLblStatus, 9, 700, 0, "Segoe UI")
GUICtrlCreateGroup("", -99, -99, 1, 1)

; Control Buttons
Local $idBtnRefresh = GUICtrlCreateButton("Refresh", 20, 308, 110, 32)
Local $idBtnLaunch  = GUICtrlCreateButton("Launch Server", 250, 308, 140, 32)
GUICtrlSetFont($idBtnLaunch, 9, 700, 0, "Segoe UI")
Local $idBtnExit    = GUICtrlCreateButton("Exit", 400, 308, 100, 32)

; Initial Scan & Render
ScanModels($idComboModels, $idLblStatus)
UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

GUISetState(@SW_SHOW, $hGUI)

; ----------------------------------------------------
; Main Application Event Loop
; ----------------------------------------------------
While 1
    Local $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE, $idBtnExit
            ExitLoop

        Case $idComboModels
            UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

        Case $idBtnRefresh
            ScanModels($idComboModels, $idLblStatus)
            UpdateModelInfo($idComboModels, $idLblPreset, $idLblMtp, $idLblVision)

        Case $idBtnLaunch
            LaunchServer($idComboModels, $idLblStatus)
    EndSwitch
WEnd

GUIDelete($hGUI)

; ----------------------------------------------------
; Helper Functions
; ----------------------------------------------------

Func ScanModels($idCombo, $idStatus)
    GUICtrlSetData($idCombo, "") ; Clear existing list
    ReDim $g_aModelName[0]
    ReDim $g_aModelPath[0]

    If Not FileExists($MODEL_DIR) Then
        GUICtrlSetData($idStatus, "Status: ERROR - Model directory not found!")
        Return
    EndIf

    Local $hSearch = FileFindFirstFile($MODEL_DIR & "\*.gguf")
    If $hSearch = -1 Then
        GUICtrlSetData($idStatus, "Status: No .gguf models found!")
        Return
    EndIf

    Local $sComboString = ""
    While 1
        Local $sFile = FileFindNextFile($hSearch)
        If @error Then ExitLoop

        ; Exclude vision projectors (mmproj) from the main dropdown
        If Not StringInStr($sFile, "mmproj", 0) Then
            _ArrayAdd($g_aModelName, $sFile)
            _ArrayAdd($g_aModelPath, $MODEL_DIR & "\" & $sFile)
            $sComboString &= $sFile & "|"
        EndIf
    WEnd
    FileClose($hSearch)

    If UBound($g_aModelName) = 0 Then
        GUICtrlSetData($idStatus, "Status: No eligible models found!")
        Return
    EndIf

    ; Remove trailing delimiter
    If StringRight($sComboString, 1) = "|" Then $sComboString = StringTrimRight($sComboString, 1)

    ; Populate Combo Box & Select First Item
    GUICtrlSetData($idCombo, $sComboString, $g_aModelName[0])
    GUICtrlSetData($idStatus, "Status: Found " & UBound($g_aModelName) & " model(s).")
EndFunc

Func GetSelectedIndex($idCombo)
    Local $sSelected = GUICtrlRead($idCombo)
    For $i = 0 To UBound($g_aModelName) - 1
        If $g_aModelName[$i] = $sSelected Then Return $i
    Next
    Return -1
EndFunc

Func UpdateModelInfo($idCombo, $idPreset, $idMtp, $idVision)
    Local $iIndex = GetSelectedIndex($idCombo)
    If $iIndex = -1 Then
        GUICtrlSetData($idPreset, "Preset File: None")
        GUICtrlSetData($idMtp, "MTP Mode: Disabled")
        GUICtrlSetData($idVision, "Vision Projector: None")
        Return
    EndIf

    Local $sSelectedName = $g_aModelName[$iIndex]
    Local $sSelectedPath = $g_aModelPath[$iIndex]

    ; Check for .preset File
    Local $sDrive, $sDir, $sFileName, $sExt
    _PathSplit($sSelectedPath, $sDrive, $sDir, $sFileName, $sExt)
    Local $sPresetFile = $sDrive & $sDir & $sFileName & ".preset"

    If FileExists($sPresetFile) Then
        GUICtrlSetData($idPreset, "Preset File: Custom (" & $sFileName & ".preset)")
    Else
        GUICtrlSetData($idPreset, "Preset File: None (Script defaults will be used)")
    EndIf

    ; Check for MTP
    If StringInStr($sSelectedName, "MTP", 0) Then
        GUICtrlSetData($idMtp, "MTP Mode: Enabled (Multi-Token Prediction)")
    Else
        GUICtrlSetData($idMtp, "MTP Mode: Disabled")
    EndIf

    ; Check for Multimodal Projector
    Local $sMmprojFile = FindMmprojFile($sSelectedName)
    If $sMmprojFile <> "" Then
        GUICtrlSetData($idVision, "Vision Projector: Attached (" & $sMmprojFile & ")")
    Else
        GUICtrlSetData($idVision, "Vision Projector: None Attached")
    EndIf
EndFunc

Func FindMmprojFile($sModelName)
    Local $sBaseName = $sModelName
    Local $aQuantifiers[38] = [ _
        "-Q2_K", "-Q2_K_S", "-Q2_K_L", _
        "-Q3_K_S", "-Q3_K_M", "-Q3_K_L", "-IQ3_XXS", "-IQ3_S", "-IQ3_M", "-IQ3_XS", _
        "-Q4_K_S", "-Q4_1", "-Q4_0", "-Q4_K_M", "-Q4_K_L", "-Q4_K_XL", "-IQ4_NL", "-IQ4_XS", _
        "-Q5_0", "-Q5_1", "-Q5_K_S", "-Q5_K_M", _
        "-Q6_K", _
        "-Q8_0", "-Q8_1", "-Q8_K", _
        "-IQ1_S", "-IQ1_M", "-IQ2_S", "-IQ2_M", "-IQ2_XXS", "-IQ2_XS", _
        "-FP16", "-BF16", "-F16", "-F32" _
    ]

    For $sQuant In $aQuantifiers
        $sBaseName = StringReplace($sBaseName, $sQuant, "", 0, 0)
    Next
    $sBaseName = StringReplace($sBaseName, ".gguf", "", 0, 0)

    Local $hMmprojSearch = FileFindFirstFile($MODEL_DIR & "\*" & $sBaseName & "*mmproj*.gguf")
    If $hMmprojSearch <> -1 Then
        Local $sFile = FileFindNextFile($hMmprojSearch)
        FileClose($hMmprojSearch)
        If Not @error Then Return $sFile
    EndIf
    Return ""
EndFunc

Func ParsePresetFile($sPresetFile)
    Local $sCustomPreset = ""
    If FileExists($sPresetFile) Then
        Local $hFile = FileOpen($sPresetFile, 0)
        If $hFile <> -1 Then
            While 1
                Local $sLine = FileReadLine($hFile)
                If @error Then ExitLoop

                $sLine = StringStripWS($sLine, 3)
                If $sLine <> "" Then
                    Local $sCleanLine = StringReplace(StringReplace($sLine, " ", ""), @TAB, "")
                    If StringLower($sCleanLine) = "#old" Then ExitLoop

                    If StringLeft($sLine, 1) <> "#" Then
                        If StringRight($sLine, 1) = "\" Then $sLine = StringTrimRight($sLine, 1)
                        $sCustomPreset &= " " & $sLine
                    EndIf
                EndIf
            WEnd
            FileClose($hFile)
        EndIf
    EndIf
    Return $sCustomPreset
EndFunc

Func LaunchServer($idCombo, $idStatus)
    Local $iIndex = GetSelectedIndex($idCombo)
    If $iIndex = -1 Then
        MsgBox(16, "Error", "No model selected from the dropdown list.")
        Return
    EndIf

    Local $sSelectedName = $g_aModelName[$iIndex]
    Local $sSelectedPath = $g_aModelPath[$iIndex]

    ; Split path for preset
    Local $sDrive, $sDir, $sFileName, $sExt
    _PathSplit($sSelectedPath, $sDrive, $sDir, $sFileName, $sExt)
    Local $sPresetFile = $sDrive & $sDir & $sFileName & ".preset"

    ; Determine MTP Parameters
    Local $sMtpParams = ""
    If StringInStr($sSelectedName, "MTP", 0) Then
        $sMtpParams = "--mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4 --no-mmap --threads 8 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0"
    EndIf

    ; Determine Multimodal Projector Parameter
    Local $sMmprojFile = FindMmprojFile($sSelectedName)
    Local $sMmprojParam = ""
    If $sMmprojFile <> "" Then
        $sMmprojParam = '--mmproj "' & $MODEL_DIR & "\" & $sMmprojFile & '"'
    EndIf

    ; Parse custom preset file if exists
    Local $sCustomPreset = ParsePresetFile($sPresetFile)

    ; Build Full Executable Command Line
    Local $sFullCmd = ""
    If FileExists($sPresetFile) Then
        $sFullCmd = '"' & $SERVER_EXE & '" -m "' & $sSelectedPath & '"' & $sCustomPreset
    Else
        Local $sDefaultArgs = '--seed 0 --host 127.0.0.1 --port 8080 --n-gpu-layers 999 --ctx-size 262144 -np 1 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --keep 4096 --timeout 600 --flash-attn on --presence-penalty 0.8 --repeat-penalty 1.1 --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}" --chat-template-file I:\AI\models\LLavacheckpoints\Qwen3.6_chat_template.jinja --cache-type-k q8_0 --cache-type-v q8_0'
        $sFullCmd = '"' & $SERVER_EXE & '" -m "' & $sSelectedPath & '" ' & $sDefaultArgs
        If $sMtpParams <> "" Then $sFullCmd &= " " & $sMtpParams
        If $sMmprojParam <> "" Then $sFullCmd &= " " & $sMmprojParam
    EndIf

    GUICtrlSetData($idStatus, "Status: Launching server for " & $sSelectedName & "...")
    Run($sFullCmd)
EndFunc

Key Enhancements in the GUI Version:

• Dropdown Box (GUICtrlCreateCombo): Automatically populates with all valid models in I:\AI\models\LLavacheckpoints.
• Live Configuration Preview: Selecting any model immediately displays whether a .preset file exists, if MTP draft parameters will be injected, or if a matching vision projector (mmproj) was found.
• Refresh Button: Allows rescanning your folder for newly downloaded models without closing the launcher.
• Non-Blocking Launch: Clicking Launch Server executes llama-server.exe in the background and updates the UI status label without freezing the window.

because I rather GUI everything. ( untested slop for now )

Have to get me "ggerganov/llama.cpp" and become familiar with it.

What is your local hardware setup ? I've got a 5060Ti on this PC and a Strix halo that has more memory, but is slower 😕 

RTX 3060 12GB. 

I GGUF everything!

[corz]

Just now, corz said:

RTX 3060 12GB. 

I GGUF everything!

Also, llama.cpp is easier than you probably think. Rather than explain, see here.