Instructions to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF", dtype="auto")

llama-cpp-python

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF",
	filename="Bootes-Qwen3_Coder-Reasoning.i1-IQ1_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
```

Unsloth Studio

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF to start chatting

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Bootes-Qwen3_Coder-Reasoning-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Bootes-Qwen3_Coder-Reasoning-i1-GGUF-Q4_K_M

List all available models

lemonade list

mradermacher commited on May 13, 2025

Commit

aeddd06

verified ·

1 Parent(s): 45ac55f

uploaded from marco

Browse files

Files changed (25) hide show

.gitattributes +24 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ1_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ1_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ2_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ2_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XS.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XXS.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ3_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ3_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XS.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XXS.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ4_NL.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-IQ4_XS.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q2_K.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q2_K_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_L.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q4_0.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q4_1.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_M.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_S.gguf +3 -0
Bootes-Qwen3_Coder-Reasoning.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 imatrix.dat filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 imatrix.dat filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Bootes-Qwen3_Coder-Reasoning.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

Bootes-Qwen3_Coder-Reasoning.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:239ad820baddfa495caf1abc56928d603cd5b6660c6d88a16877199092e19569
+size 1127019776

Bootes-Qwen3_Coder-Reasoning.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8cb74ffcd3e69372c548981d759a1df8d20d5c19528294f1960d3942d29dba40
+size 1055257856

Bootes-Qwen3_Coder-Reasoning.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:437c2f1dd8ea0d847fa74f7e5b7d114a7359665808d33f568050acadf0d641d9
+size 1512985856

Bootes-Qwen3_Coder-Reasoning.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1ef20d87ff758d457e6917e8cb31200f94a16ab3a2214b3d79f40d20328914
+size 1417303296

Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf35403ddade44a125e8f49849dfa1d04b7603ae5f7b736ddc081ffc4ece16f4
+size 1354102016

Bootes-Qwen3_Coder-Reasoning.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16b719998b1c607ad41f1a473f5be4e18cddea630d1d2619d0efafe9e3d8406c
+size 1246622976

Bootes-Qwen3_Coder-Reasoning.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9761ef6f430f9d50a224500a84441cdd0f7b8a8464eed32140b990998001d171
+size 1962898176

Bootes-Qwen3_Coder-Reasoning.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4eb4c6f2030c8ffb849c551bf1602533db5aeaa914701d04d1a1bc148eb92ed4
+size 1899533056

Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ed91861e2c59f8e1f2246d022b2a7b5d855eb5443a385e06ef62e7ffec2add6
+size 1814377216

Bootes-Qwen3_Coder-Reasoning.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b3a30066ab4c161a3a25010d56cce9ab222527c31db02f70e10294be9dff8c95
+size 1670190336

Bootes-Qwen3_Coder-Reasoning.i1-IQ4_NL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c26ad7ed010c5a2e0e00f1a352feecf536f5e96a372a0d3db1c4b84f59797a36
+size 2381345536

Bootes-Qwen3_Coder-Reasoning.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9f96ae302ea4f3221fb796a82e82d9fe98ffcf46773fc67f878eef76d2101a05
+size 2270753536

Bootes-Qwen3_Coder-Reasoning.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:35121a8615d34cf0850476f9727d3a0ac75428ef15d26d4ed0cb21aa70864306
+size 1669501696

Bootes-Qwen3_Coder-Reasoning.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16f85bf5ac29c16f1a4bd03cc7177867c81fa7397362256ca3c3dd2138d716ad
+size 1563456256

Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5ffba4af1ca1727224d1147a2d5d9d32c1a61c74ece595bd64b12d14b0e69e83
+size 2239787776

Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16c305566dcb3b6acecfcc66e2109b468701c618cc43e8edf18bb9fd2b18ca1a
+size 2075620096

Bootes-Qwen3_Coder-Reasoning.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:28a74d4ddf8dbc606af3e66be70a759edb7c70361d7d56fe42d9d2cc00b79ab0
+size 1886999296

Bootes-Qwen3_Coder-Reasoning.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e49848fbf8194ee4a4b1666e83c8dbbedc4222260de4c59b889327b1a8c46727
+size 2375774976

Bootes-Qwen3_Coder-Reasoning.i1-Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2c2de4f21cf4f13ad1267deb8bde3375240f53c56490f93324b7ae344aebc947
+size 2596631296

Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:14068160992854d30ade26c84e006ccc4b9accdfbdf0f5da69693528d6e904ea
+size 2497282816

Bootes-Qwen3_Coder-Reasoning.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:80fd79e5c89a2dc1a876dbcd483f4d9aeb895cc8b63649f8477640124eb8bf12
+size 2383311616

Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5a987474861e1b1167933582d64a612683776b2291a4cbecb3bd5840a7bf05a3
+size 2889515776

Bootes-Qwen3_Coder-Reasoning.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7a604490583374ae499a3d3e78dd34290471ed37f5fd98b8bfc6046682e193db
+size 2823713536

Bootes-Qwen3_Coder-Reasoning.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e20a87b7223de3b97e94e7d3f6856a9647869ccc543d290c75339789b21535d3
+size 3306263296