MoritzMo123/kfac-svd_uniform_llama-3-8b_0.6
Text Generation • 5B • Updated • 34
LEMS & KFAC-SVD Compression
This collection hosts the compressed models evaluated in our paper: "Layer-wise Error Modeling Search (LEMS) and KFAC-SVD".
Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.7
Text Generation • 6B • Updated • 298Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.8
Text Generation • 7B • Updated • 33Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.9
Text Generation • 7B • Updated • 35Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.6
Text Generation • 5B • Updated • 34Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.7
Text Generation • 6B • Updated • 38Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.8
Text Generation • 7B • Updated • 34Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.9
Text Generation • 7B • Updated • 36Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.6
Text Generation • 5B • Updated • 33Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.7
Text Generation • 6B • Updated • 32Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.8
Text Generation • 7B • Updated • 33Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.9
Text Generation • 7B • Updated • 32Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.6
Text Generation • 5B • Updated • 27Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.7
Text Generation • 6B • Updated • 30Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.8
Text Generation • 7B • Updated • 33Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.9
Text Generation • 7B • Updated • 30Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.6
Text Generation • 4B • Updated • 28Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.7
Text Generation • 5B • Updated • 34Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.8
Text Generation • 6B • Updated • 35Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.9
Text Generation • 7B • Updated • 35Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.6
Text Generation • 4B • Updated • 28Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.7
Text Generation • 5B • Updated • 30Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.8
Text Generation • 6B • Updated • 33Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.9
Text Generation • 7B • Updated • 34Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.6
Text Generation • 4B • Updated • 44 • 1Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.8
Text Generation • 5B • Updated • 29Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.6
Text Generation • 4B • Updated • 31 • 1Note Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.8
Text Generation • 5B • Updated • 26Note Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.