PockEngine: Sparse and Efficient Fine-tuning in a Pocket
Paper
• 2310.17752 • Published
• 15
Fine-Tuning
-
-
S-LoRA: Serving Thousands of Concurrent LoRA Adapters
Paper • 2311.03285 • Published • 31 -
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Paper • 2311.06243 • Published • 21 -
Fine-tuning Language Models for Factuality
Paper • 2311.08401 • Published • 30 -
SiRA: Sparse Mixture of Low Rank Adaptation
Paper • 2311.09179 • Published • 9 -
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning
Paper • 2311.11077 • Published • 29 -
LLaMA Pro: Progressive LLaMA with Block Expansion
Paper • 2401.02415 • Published • 54 -
BitDelta: Your Fine-Tune May Only Be Worth One Bit
Paper • 2402.10193 • Published • 21 -
How to Train Data-Efficient LLMs
Paper • 2402.09668 • Published • 43 -
A Survey on Data Selection for LLM Instruction Tuning
Paper • 2402.05123 • Published • 3 -
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Paper • 2405.12130 • Published • 50 -
Towards Modular LLMs by Building and Reusing a Library of LoRAs
Paper • 2405.11157 • Published • 31 -
Trans-LoRA: towards data-free Transferable Parameter Efficient Finetuning
Paper • 2405.17258 • Published • 16 -
In-Context Editing: Learning Knowledge from Self-Induced Distributions
Paper • 2406.11194 • Published • 20 -
Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
Paper • 2406.11817 • Published • 13 -
Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models
Paper • 2409.06277 • Published • 15 -
RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Response
Paper • 2412.14922 • Published • 88 -
Transformer^2: Self-adaptive LLMs
Paper • 2501.06252 • Published • 55 -
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Paper • 2501.17161 • Published • 124