If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs
Paper
•
2412.04144
•
Published
•
6
merging
-
-
Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
Paper • 2410.08371 • Published • 3 -
MERGE^3: Efficient Evolutionary Merging on Consumer-grade GPUs
Paper • 2502.10436 • Published • 1 -
Mergenetic: a Simple Evolutionary Model Merging Library
Paper • 2505.11427 • Published • 14 -
Evolutionary Optimization of Model Merging Recipes
Paper • 2403.13187 • Published • 58 -
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
Paper • 2410.10801 • Published • 3 -
SEA-LION: Southeast Asian Languages in One Network
Paper • 2504.05747 • Published -
What Matters for Model Merging at Scale?
Paper • 2410.03617 • Published • 9 -
Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance
Paper • 2511.13254 • Published • 134 -
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 7 -
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation
Paper • 2502.17159 • Published • 2 -
Unconstrained Model Merging for Enhanced LLM Reasoning
Paper • 2410.13699 • Published • 1 -
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
Paper • 2408.03092 • Published • 1 -
Merging Smarter, Generalizing Better: Enhancing Model Merging on OOD Data
Paper • 2506.09093 • Published -
Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent
Paper • 2501.01230 • Published -
Realistic Evaluation of Model Merging for Compositional Generalization
Paper • 2409.18314 • Published -
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 15 -
Model Merging with Functional Dual Anchors
Paper • 2510.21223 • Published • 12 -
Activation-Informed Merging of Large Language Models
Paper • 2502.02421 • Published • 6 -
Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunking
Paper • 2509.25712 • Published • 1 -
ATM: Improving Model Merging by Alternating Tuning and Merging
Paper • 2411.03055 • Published • 1 -
MergeBench: A Benchmark for Merging Domain-Specialized LLMs
Paper • 2505.10833 • Published -
Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
Paper • 2503.20641 • Published • 10 -
REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
Paper • 2510.13999 • Published • 5