Mini toolsDefinition
An ML architecture using multiple specialized sub-models (experts) and a gating mechanism to route inputs.
Detailed Explanation
A machine learning architecture, especially for large models, that uses multiple specialized sub-models ('experts') and a gating mechanism to route inputs to the most relevant expert(s), improving efficiency and performance.
Use Cases
Large language models (LLMs), sparse activation models, improving computational efficiency, scaling model capacity, handling diverse datasets.
