1 essay on this topic.
How mixture-of-experts models work: routers, top-k experts, load balancing, what experts really learn, and why total and active parameters differ.
