DeepSeek-V3TechnicalReportDeepSeek-AIresearch@deepseek.comAbstractWepresentDeepSeek-V3,astrongMixture-of-Experts(MoE)languagemodelwith671Btotalparameterswith37Bactivatedforeachtoken.Toachieveefficientinferenceandcost-effectivetraining,DeepSeek-V3adoptsMulti-headLatentAttention(MLA)andDeepSeekMoEarchitec-tures,whichwerethoroughlyvalidatedinDeepSeek-V2.Furthermore,DeepSeek-V3pioneersanauxiliary-loss-freestrategyforloadbalancingandsetsamulti-tokenpredictiontrainingobjectiveforstrongerperformance.Wepre-trainDeepSeek-V3on14.8trilliondiverseandhigh-qualitytokens,followedbySupervisedFine-TuningandReinforcementLearningstagestofullyharnessitscapabilities.Comprehen...