Running Locally on Apple Hardware
The collaboration focuses on enabling the model to run on Apple’s M5 Ultra Mac Studios using Apple’s MLX machine learning framework. Meta says the model can run on a desktop setup using two Apple M5 Ultra Mac Studios, each configured with up to 512GB of unified memory. Together, these two machines provide 1TB of memory, allowing the model to operate without cloud infrastructure.
Apple published technical details on how to distribute the model across multiple Mac Studios using high-speed communication. Apple CEO John Ternus commented on the efficiency of the setup, stating, "Amazing to run a model of Muse Spark Open’s caliber and scale this efficiently, right from your desk. Mac Studio keeps pushing the boundaries of on-device AI!"
The local execution offers potential benefits in data security and cost savings by eliminating the need to offload tasks to cloud servers. However, the hardware requirements are significant. A single Mac Studio with the M5 Ultra chip and 256GB of memory costs at least $10,799, and a 512GB version is scheduled for release later this month. A four-machine configuration can cost over $40,000.