Meta’s Muse Spark Open now out there in Amazon SageMaker JumpStart

Muse Spark Open, Meta’s coding-optimized open-weight mannequin, is now out there in Amazon SageMaker JumpStart, the machine studying (ML) hub of Amazon SageMaker AI. The mannequin was educated on actual developer workflows, resembling multi-file refactors and lengthy debugging classes, and it helps a context window of 1 million tokens. You may deploy it to a SageMaker AI endpoint in your individual AWS account in a couple of steps, with out managing the serving infrastructure your self. SageMaker JumpStart presents two variants: BF16 (Muse-Spark-Open-816B-A42B) and NVFP4 quantized (Muse-Spark-Open-816B-A42B-NVFP4).

On this put up, we introduce Muse Spark Open and present you tips on how to deploy the mannequin with Amazon SageMaker Studio and the SageMaker Python SDK, ship it a primary inference request, and clear up.

Overview of Muse Spark Open

Meta describes Muse Spark Open as a coding-optimized mannequin educated for agentic coding workflows, with a give attention to first-attempt accuracy, device calling, and understanding of enormous codebases. It’s a reasoning mannequin, and it accepts textual content, pictures, paperwork, audio, and audio-video as enter and returns textual content.

Muse Spark Open makes use of a mixture-of-experts (MoE) structure with 816 billion whole parameters, of which 42 billion are energetic for every token. The mannequin holds the capability of a really massive community, whereas the compute it spends on every token is nearer to that of a a lot smaller dense mannequin. Meta releases the weights underneath the Muse Mannequin License, which incorporates the Muse Spark Utilization Coverage.

The next desk summarizes the mannequin.

Constructed for long-running coding brokers

Meta educated Muse Spark Open on the way in which builders work in actual repositories: code technology, debugging, multi-file refactors, and duties that run for a very long time. With the 1 million-token context window, the mannequin can soak up an entire venture, together with its dependencies, legacy code, and hundreds of recordsdata. You don’t want to separate a job into smaller items to suit the context.

Meta additionally designed Muse Spark Open as an agentic mannequin from the beginning. Meta says the mannequin plans, acts, observes the end result, and corrects itself over lengthy horizons, somewhat than making single device calls. As a reasoning mannequin, it really works via a job earlier than it acts, and the SageMaker AI endpoint returns that reasoning alongside the reply, so you’ll be able to comply with what an agent is doing and why. Meta educated the mannequin to browse the online, coordinate subagents for work that may be cut up up, and function computer systems throughout desktop, cellular, and browser environments. These are capabilities of the mannequin. On SageMaker AI, your software or agent framework supplies the instruments, browser, and computer-use atmosphere that the mannequin calls. Past code, Meta studies that Muse Spark Open causes throughout pictures, paperwork, audio, and video.

Two properties of the mannequin form the way you serve it. First, sparse activation retains per-token compute low, however serving nonetheless requires sufficient GPU reminiscence to carry all 816 billion parameters, so the occasion selection comes all the way down to reminiscence capability. Second, the NVFP4 variant shops weights in NVIDIA’s 4-bit floating level format, which wants roughly 1 / 4 of the reminiscence of the BF16 weights. On SageMaker AI, that distinction decides the occasion: the NVFP4 variant runs on ml.p5e.48xlarge, and the BF16 variant runs on ml.p6-b300.48xlarge. That makes NVFP4 the variant to strive first if you measurement for value. Consider it by yourself workloads to substantiate the standard matches your use case.

With open weights on SageMaker AI, you additionally management the deployment. The endpoint runs in your AWS account, so your supply code and prompts are processed on infrastructure that you simply configure and management. You may connect the endpoint to your individual Amazon Digital Non-public Cloud (Amazon VPC), encrypt its storage quantity with a key from AWS Key Administration Service (AWS KMS), and activate community isolation for the container. You additionally select the occasion sort and scaling limits, and also you pin the mannequin model, so the mannequin solely modifications if you redeploy.

Use instances

You should use Muse Spark Open for a variety of purposes. Meta’s give attention to coding and agentic workflows fits it to the next use instances:

• Coding assistants over non-public repositories: Reply questions on massive codebases, refactor them, and debug them with the total venture in context, whereas the code is processed on an endpoint in your account.
• Lengthy-running coding brokers: Run brokers that plan a change, edit throughout a number of recordsdata, run checks in your individual sandbox, and proper course based mostly on the outcomes.
• Multimodal engineering workflows: Flip screenshots, design paperwork, or a recorded bug report with audio and video right into a replica plan or a draft repair.

Getting began with SageMaker JumpStart

You may deploy Muse Spark Open from SageMaker Studio or with the SageMaker Python SDK. Every path creates a real-time SageMaker AI endpoint in your account.

Stipulations

Earlier than you begin, just remember to have the next:

• An AWS account with permissions to create SageMaker AI fashions and endpoints.
• An AWS Identification and Entry Administration (IAM) execution function for SageMaker AI. For this walkthrough, the AmazonSageMakerFullAccess managed coverage is adequate. For manufacturing, scope the function to the actions and assets that your deployment wants. For particulars, seek advice from Tips on how to use SageMaker AI execution roles.
• A SageMaker AI area with entry to SageMaker Studio, for the Studio path. For setup directions, seek advice from Use fast setup for Amazon SageMaker AI.
• Service quota for the endpoint occasion sort: ml.p5e.48xlarge for endpoint utilization for the NVFP4 variant, or ml.p6-b300.48xlarge for endpoint utilization for the BF16 variant. To boost a quota, seek advice from Requesting a quota enhance.
• For the SDK path: Python 3.10 or later and SageMaker Python SDK v3 (pip set up --upgrade sagemaker).
• An AWS Area the place Muse Spark Open is offered in SageMaker JumpStart. At launch, Muse Spark Open is offered in US East (N. Virginia) and US East (Ohio). To test availability in different Areas, open SageMaker JumpStart in your Area.

The situations that host Muse Spark Open are multi-GPU situations, and also you pay for an endpoint for so long as it runs, whether or not or not it serves requests. Delete the endpoint if you end, as described within the clean-up part of this put up. For charges, seek advice from Amazon SageMaker AI pricing.

Deploy utilizing SageMaker Studio

The SageMaker Studio path fits a primary analysis: you discover the mannequin, overview its license, and deploy it from one mannequin element card.

To search out Muse Spark Open in SageMaker JumpStart

1. Open SageMaker Studio.
2. Within the navigation pane, select JumpStart.
3. Within the search field, enter Muse Spark Open.
4. Select Muse-Spark-Open-816B-A42B for the BF16 variant or Muse-Spark-Open-816B-A42B-NVFP4 for the NVFP4 variant to open its mannequin element card.

Determine 1 reveals the mannequin element card, which lists the mannequin description, license, and the Deploy choice.

Determine 1: The Muse Spark Open mannequin element card in SageMaker JumpStart

To deploy the mannequin

1. On the mannequin element card, select Deploy.
2. Muse Spark Open is a gated mannequin, so SageMaker Studio reveals its end-user license settlement (EULA). Assessment the settlement and settle for it to proceed.
3. For Endpoint title, maintain the generated title or enter your individual.
4. For Occasion sort, choose ml.p5e.48xlarge for NVFP4 or ml.p6-b300.48xlarge for BF16. SageMaker Studio lists solely the occasion sorts that the mannequin helps.
5. For Preliminary occasion depend, enter 1.
6. (Optionally available) Beneath Superior choices, choose a VPC and an AWS KMS key, or activate community isolation.
7. Select Deploy.

Determine 2 reveals the deployment settings for the NVFP4 variant.

Determine 2: Deployment settings for Muse Spark Open

Deployment sometimes takes 15–20 minutes whereas SageMaker AI provisions the occasion and hundreds the weights. When the endpoint standing modifications to InService, the endpoint is prepared for requests, as proven in Determine 3.

Determine 3: The Muse Spark Open endpoint in service

Deploy utilizing the SageMaker Python SDK

SageMaker JumpStart identifies every variant by a mannequin ID:

• BF16: meta-vlm-muse-spark-open-816b-a42b.
• NVFP4: meta-vlm-muse-spark-open-816b-a42b-nvfp4.

The next code deploys the NVFP4 variant with SageMaker Python SDK v3. You may run it in a SageMaker Studio pocket book, which makes use of your area’s execution function. In every other Python atmosphere with AWS credentials, go your SageMaker AI execution function with the role_arn parameter. Muse Spark Open is a gated mannequin, so you could set accept_eula=True to deploy it. To pin a particular mannequin model, add model_version to JumpStartConfig.

from sagemaker.core.jumpstart.configs import JumpStartConfig
from sagemaker.serve import ModelBuilder
from sagemaker.practice.configs import Compute

# NVFP4 variant; for BF16, use "meta-vlm-muse-spark-open-816b-a42b" with ml.p6-b300.48xlarge
model_id = "meta-vlm-muse-spark-open-816b-a42b-nvfp4"

jumpstart_config = JumpStartConfig(
    model_id=model_id,
    accept_eula=True,  # Confirms that you simply settle for Meta's license phrases
)

model_builder = ModelBuilder.from_jumpstart_config(
    jumpstart_config=jumpstart_config,
    compute=Compute(instance_type="ml.p5e.48xlarge"),
    # role_arn="",  # Required exterior SageMaker Studio
)

mannequin = model_builder.construct(model_name="muse-spark-open-nvfp4")
endpoint = model_builder.deploy(endpoint_name="muse-spark-open-nvfp4")

Run inference

When the endpoint is in service, ship it a request with the invoke methodology. The endpoint accepts an OpenAI-compatible chat payload with a messages record and returns a chat completion. The ultimate reply is in message.content material, the mannequin’s reasoning is in message.reasoning_content, and utilization.reasoning_tokens studies what number of tokens the reasoning used. As a result of Muse Spark Open is a reasoning mannequin, depart sufficient room in max_tokens for its reasoning and the ultimate reply.

import json

payload = {
    "messages": [
        {
            "role": "user",
            "content": "Write a Python function that validates ISO 8601 dates, and add pytest unit tests for it.",
        }
    ],
    "max_tokens": 4096,
}

response = endpoint.invoke(
    physique=json.dumps(payload),
    content_type="software/json",
)

end result = json.hundreds(response.physique.learn().decode("utf-8"))
message = end result["choices"][0]["message"]
print("Reasoning:", message.get("reasoning_content"))
print("Reply:", message["content"])

The endpoint additionally helps streaming. To obtain tokens as they’re generated, add "stream": true to the request physique and name endpoint.invoke_with_response_stream as a substitute of invoke.

Clear up

To cease incurring fees, delete the endpoint if you not want it. In case you deployed with the SDK, delete the mannequin, the endpoint, and the endpoint configuration:

from sagemaker.core.assets import EndpointConfig

endpoint_config = EndpointConfig.get(endpoint_config_name=endpoint.endpoint_name)
mannequin.delete()
endpoint.delete()
endpoint_config.delete()

In case you deployed from SageMaker Studio, delete the endpoint within the SageMaker AI console: select Inference, then Endpoints, choose the endpoint, and select Actions, Delete.

Conclusion

On this put up, we confirmed tips on how to deploy Muse Spark Open, Meta’s coding-optimized open-weight mannequin, from SageMaker JumpStart to an endpoint in your individual AWS account. You used SageMaker Studio or the SageMaker Python SDK and despatched the mannequin a primary inference request. Benchmark each variants by yourself repositories earlier than you select an occasion for manufacturing.

To get began, seek for Muse Spark Open in SageMaker JumpStart in SageMaker Studio. For extra deployment choices, resembling inference elements and optimized deployment configurations, seek advice from JumpStart basis mannequin utilization within the Amazon SageMaker AI Developer Information. Earlier than you deploy, overview the Muse Mannequin License, which SageMaker JumpStart reveals if you deploy the mannequin. For assist planning a manufacturing deployment, contact AWS.

Concerning the authors