Falcon 3 fashions now out there in Amazon SageMaker JumpStart

At present, we’re excited to announce that the Falcon 3 household of fashions from TII can be found in Amazon SageMaker JumpStart. On this put up, we discover how one can deploy this mannequin effectively on Amazon SageMaker AI.

Overview of the Falcon 3 household of fashions

The Falcon 3 household, developed by Expertise Innovation Institute (TII) in Abu Dhabi, represents a major development in open supply language fashions. This assortment consists of 5 base fashions starting from 1 billion to 10 billion parameters, with a deal with enhancing science, math, and coding capabilities. The household consists of Falcon3-1B-Base, Falcon3-3B-Base, Falcon3-Mamba-7B-Base, Falcon3-7B-Base, and Falcon3-10B-Base together with their instruct variants.

These fashions showcase improvements resembling environment friendly pre-training strategies, scaling for improved reasoning, and data distillation for higher efficiency in smaller fashions. Notably, the Falcon3-10B-Base mannequin achieves state-of-the-art efficiency for fashions beneath 13 billion parameters in zero-shot and few-shot duties. The Falcon 3 household additionally consists of varied fine-tuned variations like Instruct fashions and helps completely different quantization codecs, making them versatile for a variety of purposes.

At present, SageMaker JumpStart gives the bottom variations of Falcon3-3B, Falcon3-7B, and Falcon3-10B, together with their corresponding instruct variants, in addition to Falcon3-1B-Instruct.

Get began with SageMaker JumpStart

SageMaker JumpStart is a machine studying (ML) hub that may assist speed up your ML journey. With SageMaker JumpStart, you may consider, evaluate, and choose pre-trained basis fashions (FMs), together with Falcon 3 fashions. These fashions are absolutely customizable on your use case along with your information.

Deploying a Falcon 3 mannequin by SageMaker JumpStart gives two handy approaches: utilizing the intuitive SageMaker JumpStart UI or implementing programmatically by the SageMaker Python SDK. Let’s discover each strategies that can assist you select the strategy that most closely fits your wants.

Deploy Falcon 3 utilizing the SageMaker JumpStart UI

Full the next steps to deploy Falcon 3 by the JumpStart UI:

1. To entry SageMaker JumpStart, use one of many following strategies:
   1. In Amazon SageMaker Unified Studio, on the Construct menu, select JumpStart fashions beneath Mannequin growth.
      
   2. Alternatively, in Amazon SageMaker Studio, select JumpStart within the navigation pane.
      

2. Seek for Falcon3-10B-Base within the mannequin browser.
   
3. Select the mannequin and select Deploy.
   
4. For Occasion kind, both use the default occasion or select a unique occasion.
5. Select Deploy.
   After a while, the endpoint standing will present as InService and it is possible for you to to run inference in opposition to it.
   

Deploy Falcon 3 programmatically utilizing the SageMaker Python SDK

For groups seeking to automate deployment or combine with present MLOps pipelines, you should utilize the SageMaker Python SDK:

from sagemaker.serve.builder.model_builder import ModelBuilder
from sagemaker.serve.builder.schema_builder import SchemaBuilder
from sagemaker.jumpstart.mannequin import ModelAccessConfig
from sagemaker.session import Session
import logging

sagemaker_session = Session()

artifacts_bucket_name = sagemaker_session.default_bucket()
execution_role_arn = sagemaker_session.get_caller_identity_arn()


js_model_id = "huggingface-llm-falcon-3-10B-base"

gpu_instance_type = "ml.g5.12xlarge"  

response = "Hi there, I am a language mannequin, and I am right here that can assist you along with your English."

sample_input = {
    "inputs": "Hi there, I am a language mannequin,",
    "parameters": {"max_new_tokens": 128, "top_p": 0.9, "temperature": 0.6},
}

sample_output = [{"generated_text": response}]

schema_builder = SchemaBuilder(sample_input, sample_output)

model_builder = ModelBuilder(
    mannequin=js_model_id,
    schema_builder=schema_builder,
    sagemaker_session=sagemaker_session,
    role_arn=execution_role_arn,
    log_level=logging.ERROR
)

mannequin= model_builder.construct()

predictor = mannequin.deploy(model_access_configs={js_model_id:ModelAccessConfig(accept_eula=True)}, accept_eula=True)

Run inference on the predictor:

predictor.predict(sample_input)

If you wish to arrange the power to scale all the way down to zero after deployment, check with Unlock value financial savings with the brand new scale all the way down to zero characteristic in SageMaker Inference.

Clear up

To wash up the mannequin and endpoint, use the next code:

predictor.delete_model()
predictor.delete_endpoint()

Conclusion

On this put up, we explored how SageMaker JumpStart empowers information scientists and ML engineers to find, entry, and run a variety of pre-trained FMs for inference, together with the Falcon 3 household of fashions. Go to SageMaker JumpStart in SageMaker Studio now to get began. For extra data, check with SageMaker JumpStart pretrained fashions, Amazon SageMaker JumpStart Basis Fashions, and Getting began with Amazon SageMaker JumpStart.

In regards to the authors

Niithiyn Vijeaswaran is a Generative AI Specialist Options Architect with the Third-Occasion Mannequin Science group at AWS. His space of focus is generative AI and AWS AI Accelerators. He holds a Bachelor’s diploma in Laptop Science and Bioinformatics.

Marc Karp is an ML Architect with the Amazon SageMaker Service group. He focuses on serving to clients design, deploy, and handle ML workloads at scale. In his spare time, he enjoys touring and exploring new locations.

Raghu Ramesha is a Senior ML Options Architect with the Amazon SageMaker Service group. He focuses on serving to clients construct, deploy, and migrate ML manufacturing workloads to SageMaker at scale. He makes a speciality of machine studying, AI, and laptop imaginative and prescient domains, and holds a grasp’s diploma in Laptop Science from UT Dallas. In his free time, he enjoys touring and pictures.

Banu Nagasundaram leads product, engineering, and strategic partnerships for SageMaker JumpStart, SageMaker’s machine studying and GenAI hub. She is captivated with constructing options that assist clients speed up their AI journey and unlock enterprise worth.