TII Falcon-H1 fashions now obtainable on Amazon Bedrock Market and Amazon SageMaker JumpStart

This publish was co-authored with Jingwei Zuo from TII.

We’re excited to announce the supply of the Expertise Innovation Institute (TII)’s Falcon-H1 fashions on Amazon Bedrock Market and Amazon SageMaker JumpStart. With this launch, builders and knowledge scientists can now use six instruction-tuned Falcon-H1 fashions (0.5B, 1.5B, 1.5B-Deep, 3B, 7B, and 34B) on AWS, and have entry to a complete suite of hybrid structure fashions that mix conventional consideration mechanisms with State Area Fashions (SSMs) to ship distinctive efficiency with unprecedented effectivity.

On this publish, we current an summary of Falcon-H1 capabilities and present get began with TII’s Falcon-H1 fashions on each Amazon Bedrock Market and SageMaker JumpStart.

Overview of TII and AWS collaboration

TII is a number one analysis institute based mostly in Abu Dhabi. As a part of UAE’s Superior Expertise Analysis Council (ATRC), TII focuses on superior expertise analysis and growth throughout AI, quantum computing, autonomous robotics, cryptography, and extra. TII employs worldwide groups of scientists, researchers, and engineers in an open and agile surroundings, aiming to drive technological innovation and place Abu Dhabi and the UAE as a worldwide analysis and growth hub in alignment with the UAE Nationwide Technique for Synthetic Intelligence 2031.

TII and Amazon Internet Companies (AWS) are collaborating to increase entry to made-in-the-UAE AI fashions throughout the globe. By combining TII’s technical experience in constructing massive language fashions (LLMs) with AWS Cloud-based AI and machine studying (ML) companies, professionals worldwide can now construct and scale generative AI functions utilizing the Falcon-H1 sequence of fashions.

About Falcon-H1 fashions

The Falcon-H1 structure implements a parallel hybrid design, utilizing parts from Mamba and Transformer architectures to mix the sooner inference and decrease reminiscence footprint of SSMs like Mamba with the effectiveness of Transformers’ consideration mechanism in understanding context and enhanced generalization capabilities. The Falcon-H1 structure scales throughout a number of configurations starting from 0.5–34 billion parameters and offers native assist for 18 languages. In response to TII, the Falcon-H1 household demonstrates notable effectivity with printed metrics indicating that smaller mannequin variants obtain efficiency parity with bigger fashions. A number of the advantages of Falcon-H1 sequence embrace:

• Efficiency – The hybrid attention-SSM mannequin has optimized parameters with adjustable ratios between consideration and SSM heads, resulting in sooner inference, decrease reminiscence utilization, and robust generalization capabilities. In response to TII benchmarks printed in Falcon-H1’s technical weblog publish and technical report, Falcon-H1 fashions exhibit superior efficiency throughout a number of scales in opposition to different main Transformer fashions of comparable or bigger scales. For instance, Falcon-H1-0.5B delivers efficiency much like typical 7B fashions from 2024, and Falcon-H1-1.5B-Deep rivals lots of the present main 7B-10B fashions.
• Big selection of mannequin sizes – The Falcon-H1 sequence consists of six sizes: 0.5B, 1.5B, 1.5B-Deep, 3B, 7B, and 34B, with each base and instruction-tuned variants. The Instruct fashions at the moment are obtainable in Amazon Bedrock Market and SageMaker JumpStart.
• Multilingual by design – The fashions assist 18 languages natively (Arabic, Czech, German, English, Spanish, French, Hindi, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Romanian, Russian, Swedish, Urdu, and Chinese language) and may scale to over 100 languages in accordance with TII, because of a multilingual tokenizer educated on numerous language datasets.
• As much as 256,000 context size – The Falcon-H1 sequence allows functions in long-document processing, multi-turn dialogue, and long-range reasoning, exhibiting a definite benefit over rivals in sensible long-context functions like Retrieval Augmented Technology (RAG).
• Strong knowledge and coaching technique – Coaching of Falcon-H1 fashions employs an revolutionary method that introduces complicated knowledge early on, opposite to conventional curriculum studying. It additionally implements strategic knowledge reuse based mostly on cautious memorization window evaluation. Moreover, the coaching course of scales easily throughout mannequin sizes by means of a personalized Maximal Replace Parametrization (µP) recipe, particularly tailored for this novel structure.
• Balanced efficiency in science and knowledge-intensive domains – Via a rigorously designed knowledge combination and common evaluations throughout coaching, the mannequin achieves robust basic capabilities and broad world data whereas minimizing unintended specialization or domain-specific biases.

Consistent with their mission to foster AI accessibility and collaboration, TII have launched Falcon-H1 fashions beneath the Falcon LLM license. It gives the next advantages:

• Open supply nature and accessibility
• Multi-language capabilities
• Value-effectiveness in comparison with proprietary fashions
• Power-efficiency

About Amazon Bedrock Market and SageMaker JumpStart

Amazon Bedrock Market gives entry to over 100 well-liked, rising, specialised, and domain-specific fashions, so yow will discover the very best proprietary and publicly obtainable fashions to your use case based mostly on components corresponding to accuracy, flexibility, and price. On Amazon Bedrock Market you may uncover fashions in a single place and entry them by means of unified and safe Amazon Bedrock APIs. You can even choose your required variety of cases and the occasion sort to fulfill the calls for of your workload and optimize your prices.

SageMaker JumpStart helps you shortly get began with machine studying. It offers entry to state-of-the-art mannequin architectures, corresponding to language fashions, pc imaginative and prescient fashions, and extra, with out having to construct them from scratch. With SageMaker JumpStart you may deploy fashions in a safe surroundings by provisioning them on SageMaker inference cases and isolating them inside your digital non-public cloud (VPC). You can even use Amazon SageMaker AI to additional customise and fine-tune the fashions and streamline the complete mannequin deployment course of.

Answer overview

This publish demonstrates deploy a Falcon-H1 mannequin utilizing each Amazon Bedrock Market and SageMaker JumpStart. Though we use Falcon-H1-0.5B for example, you may apply these steps to different fashions within the Falcon-H1 sequence. For assist figuring out which deployment choice—Amazon Bedrock Market or SageMaker JumpStart—most closely fits your particular necessities, see Amazon Bedrock or Amazon SageMaker AI?

Deploy Falcon-H1-0.5B-Instruct with Amazon Bedrock Market

On this part, we present deploy the Falcon-H1-0.5B-Instruct mannequin in Amazon Bedrock Market.

Stipulations

To attempt the Falcon-H1-0.5B-Instruct mannequin in Amazon Bedrock Market, it’s essential to have entry to an AWS account that can comprise your AWS sources.Previous to deploying Falcon-H1-0.5B-Instruct, confirm that your AWS account has ample quota allocation for ml.g6.xlarge cases. The default quota for endpoints utilizing a number of occasion sorts and sizes is 0, so trying to deploy the mannequin with no increased quota will set off a deployment failure.

To request a quota enhance, open the AWS Service Quotas console and seek for Amazon SageMaker. Find ml.g6.xlarge for endpoint utilization and select Request quota enhance, then specify your required restrict worth. After the request is authorised, you may proceed with the deployment.

Deploy the mannequin utilizing the Amazon Bedrock Market UI

To deploy the mannequin utilizing Amazon Bedrock Market, full the next steps:

1. On the Amazon Bedrock console, beneath Uncover within the navigation pane, select Mannequin catalog.
2. Filter for Falcon-H1 because the mannequin title and select Falcon-H1-0.5B-Instruct.

The mannequin overview web page consists of details about the mannequin’s license phrases, options, setup directions, and hyperlinks to additional sources.

3. Assessment the mannequin license phrases, and if you happen to agree with the phrases, select Deploy.

4. For Endpoint title, enter an endpoint title or depart it because the default pre-populated title.
5. To attenuate prices whereas experimenting, set the Variety of cases to 1.
6. For Occasion sort, select from the record of appropriate occasion sorts. Falcon-H1-0.5B-Instruct is an environment friendly mannequin, so ml.m6.xlarge is ample for this train.

Though the default configurations are usually ample for primary wants, you may customise superior settings like VPC, service entry permissions, encryption keys, and useful resource tags. These superior settings may require adjustment for manufacturing environments to take care of compliance along with your group’s safety protocols.

7. Select Deploy.
8. A immediate asks you to remain on the web page whereas the AWS Id and Entry Administration (IAM) position is being created. In case your AWS account lacks ample quota for the chosen occasion sort, you’ll obtain an error message. On this case, confer with the previous prerequisite part to extend your quota, then attempt the deployment once more.

Whereas deployment is in progress, you may select Market mannequin deployments within the navigation pane to observe the deployment progress within the Managed deployment part. When the deployment is full, the endpoint standing will change from Creating to In Service.

Work together with the mannequin within the Amazon Bedrock Market playground

Now you can take a look at Falcon-H1 capabilities immediately within the Amazon Bedrock playground by deciding on the managed deployment and selecting Open in playground.

Now you can use the Amazon Bedrock Market playground to work together with Falcon-H1-0.5B-Instruct.

Invoke the mannequin utilizing code

On this part, we exhibit to invoke the mannequin utilizing the Amazon Bedrock Converse API.

Change the placeholder code with the endpoint’s Amazon Useful resource Title (ARN), which begins with arn:aws:sagemaker. You will discover this ARN on the endpoint particulars web page within the Managed deployments part.

import boto3
bedrock_runtime = boto3.shopper("bedrock-runtime")
endpoint_arn = "{ENDPOINT ARN}" # Change with endpoint ARN
response = bedrock_runtime.converse( modelId=endpoint_arn, messages=[{"role": "user", "content": [{"text": "What is generative AI?"}]}], inferenceConfig={"temperature": 0.1, "topP": 0.1})

print(response["output"]["message"]["content"][0]["text"])

To be taught extra in regards to the detailed steps and instance code for invoking the mannequin utilizing Amazon Bedrock APIs, confer with Submit prompts and generate response utilizing the API.

Deploy Falcon-H1-0.5B-Instruct with SageMaker JumpStart

You’ll be able to entry FMs in SageMaker JumpStart by means of Amazon SageMaker Studio, the SageMaker SDK, and the AWS Administration Console. On this walkthrough, we exhibit deploy Falcon-H1-0.5B-Instruct utilizing the SageMaker Python SDK. Seek advice from Deploy a mannequin in Studio to discover ways to deploy the mannequin by means of SageMaker Studio.

Stipulations

To deploy Falcon-H1-0.5B-Instruct with SageMaker JumpStart, it’s essential to have the next conditions:

• An AWS account that can comprise your AWS sources.
• An IAM position to entry SageMaker AI. To be taught extra about how IAM works with SageMaker AI, see Id and Entry Administration for Amazon SageMaker AI.
• Entry to SageMaker Studio with a JupyterLab area, or an interactive growth surroundings (IDE) corresponding to Visible Studio Code or PyCharm.

Deploy the mannequin programmatically utilizing the SageMaker Python SDK

Earlier than deploying Falcon-H1-0.5B-Instruct utilizing the SageMaker Python SDK, be sure you have put in the SDK and configured your AWS credentials and permissions.

The next code instance demonstrates deploy the mannequin:

import sagemakerfrom sagemaker.jumpstart.mannequin
import JumpStartModelfrom sagemaker
import Session
import boto3
import json

# Initialize SageMaker session
session = sagemaker.Session()
position = sagemaker.get_execution_role()

# Specify mannequin parameters
model_id = "huggingface-llm-falcon-h1-0-5b-instruct"
instance_type = "ml.g6.xlarge" # Select acceptable occasion based mostly in your wants

# Create and deploy the mannequin
mannequin = JumpStartModel( model_id=model_id, position=position, instance_type=instance_type, model_version="*" # Newest model)

# Deploy the mannequin
predictor = mannequin.deploy( initial_instance_count=1, accept_eula=True # Required for deploying basis fashions)

print("Endpoint title:")
print(predictor.endpoint_name)

When the earlier code phase completes efficiently, the Falcon-H1-0.5B-Instruct mannequin deployment is full and obtainable on a SageMaker endpoint. Word the endpoint title proven within the output—you’ll substitute the placeholder within the following code phase with this worth.The next code demonstrates put together the enter knowledge, make the inference API name, and course of the mannequin’s response:

import json
import boto3

session = boto3.Session() # Ensure that your AWS credentials are configured
sagemaker_runtime = session.shopper("sagemaker-runtime")

endpoint_name = "{ENDPOINT_NAME}" # Change with endpoint title from deployment output

payload = { "messages": [ { "role": "user", "content": "What is generative AI?" } ], "parameters": { "max_tokens": 256, "temperature": 0.1, "top_p": 0.1 } }

# Carry out inference
response = sagemaker_runtime.invoke_endpoint( EndpointName=endpoint_name, ContentType="utility/json", Physique=json.dumps(payload))

# Parse the response
consequence = json.masses(response["Body"].learn().decode("utf-8"))generated_text = consequence["choices"][0]["message"]["content"].strip()
print("Generated Response:")
print(generated_text)

Clear up

To keep away from ongoing expenses for AWS sources used whereas experimenting with Falcon-H1 fashions, be certain that to delete all deployed endpoints and their related sources once you’re completed. To take action, full the next steps:

1. Delete Amazon Bedrock Market sources:
   1. On the Amazon Bedrock console, select Market mannequin deployment within the navigation pane.
   2. Beneath Managed deployments, select the Falcon-H1 mannequin endpoint you deployed earlier.
   3. Select Delete and ensure the deletion if you happen to not want to make use of this endpoint in Amazon Bedrock Market.
2. Delete SageMaker endpoints:
   1. On the SageMaker AI console, within the navigation pane, select Endpoints beneath Inference.
   2. Choose the endpoint related to the Falcon-H1 fashions.
   3. Select Delete and ensure the deletion. This stops the endpoint and avoids additional compute expenses.
3. Delete SageMaker fashions:
   1. On the SageMaker AI console, select Fashions beneath Inference.
   2. Choose the mannequin related along with your endpoint and select Delete.

At all times confirm that each one endpoints are deleted after experimentation to optimize prices. Seek advice from the Amazon SageMaker documentation for extra steerage on managing sources.

Conclusion

The supply of Falcon-H1 fashions in Amazon Bedrock Market and SageMaker JumpStart helps builders, researchers, and companies construct cutting-edge generative AI functions with ease. Falcon-H1 fashions supply multilingual assist (18 languages) throughout numerous mannequin sizes (from 0.5B to 34B parameters) and assist as much as 256K context size, because of their environment friendly hybrid attention-SSM structure.

Through the use of the seamless discovery and deployment capabilities of Amazon Bedrock Market and SageMaker JumpStart, you may speed up your AI innovation whereas benefiting from the safe, scalable, and cost-effective AWS Cloud infrastructure.

We encourage you to discover the Falcon-H1 fashions in Amazon Bedrock Market or SageMaker JumpStart. You need to use these fashions in AWS Areas the place Amazon Bedrock or SageMaker JumpStart and the required occasion sorts can be found.

For additional studying, discover the AWS Machine Studying Weblog, SageMaker JumpStart GitHub repository, and Amazon Bedrock Consumer Information. Begin constructing your subsequent generative AI utility with Falcon-H1 fashions and unlock new potentialities with AWS!

Particular because of everybody who contributed to the launch: Evan Kravitz, Varun Morishetty, and Yotam Moss.

Concerning the authors

Mehran Nikoo leads the Go-to-Market technique for Amazon Bedrock and agentic AI in EMEA at AWS, the place he has been driving the event of AI methods and cloud-native options over the past 4 years. Previous to becoming a member of AWS, Mehran held management and technical positions at Trainline, McLaren, and Microsoft. He holds an MBA from Warwick Enterprise College and an MRes in Laptop Science from Birkbeck, College of London.

Mustapha Tawbi is a Senior Associate Options Architect at AWS, specializing in generative AI and ML, with 25 years of enterprise expertise expertise throughout AWS, IBM, Sopra Group, and Capgemini. He has a PhD in Laptop Science from Sorbonne and a Grasp’s diploma in Information Science from Heriot-Watt College Dubai. Mustapha leads generative AI technical collaborations with AWS companions all through the MENAT area.

Jingwei Zuo is a Lead Researcher on the Expertise Innovation Institute (TII) within the UAE, the place he leads the Falcon Foundational Fashions workforce. He obtained his PhD in 2022 from College of Paris-Saclay, the place he was awarded the Plateau de Saclay Doctoral Prize. He holds an MSc (2018) from the College of Paris-Saclay, an Engineer diploma (2017) from Sorbonne Université, and a BSc from Huazhong College of Science & Expertise.

John Liu is a Principal Product Supervisor for Amazon Bedrock at AWS. Beforehand, he served because the Head of Product for AWS Web3/Blockchain. Previous to becoming a member of AWS, John held numerous product management roles at public blockchain protocols and monetary expertise (fintech) firms for 14 years. He additionally has 9 years of portfolio administration expertise at a number of hedge funds.

Hamza MIMI is a Options Architect for companions and strategic offers within the MENAT area at AWS, the place he bridges cutting-edge expertise with impactful enterprise outcomes. With experience in AI and a ardour for sustainability, he helps organizations architect revolutionary options that drive each digital transformation and environmental duty, reworking complicated challenges into alternatives for progress and optimistic change.