Streamline exterior entry to Amazon SageMaker MLflow utilizing a REST API proxy

Machine studying (ML) groups use MLflow to handle their ML lifecycle successfully. Amazon SageMaker MLflow supplies complete ML experiment monitoring and mannequin administration capabilities. Nonetheless, many enterprises have current infrastructure necessities that want HTTPS-based integrations quite than direct SDK utilization.

Many organizations must combine Amazon SageMaker MLflow with their established methods whereas sustaining their safety and infrastructure patterns. This integration problem impacts groups who can’t use the SDK immediately due to company safety insurance policies, community restrictions, or legacy system constraints.

On this submit, we exhibit how you can construct a safe Flask-based MLflow proxy service that gives HTTPS entry to Amazon SageMaker MLflow with out requiring the MLflow SDK. This resolution is for organizations present process cloud transformation who wish to protect their current ML workflows whereas adopting cloud-native companies.

This submit covers the next subjects:

• Implementing the MLflow proxy service for MLflow HTTPS requests.
• Configuring AWS Id and Entry Administration (IAM) authentication for safe entry.
• Managing URL pre-signing and request transformation.

After implementing this resolution, you may:

• Entry SageMaker MLflow securely via commonplace HTTPS endpoints.
• Keep compliance together with your group’s safety necessities.
• Combine MLflow with current enterprise methods.
• Scale back implementation complexity and upkeep overhead.

Answer overview

A light-weight Flask-based MLflow proxy structure supplies safe integration between enterprise methods and Amazon SageMaker MLflow via three key elements.

Element 1: Utility Load Balancer (ALB)

An AWS Utility Load Balancer serves because the upstream router, offering the next:

• Site visitors distribution for MLflow UI and REST API requests.
• Preliminary request dealing with and routing.
• Assist for customized domains and SSL termination.

Observe: This implementation makes use of ALB, however you may alternatively use different routing options similar to Nginx primarily based in your necessities.

Element 2: Flask MLflow Proxy Service

On the coronary heart of the structure, a Python-based Flask software handles the next:

• Intercepting and processing incoming HTTPS requests.
• Managing AWS authentication and request signing.
• Remodeling URLs for safe MLflow endpoint entry.
• Dealing with response routing again to shoppers.

Element 3: Amazon SageMaker MLflow

The AWS managed SageMaker MLflow service supplies the next:

• Assist for 2 MLflow deployment modes:
  • MLflow Monitoring Server – managed MLflow monitoring server.
  • MLflowApp – serverless MLflow software.
• Backend metadata retailer for monitoring info.
• Storage for mannequin recordsdata and information.

This structure supplies safe communication whereas sustaining compatibility with current enterprise methods. The proxy service acts as a bridge, reworking commonplace HTTPS requests into authenticated AWS API calls that may work together with SageMaker MLflow.

Structure and request workflow

The next diagram reveals how the Flask proxy service supplies safe communication between exterior shoppers and Amazon SageMaker MLflow.

Determine 1: Structure diagram exhibiting the Flask proxy service integration with Amazon SageMaker MLflow

The structure diagram reveals three most important elements:

• An ALB that handles incoming site visitors.
• A Flask proxy service that manages authentication and request transformation.
• Amazon SageMaker MLflow that processes ML operations.

Request workflow

Let’s discover how requests stream via this structure to offer safe MLflow entry.

When a consumer initiates an HTTPS request, it first reaches the ALB, which acts because the entry level for all incoming site visitors. The ALB then routes these requests to the MLflow proxy service.

When it receives the request, the MLflow proxy service performs a number of vital features:

• Handles authentication via AWS IAM integration.
• Transforms URLs and pre-signs them for safe entry.
• Processes the MLflow REST API endpoints as wanted.

The MLflow proxy service transforms the incoming request into an authenticated AWS request earlier than making the API name to SageMaker MLflow REST endpoints. After SageMaker MLflow processes the request, it returns a response which the MLflow proxy service processes and routes again to the unique consumer.

This workflow maintains safety whereas offering integration between enterprise methods and SageMaker MLflow.

Stipulations

To comply with this walkthrough, be sure you have the next:

• An AWS account.
• A workstation with the next instruments put in:
  • AWS Command Line Interface (AWS CLI) configured with permissions to create:
    • Amazon Digital Personal Cloud (Amazon VPC) and related networking elements.
    • Amazon Elastic Compute Cloud (Amazon EC2) cases.
    • Amazon SageMaker AI sources.
    • Amazon Easy Storage Service (Amazon S3) buckets.
    • AWS Id and Entry Administration (IAM) roles and insurance policies.
    • AWS CloudFormation stacks.
    • AWS Utility Load Balancers.
  • Node.js model 18.0.0 or later.
  • NPM.
  • AWS Cloud Growth Package (AWS CDK) CLI model 2.100.0 or later.
  • Python 3.x with pip or pip3.
• Required information:
  • Primary understanding of AWS companies and IAM permissions.
  • Familiarity with Python and Flask functions.
  • Understanding of MLflow ideas and operations.
• Price concerns:
  • This resolution creates AWS sources that may incur prices.
  • Key cost-driving sources embrace:
    • Amazon EC2 cases.
    • Utility Load Balancer.
    • Amazon SageMaker AI sources.
    • Amazon S3 storage.

For details about AWS service pricing, see AWS Pricing Calculator.

Deploy the answer

This part walks you thru deploying the answer in your AWS account and validating it. The deployment course of takes roughly 40 minutes.

Step 1: Deploy the infrastructure utilizing AWS CDK

1. Obtain the answer code and set up dependencies:

   # Clone the repository
   git clone https://github.com/aws-samples/sample-sagemaker-mlflow-rest-apis.git
   
   # Navigate to challenge listing and set up dependencies
   cd sample-sagemaker-mlflow-rest-apis
   npm ci

2. Bootstrap your surroundings for AWS CDK. Skip this step in case your AWS account and Area are already bootstrapped for AWS CDK.Bootstrap the AWS account and Area for CDK:

   npx cdk bootstrap aws:///

3. Deploy the required sources in your AWS account.The answer consists of 4 CDK stacks:
   • Networking stack — creates the VPC and networking elements.
   • SageMaker AI area stack — units up the SageMaker area.
   • SageMaker MLflow stack — deploys the MLflow monitoring server or MLflow serverless app.
   • Flask software stack — deploys the MLflow proxy service.

   Deploy all of the stacks with one of many following instructions.

   For monitoring server primarily based deployment:

   npx cdk deploy --all --require-approval=by no means -c mlflowType=monitoring

   For serverless app primarily based deployment:

   npx cdk deploy --all --require-approval=by no means -c mlflowType=serverless

Step 2: Set up and configure the Flask MLflow proxy service

1. Connect with the EC2 occasion:
   1. Observe the Amazon EC2 occasion ID from the CDK output or from the sagemaker-infra-flaskapp-{mlflowType} AWS CloudFormation stack output part.
   2. Use AWS Methods Supervisor Session Supervisor to attach. Comply with the Session Supervisor connection information.
2. Set up Python 3.13 and dependencies.Set up Python packages:

   # Swap to root consumer
   sudo su -
   cd /root
   
   # Set up Python and dependencies
   chmod +x install_python13.sh
   ./install_python13.sh

   Observe: This script is designed for Ubuntu-based methods. For different Linux distributions, set up Python 3.12+, PIP3, and Virtualenv utilizing your system’s package deal supervisor.

3. Set up and begin the MLflow proxy service:

   chmod +x setup_mlflow_proxy_app.sh
   ./setup_mlflow_proxy_app.sh

4. Test the Flask MLflow proxy service standing:

   systemctl standing mlflowproxy

   Observe: If the service isn’t operating, examine logs with the next command:

   journalctl -u mlflowproxy

Step 3: Validate MLflow REST API entry

This part demonstrates how you can work together with MLflow REST APIs via the ALB.

Observe: These examples use the HTTP (unsecured) protocol. For manufacturing environments, we advocate HTTPS. We use curl to make the API requests on this submit, however you need to use any device you favor. The offered curl instructions work identically for each monitoring server and serverless modes; the proxy service handles the variations transparently.

1. Get your ALB DNS identify by operating the next command in your workstation:

   aws cloudformation describe-stacks --stack-name sagemaker-infra-flaskapp-{mlflowType} --query 'Stacks[0].Outputs[?OutputKey==`ALBUrl`].OutputValue' --output textual content

2. Check MLflow API endpoints by operating the next instructions in your workstation. Exchange , , , and with applicable values.
   1. Create an experiment:

      curl -X POST http:///ajax-api/2.0/mlflow/experiments/create -H "Content material-Kind: software/json" -d '{"identify": "mlflow-experiment"}'

   2. Search experiments:

      curl -X POST http:///ajax-api/2.0/mlflow/experiments/search -H "Content material-Kind: software/json" -d '{"max_results": 5}'

   3. Get an experiment:

      curl -X GET 'http:///ajax-api/2.0/mlflow/experiments/get?experiment_id=0'

   4. Create a run inside an experiment:

      curl -X POST http:///ajax-api/2.0/mlflow/runs/create -H "Content material-Kind: software/json" -d '{"experiment_id": , "run_name": ""}'

   5. Listing artifacts from a run:

      curl -X GET "http:///ajax-api/2.0/mlflow/artifacts/listing?run_id="

   6. Set a tag on a run:

      curl -X POST "http:///ajax-api/2.0/mlflow/runs/set-tag" -H "Content material-Kind: software/json" -d '{"run_id": "", "key": "model_type","worth": "api-test"}'

   7. Delete a run:

      curl -X POST http:///ajax-api/2.0/mlflow/runs/delete -H "Content material-Kind: software/json" -d '{"run_id": ""}'

   Observe: You too can open the MLflow UI and look at the adjustments you make utilizing the previous curl instructions. For directions on launching the MLflow UI, see Launch the MLflow UI utilizing a presigned URL.

Cleanup

To keep away from ongoing expenses and take away the sources created by this resolution, comply with these cleanup steps:

1. Delete CDK-managed sources.Navigate to the foundation listing of the cloned repository in your workstation and run the next.For monitoring server primarily based deployment:

   npx cdk destroy --all -c mlflowType=monitoring

   For serverless app primarily based deployment:

   npx cdk destroy --all -c mlflowType=serverless

   Observe: The networking and SageMaker area stacks are shared throughout each deployment modes. AWS CDK solely deletes them when the final MLflow or Flask app stack pair is eliminated.

2. Guide useful resource cleanup. Some sources may require guide deletion due to retention insurance policies or dependencies:
   1. Amazon S3 buckets:
      1. Navigate to the Amazon S3 console.
      2. Establish the buckets created by this resolution.
      3. Empty every bucket and delete it.
   2. Amazon CloudWatch log teams:
      1. Within the CloudWatch console, discover the log teams related to this resolution.
      2. Delete these log teams.

Safety concerns

Once you deploy this resolution in a manufacturing surroundings, contemplate the next safety measures:

• Configure Amazon CloudWatch monitoring for the Flask-based proxy service to trace software well being, detect anomalies, and arrange alerts for suspicious actions.
• Implement fee limiting for the Flask-based proxy service to guard in opposition to potential denial-of-service (DoS) assaults and management the variety of requests from particular person shoppers. You need to use AWS WAF (internet software firewall) with the ALB to implement rate-based guidelines.
• Deploy an inner (non-internet-facing) ALB to limit proxy entry to your personal community. This setup makes positive that solely site visitors from inside your VPC or related networks can attain the service. Join via VPC peering or AWS Transit Gateway.
• Allow HTTPS termination on the ALB degree for safe communication between shoppers and your software. You need to use AWS Certificates Supervisor (ACM) to provision and handle SSL/TLS certificates on your software. For directions on configuring HTTPS listeners, see the Utility Load Balancer HTTPS listeners documentation.

These safety measures assist defend the Flask software in opposition to widespread internet vulnerabilities and supply safe communication between elements.

Conclusion

On this submit, we confirmed how you can construct a safe Flask-based proxy service that gives HTTPS entry to Amazon SageMaker MLflow. This resolution helps organizations bridge their current infrastructure with AWS managed MLflow capabilities whereas sustaining enterprise safety necessities.

Answer advantages:

• Integration with current enterprise safety controls.
• Minimal adjustments to current ML workflows.
• Decreased deployment complexity.
• REST API integration.
• Compatibility with enterprise proxy companies.

Subsequent steps

To be taught extra about Amazon SageMaker MLflow and associated subjects, you may:

Do this resolution in your individual surroundings and tell us your expertise within the feedback.

In regards to the authors