Generative AI is revolutionizing the way in which builders strategy programming by offering clever help and automation all through the coding course of. With the ability of superior language fashions and machine studying (ML) algorithms, generative AI can perceive the context and intent behind a programmer’s code, providing precious options, finishing code snippets, and even producing complete features or modules based mostly on high-level descriptions. This know-how empowers builders to deal with higher-level problem-solving and structure, whereas the AI handles the tedious and repetitive features of coding. One of many key benefits of huge language fashions (LLMs) in programming is their capability to study from the huge quantities of current code and programming patterns they had been skilled on. This data permits them to generate context-aware code, detect potential bugs or vulnerabilities, and provide optimizations to enhance code high quality and efficiency.
On this put up, we spotlight how the AWS Generative AI Innovation Middle collaborated with SailPoint Applied sciences to construct a generative AI-based coding assistant that makes use of Anthropic’s Claude Sonnet on Amazon Bedrock to assist speed up the event of software program as a service (SaaS) connectors.
Amazon Bedrock is a totally managed service that provides a selection of high-performing basis fashions (FMs) from main AI corporations like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon by way of a single API, together with a broad set of capabilities to construct generative AI purposes with safety, privateness, and accountable AI.
SailPoint makes a speciality of enterprise id safety options. Over 3,000 enterprises worldwide use SailPoint to assist defend towards in the present day’s dynamic, identity-centric cyber threats whereas enhancing productiveness and effectivity. Their merchandise are designed to handle and safe entry to purposes and information by way of the lens of id, at velocity and scale, for customers inside a company and for exterior events resembling non-employees. SailPoint’s unified, clever, and extensible atmosphere offers complete id governance capabilities, together with entry certifications, coverage administration, entry request and provisioning, password administration, and information entry governance. This helps organizations ensure the correct people have the correct entry to the correct sources on the proper instances, thereby implementing safety insurance policies and compliance necessities. Based in 2005, SailPoint has grown to be a key participant in id safety, serving clients globally throughout varied industries.
SailPoint connectors and SaaS connectivity
SailPoint’s id safety options interface with varied software program as a service (SaaS) purposes to retrieve the required info, resembling account and entry info, from an id safety standpoint. Every SaaS software implements these functionalities in barely alternative ways and may expose their implementation by way of REST-based net APIs which can be sometimes supported by OpenAPI specs. SailPoint connectors are TypeScript modules that interface with a SaaS software and map the related id safety info (resembling accounts and entitlements) to a standardized format understood by SailPoint. Based mostly on the APIs uncovered by the applying, SailPoint connectors can create, replace, and delete entry on these accounts. SailPoint connectors assist handle person identities and their entry rights throughout completely different environments inside a company, supporting the group’s compliance and safety efforts.
Though a typical connector exposes a number of features, for this put up, we deal with growing the listing person operate of a connector that connects to an API endpoint for itemizing customers, retrieving all of the customers, and reworking them into the format required by SailPoint.
Within the following sections, we element how we used Anthropic’s Claude Sonnet on Amazon Bedrock to robotically create the listing person connector, a crucial element of the broader SaaS connectivity.
Understanding the listing person connector
Connectors are modules that may hook up with an exterior service and retrieve and replace related info from a SaaS software. To higher perceive how connectors are constructed, we give an instance of the connector operate that connects to DocuSign’s REST API’s getUsers endpoint. The next TypeScript code defines an asynchronous operate listUsers that retrieves a listing of person accounts from an exterior API and constructs a structured output for every person:
The next is a breakdown of what every a part of the code does:
- Imports – The code imports a number of sorts and interfaces from
@sailpoint/connector-sdk
. These embraceContext, Response, StdAccountListHandler, and StdAccountListOutput
, that are used to deal with the enter and output of the operate in a standardized means inside a SailPoint atmosphere. - Perform definition –
listUsers
is outlined as an asynchronous operate suitable with theStdAccountListHandler
It makes use of theContext
to entry configuration particulars like API keys and the bottom URL, and aResponse
to construction the output. - Retrieve API key and host URL – These are extracted from the
context
parameter. They’re used to authenticate and assemble the request URL. - URL building – The operate constructs the preliminary URL utilizing the hostUrl and
organizationId
from thecontext
. This URL factors to an endpoint that returns customers related to a selected group. - Loop by way of pages – The
whereas
loop continues so long as there are extra pages of information (hasMore
is true). It serves the next features:- Fetch information – Contained in the
whereas
loop, afetch
request is made to the API endpoint. The request contains anAuthorization
header that makes use of theapiKey
. The API’s response is transformed to JSON format. - Course of customers – Contained in the
whereas
loop, it extracts person information from the API response. The method loops by way of every person, developing anStdAccountListOutput
object for each. This object contains person identifiers and attributes like person names, names, standing, electronic mail, and group IDs. - Pagination – Contained in the whereas loop, it checks if there’s a subsequent web page URL within the pagination info (
outcomes.paging.subsequent
). If it exists, it updates theurl
for the following iteration of the loop. If not, it units hasMore to false to cease the loop.
- Fetch information – Contained in the
Understanding this instance helps us perceive the step-by-step strategy of constructing this operate in a connector. We goal to breed this course of utilizing an LLM with a immediate chaining technique.
Generate a TypeScript connector utilizing an LLM immediate chain
There are a number of approaches to utilizing pre-trained LLMs for code technology, with various ranges of complexity:
- Single immediate – You should use fashions like Anthropic’s Claude to generate code by direct prompting. These fashions can generate code in quite a lot of languages, together with TypeScript, however they don’t inherently possess domain-specific information related to the duty of constructing a connector. All of the required info, together with API specs and formatting directions, have to be supplied within the immediate, much like the directions that might be given to a developer. Nevertheless, LLMs are inclined to battle when given an extended listing of advanced directions. It’s additionally tough for the immediate engineer to know which steps are difficult for the LLM.
- Agentic frameworks with LLMs – Brokers are a complicated framework that may use instruments to carry out a sequence of advanced duties. On this case, the agent begins by breaking down the person requests into steps, searches for obligatory info utilizing instruments (a information base or net browser), and autonomously generates code from begin to end. Though they’re highly effective, these frameworks are advanced to implement, typically unstable of their conduct, and fewer controllable in comparison with different strategies. Brokers additionally require many LLM calls to carry out a job, which makes them relatively gradual in observe. Within the case the place the logic to carry out a job is a set sequence of steps, brokers usually are not an environment friendly choice.
- Immediate chain – An answer that finds trade-off between the 2 earlier approaches includes utilizing a immediate chaining method. This technique breaks the advanced downside right into a sequence of extra manageable steps and integrates them to craft the ultimate code. Every step has clear directions which can be simpler for the LLM to observe, and a human within the loop can management the output of every step and proper the LLM if wanted. This strategy strikes a steadiness between flexibility and management, avoiding the extremes of the opposite two strategies.
We initially examined the LLM’s capability to generate connector code based mostly on a single immediate and realized that it struggles to generate code that addresses all features of the issue, resembling pagination or nested information constructions. To verify the LLM would cowl all the required elements of the connector features, and since making a connector follows a set sequence of steps, immediate chaining was essentially the most pure strategy to enhance the generated code.
The chain we used for connector technology consists of the next high-level steps:
- Parse the information mannequin of the API response into prescribed TypeScript courses.
- Generate the operate for person flattening within the format anticipated by the connector interface.
- Perceive the pagination of the API specs and formulate a high-level resolution.
- Generate the code for the
ListUsers
operate by combining all of the intermediate steps.
Step 1 is used as an enter to Step 2, however Step 3 is separate. Each Step 2 and Step 3 outcomes are fed to Step 4 for the ultimate outcome. The next diagram illustrates this workflow.
Within the following sections, we are going to dive into the prompting methods we used for every of those steps.
System immediate
The system immediate is an integral part of LLM prompting that sometimes offers the preliminary context to information the mannequin’s response. For all of the prompts within the chain, we used the next system immediate:
Extra particularly, the system immediate is used to determine the function of the LLM (professional net developer), give it a common purpose (perceive API specs and write TypeScript code), give high-level directions (add feedback within the code) and set boundaries (don’t make up info).
Information mannequin parsing
On this step, we immediate the LLM to know the construction of the API response and create TypeScript courses akin to the objects within the response. Though this step isn’t strictly obligatory for producing the response, it could assist the LLM immensely in producing an accurate connector. Just like chain-of-thought reasoning for arithmetic issues, it’s forcing the LLM to “assume” earlier than responding.
This step affords two main advantages:
- Verbose API response simplification – API responses specified within the documentation might be fairly verbose. By changing the response construction into TypeScript courses, we compress the data into fewer traces of code, making it extra concise and simpler for the LLM to understand. This step helps be sure that the important info is prominently displayed firstly.
- Dealing with fragmented person responses – In some APIs, the person response consists of a number of fragments due to the reuse of information constructions. The OpenAPI specification makes use of the
$ref
tag to reference these reusable elements. By changing the person response into TypeScript courses, we will consolidate all of the related info right into a single location. This consolidation simplifies the downstream steps by offering a centralized supply of knowledge.
We use the next job immediate to transform the API response into prescribed TypeScript courses:
Within the previous immediate template, the variable {api_spec}
is changed with the API specification of the endpoint. A selected instance for a DocuSign ListUsers
endpoint is supplied within the appendix.
The next code is an instance of the LLM-generated courses when utilized to the DocuSign API specs. This has been parsed out of the
tags.
Person flattening operate technology
The anticipated construction for every person is an object consisting of two properties: an identifier and a dictionary of attributes. The attributes dictionary is a map that associates string keys with both primitive attributes (quantity, Boolean, or string) or an array of primitive attributes. due to the potential for arbitrarily nested JSON object constructions within the response, we use the capabilities of an LLM to generate a person flattening and conversion operate. Each the person ID and the attributes are extracted from the response. By using this strategy, we successfully separate the intricate job of changing the person construction from the REST API response into the required format for the SailPoint connector SDK (hereafter known as the connector SDK).
The advantages of this strategy are twofold. First, it permits for a cleaner and extra modular code design, as a result of the advanced conversion course of is abstracted away from the primary code base. Second, it allows higher flexibility and flexibility, as a result of the conversion operate might be modified or regenerated to accommodate adjustments within the API response construction or the connector SDK necessities, with out necessitating intensive modifications to the encircling code base.
We use the next immediate to generate the conversion operate, which takes as enter the information mannequin generated within the earlier step:
Within the previous immediate template, we change the {data_model}
variable with the information mannequin of TypeScript courses extracted within the earlier technology step of parsing the information mannequin.
The next code is an instance of the LLM-generated person flattening operate when utilized to the DocuSign API:
Pagination understanding
As talked about earlier, the REST API can implement a number of pagination schemes. Usually, the pagination particulars aren’t explicitly talked about. Throughout the growth of the chain, we discovered that when there are a number of pagination schemes, the LLM would combine up parts of various pagination schemes and output code that isn’t coherent and generally additionally incorporates errors. As a result of looping over the paged outcomes is an important step, we separate out this step within the code technology to let the LLM perceive the pagination scheme carried out by the API and formulate its response at a excessive stage earlier than outputting the code. This permits the LLM to assume step-by-step in formulating the response. This step generates the intermediate reasoning, which is fed into the following and closing step: producing the listing customers operate code.
We use the next immediate to get the pagination logic. As a result of we’re utilizing Anthropic’s Claude Sonnet on Amazon Bedrock, we ask the LLM to output the logic in XML format, which is understood to be an environment friendly strategy to construction info for that mannequin.
Within the previous immediate template, the variable {api_spec}
is changed with the API specification. An instance of the DocuSign API is supplied within the appendix on the finish of this put up. The variable {api_info}
might be changed with extra API documentation in pure language, which is left as an empty string within the DocuSign instance.
The next is the LLM’s response for the pagination logic extraction within the case of the DocuSign API, parsed out of the
tags:
ListUsers operate technology
This closing step within the chain combines the data extracted within the earlier steps along with the person flattening operate generated within the earlier steps to formulate the ultimate response, which is the TypeScript operate that retrieves a listing of customers from the supplied API.
We use the next immediate to generate the whole TypeScript operate:
On this immediate, we change {flatten_user_function}
with the flattenUser
that was generated earlier and {pagination_logic}
with the one which was generated earlier. We offer a template for the listUsers
operate to ensure the ultimate output meets the necessities for the connector operate. The ensuing output is the next listUsers
operate, which makes use of the flattenUser
operate from earlier:
Classes realized
On this put up, we demonstrated how LLMs can deal with advanced code technology issues by using varied core prompting rules and the immediate chaining method. Though LLMs excel at following clearly outlined directions and producing small code snippets, this use case concerned a considerable quantity of contextual info within the type of API specs and person directions. Our findings from this train are the next:
- Decomposing advanced issues – Breaking down a posh code technology downside into a number of intermediate steps of decrease complexity enhances the LLM’s efficiency. Offering a single advanced immediate may end up in the LLM lacking some directions. The immediate chaining strategy enhances the robustness of the technology, sustaining higher adherence to directions.
- Iterative optimization – This technique permits for iterative optimization of intermediate steps. Every a part of the chain might be refined independently earlier than transferring to the following step. LLMs might be delicate to minor adjustments in directions, and adjusting one facet can unintentionally have an effect on different targets. Immediate chaining affords a scientific strategy to optimize every step independently.
- Dealing with advanced selections – Within the part on understanding pagination, we illustrated how LLMs can purpose by way of varied choices and make advanced selections earlier than producing code. As an illustration, when the enter API specification helps a number of pagination schemes, we prompted the LLM to determine on the pagination strategy earlier than implementing the code. With direct code technology, with out utilizing an intermediate reasoning step, the LLM tended to combine parts of various pagination schemes, leading to inconsistent output. By forcing decision-making first, in pure language, we achieved extra constant and correct code technology.
By means of automated code technology, SailPoint was capable of dramatically scale back connector growth time from hours or days to mere minutes. The strategy additionally democratizes code growth, so that you don’t want deep TypeScript experience or intimate familiarity with SailPoint’s connector SDK. By accelerating connector technology, SailPoint considerably shortens the general buyer onboarding course of. This streamlined workflow not solely saves precious developer time but additionally allows sooner integration of numerous programs, in the end permitting clients to make use of SailPoint’s id safety options extra quickly and successfully.
Conclusion
Our AI-powered resolution for producing connector code opens up new potentialities for integrating with REST APIs. By automating the creation of connectors from API specs, builders can quickly construct strong connections to any REST API, saving developer time and lowering the time to worth for onboarding new clients. As demonstrated on this put up, this know-how can considerably streamline the method of working with numerous APIs, permitting groups to deal with utilizing the information and performance these APIs present relatively than getting overwhelmed by connector code particulars. Take into account how such an answer may improve your individual API integration efforts—it might be the important thing to extra environment friendly and efficient use of the myriad APIs accessible in in the present day’s interconnected digital panorama.
In regards to the Authors
Erik Huckle is the product lead for AI at SailPoint, the place he works to unravel crucial buyer issues within the id safety ecosystem by way of generative AI and information applied sciences. Previous to SailPoint, Erik co-founded a startup in robotic automation and later joined AWS as the primary product rent at Amazon One. Erik mentors native startups and serves as a board member and tech committee lead for a EdTech nonprofit group.
Tyler McDonnell is the engineering head of AI at SailPoint, the place he leads the event of AI options to drive innovation and impression in id safety world. Previous to SailPoint, Tyler led machine studying analysis and engineering groups at a number of early to late-stage startups and revealed work in domains spanning software program upkeep, info retrieval, and deep studying. He’s obsessed with constructing merchandise that use AI to carry constructive impression to actual individuals and issues.
Anveshi Charuvaka is a Senior Utilized Scientist on the Generative AI Innovation Middle, the place he helps clients undertake Generative AI by implementing options for his or her crucial enterprise challenges. With a PhD in Machine Studying and over a decade of expertise, he makes a speciality of making use of modern machine studying and generative AI methods to deal with advanced real-world issues.
Aude Genevay is a Senior Utilized Scientist on the Generative AI Innovation Middle, the place she helps clients deal with crucial enterprise challenges and create worth utilizing generative AI. She holds a PhD in theoretical machine studying and enjoys turning cutting-edge analysis into real-world options.
Mofijul Islam is an Utilized Scientist II on the AWS Generative AI Innovation Middle, the place he helps clients deal with advanced, customer-centric analysis challenges utilizing generative AI, giant language fashions (LLM), multi-agent studying, and multimodal studying. He holds a PhD in machine studying from the College of Virginia, the place his work centered on multimodal machine studying, multilingual NLP, and multitask studying. His analysis has been revealed in top-tier conferences like NeurIPS, ICLR, AISTATS, and AAAI, in addition to IEEE and ACM Transactions.
Yasin Khatami is a Senior Utilized Scientist on the Generative AI Innovation Middle. With greater than a decade of expertise in synthetic intelligence (AI), he implements state-of-the-art AI merchandise for AWS clients to drive effectivity and worth for buyer platforms. His experience is in generative AI, giant language fashions (LLM), multi-agent methods, and multimodal studying.
Karthik Ram is a Principal Options Architect with Amazon Net Providers based mostly in Columbus, Ohio. He works with Impartial Software program Distributors (ISVs) to construct safe and modern cloud options, together with serving to with their merchandise and fixing their enterprise issues utilizing data-driven approaches. Karthik’s space of depth is Cloud Safety with a deal with Infrastructure Safety and menace detection.
Appendix
The next API specs had been used for the experiments on this put up: