This put up is co-written with Taras Tsarenko, Vitalil Bozadzhy, and Vladyslav Horbatenko.
As organizations worldwide search to make use of AI for social impression, the Danish humanitarian group Bevar Ukraine has developed a complete digital generative AI-powered assistant known as Victor, geared toward addressing the urgent wants of Ukrainian refugees integrating into Danish society. This put up particulars our technical implementation utilizing AWS providers to create a scalable, multilingual AI assistant system that gives automated help whereas sustaining knowledge safety and GDPR compliance.
Bevar Ukraine was established in 2014 and has been on the forefront of supporting Ukrainian refugees in Denmark for the reason that full-scale conflict in 2022, offering help to over 30,000 Ukrainians with housing, job search, and integration providers. The group has additionally delivered greater than 200 tons of humanitarian support to Ukraine, together with medical provides, mills, and important gadgets for civilians affected by the conflict.
Background and challenges
The mixing of refugees into host nations presents a number of challenges, notably in accessing public providers and navigating advanced authorized procedures. Conventional assist techniques, relying closely on human social employees, usually face scalability limitations and language limitations. Bevar Ukraine’s answer addresses these challenges via an AI-powered system that operates constantly whereas sustaining excessive requirements of service high quality.
Answer overview
The answer’s spine includes a number of AWS providers to ship a dependable, safe, and environment friendly generative AI-powered digital assistant for Ukrainian refugees. The workforce consisting of three volunteer software program builders developed the answer inside weeks.
The next diagram illustrates the answer structure.
Amazon Elastic Compute Cloud (Amazon EC2) serves as the first compute layer, utilizing Spot Cases to optimize prices. Amazon Easy Storage Service (Amazon S3) gives safe storage for dialog logs and supporting paperwork, and Amazon Bedrock powers the core pure language processing capabilities. Bevar Ukraine makes use of Amazon DynamoDB for real-time knowledge entry and session administration, offering low-latency responses even underneath excessive load.
Within the technique of implementation, we found that Anthropic’s Claude 3.5 massive language mannequin (LLM) is greatest suited as a result of its superior dialogue logic and talent to take care of a human-like tone. It’s greatest for thorough, reasoned responses and producing extra inventive content material, which makes Victor’s replies extra pure and fascinating.
Amazon Titan Embeddings G1 – Textual content v1.2 excels at producing high-quality vector representations of multilingual textual content, enabling environment friendly semantic search and similarity comparisons. That is notably precious when Victor must retrieve related info from a big data base or match customers’ queries to beforehand seen inputs. Amazon Titan Embeddings additionally integrates easily with AWS, simplifying duties like indexing, search, and retrieval.
In real-world interactions with Victor, some queries require quick, particular solutions, whereas others want inventive technology or contextual understanding. By combining Anthropic’s Claude 3.5. for technology and Amazon Titan Embeddings G1 for semantic retrieval, Victor can route every question via probably the most applicable pipeline, retrieving related context via embeddings and producing a response, leading to extra correct and context-aware solutions.
Amazon Bedrock gives a exceptional interface to name Anthropic’s Claude 3.5 and Amazon Titan Embeddings G1 (together with different fashions) with out creating separate integrations for every supplier, simplifying growth and upkeep.
For multilingual assist, we used embedders that assist multi-language embeddings and translated our supplies utilizing Amazon Translate. This enhances the resilience of our Retrieval Augmented Era (RAG) system. The appliance is constructed securely and makes use of AWS providers to perform this. AWS Key Administration Service (AWS KMS) simplifies the method of encrypting knowledge inside the utility, and Amazon API Gateway helps the functions REST endpoints. Person authentication and authorization capabilities are supported by Amazon Cognito, which gives safe and scalable buyer identification and entry administration (CIAM) capabilities.
The appliance runs on AWS infrastructure utilizing providers which are designed to be safe and scalable like Amazon S3, AWS Lambda, and DynamoDB.
Suggestions and proposals
Constructing an AI assistant answer for refugees utilizing Amazon Bedrock and different AWS providers has supplied precious insights into creating impactful AI-powered humanitarian options. By way of this implementation, we found key issues that organizations ought to be mindful when creating comparable options. The expertise highlighted the significance of balancing technical capabilities with human-centric design, offering multilingual assist, sustaining knowledge privateness, and creating scalable but cost-effective options. These learnings can function a basis for organizations trying to make use of AI and cloud applied sciences to assist humanitarian causes, notably in creating accessible and useful digital help for displaced populations. The next are the primary
- Use the Amazon Bedrock playground to check a number of LLMs facet by facet utilizing the identical immediate. This helps you discover the mannequin that provides the very best quality, type, and tone of response to your particular use case (for instance, factual accuracy vs. conversational tone).
- Experiment with prompts and settings to enhance responses.
- Preserve prices in thoughts; arrange monitoring and budgets in AWS.
- For duties involving info retrieval or semantic search, choose an embedding mannequin whereas ensuring to choose the suitable settings. Take note of the scale of the embeddings, as a result of bigger vectors can seize extra that means however may improve prices. Additionally, test that the mannequin helps the languages your utility requires.
- For those who’re utilizing a data base, use the Amazon Bedrock data base playground to experiment with how content material is chunked and what number of passages are retrieved for every question. Discovering the appropriate variety of retrieved passages could make a giant distinction in how clear and targeted the ultimate solutions are—generally fewer, high-quality chunks work higher than sending an excessive amount of context.
- To implement security and privateness, use Amazon Bedrock Guardrails. Guardrails may also help stop the mannequin from leaking delicate info, akin to private knowledge or inside enterprise content material, and you’ll block dangerous responses or implement a selected tone and formatting type.
- Begin with a easy prototype, check the embedding high quality in your area, and broaden iteratively.
Integration and enhancement layer
Bevar Ukraine has prolonged the core AWS infrastructure with a number of complementary applied sciences:
- Pinecone vector database – For environment friendly storage and retrieval of semantic embeddings
- DSPy framework – For structured immediate engineering and optimization of Anthropic’s Claude 3.5 Sonnet responses
- EasyWeek – For appointment scheduling and useful resource administration
- Telegram API – For UI supply
- Amazon Bedrock Guardrails – For safety coverage enforcement
- Amazon Rekognition – For doc verification
- GitHub-based steady integration and supply (CI/CD) pipeline – For speedy characteristic deployment
Key technical insights
The implementation revealed a number of essential technical issues. The DSPy framework was essential in optimizing and enhancing our language mannequin prompts. By integrating extra layers of reasoning and context consciousness instruments, DSPy notably improved response accuracy, consistency, and depth. The workforce discovered that designing a strong data base with complete metadata was elementary to the system’s effectiveness.
GDPR compliance required cautious architectural selections, together with knowledge minimization, safe storage, and clear consumer consent mechanisms. Value optimization was achieved via strategic use of EC2 Spot Cases and implementation of API request throttling, leading to important operational financial savings with out compromising efficiency.
Future enhancements
Our roadmap contains a number of technical enhancements to boost the system’s capabilities:
- Implementing superior context dispatching utilizing machine studying algorithms to enhance service coordination throughout a number of domains
- Creating a classy human-in-the-loop validation system for advanced circumstances requiring professional oversight
- Migrating appropriate parts to a serverless structure utilizing Lambda to optimize useful resource utilization and prices
- Enhancing the data base with superior semantic search capabilities and automatic content material updates
Outcomes
This answer, which serves a whole bunch of Ukrainian refugees in Denmark each day, demonstrates the potential of AWS providers in creating scalable, safe, and environment friendly AI-powered techniques for social impression. Consequently, volunteers and staff of Bevar Ukraine have saved 1000’s of hours, and as an alternative of answering repetitive questions from refugees, can assist them in additional difficult life conditions. For refugees, the digital assistant Victor is a lifeline assist that enables customers to get responses to probably the most urgent questions on public providers in Denmark and plenty of different questions in seconds as an alternative of getting to attend for an out there volunteer to assist. Given the huge data base Victor is utilizing to generate responses, the standard of assist has improved as properly.
Conclusion
By way of cautious structure design and integration of complementary applied sciences, we’ve created a platform that successfully addresses the challenges confronted by refugees whereas sustaining excessive requirements of safety and knowledge safety.
The success of this implementation gives a blueprint for comparable options in different social service domains, doubtlessly supporting refugees and different folks in want all over the world, highlighting the significance of mixing sturdy cloud infrastructure with considerate system design to create significant social impression.
Concerning the Authors
Taras Tsarenko is a Program Supervisor at Bevar Ukraine. For over a decade on the earth of know-how, Taras has led all the things from tight-knit agile groups of 5 or extra to an organization of 90 those who turned the most effective small IT firm in Ukraine underneath 100 folks in 2015. Taras is a builder who thrives on the intersection of technique and execution, the place technical experience meets human impression, whether or not it’s streamlining workflows, fixing advanced issues, or empowering groups to create significant merchandise. Taras makes a speciality of AI-driven options and knowledge engineering, leveraging applied sciences like machine studying and generative AI utilizing Amazon SageMaker AI, Amazon Bedrock, Amazon OpenSearch Service, and extra. Taras is an AWS Licensed ML Engineer Affiliate.
Anton Garvanko is a Senior Analytics Gross sales Specialist for Europe North at AWS. As a finance skilled turned salesman, Anton spent 15 years in varied finance management roles in provide chain and logistics in addition to monetary providers industries. Anton joined Amazon over 5 years in the past and has been a part of specialist gross sales groups specializing in enterprise intelligence, analytics, and generative AI for over 3 years. He’s obsessed with connecting the worlds of finance and IT by ensuring that enterprise intelligence and analytics powered by generative AI assist on a regular basis decision-making throughout industries and use circumstances.
Vitalii Bozadzhy is a Senior Developer with intensive expertise in constructing high-load, cloud-based options, specializing in Java, Golang, SWIFT, and Python. He makes a speciality of scalable backend techniques, microservice architectures designed to automate enterprise processes, in addition to constructing dependable and safe cloud infrastructures. Moreover, he has expertise in optimizing compute sources and constructing superior options built-in into merchandise. His experience covers the complete growth cycle—from design and structure to deployment and upkeep—with a powerful give attention to efficiency, fault tolerance, and innovation.
Vladyslav Horbatenko is a pc science pupil, Professor Assistant, and Information Scientist with a powerful give attention to synthetic intelligence. Vladyslav started his journey with machine studying, reinforcement studying, and deep studying, and step by step turned extra taken with massive language fashions (LLMs) and their potential impression. This led him to deepen his understanding of LLMs, and now he works on creating, sustaining, and bettering LLM-based options. He contributes to progressive tasks whereas staying updated with the newest developments in AI.
