Steer AGI

Page 1: Multi Agent based Medical Assistant for Edge Devices Sakharam Gawade*, Shivam Akhouri*, Chinmay Kulkarni*, Jagdish Samant*, Pragya Sahu*, Aastik, Jai Pahal, Saswat Meher Samsung Research Institute Bangalore, India Team Name: Agent Lite Abstract Large Action Models (LAMs) have revolution- ized intelligent automation, but their applica- tion in healthcare faces challenges due to pri- vacy concerns, latency, and dependency on in- ternet access. This report introduces an on- device, multi-agent healthcare assistant that overcomes these limitations. The system uti- lizes smaller, task-specific agents to optimize resources, ensure scalability and high perfor- mance. Our proposed system acts as a one-stop solution for health care needs with features like appointment booking, health monitoring, med- ication reminders, and daily health reporting. Powered by the Qwen Code Instruct 2.5 7B model, the Planner and Caller Agents achieve an average RougeL score of 85.5 for planning and 96.5 for calling for our tasks while being lightweight for on-device deployment. This in- novative approach combines the benefits of on- device systems with multi-agent architectures, paving the way for user-centric healthcare solu- tions. Check our code here. 1 Introduction Large Action Models (LAMs) have emerged as a groundbreaking paradigm in intelligent automa- tion, enabling systems to execute complex tasks by dividing them into modular, agent-specific com- ponents. These systems excel in domains like e- commerce and logistics, where cloud-based archi- tectures leverage vast computational resources to deliver high precision and scalability. However, ap- plying this framework to healthcare poses unique challenges that cloud-centric systems are ill-suited to address. Healthcare applications require handling sensi- tive personal data, ensuring real-time responsive- ness, and maintaining uninterrupted functionality, even in offline scenarios. Cloud-based solutions, while powerful, are often constrained by privacy *Equal Contributionrisks, latency issues, and dependency on continu- ous internet access. These challenges necessitate a shift toward on-device implementations, where user data remains private , and operations are opti- mized for local execution. The computational limitations of edge devices pose an additional challenge when deploying large, monolithic models. This is where the power of multi-agent systems comes into play. By utiliz- ing smaller, task-specific agents, we can effectively distribute workloads . This leads to optimized resource allocation , where compact models are designed to operate within the constraints of edge devices, ensuring efficient use of available compu- tational resources. Moreover, the system’s func- tionality can be easily scaled by introducing addi- tional agents, allowing for incremental growth and adaptation. This setup fosters “modular collabora- tion”, where each agent functions independently, focusing on its specific task, while also working in harmony with others to manage more complex workflows. Our solution uses a multi-agent architecture with on-device data and specialized models for intelli- gent diagnosis, appointment booking, emergency services, etc. It is integrated with a user-friendly ap- plication, enabling seamless interaction and person- alized data retrieval, enhancing agent capabilities through retrieval augmented generation. In this project, we contribute the following: 1.A multi-agent healthcare assistant designed foredge devices , ensuring privacy andscala- bility (refer section 4). 2.Adata creation pipeline that generates syn- thetic data and enhances it to address chal- lenges relevant to AI agents (refer section 6). 3.The fine-tuned planner and caller agents that achieve an average RougeL score of 85.5 forarXiv:2503.05397v1 [cs.MA] 7 Mar 2025 Page 2: planning and 96.5 for calling using our gen- erated data, demonstrating effectiveness for our target use cases (refer to section 7). 4.Anapplication that connects users with the multi-agent system, enabling personalized data interaction (refer section 5). 2 Motivation In the midst of today’s hectic lifestyle, staying on top of one’s health can feel like an uphill battle. Juggling medications, monitoring vital signs, and keeping track of appointments can easily become overwhelming, especially for vulnerable groups such as children and the elderly, who often require more frequent medical attention. Our multi-agent healthcare assistant is designed to simplify and streamline this process, empowering users of all ages to take charge of their health journey while en- suring the utmost privacy. By harnessing the power of local, on-device processing and seamless inte- gration with smartwatches, we offer a secure, user- friendly solution that makes health management efficient and accessible for everyone, regardless of age or technological expertise. 3 Problem Statement and Use-cases We develop a multi-agent system that manages per- sonal health data securely and efficiently while pro- viding users with timely reminders, monitoring vi- tals, and handling appointment requests based on symptoms. It operates locally on the user’s device, ensuring data privacy and seamless integration with smart watches for enhanced user experience. The primary usecases for the project are: Intelligent Diagnosis and Appointment Scheduling: The healthcare assistant engages in multi-turn conversations with the user, allowing for intelligent diagnosis by gathering comprehensive information about the user’s health concerns. This iterative questioning process is key to accurately identifying the most appropriate medical specialist, ensuring targeted and effective care. It also uses past interactions and profile information to identify the right specialist. The aim is not to diagnose or provide remedies, but to identify the most suitable specialist based on the user’s needs and availability. This end-to-end functionality reduces user burden and ensures timely consultations without unnecessary hassle. Emergency SOS: The system is equipped to handle critical emergency situations where imme-diate assistance is required. Users can manually trigger an SOS signal, which prompts the system to take swift action. We call this as Hard SOS where we are sure of an emergency situation. These ac- tions may include notifying emergency services (e.g., calling an ambulance), contacting the user’s designated emergency contacts, and providing real- time updates to responders. The system is designed to ensure prompt escalation of emergencies, en- hancing the user’s safety and response time during life-threatening scenarios. Vitals Tracking and Abnormality Detection: The system continuously monitors the user’s vi- tal signs through smart devices like wearable health trackers or smartphones. This feature en- sures proactive health management by identifying anomalies or irregularities in real time, such as ab- normal heart rate, irregular blood oxygen levels, or deviations in other key health metrics. When a abnormal situation is detected, a SOS is triggered, alerting the user and providing actionable recom- mendations. We call it Soft SOS as unlike Hard SOS, it only alerts the user. For this project, we integrate with a simulator smart watch. Schedule Reminders: Managing healthcare rou- tines can be challenging, especially for users with multiple prescriptions or treatment plans. The sys- tem automates this process by extracting relevant details from prescriptions or user inputs and setting personalized reminders. Daily Health Report: The system generates a comprehensive daily report to enhance health track- ing, summarizing the user’s essential health met- rics and any detected anomalies throughout the day. This report is provided to the user each evening, presenting a clear overview of their health status. Furthermore, the reports are stored long-term, al- lowing users and healthcare providers to review his- torical data for well-informed decisions and trend analysis. 4 System Architecture In our proposed architecture, three key components collaborate to deliver a comprehensive healthcare assistance experience: the Action Manager, Health Manager, and Memory Unit. Figure 1a show the in- teraction between these three modules. The Health Manager’s agents independently handle schedul- ing reminders, monitoring vitals, and generating reports. It connects with the Action Manager for user interaction and tool execution, utilizing the Page 3: (a) Communication in the multi-agent system (b) Communication between the planner and caller in Action Manager Figure 1: Multi-Agent Design for Healthcare Assistant Action Manager’s agents for planning and execu- tion. The Memory Unit enables user-agent and agent-agent interaction, personalizing the system. We chose the Qwen 2.5-Coder-7B-Instruct model 1as the backbone model for our architecture due to its size which can fit in a flash memory and per- formance(Alizadeh et al., 2023). The following subsections delve into the details of each compo- nent. 4.1 Health Manager The Health Manager is a crucial component of our multi-agent healthcare system that uses multiple agents to monitor user health, generate reports, and manage medication schedules. It consists of three key sub-components: the Report Generator, Health Monitor, and Scheduler. 4.1.1 Report Generator The Report Generator creates a comprehensive summary of each session using Qwen7B. It pro- cesses session data, including user queries, exe- cuted actions, vital health data, medicine prescrip- tions, and key insights gathered during the session, to generate a detailed and coherent report. This ensures that the session’s key details are accurately captured and presented in a user-friendly format, enhancing the overall effectiveness and reliability of the system. 4.1.2 Health Monitor The Health Monitor continuously analyzes vital signs from the user’s wearable devices, such as smartwatches or smart rings, which provide real- time data on key health metrics like heart rate, blood pressure, and oxygen saturation. If the vitals deviate significantly from normal ranges, the sys- tem automatically triggers a soft SOS alert, serving 1Qwen2.5-Coder-7B-Instruct: https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instructas an early warning mechanism and ensuring that appropriate measures can be taken promptly. 4.1.3 Scheduler The Scheduler assists users in managing their medi- cation schedules effectively by analyzing the user’s prescription and automatically extracting essential details such as medicine names, dosage timing, frequency, and duration. Using this information, the Scheduler creates personalized reminders that are sent to the user, helping them adhere to their prescribed treatment plan. 4.2 Action Manager The role of Action Manager is to execute a task using available tools. This task is accomplished by two agent models, planner and caller. Since a model trained on code has function knowledge and improved reasoning (Ma et al., 2023; Zhang et al., 2024b), we choose an instruct finetuned model which has been also finetuned on coding tasks. We LoRA finetune it for planning and calling to incor- porate planning and function calling abilities into the model and align with our usecases. 4.2.1 Planner The planner takes the user’s query and based on the information of the tools, it generates an action along with its reasoning (ReAct framework (Yao et al., 2022)). After the action is executed, the result along with the trajectory is passed to the planner, and the subsequent action and its reasoning is gener- ated. Based on the results of the actions, next action is generated until the task is completed. This style of planning is called interleaved planning ((Huang et al., 2024)). We chose this over generating plan in a single-go for robustness in task execution. We LoRA finetune the Instruct-LLM model using our synthetic planning data (refer section 6). Page 4: 4.2.2 Caller The Caller is responsible for identifying the cor- rect parameters to be passed to the tool call. Our experiments revealed that function calling models perform poorly when dealing with a large num- ber of tools (examples in section B). The Caller acts as the execution engine for various tasks, dy- namically invoking appropriate functions based on instructions generated by the Planner (Zhang et al., 2024a; Schick et al., 2023). These tasks may include interacting with external APIs, retrieving or updating user-specific data, engaging with the user, and performing actions such as scheduling reminders, sending alerts, or interfacing with wear- able devices. 4.3 Memory The Memory Component is a crucial element in our multi-agent healthcare system, enabling a personal- ized user experience by storing and managing both long-term and short-term information. This compo- nent allows the system to tailor its interactions and provide more accurate, context-aware assistance (Sumers et al., 2024; Zhang et al., 2024c). 4.3.1 Short-Term Memory The Short-Term Memory (STM) maintains context and temporary data during active sessions, such as the user’s current symptoms and ongoing in- teractions. This enables the agents to maintain a coherent conversation and adapt to the user’s im- mediate needs, ensuring a seamless and responsive experience. 4.3.2 Long-Term Memory The Long-Term Memory (LTM) stores the user’s personal information, previous symptoms, and other relevant data that can be retrieved by the Plan- ner agent for improved decision-making. By lever- aging this historical information, the Planner can better identify underlying issues and make more informed decisions when generating plans for the user’s healthcare needs (Jiang et al., 2024). For example, knowing a patient’s past complaints and vital history can help the Planner refer to the correct specialist, resulting in more targeted and effective care. For retrieving relevant information from the Long-Term Memory (LTM), we employ the retrieval model from the Spacy library’s en_core_web_trf2to extract pertinent data based 2Spacy’s en_core_web_trfon user queries and context. To parse the retrieved information, we utilize the EasyOCR library3, which enables text recognition and extraction from various documents such as prescription. 5 Application Design Figure 2: System flow diagram of the E2E application Our application follows a three-tier architecture, as illustrated in Figure 2. The user interacts with the system through the frontend, which serves as the topmost layer of the application. All user ac- tions are processed at the backend, where appropri- ate responses are generated based on the specific action. The data layer, which consists of a SQLite database, handles data storage and retrieval for var- ious queries and functionalities. Following are the main components of app ar- chitecture: •Frontend – provides UI interface to the user for app interaction •Backend – Intermediate processor between application services and frontend • SQLite – SQL db to store necessary data • Celery – Task queue to process async tasks •Twilio – Sending messages to ambulance and emergency contacts 3EasyOCR: https://github.com/JaidedAI/EasyOCR Page 5: 5.1 Backend The Django backend serves as the core of the ap- plication, handling various tasks such as process- ing user queries, managing user data, creating re- minders based on uploaded prescriptions, trigger- ing SOS calls, and interacting with the agent mod- els. User interactions and histories are stored in a SQLite database, with Django ORM facilitating seamless data operations. The backend provides several key endpoints, in- cluding: • Receiving user input for the agent model •Presenting a summary of tracked vitals and suggestions •Triggering SOS calls in case of abnormality detection •Reminding users to take medication based on their prescription The primary responsibilities of the backend in- clude: • Hosting multiple large action models •Managing API endpoints for model interac- tions •Providing a secure and scalable backend for the chat system and managing each user’s his- tory •Analyzing recorded user vitals and providing suggestions based on the analysis •Sending SMS to friends and ambulance ser- vices based on the severity of the detected abnormality 5.2 Frontend The frontend leverages Next.js4for dynamic and responsive user interactions, managed under a Tur- borepo structure to streamline development across teams. Next.js, a React-based framework provides server-side rendering and client-side interactions for optimal user experience and at the same time provides room for scalability and improvement with well defined structure. Turborepo is used to manage multiple repositories under a single struc- ture, optimizing builds and deployments, reducing development overhead for future. 4Next.js - https://nextjs.org/5.2.1 UI Interface The visual components are styled using tailwind CSS and styled-components, which provides pre- defined styling framework for fast as well as re- sponsive development. The functional components present in the UI are as follows: •A real-time chat-like setup for users to interact with the planner model. •Powered by httpx and websockets for live up- dates and responsiveness. •Utilizes Fetch API to call Django REST end- points. •Error handling and loading states ensure a smooth user experience. •Watch and SOS simulator have been included for demo purpose. •Agent logs are added to help understand the logic behind agent’s inference for user input. 5.2.2 State Management •State is managed using React Context API for lightweight and efficient state sharing. •Server-side rendering (SSR) for initial page load and client-side fetching for subsequent interactions to save on data and improve re- sponse time. 6 Data Creation We curate a synthetic dataset to LoRA finetune the planner and caller (refer table 1). We present our data generation process in the following subsec- tions. Figure 3: Data Creation Process Page 6: Task Usecase Planner Caller Train Combined 374,000 328,000 TestGeneral 700 800 Counter 876 976 Negative 492 589 Dietician 1,273 1,423 Soft SOS 200 100 Hard SOS - Start 700 600 Hard SOS - End 600 500 Table 1: Statistics of Synthetic Data Created for Plan- ner and Caller agents. Combined indicates mixture of appointment usecases (general, counter, negative, di- etician), hard SOS (start and end) and soft SOS. Gen- eral: Appointment booked after finding specialist for the symptoms, Counter: Follow-up questions asked to understand symptoms better, Negative: User declines the appointment, Dietician: Use is referred to dietician based on the symptoms 6.1 Data Gathering We first begin by standardizing the format of the data. We create a trajectory containing communica- tion between the planner and caller to complete the task. The planner uses the ReAct framework where based on the task the next action and its reasoning is generated. The caller generates the function call along with appropriate parameters. After finaliz- ing format of trajectory, we generate data using OpenAI’s gpt-4o-mini. 6.1.1 Data Format This section introduces the data format used in our multi-agent healthcare system. A trajectory represents an interleaved history of the interaction between the user, multiple agents, and the environ- ment to complete a task. Following is the format of each state in the trajectory: 1. System { " from ": " system ", " value ": { " user_details ": { " user_id ": " BKYA569367 ", " name ": " John Doe ", " timestamp ": "2024 -12 -17 T23 :59:59" } } } 2. User{ " from ": " user ", " value ": " Natural language input from user " } 3. Planner { " from ": " planner ", " value ": { " reason ": " Reasoning Generated by LLM based on previous actions ", " action ": " Next action based on LLM's reasoning . <END > for completed task " } } 4. Caller { " from ": " caller ", " value ": { " tool ": " function_name ", " parameters ": { " param1 ": " value1 ", " param2 ": " value2 ", ... " paramN ": " valueN " } } } 5. Observation { " from ": " observation ", " value ": { " result ": Object_returned_by_function } } Consolidating the states, the format of trajec- tory looks as follows: { " from ": " system ", " value ": {...} }, { " from ": " user ", " value ": "" }, { " from ": " planner ", " value ": {...} }, { " from ": " caller ", " value ": {...} }, { " from ": " observation ", " value ": {...} } ... { " from ": " planner ", " value ": { Page 7: " reason ": " The task is sucessfully completed .", " action ": "<END >" } } For our usecases, examples of trajectories are shown in section D. 6.1.2 Data Generation To generate data for our multi-agent healthcare system, we use a 1-shot prompt with OpenAI’s gpt-4o-mini, providing a list of tools, a description of the use case, and an example trajectory. For instance, when generating appointment data, we want the Planner to suggest a specialist without diagnosing a disease while maintaining data diver- sity. To achieve this, we include a specific disease in the prompt and instruct the LLM to generate symptoms for that disease and suggest a specialist accordingly. 6.2 Data Enhancement After generating the data, we noticed that it had repetitive names, dates, and times. To fix this, we replaced them with random values. Since the input and output formats of tools for the named entities are standard (e.g. YYYY-MM-DD for date), we used regular expressions to find these entities. For dates and times, we made sure that in cases like appointments, the date of past symptoms comes before the user’s query date, and the suggested appointment is within a reasonable time after the user’s query. The tools and their descriptions in the input to the Planner serve as premises for reasoning and generating the next action. As demonstrated by (Chen et al., 2024), shuffling the premises can hin- der the model’s reasoning ability. Therefore, we shuffle the tools for each sample to enhance the model’s robustness and adaptability.. 6.3 Data Verification We analyze the generated data for each usecase and identify the scope of improvements. Apart from data augmentation, we check cases where the trained models might fail with the current data and adapt it for such exigencies. For example, when user gives a few symptoms and planner might select an incorrect specialist for appointment. To tackle this, we allow the planner to counter questions and prompt gpt-4o-mini to generate data where user inputs vague symptoms at first.6.4 Data Interleaving The trajectory consists of a sequence of states that include system, user, planner, and caller interac- tions. To create datasets for the Planner and Caller agents, we employ data interleaving. For each Plan- ner state in the trajectory, we select the tools, pre- vious interactions, and observations from function calls as input, while the reason and action are cho- sen as output. Similarly, for each Caller state, the function call and its parameters are set as output. In simpler terms, we extract relevant information from the trajectory to create training data for the Planner and Caller agents. This process involves identifying the appropriate input and output for each agent based on their specific roles within the sequence of states. By creating these datasets, we enable the fine-tuning of the Planner and Caller models to improve their performance in generating accurate reasoning, actions, and function calls. We enclose the reason, action, function and pa- rameters in tags for convenient post-processing. We also add these as special tokens in the model. For example, the output of planner for booking a specialist for the user: <reason >Dr. Gabriel Lopez ( Neurologist ) is available on the user 's preferred date and time .</ reason > <action > Suggest the appointment to the user and proceed with booking if confirmed . Dr. Gabriel Lopez ( Neurologist ) on 2024 -12 -01 between 10:00 -10:30 </ action > 7 Experimental Setup and Results We fine-tuned the Qwen2.5-Coder-7B-Instruct model for our multi-agent healthcare assistant us- ing synthetic data and LoRA adaptation with rank 8 and alpha 16. Context length was set to 4,096 tokens. The model was trained for 5,000 steps with a batch size of 64. By fine-tuning on separate Plan- ner and Caller datasets, we achieved a modular sys- tem for edge device deployment, with each LoRA adapter taking only 10MB before compression. We evaluate the performance of our multi-agent healthcare system using distinct metrics for the Planner and Caller agents. For the Planner, we assess the correctness of its reasoning and action using BLEU and ROUGE scores. ROUGE and BLEU scores are apt for checking the correctness of the Planner’s reasoning and action as they quan- titatively assess the similarity between the gener- ated text and reference text, providing a standard- ized way to evaluate the quality of the Planner’s Page 8: Category BLEU Rouge1 Rouge2 RougeL General 94.79 0.97 0.95 0.97 Counter 78.97 0.84 0.76 0.83 Negative 78.05 0.86 0.76 0.84 Dietician 74.44 0.80 0.71 0.78 Table 2: Evaluation results of Planner for appointment usecase. General: Appointment booked after finding specialist for the symptoms, Counter: Follow-up questions asked to understand symptoms better, Negative: User declines the appointment, Dietician: Use is referred to dietician based on the symptoms Category BLEU Rouge1 Rouge2 RougeL ToolAcc ParamAcc ValuesAcc General 98.88 1.0 0.99 0.99 1.0 1.0 0.9 Counter 96.95 0.98 0.96 0.98 1.0 0.99 0.77 Negative 89.64 0.92 0.88 0.92 1.0 0.93 0.57 Dietician 96.06 0.98 0.95 0.97 1.0 1.0 0.76 Table 3: Evaluation results of Caller for appointment usecase. General: Appointment booked after finding specialist for the symptoms, Counter: Follow-up questions asked to understand symptoms better, Negative: User declines the appointment, Dietician: Use is referred to dietician based on the symptoms responses. In the case of the Caller, we measure the accuracy of the tool called (ToolAcc), parameters passed to the tool (ParamAcc) and values of the pa- rameters (ValuesAcc). We also employ BLEU and ROUGE scores for the Caller, as they prove valu- able when a natural language sentence is provided as input to the tool. On our test set, we achieve an average RougeL score of 85.5 for planner and 96.5 for caller for the appointment usecase. Notably, the Caller consis- tently predicts the correct tool. For detailed scores for appointment usecase, refer to Table 2 and Table 3. For SOS usecases, we observe 100% scores for both planner and caller because of a fixed sequence of actions. 8 Conclusion Our multi-agent healthcare system demonstrates the potential for AI-powered assistants to revolu- tionize personal health management. By leveraging the power of large language models and fine-tuning them for specific tasks, we have created a modular and efficient system that can effectively support users in managing their health and well-being. The Action Manager, Health Manager, and Mem- ory Unit work together to provide a personalized healthcare experience. The Action Manager coor- dinates tasks, the Health Manager monitors health and manages schedules, while the Memory Unit stores and retrieves user-specific information for context-aware assistance. Through extensive data generation, enhance-ment, and fine-tuning, we have achieved a RougeL scores of 85.5 for planning and 96.5 for calling. This demonstrates the system’s ability to generate relevant and accurate responses, while the Caller’s consistent prediction of the correct tool showcases its reliability in executing tasks. 9 Future Work We plan to enhance our multi-agent system by con- verting the trained modules into on-device compat- ible formats like ONNX/GGUF, enabling smooth system integration into mobile devices and lower latency. We’ll also integrate the system with var- ious health monitoring devices and provide sup- port for user onboarding of additional devices like smart rings and smart weighing scales. To enhance healthcare capabilities, we’ll extend the system into a multimodal understanding engine, empowering it to perform image-based diagnosis. Furthermore, we aim to expand the system to cover additional targeted scenarios, such as women’s health tracking and post hospitalization care. References Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. 2023. Llm in a flash: Efficient large language model inference with limited memory. arXiv preprint arXiv:2312.11514 . Xinyun Chen, Ryan A. Chi, Xuezhi Wang, and Denny Page 9: Zhou. 2024. Premise order matters in reasoning with large language models. Preprint , arXiv:2402.08939. Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruim- ing Tang, and Enhong Chen. 2024. Understanding the planning of llm agents: A survey. Preprint , arXiv:2402.02716. Xun Jiang, Feng Li, Han Zhao, Jiaying Wang, Jun Shao, Shihao Xu, Shu Zhang, Weiling Chen, Xavier Tang, Yize Chen, Mengyue Wu, Weizhi Ma, Mengdi Wang, and Tianqiao Chen. 2024. Long term mem- ory: The foundation of ai self-evolution. Preprint , arXiv:2410.15665. Yingwei Ma, Yue Liu, Yue Yu, Yuanliang Zhang, Yu Jiang, Changjian Wang, and Shanshan Li. 2023. At which training stage does code data help llms reasoning? Preprint , arXiv:2309.16298. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Preprint , arXiv:2302.04761. Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. 2024. Cognitive architectures for language agents. Preprint , arXiv:2309.02427. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 . Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yi- hao Feng, Tulika Awalgaonkar, Rithesh Murthy, Eric Hu, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, and Caiming Xiong. 2024a. xlam: A family of large ac- tion models to empower ai agent systems. Preprint , arXiv:2409.03215. Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang, Lichang Chen, William Yang Wang, and Linda Ruth Petzold. 2024b. Unveiling the impact of coding data instruction fine-tuning on large language models rea- soning. Preprint , arXiv:2405.20535. Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji- Rong Wen. 2024c. A survey on the memory mecha- nism of large language model based agents. Preprint , arXiv:2404.13501. A Snapshots of Application In this section, we present snapshots showcasing the key features of our application. These include intelligent appointment scheduling based on usersymptoms, reminder scheduling by extracting pre- scription information, soft SOS triggers for abnor- mal vitals, and user-initiated hard SOS with loca- tion tracking and emergency notifications. A.1 Intelligent Appointment Scheduling Figure 4 shows the appointment booking use case where the user complains about having abdominal pain and feeling feverish. The Planner and Caller ask follow-up questions and find a specialist that fits the user’s schedule. A.2 Reminder Scheduling from Prescription As shown in figure 5a, the user can upload a pre- scription. The prescription is parsed and analyzed by LLM to extract information about medication such as medicine name, time, and duration. This information is then added to the user’s list of re- minders, as shown in figure 5b. A.3 Soft Trigger from Abnormal Vitals If a user’s vitals are abnormal, the monitor agent triggers a soft SOS. At that time, the user’s vitals are fetched (simulated for our demo) and a small report is displayed to the user (refer to figure 6). A.4 Hard SOS Triggered by User The user can trigger a Hard SOS as shown in figure 7a. The user’s location is fetched and the nearest ambulance is identified. The location is then sent to the nearest ambulance and emergency contacts. A sample of the message sent to an emergency contact is shown in figure 7b. B Experimentation with function calling models We conducted experiments with xLAM 1B and 7B models to assess their suitability for planning and calling tasks. As illustrated in Figure 10, we pro- vided a list of 28 tools for the task of finding good movies in nearby theaters. However, both the 1B and 7B models failed to identify the correct func- tion calls in steps 1 and 2, respectively (refer figure 8 and 9). Specifically, the 7B model attempted to check movie availability before fetching the list of movies. To address these issues, we fine-tuned the Qwen model to improve its performance and avoid such situations. C List of Diseases During data generation, the prompt instructs to generate symptoms based on a given disease. We Page 10: Figure 4: Appointment Booking (a) Medication Uploaded schedule Reminders (b) Reminder Added Figure 5: Adding Reminder from Prescription (a) Soft SOS triggered due to abnormal vitals (simulated) (b) User’s vitals analyzed Figure 6: Soft SOS triggered and vitals analyzed Page 11: (a) Hard SOS-Interface (b) Hard SOS-SMS Figure 7: Triggering Hard SOS and notifying emergency contacts with GPS Figure 8: Planning and function calling using xLAM 1b Figure 9: Planning and function calling using xLAM 7b Page 12: Figure 10: Prompt to xLAM 7b randomly sample from a list of 153 diseases. The list was created based on different specialization such as orthologist, cardiologist, gastroenterolo- gists, etc. They are as follows: Influenza, Diabetes Mellitus, Hypertension, Asthma, Coronary Artery Disease, Stroke, HIV/AIDS, Tuberculosis, Hepati- tis B, Hepatitis C, Dengue Fever, Malaria, Chicken- pox, Measles, Mumps, Rubella, Zika Virus, Ebola, COVID-19, Cancer, Alzheimer’s Disease, Parkin- son’s Disease, Arthritis, Lupus, Multiple Sclero- sis, Crohn’s Disease, Ulcerative Colitis, Gastroe- sophageal Reflux Disease (GERD), Celiac Disease, Anemia, Bronchitis, Pneumonia, Sinusitis, Chlamy- dia, Gonorrhea, Syphilis, Herpes, Human Papillo- mavirus (HPV), Tetanus, Rabies, Polio, Yellow Fever, Smallpox, Typhoid Fever, Leprosy, Plague, Hemophilia, Sickle Cell Disease, Thalassemia, Hemochromatosis, Meningitis, Encephalitis, Glau- coma, Cataracts, Macular Degeneration, Otitis Me- dia, Tonsillitis, Appendicitis, Hemorrhoids, Diver- ticulitis, Irritable Bowel Syndrome (IBS), Kidney Stones, Chronic Kidney Disease, Renal Failure, Osteoporosis, Osteopenia, Rheumatoid Arthritis, Gout, Psoriasis, Eczema, Vitiligo, Alopecia, Hy- pothyroidism, Hyperthyroidism, Cushing’s Syn- drome, Addison’s Disease, Prostate Cancer, Breast Cancer, Lung Cancer, Colorectal Cancer, Pancre- atic Cancer, Liver Cancer, Skin Cancer, Melanoma, Leukemia, Lymphoma, Myeloma, Basal Cell Car- cinoma, Squamous Cell Carcinoma, EsophagealCancer, Gastric Cancer, Ovarian Cancer, Cervi- cal Cancer, Endometrial Cancer, Bladder Cancer, Kidney Cancer, Testicular Cancer, Thyroid Can- cer, Mesothelioma, Brain Tumors, Spinal Cord Tu- mors, Heart Failure, Atrial Fibrillation, Cardiomy- opathy, Angina, Atherosclerosis, Pericarditis, My- ocarditis, Endocarditis, Congenital Heart Disease, Valvular Heart Disease, Peripheral Artery Disease, Deep Vein Thrombosis, Pulmonary Embolism, Varicose Veins, Hypertensive Heart Disease, Coro- nary Microvascular Disease, Dyslipidemia, Famil- ial Hypercholesterolemia, Hypertrophic Cardiomy- opathy, Dilated Cardiomyopathy, Restrictive Car- diomyopathy, Takotsubo Cardiomyopathy, Sleep Apnea, Insomnia, Narcolepsy, Restless Leg Syn- drome, Epilepsy, Migraine, Cluster Headache, Ten- sion Headache, Trigeminal Neuralgia, Multiple System Atrophy, Amyotrophic Lateral Sclerosis (ALS), Huntington’s Disease, Dystonia, Myasthe- nia Gravis, Guillain-Barre Syndrome, Spinal Mus- cular Atrophy, Muscular Dystrophy, Fibromyal- gia, Chronic Fatigue Syndrome, Complex Regional Pain Syndrome, Temporomandibular Joint Dis- orders, Sjogren’s Syndrome, Raynaud’s Disease, Scleroderma, Mixed Connective Tisse Disease, Be- hcet’s Disease, Wegener’s Granulomatosis, Giant Cell Arteritis, Polyarteritis Nodosa and Sarcoido- sis. Page 13: D Example of Trajectory D.1 Appointment - Generic { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " JICC571413 ", " name ": " Sakura Tominaga ", " timestamp ": "2024 -09 -02 T10 :57:00" } } }, { " from ": " user ", " value ": "I 've been feeling extremely fatigued with chills , body aches , and a sore throat . It's becoming hard to get through the day ." }, { " from ": " planner ", " value ": { " reason ": " User reports fatigue , chills , body aches , and a sore throat which could indicate a viral infection such as influenza .", " action ": " Check for past similar complaints using the retrieve_past_complaints tool ." } }, { " from ": " caller ", " value ": { " tool ": " retrieve_past_complaints ", " parameters ": { " user_id ": " JICC571413 ", " symptoms ": " fatigue , chills , body aches , sore throat " } } }, { " from ": " observation ", " value ": { " result ": [ { " date ": "2024 -06 -02" , " symptoms ": " mild body aches , slight fever " } ] } }, { " from ": " planner ", " value ": { " reason ": " Past records indicate a mild common cold , but the current symptoms are more severe . A consultation with a general physician is advisable ."," action ": " Use the get_available_specialists tool to find a general physician available for an appointment ." } }, { " from ": " caller ", " value ": { " tool ": " get_available_specialists ", " parameters ": { " symptoms ": " fatigue , chills , body aches , sore throat ", " specialization ": " general physician " } } }, { " from ": " observation ", " value ": { " result ": { " specialist_id ": " AECJ317777 ", " name ": "Dr. Diego Arroyo ( General Physician )", " available_slot ": { " date ": "2024 -11 -30" , " time ": "11:00 -11:30" } } } }, { " from ": " planner ", " value ": { " reason ": "Dr. Diego Arroyo ( General Physician ) is available on the user 's preferred date and time .", " action ": " Suggest the appointment to the user and proceed with booking if confirmed . Dr. Diego Arroyo ( General Physician ) on 2024 -11 -30 between 11:00 -11:30" } }, { " from ": " caller ", " value ": { " tool ": " get_input_from_user ", " parameters ": { " user_id ": " JICC571413 ", " questions ": "Dr. Diego Arroyo ( General Physician ) has an opening at 11:00 AM on November 30 th. Would you like to schedule an appointment ?" } } }, { " from ": " observation ", " value ": { " result ": "Yes , please " } }, { " from ": " planner ", " value ": { Page 14: " reason ": " User has confirmed the appointment ", " action ": "I should confirm the appointment " } }, { " from ": " caller ", " value ": { " tool ": " confirm_appointment ", " parameters ": { " user_id ": " JICC571413 ", " specialist_id ": " AECJ317777 ", " appointment_time_date ": "11:00 -11:30 , 30/11/2024" } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " Appointment confirmed . Next , I should save the appointment history .", " action ": " Save the appointment history ." } }, { " from ": " caller ", " value ": { " tool ": " save_appointment_history ", " parameters ": { " user_id ": " JICC571413 ", " symptoms ": " fatigue , chills , body aches , sore throat ", " specialist_id ": " AECJ317777 ", " appointment_time_date ": "11:00 -11:30 , 30/11/2024" } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " Appointment confirmed and stored .", " action ": " Also store the current symptoms for future reference ." } }, { " from ": " caller ", " value ": { " tool ": " store_symptoms ", " parameters ": { " user_id ": " JICC571413 "," symptoms ": " fatigue , chills , body aches , sore throat ", " timestamp ": "2024 -09 -02 T10 :57:00" } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " Appointment confirmed and symptoms stored .", " action ": " Inform the user of the successful booking ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " JICC571413 ", " message ": " Your appointment with Dr. Diego Arroyo ( General Physician ) is confirmed for 11:00 AM on November 30 th. Your symptoms have been recorded for future reference ." } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The task is completed successfully .", " action ": "<END >" } } ] } D.2 Appointment - Reject Appointment { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " BKYA569367 ", " name ": " Tomomi Gao ", " timestamp ": "2023 -11 -08 T10 :54:00" } } }, { Page 15: " from ": " user ", " value ": "I 've been feeling very unwell with a high fever , body aches , constant coughing , and sore throat for the past few days ." }, ... ( similar to Appointment Trajectory ) { " from ": " observation ", " value ": { " result ": { " specialist_id ": " CXAE230642 ", " name ": "Dr. Diego Perez ( General Practitioner )", " available_slot ": { " date ": "2024 -11 -30" , " time ": "10:00 -10:30" } } } }, { " from ": " planner ", " value ": { " reason ": "Dr. Diego Perez ( General Practitioner ) is available on the user 's preferred date and time .", " action ": " Suggest the appointment to the user and check if they want to schedule it. Dr. Diego Perez ( General Practitioner ) on 2024 -11 -30 between 10:00 -10:30" } }, { " from ": " caller ", " value ": { " tool ": " get_input_from_user ", " parameters ": { " user_id ": " BKYA569367 ", " questions ": "Dr. Diego Perez ( General Practitioner ) has an opening at 10:00 AM on November 30 th. Would you like to schedule an appointment ?" } } }, { " from ": " observation ", " value ": { " result ": "No , not at this time ." } }, { " from ": " planner ", " value ": { " reason ": " User has declined the appointment .", " action ": " Proceed to store the symptoms for future reference without scheduling an appointment ." } }, {" from ": " caller ", " value ": { " tool ": " store_symptoms ", " parameters ": { " user_id ": " BKYA569367 ", " symptoms ": " high fever , body aches , cough , sore throat ", " timestamp ": "2023 -11 -08 T10 :54:00" } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " Symptoms stored successfully .", " action ": " Inform the user that the symptoms have been recorded for future reference ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " BKYA569367 ", " message ": " Your symptoms have been recorded for future reference . Please don 't hesitate to reach out if you decide to see a doctor ." } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The process is completed with symptoms stored and user notified .", " action ": "<END >" } } ] } D.3 Appointment - Follow-up for Unclear Symptoms { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " BLTA888285 ", Page 16: " name ": " Juan Martinez ", " timestamp ": "2024 -08 -05 T04 :20:00" } } }, { " from ": " user ", " value ": "I 've been having some trouble with my movement lately ." }, { " from ": " planner ", " value ": { " reason ": " User reports movement issues , but I need to clarify if it 's weakness , stiffness , or something else .", " action ": " Ask user if they are experiencing weakness , stiffness , or difficulties in coordination ." } }, { " from ": " caller ", " value ": { " tool ": " get_input_from_user ", " parameters ": { " user_id ": " BLTA888285 ", " questions ": " Could you describe if you 're feeling weakness , stiffness , or issues with coordination ?" } } }, { " from ": " observation ", " value ": { " result ": { " user ": "I feel a bit weak and sometimes my hand shakes ." } } }, { " from ": " planner ", " value ": { " reason ": " User mentions weakness and hand shaking , which could indicate a need for rehabilitation .", " action ": " Ask if they are experiencing any pain or if the weakness is constant ." } }, { " from ": " caller ", " value ": { " tool ": " get_input_from_user ", " parameters ": { " user_id ": " BLTA888285 ", " questions ": " Are you experiencing any pain , or is the weakness constant throughout the day ?" } } },{ " from ": " observation ", " value ": { " result ": { " user ": "There 's no pain , but the weakness seems to come and go ." } } }, .... ( Rest similar to Appointment Trajectory ) ] } D.4 Soft SOS { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " PKFG301655 ", " name ": " Aaliyah Sousa ", " timestamp ": "2025 -02 -01 T08 :11:00" } } }, { " from ": " user ", " value ": " Soft SOS triggered . Abnormal Vitals : { 'oxygen ': 85, 'heart_rate ': 41, 'sleep ': {' deep ': 75, 'light ': 238 , 'rem': 94, 'awake ': 44}}" }, { " from ": " planner ", " value ": { " reason ": " User has triggered a soft SOS , so the system needs to notify the user about abnormal vitals ", " action ": " Call notify_user to inform the user that the SOS process is being triggered ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " PKFG301655 ", " symptoms ": " Soft SOS triggered . Abnormal vitals detected .\ nIf you are feeling unwell , contact emergency services or book an appointment .\n\ nYour Vitals -\ nHeart Rate : 41 bps \ nOxygen : 85%\ nDeep Sleep : 75 minutes \ nLight Sleep : 238 minutes \ nREM : 94 minutes \ nAwake : 44 minutes " } } }, { Page 17: " from ": " planner ", " value ": { " reason ": " The user has been informed about the abnormal vitals . The Soft SOS process is completed successfully .", " action ": "<END >" } } ] } D.5 Hard SOS - Start { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " HNNT232992 ", " name ": " Jace Cardoso ", " timestamp ": "2024 -03 -13 T06 :59:00" } } }, { " from ": " user ", " value ": " Hard SOS triggered " }, { " from ": " planner ", " value ": { " reason ": " User has triggered a hard SOS , so the system needs to notify the user that the SOS process is starting .", " action ": " Call notify_user to inform the user that the SOS process is being triggered ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " HNNT232992 ", " symptoms ": " Hard SOS triggered . We are initiating emergency response procedures ." } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The user has been notified about the SOS initiation . The system now needs to retrieve the user 's location to proceed with the emergency response ."," action ": " Call get_location to fetch the user 's current coordinates ." } }, { " from ": " caller ", " value ": { " tool ": " get_location ", " parameters ": {} } }, { " from ": " observation ", " value ": { " result ": { " latitude ": 23.5326 , " longitude ": 139.7524 } } }, { " from ": " planner ", " value ": { " reason ": " The user 's current location has been retrieved . Next , the system needs to find the nearest ambulance based on this location .", " action ": " Call search_ambulance with the location to find the nearest available ambulance ." } }, { " from ": " caller ", " value ": { " tool ": " search_ambulance ", " parameters ": { " location ": { " latitude ": 23.5326 , " longitude ": 139.7524 } } } }, { " from ": " observation ", " value ": { " result ": { " ambulance_id ": " AMBpF0E ", " phone_no ": "+146910850030" } } }, { " from ": " planner ", " value ": { " reason ": " The nearest ambulance has been identified . The system needs to notify the ambulance about the user 's location and details .", " action ": " Call send_message to notify the ambulance with the user 's location and details ." } }, { " from ": " caller ", Page 18: " value ": { " tool ": " send_message ", " parameters ": { " phone_no ": "+146910850030" , " text ": " Ambulance needed at location { latitude : 23.5326 , longitude : 139.7524} by user HNNT232992 - Jace Cardoso " } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The ambulance has been notified . Next , the system needs to alert the user 's emergency contacts about the SOS .", " action ": " Call send_message to notify emergency contacts about the SOS and user 's location ." } }, { " from ": " caller ", " value ": { " tool ": " send_message ", " parameters ": { " text ": " SOS triggered by Jace Cardoso at location { latitude : 23.5326 , longitude : 139.7524}" , " to_emergency_contacts ": true } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The emergency contacts have been notified . The system now needs to inform the user about the actions taken , including ambulance and emergency contact notifications .", " action ": " Call notify_user to inform the user about the completion of the SOS process ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " HNNT232992 "," symptoms ": " Ambulance ( AMBpF0E ) with contact HNNT232992 has been informed and is on its way . Your emergency contacts have also been notified ." } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The user has been informed about the actions taken . The SOS process is completed successfully .", " action ": "<END >" } } ] } D.6 Hard SOS - End { " interaction_trajectory ": [ { " from ": " system ", " value ": { " user_details ": { " user_id ": " EORZ618635 ", " name ": " Leah Lima ", " timestamp ": "2023 -11 -29 T01 :52:00" } } }, { " from ": " user ", " value ": " End SOS triggered " }, { " from ": " planner ", " value ": { " reason ": " User has triggered an End SOS . The system needs to notify the user that the End SOS process is being initiated .", " action ": " Call notify_user to inform the user about the initiation of the End SOS process ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " EORZ618635 ", " symptoms ": " End SOS triggered . We are notifying all relevant parties ." } Page 19: } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The user has been informed about the initiation of the End SOS process . The system now needs to get the assigned ambulance details .", " action ": " Call get_assigned_ambulance to retrieve the ambulance details ." } }, { " from ": " caller ", " value ": { " tool ": " get_assigned_ambulance ", " parameters ": { " user_id ": " EORZ618635 " } } }, { " from ": " observation ", " value ": { " result ": { " ambulance_id ": " AMBUaTg ", " phone_no ": "+235135781046" } } }, { " from ": " planner ", " value ": { " reason ": " The assigned ambulance details have been retrieved . The system now needs to notify the ambulance about the SOS being ended .", " action ": " Call send_message to inform the ambulance about the End SOS ." } }, { " from ": " caller ", " value ": { " tool ": " send_message ", " parameters ": { " phone_no ": "+235135781046" , " text ": " The SOS triggered by user EORZ618635 - Leah Lima has been resolved . Ambulance services are no longer required ." } } }, { " from ": " observation ", " value ": {" result ": true } }, { " from ": " planner ", " value ": { " reason ": " The ambulance has been notified about the End SOS . Next , the system needs to notify the user 's emergency contacts about the resolution .", " action ": " Call send_message to inform emergency contacts about the SOS resolution ." } }, { " from ": " caller ", " value ": { " tool ": " send_message ", " parameters ": { " text ": " The SOS triggered by Leah Lima has been resolved . No further assistance is required .", " to_emergency_contacts ": true } } }, { " from ": " observation ", " value ": { " result ": true } }, { " from ": " planner ", " value ": { " reason ": " The emergency contacts have been notified about the SOS resolution . The system now needs to inform the user that the End SOS process is completed .", " action ": " Call notify_user to inform the user about the successful completion of the End SOS process ." } }, { " from ": " caller ", " value ": { " tool ": " notify_user ", " parameters ": { " user_id ": " EORZ618635 ", " symptoms ": " The SOS process has been successfully ended . Ambulance and emergency contacts have been informed ." } } }, { " from ": " observation ", " value ": { " result ": true } }, Page 20: { " from ": " planner ", " value ": { " reason ": " The user has been informed about the successful completion of the End SOS process . The task is completed .", " action ": "<END >" } } ] } E List of Tools Following is a list of tools for which planner and caller is trained: [ { " Name ": " get_location ", " Description ": " Fetches the GPS coordinates of the user 's current location .", " Parameters ": [], " Required Parameters ": [], " Returns ": { " type ": " dictionary ", " description ": " Returns the user 's current location as { latitude : float , longitude : float }." } }, { " Name ": " search_ambulance ", " Description ": " Finds the nearest available ambulance based on the given location .", " Parameters ": [ { " param_name ": " location ", " type ": " dictionary ", " default ": " None ", " description ": " The user 's current location as { latitude : float , longitude : float }." } ], " Required Parameters ": [ " location " ], " Returns ": { " type ": " dictionary ", " description ": " Returns the ambulance details { ambulance_id : string , phone_no : string }." } }, { " Name ": " send_message ", " Description ": " Sends a text message to a specified phone number or emergency contacts .", " Parameters ": [ { " param_name ": " phone_no ", " type ": " string ", " default ": " None "," description ": " The phone number to send the message to ." }, { " param_name ": " text ", " type ": " string ", " default ": " None ", " description ": " The message text to send ." }, { " param_name ": " to_emergency_contacts ", " type ": " boolean ", " default ": " False ", " description ": " Whether to send the message to all emergency contacts ." } ], " Required Parameters ": [ " text " ], " Returns ": { " type ": " boolean ", " description ": " Returns True if the message was sent successfully ." } }, { " Name ": " get_available_specialists ", " Description ": " Fetches a list of specialists and their availability .", " Parameters ": [ { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " List of symptoms derived from user input ." }, { " param_name ": " specialization ", " type ": " string ", " default ": " None ", " description ": " Specialization name for the appointment ." }, { " param_name ": " user_schedule ", " type ": " object ", " default ": " None ", " description ": "User 's preferred schedule for appointments in key - value pairs (e.g., { 'date ':' YYYY -MM -DD ','time_range ':'HH: MM -HH:MM '}) ." } ], " Required Parameters ": [ " symptoms ", " user_schedule " ], " Returns ": { " type ": " dictionary ", " description ": " Returns single best schedule { specialist_id : string , name : string , available_slot : string , date : string }." Page 21: } }, { " Name ": " confirm_appointment ", " Description ": " Confirms an appointment slot and stores it in the hospital 's database .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " specialist_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the chosen specialist ." }, { " param_name ": " appointment_time_date ", " type ": " string ", " default ": " None ", " description ": " The selected time slot for the appointment as time in HH:MM -HH:MM format and date in DD/MM/YY format ." } ], " Required Parameters ": [ " user_id ", " specialist_id ", " appointment_time_date " ], " Returns ": { " type ": " boolean ", " description ": " Always returns True ." } }, { " Name ": " save_appointment_history ", " Description ": " Saves appointment information in the user 's database for later reference and recurring use cases .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " The symptoms described by the user ." }, { " param_name ": " specialist_id ", " type ": " string ", " default ": " None "," description ": " Unique identifier for the chosen specialist ." }, { " param_name ": " appointment_time_date ", " type ": " string ", " default ": " None ", " description ": " The selected time slot for the appointment as time in HH:MM -HH:MM format and date in DD/MM/YY format ." } ], " Required Parameters ": [ " user_id ", " symptoms ", " specialist_id ", " appointment_time_date " ], " Returns ": { " type ": " boolean ", " description ": " Always returns True ." } }, { " Name ": " get_available_specialists ", " Description ": " Fetches a list of specialists and their availability .", " Parameters ": [ { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " List of symptoms derived from user input ." }, { " param_name ": " specialization ", " type ": " string ", " default ": " None ", " description ": " specialization name for the appointment " }, { " param_name ": " user_schedule ", " type ": " object ", " default ": " None ", " description ": "User 's preferred schedule for appointments in key - value pairs (e.g., { 'date ':' YYYY -MM -DD ','time_range ':'HH: MM -HH:MM '}) ." } ], " Required Parameters ": [ " symptoms ", " specialization " ], " Returns ": { " type ": " dictionary ", " description ": " returns single best schedule { specialist_id ,name , available_slot including time in HH:MM -HH:MM format and date in DD/MM/YY format }." } }, Page 22: { " Name ": " confirm_appointment ", " Description ": " Confirms an appointment slot and stores it in the hospital 's database .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " specialist_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the chosen specialist ." }, { " param_name ": " appointment_time_date ", " type ": " string ", " default ": " None ", " description ": " The selected time slot for the appointment as time in HH:MM -HH:MM format and date in DD/MM/YY format ." } ], " Required Parameters ": [ " user_id ", " specialist_id ", " appointment_time " ], " Returns ": { " type ": " boolean ", " description ": " Always returns True ." } }, { " Name ": " save_appointment_history ", " Description ": " Saves appointment information in the user 's database for later reference and recurring use cases .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " The symptoms described by the user ." }, { " param_name ": " specialist_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the chosen specialist ." },{ " param_name ": " appointment_time_date ", " type ": " string ", " default ": " None ", " description ": " The selected time slot for the appointment as time in HH:MM -HH:MM format and date in DD/MM/YY format ." } ], " Required Parameters ": [ " user_id ", " symptoms ", " specialist_id ", " appointment_time_date " ], " Returns ": { " type ": " boolean ", " description ": " Always true " } }, { " Name ": " get_appointment_history ", " Description ": " Retrieves the user 's appointment history for analysis and reminders .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." } ], " Required Parameters ": [ " user_id " ], " Returns ": { " type ": " array ", " description ": " Array containing past appointment records ." } }, { " Name ": " retrieve_past_complaints ", " Description ": " Fetches the user 's past complaints matching the given symptoms for analysis and reference .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " List of symptoms to search for in past complaints ." }, { " param_name ": " date_range ", " type ": " object ", " default ": " None ", Page 23: " description ": " Optional date range filter in the format { ' start_date ':'YYYY -MM -DD ',' end_date ':'YYYY -MM -DD '}." } ], " Required Parameters ": [ " user_id ", " symptoms " ], " Returns ": { " type ": " array ", " description ": " Array of past complaints related to the specified symptoms ." } }, { " Name ": " follow_up_with_user ", " Description ": " Initiates a follow -up interaction with the user based on their past complaints and current symptoms .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " past_complaints ", " type ": " array ", " default ": "[]" , " description ": " List of past complaints to reference during the follow -up ." }, { " param_name ": " current_symptoms ", " type ": " string ", " default ": " None ", " description ": " Current symptoms reported by the user ." }, { " param_name ": " preferred_contact_method ", " type ": " string ", " default ": " None ", " description ": "User 's preferred method for follow -up (e.g., ' call ','email ','chat ') ." } ], " Required Parameters ": [ " user_id ", " current_symptoms " ], " Returns ": { " type ": " object ", " description ": " Details of the follow -up initiated , including method and next steps ." } }, { " Name ": " notify_user "," Description ": " Sends a notification to the user ", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " message ", " type ": " string ", " default ": " None ", " description ": " The notification message to send ." } ], " Required Parameters ": [ " user_id ", " message " ], " Returns ": { " type ": " boolean ", " description ": " Always returns status as True " } }, { " Name ": " get_input_from_user ", " Description ": " Collects input from the user for specified parameters .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " questions ", " type ": " string ", " default ": "[]" , " description ": " Question to ask the user ." } ], " Required Parameters ": [ " user_id ", " questions " ], " Returns ": { " type ": " string ", " description ": "User 's response to the specified questions ." } }, { " Name ": " store_symptoms ", " Description ": " Stores the symptoms reported by the user and initiates a follow -up process to gather additional details for a more accurate analysis .", " Parameters ": [ { " param_name ": " user_id ", " type ": " string ", Page 24: " default ": " None ", " description ": " Unique identifier for the user ." }, { " param_name ": " symptoms ", " type ": " string ", " default ": " None ", " description ": " Symptoms of user ." }, { " param_name ": " timestamp ", " type ": " object ", " default ": " None ", " description ": " Save date and time of the event ", " Required Parameters ": [ " user_id ", " symptoms ", " timestamp " ], " Returns ": { " type ": " boolean ", " description ": " Status always true " } } ] } ]