Build an Expert Agent

Business users can leverage the power of an assistant to streamline their workflow, increase efficiency, and ultimately enhance their ability to understand and serve their clients effectively. By harnessing the capabilities of an assistant to help manage emails, organize databases, and facilitate preparation for client meetings, financial advisors can save time and focus on what truly matters – building strong relationships and providing valuable financial advice. Join us as we dive into practical strategies and tools to optimize productivity and achieve success in the fast-paced world of financial planning.

Sample Use Case: While serving a client, partners often grapple with manually retrieving client details from emails, company databases, and appointment schedules to assess ISA allowances and provide tailored recommendations. This cumbersome process can be time-consuming and prone to errors. However, you can create an Advice Assistant to revolutionize this workflow by automating data retrieval and consolidation. With a simple client ID input, partners can instantly access comprehensive information, including past meetings and notes, ISA allowances and scheduled appointments, through the application. This automation enables partners to shift their focus from administrative tasks to delivering insightful recommendations and creating a more personalized client experience, ultimately elevating client engagement and satisfaction.

Users must have the Gen AI User policy to access the journey on how to build an expert agent. 

This guide will walk you through the steps on how to build an expert agent with the help of Purple Fabric.

1. Create an asset
2. Select a prompt template
3. Select a model and set model configurations
4. Provide the system instruction, action and examples
5. Run the model and view results
6. Validate and benchmark the asset
7. Publish the asset
8. Consume the asset

Step 1: Create an asset

1. Head to the Gen AI Studio module and click Create Asset.
   
   
   
   
2. In the Create Gen AI asset window that appears, enter a unique Asset name, for example, “Advice_Assistant” to easily identify it within the platform.
   
   
   
   
3. Optional: Enter a brief description and upload an image to provide additional context or information about your Asset.
4. In Type, choose the Conversational Agent and click Create.

Step 2: Select a prompt template

1. On the Gen AI Asset creation page that appears, choose ReAct template.
   
   
   
   
   For more information on Default, RAG and ReAct templates, see Basics of Prompt Engineering course.

Step 3: Select a model and set model configurations

1. Select a model from the available list, taking into account aspects such as model size and performance.
   
   
   
   
2. Click and then set the following tuning factors/parameters to optimize its performance, if you wish to fine-tune the configurations of your Conversational Agent. 

Note: Initially, keep the factors at their default levels and run the prompt to assess if the answer aligns with your expectations. If you desire a more creatively crafted answer, consider increasing the temperature, top_p, and top_k slightly, as this may enhance the output. However, if the model begins to produce excessively quirky responses, maintain a high temperature while adjusting the top_p/top_k settings for more controlled results.

• temperature: This parameter controls the level of randomness or creativity in the AI-generated text. Lower temperatures produce more conservative and predictable responses, while higher temperatures yield more diverse and unpredictable outputs.
  
  
• top_k: This parameter limits the AI model to considering only the top k most probable words for each token generated, aiding in controlling the generation process. For example, setting top_k to 10 means only the top 10 most likely words will be considered for each word generated.
  
  
• top_p: This parameter sets a threshold for cumulative probability during word selection, refining content by excluding less probable words. For example, setting top_p to 0.7 ensures words contributing to at least 70% of likely choices are considered, refining responses.

Step 4: Provide the system instructions, Knowledge Base and Examples

Provide System Instructions 

A system instruction refers to a command or directive provided to the model to modify its behavior or output in a specific way. For example, a system instruction might instruct the model to summarize a given text, answer a question in a specific format, or generate content with a particular tone or style.

1. Enter the system instructions by crafting a prompt that guides the agent in helping advisors.
    
   
   
   

Add Actions

1. In the Actions, click on Add.
   
   
   
   
2. In the Actions window that appears,  use the Search bar to find the required tools. For example, Advisor meeting details 1, client and plan details and Get IST Time and click Add.
   
   
   
   
3. After adding the tool, provide a description so that LLM can understand the context better.
   
   
   
   

Provide Examples

1. Examples help the content creation task at hand to enhance the agent’s understanding and response accuracy. These examples help the agent learn and improve over time.
   
   
2. In the Examples section, click Add. 
   
   
   
   
   
3. Enter the example Question, Thought, Action, Action Input, Observation, Thought and Final Answer.
   
   
   
   

Step 5: Run the model and view results

1. In the Debug and Preview section, enter the prompt in the query bar to seek the required answers.
   
   
   
   
2. Click or press Enter key to run the prompt. 
   
   
   
   
3. Review the generated response to ensure it adequately addresses or clarifies your query.
4. If necessary, provide examples to enhance the conversational agent’s understanding and response accuracy for answering questions.

Note: If the answer falls short of your expectations, provide additional context or rephrase your prompt for better clarification.

Step 6: Validate and benchmark the asset

1. In the Debug and preview section, click Benchmark.
   
   
   
   
2. In the Benchmarks window that appears, click Start New to benchmark against predefined metrics to determine the most effective model.
   
   
   
   
3. In the Input and Expected output , enter the example input and the expected output.
   
   
   
   
4. Click to add another model to benchmark the response against.
   
   
   
   
5. Click and adjust the metrics such as temperature, top_k and top_p as required to compare the output of the models against each other.
   
   
   
   

6. Click Re-run prompt. 
   
   
   
   
7. Compare the response of the models based on the tokens, score, latency and cost which will determine which is the best suited model to be deployed for your use case.
   
8. Preview, like or dislike the results to notify fellow team members. 

Definition of Metrics

• “tokens used” typically refers to the number of these units processed to generate a response. Each token used consumes computational resources and may be subject to pricing.
• The score refers to the accuracy percentage which can be evaluated by comparing the model’s responses to a set of reference answers.
• Latency refers to the time delay between a user’s input and the model’s output.Latency can be influenced by various factors such as the complexity of the task, the model’s size, and the computational resources available. Lower latency indicates faster response times.
• Cost refers to the financial expenditure associated with using the language model. Costs vary based on the number of tokens processed, the level of accuracy required, and the computational resources utilized. 

Step 7: Publish the asset

1. If the desired accuracy and performance for getting answers from the document has been achieved, click Publish.
   
   
    
   
2. In the Asset Details page that appears, enter the Welcome Message, Conversation Starters and Asset Disclaimer.
   
   
   
   

• Name:Advice_Assistant_Asset
• Description: This asset is an intelligent advice assistant.
• Welcome Message: Welcome to the Advice Assistant for Advisors and Partners! Here, we empower you with the tools and information you need to excel in client meetings and planning sessions. Whether you’re looking for data, insights, or strategies, I’m here to assist you every step of the way. Let’s collaborate to ensure your clients receive the best guidance and solutions tailored to their needs. Your success is our priority!
• Conversation starters:
  • “What are the client and plan details related to ISA product for all the clients who has meeting with advisor “999999X”” 
  • “What are remaining allowance details for plans related to ISA product for all the clients who has meeting with advisor “999999X”. Show me the output in a tabular format including the details Client Name, Client Address, Plan Number, Remaining Allowance”
  • “For all clients meeting with advisor “999999X” today, how many clients have plans with ISA product?” 
  • Asset Disclaimer:  This Asset  provides responses based on the the tools that get time, client details and advisor details. While every effort is made to ensure accuracy, the information should be used as a supplementary guide and may not always reflect the most updated or specific details.

3. Optional: Upload a sample image for a visual representation.
4. Click Publish and the status of the Asset changes to Published then it can be accessed in the Gen AI Studio.
   
   
   

Step 8: Consume the asset

1.  Head to the GenAI Studio module. Use the Search bar to find an Asset.  
   
   
   
   
2. Select an Asset that you wish to consume.
   
   
   
   
3. In the Conversational Assistant that appears, initiate a conversation by asking the asset a question based on your documents. An example could be “show remaining allowance details for ISA products for all the clients that the advisor “999999X” meeting today?”