Generative AI Action
This action allows you to prompt the AI model, which will respond based on its inbuilt general knowledge or an uploaded source file. You can also provide detailed instructions to guide the AI’s response. Additionally, you can select from multiple registered AI models to suit your needs.
In this documentation chapter, let’s create a Generative AI block, that will answer questions about workplace policy.
Step 1. Create a Generative AI Action by pressing the button at the bottom of an Action Group :
Step 2. A new Generative AI Action will appear and be ready to be filled in. Let’s set up this block step by step.
A – Help Dialog
For guidance on the action setup, click the Information Icon. A help dialogue with instructions and examples will appear.
B – Select the Generative AI Destination
Select a Generative AI Destination (Model) for generating the response. You can choose between using the Default Generative AI Destination, which is assigned to the current AI Model in the Generative AI Settings section, or selecting a specific one from the list.
In this example, the Ollama Model is selected.
C – Set the Input Prompt
The Input Prompt field holds the question asked to the AI. It usually includes the User Phrase ( the exact phrase the user writes), which is combined with Instruction and Source Documents to create the Context for Generative AI.
You can enhance {{user_phrase}} by adding additional text or memory parameters. You can choose to remove {{user_phrase}} entirely. If removed, the Generative AI will generate output based solely on the instructions provided and will ignore user messages.
Example Input Prompt 1: {{user_phrase}}
Example Input Prompt 2: Answer this question: {{user_phrase}} in context of {{memory.rt_leave_request_type}}.
In this example, we only use {{user_phrase}} without any additions.
D – Write the Instruction
The text in the Instruction field guides Generative AI in crafting its response. Together with Input Prompt and Source Documents, it forms the Context for Generative AI. Attached Source Documents are added at the end of the Context by default. If you need to refer to the specific source document in the instruction body, you can use tag mask {{document.document_name}}, where document_name – the name of the attached document (including file extension) in this Action.
Example Instruction 1:
You are an experienced support agent, produce the reply for the input prompt in the context of user support with one sentence with no additional information, using only the information attached below.
Example Instruction 2:
Imagine you are a sales agent, and your aim is to sell the XYZ and ABC products. Use the {{document.Instruction for product XYZ.txt}} for providing the reply concerning product XYZ and {{document.instruction for product ABC.txt}} for providing the reply about the product ABC. If the user asks about topics other than products XYZ and ABC, try to convince them to switch the topic to the products discussion. The answer should be 2 sentences maximum.
NOTE: If your input prompt exceeds six lines, you can expand the text area for more comfortable editing.
Click the Expand button in the top right corner, next to the instruction field. When you finish editing the instruction field, press the Save button. The text area will return to its default size. If you need to expand it again, press the Edit button and click Expand once more.
Step 3. Upload the Source Documents.
Generative AI Action allows to attach source files that, along with the Input Prompt and Instruction, will be used by Generative AI to generate a response. If no document is provided, the AI will generate responses based on general knowledge.
- Usage: You can upload one or more documents, or leave this field empty if not required.
- Character Limit: The total size of the attached documents is restricted by the Max Size of Context setting in the AI Model. For instance, if the model’s Max Context Size is set to 2048 tokens (approximately 8,192 characters, without spaces), the content of the documents must fit within this limit. This limit applies to both the input prompt and the generated output, combined.
- Important: If the file exceeds the character limit and RAG (Retrieval-Augmented Generation) is activated in your AI Model, you will receive a prompt to convert the attachment to RAG. This enhances search accuracy by retrieving relevant information from large data sources. If RAG is not enabled or the file still exceeds the limit, the document must be reduced in size or reworked to fit within the allowed character count.
- Restriction: With a 2048-token limit (~8,192 characters), the attached documents should not exceed 1-2 pages of text (in a standard MS Word document, 11-point font).
- Examples:
- 1 document of 1-2 pages (~8,000 characters)
- 2 documents of 1 page each (~4,000 characters per document)
In order to upload documents press the Upload button.
Select a file you want to upload:
If the file doesn’t exceed size or character count limits, it will be saved as FLAT.
If the file does exceed at least one limit, it will be converted to RAG file while uploading.
File will start uploading, the progress can be visible next to the file name.
For our example, we uploaded one bigger file that was converted to RAG – “Staff_Handbook.txt”, and one smaller Flat file – “Vacation_Quotas.txt”.
NOTE: The document formats allowed to be uploaded are set in Generative AI Settings.
Step 4. Additional Parameters
Parameters for the Generative AI Destination are applied by default. You can view these parameters in the AI Model Configurator application for the respective connected Generative AI. To customize the AI’s behavior further, you can redefine existing parameters or add new ones using the Add Parameter function. Save changes in the end.
Available Parameters:
- Ollama Models: This link provides a comprehensive list of valid parameters and values for Ollama models.
- OpenAI Models: This link provides a comprehensive list of valid parameters and values for Open AI models.
For other Generative AI models refer to the respective documentation of the model provider.
NOTE: Changing default parameters can significantly influence the AI Model’s performance and responses. Therefore, it is recommended to proceed with caution when making changes.
Step 5. Select the reply output type.
There are two options for reply output.
Send reply directly – reply will be outputted directly to the chat with the end user.
OR Save reply to a variable for later use.
The AI reply will be saved in the memory parameter specified in the ‘Output Parameter’, rt_genai_output by default. To refer to this parameter in future messages, use the following syntax: {{memory.rt_genai_output}}.
Step 5. Save your changes when done editing, and check the Generative AI block.
Open a Webchat Preview window and ask a question. In this example we saved the response to a variable, and displayed it after a hardcoded “This is your Generative AI Response” text :
