AWS nova medium security documentation change
Summary
Updated RAG documentation with enhanced prompt engineering guidance, added citation requirements, and clearer instructions to prevent hallucinations
Security assessment
The change adds explicit instructions to 'DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS!' and implements citation requirements to ground responses. This directly addresses security concerns about AI hallucination and unverified information disclosure by enforcing source validation and restricting model access to predefined data sources.
Diff
diff --git a/nova/latest/userguide/prompting-tools-rag.md b/nova/latest/userguide/prompting-tools-rag.md index 61af87890..36400e50d 100644 --- a//nova/latest/userguide/prompting-tools-rag.md +++ b//nova/latest/userguide/prompting-tools-rag.md @@ -11 +11 @@ When constructing your own _retrieval augmented generation_ (RAG) system, you ca -###### Tip +###### Leverage the system prompt @@ -13 +13 @@ When constructing your own _retrieval augmented generation_ (RAG) system, you ca -Leverage the System Prompts: As with other functionalities, enhancing the system prompt can be beneficial. You can define the RAG Systems description in the system prompt, outlining the desired persona and behavior for the model. +As with other functionalities, enhancing the system prompt can be beneficial. You can define the RAG Systems description in the system prompt, outlining the desired persona and behavior for the model. @@ -15 +15 @@ Leverage the System Prompts: As with other functionalities, enhancing the system -###### Tip +###### Use Model Instructions @@ -17 +17 @@ Leverage the System Prompts: As with other functionalities, enhancing the system -Use Model Instructions: Additionally, you can include a dedicated `"Model Instructions:"` section within the system prompt, where you can provide specific guidelines for the model to follow. +You can include a dedicated `"Model Instructions:"` section within the system prompt, where you can provide specific guidelines for the model to follow. For instance, you can list instructions such as: @@ -19 +19 @@ Use Model Instructions: Additionally, you can include a dedicated `"Model Instru -For instance, you can list instructions such as: `In this example session, the model has access to search results and a user's question, its job is to answer the user's question using only information from the search results.` +`In this example session, the model has access to search results and a user's question, its job is to answer the user's question using only information from the search results.` @@ -39 +39 @@ For instance, you can list instructions such as: `In this example session, the m -###### Tip +###### Avoid Hallucination by restricting the instructions @@ -41 +41 @@ For instance, you can list instructions such as: `In this example session, the m -Avoid Hallucination by restricting the instructions: Bring more focus to instructions by clearly mentioning "DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS!" as a model instruction so the answers are grounded in the provided context. +Bring more focus to instructions by clearly mentioning "DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS!" as a model instruction so the answers are grounded in the provided context. @@ -47 +47 @@ Avoid Hallucination by restricting the instructions: Bring more focus to instruc -###### Tip +###### Provide an input query followed by search results @@ -49 +49 @@ Avoid Hallucination by restricting the instructions: Bring more focus to instruc -Provide an input query followed by search results: Provide an input query followed by the retriever search results or contextual chunks. The model works best when the chunk results are provided after `Resource: Search Results:` +Provide an input query followed by the retriever search results or contextual chunks. The model works best when the chunk results are provided after `Resource: Search Results:` @@ -55,0 +56,12 @@ Provide an input query followed by search results: Provide an input query follow +###### Citations + +Citations serve as helpful references back to the context provided to answer the question. Citations are generally utilized primarily to ground the LLM Answers. Citations are employed as an evaluation tool, enabling users to refer back to the cited sources from the context to assess whether the answers remain faithful to the provided information. + +This is a sample prompt that should be added in the “Model Instructions” in your system prompt to enable the model to focus on producing citations in the answer: + + + - Make sure to always add citations + to your response using markers like + %[1]%, %[2]%, %[3]%, and for the corresponding + passage that supports the response. + @@ -57,0 +70,3 @@ You can combine all of the previous recommendations with the following prompt te +Role | Prompt +---|--- +System | In this session, the model has access to search results and a user's question, your job is to answer the user's question using only information from the search results. Model Instructions: @@ -59,4 +74,5 @@ You can combine all of the previous recommendations with the following prompt te - In this session, the model has access - to search results and a user's question, your job - is to answer the user's question using only - information from the search results. + * You should provide concise answer to simple questions when the answer is directly contained in search results, but when comes to yes/no question, provide some details. + * In case the question requires multi-hop reasoning, you should find relevant information from search results and summarize the answer based on relevant information with logical reasoning. + * If the search results do not contain information that can answer the question, please state that you could not find an exact answer to the question, and if search results are completely irrelevant, say that you could not find an exact answer, then summarize search results. + * Remember to add a citation to the end of your response using markers like %[1]%, %[2]%, %[3]%, etc for the corresponding passage supports the response. + * DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS! @@ -64,19 +79,0 @@ You can combine all of the previous recommendations with the following prompt te - Model Instructions: - - You should provide concise answer to simple - questions when the answer is directly contained in - search results, but when comes to yes/no question, - provide some details. - - In case the question requires multi-hop reasoning, - you should find relevant information from search - results and summarize the answer based on relevant - information with logical reasoning. - - If the search results do not contain information - that can answer the question, please state that you - could not find an exact answer to the question, and - if search results are completely irrelevant, say - that you could not find an exact answer, then summarize - search results. - - Remember to add a citation to the end of your response - using markers like %[1]%, %[2]%, %[3]%, etc for the - corresponding passage supports the response. - - DO NOT USE INFORMATION THAT IS NOT IN SEARCH RESULTS! @@ -84,2 +81 @@ You can combine all of the previous recommendations with the following prompt te - {Query} - Resource: {search_results} +User | {Query} Resource: Search Results: {search_results}