Urgent: Agent Not Referring to Vector DB – Returning Inaccurate Responses

Question

Urgent: Agent Not Referring to Vector DB – Returning Inaccurate Responses

Vibhuti Singh 5

We have deployed an AI Agent on Microsoft Teams, which was functioning well until recently. The agent is designed to:

Pull company and contact information via web data and OpenAI.

Fetch case studies from a deployed Vector Database.

Since yesterday, the agent has stopped retrieving data from the Vector DB. Instead, it is generating inaccurate or imaginary responses. This issue is critically impacting our business operations, as our teams rely on accurate case study information during client interactions.

We request immediate assistance to investigate and resolve the issue. Kindly prioritize this case and let us know if any additional logs or access are required from our side.

Thank you for your prompt attention.

Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-04T07:48:59.6166667+00:00
Hi Vibhuti Singh

Sorry for the trouble and delay in response.

It should be vector DB connection issue or recent stricter requirement on Azure OpenAI side.

Could you confirm that you have enable managed identity on Azure OpenAI resource and added the as contributor access or Search Service Contributor on your Azure AI search resource (if not an inbuilt vector store)

Temperature in Azure OpenAI less than 0.4

Adjust Top_K and Top_P from Azure OpenAI deployment setting

Modify the system prompt " your instruction. not to confuse if there is not a match in answer, instead say could not find"

Looking forward to hearing from you.

Thank you
Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-05T02:46:36.92+00:00

Hi Vibhuti Singh

Could you share the vector indexing configuration too and let us know if the situation improved with above pointers.

Thank you
Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-06T04:38:19.78+00:00

Hi Vibhuti Singh

We could not receive any inputs from you. Hope the pointers helped.

Thank you
Vibhuti Singh 5 Reputation points

2025-08-07T07:20:29.9966667+00:00

Thanks @Manas Mohanty , please allow me some time; I will reply to all your points

Your answer

Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-04T07:48:59.6166667+00:00

Hi Vibhuti Singh

Sorry for the trouble and delay in response.

It should be vector DB connection issue or recent stricter requirement on Azure OpenAI side.

Could you confirm that you have enable managed identity on Azure OpenAI resource and added the as contributor access or Search Service Contributor on your Azure AI search resource (if not an inbuilt vector store)

Temperature in Azure OpenAI less than 0.4

Adjust Top_K and Top_P from Azure OpenAI deployment setting

Modify the system prompt " your instruction. not to confuse if there is not a match in answer, instead say could not find"

Looking forward to hearing from you.

Thank you
Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-05T02:46:36.92+00:00

Hi Vibhuti Singh

Could you share the vector indexing configuration too and let us know if the situation improved with above pointers.

Thank you
Manas Mohanty 8,325 Reputation points Microsoft External Staff Moderator

2025-08-06T04:38:19.78+00:00

Hi Vibhuti Singh

We could not receive any inputs from you. Hope the pointers helped.

Thank you
Vibhuti Singh 5 Reputation points

2025-08-07T07:20:29.9966667+00:00

Thanks @Manas Mohanty , please allow me some time; I will reply to all your points

Share via

Urgent: Agent Not Referring to Vector DB – Returning Inaccurate Responses

Your answer