I had an interesting question from a customer today regarding testing of Agentforce and how it’s feasible in non-production orgs (“environments”) that may not have as must customer data or even contain masked values for some fields. Below is my response.
I think it’s important to distinguish between having “the right amount/variance/volume of data” and having “the right data” in the org when testing or working with agents. In Agentforce we do not train or fine tune any models so the amount of data is less important than having the right data. By this I mean that testing an agent in Salesforce is altogether feasible with only a single record if it’s the right record.
I know this may be bit of a bold statement, but here’s the reason. When generating any response to the user of the agent, Agentforce will look up the information needed or ground any prompt (see explainer below) with the information needed. Performing lookups are done by its actions, i.e. having the right test data set makes it feasible to test with a very limited data set. If grounding a prompt the information used for the grounding will either be a) connected to the user or b) publicly available information i.e. not customer specific / sensitive. Again totally feasible with a limited data set.
Explainer: “Grounding a prompt” is the act of inserting the context needed for the LLM to create the response into the prompt itself. The important part here is that the context is provided in the prompt and is not stored in the model. The model is in this context replaceable and should be chosen based on use case. (i.e. speed or cost ´).
Considering the above, the volume of data in the org is therefore less important than having the right data there.
The above is true for both structured and unstructured data. That is not to say that when testing an agent it doesn’t make sense to have the knowledge base or product manuals indexed in Data Cloud but it does mean that production data or even unmasked data is not required for agent testing.