Model is failing on prompts with special characters.
#6
by sourabh89 - opened
Hi , I tried a few simple examples below . It predicts them as injection.
a,b
a#b
a@b
a>b
I think the model predicts safe only if the instruction is well formatted with different special characters.
Hey @sourabh89 , thanks a lot for the info. It's true, we trained the model to be comparatively strict to only allow queries that comply with a specific use case (people searching for information in a textual database).
If your use case requires very different queries (this looks like mathematical operations?), you may want to add a representative set of legitimate queries to the data set of injections and retrain.
Hope that helps?