Guard issues #44
Description
the test message I use for scan is always: 'corporate sales number are 10k filling, [[email protected]](mailto:[email protected])'
-
default threshold should be changed from '>= 0.0' to '> 0.0' otherwise even guards with actual_value 0.0 should be activated.
GuardMatch(response_match=<ResponseMatch.NO: 'NO'>, guard_name=<GuardName.HARMFUL_CONTENT: 'harmful'>, actual_value=0.0, threshold='>= 0.0', match_count=0) -
for Guard that can be now also created by directly passing arguments, it is always asking for matches even if they are not necessary:
In [51]: gc1=Guard('jailbreak', threshold='1')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[51], line 1
----> 1 gc1=Guard('jailbreak', threshold='1')
TypeError: Guard.__init__() missing 1 required positional argument: 'matches'
Might be better to allow only one option for Guard creation
- In default guard, modality is matched even though actual_value is indicated as 0.0 - so looks like modality does not work with threshold and any threshold passed to guard would cause it to fail, does it work only with count threshold ? In any way nothing from the guards indicates why it was matched
res=c.apex.scan(message)
res.match_details[0].matched_checks
GuardMatch(response_match=<ResponseMatch.YES: 'YES'>, guard_name=<GuardName.MODALITY: 'modality'>, actual_value=0.0, threshold='>= 0.0', match_count=0, match_values=[]),
- if count_threshold does not pass, no matched values are visible even though 2 were detected
gc=Guard.create('pii_detector',count_threshold=3, matches={'email_address':None})
res=c.apex.scan(message, guard_config=[gc])
res.match_details[0].all_checks
GuardMatch(response_match=<ResponseMatch.NO: 'NO'>, guard_name=<GuardName.PII_DETECTOR: 'pii_detector'>, actual_value=1.0, threshold='>= 0.0', match_count=2, match_values=[]
- I would add count_threshold to the GuardMatch so it is clear what is match_count compared to