Added validator logs and organized code

[rkritika1508]

Summary

Target issue is #19.
Explain the motivation for making this change. What existing problem does the pull request solve?
Currently, we are using adding request logs, which work at the API level. Now, we will also have validator logs. So, for each validator that is triggered by the API, we will have the input_before_validation, input_after_validation, outcome, error, validation type, etc. This will make the logs more comprehensive and useful for NGOs to understand where the guardrails are failing and what are the reasons for the failure.

The guard.history parameter contains the validator logs which we are using.
Adding a sample schema from where we are extracting the logs -

Call(
    id='5155853312',
    iterations=[
        Iteration(
            id='5155864112',
            index=0,
            call_id='5155853312',
            inputs=Inputs(
                llm_api=None,
                llm_output='input-message',
                messages=None,
                prompt_params={},
                num_reasks=0,
                metadata={},
                full_schema_reask=False,
                stream=False
            ),
            outputs=Outputs(
                llm_response_info=LLMResponse(
                    prompt_token_count=None,
                    response_token_count=None,
                    output='input-message',
                    stream_output=None,
                    async_stream_output=None
                ),
                raw_output=None,
                parsed_output='input-message',
                validation_response='validated-input',
                guarded_output='validated-input',
                reasks=[],
                validator_logs=[
                    ValidatorLogs(
                        validator_name='PIIRemover',
                        registered_name='pii-remover',
                        instance_id=4827966800,
                        property_path='$',
                        value_before_validation='input-message',
                        value_after_validation='validated-input',
                        validation_result=FailResult(
                            outcome='fail',
                            error_message='PII detected in the text.',
                            fix_value='validated-input',
                            error_spans=None,
                            metadata=None,
                            validated_chunk=None
                        ),
                        start_time=datetime.datetime(2026, 1, 20, 13, 38, 2, 692890),
                        end_time=datetime.datetime(2026, 1, 20, 13, 38, 2, 976430)
                    )
                ],
                error=None,
                exception=None
            )
        )
    ],
    inputs=CallInputs(
        llm_api=None,
        llm_output=None,
        messages=None,
        prompt_params={},
        num_reasks=0,
        metadata={},
        full_schema_reask=False,
        stream=False,
        args=[],
        kwargs={}
    ),
    exception=None
)

Checklist

Before submitting a pull request, please ensure that you mark these task.

• Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
• If you've fixed a bug or added code that is tested and has test cases.

Notes

Please add here if any other information is required for the reviewer.

Summary by CodeRabbit

• New Features

  • Per-validator logging persisted: each validator now records name, input, output, error, and PASS/FAIL outcome for auditability.
  • New request logging model added to track request lifecycle and responses.

• Chores

  • Guardrail flows now persist validator logs and validate request IDs, returning failures for invalid IDs.
  • Logging model adjustments: explicit outcome enum, nullable output/error, and timestamped records for clearer tracking.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds per-validator logging and request logging models: migration updates validator_log schema; new ValidatorLogCrud persists validator logs; guardrails routes instantiate and pass ValidatorLogCrud, persist per-validator entries, and validate request_id; models reorganized under app.models.logging.

Changes

Cohort / File(s)	Summary
Database Migration backend/app/alembic/versions/002_added_validator_log.py	Added non-nullable name (AutoString) and non-nullable outcome (enum validatoroutcome / ValidatorOutcome with PASS/FAIL); changed output and error to nullable; downgrade still drops table.
Validator Logging CRUD backend/app/crud/validator_log.py	New ValidatorLogCrud(session: Session) with create(log: ValidatorLog) -> ValidatorLog — sets updated_at, adds, commits, refreshes, and returns the persisted entry.
Guardrails Integration backend/app/api/routes/guardrails.py	Instantiate and pass ValidatorLogCrud through guard flows; _validate_with_guard signature updated to accept validator_log_crud; added add_validator_logs helper to persist per-validator input/output/outcome; request_id extraction includes validation and failure handling.
Models: logging reorg & updates backend/app/models/__init__.py, backend/app/models/logging/validator.py, backend/app/models/guardrail_config.py	Re-export models from app.models.logging.*; added ValidatorOutcome enum; ValidatorLog adds name and outcome, input non-nullable, output/error nullable; validator-config imports switched to app.models.validators.*.
RequestLog model new module / import update backend/app/models/logging/request.py, backend/app/crud/request_log.py	New RequestLog, RequestLogUpdate, and RequestStatus in app.models.logging.request; crud/request_log.py imports switched to use app.models.logging.request.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant API as Guardrails API
    participant Guard as Guard (validators)
    participant LogCrud as ValidatorLogCrud
    participant DB as Database

    Client->>API: call run_input_guardrails / run_output_guardrails(request)
    API->>API: derive and validate request_id
    alt invalid request_id
        API-->>Client: return failure (invalid request id)
    else valid request_id
        API->>LogCrud: instantiate ValidatorLogCrud(session)
        API->>Guard: _validate_with_guard(payload, validator_log_crud, request_log_id)
        Guard->>Guard: execute validators -> produce results + history
        Guard-->>API: validation result + history
        API->>LogCrud: add_validator_logs(guard, request_log_id, validator_log_crud)
        LogCrud->>DB: create(ValidatorLog entries)
        DB-->>LogCrud: persisted entries
        alt validation success
            API-->>Client: return success (sorted output, request_id)
        else validation failure
            API-->>Client: return failure (generic error, request_id)
        end
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🐰
I hopped through code and left a trail,
Each validator now tells its tale,
Pass or fail, the ledger grows,
Quiet logs where insight flows,
Happy hops for guarded goals!

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	❓ Inconclusive	The title 'Added validator logs and organized code' is overly broad and vague. While it mentions adding validator logs (a core part of the change), it also includes 'organized code' which is non-descriptive and doesn't convey the actual scope of changes (database migrations, CRUD operations, model reorganization, and integration with guardrails routes).	Consider a more specific title like 'Add validator-level logging to guardrails validation flows' or 'Implement validator logging with database schema and CRUD operations' to better capture the main architectural changes.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@backend/app/api/routes/guardrails.py`:
- Around line 24-27: The code calls UUID(payload.request_id) which can raise
ValueError and yield a 500; update the guardrails route to validate request_id
and return a 4xx instead: either change the request Pydantic model to use
typing.UUID for request_id (so FastAPI will validate automatically) or catch
ValueError around the UUID(...) conversion in the handler that uses
RequestLogCrud and ValidatorLogCrud, and return an appropriate HTTP 400/422
response with a clear message; locate the UUID(...) usage and surrounding logic
where RequestLogCrud.create and ValidatorLogCrud are called to apply the change.
- Around line 146-160: iteration.outputs.validator_logs handling writes the
guardrail outcome as-is, but Guardrails returns lowercase "pass"/"fail" which
fails the ValidatorOutcome enum; update the persistence call by normalizing
result.outcome to uppercase and converting it to the ValidatorOutcome enum
before calling validator_log_crud.create. Specifically, where FailResult and
result are used in the loop (iteration.outputs.validator_logs, FailResult,
result.outcome), transform outcome via ValidatorOutcome(result.outcome.upper())
and pass that value into validator_log_crud.create's outcome parameter.

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@backend/app/api/routes/guardrails.py`:
- Around line 164-192: In add_validator_logs, convert non-string validator
values to safe strings before constructing the ValidatorLog: when building
ValidatorLog (class ValidatorLog) use a defensive conversion for
log.value_before_validation and log.value_after_validation (e.g., coerce None to
empty string and wrap non-str values with str() or json.dumps for complex
objects) so input and output are always non-null strings; keep existing
error_message logic and call validator_log_crud.create as before. Also ensure
the file ends with a trailing newline.

♻️ Duplicate comments (1)

backend/app/crud/validator_log.py (1)

8-17: LGTM!

The ValidatorLogCrud class follows a consistent pattern with RequestLogCrud and correctly handles the database operations for persisting validator logs. Note: Add a trailing newline at end of file per linting rules.

🧹 Nitpick comments (1)

backend/app/crud/validator_log.py (1)

1-1: Remove unused imports.

UUID and uuid4 are imported but never used in this file.

♻️ Proposed fix

-from uuid import UUID, uuid4
-
 from sqlmodel import Session

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@backend/app/models/logging/request.py`:
- Around line 66-68: The file ends without a trailing newline causing Ruff W292;
add a single newline character at EOF so the class definition for
RequestLogUpdate (response_text: str, response_id: UUID) is followed by a
newline, then save the file to satisfy the linter.

♻️ Duplicate comments (1)

backend/app/models/logging/validator.py (1)

63-63: Add trailing newline (W292).

File is missing a newline at EOF.

🧹 Nitpick comments (3)

backend/app/models/logging/request.py (1)

59-63: updated_at won't auto-update on record modifications.

The default_factory=now only sets the value at insert time. To automatically update this timestamp when the record is modified, use SQLAlchemy's onupdate parameter.

♻️ Suggested fix

     updated_at: datetime = Field(
         default_factory=now, 
         nullable=False,
-        sa_column_kwargs={"comment": "Timestamp when the entry was last updated"},
+        sa_column_kwargs={
+            "comment": "Timestamp when the entry was last updated",
+            "onupdate": now,
+        },
     )

backend/app/models/logging/validator.py (2)

38-46: Consider adding explicit default=None for nullable fields.

While SQLModel may infer the default for str | None types, explicitly setting default=None improves clarity and ensures consistent behavior.

♻️ Suggested fix

-    output: str | None = Field(
+    output: str | None = Field(
+        default=None,
         nullable=True,
         sa_column_kwargs={"comment": "Output message post validation"},
     )

-    error: str | None = Field(
+    error: str | None = Field(
+        default=None,
         nullable=True,
         sa_column_kwargs={"comment": "Error message if the validator throws an exception"},
     )

---

59-63: updated_at won't auto-update on record modifications.

Same as in request.py: use onupdate in sa_column_kwargs to automatically update this timestamp when the record is modified.

♻️ Suggested fix

     updated_at: datetime = Field(
         default_factory=now,
         nullable=False,
-        sa_column_kwargs={"comment": "Timestamp when the entry was last updated"},
+        sa_column_kwargs={
+            "comment": "Timestamp when the entry was last updated",
+            "onupdate": now,
+        },
     )
+