-
Notifications
You must be signed in to change notification settings - Fork 900
Open
Description
I want to insert a JSON document into MongoDB that contains a date field expressed in MongoDB Extended JSON format (see https://www.mongodb.com/docs/manual/reference/mongodb-extended-json/#mongodb-extended-json--v2-). This is valid JSON — an object with a "$date" field containing an ISO8601 string.
Example input:
{
"thing_id": "t1",
"timestamp": {
"$date": "2025-10-13T09:13:52Z"
}
}Expected behavior:
- Connect should forward this JSON unchanged to MongoDB. MongoDB should interpret { "$date": "2025-10-13T09:13:52Z" } as a Date and store ISODate("2025-10-13T09:13:52Z").
Actual behavior:
Instead of inserting the document, Connect crashes with the following error mesage:
rpk connect run config.yaml
INFO[2025-11-17T17:38:01+01:00] Running main config from specified file @service=redpanda-connect benthos_version=4.70.0 path=config.yaml
INFO[2025-11-17T17:38:01+01:00] Successfully loaded Redpanda license @service=redpanda-connect expires_at="2035-11-15T17:38:01+01:00" license_org="" license_type="open source"
INFO[2025-11-17T17:38:01+01:00] Listening for HTTP requests at: http://0.0.0.0:4195 @service=redpanda-connect
INFO[2025-11-17T17:38:01+01:00] Input type file is now active @service=redpanda-connect label="" path=root.input
INFO[2025-11-17T17:38:01+01:00] Launching a Redpanda Connect instance, use CTRL+C to close @service=redpanda-connect
INFO[2025-11-17T17:38:01+01:00] Output type mongodb is now active @service=redpanda-connect label="" path=root.output
ERRO[2025-11-17T17:38:01+01:00] Failed to send message to mongodb: failed to execute document_map: invalid JSON input; expected { @service=redpanda-connect label="" path=root.output
INFO[2025-11-17T17:38:01+01:00] Pipeline has terminated. Shutting down the service @service=redpanda-connectHow to reproduce:
- Start MongoDB:
docker run -d -p 27017:27017 --name=mongo-example1 mongo:latest- Create a config.yaml file:
input:
file:
paths: [ ./*.json ]
scanner:
json_documents: {}
output:
mongodb:
url: mongodb://localhost
database: admin
collection: message
operation: insert-one
write_concern:
w: majority
j: false
w_timeout: ""
document_map:
root = this- Create a input.json file:
{
"thing_id": "t1",
"timestamp": {
"$date": "2025-10-13T09:13:52Z"
}
}- Run
rpk connect run config.yamlStrangely enough, the following JSON file works fine without an error. So expect that something happen if Connect tries to process $date.
{
"$device_id": "fs1",
"timestamp": {
"$someThing": "2025-10-13T09:13:52Z"
}
}I hope this fits as an issue; I don't think it's a discussion.
Metadata
Metadata
Assignees
Labels
No labels