Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions modules/ROOT/pages/types.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Neo4j and Cypher provide a link:https://neo4j.com/docs/java-reference/current/extending-neo4j/values-and-types/[type system]
that describes how values are stored in the database, but these types do not always exactly match what Spark provides.

In some cases, there are data types that Neo4j provides that Spark does not have an equivalent for, and vice versa.
In some cases, there are data types that Neo4j provides that Spark does not have an equivalent for, and vice versa.

== Data type mappings

Expand All @@ -17,13 +17,19 @@ In some cases, there are data types that Neo4j provides that Spark does not have
|Example: `"Hello"`

|`Integer`
|`long`
|`long`, `short`, `byte`
|Example: `12345`

|`Float`
|`double`
|Example: `3.141592`

|`String`
|`decimal`
|Example: `"16725.77423461"`
This mapping only applies when _writing_ to Neo4j.
If you need to read a spark `decimal`, please parse the string representation in an appropriate way for your use case.

|`Boolean`
|`boolean`
|Example: `true`
Expand Down Expand Up @@ -58,6 +64,14 @@ If you need to write a `DateTime`, use a xref:write/query.adoc[Cypher write quer
|`struct { type: string, months: long, days: long, seconds: long, nanonseconds: integer, value: string }`
|See link:https://neo4j.com/docs/cypher-manual/current/values-and-types/temporal/#cypher-temporal-durations[Temporal functions: duration]

|`Duration`
|`duration` or `period` objects
|Example: `java.time.Duration.ofDays(42)` or `java.time.Period.ofMonths(5)`

|`Duration`
|Spark SQL `INTERVAL` types
|Example: `INTERVAL '10 05:30' DAY TO MINUTE`, `INTERVAL '4-5' YEAR TO MONTH` or `timestamp('2025-01-02 18:30:00.454') - timestamp('2024-01-01 00:00:00')`

|`Node`
|`struct { <id>: long, <labels>: array[string], (PROPERTIES) }`
|Nodes in Neo4j are represented as property containers; that is they appear as structs with properties corresponding to whatever properties were in the node. _For ease of use it is usually better to return individual properties than a node from a query._
Expand All @@ -70,6 +84,10 @@ If you need to write a `DateTime`, use a xref:write/query.adoc[Cypher write quer
|`string`
|Example: `path[(322)<-[20280:AIRLINE]-(33510)]`. _For ease of use it is recommended to use link:https://neo4j.com/docs/cypher-manual/current/values-and-types/lists/[path functions] to return individual properties/aspects of a path from a query._

|`ByteArray`
|`binary`, `array[byte]`
| Binary writes are treated specifically as Neo4j's `ByteArray` type.

|`[Array of same type]`
|`array[element]`
|In Neo4j, arrays must be consistently typed (for example, an array must contain only `Float` values). The inner Spark type matches the type mapping above.
Expand Down Expand Up @@ -261,6 +279,7 @@ MyNodeWithFlattenedMap {
.Spark to Cypher constraint type mapping
|===
|Spark type |Neo4j Type

|BooleanType |BOOLEAN
|StringType |STRING
|IntegerType |INTEGER
Expand All @@ -284,4 +303,4 @@ MyNodeWithFlattenedMap {

|===

For the arrays in particular we use the version without null elements as Neo4j does not allow to have them in arrays.
For the arrays in particular we use the version without null elements as Neo4j does not allow to have them in arrays.
5 changes: 5 additions & 0 deletions modules/ROOT/pages/write/options.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ The DataSource Writer has several options to connect and persist data into Neo4j
|0
|No

|`index.await.timeout`
|Adjust the maximum timeout to wait for index generation. Before writing, Cpyher procedure `db.awaitIndexes` will be called with specified timeout in whole seconds. Set to `0` to disable
|300
|No

4+|*Node specific options*

|`node.keys`
Expand Down