ArangoDB v3.9 reached End of Life (EOL) and is no longer supported.
This documentation is outdated. Please see the most recent version at docs.arangodb.com
HTTP Interface for Bulk Imports
ArangoDB provides an HTTP interface to import multiple documents at once into a collection. This is known as a bulk import.
The data uploaded must be provided in JSON format. There are two mechanisms to import the data:
- self-contained JSON documents: in this case, each document contains all attribute names and values. Attribute names may be completely different among the documents uploaded
- attribute names plus document data: in this case, the first array must contain the attribute names of the documents that follow. The following arrays containing only the attribute values. Attribute values will be mapped to the attribute names by positions.
The endpoint address is /_api/import for both input mechanisms. Data must be sent to this URL using an HTTP POST request. The data to import must be contained in the body of the POST request.
The collection query parameter must be used to specify the target collection for the import. Importing data into a non-existing collection will produce an error.
The waitForSync query parameter can be set to true to make the import only return if all documents have been synced to disk.
The complete query parameter can be set to true to make the entire import fail if any of the uploaded documents is invalid and cannot be imported. In this case, no documents will be imported by the import run, even if a failure happens at the end of the import.
If complete has a value other than true, valid documents will be imported while invalid documents will be rejected, meaning only some of the uploaded documents might have been imported.
The details query parameter can be set to true to make the import API return details about documents that could not be imported. If details is true, then the result will also contain a details attribute which is an array of detailed error messages. If the details is set to false or omitted, no details will be returned.
imports document values
imports documents from JSON-encoded lists
POST /_api/import
Query Parameters
-
collection (string, required): The collection name.
-
fromPrefix (string, optional): An optional prefix for the values in
_from
attributes. If specified, the value is automatically prepended to each_from
input value. This allows specifying just the keys for_from
. -
toPrefix (string, optional): An optional prefix for the values in
_to
attributes. If specified, the value is automatically prepended to each_to
input value. This allows specifying just the keys for_to
. -
overwrite (boolean, optional): If this parameter has a value of
true
oryes
, then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preserved. -
waitForSync (boolean, optional): Wait until documents have been synced to disk before returning.
- onDuplicate (string, optional):
Controls what action is carried out in case of a unique key constraint
violation. Possible values are:
- error: this will not import the current document because of the unique key constraint violation. This is the default setting.
- update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preserved.
- replace: this will replace an existing document in the database with the data specified in the request.
- ignore: this will not update an existing document and simply ignore the
error caused by the unique key constraint violation.
Note that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.
-
complete (boolean, optional): If set to
true
oryes
, it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported. - details (boolean, optional):
If set to
true
oryes
, the result will include an attributedetails
with details about documents that could not be imported.
Request Body
(string, required)
The body must consist of JSON-encoded arrays of attribute values, with one line per document. The first row of the request must be a JSON-encoded array of attribute names. These attribute names are used for the data in the subsequent lines.
Creates documents in the collection identified by collection-name
.
The first line of the request body must contain a JSON-encoded array of
attribute names. All following lines in the request body must contain
JSON-encoded arrays of attribute values. Each line is interpreted as a
separate document, and the values specified will be mapped to the array
of attribute names specified in the first header line.
The response is a JSON object with the following attributes:
-
created
: number of documents imported. -
errors
: number of documents that were not imported due to an error. -
empty
: number of empty lines found in the input (will only contain a value greater zero for typesdocuments
orauto
). -
updated
: number of updated/replaced documents (in caseonDuplicate
was set to eitherupdate
orreplace
). -
ignored
: number of failed but ignored insert operations (in caseonDuplicate
was set toignore
). -
details
: if query parameterdetails
is set to true, the result will contain adetails
attribute which is an array with more detailed information about which documents could not be inserted.
Responses
HTTP 201: is returned if all documents could be imported successfully.
HTTP 400: is returned if type
contains an invalid value, no collection
is
specified, the documents are incorrectly encoded, or the request
is malformed.
HTTP 404: is returned if collection
or the _from
or _to
attributes of an
imported edge refer to an unknown collection.
HTTP 409: is returned if the import would trigger a unique key violation and
complete
is set to true
.
HTTP 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.
Examples
Importing two documents, with attributes _key
, value1
and value2
each. One
line in the import data is empty
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
[ "foo", "bar", "baz" ]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Importing into an edge collection, with attributes _from
, _to
and name
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links <<EOF
[ "_from", "_to", "name" ]
[ "products/123","products/234", "some name" ]
[ "products/332", "products/abc", "other name" ]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Importing into an edge collection, omitting _from
or _to
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&details=true <<EOF
[ "name" ]
[ "some name" ]
[ "other name" ]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 281
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Violating a unique constraint, but allow partial imports
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&details=true <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["abc", "bar", "baz" ]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 244
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Violating a unique constraint, not allowing partial imports
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&complete=true <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["abc", "bar", "baz" ]
EOF
HTTP/1.1 409 Conflict
content-type: application/json
connection: Keep-Alive
content-length: 85
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Using a non-existing collection
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["foo", "bar", "baz" ]
EOF
HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 97
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Using a malformed body
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
{ "_key": "foo", "value1": "bar" }
EOF
HTTP/1.1 400 Bad Request
content-type: application/json
connection: Keep-Alive
content-length: 92
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
imports documents from JSON
imports documents from JSON
POST /_api/import
Query Parameters
- type (string, required):
Determines how the body of the request will be interpreted.
type
can have the following values:documents
: when this type is used, each line in the request body is expected to be an individual JSON-encoded document. Multiple JSON objects in the request body need to be separated by newlines.list
: when this type is used, the request body must contain a single JSON-encoded array of individual objects to import.auto
: if set, this will automatically determine the body type (eitherdocuments
orlist
).
-
collection (string, required): The collection name.
-
fromPrefix (string, optional): An optional prefix for the values in
_from
attributes. If specified, the value is automatically prepended to each_from
input value. This allows specifying just the keys for_from
. -
toPrefix (string, optional): An optional prefix for the values in
_to
attributes. If specified, the value is automatically prepended to each_to
input value. This allows specifying just the keys for_to
. -
overwrite (boolean, optional): If this parameter has a value of
true
oryes
, then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preserved. -
waitForSync (boolean, optional): Wait until documents have been synced to disk before returning.
- onDuplicate (string, optional):
Controls what action is carried out in case of a unique key constraint
violation. Possible values are:
- error: this will not import the current document because of the unique key constraint violation. This is the default setting.
- update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preserved.
- replace: this will replace an existing document in the database with the data specified in the request.
- ignore: this will not update an existing document and simply ignore the
error caused by a unique key constraint violation.
Note that that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.
-
complete (boolean, optional): If set to
true
oryes
, it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported. - details (boolean, optional):
If set to
true
oryes
, the result will include an attributedetails
with details about documents that could not be imported.
Request Body
(string, required)
The body must either be a JSON-encoded array of objects or a string with multiple JSON objects separated by newlines.
Creates documents in the collection identified by collection-name
.
The JSON representations of the documents must be passed as the body of the
POST request. The request body can either consist of multiple lines, with
each line being a single stand-alone JSON object, or a singe JSON array with
sub-objects.
The response is a JSON object with the following attributes:
-
created
: number of documents imported. -
errors
: number of documents that were not imported due to an error. -
empty
: number of empty lines found in the input (will only contain a value greater zero for typesdocuments
orauto
). -
updated
: number of updated/replaced documents (in caseonDuplicate
was set to eitherupdate
orreplace
). -
ignored
: number of failed but ignored insert operations (in caseonDuplicate
was set toignore
). -
details
: if query parameterdetails
is set to true, the result will contain adetails
attribute which is an array with more detailed information about which documents could not be inserted.
Responses
HTTP 201: is returned if all documents could be imported successfully.
HTTP 400: is returned if type
contains an invalid value, no collection
is
specified, the documents are incorrectly encoded, or the request
is malformed.
HTTP 404: is returned if collection
or the _from
or _to
attributes of an
imported edge refer to an unknown collection.
HTTP 409: is returned if the import would trigger a unique key violation and
complete
is set to true
.
HTTP 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.
Examples
Importing documents with heterogenous attributes from a JSON array
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
[
{
"_key" : "abc",
"value1" : 25,
"value2" : "test",
"allowed" : true
},
{
"_key" : "foo",
"name" : "baz"
},
{
"name" : {
"detailed" : "detailed name",
"short" : "short name"
}
}
]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Importing documents from individual JSON lines
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "_key": "abc", "value1": 25, "value2": "test","allowed": true }
{ "_key": "foo", "name": "baz" }
{ "name": { "detailed": "detailed name", "short": "short name" } }
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Using the auto type detection
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=auto <<EOF
[
{
"_key" : "abc",
"value1" : 25,
"value2" : "test",
"allowed" : true
},
{
"_key" : "foo",
"name" : "baz"
},
{
"name" : {
"detailed" : "detailed name",
"short" : "short name"
}
}
]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Importing into an edge collection, with attributes _from
, _to
and name
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=documents <<EOF
{ "_from": "products/123", "_to": "products/234" }
{"_from": "products/332", "_to": "products/abc", "name": "other name" }
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 72
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Importing into an edge collection, omitting _from
or _to
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=list&details=true <<EOF
[
{
"name" : "some name"
}
]
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 182
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Violating a unique constraint, but allow partial imports
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&details=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF
HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 244
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Violating a unique constraint, not allowing partial imports
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&complete=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF
HTTP/1.1 409 Conflict
content-type: application/json
connection: Keep-Alive
content-length: 85
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Using a non-existing collection
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "name": "test" }
EOF
HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 97
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Using a malformed body
shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
{ }
EOF
HTTP/1.1 400 Bad Request
content-type: application/json
connection: Keep-Alive
content-length: 95
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff