Using the ArangoDB Swagger.io Interactive API Documentation
ArangoDB bundles its regular API Documentation also in Swagger.IO API description format. You can browse and explore it interactively via the ArangoDB Webinterface.
Read more
Using GraphQL with ArangoDB: A NoSQL Database Solution
GraphQL is a query language created by Facebook for modern web and mobile applications as an alternative to REST APIs. Following the original announcement alongside Relay, Facebook has published an official specification and reference implementation in JavaScript. Recently projects outside Facebook like Meteor have also begun to embrace GraphQL.
Users have been asking us how they can try out GraphQL with ArangoDB. While working on the 2.8 release of our NoSQL database we experimented with GraphQL and published an ArangoDB-compatible wrapper for GraphQL.js. With the general availability of ArangoDB 2.8 you can now use GraphQL in ArangoDB using Foxx services (JavaScript in the database).
A GraphQL primer
GraphQL is a query language that bears some superficial similarities with JSON. Generally GraphQL APIs consist of three parts:
The GraphQL schema is implemented on the server using a library like graphql-sync and defines the types supported by the API, the names of fields that can be queried and the types of queries that can be made. Additionally it defines how the fields are resolved to values using a backend (which can be anything from a simple function call, a remote web service or accessing a database collection).
The client sends queries to the GraphQL API using the GraphQL query language. For web applications and JavaScript mobile apps you can use either GraphQL.js or graphql-sync to make it easier to generate these queries by escaping parameters.
The server exposes the GraphQL API (e.g. using an HTTP endpoint) and passes the schema and query to the GraphQL implementation, which validates and executes the query, later returning the output as JSON.
GraphQL vs REST
Whereas in REST APIs each endpoint represents a single resource or collection of resources, GraphQL is agnostic of the underlying protocols. When used via HTTP it only needs a single endpoint that handles all queries.
The API developer still needs to decide what information should be exposed to the client or what access controls should apply to the data but instead of implementing them at each API endpoint, GraphQL allows centralising them in the GraphQL schema. Instead of querying multiple endpoints, the client can pick and choose from the schema when defining the query and filter the response to only contain the fields it actually needs.
For example, the following GraphQL query:
query {
user(id: "1234") {
name
friends {
name
}
}
}
could return a response like this:
{
"data": {
"user": {
"name": "Bob",
"friends": [
{
"name": "Alice"
},
{
"name": "Carol"
}
]
}
}
}
whereas in a traditional REST API accessing the names of the friends would likely require additional API calls and filtering the responses to certain fields would either require proprietary extensions or additional endpoints.
GraphQL Demo Service
If you are running ArangoDB 2.8 you can install the Foxx service demo-graphql from the Store. The service provides a single HTTP POST endpoint /graphql
that accepts well-formed GraphQL queries against the Star Wars data set used by GraphQL.js.
It supports three queries:
hero(episode)
returns thehuman
ordroid
that was the hero of the given episode or the hero of the Star Wars saga if no episode is specified. The valid IDs of the episodes are"NewHope"
,"Empire"
,"Jedi"
and"Awakens"
corresponding to episodes 4, 5, 6 and 7.human(id)
returns the human with the given ID (a string value in the range of"1000"
to"1007"
). Humans have anid
,name
and optionally ahomePlanet
.droid(id)
does the same for droids (with IDs"2000"
,"2001"
and"2002"
). Droids don't have ahomePlanet
but may have aprimaryFunction
.
Both droids and humans have friends
(which again can be humans or droids) and a field appearsIn
mapping them to episodes (which have an id
, title
and description
).
For example, the following query:
{
human(id: "1007") {
name
friends {
name
}
appearsIn {
title
}
}
}
returns the following JSON:
{
"data": {
"human": {
"name": "Wilhuff Tarkin",
"friends": [
{
"name": "Darth Vader"
}
],
"appearsIn": [
{
"title": "A New Hope"
}
]
}
}
}
It's also possible to do deeply nested lookups like "what episodes have the friends of friends of Luke Skywalker appeared in" (but note that mutual friendships will result in some duplication in the output):
{
human(id: "1000") {
friends {
friends {
appearsIn {
title
}
}
}
}
}
Additionally it's possible to make queries about the API itself using __schema
and __type
. For example, the following tells us the "droid" query returns something of a type called "Droid"
:
{
__schema {
queryType {
fields {
name
type {
name
}
}
}
}
}
And the next query tells us what fields droids have (so we know what fields we can request when querying droids):
{
__type(name: "Droid") {
fields {
name
}
}
}
GraphQL: The Good
GraphQL shifts the burden of having to specify what particular subset of information should be returned to the client. Unlike traditional REST based solutions this is built into the language from the start: a client will only see information they explicitly request, they don't have to know about anything they're not already interested in.
At the same time a single GraphQL schema can be written to represent the entire global state graph of an application domain without having to hard-code any assumptions about how that data will be presented to the user. By making the schema declarative GraphQL avoids the necessary duplication and potential for subtle bugs involved in building equally exhaustive HTTP APIs.
GraphQL also provides mechanisms for introspection, allowing developers to explore GraphQL APIs without external documentation.
GraphQL is also protocol agnostic. While REST directly builds on the semantics of the underlying HTTP protocol, GraphQL brings its own semantics, making it easy to re-use GraphQL APIs for non-HTTP communication (such as Web Sockets) with minimal effort.
GraphQL: The Bad
The main drawback of GraphQL as implemented in GraphQL.js is that each object has to be retrieved from the data source before it can be queried further. For example, in order to retrieve the friends of a person, the schema has to first retrieve the person and then retrieve the person's friends using a second query.
Currently all existing demonstrations of GraphQL use external databases with ORMs or ODMs with complex GraphQL queries causing multiple consequent network requests to an external database. This added cost of network latency, transport overhead, serialization and deserialization makes using GraphQL slow and inefficient compared to an equivalent API using hand-optimized database queries.
This can be mitigated by inspecting the GraphQL Abstract Syntax Tree
to determine what fields will be accessed on the retrieved document. However, it doesn't seem feasible to generate efficient database queries ad hoc, foregoing a lot of the optimizations otherwise possible with handwritten queries in databases.
Conclusion
Although there doesn't seem to be any feasible way to translate GraphQL requests into database-specific queries (such as AQL), the impact of having a single GraphQL request result in a potentially large number of database requests is much less significant when implementing the GraphQL backend directly inside the database.
While RESTful HTTP APIs are certainly here to stay and GraphQL like any technology has its own trade-offs, the advantages of having a standardized yet flexible interface for accessing and manipulating an application's global state graph are undeniable.
GraphQL is a promising fit for schema-free databases and dynamically typed languages. Instead of having to spread validation and authorization logic across different HTTP endpoints and native database format restrictions a GraphQL schema can describe these concerns. Thus guaranteeing that sensitive fields are not accidentally exposed and the data formats remain consistent across different queries.
We're excited to see what the future will hold for GraphQL and encourage you to try out GraphQL in the database with ArangoDB 2.8 and Foxx today. Have a look at the demo-graphql from the Store. If you have built or are planning to build applications using GraphQL and ArangoDB, let us know in the comments.
Using Bind Parameters in the AQL Editor: ArangoDB
The AQL editor in the web interface is useful for running ad hoc AQL queries and trying things out. It provides a feature to explain the query and inspect its execution plan. This can be used to check if the query uses indexes, and which.
So far the AQL editor only supported using query string literals, but it lacked support for bind parameters. Queries issued by application code however often will use bind parameters for security reasons. Often enough this prevented copying & pasting queries from the application code into the AQL editor and vice versa without making manual adjustments. Read more
Running V8 Isolates in Multi-Threaded ArangoDB
ArangoDB allows running user-defined JavaScript code in the database. This can be used for more complex, stored procedures-like database operations. Additionally, ArangoDB’s Foxx framework can be used to make any database functionality available via an HTTP REST API. It’s easy to build data-centric microservices with it, using the scripting functionality for tasks like access control, data validation, sanitation etc.
We often get asked how the scripting functionality is implemented under the hood. Additionally, several people have asked how ArangoDB’s JavaScript functionality relates to node.js.
This post tries to explain that in detail.
ArangoDB 2.6 API Changes: Updates & Enhancements
ArangoDB 2.6 comes with new and changed APIs as well as changed behavior regarding document keys and several graph functions.
If you use Travis-CI for your tests you can download the Travis-CI ArangoDB build here: Travis-CI/ArangoDB-2.6.0-alpha2.tar.gz
The changes so far:
APIs added
- added batch document removal and lookup APIs:
These APIs can be used to perform multi-document lookup and removal operations efficiently. The arguments to these APIs are the name of the collection plus the array of document keys to fetch or remove.
The endpoints for these APIs are as follows:
PUT /_api/simple/lookup-by-keys PUT /_api/simple/remove-by-keys
Example call to fetch documents:
curl -X PUT \ http://127.0.0.1:8529/\_db/\_system/_api/simple/lookup-by-keys \ --data '{"collection":"myCollection","keys":["test1","test3"]}'
The documents will be returned in an attribute
documents
of the HTTP response.documents
is an array containing all documents found. Only those documents that were actually found will be returned. Documents that were searched but do not exist will not be returned and do not trigger any errors. (more…)
LoopBack Connector for ArangoDB: Seamless Integration
ArangoDB can be used as a backend data source for APIs that you compose with the popular open-source LoopBack Node.js framework.

In a recent blog article on StrongLoop, Nicholas Duffy explains how to use his new loopback-connector-arango connector to access ArangoDB:
Getting Started with the Node.js LoopBack Connector for ArangoDB
The tutorial uses the loopback-connector-arango which is available as npm
and a demo application which is available from Github. (more…)
Bulk Document Lookups: Efficient Data Retrieval with ArangoDB
ArangoDB 2.6 comes with a specialized API for bulk document lookups. The new API allows fetching multiple documents from the server using a single request, making bulk document retrieval more efficient than when using one request per document to fetch.
Provided the documents keys are known, all the client application needs to do is to call the collection’s lookupByKeys
method:
// list of document keys
var keys = [ "foo", "bar", "baz", ...];
var results = db.test.lookupByKeys(keys);
// now all documents are contained in variable 'results'
Additionally, the server-side REST API method for bulk document lookups can be invoked directly via HTTP as follows:
curl \
-X PUT \
http://127.0.0.1:8529/_api/simple/lookup-by-keys \
--data '{"collection":"test","keys":["foo","bar","baz"]}'
Jan compared the functionality with single document requests in his latest blog post.
Full-Text Index Enhancements: ArangoDB Search Optimization
This post is about improvements for the fulltext index in ArangoDB 2.6. The improvements address the problem that non-string attributes were ignored when fulltext-indexing.
Effectively this prevented string values inside arrays or objects from being indexed. Though this behavior was documented, it was limited the usefulness of the fulltext index much. Several users requested the fulltext index to be able to index arrays and object attributes, too.
Finally this has been accomplished, so the fulltext index in 2.6 supports indexing arrays and objects!
Read on in Jan’s blog post about Fulltext Index Enhancements.
Exporting Data for Offline Processing in PHP: ArangoDB Guide
A few weeks ago I wrote about ArangoDB’s specialized export API.
The export API is useful when the goal is to extract all documents from a given collection and to process them outside of ArangoDB.
The export API can provide quick and memory-efficient snapshots of the data in the underlying collection, making it suitable for extract all documents of the collection. It will be able to provide data much faster than with an AQL query that will extract all documents.
In this post I’ll show how to use the export API to extract data and process it with PHP.
Please read the full blog post Exporting Data for Offline Processing.
Creating Multi-Game Highscore Lists: ArangoDB Techniques
I just came across a question about how to create highscore lists or leaderboards in ArangoDB, and how they would work when compared to Redis sorted sets.
This blog post tries to give an answer on the topic and also detailed instructions and queries for setting up highscore lists with ArangoDB. The additional section “Extensions” explains slightly more advanced highscore list use cases like multi-game highscore lists, joining data and maintaining a “last updated” date.
(more…)
Get the latest tutorials,
blog posts and news:
Thanks for subscribing! Please check your email for further instructions.