ArangoDB 2.1 RC 1: Test the Latest Features | ArangoDB

Dear ArangoDB users,

we are proud to announce the first release candidate for ArangoDB 2.1 two months after the first sharding release 2.0. It is a major step forward containing a lot of improvements.

Please note that we need your help testing the release candidate for any glitches and bugs. For instance, we moved to C++11. We tested the RPM and DEB packages we have built, but as there are so many different set-ups we certainly did not find all incompatibilities.

You can download packages from our website. Please report any bugs to us using the issue tracker of Github.

Thanks in advance for your help
Frank

Features and Improvements
=========================

The following list shows in detail which features have been added or improved in
ArangoDB 2.1. ArangoDB 2.1 also contains several bugfixes that are not listed
here.

New Edges Index
—————

The edges index (used to store connections between nodes in a graph) internally
uses a new data structure. This data structure improves the performance when
populating the edge index (i.e. when loading an edge collection). For large
graphs loading can be 20 times faster than with ArangoDB 2.0.

Additionally, the new index fixes performance problems that occurred when many
duplicate `_from` or `_to` values were contained in the index. Furthermore, the
new index supports faster removal of edges.

Finally, when loading an existing collection and building the edges index for
the collection, less memory re-allocations will be performed.

Overall, this should considerably speed up loading edge collections.

The new index type replaces the old edges index type automatically, without any
changes being required by the end user.

The API of the new index is compatible with the API of the old index. Still it
is possible that the new index returns edges in a different order than the old
index. This is still considered to be compatible because the old index had never
guaranteed any result order either.

AQL Improvements
—————-

AQL offers functionality to work with dates. Dates are no datatypes of their own
in AQL (neither they are in JSON, which is often used as a format to ship data
into and out of ArangoDB). Instead, dates in AQL are internally represented by
either numbers (timestamps) or strings. The date functions in AQL provide
mechanisms to convert from a numeric timestamp to a string representation and
vice versa.

There are two date functions in AQL to create dates for further use:

– `DATE_TIMESTAMP(date)` Creates a UTC timestamp value from `date`

– `DATE_TIMESTAMP(year, month, day, hour, minute, second, millisecond)`:
Same as before, but allows specifying the individual date components separately.
All parameters after `day` are optional.

– `DATE_ISO8601(date)`: Returns an ISO8601 datetime string from `date`.
The datetime string will always use UTC time, indicated by the `Z` at its end.

– `DATE_ISO8601(year, month, day, hour, minute, second, millisecond)`:
same as before, but allows specifying the individual date components separately.
All parameters after `day` are optional.

These two above date functions accept the following input values:

– numeric timestamps, indicating the number of milliseconds elapsed since the UNIX
epoch (i.e. January 1st 1970 00:00:00 UTC).
An example timestamp value is `1399472349522`, which translates to
`2014-05-07T14:19:09.522Z`.

– datetime strings in formats `YYYY-MM-DDTHH:MM:SS.MMM`, `YYYY-MM-DD HH:MM:SS.MMM`, or
`YYYY-MM-DD`. Milliseconds are always optional.

A timezone difference may optionally be added at the end of the string, with the
hours and minutes that need to be added or subtracted to the datetime value.
For example, `2014-05-07T14:19:09+01:00` can be used to specify a one hour offset,
and `2014-05-07T14:19:09+07:30` can be specified for seven and half hours offset.
Negative offsets are also possible. Alternatively to an offset, a `Z` can be used
to indicate UTC / Zulu time.

An example value is `2014-05-07T14:19:09.522Z` meaning May 7th 2014, 14:19:09 and
522 milliseconds, UTC / Zulu time. Another example value without time component is
`2014-05-07Z`.

Please note that if no timezone offset is specified in a datestring, ArangoDB will
assume UTC time automatically. This is done to ensure portability of queries across
servers with different timezone settings, and because timestamps will always be
UTC-based.

– individual date components as separate function arguments, in the following order:
– year
– month
– day
– hour
– minute
– second
– millisecond

All components following `day` are optional and can be omitted. Note that no
timezone offsets can be specified when using separate date components, and UTC /
Zulu time will be used.

The following calls to `DATE_TIMESTAMP` are equivalent and will all return
`1399472349522`:

DATE_TIMESTAMP(“2014-05-07T14:19:09.522”)
DATE_TIMESTAMP(“2014-05-07T14:19:09.522Z”)
DATE_TIMESTAMP(“2014-05-07 14:19:09.522”)
DATE_TIMESTAMP(“2014-05-07 14:19:09.522Z”)
DATE_TIMESTAMP(2014, 5, 7, 14, 19, 9, 522)
DATE_TIMESTAMP(1399472349522)

The same is true for calls to `DATE_ISO8601` that also accepts variable input
formats:

DATE_ISO8601(“2014-05-07T14:19:09.522Z”)
DATE_ISO8601(“2014-05-07 14:19:09.522Z”)
DATE_ISO8601(2014, 5, 7, 14, 19, 9, 522)
DATE_ISO8601(1399472349522)

The above functions are all equivalent and will return `”2014-05-07T14:19:09.522Z”`.

The following date functions can be used with dates created by `DATE_TIMESTAMP` and
`DATE_ISO8601`:

– `DATE_DAYOFWEEK(date)`: Returns the weekday number of `date`. The return values have
the following meanings:
– 0: Sunday
– 1: Monday
– 2: Tuesday
– 3: Wednesday
– 4: Thursday
– 5: Friday
– 6: Saturday

– `DATE_YEAR(date)`: Returns the year part of `date` as a number.

– `DATE_MONTH(date)`: Returns the month part of `date` as a number.

– `DATE_DAY(date)`: Returns the day part of `date` as a number.

– `DATE_HOUR(date)`: Returns the hour part of `date` as a number.

– `DATE_MINUTE(date)`: Returns the minute part of `date` as a number.

– `DATE_SECOND(date)`: Returns the seconds part of `date` as a number.

– `DATE_MILLISECOND(date)`: Returns the milliseconds part of `date` as a number.

The following other date functions are also available:

– `DATE_NOW()`: Returns the current time as a timestamp.

Note that this function is evaluated on every invocation and may return different
values when invoked multiple times in the same query.

The following other AQL functions have been added in ArangoDB 2.1:

– `FLATTEN`: this function can turn a list of sub-lists into a single flat list. All
list elements in the original list will be expanded recursively up to a configurable
depth. The expanded values will be added to the single result list.

Example:

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ])

will expand the sub-lists on the first level and produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, [ 9, 10 ] ]

To fully flatten the list, the maximum depth can be specified (e.g. with a value of `2`):

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ], 2)

This will fully expand the sub-lists and produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

– `CURRENT_DATABASE`: this function will return the name of the database the current
query is executed in.

– `CURRENT_USER`: this function returns the name of the current user that is executing
the query. If authorization is turned off or the query is executed outside of a
request context, no user is present and the function will return `null`.

Cluster Dump and Restore
————————

The dump and restore tools, _arangodump_ and _arangorestore_, can now be used to
dump and restore collections in a cluster. Additionally, a collection dump from
a standalone ArangoDB server can be imported into a cluster, and vice versa.

Web Interface Improvements
————————–

The web interface in version 2.1 has a more compact dashboard. It provides
charts with time-series for incoming requests, HTTP transfer volume and some
server resource usage figures.

Additionally it provides trend indicators (e.g. 15 min averages) and
distribution charts (aka histogram) for some figures.

Foxx Improvements
—————–

To easily access a file inside the directory of a Foxx application from within
Foxx, Foxx’s `applicationContext` now provides the `foxxFilename()` function. It
can be used to assemble the full filename of a file inside the application’s
directory. The `applicationContext` can be accessed as global variable from any
module within a Foxx application.

The filename can be used inside Foxx actions or setup / teardown scripts,
e.g. to populate a Foxx application’s collection with data.

The `require` function now also prefers local modules when used from inside a
Foxx application. This allows putting modules inside the Foxx application
directory and requiring them easily. It also allows using application-specific
versions of libraries that are bundled with ArangoDB (such as underscore.js).

Windows Installer
—————–

The Windows installer shipped with ArangoDB now supports installation of
ArangoDB for the current user or all users, with the required privileges. It
also supports the installation of ArangoDB as a service.

Fixes for 32 bit systems
————————

Several issues have been fixed that occured only when using ArangoDB on a 32 bits
operating system, specifically:

– a crash in a third party component used to manage cluster data

– a third party library that failed to initialize on 32 bit Windows, making arangod
and arangosh crash immediately.

– overflows of values used for nanosecond-precision timeouts: these overflows
have led to invalid values being passed to socket operations, making them fail
and re-try too often

Updated drivers
—————

Several drivers for ArangoDB have been checked for compatibility with 2.1. The
current list of drivers with compatibility notes can be found online
[here](https://test.arangodb.com/driver).

C++11 usage
———–

We have moved several files from C to C++, allowing more code reuse and reducing
the need for shipping data between the two. We have also decided to require
C++11 support for ArangoDB, which allows us to use some of the simplifications,
features and guarantees that this standard has in stock.

That also means a compiler with C++11 support is required to build ArangoDB from
source. For instance GNU CC of at least version 4.8.

Miscellaneous Improvements
————————–

– Cancelable asynchronous jobs: several potentially long-running jobs can now be
cancelled via an explicit cancel operation. This allows stopping long-running
queries, traversals or scripts without shutting down the complete ArangoDB
process. Job cancellation is provided for asynchronously executed jobs as is
described in @ref HttpJobCancel.

– Server-side periodic task management: an ArangoDB server now provides
functionality to register and unregister periodic tasks. Tasks are
user-defined JavaScript actions that can be run periodically and
automatically, independent of any HTTP requests.

The following task management functions are provided:

– require(“org/arangodb/tasks”).register(): registers a periodic task
– require(“org/arangodb/tasks”).unregister(): unregisters and removes a periodic task
– require(“org/arangodb/tasks”).get(): retrieves a specific tasks or all existing tasks

An example task (to be executed every 15 seconds) can be registered like this:

var tasks = require(“org/arangodb/tasks”);
tasks.register({
name: “this is an example task with parameters”,
period: 15,
command: function (params) {
var greeting = params.greeting;
var data = JSON.stringify(params.data);
require(‘console’).log(‘%s from parameter task: %s’, greeting, data);
},
params: { greeting: “hi”, data: “how are you?” }
});

Please refer to the section @ref Tasks for more details.

– The `figures` method of a collection now returns data about the collection’s
index memory consumption. The returned value `indexes.size` will contain the
total amount of memory acquired by all indexes of the collection. This figure
can be used to assess the memory impact of indexes.

– Capitalized HTTP response headers: from version 2.1, ArangoDB will return
capitalized HTTP headers by default, e.g. `Content-Length` instead of
`content-length`. Though the HTTP specification states that headers field
name are case-insensitive, several older client tools rely on a specific case
in HTTP response headers. This changes make ArangoDB a bit more compatible
with those.

– Simplified usage of `db._createStatement()`: to easily run an AQL query, the
method `db._createStatement` now allows passing the AQL query as a string.
Previously it required the user to pass an object with a `query` attribute
(which then contained the query string).

ArangoDB now supports both versions:

db._createStatement(queryString);
db._createStatement({ query: queryString });