home shape

More Efficient Data Exports with new Export API

ArangoDB 2.6 provides a specialized export API for exporting all documents from a collection and shipping them to a client application. It is rather limited but faster than the general-purpose AQL cursor API and can store its snapshots using less memory.

export_api

A side effect of the speedup is that the first results will arrive much earlier in the client application. This will help in reducing client connection timeouts in case clients are enforcing them on temporarily non-responding connections.

The following table shows the execution times for fetching the first 1,000 documents from collections of different sizes, both with via the cursor API and the export API. Figures for the cursor API are shown for ArangoDB 2.5 and 2.6 (the version in which it was rewritten):
export_api_performance

Read more about the new export API in Jan’s blog post on more efficient data exports.

Ingo

Ingo Friepoertner

Ingo is dealing with all the good ideas from the ArangoDB community, customers and industry experts to improve the value provided by the company’s native multi-model approach. In former positions he worked as a product owner and tech consultant, building custom software solutions for large companies in various industries. Ingo holds a diploma in business informatics from FHDW University of Applied Sciences.

3 Comments

  1. CoDEmanX on April 8 2015, at 10:53 am

    Really nice! Any plans on extending the export API yet? Like a consistent snapshot of an entire database? Alternatively, a read-only maintenance mode would also do, and maybe even work in clusters.

    • jsteemann on May 8 2015, at 4:16 pm

      Sorry for the delay, but I stumbled upon your comment just now.

      So far the export API is pretty limited. It is designed purely to get a single collection’s documents out as efficiently as possible. There are no concrete thoughts yet about extending the export API into the directions you mentioned, though I won’t exclude that this is a future option.

      Regarding the snapshotting:
      For creating a snapshot of an entire database, there is arangodump. AFAIK it should create a consistent snapshot of the database even if data modifications happen while its dumping. There are some restrictions though: the dump won’t be consistent in a cluster and there might be issues if collections are dropped while the dump is ongoing. Finally, a full arangodump run may take a long time when run on a bigger database. But for smaller instances arangodump should be fine already.

  2. ArangoDB database on January 10 2018, at 9:47 pm

    Sorry for the late reply… Yes, we released a new arangoexport feature. Please read more here: https://www.arangodb.com/2017/02/arangoexport-tool-exporting-data-arangodb/

Leave a Comment





Get the latest tutorials, blog posts and news: