home shape

ArangoDB Spark Connector: Seamless Integration

Currently we are diving deeper into the Apache Spark world. We started with an implementation of a Spark-Connector written in Scala. The connector supports loading of data from ArangoDB into Spark and vice-versa. Today we release a first prototype with an aim of including our community into the development process early to build a product that fits your needs. Your feedback is more than welcome!


Setup SparkContext

First you need to initialize a SparkContext with the configuration for the Spark-Connector and the underlying Java Driver (see corresponding blog post here) to connect to your ArangoDB server.


val conf = new SparkConf()
    .set("arangodb.host", "")
    .set("arangodb.port", "8529")
    .set("arangodb.user", "myUser")
    .set("arangodb.password", "myPassword")
val sc = new SparkContext(conf)


SparkConf conf = new SparkConf()
    .set("arangodb.host", "")
    .set("arangodb.port", "8529")
    .set("arangodb.user", "myUser")
    .set("arangodb.password", "myPassword");
JavaSparkContext sc = new JavaSparkContext(conf);

Load data from ArangoDB

To load data from ArangoDB, use the function load – from the object ArangoSpark – with the SparkContext, the name of your collection and the type of your bean to load data in. If needed, there is an additional load function with extra read options like the name of the database.


val rdd = ArangoSpark.load[MyBean](sc, "myCollection")


ArangoJavaRDD rdd = ArangoSpark.load(sc, "myCollection", MyBean.class);

Save data to ArangoDB

To save data to ArangoDB, use the function save – from the object ArangoSpark – with the SparkContext and the name of your collection. If needed, there is an additional save function with extra write options like the name of the database.

Scala / Java

ArangoSpark.save(rdd, "myCollection")

It would be great if you try it out and give us feedback on what you think.


Michele Rastelli

Michele is a software engineer with experience in algorithms, web architectures, and data modeling. He has been working as a technical leader for web startups, designing and implementing scalable and resilient microservices applications. He currently works at ArangoDB on the development of the Java driver and frameworks integrations.


  1. Aravindan on January 28, 2020 at 7:15 am

    Is there a Pyspark API/wrapper available to connect with ArangoDB?

  2. Michele Rastelli on February 4, 2020 at 3:26 pm

    Unfortunately not yet. You can check the available Python drivers here: https://www.arangodb.com/arangodb-drivers/

Leave a Comment

Get the latest tutorials, blog posts and news: