SatelliteGraph Details

Below you find usage examples and advanced configuration possibilities for SatelliteGraphs. The examples use arangosh and the @arangodb/satellite-graph module. You can also manage SatelliteGraphs via the HTTP API.

How to create a SatelliteGraph

SatelliteGraphs enforce and rely on special properties of the underlying collections and hence can only work with collections that are either created implicitly through the SatelliteGraph interface, or manually with the correct properties:

• There needs to be a prototype collection with replicationFactor set to "satellite"
• All other collections need to have distributeShardsLike set to the name of the prototype collection

Collections can be part of multiple SatelliteGraphs. This means that in contrast to SmartGraphs, SatelliteGraphs can be overlapping. If you have a larger SatelliteGraph and want to create an additional SatelliteGraph which only covers a part of it, then you can do so.

Create a graph

To create a SatelliteGraph in arangosh, use the satellite-graph module:

arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
arangosh> satelliteGraphModule._graph("satelliteGraph");

{[SatelliteGraph] 
}

In contrast to General Graphs and SmartGraphs, you do not need to take care of the sharding and replication properties. The properties distributeShardsLike, replicationFactor and numberOfShards will be set automatically.

Add vertex collections

Adding vertex collections is analogous to General Graphs:

arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
arangosh> graph._addVertexCollection("aVertexCollection");

(Empty output)

If the collection "aVertexCollection" doesn’t exist yet, then the SatelliteGraph module will create it automatically with the correct properties. If it exists already, then its properties must be suitable for a SatelliteGraph (see prototype collection). Otherwise it will not be added.

Define relations

Adding edge collections works the same as with General Graphs, but again, the collections are created by the SatelliteGraph module with the right properties if they don’t exist already.

arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
arangosh> var relation = satelliteGraphModule._relation("isFriend", ["person"], ["person"]);
arangosh> graph._extendEdgeDefinitions(relation);

(Empty output)

Existing edge collections can be added, but they require the distributeShardsLike property to reference the prototype collection.

The prototype collection

Every SatelliteGraph needs exactly one document collection with replicationFactor set to "satellite". This automatically leads to the collection having an exact amount of one shard per collection. This collection is selected as prototype.

All other collections of the SatelliteGraph need to inherit its properties by referencing its name in the distributeShardsLike property.

If collections are created implicitly through the SatelliteGraph module, then this is handled for you automatically. If you want to create the collections manually before adding them to the SatelliteGraph, then you need to take care of these properties.

Prototype collection examples

Creating an empty SatelliteGraph: No prototype collection is present.

arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
arangosh> satelliteGraphModule._create("satelliteGraph");

{[SatelliteGraph] 
}

Creating an empty SatelliteGraph, then adding a document (vertex) collection. This leads to the creation of a prototype collection "myPrototypeColl" (assuming that no collection with this name existed before):

arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
arangosh> graph._addVertexCollection("myPrototypeColl");

(Empty output)

Creating an empty SatelliteGraph, then adding an edge definition. This will select the collection "person" as prototype collection, as it is the only document (vertex) collection. If you supply more than one document collection, then one of the collections will be chosen arbitrarily as prototype collection.

arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
arangosh> var graph = satelliteGraphModule._create("satelliteGraph");
arangosh> var relation = satelliteGraphModule._relation("isFriend", ["person"], ["person"]);
arangosh> graph._extendEdgeDefinitions(relation);

(Empty output)

The prototype collection can and also will be automatically selected during the graph creation process if at least one document (vertex) collection is supplied directly. If more then one are available, they will be chosen randomly as well, regardless whether they are set inside the edge definition itself or set as a vertex/orphan collection.

Utilizing SatelliteGraphs

Obviously, a SatelliteGraph must be created before it can be queried. Valid operations that can then be optimized are (k-)shortest path(s) computations and traversals. Both also allow for combination with local joins or other SatelliteGraph operations.

Here is an example showing the difference between the execution of a General Graph and a SatelliteGraph traversal query:

1. First we setup our graphs and collections.

arangosh> var graphModule = require("@arangodb/general-graph");
arangosh> var satelliteGraphModule = require("@arangodb/satellite-graph");
arangosh> graphModule._create("normalGraph", [ graphModule._relation("edges", "vertices", "vertices") ], [], {});
arangosh> satelliteGraphModule._create("satelliteGraph", [ satelliteGraphModule._relation("satEdges", "satVertices", "satVertices") ], [], {});
arangosh> db._create("collection", {numberOfShards: 8});

{[Graph] 
  "edges" : [ArangoCollection 10073, "edges" (type edge, status loaded)], 
  "vertices" : [ArangoCollection 10072, "vertices" (type document, status loaded)] 
}
{[SatelliteGraph] 
  "satEdges" : [ArangoCollection 10079, "satEdges" (type edge, status loaded)], 
  "satVertices" : [ArangoCollection 10077, "satVertices" (type document, status loaded)] 
}
[ArangoCollection 10081, "collection" (type document, status loaded)]

1. Let us analyze a query involving a traversal:

arangosh> db._explain(`FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "normalGraph" RETURN [doc,v,e,p]`, {}, {colors: false});

Query String (99 chars, cacheable: true):
 FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "normalGraph" RETURN [doc,v,e,p]

Execution plan:
 Id   NodeType                  Site  Est.   Comment
  1   SingletonNode             DBS      1   * ROOT
  2   EnumerateCollectionNode   DBS      0     - FOR doc IN collection   /* full collection scan, 8 shard(s)  */
  8   RemoteNode                COOR     0       - REMOTE
  9   GatherNode                COOR     0       - GATHER   /* unsorted */
  3   TraversalNode             COOR     1       - FOR v  /* vertex */, e  /* edge */, p  /* paths: vertices, edges */ IN 1..1  /* min..maxPathDepth */ OUTBOUND 'vertices/start' /* startnode */  GRAPH 'normalGraph'
  4   CalculationNode           COOR     1         - LET #6 = [ doc, v, e, p ]   /* simple expression */   /* collections used: doc : collection */
  5   ReturnNode                COOR     1         - RETURN #6

Indexes used:
 By   Name   Type   Collection   Unique   Sparse   Selectivity   Fields        Ranges
  3   edge   edge   edges        false    false       100.00 %   [ `_from` ]   base OUTBOUND

Traversals on graphs:
 Id  Depth  Vertex collections  Edge collections  Options                                  Filter / Prune Conditions
 3   1..1   vertices            edges             uniqueVertices: none, uniqueEdges: path                           

Optimization rules applied:
 Id   RuleName
  1   scatter-in-cluster
  2   remove-unnecessary-remote-scatter

Optimization rules with highest execution times:
 RuleName                                        Duration [s]
 scatter-in-cluster                                   0.00001
 restrict-to-single-shard                             0.00001
 use-indexes                                          0.00000
 remove-unnecessary-remote-scatter                    0.00000
 optimize-traversals                                  0.00000

59 rule(s) executed, 1 plan(s) created

You can see that the TraversalNode is executed on a Coordinator, and only the EnumerateCollectionNode is executed on DB-Servers. This will happen for each of the 8 shards in collection.

1. Let us now have a look at the same query using a SatelliteGraph:

arangosh> db._explain(`FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "satelliteGraph" RETURN [doc,v,e,p]`, {}, {colors: false});

Query String (102 chars, cacheable: true):
 FOR doc in collection FOR v,e,p IN OUTBOUND "vertices/start" GRAPH "satelliteGraph" RETURN 
 [doc,v,e,p]

Execution plan:
 Id   NodeType                  Site  Est.   Comment
  1   SingletonNode             DBS      1   * ROOT
  2   EnumerateCollectionNode   DBS      0     - FOR doc IN collection   /* full collection scan, 8 shard(s)  */
 10   TraversalNode             DBS      1       - FOR v  /* vertex */, e  /* edge */, p  /* paths: vertices, edges */ IN 1..1  /* min..maxPathDepth */ OUTBOUND 'vertices/start' /* startnode */  satEdges /* local graph node, used as satellite */
  4   CalculationNode           DBS      1         - LET #6 = [ doc, v, e, p ]   /* simple expression */   /* collections used: doc : collection */
 13   RemoteNode                COOR     1         - REMOTE
 14   GatherNode                COOR     1         - GATHER   /* parallel, unsorted */
  5   ReturnNode                COOR     1         - RETURN #6

Indexes used:
 By   Name   Type   Collection   Unique   Sparse   Selectivity   Fields        Ranges
 10   edge   edge   satEdges     false    false       100.00 %   [ `_from` ]   base OUTBOUND

Traversals on graphs:
 Id  Depth  Vertex collections  Edge collections  Options                                  Filter / Prune Conditions
 10  1..1   satVertices         satEdges          uniqueVertices: none, uniqueEdges: path                           

Optimization rules applied:
 Id   RuleName
  1   scatter-in-cluster
  2   scatter-satellite-graphs
  3   remove-satellite-joins
  4   distribute-filtercalc-to-cluster
  5   remove-unnecessary-remote-scatter
  6   parallelize-gather

Optimization rules with highest execution times:
 RuleName                                        Duration [s]
 scatter-satellite-graphs                             0.00001
 scatter-in-cluster                                   0.00001
 remove-satellite-joins                               0.00001
 parallelize-gather                                   0.00000
 restrict-to-single-shard                             0.00000

59 rule(s) executed, 1 plan(s) created

Note that now the TraversalNode is executed on each DB-Server, leading to a great reduction in required network communication, and hence potential gains in query performance.

Convert General Graphs or SmartGraphs to SatelliteGraphs

If you want to transform an existing General Graph or SmartGraph into a SatelliteGraph, then you need to dump and restore your previous graph. This is necessary for the initial data replication and because some collection properties are immutable.

Use arangodump and arangorestore. The only thing you have to change in this pipeline is that you create the new collections during creation with the SatelliteGraph module or add collections manually to the SatelliteGraph before starting the arangorestore process.