Skip to content 
Configuring ArangoDB-PHP to use active failover
This article is about setting up active failover for ArangoDB-PHP, the PHP client driver for ArangoDB. It requires ArangoDB-PHP 3.3.2 or higher, and an ArangoDB server version of 3.3.4 or higher.
Interested in trying out ArangoDB? Fire up your cluster in just a few clicks with ArangoDB ArangoGraph: the Cloud Service for ArangoDB. Start your free 14-day trial here
Active failover: basic setup
Historically, ArangoDB-PHP has been able to connect to a single ArangoDB endpoint, i.e. one combination of IP address and port number.
To connect to an ArangoDB server that is running on localhost or on a remote server, simply set the `OPTION_ENDPOINT` item in the `ConnectionOptions` and connect: Read more
ArangoDB | Milestone2: ArangoDB 3.3 New Data Replication
We’re pleased to announce the availability of the Milestone 2 of ArangoDB 3.3. There are a number of improvements, please consult the changelog for a complete overview of changes.
This milestone release contains our new and improved data replication engine. The replication engine is at the core of every distributed ArangoDB setup: whether it is a typical master/slave setup between multiple single servers or a full-fledged cluster. During the last month we:
- redesigned the replication protocol to be more reliable
- refactored and modernized the internal infrastructure to better support continuous asynchronous replication
- added a new global asynchronous replication API, to allow you to automatically and continuously mirror an entire ArangoDB single-instance (master) onto another one (or more)
- added support for automatic failover from a master server to one of his replica-slaves, if the master server becomes unreachable
Milestone 1 ArangoDB 3.3: Datacenter to Datacenter Replication
Every company needs a disaster recovery plan for all important systems. This is true from small units like single processes running in some container to the largest distributed architectures. For databases in particular this usually involves a mixture of fault-tolerance, redundancy, regular backups and emergency plans. The larger a data store, the more difficult is it to come up with a good strategy.
Therefore, it is desirable to be able to run a distributed database in one datacenter and replicate all transactions to another datacenter in some way. Often, transaction logs are shipped over the network to replicate everything in another, identical system in the other datacenter. Some distributed data stores have built-in support for multiple datacenter awareness and can replicate between datacenters in a fully automatic fashion.
This post gives an overview over the first evolutionary step of ArangoDB towards multi-datacenter support, which is asynchronous datacenter to datacenter replication.
Setting up Datacenter to Datacenter Replication in ArangoDB
Please note that this tutorial is valid for the ArangoDB 3.3 milestone 1 version of DC to DC replication!
Interested in trying out ArangoDB? Fire up your cluster in just a few clicks with ArangoDB ArangoGraph: the Cloud Service for ArangoDB. Start your free 14-day trial here
This milestone release contains data-center to data-center replication as an enterprise feature. This is a preview of the upcoming 3.3 release and is not considered production-ready.
In order to prepare for a major disaster, you can setup a backup data center that will take over operations if the primary data center goes down. For a server failure, the resilience features of ArangoDB can be used. Data center to data center is used to handle the failure of a complete data center.
Data is transported between data-centers using a message queue. The current implementation uses Apache Kafka as message queue. Apache Kafka is a commonly used open source message queue which is capable of handling multiple data-centers. However, the ArangoDB replication is not tied to Apache Kafka. We plan to support different message queues systems in the future.
The following contains a high-level description how to setup data-center to data-center replication. Detailed instructions for specific operating systems will follow shortly. Read more
Webinar: Use ArangoDB Agency as fault-tolerant persistent data store
Join our Sr Distributed System Engineer, Kaveh Vahedipour, to learn more about ArangoDB Agency on September 19th, 2017 (6PM CEST/12PM ET/ 9AM PT) – View the Recording.
Distributed systems have become the standard topology on which modern appliances live. While the advantages of distributing workload for both performance as well as fault-tolerance are obvious, the runtime flexible configuration of such deployment becomes non-trivial.
ArangoDB clusters are no different in that regard. A potentially large database cluster’s configuration is manipulated at runtime by addition, alteration and removal of collections, indexes, and even servers. All servers need to trust in a fault-tolerant centralized configuration tree, which we call “the agency” in arango-speak. Read more
ArangoDB | Pronto Move Shard – Multi-Model NoSQL Database
In July Adobe announced that they plan the End-of-Life for flash at around 2020.
As HTML5 progressed and due to a long history of critical security vulnerabilities this is – technologically speaking – certainly the right decision. However I tended to also become a bit sad.
Flash was the first technology that brought interactivity to the web. We tend to forget how static the web was in the early 2000s. Flash brought life to the web and there were plenty of stupid trash games and animations which I really enjoyed at the time. As a homage to the age of trashy flash games I created a game which resembles the games of this era: Read more
The new SatelliteCollections Feature of ArangoDB
With the new Version 3.2 we have introduced a new feature called SatelliteCollections. This post explains what this is all about, how it can help you, and explains a concrete use case for which it is essential.
Background and Overview
Join operations are very useful but can be troublesome in a distributed database. This is because quite often, a join operation has to bring together different pieces of your data that reside on different machines. This leads to cluster internal communication and can easily ruin query performance. As in many contexts nowadays, data locality is very important to avoid such headaches. There is no silver bullet, because there will be many cases in which one cannot do much to improve data locality.
One particular case in which one can achieve something, is if you need a join operation between a very large collection (sharded across your cluster) and a small one, because then one can afford to replicate the small collection to every server, and all join operations can be executed without network communications.
ArangoDB 3.2 Beta: RocksDB Storage Engine & Distributed Graph Cluster
We’re excited to release today the beta of ArangoDB 3.2. It’s feature rich, well tested and hopefully plenty of fun for all of you. Keen to take it for a spin? Get ArangoDB 3.2 beta here.
With ArangoDB 3.2, we’re introducing the long-awaited pluggable storage engine and its first new citizen, RocksDB from Facebook
- RocksDB: You can now use as much data in ArangoDB as you can fit on your disk. Plus, you can enjoy performance boosts on writes by having only document-level locks (more info below).
- Pregel: Furthermore, we implemented distributed graph processing with Pregel for discovering hidden patterns, identify communities and perform in-depth analytics of large graph data sets.
- ClusterFoxx: Another important upgrade is what we internally and playfully call the ClusterFoxx. The Foxx management internals have been rewritten from the ground up to make sure multi-coordinator cluster setups always keep their services in sync and new coordinators are fully initialised even when all existing coordinators are unavailable.
- Enterprise: Working with some of our largest customers, we’ve added further security and scalability features to ArangoDB Enterprise like LDAP integration, Encryption at Rest, and the brand new Satellite Collections.
The goal of the whole ArangoDB 3 release cycle has been to scale the multi-model idea to new heights. Getting ‘ready’ for large scale applications is not done overnight and it’s definitely not possible without the help of a strong community. We’d like to invite all of you to lend us a helping hand to make ArangoDB 3.2 the best release ever. Please push this beta to its limits: test it for your use cases and compare the performance of the new features like RocksDB. Let us know on Github any bug that you find. Don’t worry about hurting our feelings: we want to fix any problems.
Join the Beta Bug Hunt Challenge and win a $200 Amazon Gift Card as first prize. You can find more details about this reward program at the end of this post.
New Storage Engine RocksDB
ArangoDB now comes with two storage engines: mmfiles and RocksDB. If you want to compare the engines, you can use arangodump to export data from either engine and arangorestore to import into the other. MMFILES are generally well suited for use-cases that fit into main memory, while RocksDB allows larger than memory work-sets.
RocksDB has plenty of configuration options; we have selected the general purpose options. Please let us know how it works for your use case so that we can further optimize the implementation. Also notice that we do many tests under Linux, Windows and macOS. However, we optimize for Linux. Any feedback regarding other operating systems is very welcome. Check out the step by step guide to compare both storage engines for your use case and OS!
Benefits of RocksDB Storage Engine:
- Document-level locks: performance boost for write intensive applications. Writes don’t block reads, and reads don’t block writes
- Support for large datasets: go beyond the limit of main memory and stay performant
- Persistent indexes: faster index build after restart
Things to consider before switching to RocksDB
- RocksDB allows concurrent writes: Write conflicts can be raised. Applications switching from MMFiles must be prepared for exceptions
- Transaction Limit in RocksDB: The total size of transactions is limited in RocksDB. Modifying statements on large amounts of documents have to commit in between -- with AQL this is done by default.
- Engine Selection on Server/Cluster Level: It’s not possible to mix both storage engines within a single instance or cluster installation. Transaction handling and write ahead log formats are different.
Find all important details about RocksDB in the storage engine documentation, as well as answers to common questions about our RocksDB integration in the FAQ.
Please note that ArangoDB 3.2 beta is fully tested, but not yet fully optimized (known-issues RocksDB). If you find something that is much slower with RocksDB compared to your current queries with the MMFiles engine, please create a github ticket. Please check the comparison guide here.
New Distributed Graph Processing
With the new implementation of distributed graph processing, you are now able to analyze even very large graph data sets as a whole. Internally, we implemented the pregel computing model to enable ArangoDB to support arbitrary graph algorithms, which will scale with your data -- or with the size of your database cluster.
You can already use a number of well-known graph algorithms:
- PageRank
- Weakly Connected Components
- Strongly Connected Components
- HITS (hubs and authorities)
- Single-Source Shortest Path
- Community Detection via Label Propagation
- Vertex Centrality measures
- Closeness Centrality via Effective Closeness
- Betweenness Centrality via LineRank
By using these new capabilities, you are now able, for example, to detect communities within your graph, shard your data according to these communities and leverage ArangoDB SmartGraphs to perform highly efficient graph traversals even in a cluster setup.
New ClusterFoxx
Managing your JavaScript microservices is now easier and more reliable than ever before. The Foxx management internals have been rewritten to make sure multi-coordinator cluster setups always keep their services in sync and new coordinators are fully initialised even when all existing coordinators are unavailable.
Additionally, the new fully documented REST API for managing Foxx services enables you to install, upgrade and configure your services using your existing devops processes. And if your service only consists of a single JavaScript file, you can now forego the manifest and upload that file directly, instead of creating a full bundle.
Further useful new features included in ArangoDB 3.2 beta
- geo_cursor: Get documents sorted by distance to a certain point in space. You can also apply filters and limits to geo_cursor.
- arangoexport: Export data as JSON, JSONL and even graphs as XGMML for visualisation in Cytoscape. You can find details in the Alpha2 release post.
Download ArangoDB 3.2 beta Community
New Enterprise Edition Features in 3.2
The Enterprise Edition of ArangoDB is focused to solve enterprise-scale problems and meet high security standards. In version 3.1, we introduced SmartGraphs to bring fast traversal times to sharded graph datasets. With SatelliteCollections, we enable the same performance boost to join operations at scale.
SatelliteCollections
From genome-sequencing projects to massive online games and beyond, we see the need for join operations including sharded collections and sub-second response times.
With SatelliteCollections, you can define collections to shard to a cluster and collections to replicate to each machine. The ArangoDB query optimizer knows where each shard is located and sends the requests to the DBServers involved, which then executes the query, locally. With this approach, network hops during join operations on sharded collections can be avoided and response times can be close to that of a single instance.
In the example below, collection C is large and sharded to multiple machines while the smaller satellites (S1-S5) are replicated to each machine.
We are super excited to see what you will create with this new feature and welcome any feedback you can provide. The Enterprise Edition of ArangoDB is forever free for evaluation. So feel free to take it for a spin.
Encryption at Rest
With RocksDB, you can encrypt the data stored on disk using a highly secure AES algorithm. With this upgrade, ArangoDB takes another big step towards HIPPA compliance. Even if someone steals one of your discs, they won’t be able to access the data.
Enhanced Authorisation with LDAP
Normally, users are defined and managed in ArangoDB itself. With LDAP, you can use an external server to manage your users. We have implemented a common schema which can be extended. If you have special requirements that do not fit into this schema, please let us know (#feedback32 channel). A general note: The final release will also support read-only users. With this beta, only read/write users are supported.
Download ArangoDB 3.2 beta Enterprise
Bug Hunt Competition
We’d love to invite all of you to try ArangoDB 3.2 beta and report all bugs you can find -- we’re hoping there won’t be any, but there always are some. Everyone reporting bugs for 3.2 beta on Github will take part in the Beta Bug Hunt Competition. All Github issues count for version 3.2 beta which are marked with bug by the ArangoDB team. The hunter with the most reported bugs wins. The first three runner-ups will receive an honorable mention in the Bug Hunt Challenge post.
How the Bug Hunt Competition works:
- Duration: from June 13th until June 27th
- Bugs: Any Github issue for 3.2 beta release which is marked as bug by ArangoDB team
- Who can participate: everyone
- First Prize: a $200 Amazon Gift Card + ArangoDB Swag Package
- Second Prize: $100 Amazon Gift Card + ArangoDB Swag Package
- Runner-Up: ArangoDB Swag Package
All winners will receive an honorable mention in the bug hunt post and tweets after the challenge. Please note that our team has to be able to reproduce the bug.
Therefore good bug reports
- Have only necessary infos included
- Provide a step by step description to reproduce the bug
- Provide demo data via e.g. gist (if necessary)
We hope you will enjoy this new release - “Gute Jagd!”
Legal
In connection with your participation in this program you agree to comply with all applicable local and national laws. You must not disrupt or compromise any data that is not your own.
ArangoDB reserves the right to discontinue this program or change or modify its terms at any time. The ultimate decision over an award -- whether to give one and in what amount -- is entirely at ArangoDB’s discretion.
You are exclusively responsible for paying any taxes on any reward you receive.
Vulnerabilities obtained by exploiting ArangoDB users or instances on the Internet are not eligible for an award and will result in immediate disqualification from the program.
ArangoDB: Consensus for Enhanced Data Stability
– law of the polish-lithuanian common-wealth, 1505
A warning aforehand: this is a rather longish post, but hang in there it might be saving you a lot of time one day.
Introduction
Consensus has its etymological roots in the latin verb consentire, which comes as no surprise to mean to consent, to agree. As old as the verb equally old is the concept in the brief history of computer science. It designates a crucial necessity of distributed appliances. More fundamentally, consensus wants to provide a fault-tolerant distributed animal brain to higher level appliances such as deployed cluster file systems, currency exchange systems, or specifically in our case distributed databases, etc. Read more
Starting an ArangoDB cluster the easy way
Recently, we have got a lot of feedback about the fact that standing up an ArangoDB cluster “manually” is an awkward and error-prone affair. We have been aware of this for some time, but always expected that most users running ArangoDB clusters would do so on Apache Mesos or DC/OS, where deployment is a breeze due to our ArangoDB framework.
However, for various valid reasons people do not want to use Apache Mesos and thus are back to square one with the problem of deploying an ArangoDB cluster without Apache Mesos. Read more
- « Previous
- 1
- 2
- 3
- 4
- Next »