home shape
AskBlue-logo
right blob img min

Creating A Location-Aware Recommendation Engine In 4 Weeks

“AskBlue is a company that provides specialized consulting services in the financial sector and information technology with proven financial sector knowledge in complex, high-level, multinational organizational contexts. Currently, they employ 150 people, who are mostly working on projects for their main customers, like Santander, Caixa Geral de Depóstitos, Bankinter, BPI, Millenium, and CTT.

For the startup BeOnit, a company focusing on location-aware apps from Portugal, Alex Pavlov and his team at AskBlue led the software development of the location-based recommendation engine and storage backend based on ArangoDB. The application BeOnit was introduced to the public by the company BeOnit at WebSummit 2017 and finally launched in January 2018. The App is now available for Android and iOS.”

by Case Study by Alex Pavlov (Solution Architect & Lead Software Engineer, AskBlue)

The Challenge

To understand the challenge presented in developing the BeOnit application, you need to understand its purpose. BeOnit is a B2C app that connects people to local businesses or points of interest around them. Based on the geolocation of a user which is determined by GPS, relevant content and recommendations are displayed for the user.

The BeOnit app connects via Wifi, beacon or GPS to the respective location and displays important details or recommendations for this or multiple locations (e.g. restaurants nearby). In addition to this core functionality, users can also book tables, buy tickets for the cinema, or get relevant details. Recommendations have to be very fast and precise in terms of geo-location.

We needed a data store that met the performance requirements, but more importantly we wanted to test a new approach to save on the development side. We needed graphs to go with the recommendations presented to users, but also common join operations as well as a database for storing and processing of geo data.

In general, we wanted an open-source database that could scale horizontally and allow us to write clean, easily understandable code that our client, BeOnit, could afterwards easily maintain themselves. Essentially, we needed a user-friendly NoSQL database which supports real joins with a query language that was easy to learn, and one that supports array as well as custom functions — but can also connect via NodeJS and run smoothly in a cluster setting. That may seem like a tall order, but ArangoDB fit the bill.

right blob min

Why ArangoDB

ArangoDB is open-source, easy to learn and it supports all of the functionality we needed. As it turned out, we got even more than we had hoped.

My team and I prefer to work schemaless, if possible. We don’t like to invest too much in developing a schema that will need to be changed many times. It’s very frustrating to do so in a production database.

In previous projects, we used MongoDB for working with document-oriented projects. However, ArangoDB is an excellent solution because it has everything we need and even non-experts can work with it. For instance, MongoDB only supports the JSON notation. This requires plenty of adjust time for many developers who are new to it. With AQL (ArangoDB Query Language), anyone can easily understand what someone else did. Plus, queries tend to be fast from the get-go.

Since our applications sometime use an SQL-logic, we need native joins. MongoDB is not good at that, but ArangoDB is. What we like the most is the multi-model approach that ArangoDB offers. We can choose freely which data models best suits a given part of our application. We’re not forced to build a data model imperatively into our collections to avoid joins. The freedom in ArangoDB is fantastic!

Actual Use Case

We use ArangoDB for all of the storage needs of the BeOnit app: storing and processing of geo locations (i.e., beacons, router, GPS information), storing and sending notifications, and processing campaigns. We even run the proximity based algorithm on ArangoDB to decide which notification or campaign will be shown to the user.

In the simple image below, you can see the general setup we use. Data from routers, GPS or beacons define the geo-location of the user’s mobile phone. This information gets sent to our NodeJS applications and ArangoDB where calculations are made for the best recommendations and campaigns.

image1-2

We use a few collections to build a complex app. We don’t need to create forty tables with schema as we would with an SQL database. This makes it easy to understand and easy to maintain.

image2-1

We can’t say much about the amount of data that will be stored in ArangoDB since the app was launched only a month ago. But with the target market for this app in Portugal and Brazil, which is a huge market, we expect a steady growth in the size of our dataset.

one query background

Benefits to your Company

First of all, we were impressed how quickly we can develop with ArangoDB. Our clients were surprised to have a working app in their hands after just four weeks.

Previously, I preferred MongoDB. However, it is one thing to write queries in JSON notation and another to use a much simpler query language to compose natural queries. Even the most complex queries in ArangoDB can be understood, built and executed by our colleagues without advanced technical skills. As mentioned before, the whole team really appreciates the AQL. It facilitates convergence in the project and creates a unique environment.

ArangoDB has very important and useful functions to manipulate arrays that many other NoSQL systems don’t have. For example, it can do something like:

LET arr = [[1,2,],3,[4,5],6]
RETURN arr[** FILTER CURRENT % 2 ==0]

In only two lines, with very clear code, we get a result that MongoDB would require a whole block!

Another important feature that we don’t find in many other databases is the ability to execute complex, yet elegant subqueries. The really great thing about AQL is, that also people with basic knowledge can either write queries that are already fast from the get-go or read complex queries and quickly understand what they do.

In the example query below, we do the following:

1. We list promotions and access points (beacons) before we perform the update
2. Perform an UPDATE
3. We list promotions and access points (beacons) after we have performed the update
4. We compare access points and promotions list (before vs after)
5. If we have some new access points and/or new promotions, they are being activated and a notification gets sent out to our customers, something like: “Hello, you have a new access points /promotions detected!”
6. And finally, we store the gps/beacon data in the tracking table.

In AQL this is all possible in one query and it looks like this:

LET isNotify = (FOR doc in config
			FILTER doc.id=='notify'
			RETURN doc.value)
		LET promotionsBefore = (
				FOR doc1 IN promotions
					FILTER doc1.watched ANY == ${userId} && doc1.active==true && doc1.weekdays ANY == ${weekday} && doc1.startdate <= ${myData} && doc1.finishdate >= ${myData}
					RETURN doc1.id_beonit)		
		LET beaconsBefore = (
				FOR doc1 IN accesspoints
				    FILTER POSITION(doc1.users, ${userId}) && doc1.startdate <= ${myData} && doc1.finishdate >= ${myData}
			RETURN doc1.id_beonit) 
		LET apList = (
					FOR doc IN accesspoints 
					FILTER doc.accesspoint_id == ${minor} && doc.type == "beacon"
					RETURN doc.id_beonit)
	    LET apUpdate = (
	        FOR doc in accesspoints
		        FOR ap in apList
			        FILTER doc.id_beonit == ap && doc.startdate <= ${myData} && doc.finishdate >= ${myData}
			            UPDATE 
		                doc with {'users':PUSH(doc.users, ${userId},true)} in accesspoints
		                    RETURN NEW.id_beonit)
		LET promotionsList = (
		    FOR doc in promotions
		     FOR acp in doc.accesspoints
		        FILTER acp IN apList && doc.id_beonit != NULL 
		          RETURN doc.id_beonit)
		LET promotionsUpdate = (FOR doc in promotions
		   FOR pl in promotionsList
			FILTER doc.id_beonit==pl && doc.active==true && doc.weekdays ANY == ${weekday} && doc.startdate <= ${myData} && doc.finishdate >= ${myData} 
			UPDATE 
		doc with {'watched':PUSH(doc.watched, ${userId},true)} in promotions
		RETURN NEW.id_beonit)
		INSERT ${req.body} into tracking
		RETURN {'notificationsActive': isNotify[0],'unchangedPromotions': promotionsUpdate ALL IN promotionsBefore, 'unchangedBeacons': apUpdate ALL IN beaconsBefore}

Here we have several subqueries. There’s no way at this time to do something similar in MongoDB in a single query. It’s so clear and elegant in ArangoDB!

It’s worth noting that ArangoDB not only stores the data, but executes the algorithms and makes the decision to show or not the information in the app (i.e., campaigns, beacons, notifications, etc.). This is important because usually that decision is made by the server and not by the database. This puts ArangoDB at a higher level.

Overall Experience

ArangoDB just runs. It’s very stable, very fast and we’ve had no problems at all with maintaining it. It’s a friendly database in which all of the pieces are well assembled. Plus, we were able to learn it quickly.

After only two weeks, we were able to use ArangoDB with NodeJS and present a working prototype to our clients. We are currently working on our next projects using ArangoDB in the field of legal document repositories, i-voting systems, accounting and much more.

Importance of different ArangoDB Characteristics for Beonit

Factornot important important very important
Performance x
Clusterx
Documentation x
Active community x
Price x

Feature set
not importantimportantvery important
Multi-model x
AQL / JOINs x
Foxx Microservices x

If you have any question about this project or the experiences of Alex, please ping jan.stuecke at arangodb.com. He will forward your questions to the author.

Big thanks to Alex and his team for taking their precious time to wrote this excellent case study.