Oct 31, 2023 by Sébastien Mitea | 296 views
You have heard of RDBMS, for Relational Database Management System, or may even intuitively associate the word "database" with a tabular representation of data that you can query using a language such as SQL. Depending on your technical affinity, or the scale of your problem, you would then turn to your tool of choice to store and access your data in a CSV text file, a spreadsheet, ... or table(s) inside a proper database. Ouch! Who says that one of those is better than another? And why even learn about anything else than a tabular model?
It comes down to how to best model the elements that make up your problem and their interaction. Surely your mental model when grocery shopping is a one-dimensional bullet point list. Whether it fills up multiple columns on a Post-It note is irrelevant; the point remains that you store all grocery items in some data container under a single title, such as Stuff to buy.
When it comes to data persistence, paper notes have a tendency to vanish without warning. The same goes for text files or spreadsheets. This is when proper database systems come in handy; adding back-ups, storage and query optimization, automatic checks of data consistency (all phone numbers with a prefix, names only in lower case, ...), and many more. But now we are back to the initial question of tabular or not, RDBMS and SQL or not?
Imagine that you want to keep track of any of the following:
You might create tables having columns such as: title, author, organizer, APT_name, opinion_category, number_of_likes, short_description, date, personal_comments, ... You will then fix these attributes or column names along with their data type (e.g. integers, strings, arrays) into the so-called schema of (several tables in) your database. Once defined, this data model is rigid and is not meant to be changed at every step of your project. Obviously it is always possible to copy or recreate your tables with different columns; SQL-based databases can thus evolve within reasonable limits. Plenty more reasons can explain why RDBMS have been around for half a century. But what about your mental model?
It is a bit of misnomer to refer to relational databases when, in fact, they do not track the relations between entities. For instance, take 5 seconds for each of the problems below, and imagine how you would explain your approach to your parents, to a colleague or in your draft notebook:
I dare think (hope) that the image that came to your mind for these problems did not include several mini-spreadsheets connected together. It would probably look more like the figure above. In this case the problem can be formulated like this: . A query in SQL would require inefficiently joining multiple times the same tables, and the syntax would be cryptic to anyone but the developer. In a so-called graph database, the same query would look like this:
FOR [Any researcher] STARTING FROM [Targeted researchers] SEPARATED BY [2 steps] # one for each vertex (researcher or organization) in the graph RETURN [Connecting researcher] COMPUTE SHORTEST PATH FROM [You] TO [Connecting researcher]
The code for this query is given below but, first, now is the time to introduce ArangoDB; a multi-model database. In practice, this means that it operates natively and simultaneously like a graph database such as neo4j, a document database à la MongoDB, a key-value store like redis and even a search engine such as elastic.
Of course not, but that presents serious advantages. The databases mentioned in the previous paragraph are generically referred to as NoSQL. Irrespective of your preferred urban legend about this acronym ( 🤓 is it even an acronym? 😈 ), in short, they allow but do not impose the use of a schema. Despite that freedom, they have competitive performances. Thanks to that freedom, they can do things that RDBMS can´t.
For the problem above about the network of researchers and organizations, the code in AQL (working out this acronym is your homework for today) would be:
FOR researcher, connection, path IN 2 ANY "list_of_researchers/targeted_researcher_x" list_of_connections FILTER IS_SAME_COLLECTION( "list_of_researchers", path.vertices ) RETURN path.vertices.name
Do that with each of the Targeted researchers then find the Connecting researcher they all have in common. Next, for the shortest path between You and the Connecting researcher, the AQL code is:
FOR vertex, edge IN ANY SHORTEST_PATH "list_of_researchers/you" TO "list_of_researchers/connecting_researcher" list_of_connections FILTER IS_SAME_COLLECTION( "list_of_researchers", vertex ) RETURN vertex.name
The equivalent in AQL of the RDBMS terminology of Tables and Records is Collections and Documents. Each data record in ArangoDB is actually a JSON document, a highly efficient and very human readable file format (the power of which is discussed in a subsequent blog post). Having all data stored in JSON in the background ensures interoperability with a wide range of existing software and programming languages.
It is easy to import or export data, for instance as JSON or CSV files, either manually through the web interface above, or programmatically via scripts in
python, ... (a subsequent blog post will explain how to use the python driver).
ArangoDB is intuitive, flexible and powerful to use just as a schema-free data store. But in order to benefit from the power of graph algorithms, an additional collection type exists: the Edges. They can be used as a simple list of pairs of "_from: document_x" and "_to: document_y". In this case, they inform AQL of where to look for paths during a graph traversal like in the problem above. But edges can have any number of attributes assigned to them, e.g.: the duration of an employment relationship, the distance in kilometres between two geographical points, the weight of a connection, ... significantly increasing the possibilities of network analysis.
In the hope that this short introduction has piqued your interest in graph databases in general, and in ArangoDB in particular, more blog posts related to this theme will be published here.