In the modern business world, data is the decisive factor for success. Companies of all sizes are faced with the challenge of efficiently storing and analysing huge amounts of data. In this context, NoSQL databases have become increasingly popular in recent years. But what exactly does the term ‘NoSQL’ mean and how does it differ from traditional database systems? In this article, we want to give an overview of NoSQL databases, highlight their advantages and possible applications and answer the question of whether SQL databases have now become superfluous.

What are NoSQL databases?

The term ‘NoSQL’ first appeared at the end of the 1990s with Carlo Strozzi, whose relational database management system cannot be queried using the SQL database language. Today's understanding of NoSQL (hereafter short for ‘Not Only SQL’ or ‘Non-SQL’) arose from the need to overcome the limitations of conventional relational databases.

Relational databases, as developed in the 1970s by companies such as IBM and Oracle, offer a proven method for managing structured data using tables and predefined schemas. However, with the advent of the internet and digital transformation, data volumes and requirements increased rapidly. In the 2000s, when large technology companies such as Google, Amazon and Facebook were confronted with gigantic amounts of data, they began to develop their own solutions for data management. These efforts led to the birth of NoSQL databases, which to this day offer flexible, scalable and high-performance alternatives to traditional relational models. The term ‘NoSQL’ is therefore somewhat misleading nowadays, as the concept refers more to the differentiation from relational organisation than to the SQL query language itself - NoSQL databases can also be partially queried using SQL.

Comparison: NoSQL vs. relational databases

Structure und schema:
Relational databases use a fixed schema that must be defined before the data is stored. This means that the structure of the data (tables, columns, data types) is fixed and subsequent changes are complex and expensive, and in most cases involve downtime for the database and therefore the system. NoSQL databases, on the other hand, are schema-free and can therefore store data in flexible, non-predefined structures - these are predominantly JSON-like objects. This enables faster adaptation to changing requirements. New data records do not necessarily have to correspond to the structure of the previously inserted data records, as long as the accessing system can understand and process all stored formats. Fast iterations in product development can thus be realised much more easily.

Scalability:
Relational databases scale vertically. Performance is increased by adding more resources to a single server. NoSQL databases, on the other hand, are designed for horizontal scaling, which means that they can be expanded by adding more servers. This makes them ideal for processing large amounts of data, as adding more machines to a network is much easier than upgrading one machine over and over again. This means that the resources required can be optimally distributed across the various servers, thereby reducing costs. In addition, distribution across several servers also creates greater reliability compared to relational models.

Performance and consistency:
Relational databases offer ACID transactions, which ensure a high level of data security and consistency. NoSQL databases often rely on the BASE model, which enables higher availability and scalability, but dispenses with immediate consistency. The system is available at all times and can answer queries without necessarily working with the current data status. However, a functioning system will achieve a consistent data status after a sufficient amount of time has elapsed. However, these limitations make NoSQL databases unusable for some use cases, such as payment transactions, where the assurances of ACID transactions are indispensable.

Types of NoSQL databases

There are different types of NoSQL databases, each of which is optimised for different use cases:

1. Document-based databases (document store): These databases store data in documents that are available in formats such as JSON or BSON. Examples include MongoDB and CouchDB. They are ideal for applications that require flexible and complex data structures.
2. Key-value databases (key-value store): These store data as key-value pairs. With their ease of use and speed, they are particularly suitable for applications with high throughput and low latency. Common representatives are Redis and Riak.
3. Column-orientated databases (wide-column store): Here the data is stored in column families instead of rows, which makes them particularly attractive for analytical applications and large amounts of data. Well-known databases in this category include Cassandra and HBase.
4. Graph databases (graph database): These store data in nodes and edges and are ideal for applications that need to manage complex relationship data, such as social networks or recommendation engines. Examples are Neo4j and ArangoDB.

Use cases for NoSQL databases

NoSQL databases are particularly suitable for:

• Big data and real-time analytics: Thanks to their scalability and high performance, NoSQL databases can process and analyse large amounts of data in real time.
• Content management and personalisation: Their flexibility makes them ideal for managing unstructured or semi-structured data such as blog posts, product catalogues and personalised user experiences.
• Internet of Things (IoT): The ability to quickly store and process large amounts of sensor and device data makes NoSQL databases the first choice for IoT applications.
• E-commerce: High availability and scalability are crucial for high-performance online shops, especially during sales promotions or peak periods.
• Machine learning and AI: Due to the schema-freedom of NoSQL databases and their ability to process large amounts of data at high speed and scale easily if necessary, they are ideal for containing data for machine learning algorithms or AI models. The vector databases used for large Large Language Models (LLMs), which help the models to recognise relationships or similarities, can be easily integrated here. In this way, the vector embeddings can be stored together with the original data, which saves resources, simplifies data processes and increases their performance.

NoSQL databases also repeatedly play an important role in our projects. AWS DynamoDB, MongoDB or the Azure resources CosmosDB and Storage Accounts as well as Redis Caches have already been used to persist customer master data or trip records in the automotive sector. We have also successfully implemented projects in the construction and energy sectors with CouchDB and Influx. We do not recommend NoSQL databases for use cases where data consistency is a top priority, such as financial transactions.

NoSQL = No more SQL?

Despite the advantages of NoSQL databases, relational databases are by no means obsolete today. Both technologies have their strengths and weaknesses and can be used in a complementary way depending on the application. Relational databases continue to offer unbeatable advantages in terms of transaction security and consistency, while NoSQL databases offer great flexibility and scalability. The so-called schema-freedom of NoSQL databases is sometimes paid for by the fact that in the applications that use the databases, even more attention must be paid to the validity of the data during development. This sometimes higher development effort must of course be included in the project planning and well calculated.

In many companies, a hybrid approach is pursued in which both relational and NoSQL databases are used to optimally fulfil the specific requirements of different applications. The choice of the right database is therefore inextricably linked to the individual requirements and objectives of the respective project.

Conclusion

NoSQL databases have undoubtedly found their place in modern data processing. They offer innovative solutions to the challenges posed by the explosive growth of data and the growing requirements for its processing. Nevertheless, SQL and relational models remain an indispensable part of the database landscape. Companies benefit most when they use both technologies skilfully to achieve their business goals.