Most of the time, databases are considered separate entities with their own development cycles. As a result, there is a major disconnect between database development and DevOps. You often see a situation where the application gets updated quickly with bug fixes and new features, yet the databases are lagging behind, slowing down the release process.
The Isolation of databases from DevOps created a gap between the application developers and database admins/devs that led to bottlenecks and errors in the development process.
Developers often have to wait a long time for database modifications, resulting in delays in development. In situations where multiple database changes overlap, the lack of collaboration and communication ultimately leads to errors, crashes, and downtime.
To mitigate all these issues and get all your teams to collaborate effectively, you need to include the database as an integral part of your DevOps pipeline. In this article, we will explore using GitLab to create an end-to-end DevOps pipeline extending to the database via DBmaestro.
What is GitLab?
GitLab is an open DevOps platform that provides all the necessary tools to create and manage a DevOps pipeline as a single application. It includes everything from a web-based git repository, CI/CD to reporting and security tools.
GitLab was first started as an open-source project aimed at improving team collaboration in software development. Now it has evolved into a complete DevOps and GitOps platform that streamlines the software development lifecycle. The GitLab project is continuously evolving with over 3000 active contributors.
Advantages of GitLab
Complete DevOps Platform
GitLab is not simply a code repository but a complete DevOps platform. It offers everything as a single application eliminating the need to rely on different tools for different functionalities in the DevOps pipeline.
Besides, GitLab supports integrations with third-party services and tools such as JIRA, Jenkins, Elasticsearch, etc., allowing you to extend the functionality easily.
Free and Open-source
GitLab offers a completely free tier in their cloud solution as well as a complete downloadable GitLab server that you can install anywhere. GitLab Inc is committed to providing these free versions and keeping the GitLab core open-source.
Implementation Flexibility
GitLab offers both a SaaS solution as well as self-managed self-hosted products so that users can select either of these options depending on their use case. GitLab’s SaaS solution offers both a free tier and a paid subscription-based model offering some advanced features like compliance management and advanced security testing.
Ease of Use
GitLab is simple to configure and use. Unlike many other tools, GitLab can be installed anywhere from on-premise installations to containers. Furthermore, it provides an intuitive UI to manage the entire DevOps pipeline from a centralized location for improved user experience.
Increased Security
GitLab is built with security and compliance in mind. It offers complete control over your codebase, from managing user access to deciding code storage location. On top of that, GitLab provides security tools that can be easily integrated into the SDLC, such as Static Application Security Testing, Dynamic Application Security Testing, Container Scanning, Dependency Scanning, etc. Furthermore, its comprehensive monitoring suite offers users an additional layer of security with monitoring capabilities.
Why you need to Integrate GitLab into end-to-end Release Pipeline
As a complete DevOps platform, GitLab offers all the necessary tools to carry out all the stages within the DevOps pipeline, from planning, packaging to deployment and monitoring. Think of a simple scenario where you have a cloud-native application with a managed database hosted in a cloud service. How can you manage the lifecycle of that application?
The solution is to create an end-to-end release pipeline integrating GitLab as its core. This way, you have a single application to control your whole pipeline. Commit both the application and database code changes to a GitLab repository, validate and test the code, package, and finally deploy the changes. This provides you with a single data source with all the necessary tools for testing, security, compliance, auditing, deploying, and monitoring. You can further extend the pipeline by integrating it with project management tools like JIRA to keep track of changes and automatically trigger builds.
Extend your CI/CD to the database with DBmaestro
Using GitLab as the core while extending your CI/CD to the database with DBmaestro reduces the chances for bottlenecks and delays in the pipeline.
By bringing the database into the process with DBmaestro, developers and DBAs can have visibility on modifications and improvements done to both the application and database by managing the application and database code as a single entity. This also reduces the potential for bottlenecks in both the application and the database development process.
The database will always be in tune with the application to provide an optimized experience, and developers do not have to wait for DBAs to modify the database. When a fix or an improvement is made, you can create a pull request which will trigger the pipeline to execute and automatically carry out reviews, quality, and functionality testing and provide feedback. Likewise, everything is included in the pipeline reducing the overall feedback cycle and improving the efficiency and development speed.
Besides, this automated release pipeline greatly improves team collaboration since everyone has a better understanding of the workload of each team member and can effectively divide the work without conflicting with each other. You can reduce errors in the source code further using inbuilt code review and quality analysis tools. Using both automated testing and manual reviews improves the quality of the code across the organization, allowing them to comply with compliance requirements easily.
How to Build an end-to-end Release Pipeline with Gitlab and DBmaestro
Now let’s see how to configure an automated end-to-end release pipeline using GitLab and DBmaestro. This release pipeline will consist of three main parts: Source Control, Continuous Integration, and Continuous Delivery.
Step 1 – The first step is to declare the database pipeline within DBmaestro. This enables you to track and understand the database release process.
Step 2 – In this step, any changes to your database code can be integrated into a GitLab repository along with the application code. Each change will be version-controlled, making it easier to track and manage overall changes to the application and database.
Step 3 – This step allows you to trigger the GitLab CI/CD pipeline manually or using a web trigger. It will execute each defined stage in the pipeline.
The first stage in the above pipeline is the application build process. Then the pipeline will move on to the database CI process, where it will get the database scripts, create a database release package and carry out the prechecks.
The release candidates are created as immutable release packages enabling you to test the releases as a single unit.
In the precheck stage, you can configure code analysis to scan your code to identify any issues and enforce compliance standards (Ex: naming conventions, security best practices). Any identified issues will be automatically sent to the developer.
In the above example, a policy has been set to block any DROP table statements, which would cause data loss. This is immediately identified in the precheck stage and quickly notified to the developer.
Additionally, it will also include a dry run in a provisional environment to identify any execution errors and inform the developer. This will ensure that no execution errors will occur during the actual deployment.
Step 4 – With all the tests completed and all the identified issues fixed, you can be confident that your database package is ready for production and would not cause any downtime or issues. Then the pipeline will move on to the CD process upon successful completion of the CI process.
With DBmaestro, you can simply define a proper release schedule without worrying about the deployment strategy or configurations. DBmaestro will take care of them and create a repeatable deployment.
Additionally, DBmaestro will track every aspect of the deployment, from server outputs to any configuration discrepancies. Moreover, it will create an easily digestible report at the end of a successful deployment which contains all the modifications done to databases.
DBmaestro will immediately notify users in case of an issue such as a configuration discrepancy between the database package and the database, allowing them to decide how to proceed with the deployment.
The above screenshot depicts how DBmaestro will show a discrepancy. In this scenario, a hotfix has been applied to the database, which is not included in the release. Since it was detected, the user can properly address the issue, negating any risks to the database. This shows that it is difficult to identify issues in deployment without proper detection capabilities. No one will notice an issue until a production bug is identified in the system.
Step 5 – Finally, the application will be moved to the desired environment once all the modifications are successfully deployed to all the databases.
Benefits of Integrating GitLab and DBmaestro
Following are some of the benefits you can gain by utilizing GitLab and DBmaestro for your DevOps pipeline.
- Increased speed and efficiency in the overall DevOps process while improving quality
- Comprehensive Release, Security and Compliance automation via GitLab
- Shorter and faster feedback loop to address any issues during CI/CD
- Enforce organizational policies, coding, and security best practices across the codebase to enhance code quality and security
- Mitigate risks with release checks, dry runs, and deployment risk management tools
- Database drift detection
- Role-based security management
- Complete audit trails to monitor user behavior
Conclusion
The combination of GitLab with DBmaestro allows you to create a comprehensive, secure, and automated end-to-end release pipeline that includes both the application and database code bases. Leveraging the strengths of GitLab to create the release pipeline with DBmaestro helps users manage all the database-related tasks.
