Camunda and the renaissance of the BPM market
When the Business Process Management (BPM) concept emerged, it promised to be the backbone of business automation that many businesses needed – to support them in allocating resources, measuring risks, and revealing weak links and bottlenecks in processes for overall business process improvement.
After an optimistic start in the 2000s, the BPM market cooled down because the concept was not fulfilling some initial promises and was too heavy and expensive. Camunda comes with the promise to fix this.
In this article, we will explore BPM’s initial promises and market challenges, and the benefits of using Camunda as an alternative to traditional BPM solutions.
What is Business Process Management?
Business process management (BPM) is a structured methodology that aims to improve the processes organizations use to achieve their goals – complete tasks, serve their customers and generate business value. BPM uses diverse methods and tools to improve a business’s processes – for example, by analyzing it, modeling how it would work in different scenarios, setting changes, implementing, monitoring new processes, and continuously improving it to achieve the target outcomes. For some businesses, especially ones where automated processing steps are mixed with manual activities, the BPM is an “almost standard” approach.
Throughout the 2000s, as certain companies found their business processes to be too complex to manage without the support of automated tools, BPM concepts and tools emerged:
BPEL
In 2002, the first BPEL (Business Process Execution Language) was created, and shortly afterward the “big providers” created the first implementations. In hindsight, the first BPEL integration was very “techy”, heavy, expensive, and outdated.
BPMN
In 2005, the standard for BPMN (Business Process Modelling Notation) was defined, which added a graphical representation of processes and simplified it. In Emergn’s portfolio, there are several projects using it. BPMN depicts the business process as a standardized, executable diagram. BPMN diagrams contain predefined logic of business transactions, which consist of:
- Automated process steps.
- Manual (potentially long-running) activities.
- Call of subprocesses.
- Sending messages to start asynchronous processes.
- Decision tables.
- And many other elements.
BPM engines store process variables (process context) in the database, which allows users to view the context of active and historical process instances – thus providing audit trails and insights into incidents out of the box.
DMN
DMN (Decision Model and Notation), often called decision tables. In this example, you can see a liability calculation in DMN, where it defines:
- If the product is “Motor” and the incident type is “Theft”, and the participant role is “Policy Holder”.
- Liability status is “Fault,” and no claims bonus is “Disallowed”.
Why is DMN notation so important for BPM?
- Conditions can change often.
- They can be written in simple table format.
- They allow you to change outside of the “real” program code. These cannot be changed physically by businesspeople in a production environment. However, conceptually they can be owned by businesses. Businesspeople can read, understand and define those changes. To change such conditions, we do not need real programming skills. So, maintenance and changes are potentially faster, simpler, and come with fewer errors.
The image below shows how the DMN from the first image is integrated into the process flow. This is one way we can use DMN. We can also leverage DMN by calling from the program code to get results. The important thing is that we define the DMN once and then integrate it everywhere it is needed.
How did BPM match its initial promises?
”Businesspeople will be able to change process independently from developers”
To be honest, I have never seen such a degree of independence in real life. Still, using BPMN and DMN standards gives businesspeople more control over their business process definitions and improves maintainability. In reality, the BPMN/DMN artifacts still should be verified by developers and testers before being implemented, because “techy” stuff is not visible when you define the artifact but appears when you integrate and run it.
“It will be faster to market”
It is always a question of balance: Applying BPMN and DMN standards can improve time to market on its own if done according to best practices. On the other hand, many “traditional” platforms implementing BPMN are expensive and heavy. They dictate the rules on how software should be structured and deployed, making the entire process slower and more costly. In many larger organizations, it is merely a strategic choice. Still, compared to “heavy predecessors”, Camunda is more efficient because of its openness to integration.
Although BPM, in theory, kept its initial promises, when it was introduced to larger companies in the market, it developed provider-specific standard extensions that transform it into a very complex, big, and expensive solution that dictates the overall software architecture. As a result, BPM software projects became slow and expensive, and the reputation of BPM lost traction.
Why is BPM still at the heart of many important businesses?
There are still some businesses where BPM can be the best option, such as:
- Banking Business (issuing loans)
- Insurance Business (processing insurance claims)
Those processes combine automated and manual processing activities. An example of manual activities would be we wait for a response from a customer or third parties. Typically, there are many questions we need to answer, like:
- How many active process instances do we have?
- In what exact state is every instance?
- When should the next actions take place, and who is responsible?
- What has happened exactly in every process instance (for audit, reports, metrics, BI)?
BPM provides answers to those questions out of the box, “per design”. Usually, BPM tools have data stores to automatically store this information and frontend which allows users to work with it.
How is Camunda different?
Camunda BPM is a lightweight, Java-based framework. It can be used as a standalone process engine server or embedded inside custom Java applications. It offers non-Java developers a REST API and dedicated client libraries to build applications connected to a remote workflow engine.
Camunda is different from other established BPM solutions from big providers – like SAP, Oracle, Microsoft, and Pega – because:
- It is lightweight – you can integrate it into your pet project, and usually get something to work within one day
- It can be open source (there is Enterprise Edition too) and available in different distributions, including Docker container
- It is standards-based, where most of the solutions from big providers don’t stick to standards
- It has end-to-end orchestration
- It doesn’t dictate how to integrate it (open architecture)
How would Camunda impact your organization?
The following illustration shows the most important Camunda tools and common users in an organization. Any organization can personalize this structure based on its unique needs.
Nice extra tools of Camunda’s commercial version
Cawemo
Cawemo helps to separate concerns between business analysts and developers.
- Business analysts collaborate to create conceptual diagrams in the cloud platform Cawemo, focusing on the business context. The BPMN/DMN standards are still used there but some technical details can be ignored. This is beneficial because some of those details may not be relevant and may be too challenging for businesspeople to understand.
- Developers can take it from there, adding their technical expertise that may not be visible on a conceptual level (for example, data mappings, transaction control functions, etc.). The result is integrated and tested; then stored on an open-source repository, like git.
- The final BPMN / DMN artifacts can be automatically transferred back to the Cawemo platform, to create a more precise basis for the next versions. The automated synchronization is triggered at deployment time to the environment.
Optimize
Camunda Optimize is especially useful for business stakeholders because it offers business intelligence tools. It leverages data collected during the process execution to help you improve business processes. Some of the features include:
- Access reports and insights on your BPMN/DMN instances
- Uncover process bottlenecks using heatmaps
- Analyze process variants to make data-driven improvements
Some learnings from real projects
There is more than one Emergn customer project using Camunda. The ideas from this article are inspired by the work Emergn did for an international insurance company that has put Camunda in heart of the “Insurance as a service” platform. Here are some valuable learnings from these projects:
Don’t always believe what you see
To understand why you should get to know transaction management in Camunda. In case of an error, Camunda rolls back to the previously successful transaction.
This topic is interesting but not so trivial, read more about it here.
Follow Camunda’s best practices
The concepts of BPMN and DMN are implemented in Camunda using underlying techniques/solutions for communication, transaction management, and others that influence stability and performance. You can start very fast with Camunda, but if you’re looking to build a scalable, complex, robust system, you should follow Camunda’s best practices guide documents.
P.S. Keep in mind that the learning curve is steep.
Don’t overdrive orchestration with Camunda
Don’t use Camunda only to orchestrate the data entry in the database. “The usual” web application can do it. We did this at Emergn and the process was overkill. Even if it is a light solution, Camunda adds complexity to the project.
For teams looking to orchestrate long-running manual process activities, Camunda fits perfectly.
Don’t copy blindly the samples from the documentation
This sample will lead to an endless loop that will degrade the performance of the whole system. To fix this, you should decrement the “retries” value with every call.
Before you go
If you haven’t done so yet, I encourage you to get in touch with Camunda. You can use the Camunda tutorial to guide you through the modeling and implementation of your first workflow with its platform.
If you’re thinking about starting a small “pet project”, I encourage you to consider Camunda as a backend to save your application data as process context. The process support and orchestration of user tasks is a nice additional out-of-the-box option.
Camunda is also a great alternative for big, robust, distributed, scalable, enterprise BPM systems—it is light and open, compared to the traditional BPM solutions that dominated the market in the early 2000s. Still, it is important to consider the steep learning curve before diving in.
Additional Insights
The cultural dimension to AI productivity
How AI tools impact the way we develop software: our GitHub Copilot journey
Are we overreacting when it comes to AI?
