When you need a data bus: signs you’ve reached the point of no return in IT

Problem in simple terms: integrations start getting in the way

Almost everyone starts the same way: you need to quickly connect two systems. For example, CRM must send orders to the accounting system, and the warehouse must confirm shipments. You build a point-to-point integration via a file, a simple script, or a direct API. It works and feels like a win.

Problems come later, when the number of connections grows. Each new change attaches to old rules, and a change in one system unexpectedly breaks another. Integrations stop helping and start dictating how the business must operate.

The worst part is the cost of a new integration increases over time. At first it's 1–2 days to set up. Then it becomes a week of approvals, tests, and conversations like “why it works for Petya but not for us.” At the same time, questions surface that no one fixed beforehand: who owns the data, where to look for errors, how to roll back if something goes wrong.

Business usually notices this through familiar symptoms: data arrives late, errors appear because of different reference data (items, counterparties, prices), people make manual reconciliations in Excel "just in case", and meetings turn into arguments about which system is authoritative and who’s to blame when numbers don’t match.

A one-off failure almost always has a concrete reason: a server crashed, a token expired, someone made a mistake. A systemic problem repeats in the same pattern: each change brings back manual work, queues of tasks grow, and fixes happen at night because you can’t stop processes during the day.

If you increasingly catch yourself thinking “we need a data bus to stop this patching,” then point integrations have already become a risk source. It’s time to discuss not another script, but a minimal integration framework that will handle growth.

Quick diagnosis: what to measure before talking about a bus

Before discussing a data bus (ESB), it’s useful to do a short inventory. The goal is simple: understand whether you still have "a couple of integrations" or a network that’s hard to maintain without a common approach.

Start with a map of systems and exchanges. List the main applications (1С/ERP, CRM, website, warehouse, accounting, BI, service desk, banks, government services) and mark who exchanges data with whom. Then count the "system-to-system" links. If exchanges grow faster than the number of systems (each new system needs multiple connections right away), you’re approaching the point where point solutions become problematic.

The second indicator is change velocity. Measure on a real example how many days it takes to add a new exchange or change a single field in an existing one (for example, add a delivery flag to an order). Count the full cycle: approvals, edits in two systems, tests, deployment, incident analysis. If even a small change turns into a mini-project, that’s a signal.

The third indicator is "who knows the wiring." How many people actually understand who’s connected where? Who can quickly answer: “what will break if we change the counterparty directory”? If it’s one person or “everything rests on a developer who did it once,” the risk is managerial, not just technical.

Finally, identify critical data and the cost of errors. Usually these are orders, payments, inventory balances and personal data. Note which data must be near real-time, what happens on loss or duplicates (money, fines, downtime, reputation), and where personal data passes and who is responsible for it.

These four steps give a clear picture of where it hurts most and where to start moving to a tidier integration model.

Signs of the “point of no return” — when it’s time to think about a bus

The ESB conversation usually starts not because “it’s the right thing to do,” but because integrations stop being background tasks and start consuming the team’s time. If you talk about data exchanges more than about business tasks, that’s a sign.

The main sign is an increase in the number of connections between systems. While there are 2–3 connections you can live with direct integrations. But when links multiply, any change requires edits in several places: extract, processing, directories, reports.

There are also everyday symptoms that are easy to recognize:

• Any integration change becomes a chain of tasks across multiple systems and developers.
• Data constantly needs format conversions, and rules live in code and chats.
• Everything relies on scripts, manual fixes and the knowledge of one or two people.
• On a failure, it’s unclear where to look: logs are scattered, monitoring is point-based or missing.
• After system updates, a queue of “fix exchange” tasks regularly appears and becomes the norm.

A simple check: if you cannot answer within 10 minutes which systems a new integration will affect and where to look for errors, connection management is already out of control.

Example: a company updated the ERP, status codes and date format changed. Direct links caused failures in warehouse, delivery and the executive dashboard. Fixes were needed in three places, and some data had to be corrected manually. A bus helps here not by magic but by providing a single routing point, common transformation rules and a clear place to observe exchanges.

When a bus is not needed: common false reasons

A data bus does not fix what is unrelated to integration. If processes aren’t described, roles aren’t assigned, and directories diverge, a bus will only spread errors faster and make them more visible.

The first false reason is trying to “fix” bad data with the bus. If each department has its own customer record or item codes, the problem isn’t transport but master data: who owns the directory, how records change, how quality is checked.

The second false reason is having few, infrequent exchanges. When integrations are 2–3 and stable (for example, accounting gets sales once a day), agreeing on one API style, an error format and a simple runbook—who’s responsible, how we monitor, what to do on failure—is often enough.

A common mistake is buying a platform "for the future" without scenarios. Symptoms are usually the same: no list of integrations and owners, no defined events or required data, no versioning and testing rules, expecting the bus to unify formats by itself while monitoring and incident response are left for later.

Example: HR requests integration between the HR system and the access control system. If the task is to export a list of active employees once a day, a bus is not required. Start with a data contract and clear responsibility. But if exchanges grow, change frequently and begin to conflict, then a bus becomes practical.

Integrators, including GSE.kz, usually start not by choosing a platform but by describing scenarios and minimal rules. That quickly shows whether a bus is needed at all.

Minimal bus architecture that won’t overload a small IT team

A bus is not a "huge platform" but a set of supporting elements so new integrations are predictable and don’t become one-off hacks.

The minimum set usually includes:

• A central transport for asynchronous exchanges (a message queue or event stream) so systems don’t call each other directly.
• An API layer for synchronous "right now" requests (for example, checking an order status during a customer call).
• Simple message schemas and versioning so format changes don’t break consumers.
• A single log and basic monitoring that shows whether a message arrived, where it failed, how many attempts and what error occurred.
• A minimal registry of integrations: who owns it, what is transmitted, frequency, and expected delivery time.

To avoid turning this into bureaucracy, a few rules are enough:

1. Each message has a data owner and a channel owner.
2. Any format change goes through a new version; the old one lives for a transition period.
3. Errors are not lost: they appear in the log and get a clear status.

Example: accounting needs supply details from the warehouse system. Send an asynchronous event "Delivery confirmed", and provide a synchronous API "Show delivery details" on demand. That way, warehouse failures don’t block accounting, and you can quickly see what didn’t arrive and who is responsible for fixing it.

How to roll out step by step: from pilot to first rules

Buses fail less because of technology and more because people try to connect everyone at once. It’s safer to proceed in short iterations so a small IT team can learn the consequences without turning the project into an endless migration.

Start with a pilot that truly hurts. Choose 1–2 integrations where failures cost money or time: for example, CRM orders not reaching accounting, or shipment statuses diverging.

A simple plan helps:

• Define the pilot scenario and success criteria: which statuses must match, in what time, and who verifies the result.
• Agree on an events and fields "dictionary" in plain language: what "order created" means, when it’s considered created, which fields are mandatory.
• Bring up transport and logging: record every message, set alerts for 3–4 typical errors (not delivered, parse error, duplicate, timeout).
• Connect systems via adapters: each system talks to the bus, not to each other.
• Introduce a change process: message versions, a test environment, a rollout window and a short pre-release checklist.

A small example: if the warehouse starts returning a new status "Partially picked", without versioning this can break ERP processing. With event versions and a test environment you can add the status as an extension and update subscribers calmly.

After the pilot, don’t rush expansion. First lock in the initial rules, appoint an owner for the dictionary and analyze every incident from the log. This yields faster improvements than an "ideal" architecture on paper.

Data and responsibility: without this, a bus won’t save you

A bus solves the wiring problem between systems but does not answer the main question: who is responsible for what. If this isn’t set, even a tidy project quickly becomes a debate about whose data is correct and why integrations break after updates.

Start simple: each key dataset must have an owner. For example, HR owns employees, accounting owns counterparties and invoices, training owns students and groups. The owner doesn’t necessarily store the data but decides what is the truth, who can change the format, and how to notify other teams about edits.

You don’t need complex committees for duplicates and directory conflicts. Minimal rules suffice: one source of truth per directory, a single key (at least an internal ID) and a clear path to escalate disputed records for resolution.

Minimum to log

To find errors by facts rather than chats, agree on basic logging:

• message identifier (correlation)
• send and processing timestamps
• source and recipient
• error code and reason text (if any)

Then it’s easier to talk about access and personal data. Don’t overcomplicate: separate who can see content and who can see technical metadata. Often integrators only need metadata (IDs, statuses), and personal fields can be masked or sent only where truly required.

Metrics that show the real picture

Look not at "how many integrations were done" but how they operate:

• delivery latency (average and 95th percentile)
• error rate by type
• share of redeliveries and retry counts
• number of messages in queues (if growing — there’s a bottleneck)

This creates transparency: who changes formats, who fixes failures and why they happened.

Common mistakes and pitfalls when implementing a bus

The most common mistake is starting with the slogan "from now on all integrations go through the bus" and trying to migrate everything quickly. The team drowns in migrations and the business sees only new delays. It’s better if the bus first solves one concrete pain.

The second trap is putting business rules into the bus. Once routing contains discounts, limits, approval statuses and exceptions, the bus becomes a monolith people are afraid to change. The bus should carry events and commands; business logic must live in the owner systems.

Another error is creating a separate data format for each system ("for accounting this way", "for CRM another"). That quickly increases transformations and failures. It’s simpler to agree on a few common events (e.g., "Counterparty updated", "Order created") and treat them as a product.

Observability is a separate topic. Without it the bus becomes a black box: "nothing arrived" tickets take hours to investigate. Minimum needed from day one:

• a single request/message identifier for end-to-end tracing
• clear delivery statuses and rejection reasons
• a retry and manual replay log
• basic queue and processing time metrics

Finally, many ignore load growth. Today 50 messages per minute, in six months 500 — suddenly duplicates, stuck queues and a flood of retries appear. Plan retries (with backoff), deduplication, processing limits and rules for what to do when a recipient is unavailable.

Checklist: is your company ready for a bus?

A bus starts to bring value when you have not just one "cable" between systems but a network of dependencies where it’s unclear what will break after each change.

Check five things. If two or three are missing, fix those gaps first:

• An up-to-date list of integrations: which systems are connected, why, via what channels, and who is responsible for each link.
• Agreement on key events and their structure. Usually 5–10 events suffice (e.g., "order created", "invoice status changed") with clear fields and versions.
• Logs and monitoring set up so incidents are investigated step by step: who sees the alert, where to look, how to record conclusions and how to prevent recurrence.
• A test environment and a simple rollout process: where we test integrations, who approves, how to roll back.
• Understanding which links will remain point-to-point and why (for example, a one-off monthly file exchange or integration with a soon-to-be-retired system).

Example: accounting changes the counterparty directory, CRM pulls data, and the warehouse stops accepting orders because of a new field. Without event versions, logs and an integration owner, root cause analysis turns into "it’s not our fault" messaging. A bus won’t fix that by itself but will enforce rules: what is an event, where it’s visible and who reacts.

If the checklist is mostly covered, start small: one or two critical integrations and a couple of events. The benefit of order and transparency usually appears quickly.

Practical example: how a bus reduces chaos without a "big reform"

A mid-sized company: sales work in CRM, warehouse in a separate accounting system, accounting in 1С. Weekly meetings repeat the same issues: balances don’t match, shipments disappear, payments appear late. A customer was promised a product that the warehouse no longer has.

Exchanges were previously based on files and manual reconciliations. A manager exported Excel, the warehouse imported it "as is", and accounting adjusted statuses by phone. When something broke, no one could quickly say where the error was and who introduced it.

The solution was implemented without replacing systems: a data bus (ESB) was added as a thin event layer. Not "integrate everything with everything," but agree on a few key events and a common format.

Three or four signals were enough: "order created or updated", "shipment confirmed", "payment received", "return processed." Systems kept working independently, but each event passed through the bus and was logged: time, source, recipients, result.

Within a month the effect was visible without complex analytics: fewer manual reconciliations, faster resolution of disputes, and lost documents found via the event chain rather than chat threads.

Limitations remained, and that’s normal. A bus doesn’t fix bad directories or replace change discipline. If departments keep editing data "on request" without rules, chaos will return — only now with a log.

Next steps: how to start without unnecessary risk

Don’t start with a "big rollout." Choose a small but illustrative contour where impact can be measured and mistakes won’t halt the business.

First, agree on 2–3 goals that are measurable. For example: reduce integration change time from 10 days to 2, halve the number of overnight exchange failures, remove manual exports between two key systems.

Then follow a short plan:

• Gather a list of systems and current exchanges (who talks to whom, what data, how often, who owns it).
• Pick a 6–10 week pilot: 1–2 integrations that cause pain (frequent errors, lots of manual work, critical deadlines).
• Decide where infrastructure will run: on-premises or in a data center.
• Predefine support for critical exchanges: who’s on-call 24/7, how escalation works, which alerts are "red".
• Fix a minimum set of rules: unified log format, queue and error monitoring, versioned data contracts.

A safe-start criterion is simple: the pilot should make the team’s life easier, not add another layer of manual checks.

If you lack experience designing integration architecture or need a reliable base for a bus and monitoring, consider involving an integrator. For example, GSE.kz as a system integrator can help design the exchange contour and select infrastructure, including servers and support processes aligned with your internal policies.