Financial systems don’t get a maintenance window because the business they run doesn’t stop. Card authorizations keep arriving at 3 a.m. Payroll runs hit on fixed schedules that don’t care about your migration timeline. A ledger that’s unavailable for even ninety seconds isn’t a minor inconvenience, it’s a declined transaction, a failed payout, or a support ticket from a customer who now wonders if their money is safe. The absence of a window is the defining constraint of these migrations, and it changes almost everything about how the cutover has to be built.
This piece assumes you’ve already picked the destination: a new ledger, a new database engine, a new payment processor. The harder problem, and the one that actually determines whether the migration is invisible or an incident, is the mechanics of getting from the old system to the new one while both are live. That’s dual-write validation, reconciliation, traffic shifting, and rollback design. Get those four right and the destination almost doesn’t matter. Get them wrong and the best database in the world won’t save you.
Why a maintenance window isn’t an option for financial systems
In most software domains, a maintenance window is an inconvenience you schedule around low traffic. In financial systems, it’s a liability with a dollar figure attached. Every minute of downtime on a payment system is a measurable count of failed transactions, and every one of those failures has a customer attached to it who will remember it.
Regulatory expectations compound this. Many payment processors and banking partners have uptime commitments written into their agreements, and some jurisdictions treat extended unavailability of a core ledger as a reportable incident. A migration that requires taking the ledger offline isn’t just risky, it may not be permitted under the terms you’ve already signed.
The practical consequence is that the migration has to run as a long-lived parallel operation rather than a single event. Old and new systems coexist, often for weeks, while traffic gradually moves from one to the other. That coexistence period is where almost every migration failure actually originates, not in the final cutover moment itself.
Dual-write and shadow traffic: running old and new systems in parallel without corrupting either
Dual-write means every write that hits the old system also gets sent to the new one, so the new system’s data stays current while you’re still deciding whether to trust it. It sounds simple. It’s the part of the migration most likely to quietly corrupt your data if you don’t think through the failure modes.
The first decision is ordering. If a write succeeds on the old system but fails on the new one, what happens? You need a defined answer, not an assumption, because the default behavior of most application code is to swallow the second write’s failure and move on, leaving the new system silently behind. The safest pattern is to treat the old system as the system of record during the dual-write phase and make the new system’s write best-effort but logged. Every failed write to the new system goes into a queue for retry and, if retries exhaust, into a list that reconciliation has to catch.
The second decision is idempotency. Dual-write means the same logical transaction is being processed twice, by two different systems, possibly with different retry timing. If either system isn’t built to safely process the same transaction ID twice, you’ll end up with double-posted entries. Every write path in the dual-write phase needs an idempotency key that both systems respect, and that key has to survive retries, timeouts, and manual replays.
Shadow traffic is a lighter-weight companion to dual-write, used for read paths rather than write paths. Instead of writing to both systems, you send a copy of production read traffic to the new system and compare its responses against the old system’s responses without ever serving the new system’s answer to a real user. This is how you validate that the new system computes balances, applies interest, or calculates fees correctly, under real traffic patterns, before a single customer is exposed to it. Shadow traffic catches a category of bug that unit tests never will: the one that only shows up under the specific, messy combination of inputs your production data actually produces.
Run shadow traffic for long enough to see your real volume patterns, not just a sample. A new system that handles Tuesday afternoon volume fine but chokes on month-end payroll volume will only reveal that problem if you let the shadow run through a month-end cycle.
Reconciliation jobs: catching ledger drift before it becomes a customer-facing liability
Dual-write will drift. Not because the design is wrong, but because networks fail, retries race, and edge cases in currency rounding or timestamp handling behave differently in two codebases that were never meant to be identical. The question isn’t whether drift happens, it’s whether you catch it before a customer does.
A reconciliation job compares the old and new systems at the record level, not just at the aggregate level. Comparing total account balances across both systems will tell you the two systems roughly agree, but it won’t tell you that one customer’s balance is off by exactly the amount of another customer’s error, because the two mistakes happened to cancel out in the aggregate. Reconciliation needs to walk every account, every transaction, and flag any mismatch, however small.
Build the reconciliation job to run continuously during the dual-write phase, not as a one-time check before cutover. Run it on a schedule tight enough that drift gets caught in hours, not days. For a ledger processing meaningful transaction volume, that usually means hourly or more frequent, because the cost of letting a discrepancy compound overnight is higher than the cost of running the comparison more often.
Classify mismatches by type and severity. A one-cent difference caused by a rounding rule that genuinely differs between systems is a design decision you need to resolve, not an emergency. A missing transaction, where the new system simply never received a write that the old system processed, is a different category entirely and should page someone. Build the alerting around that distinction from the start, because a reconciliation job that pages on every one-cent variance will get ignored within a week, and then it won’t catch the missing transaction either.
Keep a permanent record of every discrepancy found and how it was resolved. Auditors and regulators will eventually ask how you validated the migration, and “the reconciliation job ran for six weeks and found these forty-one discrepancies, all understood and resolved” is a defensible answer. “We spot-checked it and it looked fine” is not.
Traffic shifting with feature flags: moving load gradually instead of flipping a switch
Once the new system has been validated under shadow traffic and dual-write reconciliation has run clean for a meaningful stretch, the next question is how to move real traffic over. The instinct to flip a single switch and route everything to the new system at once is understandable and wrong. A gradual shift, controlled by feature flags, turns a binary cutover into a continuum you can stop at any point.
Shift by a dimension that lets you isolate problems. Internal test accounts first, then a small percentage of real accounts chosen at random, then a larger percentage, then everything. Some teams shift by account cohort instead of percentage, moving a specific customer segment, like accounts opened in the last month, before moving the larger and older population. Either approach works as long as you can answer, at any moment, exactly which accounts are on which system.
At each stage, hold the traffic level steady long enough to observe it properly rather than ramping on a fixed clock. Five percent of traffic for fifteen minutes doesn’t tell you much. Five percent of traffic for a full business day, including peak hours, tells you whether the new system holds up under real conditions.
Feature flags give you the control plane for this, but they need to be built to flip per-account, not just globally. A global flag that’s either fully on or fully off defeats the purpose of a gradual shift. The flag check needs to sit as close to the entry point of the transaction as possible, so that a single decision routes the entire transaction’s lifecycle to one system rather than splitting it across both partway through.
Watch the same metrics on both systems during the shift: latency, error rate, and transaction success rate, side by side. A new system that’s slightly slower at five percent of traffic will often get dramatically slower at fifty percent, and you want to see that curve before you’re committed to the higher percentage, not after.
Rollback design: building an undo path before you need one
Rollback isn’t something you improvise once something has gone wrong. By the time you need it, you need it to already work, and the only way to know it works is to have built it, and tested it, before the migration starts shifting any real traffic.
The central design question is: when traffic moves back to the old system, does the old system have every transaction that happened while the new system was serving traffic? If the old system stopped receiving writes once a given account moved to the new system, rollback for that account means replaying everything the new system processed back into the old one, in order, without duplication. This is why dual-write in the other direction, from new system back to old, is worth considering even during the forward migration: it keeps the old system current enough that rollback doesn’t require a separate replay mechanism built under pressure.
Define your rollback trigger criteria before you start shifting traffic, not during an incident. Decide in advance what error rate, what latency threshold, or what specific failure class automatically triggers a rollback for the affected cohort, and write it down somewhere the on-call engineer can find it at 2 a.m. without needing to make a judgment call about whether the problem is “bad enough.” Ambiguity about whether to roll back is how teams end up riding out a problem for an extra hour while the decision gets escalated.
Test the rollback path itself, not just the forward migration. Roll a test cohort forward to the new system, then roll it back, and verify the account’s transaction history is complete and correct on the old system afterward. If you’ve never actually executed a rollback before the day you need one for real, you don’t have a rollback plan, you have a rollback plan on paper.
Testing the cutover itself, not just the destination system
It’s tempting to spend all your testing effort validating that the new system is correct, fast, and reliable, and treat the cutover mechanics as plumbing that doesn’t need the same scrutiny. That’s backwards. The new system has presumably been tested extensively as a standalone piece of infrastructure. The cutover process, dual-write, reconciliation, flag-based routing, rollback, is often newly built specifically for this migration and has had far less real-world exposure.
Test the cutover process under the failure conditions it’s actually likely to encounter: a network partition between the dual-write targets, a reconciliation job that itself fails partway through a run, a feature flag service that becomes unavailable mid-shift. Each of these is a plausible production event, and each one needs a defined, tested behavior rather than an assumption that it won’t happen during your migration window.
This is closely related to the broader practice of deliberately injecting failure into production-adjacent systems to see how they actually behave, rather than how you assume they behave. If that discipline isn’t already part of how your team validates infrastructure changes, our guide on chaos engineering for financial systems covers how to build that testing practice before you need it for a migration this consequential.
Choosing a cutover window and monitoring plan for the first 72 hours after go-live
“Zero downtime” doesn’t mean zero attention. Even a well-executed gradual shift has a point where the last percentage of traffic moves over and the old system stops receiving new writes. Choose that moment for a period of naturally lower transaction volume and lower operational risk, such as a weekday outside of month-end or payroll cycles, so that if something does surface, you’re dealing with it against a smaller blast radius.
Staff the 72 hours after full cutover deliberately. This isn’t the time for the engineers who built the migration to be off-call. Keep the people who understand both systems, old and new, reachable and awake during the hours when your highest transaction volume historically occurs.
Keep the old system running and able to receive data, even after traffic has fully moved, for the entire 72-hour window at minimum. Decommissioning the old system immediately after cutover removes your rollback path exactly when you’ve had the least time to prove the new system holds up under full load. Reconciliation should keep running during this period too, comparing the new system’s output against what the old system would have produced had it still been processing those transactions, wherever that comparison remains possible.
Define, before go-live, exactly what “stable” looks like: what error rate, what latency, and over what duration, before you consider the migration complete and the old system safe to retire. Writing that definition down in advance keeps the decision from becoming a judgment call made under the pressure of wanting the project to be finished.
A migration that holds up isn’t the one where nothing went wrong. It’s the one where everything that could go wrong had already been tested, flagged, and given a defined response before the first real transaction ever touched the new system.