Why SIEM migrations go wrong
A SIEM accumulates undocumented dependencies faster than almost any other system in the estate. Detections reference field names that exist because of a parser written years ago. Dashboards are load-bearing for a compliance report nobody remembers commissioning. A handful of integrations were built by someone who has left, and the only record of what they do is the code.
None of that is visible when you scope the project, which is why migration estimates are usually wrong in the same direction. The work is not standing up the new platform; it is discovering what the old one was actually doing.
The second failure mode is treating parity as the goal. If the objective is to reproduce every existing detection in the new platform, you have signed up to carry forward years of accumulated tuning debt, and you have given up most of the benefit of moving to a system that detects behaviourally rather than by rule.
Before you start: three inventories
The preparation that determines whether a migration runs to plan is unglamorous and mostly consists of finding out what you have. Three inventories are worth completing before the first byte is routed anywhere.
- Data sources, with the owner, the volume, the retention obligation and, critically, whether anyone would notice if the source stopped arriving. A surprising proportion of ingest volume turns out to feed nothing.
- Detections, with the last time each one fired, the last time each one produced a true positive, and the analyst who owns it. Detections that have never fired a true positive are candidates for retirement, not migration.
- Downstream consumers, meaning every report, dashboard, ticket automation and compliance artefact that reads from the SIEM. These are what break silently at cutover, because nobody tests a quarterly report during a migration.
Phase 1: Parallel ingestion
Route a subset of high-value telemetry to the AI SIEM alongside the incumbent, with no production dependency on the new platform. The objective is narrow and technical: prove that ingestion works, that parsing is accurate on your data rather than on a reference schema, and that search performs acceptably at your volume.
Choose the subset deliberately. Identity, endpoint and one cloud estate usually give the broadest signal for the smallest ingest cost, and they exercise the parsers most likely to reveal problems. Resist the temptation to start with the noisiest source on the theory that it is the biggest win; start with the one whose correctness you can verify.
The exit criterion for this phase is not that data has arrived. It is that you have taken a sample of events, traced them through parsing and normalisation, and confirmed that the fields your detections will depend on are populated correctly and consistently.
- Verify field-level parsing against known events rather than against record counts.
- Measure search latency at realistic time ranges, not on the last fifteen minutes.
- Confirm timestamp handling and time zone normalisation explicitly, because this is where cross-source correlation quietly breaks.
- Establish what the platform costs at this ingest volume before expanding it.
Phase 2: Detection parity, and deliberate divergence
This phase reproduces the detections that matter and, just as importantly, decides which ones do not. Work from the detection inventory: for each rule, decide whether to port it, replace it with a behavioural equivalent, or retire it.
Rules that encode a specific known-bad indicator usually port directly. Rules that were written to approximate abnormality, which is most of the long tail, are the ones to replace rather than reproduce. This is the substantive difference an AI SIEM makes: instead of a rule attempting to describe what unusual looks like, the platform baselines behaviour and reasons about deviation. Porting the rule forward means keeping the approximation and paying for the reasoning engine without using it.
Run both platforms in shadow mode against production for a defined period, and compare on findings rather than on alert counts. Alert count is the wrong measure: a successful migration should reduce it substantially while missing nothing that mattered.
- Track every detection that fires in the incumbent but not in the AI SIEM, and resolve each one explicitly as either a gap or an improvement.
- Track the reverse as well. Detections that fire only in the new platform are either the value you are buying or a tuning problem, and you need to know which.
- Keep a written record of every rule you retired and why. This is the artefact your auditor will ask for.
- Set the shadow period long enough to cover a full business cycle, including a month-end and any scheduled maintenance window.
Phase 3: Operational cutover
Cutover moves analyst workflow to the AI SIEM while keeping the incumbent live but read-only. Nothing about this phase is technical in the way the earlier phases were; it succeeds or fails on whether analysts adopt the new surface.
The last-mile integrations decide that. Ticketing, incident response tooling, threat intelligence and the on-call paging path all need to work on day one, because an analyst who has to copy context between systems at three in the morning will go back to the tool that does not make them.
Run the first weeks with the incumbent still available for reference and with an explicit, publicised path for analysts to escalate when the new platform gives them something they do not trust. Suppressing that feedback does not make the problems go away; it just means you find them later and from an incident rather than from a colleague.
- Cut over one shift or one team first where the SOC structure allows it.
- Publish a rollback trigger: the specific condition under which you would move analyst workflow back, decided before you need it.
- Rebuild compliance and management reporting before cutover, not after. This is the most commonly deferred task and the most commonly regretted.
- Keep the incumbent's alerting on for a short overlap so a missed detection produces a signal rather than a silence.
Phase 4: Decommission
Retire the incumbent once retention and audit obligations are satisfied, and not before. The temptation to decommission early is strong because the licence cost is visible and the risk is not.
Historical data is the question that determines the shape of this phase. Decide whether you are migrating history into the new platform, archiving it to cheaper storage with a documented retrieval path, or retaining the old system in a minimal read-only configuration until its retention window closes. Each is defensible; what is not defensible is discovering the answer after the contract has lapsed.
Document the cutover itself: what moved when, what was retired and on whose authority, and where historical data now lives. An auditor asking about an incident from before the migration should not require an archaeological expedition.
What to measure, and what not to
Migrations are usually reported on the wrong metrics because the wrong metrics are the easy ones. Ingest volume, number of detections ported and percentage of sources onboarded all describe activity rather than outcome.
The measures worth reporting are the ones that describe whether security operations improved: time from detection to a conclusion an analyst will act on, the proportion of alerts closed without human touch that were correctly closed, and the number of detections retired without a corresponding gap. The last one is the clearest sign that the migration achieved something a like-for-like replacement would not have.
Common mistakes worth naming
These recur often enough across migrations to be worth stating plainly.
- Cutting over on a fixed calendar date rather than on exit criteria, which converts a quality decision into a scheduling decision.
- Porting the full detection portfolio because retiring rules requires an argument and porting them does not.
- Leaving compliance reporting until after cutover, then discovering a report depends on a field that no longer exists.
- Running the shadow phase too short to include an unusual week.
- Treating the migration as a platform project rather than a SOC project, so the people whose work changes hear about it late.
- Underestimating parallel running cost, then compressing the overlap to save budget at exactly the moment the overlap is protecting you.
Where Spharaka fits
Spharaka AI SIEM is the analytics core of Sphere, ingesting logs and telemetry across endpoint, network, identity, cloud, SaaS and OT, and applying AI correlation rather than a static rule set. In a migration that matters mainly in Phase 2: the behavioural equivalents that let you retire rules instead of porting them are what makes the reduction possible.
Because Sphere unifies the SIEM function with SOAR, XDR and EDR, a migration is often also a consolidation, which changes the sequencing. Deployment supports cloud, on-premises and hybrid topologies, so organisations with data localisation obligations can keep telemetry inside the required jurisdiction throughout the migration rather than making an exception for the transition period.
Frequently asked questions
How long does a typical migration take?
Six to twelve months for a mid-sized or large enterprise, driven mainly by the size of the existing detection portfolio and the number of data sources with undocumented dependencies. The new platform is rarely the constraint; the archaeology on the old one usually is.
Do we need to rewrite every detection?
No, and trying to is the most common way these projects overrun. Rules encoding specific known-bad indicators usually port directly. Rules written to approximate abnormal behaviour are better replaced with behavioural detection, and a substantial number of long-tail rules that have never produced a true positive should simply be retired.
Can we run both SIEMs at once, and for how long?
Yes, and you should. Parallel running is what turns a cutover from a leap into a comparison. Plan for an overlap spanning at least one full business cycle, and budget for it explicitly, because compressing the overlap to save licence cost removes the protection at the moment you most need it.
What happens to our historical data?
Three options are defensible: migrate history into the new platform, archive it with a documented and tested retrieval path, or keep the incumbent in a minimal read-only state until its retention window closes. Decide before decommissioning, and record where the data ended up.
What is the most common cause of overrun?
Undocumented dependencies discovered mid-project, usually parsing quirks that detections rely on or downstream reports nobody knew existed. The three inventories described above are the cheapest insurance against this.
How do we know when we are ready to cut over?
When every detection firing in the incumbent but not in the new platform has been explicitly resolved as either a gap or an improvement, the last-mile integrations work, compliance reporting has been rebuilt, and a rollback trigger has been agreed. A calendar date is not a readiness criterion.
Should the migration be run by the platform team or the SOC?
The SOC has to own it, with platform engineering supporting. The work that decides success is detection triage and analyst adoption, and both belong to the people who run operations rather than to the people who run infrastructure.