On‑Prem Corporate Email Archive: Indexing and Retention

Why you should fix requirements in advance, not "as it goes"

An on-prem corporate email archive that’s launched without clear rules may look usable at first. Problems surface later — during an audit, an internal investigation or a legal request. Then you discover that an "archive exists," but results are not predictable.

The most common pain point is search. Two people run the same query and get different results. The causes are usually simple: one can see attachments while the other cannot; only subjects and senders are indexed in some places; some messages arrived in the archive with delay; different mailboxes are handled by different rules. If requirements aren’t documented in advance, you won’t be able to explain why this happened or fix it quickly.

The second problem is retention. If messages are deleted earlier than their retention period (from the mailbox or even from the archive), evidence and context are lost. Even if “important things” are usually kept, a single cleanup script or policy change can leave a hole in correspondence for a needed period.

Access rights are a separate risk. Legal or security teams may have the right to search but not to view content, or vice versa. As a result, a request can stall: only an admin may produce an export, but the admin shouldn’t see the data.

Manual exports from mailboxes are especially vulnerable. They almost always use different parameters (folders, date ranges, attachments), it’s hard to prove completeness and immutability, and chains or hidden attachments can be missed. The result depends on the executor and their permissions.

It’s more reliable to agree and document in advance: what is indexed, what retention periods apply, who and how performs searches and exports, and how this is confirmed in an audit. Then any request is handled consistently, regardless of people or shifts.

What exactly do you archive: data composition and boundaries

An on-prem corporate email archive usually "breaks" less on hardware and more on disputes over what counts as a message and where it comes from. So start by recording the archive object types, then move on to indexing and retention.

To make search and legal requests predictable, you typically need at least:

• the message body (including HTML and plain-text versions if they differ)
• attachments as separate files plus the link "which attachment belongs to which message"
• headers and metadata: From/To/Cc/Bcc, Message-ID, dates, subject, size, language, encodings
• thread links and relations (In-Reply-To/References) so conversations can be reconstructed without guessing

If actions on messages matter for investigations, explicitly decide whether you store service events (forward, reply, status change, deletion, move) and in what form.

Next, define sources. If the archive only pulls data from the mail server, you might miss what passed through a gateway (antivirus, antispam, DLP) or was rejected before delivery. When provability matters, decide in advance whether to ingest gateway logs and transport logs as proof of sending and routing.

Boundaries are as important as composition. Clearly list what is in scope and what is not: personal mailboxes, shared mailboxes and functional accounts, aliases, external forwards. Decide separately how you handle messengers and group chats: either they are outside the archive, or they require a separate storage and search environment.

Practical example: in a government agency, a shared mailbox "tender@" is managed by three people. If you only archive personal mailboxes, tender-related correspondence becomes "holey" and eDiscovery search will be incomplete.

Indexing requirements: what should be searchable and how fast

Indexing solves a simple problem: you must find what is actually stored, and find it the same way every time. If the index is incomplete or handles formats poorly, an on-prem corporate email archive becomes a warehouse where answers depend on chance.

First, record exactly what is indexed. Typically this includes:

• message body (text and HTML)
• headers: From/To/Cc/Bcc, subject, Message-ID, date, routing
• attachments (content and filename)
• embedded messages (for example, .eml inside a message)

Resolve the most common conflict immediately: "attachment exists but is not readable." Specify supported formats (docx, xlsx, pdf, jpg) and rules for rare types: skip, store without index or convert. For PDFs and scans separately decide whether OCR is required. Without OCR you cannot search text in scans, and users will treat that as an archive error.

For Kazakhstan, Cyrillic and mixed languages (ru, kk, en) are almost always important. Requirements should explicitly state support for UTF‑8 and legacy encodings, correct searching across word forms (at least basic), and rules for Kazakh characters and transliteration when present.

Normalization also affects predictability. The same message may appear as original, forwarded or copied across multiple mailboxes. Decide in advance how to handle duplicates: remove by Message-ID and attachment hash or keep all copies but show grouped results.

Finally — speed. Record target metrics for typical queries (by sender, subject, phrase in an attachment) and for a typical volume (for example, 1 month across 5,000 mailboxes). A practical guideline: first results within 3–10 seconds, and exports of large sets should show progress and estimated time so a legal request does not hang "forever."

Retention and legal hold: storage rules and exceptions

Retention defines a clear rule: how long to keep and what to delete when the time is up. In an on-prem corporate email archive this is critical because users often assume "the archive keeps everything forever," while lawyers expect stable, repeatable behavior.

Start by separating data types. Messages and attachments have different lifecycles: messages are often needed for investigations, attachments take most space and often fall under separate rules (for example, contracts or medical documents). Describe shared mailboxes and service addresses (support@, procurement@) separately — there is a higher risk that a retention deletion will affect business processes.

Policies are rarely uniform. In practice differences typically include: different retention for departments (finance, HR, sales), special rules for projects and tenders, longer retention for shared mailboxes and executives, exceptions for regulatory categories (by internal list), and a clear start date (sent date, received date or last modified).

Legal hold is needed to "freeze" without manual workarounds. A legal request must not extend retention for everyone. It should block deletion selectively for chosen people, mailboxes, periods and topics. It’s important that a hold covers both messages and attachments, and that removing a hold returns objects to normal rules rather than accidentally making them permanent.

Deletion by policy must be controlled: who approves policies, who can trigger deletion, and how the action is recorded (log, report, identifiers, timestamp). If policies conflict (retention expires but legal asks to keep), legal hold takes precedence. Document the process: who places a hold, within what timeframe, how to verify deletion is stopped, and who confirms hold removal after the case is closed.

Access rights: who can search, view and export

Describe archive access rights as strictly as the mail system. Otherwise the on-prem corporate email archive becomes a place where "anyone can do everything," and search and export results are hard to reproduce and protect.

Roles are easier to assign by actions rather than job titles. Searching metadata and reading message content are different risks. Export is the most sensitive step because data can easily leave the controlled environment.

A good scheme follows least privilege and separation of duties so one person cannot search, read and export correspondence without oversight. For example:

• an administrator maintains the system but by default does not read messages
• InfoSec configures access policies and reviews audit logs
• lawyers create requests, run searches and prepare sets for disclosure
• HR works only on employee cases within approved justification
• managers receive summary reports or extracts but do not run searches themselves

Specify rules for shared and delegated mailboxes. In mail a delegate may read "on behalf" of an owner, but in the archive that does not always imply the right to read historical messages. Decide in advance whether delegated rights are inherited, how mailbox owner changes are handled, and who approves archive access after an employee leaves.

For investigations and legal requests prefer issuing temporary access with a fixed scope and reason. A working procedure usually includes a request form with period and mailbox list, role assignment with expiry (auto-revocation), two-step export approval (e.g., lawyer + InfoSec), delivery of the export in an agreed format and case closure with audit review.

The more precisely rights and responsibilities are defined, the more predictable eDiscovery becomes and the fewer disputes over "who viewed what" and "why this was exported."

Audit and provability: so results can be defended

Organizations often implement an on-prem corporate email archive to achieve predictability. That predictability ends where you cannot prove who searched, what exact results were found, and whether an export was altered on its way to lawyers or a regulator.

First layer — activity audit. Logs should record not only logins but steps that affect a request result: search parameters (filters, date ranges), viewing messages and attachments, export and its format, changes to retention policies, legal holds and access rights, and admin operations on indexes and data sources.

Second layer — immutability. Decide in advance the required level of protection: deletion prohibition, WORM storage, separate roles for admins and a compliance officer, separate storage of encryption keys. If an admin can quietly edit an archived message or swap an attachment, even a perfect search won’t help.

Third layer — chain of custody. When an export leaves the archive you must answer: who requested, who approved, who generated, who received it, where a copy is stored, and how access is controlled. It’s practical to deliver exports as a package with a report and integrity checks.

Agree in advance which artifacts are kept with an export. Usually these are checksums (for files and the whole package), a request report (criteria, time, number of items found), an action log for the case with user IDs, and the version of policies and settings in effect at the search time.

Simple example: a lawyer requests correspondence for a contract covering 30 days. If a dispute arises a month later, you must present not only the messages but also prove they were extracted by a specific query, at a specific time, under the policies in effect, and that the package wasn’t modified after export.

Step-by-step: how to form requirements before deployment

Start not from product selection, but from who is responsible for rules. For an on-prem corporate email archive ownership of requirements is usually collective: IT is responsible for operability and integrations, InfoSec for access and risk, lawyers for provability and retention, and the business for practical scenarios. Appoint a coordinator and agree who approves the final version.

Collect 5–10 typical situations where the archive is actually needed: court request, internal incident review, procurement dispute, policy compliance check, HR investigation. Describe not only "what we search for" but also "who can request it," "how fast results are needed" and "what constitutes an acceptable output."

Minimum set of requirements to document

It’s easier to write requirements as a short document that can be validated with a test:

• data composition and boundaries: which mailboxes, periods, attachments, calendar items, contacts, service logs
• search fields and expected behavior: sender, recipient, period, subject, keywords, attachment presence and type
• retention policies and exceptions: retention by data type and when legal hold applies
• roles and access procedures: who can search, who can view full messages, who can export, who approves
• quality metrics: indexing window, search response time for typical queries, result completeness

Test run before purchase and rollout

Use a real sample (for example, 2–4 weeks of messages from several departments) and run scenarios. Example: a lawyer requests correspondence with a counterparty over 30 days including attachments, while InfoSec requires access to be granted only for a specific case and fully logged.

Record figures: how many messages found, how many missed, how long it took, what permissions were required. If results “float,” refine requirements and repeat tests before deployment.

Common mistakes that make search and requests unpredictable

Even a good on-prem corporate email archive starts to fail if requirements are not translated into concrete settings and checks. Problems usually surface not in the pilot, but when a legal request arrives and the deadline is tight.

Mistakes that break expected results

Unpredictability most often stems from details nobody defined as mandatory:

• the index does not cover all attachment types: PDFs are searchable but scans, archives or embedded messages (.eml/.msg) are not
• times in messages and logs use different time zones, and a filter "from 01 to 30" excludes part of the correspondence
• access rights are configured so searches run only against some sources while the system silently excludes others
• legal hold is recorded as an email or task, but no technical block appears in the archive
• exports are done manually "however it comes out": different formats, no checksums and no action log

Small real-life example

A lawyer requests correspondence for a project over 30 days expecting date and keyword filters to yield a complete set. But some messages fall outside the period because of time zone differences, and key contracts are not found because scanned attachments were not indexed. The team spends time on manual review and the delivered package becomes contested.

To avoid this, agree on validations in advance: which attachment types must be indexed, which time zone is the authoritative one, which roles can search and export, and what the system should do when a hold is placed.

Short checklist: what to verify before launch and regularly

Before launching an on-prem corporate email archive agree on checks to run on a schedule. This is cheaper than troubleshooting under pressure when a legal request arrives and "nothing is found."

Minimum checks for indexing and search

Ensure search works on the data that matters in real cases, not only on "subject." Verify and record:

• which fields are actually indexed (From/To/Cc/Bcc, subject, body, date, Message-ID, attachments)
• which fields are searchable and how queries are interpreted (exact match, partial word, morphology, phrase search)
• the time between message arrival and appearance in results (indexing window)
• how problematic objects are handled (encrypted attachments, nested archives, unusual encodings)

Storage, legal hold and rights

Then verify retention rules and how they cope with personnel and organizational changes:

• retention for key mailboxes and shared addresses: period, start event, what counts as deletion
• how legal hold is enabled: who initiates and approves, how fast it applies, whether attachments are included
• who can search, view and export: roles, least-privilege principle, separation of duties
• export: format, marking of result, where export is stored and who can access it
• audit log: where it’s kept, what is recorded (search, view, export, policy changes), retention period

A test case is often missed. Use a short scenario with a clear expected result (for example, 10 messages with a PDF attachment, 2 messages with the same subject, one deleted) to regularly confirm predictable search and policy behavior.

Example scenario: legal request for 30 days of correspondence

Imagine lawyers ask the company to produce correspondence for the last 30 days about a disputed contract. For an on-prem corporate email archive it’s important that results are repeatable: today you find X, tomorrow the same query returns the same X.

Lawyers should provide scope, and it’s best to record it in writing:

• period: precise dates and time zone
• participants: senders, recipients, Cc and Bcc
• keywords and variations (project name, contract number)
• attachments: which types matter (PDFs, scans, spreadsheets) and whether versions are needed
• exclusion rules: personal mail, mass mailings, system notifications

Then narrow the query without losing relevance. Usually you start broad (period + participants) then add keywords. Don’t lose "indirect" messages: replies in threads without the keyword, forwards, and messages where relevant text is inside an attachment. Validate the search against 3–5 control messages that must be in the results.

Apply legal hold for the specific mailboxes and period to prevent deletion or modification in the archive even if normal retention would have removed them. Release the hold only after lawyers confirm case closure.

Prepare the export so it can be reviewed and verified: a single format (for example, PST/EML + folder tree), folders by mailbox, a description file with criteria, dates, time zone and item counts.

And separately — audit. It must be visible who ran the search, which filters were used, who exported, who received a copy and when. This protects IT and legal teams if completeness of the production is later questioned.

Next steps: infrastructure, pilot and responsible operation

Once search, retention and access requirements are documented, the next risk is "set and forget." For an on-prem corporate email archive move immediately to a plan: how much data you will have, what infrastructure is required and how you prove the system behaves predictably.

Infrastructure: count more than mailboxes

Start with a simple estimate: how many active mailboxes, average monthly growth, what share is attachments and how long data is retained. These numbers determine storage, index load and backup windows.

To avoid "worked yesterday, fails today," plan for:

• spare capacity for growth and peaks (for example, mass mailings)
• fault tolerance: what happens if a node, disk or site fails
• special attention to indexes: their recovery is often harder than message recovery
• regular backup checks and clear RPO/RTO for the archive

Pilot and operation: acceptance criteria and discipline

The pilot should be small in scope but realistic in scenarios: legal request, export, audit trail, message and index recovery. Agree who owns the process, who administers and who can search and export.

For acceptance, define measurable criteria:

• search speed for typical queries (by sender, subject, attachment)
• completeness: how many messages and attachments actually reach the archive
• reporting on retention and legal hold without manual spreadsheets
• audit: who searched, viewed and exported
• regular restore test (messages and index)

If you need an on-prem turnkey project, GSE.kz can help with selecting and supplying servers and workstations, system integration and 24/7 support so the pilot does not stall on infrastructure and ongoing operation.