Skip to content

Manual Guide: Data Archiving

The platform includes a powerful, built-in Data Archiving & Retention Engine that works out-of-the-box (OOTB) to maintain optimal database performance and manage transactional log lifecycles without requiring heavy manual maintenance.

1. Core Capabilities & Architecture

  • Automated Daily Scheduler: Runs background archiving automatically every day at 11:00 PM (23:00) based on a configured retention window (default: 7 days for successful, completed, or sent terminal records). Configurable under Logic Flow > MDLWR > Archive Transaction Table.
  • Smart Cascading (trace_id Linkage): Core transactional modules—including Jobs, Job Rows, Logic Flow Step Logs, and API Request Logs—are archived intelligently. When a parent Job qualifies, all related child records linked by the same trace_id are bundled and moved together to prevent orphan records. (Note: Mailbox / Email logs operate independently based on their own delivery timestamp and status).
  • Manual Toolbar Triggers: Administrators can trigger archiving on-demand using the dedicated Archive action buttons positioned directly above the respective active data tables (e.g., Incoming Logs, Jobs, and Mailbox).
  • Smart Restore: Archived datasets are never permanently lost. They remain fully accessible, searchable, and auditable under Custom Table > Data (filtered under the "MDLWR Archive Table" category). Each record features a built-in Restore button that safely reverts archived data (along with its related child chains) back to active production tables seamlessly.

2. Accessing & Restoring Archived Data (Smart Restore)

Archived records are securely relocated and can be fully managed or reversed via the UI:

  1. Navigate to Custom Table > Data.
  2. Select the category filter "MDLWR Archive Table".
  3. Choose the target archive table (e.g., mdlwr_jobs_archive, mdlwr_api_request_log_archive, etc.).
  4. Smart Restore Button: Each record or dataset in the archive view features a dedicated Restore button.
  5. When triggered on a Job or transactional log, the system intelligently restores the main record along with all associated child data linked by the same trace_id back to the active production tables seamlessly.
  6. Mailbox restores operate independently per email log entry.

3. Extensibility & Custom Table Archiving

If you create new custom tables in the future and want to implement archiving for them, the standard approach is to follow the one-to-one independent pattern (similar to the Mailbox module):

  • Mirrored Archive Table: Provision a matching archive table by appending _archive to the table name (e.g., pairing your_custom_table with your_custom_table_archive).
  • Explicit Column Definition: To ensure strict database stability and avoid query or wildcard errors, explicitly define the column list inside the archive script rather than using wildcard selectors.
  • Dedicated Custom Scheduler: You can set up a new individual scheduler under Logic Flow > MDLWR following the existing template logic, pointing to your custom table's timestamp/status conditions for straightforward independent archiving.

4. Database Backup & Disaster Recovery

  • Database Snapshots: Regular database backups should be performed at the engine level (such as PostgreSQL pg_dump), capturing both active schemas and archive tables prior to major upgrades.
  • Full Disaster Recovery: Restore the complete PostgreSQL database dump to a staging instance to verify data integrity before applying to production environments.

5. Standard Table Operations

For complete guidelines on how to use standard table utility features (such as ColVis, copying data, Export/Import, Refresh, Bulk Delete, etc.), please refer to the Standard Tables Guide.