Backup Procedures
Last updated
Last updated
The SanteDB iCDR and dCDR installed at clinics should be regularly backed up. As part of the Develop Operational Technology Architecture activities performed during the planning phase of your deployment, there should have been an effort to establish a Maximum Tolerable Outage (MTO).
The MTO is the point at which it is tolerable to lose data. The obvious desire would be to never lose data, however the amount of additional equipment to provide that amount of system redundancy is almost always cost-prohibitive.
This value of the MTO will dictate the frequency of your backup schedule. For example, if operating SanteMPI in a context where real-time registration is performed without any re-queueing mechanisms from the client, then that would mean data corruption of the SanteMPI server would result in lost data.
If a backup is taken every 24 hours, then the maximum amount of data that would be lost in the case of a catastrophic failure would be the number of hours that have elapsed since the previous day's backup was completed.
The method of backup for the iCDR will depend on the requirements of your environment. The process of your backup and the system architecture of the backup design will depend somewhat on your recovery and storage space available.
Whatever the method of backup, it is generally a good idea to keep backups of the iCDR in several places:
Online "Hot" Backups: A "hot" backup should be online and accessible by the host environment and usually provides one or two of the most recent backups (imagine on a daily schedule that the previous 2 days of backups are kept). Example approaches include using:
Local Network Attached Storage (NAS) devices
Local Attached Disk devices
Cloud Based Storage options
Offline "Cold" Backups: These are backups which are available only after media (such as tapes, removable media, etc.) is retrieved and restored. These are typically used for longer term backups and mitigate against active malware, randomware, or viruses which may lay dormant (i.e. a copy of the backup prior to infection can be restored). Examples include:
Tape / Removable Mass Media Storage
Disconnected disk devices
Additionally, copies of backups should be kept both in a location which is onsite (rapidly available for restoration) as well as offsite (in case of natural disaster, loss or damage to the data center).
Backup Technology | Benefits | Risks |
---|---|---|
The 3-2-1 backup strategy is highly recommended for use in production deployments of the SanteDB iCDR. This strategy is illustrated below.
3 Copies of Data: At least 3 copies of the data should be available. At minimum the active (or primary copy) and at least 2 other copies should be available.
2 Different Media: The copies of data should be stored on at least 2 different media or hosts. For example, the data exists on production server and a NAS and a cloud provider.
1 Copy Offsite: At least one copy of the backup should always be kept offsite on a cloud provider or remote physical media.
There are many ways to realize this backup strategy using various technologies, to illustrate this, the SanteDB community server backup strategy is shown below:
The backup on the community server:
Every night at 12:00 AM Microsoft Windows Server Backup performs an incremental backup of the most recent copy of all Virtual Disks from the physical host to a FreeNAS server using UNC Windows Shares and Volume Shadow Copy (live backups) to a fast pool (using SSD based disks in a RAID Z1). Snapshots of the fast pool are taken every 24 H and only 2 snapshots are kept on fast storage (2 days of backups). This pool is used for immediate restore if issues are detected within 24 H (2 copies of the data now exist)
The FreeNAS server then, every morning at 6:00 AM replicates the fast pool on SSD disks to a slow pool using 7200 RPM disks in RAID Z1. Snapshots of this pool are taken every 48 H and 30 snapshots are kept (30 days of backups). This pool is used for restore of infrastructure in the event data corruption occurred prior to a 48 H window (3 copies of the data now exist)
The FreeNAS server then, copies older snapshots from the slow storage pool to a physically attached USB hard drive. This hard drive has no limitation (beyond storage space) on snapshot lifetimes. After 30 days or earlier (when full), the drive is unmounted and stored at a different location.
The community server is not mission critical and therefore an MTO of months is permissible. If your deployment has a MTO value which indicates that only 24 H or 48 H of missing data is tolerable, then a different strategy should be employed.
The simplest way to backup your SanteDB VM is to back up the virtual hard drives on which the SanteDB iCDR runs. This solution provides several advantages:
Restoration of the iCDR is relatively straightforward (i.e. restore the VM)
No need for special backup software or processes (many operating systems and hypervisors provide methods of copying VMs)
Incremental backups can be taken using snapshotting technology (for example, Hyper-V can perform live, in-flight backups using Volume Shadow Copies).
No need to understand complex database restore procedures.
There are disadvantages to virtual disk backups, these can include:
The amount of space required to store the files may be larger (this can be mitigated by separating "data" partitions from "OS" partitions and only backing up the data partitions)
The amount of data which needs to be shared/shipped to network backups may be larger.
Some hypervisors may require the VM to be shut down during backup
It is also possible to leverage database backups for your backup strategy of the iCDR. Implementers may use the built-in pg_dump
backup method (see tutorial here) or they may use a third party software tools (see Bacula as an example).
The advantages of performing data dump backups are:
Size of the backup files are typically smaller
Does not rely on a hypervisor technology (works on bare-metal deployments, cloud deployments, docker, etc.)
Data backups can be encrypted and compressed by piping output to different utilities
The disadvantages of performing data dump backups are:
Restoration is more complex (requires a new PostgreSQL server to restore the copy of the data to, and often binary backups require the same version of PostgreSQL)
Foreign key and data constraints can interrupt backup restore procedures.
Slower to restore data (especially if using a SQL based dump).
To perform a backup of the SanteDB database you can use the following command:
If you wish to encrypt and compress the backup, then you may additionally use the 7ZIP tool
Restoring a backup which was created using the data dump pattern is relatively straightforward process.
Decrypt your backup file (if you encrypted it)
Decompress the backup file
Run the contents through pg_restore
or the psql
command.
For example, to extract a backup to a new database called newdb
from an encrypted backup in 7zip.
The following dCDR products automatically backup their local database upon system service startup and according to the backup job in their software:
SanteDB dCDR Gateway
SanteDB dCDR Windows Client
SanteDB dCDR Android Client
Each of these dCDR instances will provide a system job for the backup task
Backups are encrypted AES256 encryption with a passphrase matching the name of the device which produced the backup. It is important to keep the name of the machine in order to restore the backup.
Backup files can be accessed in the following locations:
On Windows Operating Systems the backups are stored in %appdata%\local\SanteDB\<instance-name>\backup
On Linux Operating Systems backups are stored in ~/.local/SanteDB/<instance-name>/backup
On Android Operating Systems backups are stored in ~/Documents/SanteDB
To restore a dCDR backup is a straightforward process which differs based on the technology used.
Performing a restore will overwrite any data in the current dCDR instance. Ensure that all necessary data is properly synchronized prior to continuing a restore operation.
To restore on the dCDR gateway, administrators should run the santedb-dcg
command with the --restore
option:
The --sysrestore
should be used in an elevated command prompt on Windows and is used to restore the backup not to the current user's %appdata%
directory but to the System %appdata%
directory usually located in C:\Windows\SysWOW64\config\systemprofile
.
You will be prompted for a backup password (the name of the original device which produced the backup). On Windows the restore process:
Shuts down the SanteDB dCDR (on Windows)
Decrypts and restores the data files to the appropriate directories
Starts the SanteDB dCDR (on Windows)
To restore files on the Windows (or Linux in the future) standalone clients, users should follow the process:
Ensure that the Windows application is configured with the same device name as the backup file.
Copy the sdbk
file to the %localappdata%\santedb\<instance-name>\restore
directory
Start the SanteDB Windows Application
Backup will automatically be processed and imported.
To restore a database on Android:
Ensure the *.sdbk
file you want to restore is placed in the Documents
folder of the tablet.
Wipe the data for the SanteDB dCDR application (in the Application Manager clear the data for the application)
Configure the application using the same tablet name and subscription settings as before
When prompted at startup, answer "YES" to the restore from backup prompt provided