Overview
System administrators can manage whether to create metadata for backups. Every time that cPanel & WHM creates a backup, it creates metadata for that backup. You can also manually generate backup metadata with the backups_create_metadata script. There are several settings that disable metadata creation. You may want to disable metadata creation for testing, before an upgrade, or if your server experiences performance issues.
Note:
- The
backups_create_metadatascript will generate metadata for all directories under the/home/usernamedirectory, except for the/mailand/.cpaneldirectories. - Backups and metadata must exist on the local disk before users can access the following interfaces in cPanel & WHM version 72 and later:
- cPanel's File and Directory Restoration interface (cPanel >> Home >> Files >> File and Directory Restoration).
- WHM's File and Directory Restoration interface (WHM >> Home >> Backups >> File and Directory Restoration).
Enable metadata creation
Metadata for backups version 3.1
Every time that cPanel & WHM creates a backup, it creates metadata for that backup. The system stores that metadata as entries in a username.db database (where username represents the cPanel account's username). Then, the system saves the database to the .meta directory under your configured backup directory. The metadata databases store the indexed information of their related backups. These smaller database entries provide a faster information-retrieval method than their source backup.
Metadata database tables
The following tables describe the fields of these smaller databases, the backup_paths table, the backups table, the file_changes table, the metadata table, and the seen_files table.
The backup_paths table
The backup_paths table lists the backup files' paths.
| Field | Type | Description | Possible values | Example |
|---|---|---|---|---|
backup_path | string | The backup files's filepath, relative to the configured backup directory. | A string value. | /backup/2018-04-12/accounts |
backup_id | integer | The backup file's identification number. | A positive integer. This value references the | 1 |
The backups table
The backups table lists the backup files on the disk.
| Field | Type | Description | Possible values | Example |
|---|---|---|---|---|
backup_id | integer | The backup file's identification number. | An automatically-incrementing positive integer. | 1 |
timestamp | timestamp | The backup files's creation date in Universal Time Coordinated (UTC) format. | An integer. | 1523642274 |
does_exist | Boolean | Whether the backup file exists. |
|
|
The file_changes table
The file_changes table lists backup file changes when any of the following actions occur:
- The first time the system backs up the file.
- The user modifies the file.
- The user removes the file.
| Field | Type | Description | Possible values | Example |
|---|---|---|---|---|
seen_files_id | integer | The filepath's identification number. | A positive integer. This value references the | 1 |
backup_id | integer | The backup file's identification number. | A positive integer. This value references the | 1 |
size | integer | The backup file's size, in bytes. |
| 660 |
mtime | integer | The date when the user last modified the file, in UTC format. | An integer. | 1523642274 |
operation | integer | The change type. |
| 0 |
type | integer | The backup file's type. |
| 0 |
The metadata table
This table stores metadata keys and values.
| Field | Type | Description | Possible values | Example |
|---|---|---|---|---|
key | string | The metadata code type. |
| schema_version |
value | string | The metadata code's version number. | A string value. | 3.1 |
The seen_files table
This table stores filenames.
| Field | Type | Description | Possible values | Example |
|---|---|---|---|---|
file_id | integer | The file's identification number. | An automatically-incrementing positive integer. | 1 |
path | string | The file's filepath, relative to the /home/username directory. | A string value. | /public_html/ |
The .master.meta file
The .master.meta file stores relevant data about cPanel user backups that is not stored in the metadata database tables. This information, unlike the information in the tables, doesn't change.
Every accounts directory inside the daily, weekly, and monthly backup directories contains a .master.meta file.
The /backup directory structure
The system stores backups inside the /backup directory in one of three different formats: compressed, uncompressed, and incremental. Each format uses a different filename extension.
Backup format | Filename extension | Example |
|---|---|---|
| compressed | .tar.gz | username.tar.gz |
| uncompressed | .tar | username.tar |
| incremental | None | username |
The system distinguishes each backup by its filepath, even if the system stores the backup information under the same name. The filepath includes the backup directory where the backup resides. The following example shows the layout and structure of a /backup directory.
This directory includes daily, monthly, and weekly compressed backups. You can distinguish each backup type by its complete filepath. This filepath is the file's full path. The system stores the metadata for this path in the backup_paths table, under the backup_path field.
The system also stores metafiles in the /backup directory and distinguished by their filepath. The system stores metafiles inside the /backup directory as entries in a username.db database (where username represents the cPanel account's username).
Note:
- The system enables metadata creation automatically in cPanel & WHM version 66 and 68. You cannot disable metadata creation in these versions.
- If you upgrade from version 66 or 68 to later versions of cPanel & WHM, your backup configuration may affect whether the File Restoration interface (version 70) or the File and Directory Restoration interface (version 72 and later) is disabled or visible. For more information, see the cPanel & WHM version 70 or cPanel & WHM version 72 or later tabs in the Disable metadata creation section of this document.
Metadata for backups version 1.0
Every time cPanel & WHM creates a backup, it also generates a backup metadata file that maps the contents of that backup's directory. The metadata lists files and directories in the backup and stores the information in CSV format. You can also manually generate a backup metadata file with the backups_create_metadata script.
Metadata files store indexed information of their related backup. These smaller database entries provide a faster information-retrieval method than their source backup.
Backup metadata format
A backup metadata file, or metafile, has two sections. The first section contains the metafile's meta attributes and appears above a dotted line. The second section provides details of the backup archive's files and appears below the dotted line.
Compressed and uncompressed backups metafile example
A metafile for compressed and uncompressed backups will resemble the following example:
server,example.cpanel.net uid,2183 gid,2186 username,abc1 metaversion,1 date,2017-02-28 epoch,1488261600 archive_size,653824 uncompressed_size,531748 pkgacct_version,10 archive_version,3 file_count,108 ------------------------------------------------- 720,"2017-01-30 09:16",YWJjMS9jcC9hYmMx,/backup/2017-02-28/accounts/abc1.tar,0,2186 4221,"2017-03-01 09:11",YWJjMS9kbnN6b25lcy9hYmMxLnRsZC5kYg==,/backup/2017-02-28/accounts/abc1.tar,0,0 1731,"2017-03-01 09:11",YWJjMS9kbnN6b25lcy9hZGQxYWJjMS50bGQuZGI=,/backup/2017-02-28/accounts/abc1.tar,0,0 1675,"2017-01-16 07:44",YWJjMS9kb21haW5rZXlzL3ByaXZhdGUvYWJjMS50bGQ=,/backup/2017-02-28/accounts/abc1.tar,0,12 ....
Incremental backups metafile example
A metafile for incremental backups will resemble the following example:
server,example.cpanel.net uid,2183 gid,2186 username,abc1 metaversion,1 date,2017-03-22 epoch,1490158800 archive_size,836 uncompressed_size,533260 pkgacct_version,10 archive_version,3 file_count,110 ------------------------------------------------- 39,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvdmVyc2lvbg==,FILE,0,0 11,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvaG9tZWRpcl9wYXRocw==,FILE,0,0 0,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvaGFzX3NzbHN0b3JhZ2U=,FILE,0,0 8,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvc3NsZG9tYWlu,FILE,0,0
Meta attributes
The meta attributes section displays backup archive values, in CSV format, and uses a attribute name, attribute value convention.
| Attribute | Description | Example |
|---|---|---|
server | The server's hostname. |
example.cpanel.net
|
uid | The system user's identification number on the server. | 2183 |
gid | The user's file's group identification number. | 2186 |
username | The system user that owns the backed-up files. | abc1 |
metaversion | The metafile's format version. | 1 |
date | The date when the system backed up the file. | 2017-02-28 |
epoch | The date in Unix epoch time. | 1488261600 |
archive_size | The archive's size, in bytes. For example, the size of the | 653824 |
uncompressed_size | An uncompressed archive's size, in bytes. | 531748 |
pkgacct_version | The package account script's version. Note: cPanel & WHM uses the | 10 |
archive_version | The program version that the system uses to prepare the backup archive. | 3 |
file_count | The number of files in the backup archive. | 108 |
Backup archive files' details
This section displays backup's details in CSV format. The format uses two conventions: one for compressed and uncompressed backup files, and one for incremental backups.
Notes:
- The backup resides on the disk.
- The file details list the full path to the backup. The backup archive contains the backup's entire data.
Compressed and uncompressed backup file details
| Part | Description |
|---|---|
| File size | The backup file's size, in bytes. |
| Last modified date | The date when the system last modified the file. |
| The file's archive name | The file's name in the archive, stored in Base64 format. |
| The file's full path in the archive | The full path to the file in the archive. |
| UID | The system user's user identification number on the server. |
| GID | The user's file's group identification number. |
Incremental backup details
| Part | Description |
|---|---|
| File size | The backup's size, in bytes. |
| Last modified date | The date when the system last modified the backup. |
| The file's archive name | The backup's name in the archive, stored in Base64 format. |
| The file's full path in the archive | The full path to the backup in the archive. |
| The archived file's type | One of these archived file types:
|
| UID | The system user's identification number. |
| GID | The user's file's group identification number. |
Notes:
The system can only store the system backup (/backup/.../system) as a compressed or uncompressed backup file. The system cannot store a system backup as an incremental backup file. This condition creates a limitation when you configure a remote destination for your backups in the Additional Destinations section in WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). Because the system can only store system backup files as compressed or uncompressed files, the system copies the system backup file to the destination.
The /backup directory structure
The system stores backups inside the /backup directory in one of three different formats: compressed, uncompressed, and incremental. Each format uses a different filename extension.
| Backup format | Filename extension | Example |
|---|---|---|
| compressed | .tar.gz | username.tar.gz |
| uncompressed | .tar | username.tar |
| incremental | None | username |
The system distinguishes each backup by its filepath, even if the system stores the backup information under the same name. The filepath includes the backup directory where the backup resides. The following example shows the layout and structure of a /backup directory.
This directory includes daily, monthly, and weekly uncompressed backups. You can distinguish each backup type by its complete filepath. This filepath is the file's full path shown in the metafile's backup file details.
The system also stores metafiles in the /backup directory and distinguished by their filepath. The system stores metafiles inside the /backup directory as username-=-meta, where username represents the system user's name.
Disable metadata creation
You can manually disable metadata with WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). The following table shows which settings affect the management of metadata manually and the Backup Configuration interface. These settings function independently from one another.
Notes:
- When you configure any of these settings to disable metadata creation, the following actions occur:
- The system disables cPanel's File and Directory Restoration interface (cPanel >> Home >> Files >> File and Directory Restoration ) and WHM's File and Directory Restoration interface (WHM >> Home >> Backups >> File and Directory Restoration ). The interfaces display a notification that explains why the feature is unavailable.
- You can disable metadata with the WHM API 1
backup_config_setfunction.
| Backup configuration file setting | Backup Configuration interface setting | Type | Description | Possible values |
|---|---|---|---|---|
DISABLE_METADATA | None. You must manually configure this setting with the WHM API 1 | string | Whether the Backup system creates metadata when a backup runs. |
|
BACKUPACCTS | Backup Accounts | string | Whether the system includes cPanel user accounts in the backup. Note: System administrators can select which cPanel user accounts they wish to include in the backup through the Backup User Selection interface (WHM >> Home >> Backup >> Backup User Selection). |
|
BACKUPENABLE | Backup Status | string | Whether the WHM System Administrator wants backups enabled. |
|
BACKUPMOUNT | Mount Backup Drive as Needed | Boolean | Whether the system mounts the backup directory as a mount point before a backups runs, then unmounts it when the run completes. |
|
KEEPLOCAL | Retain backups in the default backup directory. | Boolean | Whether the system retains backups in the default local backup directory. |
|
You can manually disable metadata with WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). The following table shows which settings affect the management of metadata manually and the Backup Configuration interface. These settings function independently from one another.
Notes:
- When you configure any of these settings to disable metadata creation, the following actions occur:
- WHM disables the File and Directory Restoration interface (Home >> WHM >> Backup >> File and Directory Restoration). The interface displays a notification that indicates the setting that disabled it.
- cPanel hides the File and Directory Restoration interface (Home >> cPanel >> Files >> File and Directory Restoration) from the cPanel Home interface.
- You can disable metadata with the WHM API 1
backup_config_setfunction.
| Backup configuration file setting | Backup Configuration interface setting | Type | Description | Possible values |
|---|---|---|---|---|
DISABLE_METADATA | None. You must manually configure this setting with the WHM API 1 | string | Whether the Backup system creates metadata when a backup runs. |
|
BACKUPACCTS | Backup Accounts | string | Whether the system includes cPanel user accounts in the backup. Note: System administrators can select which cPanel user accounts they wish to include in the backup through the Backup User Selection interface (WHM >> Home >> Backup >> Backup User Selection). |
|
BACKUPENABLE | Backup Status | string | Whether the WHM System Administrator wants backups enabled. |
|
BACKUPMOUNT | Mount Backup Drive as Needed | Boolean | Whether the system mounts the backup directory as a mount point before a backups runs, then unmounts it when the run completes. |
|
KEEPLOCAL | Retain backups in the default backup directory. | Boolean | Whether the system retains backups in the default local backup directory. |
|
You can manually disable metadata with WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). The following table shows which settings affect the management of metadata manually and the Backup Configuration interface. These settings function independently from one another.
Notes:
- When you configure any of these settings to disable metadata creation, the following actions occur:
- WHM disables the File Restoration interface (Home >> WHM >> Backups >> File Restoration). The interface will display a warning that indicates the setting that disabled it.
- cPanel hides the File Restoration interface (Home >> cPanel >> Files >> File Restoration) from the cPanel Home screen.
- You can disable metadata with the WHM API 1
backup_config_setfunction.
| Backup configuration file setting | Backup Configuration interface setting | Type | Description | Possible values |
|---|---|---|---|---|
DISABLE_METADATA | None. You must manually configure this setting with the WHM API 1 | string | Whether the Backup system creates metadata when a backup runs. |
|
BACKUPACCTS | Backup Accounts | string | Whether the system includes cPanel user accounts in the backup. Note: System administrators can select which cPanel user accounts they wish to include in the backup through the Backup User Selection interface (WHM >> Home >> Backup >> Backup User Selection). |
|
BACKUPENABLE | Backup Status | string | Whether the system creates backups. |
|
KEEPLOCAL | Retain backups in the default backup directory. | Boolean | Whether the system retains backups in the default local backup directory. |
|
BACKUPMOUNT | Mount Backup Drive as Needed | Boolean | Whether the system mounts the backup directory as a mount point before a backups runs, then unmounts it when the run completes. |
|
Note:
- The system enables metadata creation automatically in cPanel & WHM version 66 and 68. You cannot disable metadata creation in these versions.
- If you upgrade from version 66 or 68 to later versions of cPanel & WHM, your backup configuration may affect whether the File Restoration interface (version 70) or the File and Directory Restoration interface (version 72 and later) is disabled or visible. For more information, see the cPanel & WHM version 70 or cPanel & WHM version 72 or later tabs in the Disable metadata creation section of this document.
Additional documentation