Import/export of Requirement Yogi data to a Confluence or Jira instance

New in 3.4.8

The backup / export / re-import feature was added in Requirement Yogi 3.4.8 (November 2022).

Prologue

Some customers in the situation of a merger-acquisition, decide to centralize all their Confluence and Jira instances, by migrating space-by-space to a new large Data Center cluster. The major difficulty is that database tables of Requirement Yogi can't simply materialize the foreign keys to entities of Confluence, because the plugin data is isolated from Confluence entities (and this is generally an excellent choice, albeit with this drawback). Therefore, the customers who chose to perform this move had to understand the data, reimport in the new instance and update the foreign keys on the fly, to point to new space keys, new page IDs, etc. It can take weeks to perform this operation, with an EDI, a database consultant and our help.

The goal is to accelerate this process to a few hours, however we can't totally automate it given the complexity of situations.

In what situations can this be used?

Situation	Is it supported?
Confluence* DC** instance → New Confluence DC instance Export and reimport a whole instance, with all the data. _{* Same with Jira} _{** Server and Data Center are both supported.}	Yes, it is supported. We recommend to use the same RY version on both sides (and we support a small range of versions). Please perform the Confluence-level import first, so pages, spaces, etc. already exist in the destination, Please create the application links, and set them up in the RY administration, before starting the RY import, Requirement Yogi settings will be imported. Page IDs, user keys, etc. must be remapped manually. This will be a long and mundane operation on your side. During the mappings, you can even change the space keys. Note that Confluence and Jira are imported separately, therefore: After a Confluence reimport, perform a Jira reimport.
Confluence DC → Same Confluence DC, but later in time.	Yes, but there is no reason to do that. Please simply backup your Confluence instance using a way recommended by Atlassian.
Confluence DC space → another Confluence DC instance. Ideal for merger-acquisitions, Ideal for centralization into a large Confluence/Jira DC cluster.	Yes, it is supported. Same specificities as above, The global entities (the settings) will not be imported, If there are pages outside the space pointing to these requirements, they will keep pointing to the old place.
Confluence space → New Confluence space with different key. Rename a space	Yes, it is supported. Please have the new space with the page IDs ready, before starting the RY import, Requirements can even exist in both spaces after import, but Jira links will only point to one of them. Jira links won't be updated, and this will be a problem. Please export-then-reimport the Jira data as well, mapping the space key with the new one.
Backups	No, this is not supported. We require the same version on both sides, and it may take some back-and-forth between the two sides, Confluence data is not part of the RY backup, so you would have to perform a Confluence backup anyway, It is not a sane way to perform backups (backup size, process, etc.).
Migrate from DC to the Cloud	Yes, you can migrate macros and Jira links, but we do not support other RY data (baselines etc.). Find more information here: Migration to the Cloud
Migrate from the Cloud to DC	No, this is not supported yet. Same, we're hoping to do it.

Of course, before committing to using the software in a particular way, please test that it fits your situation. As a reminder, to the maximum extent permitted by applicable law, this software is provided by Requirement Yogi "as is". Any express or implied warranties, including, but not limited to, the implied warranties of fitness for a particular purpose are disclaimed.

In what situations will it be difficult?

Situation	Difficulty
A space is renamed	Requirement entities will be fine, but: Cross-space references (Pages outside the space) will keep pointing to requirements in the old space. There is nothing we can do, since we're not touching the data outside the space. You will have to edit the XML of pages, directly in the database. Jira links will point to the old space. To avoid this, export-then-reimport the Jira data as well, and map the space key. This is better supported than page-to-page cross-space references, since there is no XML to edit in Jira issues. The history of Jira issues will still reference the old space keys, in case they have changed.
A Jira project is renamed	Same as above: Please also update the references in Confluence, since there are links to Jira, and this can be performed by exporting-then-reimporting the Confluence side.
Requirements are moved to a new instance	No difficulty, you just have to remap to a new Applink URL.
All page IDs have changed.	This happens when you import on a new Confluence instance. You will have to look for the page with the same title, for each page ID and page version. It is better if you gain SQL access to the database and update the table AO_32F7CE_DBBACKUPMAPPING in SQL, so you can retrieve the new page ID with a subquery.
There are requirements in Excel files.	We will reimport as-is, but the Excel attachment IDs will be wrong, since we do not remap them. Links to the Excel files will be broken, but the requirements will still exist.

What are the steps of export / re-import?

Origin instance

Destination instance

1. Export to a table*

2. Write the table to a CSV file

3. Delete the temporary table

4. Load the CSV file into the temporary table

5. Perform the mappings
(automatically, then manually)

6. Import the data

7. Re-start at step 5,
or delete the temporary table

Steps 2-3-4 → You can bypass the CSV file by copying the data yourself to the target DB.

Destructive step

It erases the target data and replaces it
with the import.

* The tables are AO_32F7CE_DBBACKUPITEM and AO_32F7CE_DBBACKUPMAPPING. Emptying those tables after working is important, since they contain an entire copy of the rest of the database.

Step details

Step 1 - Export the entities to a temporary table

Confluence	Jira

Requirement Yogi lets you choose:

The scope: It is possible to restrict the export to a single Confluence space. Restricting by Jira projects is not supported (The screenshot below doesn't contain the field 'Project key' anymore).
The backup key: In order to avoid collisions when 2 administrators are working on imports/exports, you can specify your own key. It doesn't matter what the key is, please keep it simple.

This step only copies all data into a single table, in preparation for the export. The table is as big as the entire database, therefore it will be important, later on, to empty this table.

None of those long-running task should be interrupted by, for example, a Confluence restart. If it happens, please restart the step.

Of course you can leave the page. If you lose the link, you can find the progress of the task in the "Queue" tab.

At the end, we suggest to write the file to disk. If you have lost the link to this progress screen, you can perform this operation using the main menu:

Step 2 - Write to disk

Note: This step can be skipped if you prefer (or need) to copy data manually. In such situation, please copy the contents of the tables AO_32F7CE_DBBACKUPITEM and AO_32F7CE_DBBACKUPMAPPING across to the target Confluence/Jira instance, and skip to step 5.

This step will write the contents of AO_32F7CE_DBBACKUPITEM and AO_32F7CE_DBBACKUPMAPPING into a CSV file. It can be a pretty large file, which may make you decide to copy the data manually.

The file will be written in the home directory of Confluence, i.e. you need disk access to complete this operation.

This step is not necessary if you are reimporting to the same instance.

Step 3 - Delete the temporary table

Click on the backup name in the main menu, and delete it. It is not necessary to keep it around.

Step 4 - Load the file on the new instance

The file will be read from a specific folder in the home directory of Confluence, i.e. you need disk access to complete this operation. The wizard will tell you where to place the file.

Step 5 - Perform the mappings

How to reach this step directly: In the main menu, click on the cog button next to a backup, and click "Do the mappings".

This step can be extremely menial.

For each entity (space, issue, project key, user...), you will have to provide the ID of the new entity in the target database, so that Requirement Yogi can "plug" itself to those entities. Fortunately, we provide an automatic mapping, which will lookup for the ID in the database, check that the title/details of the entity match the previous names, and/or it will attempt to find the new entities (such as the issue, based on the issue title). If an automatic mapping can be found, the status will be "MAPPED_AUTO". It is still necessary to check the value and eventually map manually.

Here are the possible statuses (matching the database column AO_32F7CE_DBBACKUPMAPPING.STATUS):

(null): Not mapped yet,
UNMAPPED: Couldn't find a match,
MAPPED_AUTO: Displayed as "MAPPED" + an exclamation mark "!" - It was guessed by the software,
MAPPED_MANUAL: Displayed as "MAPPED" + a green checkmark - It was either set manually by the user, or the user clicked the button "MAPPED" to confirm a "MAPPED_AUTO" value.

Once all entities are mapped, you can proceed to the next step.

Step 6 - Import

This step erases all data in the scope, then imports the data.

For example, if you are importing SPACE1 and have mapped SPACE1 into SPACE2, then it will erase all Requirement Yogi data related to that space, and apply the import.

Confluence must not restart during this import. If it does, reapply the import. If you close the page, it doesn't stop the process, and you can easily come back to monitoring the progress using the "Queue" tab (or reopening the previous URL).

Step 7 - Check the data

If you are unhappy with the import (wrong page ID, wrong issue ID), then please come back to the Mapping page (step 5), fix the problem, then reapply the import.

If you are happy with the import, then don't forget to delete the temporary data (delete from the UI) and the file on the disk.