Data export and import

LUY supports two file formats for exporting/importing data and the meta model:

• XML metadata interchange (XMI) as standardized by the Object Management Group (OMG)

• proprietary LUY Excel format

The XMI format is particularly designed for interoperability with other modeling tools. The Excel format in turn is intended for human data maintenance activities. In Excel, sheets, columns, and rows can be adapted to fit the needs of the data maintenance activity at hand. In particular, bulk edits of a subset of the available data can be conducted using the Excel file format to export and re-import the updated data.

Exporting data and the meta model

The export function of LUY can be opened via “reports” and "export/import".

LUY supports two distinct modes of export:

• Datamodel export containing the structure of the data modeled in LUY, with data.

• Meta model export containing the structure of the data modeled in LUY, without any data.

Additionally, empty and pre-formatted rows can be added by enabling the “add empty rows” option.

Importing data

There are up to five steps when performing an import. The screen will guide you through the steps of the import. During these steps LUY performs different checks on the supplied data with respect to format, content, and consistency.

If multiple users make changes while the import process is running, unexpected behavior can occur.

A five-step process is used to make sure that the user always knows which changes are applied to the meta model or database of LUY. Therefore, the user must acknowledge each step by checking the information given by LUY and clicking "next" to advance.

The steps are:

1. Read file and check plausibility of the data: LUY checks whether the file is a correct data file, and whether it adheres to the LUY format definition. The checks especially ensure that the supplied data is internally consistent, i.e., that no "dangling" references or unknown values exist. If an inconsistency is detected, LUY reports the detected issue to the user and cancels importing.

2. Determine meta model changes: Before starting the import, the user has the option to decide whether changes of the meta model should be taken into consideration or not. By activating the option “change meta model”, LUY checks whether all properties described in the file are already present, or whether it is possible to add missing properties to the LUY meta model. LUY presents a list of missing properties and allows the user to decide whether to apply the necessary meta model changes, i.e., create the properties or cancel the import process.
   Please note that descriptions, sequencing, and coloring of properties cannot be imported and will be ignored.

3. Write meta model changes: LUY creates any missing properties. The newly created properties are not assigned to attribute groups but are created in the default attribute group.

4. Validate data: LUY compares the data set to import with the data already present in LUY. Different checks are applied to detect any inconsistencies, i.e. possible name duplicates. The result of this comparison is presented to the user who can then decide to proceed with the next steps or to cancel the import.
   Canceling the import at this point does not roll-back the meta model changes applied in the previous step.
   

5. Write data: LUY applies the data changes identified in the preceding step to the database.

Step 5 may take longer to complete, especially for larger data sets.

Import strategies

The data in the imported file can be interpreted in different ways. Any changes will be applied to the LUY data depending on the selected import strategy.

One import strategy must be selected before a file can be imported.
As default, an additive import strategy is selected.

Two distinct cases must be considered when updating attribute values:

1. Single value assignment attribute: The importer can change the value of the attribute to another value or set the value if it was previously empty. It cannot remove the attribute value to go back to an empty state. If the according value field in the file to import is empty, it will simply be ignored.

2. Multi value assignment attributes: The importer can only add additional values to the list of current values or leave it unchanged. If, for example, the responsibility attribute "accountability" for information system "BI" is already assigned to "Alice" and "Bob" importing a file with "accountability" set to "Alice" and "Max" does not remove the entry "Bob", but simply adds "Max" to the list. Result: "Alice", "Bob" and "Max".

There are also two distinct cases for consideration when updating relations:

1. To-one relations: These are relations which are represented by a column on the sheet of the building block type they belong to, for example, the parent relation in an Excel template. Each building block can have up to one parent.

Relation assignments such as "parent" in information system or assigned information systems in capabilities will be updated to represent the state described in the imported file. This means that connections existing in LUY, but missing in the imported file, will be removed when importing.

However, both changing an assigned building block in such a relation to another building block, as well as the removal of the assignment will only be performed by the import, if all involved building blocks are listed in the import file. This means not only the changed building block and the newly assigned one need to be present in the import file, but also the formerly assigned building block.

2. To-many relations: These are relations in which a building block can be connected to a list of several other building blocks by the same relation, for example, the "based on these technical components"-relation of information systems. The handling of such relations by the importer is similar to the handling of multi-value attributes described above: The importer can only add additional connections between building blocks, not remove existing ones. If the data contains a connection between information system A and technical components B and C, but the Excel file to import only contains a connection between A and another technical component D, rather than replacing the existing connections A-B and A-C, the importer will add the connection A-D to the already existing A-B and A-C.

There is one case where the connection between two building blocks can be removed by the importer.

Example: There are information systems A, B and C in the data. A is the parent of B. The importer can change B's parent to C, which also results in B being removed from A's children.

Additive import strategy

The additive import strategy is designed to add data to LUY. Data can either be updated or inserted but not deleted.

See the limitations section below.

Conservative strategy

The conservative strategy is made for updating data in LUY without deleting building blocks.

• Even if an existing building block is not present in the imported file, it will not be deleted.

• Existing building blocks will be updated so that their new state is as described in the imported file.

  • Relation assignments such as "parent" in information system or assigned information systems in capabilities will be updated to represent the state described in the imported file. This means that connections existing in LUY, but missing in the imported file, will be removed when importing. However, connections between two building blocks will only be removed in this way, if the formerly connected building blocks are all existent in the imported file.

  • Attribute values will be set as they are in the imported file. Assignments of attribute values not mentioned in the imported file will be removed in LUY.

• Building blocks described in the imported file, which are not present in LUY, will be created.

See the limitations section below.

Overwrite strategy

The overwrite strategy is made for updating data in LUY, but also for adding new building blocks or deleting existing ones.

• If an existing building block is not present in the imported file, it will be deleted

• Existing building blocks will be updated so that their new state is as described in the imported file.

  • Relation assignments such as "parent" in information system or assigned information systems in capabilities will be updated to represent the state described in the imported file. This means that connections existing in LUY, but missing in the imported file, will be removed when importing.

  • Attribute values will be set as they are in the imported file. Assignments of attribute values not mentioned in the imported file will be removed in LUY.

• Building blocks described in the imported file, which are not present in LUY, will be created. 

If an existing element is imported with its ID removed, the import interprets this as two actions:

• delete the element with that ID (because there is no element with the ID in question in the file anymore) and

• create a new element with the same name as the existing element.

This is typically not intended and does not work, hence the import will fail in the last step. The reason for this is that the connections to the element cannot be resolved properly, they are ambiguous for the two elements in question with the same name.

Deletion of building blocks with ordered children

For some building block types, the order of an element's children can be defined explicitly. If a building block with ordered siblings is deleted in the import, the position of the siblings must be updated, and the affected building blocks are changed. For example, if the first sibling among 3 IT services is deleted, the result message is:

Following changes were applied:
IT Services changed: 2
IT Services deleted: 1
... other changes

See the limitations section below.

Tabular overview of Import Strategies

Exceptions to updates and deletions

• When importing a previously exported (and possibly modified) file, building blocks will only be deleted or updated if they were not changed since the time of the export.

The timestamp of the export has no relevance for the additive strategy.

• Removal of assigned attribute values or building block connections will only be done if the according attribute or relation as such is mentioned in the imported file. If, for example, an attribute column and relation sheet in an Excel file are removed completely, the import leaves values of that attribute and assignments in that relation untouched in LUY.

• Using the overwrite strategy, it is not possible to delete a building block and create a building block with the same name in one import. This situation is detected by the import process at stage 5, preserving the data integrity. Split the single import in two parts, first deleting the building block, and secondly creating the new building block.

Additional information/technical limitations

Updating element hierarchy

It is not recommended to mix updates to the elements’ hierarchy with other updates, like renaming or adding new elements.

Furthermore, a switch of hierarchies, i.e. A is parent of B to B is parent of A, cannot be achieved by a single import run, even though there may not be an error shown in the validation step (step 4). This is also true if B is a more distant descendant of A.

Workaround:

The tree view of the corresponding building block type can be used to switch hierarchies. Furthermore, it is possible to perform two separate import runs, i.e. the first run to remove B from parent A and the second run to attach it otherwise.

Limitations

• Due to technical constraints, there is currently no way to create attributes of the type "responsibility attribute" by adding the attribute to the excel file to be imported. When exporting landscape data or downloading the template, each responsibility attribute will be exported as text attribute. Importing data like this into LUY where the according responsibility attribute already exists works correctly, but if the attribute does not exist already the attribute type and its values will be ignored. The attribute is not added to the building block type and the attribute values are not imported.

• It is also not possible to create attributes of the type "date interval"

• Adding a new building block type or relation by adding an according sheet or column to the excel file is not possible and the attempt will result in an according error message.

• It is not possible to change whether an attribute is mandatory or not with the excel import.

• The ordering of the children of a hierarchical element may be lost after importing.

• The dates for date intervals or date attributes must be valid excel date values. Valid date values are all dates after January 1, 1900.

• When editing the attributes of a building block or association, be careful to avoid accidentally removing the ID value from the Excel row. If the ID value is removed, LUY tries to both delete the old element and create an identical one at the same time. This leads to an error during the fifth import step if using the strategies "conservative" or "overwrite". If the strategy "additive" is used, a more graceful error occurs in the fourth step.