Folder validation: technical annex

Introduction

The folder validation operation is an operation launched from folder management in the reference folder.
It can be launched in batch mode. Its aim is to perform the operations to bring the folder in a usable state, i.e. a state where connection and normal operations are possible on the corresponding endpoint.

The folder validation is a complex process that handles several cases:

• complete creation of the folder if it does not exist,
• structure changes of a folder that already exists, when:
• upgrading the reference folder,
• changing the activity codes configuration,
• setting up new languages,
• activating new modules,
• updating an history folder...

The creation of an history folder is done from the Creation of Purge folder function.

The process description has several steps, and these steps are described here.

Process steps

1 - Pre-requisite for history folders

If a history folder exists for a given folder, its revalidation is done in a second step, and can even be done much later (if for instance the environment where the history folder is installed remains as a stand-alone environment). But a history folder can only be migrated after migration of the folder it is attached to.

The history folder revalidation is only available for a migration that goes from version 150, version 6, version7, or upgrade 8 to upgrade 9 or later. Beware that this requires a minimum level of patches or maintenances depending on the version.

The following list gives the prerequisites:

2 - Preliminary controls

The folder validation first tests the parameter of the validation, with the following actions:

• If a history folder is validated, the reference folder is set to the associated exploitation folder.
• The reference folder is checked (it must be at the current version level).
• If the folder already exists, we are in a revalidation process, if not, we are in a validation process.
• The activity code values are controlled:
  - if an activity code is not set in the reference folder, it is deactivated,
  - in case of a revalidation, the list of activity codes that have been changed is updated,
  - the list of legislation activity codes updated is managed separately (some set-up updates are linked to legislations),
  - the evaluated activity code values (through dependency formulas) are computed,
  - the specific activity codes present are identified and checked.
• Additional functional controls are done (for instance, the value of some parameters).
• For history folder revalidation, check some pre-requisites (the view ADOVALHIS must be present in the reference folder: this is the case if the right preliminary patch has been installed on versions 150, 6, 7 and 8).
• The length of miscellaneous tables is checked (datatype ADI).
• The version of the engine is checked in order to ensure that optimized upgrade (migration plans) can be executed.

If some of the preliminary checks fail, the revalidation process will stop and the log file will give the reasons for this failure.

3 - Preliminary step of validation

If the folder is revalidated, the folder is first switched to mono-connection mode (no user can connect to the folder during the revalidation). This is of course only possible if no other user is already connected, otherwise the revalidation fails.

Flags defining if elements in dictionaries will have to be validated. By default, the revalidation is set to No except if we are in folder creation. In the next steps, the flag can be set to Yes by one of the upgrade procedures.

4 - Directory / Database creation

If the folder is created, a set of directories will be created in the application folder, and the scripts that create the database user with the corresponding tablespaces are run as well.

If the folder is revalidated, a check is done and the additional directories can be created (for instance when a language is added or if a new directory has been defined in the reference folder).

If the validated or revalidated folder is an history folder, the local dictionary elements (local table description) are created according to the data already stored, the dedicated menu files and text files used for the user interface are generated.

5 - Data transfer and purge

If the migration plans have been enabled:

• The list of temporary tables is set:
  - every module returns a list of tables to be purged,
  - some of them are temporary and can be purged without any problem (ALISTER, AWRKLOG, ATMPTRA for supervisor for instance, or PWRKSTT,PWRKORDERS for the purchase module),
  - some of them are purged but will be reconstructed later. This includes tables such as BALANCE, BALANA, GLCONSO for the accounting module.
• The list of movement tables is set as well.
• An entry point allows specific developments to add their own tables to the list.
• The temporary tables are purged.
• The movement tables are copied by direct SQL transfer to the dedicated U* migration tables.
• The movement tables are then purged.

6 - Table migration

The tables are now updated using the AMAJ routine located in the DOSMAJ scripts. The principles are the following:

• The indexes and the triggers are deleted.
• The columns to be added are added to the tables.
• The new tables added to the new version are created.
• The AMAJ routine in DOSMAJ* scripts are run, in the right order: for every release gap, a dedicated DOSMAJ* can run. For instance, to go from release X1 to Xn, the intermediate DOSMAJ that might exist to go from X1 to X2, from X2 to X3... from Xn-1 to Xn will run successively. This processes can, for instance, fill a new column with the content of an old column that might disappear at the next step, or fill a new table with the content of an old table that might disappear at the next step.

7 - Dictionary updates and structure changes

The table dictionary is now updated and the revalidation of all the modified tables will occur. This will drop the standard columns that are no longer present in the version and reindex the existing tables.

The movement tables transformation is also managed by this step. But if the migration plans have been used, the movement tables will be empty and the update will be performed later.

8 - Deliverables and kit management

If deliverables or kits have been added, they are copied from the reference folder (this step is not performed on a history folder).

9 - Dictionary update

When revalidating a normal folder, the dictionary content is updated for all the elements. This includes tables, views, actions, screens, objects, windows, data types, action parameters, functions, reports, activity codes, general parameters, miscellaneous tables, purge formulas, system transactions, inquiries, navigations, transactions, memo codes, type prototypes, global variables, constants, context variables, sub-programs, formula assistant contexts, URLs, documentations, data models, classes, representations, help keywords, entry points, business object related dictionaries.

The update takes into account the changes that have been made (especially new activity codes, new legislations).

The tables are now revalidated according to the new dictionary definition: columns that are no longer present are deleted, indexes are recomputed.

10 - Parameter updates and data copy

If we are updating a non history folder, depending on the changes in the setup, some data might be copied from the reference folder (for instance if a language code has been added, if new setups are delivered in a table and copied). The possible operations are the following:

• The new parameters created in the release are copied and initialized if necessary.
• The new miscellaneous tables are copied and initialized.
• The content of new tables that are delivered with automatic copy or conditional copy (if the copy has been set) is copied from the reference folder in the folder.
• The data related to kits that have been enabled is copied from the reference folder.

At this step, a link control procedure is executed on all the table except the movement tables if the migration plans are used.

11 - Dictionary generation

Still for non history folder on this step, the dictionary elements are validated:

• The views are generated.
• The local menu files are generated.
• The screens, objects, windows, inquiries are validated.
• The vocabulary elements are validated.

For history folder updates, only the tables and views are generated and validated (all the other dictionary elements are inherited from the main folder).

12 - Post upgrade data updates

If the we are in an upgrade process for non history folders, the MAJ routine in DOSMAJ* scripts are run, in the right order: for every release gap, a dedicated DOSMAJ* can run.
For instance, to go from release X1 to Xn, the intermediate DOSMAJ that might exist to go from X1 to X2, from X2 to X3... from Xn-1 to Xn will run successively.
This process can, for instance, perform additional updates. This step runs for both history folders and normal folders, but the scripts executed can of course be different.

13 - Classes and representation synchronization

If we are upgrading a non history folder, the classes and representation links are updated and the validation flag is set to trigger the validation later.

14 - Final dictionary operations

Still on a non history folder validation, the following operations are performed:

• The transactions are generated.
• If we create a mono-legislation folder, additional mass changes are done on some key linked to multi-legislation.
• If the folder has been created by the validation, default parameter values are filled according to the reference folder, default periods and fiscal year values are set.
• Menu files are generated from menu profiles.
• Triggers are generated on the tables.
• The update flags on the dictionary elements are reset.
• The BI parameter and code generation are done.
• The index files of compiled scripts are regenerated.
• Miscellaneous status flags are updated.
• The current version number is updated.
• Finally, the folder is switched back to multi-user mode.

15 - Additional step for migration if plans are selected

This step is relevant only if we are in an upgrade process and if the migration plans have been selected (otherwise, the folder validation is finished).

At this step, all the main parameters and dictionary tables are up to date, but all the movement tables are empty, and the corresponding data is located in the U* tables.

Running the migration plan steps will transfer the data by massive insertion into the final movement tables. This can be done with several steps in parallel, and is defined by the migration plan setup.

When finally this step is finished, the folder is again ready for normal operation. The U* table remain with the movement data image before the migration. They can be purged later (for instance after a given period of normal operation, to be able to analyze further data inconsistencies that were not detected).

 They need to be purged before the next upgrade will be done. This is done with the function TRTMIGDEL.

Additional comments regarding history folder upgrade

The following limitations apply to application table upgrade in history folders:

• Only the table having a standard history rule are migrated.
• The post-migration process (step 12 above) are not executed on history folder, except for balance resynchronization.

The consequence is that the following V6 fields will remain empty on the historized data:

• FIY and PER on BUD, GCOMMIT, SINVOICE (Sales/AR), PINVOICE (Purchase/AP) and PAYMENTH tables.
• UOM in GCOMMIT and BUD tables.

You should also remind that the accounting cumulated tables (BALANCE, BALANA, BLACMM, BLAQTY) are erased in V6. The migration procedure resynchronizes the balance table. If you want to use balance historized data, the GACCENTRY* and GCOMMIT* tables must also have been historized.