Automated migration procedures

Main migration stages

In case of migration of a folder from one major release to another, the following stages must be complied with:

• Controls prior to the migration in the original folder. These controls can be carried out without stopping the Sage Warehousing operations.
• The installation of Version 11.0.1 on a dedicated environment or the update of an existing environment.
• The extraction of the data to be migrated from the original folder, followed by the reintegration of these data (the HRM operation must be stopped and started again when migration is completed). Data extraction will be carried in a "flat" format for small folders, and using database extraction tools for larger folders.
• The re-validation of the folder in the environment of Version 11.0.1. This revalidation can be sub-divided into several stages, using the migration plan setup. If the folder is not too bulky, defining migration plans may not prove necessary. It becomes necessary if transfer processing routines need to be run in parallel (to benefit from the power of a multi-processor server) and their sequencing needs to be checked using resumption marks.
• Post-migration controls. This mainly relates to setup checking and adaptation.

The rest of the documentation describes all the steps.

Post-migration stages

Post-migration stages are launched in the source environment and they can be spread over a period of time, before the actual migration. In effect, they do not require the HRM operation to be stopped.

They are sub-divided as follows:

• First check that the original folder is at the minimum patch level required to run a migration. This level, which can be displayed via the ? menu > About, must be at least:
• patch 6 for a version 6.

Copy of data and adjustment of the folder parameters

This stage corresponds to the beginning of the transfer to the next version. At this stage, the operation of the solution must be stopped: it will resume when the migration is completed.

This stage is sub-divided in 4 phases:

• extraction of the data from the folder to be migrated,
• extraction of data in a "flat" format for small folders. To do so, run theextraction function used to create files in the default SVG backup directory.
• extraction of databases for larger volumes (the format depends on the database and tool used),
• copy of the files of the folder to be migrated to the V11.0.1 solution,
• creation of a copy folder of the folder tree structure in the new environment of Version 11.0.1 and import of the exported data into the new environment,
• creation of the folder record in the V11.0.1 environment. If the import is carried out via the console, the folder record is created automatically. On the other hand, if the import has been carried out via database tools, the folder record will have to be created manually after the import is completed.

You can combine these stages into a single one. To do so, use theremote import wizard available in the SAFE X3 configuration console.

If the standard process has to be relaunched from the data import in the migration folder, the TRTMIGDEL processing must be launched to clear out the tables where the migration report is stored.

Before revalidating folders, the value of activity codes must always be adjusted in the folder record. The setups are thus in compliance with the license, for introducing, for instance, new features controlled by new activity codes.The modification of the folder record, or its creation, will be carried out by logging onto the Supervisorfolder.
New validation of folders cannot be triggered without first applying this update. An error message is displayed as a reminder upon launching the validation.

Folder revalidation

Revalidating the folder changes the structure of the X3 dictionary, then of setup tables, and then data tables of the previous version in order to bring the structure up to the 9.0.0 level while transferring the data. It is triggered using the Validation button from the folder management.

It performs the supervisor and functional migration of the folder, by running a comparison between the dictionary of the new version and the dictionary of the version to migrate.

 If a migration test is first carried out in a folder containing specific developments, and if this migration can potentially stop if an error occurs, the default size of global variable GTRALIG to 200 (for optimization reasons) does not make it possible to get the set of all previous operations carried out before the system stopped.
It is necessary to, temporarily, modify the size formula of the GTRALIG global variable with the value 1.The value 200 must be entered again after the migration.
Careful, you must log out and log in again for this value to be taken into account.

Main revalidation stages

The folder revalidation triggers the following stages in the folder to migrate:

• Purge of specific temporary or totaling tables whose re-creation will be carried out during later stages. The supervisor tables impacted by this purging are the following:
  ALSTRD
  ALISTER
  AWRKLOG
  AWRKLOGIND
  AWRKLOGMES
  ATMPTRA
  
• Migration of the envelope, that is to say, the database dictionary and the supervisor tables necessary for the operating of the database (connection, user management, development environment and minimum setup). Once this migration has been carried out, it is theoretically possible to log on, even though the tables containing the flow application data are still empty.
• Functional migration: whenever there is a change in the structure of a table, the procedure will carry out the necessary changes, eventually by assigning appropriate default values, through procedures that are sequenced in stages and phases. This is the lengthy part of the migration, the part that depends on the volume of data to migrate.
• Post-migration process (they are essentially resynchronizations of the totals tables). Some of these steps can be postponed and the operations can resume but the software will be functioning in a downgraded mode during the intermediate phase. To consider the migration as completed, these stages must have been carried out.
• Final deletion of temporary tables. This is a manual phase. It can be postponed, for safety reasons, as long as there is no absolute certainty that all the flow data has been correctly migrated.

Use of the migration plans

This functional migration can be lengthy when the folder is very large. Moreover, some updated tables are potentially independent and, therefore, could be migrated in parallel, thus benefiting from multi-processor architectures to accelerate this phase.

To allow this migration to be scheduled in the best way, you can define a custom migration plan BEFORE launching the data validation. This migration plan is described in the migration procedure function (called from the Supervisor folder). This function is used to define the stages, phases and procedures of the functional and supervisor migration. A migration plan corresponds to the definition of specific migration parameters (impacted folder, number of procedures that can be run in parallel, task sequencing policy, running status). A migration plan is created by recopying all the active elements of the migration procedure. This complete set of procedures is provided as a standard. It allows all the processing routines carried out by the standard software during the migration phase to be run exhaustively.

It is possible, at this stage, to define specific procedures by writing complementary processing routines in conformity with the methodology defined in the following annex. These specific procedures will be inserted in a relevant way between the standard procedures.

Once a migration plan has been defined, you can launch it, interrupt it momentarily, resume its execution, and view its progress.

When the volume of a folder is not too large and its migration does not require any particular planning, it is not necessary to create a migration plan. In effect, when there is no migration plan, the validation operation of the folder will automatically create a migration plan. The code of this migration plan will be the folder code, except if this code already corresponds to a migration plan that does not have the Pending status. If it were the case, the plan would be created with a predefined code under the form MIGmmddM##. mmandddare the numbers representing the month and the day of the launching, ## is a sequential number.

However, to customize the migration sequencing options, you can create a migration plan with a code that must correspond to the folder name. This creation is performed in the Supervisor folder before launching the folder validation. If a migration plan exists, with the Pending status, it will be used for the functional migration of the folder.

A plan created with a name that is different from the folder to migrate will never be used by the automatic folder validation functions. Such a plan is restricted to manual launching.

Operation process in a migration plan

A migration plan is characterized by a folder code and 4 parameters:

• a title,
• the number of procedures likely to be launched simultaneously. This number is a possible maximum. Only procedures that can be run in parallel will be launched simultaneously. Caution, this number is limited by the number of batch tasks likely to be run simultaneously (defined in the batch server setup function). This number is also limited by the user license,
• an indicator specifying whether the procedures must be launched automatically and follow the logic defined by the plan. If not, the sequencing of the procedures described in the plan will be done manually,
• an indicator stating if the post-migration procedures must be linked to the migration itself. These operations are detailed below.

If the migration plan is created by default upon folder revalidation, it is created with the following values:

• Number of simultaneous taskshas the value of the NBRTACSIM parameter or, if it does not exist, it has the value 2,
• Automatic sequencing has the value Yes,
• Post-migration operations has the value Yes.

You can always modify these values from the migration plan control function.

You can view the ordered list of the plan procedures in the screen associated with the migration plan. Specific buttons make it possible to globally control the running of the plan:

• by launching its execution,
• by momentarily interrupting the launching of new procedures,
• by interrupting all the procedures in progress,
• by relaunching its execution.

Each line of the plan materializes a step in the execution, characterized by:

• the code of the procedure and its title,
• the step and phase to which the procedure belongs
• the functional module to which the procedure is linked.
• the execution rank.

The migration steps make it possible to split the functional migration. If a procedure linked to a given phase is not completed, the following phases cannot be launched. The phases are linked to the following steps:

• the initialization step is a preliminary step that is the very first to be completed. It has not been used so far. It enables specific procedures to be executed before the migration itself,
• the common data step allows data used by the following steps to be migrated,
• the module step is used to carry out migrations linked to each functional module. An operation belonging to this step is associated with a functional module: this association does not have any binding effect on the execution. For example, some data migrations linked to different modules can be run in parallel if they belong to the same phase. Their launching will follow the order of the modules, then of the execution ranks,
• the post-migration step is used to carry out synchronization operations or secondary updates whose execution is not mandatory to restart operations in a downgraded mode. It can be used, for example, to postpone the migration of old data. Please note nonetheless that even though this step is not mandatory to restart operations, it must be carried out later on for a folder to be considered as completely migrated. These post-migration operations are launched by default in a migration plan but they are optional. They are listed in a dedicated paragraphin the annex describing the migration procedures.

In one step, you will find the unitary procedures organized in phases and ranks.

• It is considered possible for two procedures defined in the same phase to be run simultaneously. As a consequence they must be independent. The assignment of a standard procedure to a given phase cannot be changed. As long as all the procedures belonging to a phase are not completed, the following phase cannot be launched.
• The rank order is purely a preferential launching order, but it can be modified, including for standard procedures, when the plan is defined.

Final purge of the entry tables

This phase is manual It is triggered by the execution of the TRTMIGDEL processing from the processing execution function. Running this phase will delete all the tables having the MIG activity code.

This phase cannot be undone. Please ensure that the migration processing routines have been carried out successfully.

Keeping the temporary tables in the folder is compatible with the resumption of normal operations. It is therefore possible to keep these tables on-line for a few weeks of operation. Should any problem occur in the weeks following the migration, you will have access to the original data for comparison or analysis purposes.

Example of sequencing on flow migration

Let us consider the following operations:

• in the initialization step, 7 operations:
• IN1, IN2 in phase 1 (ranks 3 and 7),
• IN3 and IN4, IN5 and IN6 in phase 2 (and successive ranks 1 3, 5 9),
• IN7 in phase 3,
• in the common data (TC) step, 6 operations:
• TC1, TC2, TC3, all 3 in phase 1, with ranks 3, 6, 9,
• TC4 in phase 2,
• TC5 and TC6 in phase 3,
• in the Module step, 5 operations:
• step MO1, in the Financials module, with phase 2 and rank 1,
• steps MO2 and MO3, in the Stock module, in phases 1 and 2 respectively, with ranks 1 and 2,
• steps MO4 and MO5, in the Sales module, with phases 1 and 2 respectively, with ranks 1,
• steps MO6 and MO7, in the Purchasing module, with phases 1 and 5 respectively, with ranks 1,

Let us assume that the migration plan is launched with a sequencing of tasks, and a maximum number of 2 procedures running simultaneously. The sequencing cab be the following:

• Procedures IN1 and IN2 are launched in this order,
• Let us assume that procedure IN2 is completed first. As long as procedure IN1 is not completed, no other procedure can be launched, because the following procedures are included in a phase or a step with a higher rank,
• when procedure IN1 is over, IN3 and IN4 procedures are launched in this order.
• If procedure IN4 is completed before procedure IN3, procedure IN5 will be launched. If it is completed before procedure IN3 is over, procedure IN6 will be launched and will be running at the same time as procedure IN3, which will continue to take place,
• when procedures IN3 and IN6 are both completed (and only then), the initialization step will be over.
• The procedures of the first phase of the Common Data step can then be launched: first of all, procedures TC1 and TC2, then TC3, which will be launched the moment that one of the two previous procedures have been completed. All three procedures TC1, TC2, TC3 must be completed before launching TC4, which will be running on its own since there are no other procedure in this phase. Then, TC5 and TC6 will be launched, and running in parallel.
• When procedures TC5 and TC6 are both completed, tasks belonging to the module step can be started,
• First, procedures MO2 and MO6 will be launched (in this order, these procedures run in parallel); then, as soon as one of these two procedures is completed, procedure MO4, the next one in the phase/module/rank order, will be launched in parallel with MO2 or MO6, depending on which one of them is not completed,
• then, all former procedures must be completed before procedures MO1 and MO3, belonging to phase 2, can be launched. When one of them, at least, is completed, MO5 can be launched. Then, all former tasks must be completed,
• MO7 can then be launched, as the only procedure in phase 5.

Verification further to the migration

As soon as the folder is revalidated, it can be accessed without use restriction (there can be some level of restriction if some post-migration operations have been postponed).

The only remaining task in to check and adapt some functional setups.