Automated migration procedures

Main steps of the migration

In case of migration of a folder from one major release to another, the following stages must be complied with:

• Controls prior to the migration in the original folder. These controls can be carried out without stopping the operations of Sage X3.
• Installation of version 8.0.0 in a dedicated environment.
• The extraction of the data to be migrated from the original folder, followed by the reintegration of these data (the HRM operation must be stopped and started again when migration is completed). Data extraction will be carried in a "flat" format for small folders, and using database extraction tools for larger folders.
• New validation of the folder in the environment of update 8.0.0. This revalidation can be sub-divided into several stages, using the migration plan setup. If the folder is not too bulky, defining migration plans may not prove necessary. It becomes necessary if transfer processing routines need to be run in parallel (to benefit from the power of a multi-processor server) and their sequencing needs to be checked using resumption marks.
• Postmigration controls. This mainly relates to setup checking and adaptation.

The rest of the documentation describes all the steps.

Premigration steps

Premigration steps are launched in the source environment and they can be spread over a period of time, before the actual migration. In effect, they do not require the HRM operation to be stopped.

They are sub-divided as follows:

• first, it is necessary to check that the original folder is at the minimum patch level required to run a migration. This level, which can be displayed via the ? menu > About, must be at least:
• patch 21 for a version 6,
• patch 5 for a version 7,
• then, the functional prerequisites necessary for the migration processing must be checked. These prerequisites are described in the dedicated documentation.
• Then, automated procedures can be triggered in the original folder, via processing routines with names beginning with UU. These automated procedures can be launched using the dedicated function. These processing routines are supplied upon request, according to the release number, as a specific patch list dedicated to migration. Two types of procedures are delivered:
• first, control procedures of the data contained in the original folder. The result of this control can reveal errors or inconsistencies which needs to be corrected before launching the migration itself. This control phase can be anticipated several weeks ahead of the migration, in order to have enough time to correct the data. It will nevertheless be necessary to relaunch it to check that everything is correct just before the transition to the next version. These procedures are listed in the corresponding documentation.
• then, optional loading procedures, that revalidate some tables by adding additional fields, then by populating them. Since these procedures are independent from one another, they can be planned over several weeks prior to migration (the operation can resume as soon as the fields have been added). The purpose of these procedures is to significantly accelerate the actual migration stages. However, not launching these procedures does not jeopardize the execution of the migration procedures. These procedures are listed in the corresponding documentation.

Migration plan

The preparation of this environment is carried out:

• by installing the update 8.0.0 solution, then integrating all existing patch lists to the X3 folder,
• or by upgrading an existing V7 solution, using the standard process described in the documentation Quick upgrade guide.

Operation process in a migration plan

This stage corresponds to the beginning of the transfer to the next version. At this stage, the operation of the solution must be stopped: it will resume when the migration is completed.

This stage is sub-divided in 4 phases:

• extraction of the data from the folder to be migrated,
• extraction of data in a "flat" format for small folders. The extraction function is used for that purpose. It creates the files in the default SVG directory.
• extraction of databases for larger volumes (the format depends on the database and tool used),
• copy of the folder files to be migrated to the update 8.0.0 solution,
• creation of a copy folder for the folder tree structure in the new environment of version 8.0.0, and import of the exported data into the new environment,
• creation of the folder record in the update 8.0.0 envrionment. If the import is carried out via the console, the folder record is created automatically. On the other hand, if the import has been carried out via database tools, the folder record will have to be created manually after the import is completed.

 These various stages can be combined into a single one. For that purpose, it is necessary to use theremote import wizard available in the SAFE X3 configuration console.

If the standard process has to be relaunched from the data import in the migration folder, the TRTMIGDEL processing must be launched to clear the tables storing the migration report.

Before revalidating folders, the value of activity codes must always be adjusted in the folder record.The setups are thus in compliance with license 8.0.0, for introducing, for instance, new features controlled by new activity codes.The modification of the folder record, or its creation, will be carried out by logging to the "Supervisor" folder.
New validation of folders cannot triggered without this update.An error message is displayed as a reminder upon launching the validation.

Final purge of the entry tables

Revalidating the folder changes the structure of the X3 dictionary, then of setup tables, and then data tables of the previous version in order to bring the structure up to the 8.0.0 level while transferring the data. It is triggered via the Validation button from the folder management.

It performs the supervisor and functional migration of the folder, by running a comparison between the dictionary of the new version and the dictionary of the version to migrate.

 If a migration test is first carried out in a folder containing specific developments, and if this migration can potentially stop if an error occurs, the default size of global variable GTRALIG to 200 (for optimization reasons) does not make it possible to get the set of all previous operations carried out before the system stopped.
It is necessary to, temporarily, modify the size formula of the GTRALIG global variable with the value 1.The value 200 must be entered again after the migration.
Careful, you must log out and log in again for this value to be taken into account.

Main revalidation steps

The folder revalidation triggers the following stages in the folder to migrate:

• purge of some temporary or totaling tables whose re-creation will be carried out during later stages. The supervisor tables impacted by this purging are the following:
      ALSTRD
      ALISTER
      AWRKLOG
      AWRKLOGIND
      AWRKLOGMES
      ATMPTRA
  Other temporary functional tables are also purged (in the case of Sage X3, these tables are: LABELPRN, PRTSCRWRK, BALANCE, BALANA, GLCONSO, BALCONSO, GAJOUSTA, FUP, FUPTOT, BATCH, TMPEXPENSE).
• migration of the envelope, that is to say, the database dictionary and the supervisor tables necessary for the operating of the database (connection, user management, development environment and minimum setup). Once this migration has been carried out, it is theoretically possible to log on, even though the tables containing the flow application data are still empty.
• functional migration: whenever there is a change in the structure of a table, the procedure will carry out the necessary changes, eventually by assigning appropriate default values, through procedures that are sequenced in stages and phases. This is the lengthy part of the migration, the part that depends on the volume of data to migrate.
• postmigration process (they are essentially resynchronizations of the totals tables). Some of these steps can be postponed and the operations can resume but the software will be functioning in a downgraded mode during the intermediate phase. To consider the migration as completed, these stages must have been carried out.
• final deletion of temporary tables. This is a manual phase. It can be postponed, for safety reasons, as long as there is no absolute certainty that all the the flow data has been correctly migrated.

Use of migration plans

This functional migration can be lengthy when the folder is very large. Moreover, some updated tables are potentially independent and, therefore, could be migrated in parallel, thus benefiting from multi-processor architectures to accelerate this phase.

To allow this migration to be scheduled in the best way, it is possible to define a customized migration plan before launching the data validation. This migration plan is described in the migration procedure function (called from the Supervisor folder). This function is used to define the stages, phases and procedures of the functional and supervisor migration. A migration plan corresponds to the definition of specific migration parameters (impacted folder, number of procedures that can be run in parallel, task sequencing policy, running status). A migration plan is created by recopying all the active elements of the migration procedure. This complete set of procedures is provided as a standard. It allows all the processing routines carried out by the standard software during the migration phase to be run exhaustively. A short description of these procedures is provided in a dedicated documentation.

It is possible, at this stage, to define specific procedures by writing complementary processing routines in conformity with the methodology defined in the following annex. These specific procedures will be inserted in a relevant way between the standard procedures.

Once a migration plan has been defined,it is possible to launch it, interrupt it momentarily, resume its execution, and view its progress status.

When the volume of a folder is not too large and its migration does not require any particular planning, it is not necessary to create a migration plan. In effect, when there is no migration plan, the validation operation of the folder will automatically create a migration plan. The code of this migration plan will be the folder code, except if this code already corresponds to a migration plan that does not have the Pending status. If it were the case, the plan would be created with a predefined code under the form MIGmmddM##. mmandddare the numbers representing the month and the day of the launching, ## is a sequential number.

However, to customize the migration sequencing options, it is possible to create a migration plan with a code that must correspond to the folder name. This creation will take place in the supervisor folder before launching the folder validation. If a migration plan exists, with the Pending status, it will be used for the functional migration of the folder.

 A plan created with a name that is different from the folder to migrate will never be used by the automatic folder validation functions. Such a plan is restricted to a manual launching.

Operation process in a migration plan

A migration plan is characterized by a folder code and 4 parameters:

• a title,
• the number of procedures likely to be launched simultaneously. This number is a possible maximum. Only procedures that can be run in parallel will be launched simultaneously. Caution, this number is limited by the number of batch tasks likely to be run simultaneously (defined in the batch server setup function). This number is also limited by the user license,
• an indicator specifying whether the procedures must be launched automatically and follow the logic defined by the plan. If not, the sequencing of the procedures described in the plan will occur manually,
• an indicator stating if the postmigration procedures must be linked to the migration itself. These operations are detailed below.

If the migration plan is created by default upon folder revalidation, it is created with the following values:

• Number of simultaneous taskshas the value of the NBRTACSIM parameter or, if it does not exist, it has the value 2,
• Automatic sequencing has the value Yes,
• Postmigration operations has the value Yes.

It will always be possible for the user to modify these values from the migration plan control function.

The ordered list of the plan procedures is displayed in the screen associated with the migration plan. Specific buttons make it possible to globally control the running of the plan:

• by launching its execution,
• by momentarily interrupting the launching of new procedures,
• by interrupting all the procedures in progress,
• by relaunching its execution.

Each line of the plan materializes a step in the execution, characterized by:

• the code of the procedure and its title,
• the step and phase to which the procedure belongs
• the functional module to which the procedure is linked.
• the execution rank.

The migration steps make it possible to split the functional migration. If a procedure linked to a given phase is not completed, the following phases cannot be launched. The phases are linked to the following steps:

• the initialization step is a preliminary step that is the very first to be completed. It has not been used so far. It enables specific procedures to be executed before the migration itself,
• the common data step allows data used by the following steps to be migrated,
• the module step is used to to carry out migrations linked to each functional module. An operation belonging to this step is associated with a functional module: this association does not have any binding effect on the execution. For example, some data migrations linked to different modules can be run in parallel if they belong to the same phase. Their launching will follow the order of the modules, then of the execution ranks,
• the postmigration step is used to carry out synchronization operations or secondary updates whose execution is not mandatory to restart operations in a downgraded mode. It can be used, for example, to postpone the migration of old data. Please note nonetheless that even though this step is not mandatory to restart operations, it must be carried out later on for a folder to be considered as completely migrated. These postmigration operations are launched by default in a migration plan but they are optional. They are listed in a dedicated paragraphin the annex describing the migration procedures.

One step includes unitary procedures that are organized in phases and ranks.

• It is considered possible for two procedures defined in the same phase to be run simultaneously. As a consequence they must be independent. The assignment of a standard procedure to a given phase cannot be changed. As long as all the procedures belonging to a phase are not completed, the following phase cannot be launched.
• The rank order is purely a preferential launching order, but it can be modified, including for standard procedures, when the plan is defined.

Final purge of the entry tables

This phase is manual It is triggered by the execution of the TRTMIGDEL processing from the processing execution function. Running this phase will delete all the tables having the MIG activity code.

This phase cannot be reversed.Please ensure that the migration processing routines have been carried out successfully.

Keeping the temporary tables in the folder is compatible with the resumption of normal operations. It is therefore possible to keep these tables on-line for a few weeks of operation. Should any problem occur in the weeks following the migration, it will be possible to have the original data available for comparison or analysis purposes.

Example of sequencing on flow migration

Let us consider the following operations:

• in the initialization step, 7 operations:
• IN1, IN2 in phase 1 (ranks 3 and 7),
• IN3 and IN4, IN5 and IN6 in phase 2 (and successive ranks 1 3, 5 9),
• IN7 in phase 3,
• in the common data (TC) step, 6 operations:
• TC1, TC2, TC3, all 3 in phase 1, with ranks 3, 6, 9,
• TC4 in phase 2,
• TC5 and TC6 in phase 3,
• in the Module step, 5 operations:
• step MO1, in the Financials module, with phase 2 and rank 1,
• steps MO2 and MO3, in the Stock module, in phases 1 and 2 respectively, with ranks 1 and 2,
• steps MO4 and MO5, in the Sales module, with phases 1 and 2 respectively, with ranks 1,
• steps MO6 and MO7, in the Purchasing module, with phases 1 and 5 respectively, with ranks 1,

Let us assume that the migration plan is launched with a sequencing of tasks, and a maximum number of 2 procedures running simultaneously. The sequencing cab be the following:

• Procedures IN1 and IN2 are launched in this order,
• Let us assume that procedure IN2 is completed first. As long as procedure IN1 is not completed, no other procedure can be launched, because the following procedures are included in a phase or a step with a higher rank,
• when procedure IN1 is over, IN3 and IN4 procedures are launched in this order.
• If procedure IN4 is completed before procedure IN3, procedure IN5 will be launched. If it is completed before procedure IN3 is over, procedure IN6 will be launched and will be running at the same time as procedure IN3, which will continue to take place,
• when procedures IN3 and IN6 are both completed (and only then), the initialization step will be over.
• The procedures of the first phase of the Common Data step can then be launched: first of all, procedures TC1 and TC2, then TC3, which will be launched the moment that one of the two previous procedures have been completed. All three procedures TC1, TC2, TC3 must be completed before launching TC4, which will be running on its own since there are no other procedure in this phase. Then, TC5 and TC6 will be launched, and running in parallel.
• When procedures TC5 and TC6 are both completed, tasks belonging to the module step can be started,
• First, procedures MO2 and MO6 will be launched (in this order, these procedures run in parallel); then, as soon as one of these two procedures is completed, procedure MO4, the next one in the phase/module/rank order, will be launched in parallel with MO2 or MO6, depending on which one of them is not completed,
• then, all former procedures must be completed before procedures MO1 and MO3, belonging to phase 2, can be launched. When one of them, at least, is completed, MO5 can be launched. Then, all former tasks must be completed,
• MO7 can then be launched, as the only procedure in phase 5.

Example of sequencing on flow migration

As soon as the folder is revalidated, it can be accessed without use restriction (there can be some level of restriction if some postmigration operations have been postponed).

The only remaining task in to check and adapt some functional setups. This aspect is described in the documentation on the functional postrequisites.