changed:
-
Data Migration Tools Project Summary
====================================
The goal of this project is to propose a framework for data migration (importing legacy data into Mifos) that includes a well defined data import format and to implement the import of one business object type (the Mifos Center type) as a proof of concept. This project is a first step towards the Desired Feature item Improved Tools For Data Migration.
Project Artifacts
+++++++++++++++++
* `Code, schemas, example XML <https://mifos.dev.java.net/source/browse/mifos/trunk/mifos/src/org/mifos/migration/ mifos/src/org/mifos/migration>`_
* `Test code: <https://mifos.dev.java.net/source/browse/mifos/trunk/mifos/test/org/mifos/migration/ mifos/test/org/mifos/migration>`_
* `Build file: <https://mifos.dev.java.net/source/browse/mifos/trunk/mifos/build-migration.xml mifos/build-migration.xml>`_
* Design decision documents
* (`Choice of XML for import format <DataMigrationImportTypeDesignDecision>`_)
* (`Choice of RELAX NG and W3C XML Schema support and use of JAXB <DataMigrationSchemaAndToolChoices>`_)
* (`XML schema design notes <MifosDataExchangeSchema>`_)
* Background information documents
* `Mifos specialist discussion of Grameen Koota data migration effort <DataMigrationMFTechDiscussion>`_
* [Data Migration Roadmap]
Final Project Status - 10 April 2007
++++++++++++++++++++++++++++++++++++
Work on this project (implementing a proof of concept) has been completed. A project proposal for a full implementation has been submitted.
Work completed
++++++++++++++
Week ending 13 April 2007
-------------------------
* prepared project plan for a complete Mifos Data Exchange XML import implementation
* wrote a sketch of a data migration framework using data import-- the Data Migration Roadmap
* performed a quick survey of open source ETL tools
* final checkin of project code
Week ending 6 April 2007
------------------------
* completed proof of concept import functionality for Mifos Center objects
* worked on project planning and estimation for a complete Mifos Data Exchange XML import implementation
* studied Mifos codebase to understand potential issues with a complete Mifos Data Exchange XML import implementation
* generalized import code to handle custom field entries
* revised schemas to handle custom field entries
* investigated XML comparison libraries to handle validating equality of unordered elements
* had a discussion with Terry Wong regarding prior work on a Mifos API and its relation to data migration
Week ending 30 Mar 2007
-----------------------
* initial check-in of schemas and code for reading and validating Mifos Data Exchange XML
* completed a first pass at persisting imported Mifos Center objects into the database and did a check-in
* continued refining the XML schema
* had a discussion with Arun Kanabar from MFTech about their experience as the Mifos specialist doing data migration for Grameen Koota (see [Data Migration MFTech
Discussion])
Week ending 23 Mar 2007
-----------------------
* preliminary decision taken to use RELAX NG (RNG) as the primary XML schema language for data migration and also support W3C XML Schema (WXS) (see [Data Migration Schema And Tool Choices])
* preliminary decision taken to use JAXB to generate java classes for reading,writing and validating XML
* defined and iterated on an RNG schema for the Mifos Center business object
* created an ant build file that includes targets for:
* RNG -> WXS conversion
* RNG -> RNC (RELAX NG compact syntax format)
* WXS -> java class generation using JAXB
* running tests to exercise XML validation, reading & writing
* successfully converted the RNG schema to a WXS schema
* successfully generated and compiled java files from the WXS schema using JAXB
* created a sample Mifos Data Exchange XML file for Center objects
* wrote and successfully ran initial tests to validate, read and write Mifos Data Exchange XML for Center objects
Week ending 16 Mar 2007
-----------------------
* Consider options for a Mifos data import type (e.g. Excel, CSV, XML, SQL).
* Decide on the import type that will be used (XML - see [Data Migration Import Type Design Decision])
