HDMS Wiki Home

  • ExportDataForConversion
    		•  Last edited by Steven_Beiting on Nov 20, 2003 5:43 pm

    Final Export: BCD data

    This is the penultimate step of getting your data out of a BCD (for the conversion into a Biotics 4 Oracle database)

    This page covers the steps taken to do a complete data dump for an HDMS Data Conversion.

  • If you have been doing all your work up to this point in your Production BCD System instead of a snapshot, now is when you must make the snapshot (go to the MakeASnapshotOfBCD page to find out how). Once you make the snapshot, you MUST re-run the QC steps (in order to test that the configuration is still correct after creating the snapshot--go to the DataQc page to find out more). If you are working in the snapshot, go on to the next step.

    • SYSTEM REQUIREMENTS
    This covers the system requirements of the stand-alone computer on which the snapshot BCD will reside during the export process. If you already made the snapshot, test the computer for the following requirements, and if it does not meet the requirements (and cannot be otherwise made to do so), find another computer and simply copy the snapshot to it. If you followed the instructions for making a snapshot to-the-letter, moving the BCD to another computer should be as simple as a copy.

    Recommendations:

    • Expanded memory: 4096k (no more, no less).
    • Free space on hard disk: 8 times the size of the entire BCD (xml files are huge). [Minimum size is difficult to guage since the way BCD stores data does not reflect the actual size of the data.]

    Test:

    • Log into the snapshot BCD
    • At TCL, execute WHO
    • _____ On the first screen, make sure the item "Expanded Memory" says "Active"--if not, log out and make sure the PC is giving the BCD application the right amount of memory. [TODO: put a link here to a page on guidelines for configuration. Otherwise, B4 install contact should provide document...]

    • _____ On the third screen, make sure the "EM Allocated" shows 4mb or less (if it shows more, log out and configure the PC to make sure no more than 4096K of Expanded Memory is provided--AREV is one of those few programs out there that actually does not do better with more memory). To get Arev to use expanded memory, you have to tell windows to let it have the memory, then you have to tell Arev it may use it. Right-click the shortcut you use to start Arev. If you don't have a shortcut, then create one (right-click Arev.exe, drag, and let go. Then choose "create shortcut here"). Right-click the shortcut and choose properties. Click the Memory tab. Choose 4096 for Expanded memory (EMS). Now click the Program tab. The Cmd line: field should look something like this: "c:\arev\arev.exe heritage /x m4096" Now click OK and you're done.

    The computer that does the export and the computer that has the Java Runtime installed can be different computers if necessary* (you must move the \hdms subdirectory to that other computer before executing the last step; note that moving the xml files is onerous due to their size, so zip them up before you move them, and unzip them when you get them to the other computer).

    * However, if you have optional files, and your export process calls the to_conv.bat file, the computer that does the export must have the Java Runtime installed (because the to_conv.bat file calls a Java program). (See the InstallConversionSoftware page for more information about getting a Java Runtime installed.)

    Prepare to run the Export
    1. Inside the BCD, Navigate from the Main Menu | Adjuncts | Hdms Conversion Utilities, and choose "Rebuild SYN file" (wait for this process to complete) --this is extremely important!!

    2. Inside the BCD, Navigate from the Main Menu | Adjuncts | Hdms Conversion Utilities, and choose "Edit EXPORT.BCD"

    3. When it opens in Notepad, carefully remove the leading asterisks from the lines you wish to execute (or selectively replace the text "*BIOTICS" with just "BIOTICS" for those items that affect your BCD System). If you do not make any edits at all, nothing at all will be exported from the system.

    4. Save your changes and close Notepad--the BCD will remain "frozen" until you close Notepad. Again, if you experience difficulty saving the file, you may need to change the file properties (it may be a read-only file).

    5. Run all the QC steps on the snapshot. This is the final run. You MUST RUN THIS ON THE SYSTEM FROM WHICH YOU INTEND TO EXPORT THE DATA. This is because this last run is a double-check that the configuration files are still in place and that they are being correctly and accurately accessed by the BCD during the run.

      It should be relatively fast for a final run if you have been running the QC procedures. Keep the documentation from the QC procedures as they may prove helpful during remediation of post-conversion data problems.

    6. Make a separate backup of your snapshot BCD (just zip it up). In rare cases, you may need to restore this system and this is a good point as some remaining steps may make your BCD unusable.

    7. Optional: Defragment the hard disk (or at least check to see if it needs to be defragmented). A very fragmented hard disk will cause the export to run very slowly.

    8. If advised to do so, install and execute the 'KILL_BCD' utility. As of this writing, this utility is not readily available because its purpose is to break the BCD and make it non-functional (but fast). Its actual benefit has yet to be quantified and may thus be removed from this list of steps. (Wiki link=KillBcd)

    Export data from the BCD for HDMS Conversion
    1. If advised to do so, open the XML batch file and make necessary edits--it is already set up to execute the required loads.

    2. Inside the BCD, Navigate from the Main Menu | Adjuncts | Hdms Conversion Utilities, and choose "Execute the Data Dump"

    3. When prompted, type TEST_XML_EXPORT. (Your Biotics 4 Installation Contact/Support person may advise you to execute a different batch command.)

    4. Wait, wait, wait until it finishes. Just wait. It runs pretty nicely all by itself as long as you have enough disk space and properly configured memory.

      An problem has been reported regarding exporting PCA records (possibly related to the computer's memory configuration). If you experience this problem, re-edit the EXPORT.BCD document commenting out all the files that have already been exported, plus comment out the PCA export, then re-run the export in order to complete all the file dumps. Then, go back and comment out every export except PCA. Log out of the BCD, log back in, then retry the export. If you are still experiencing problems, you may need to send your entire BCD (or at any rate, your BCDDATA subdirectory) to NatureServe's home office to have the PCA data extracted for you.

    Back to RunningTheDumpForAnNHP