TIND batch upload from MARCXML: Difference between revisions

m (ErinBlake moved page TIND batch upload to TIND batch upload from MARC without leaving a redirect)
(→‎Post-MarcEdit preparation: 005 always needs to be stripped, not just for re-loads)
 
(8 intermediate revisions by the same user not shown)
Line 13: Line 13:
* Add Folger-specific [[MARC 856 Electronic Location and Access]] information
* Add Folger-specific [[MARC 856 Electronic Location and Access]] information
* Add Folger/TIND-specific 980 and 983 fields
* Add Folger/TIND-specific 980 and 983 fields


Other changes might include:  
Other changes might include:  
* Add 1XX/7XX $e Relator term based on content of $4 Relator code (Women Writers Online records: opted not to change before load: too time-consuming)
* Add 1XX/7XX $e Relator term based on content of $4 Relator code (Women Writers Online records: opted not to change before load: too time-consuming)


See [[MARC records from vendors]] for descriptions of changes needed for each record set. In most cases, changes that can be done by machine have been bundled into a MarcEdit task specific to that record set.
 
See [[MARC records from vendors]] for descriptions of changes needed for each record set. In most cases, changes that can be done by machine have been bundled into a MarcEdit task specific to that record set.


==Post-MarcEdit preparation==
==Post-MarcEdit preparation==
# After edits are finished, use MarcValidator to make sure you didn't break anything. Fix as needed.
# After edits are finished, use MarcValidator to make sure you didn't break anything (and to identify things that were broken all along). Fix as needed.
# Save file with PostMarcEdit in filename
# Save file with PostMarcEdit in filename
# If the .mrk file is large, and you'll be using the Batch Uploader, compile it as .mrc and split it into smaller batches before proceding.
# If the .mrk file is large, and you'll be using the Batch Uploader, compile it as .mrc and split it into smaller batches before proceding.
# Convert the file to MARC21XML and save with MRKtoXML or MRCtoXML in filename
# Convert the file to MARC21XML and save with MRKtoXML or MRCtoXML in filename
# Open the .xml file in Notepad++ and run the appropriate MacroExpress batch changes to convert to TIND's flavor of MARC XML:
# Open the .xml file in Notepad++ and run the appropriate MacroExpress batch changes to convert to TIND's flavor of MARC XML (see also [[TIND MARCXML]]):
##Change <marc:leader> and <marc:leader> to <controlfield tag="000"> and </controlfield>  
##Change <marc:leader> and <marc:leader> to <marc:controlfield tag="000"> and </marc:controlfield>  
##Replace all blank spaces in 00X fields with backslash character (typically this will be the 000, 007, and 008)
##Replace all blank spaces in 000, 006, 007, and 008 fields with backslash character
##Delete MARC 005 from each record (it is system-created during the upload, and the system gets confused if there's already something there).
##Delete MARC 005 so that TIND automatically generates a new "last updated" date
# Save file with XMLtoTINDXML in filename
# Save file with TINDXML in filename


==Actual loading==
==Actual loading==
Line 35: Line 37:
#See [https://docs.tind.io/article/d2ad9k1ugx-how-to-batch-upload-metadata How to batch upload metadata] for how it's supposed to work and [https://docs.tind.io/article/g1tv6r6gbe-caveats-common-errors-and-example-metadata-files Caveats, common errors and example metadata files] for things that are likely to go wrong.
#See [https://docs.tind.io/article/d2ad9k1ugx-how-to-batch-upload-metadata How to batch upload metadata] for how it's supposed to work and [https://docs.tind.io/article/g1tv6r6gbe-caveats-common-errors-and-example-metadata-files Caveats, common errors and example metadata files] for things that are likely to go wrong.
#Hit the "Upload" button and hope for the best. Be sure to check log files for problems.
#Hit the "Upload" button and hope for the best. Be sure to check log files for problems.
Remember that re-indexing is automatic, but takes a bit of time: if you search for a keyword from the newly uploaded files, you're not going to get them all at first.
Remember that Collection building/re-building can take a long time (e.g., overnight)


[[Category:Staff policies and procedures]]
[[Category:Staff policies and procedures]]
[[Category:TIND ILS]]
[[Category:TIND ILS]]
[[Category:MARC]]

Latest revision as of 15:59, 7 April 2023

Relatively small batches MARC records (no more than 1,000 records) can be bulk uploaded directly to the catalog using the Batch Uploader at https://catalog.folger.edu/batchuploader/. Use TIND customer support for larger batches. Files use UTF8 character encoding and must be in TIND's version of MARC XML.

Save copies after each step so you can back-track if something goes wrong.

Records can also be uploaded from .csv files, but for now this article only covers MARC

Initial file preparation

Examine the MARC file for non-Folger practice or unusual fields (e.g., by using MarcReport to produce a list of all tags, indicators, and subfields in use). Use a utility such as MarcEdit to add, edit, and delete information as needed.

Common changes include:


Other changes might include:

  • Add 1XX/7XX $e Relator term based on content of $4 Relator code (Women Writers Online records: opted not to change before load: too time-consuming)


See MARC records from vendors for descriptions of changes needed for each record set. In most cases, changes that can be done by machine have been bundled into a MarcEdit task specific to that record set.

Post-MarcEdit preparation

  1. After edits are finished, use MarcValidator to make sure you didn't break anything (and to identify things that were broken all along). Fix as needed.
  2. Save file with PostMarcEdit in filename
  3. If the .mrk file is large, and you'll be using the Batch Uploader, compile it as .mrc and split it into smaller batches before proceding.
  4. Convert the file to MARC21XML and save with MRKtoXML or MRCtoXML in filename
  5. Open the .xml file in Notepad++ and run the appropriate MacroExpress batch changes to convert to TIND's flavor of MARC XML (see also TIND MARCXML):
    1. Change <marc:leader> and <marc:leader> to <marc:controlfield tag="000"> and </marc:controlfield>
    2. Replace all blank spaces in 000, 006, 007, and 008 fields with backslash character
    3. Delete MARC 005 so that TIND automatically generates a new "last updated" date
  6. Save file with TINDXML in filename

Actual loading

  1. Test on a small batch of records!
  2. Go to Administration > Batch Uploader and follow the instructions
  3. See How to batch upload metadata for how it's supposed to work and Caveats, common errors and example metadata files for things that are likely to go wrong.
  4. Hit the "Upload" button and hope for the best. Be sure to check log files for problems.

Remember that re-indexing is automatic, but takes a bit of time: if you search for a keyword from the newly uploaded files, you're not going to get them all at first.

Remember that Collection building/re-building can take a long time (e.g., overnight)