Bi-monthly 360 MARC loads brief overview
On the first and fifteenth of the month Sonoe, Tessa, Dawn, Lisa, Kristen Wilson, Emily Lynema, and Marianne receive an email from Serials Solutions with a link for copying the current 360MARC files from their server. The .mrc, .txt, and .csv files are copied and saved to the g:\Acquisitions & Discovery\BibLoads\360Marc\ folder for the current year, month, and day (1st or 15th). The new and changed .mrc files are edited in MarcEdit and loaded into Symphony. Run reports in the following order: Deleted, New, and Changed for monographs (do monographs first) and repeat for serials. All reports are saved in g:\Acquisitions & Discovery\BibLoads\360Marc\SirsiErrorReports[year].
1. Copying the files locally
Copy and paste the FTP link received from the Serials Solutions email into your file explorer. The link is ftp://JS8LB8FT5Y:K1BXCx93@ftp.serialssolutions.com
Select everything except the Archives folder and the SSJReports folder and drag or copy and paste the items into g:\Acquisitions & Discovery\BibLoads\360Marc\ folder for the current year, month, and day (1st or 15th). Alternatively, clicking on the link will take you to a file directory in a web browser where you can download files individually.
Monographs and serials delete .mrc files do not need to be modified in MarcEdit before being loaded into sirsi. For the delete files, skip to “3. Loading the files.”
2. Preliminary edits in MarcEdit
It is necessary to edit the New and the Changed files for Serials and Monographs. There is no need to edit the Delete files.
Double click on the .mrc file with the correct report date in the 360MARC folder to open the file in MarcEdit. This creates a text version of the MARC file with a .mrk extension. Make sure all the data loaded.
In MarcEdit, for a quick view of total number of records, go to Reports in MarcEdit, Field Count, and check the count for the 245 field. It’s recommended you know how many records you receive from Serials Solutions in order to compare with the amount loaded in SIRSI.
Monograph records edit
- The only file to edit is the Changed records. Open in MARCedit and delete the 035s. When the records are overlayed in SIRSI, the 035s of the original record are retained and, usually, duplicated, so we're deleting them to prevent records from having multiple, duplicate 035s.
- Go to File, Compile to MARC. This creates a new .mrc file that replaces the original .mrc file. Save the .mrk file in the bibloads\360Marc folder, along with the newly edited and compiled .mrc. file.
Serial records edits
- The Mul. 022 MarcEdit Task List is used to separate the 022 fields into print and electronic. The Task List was created by Wes Woody. For new users, a copy of the macro, Mul. 022, is found here: G:\Acquisitions & Discovery\Data Projects & Partnerships Unit\022 macro (Mul.022)/tasksfile-2013_02_27_094442850.txt. In MarcEdit, in the MarcEditor, go to Tools, Manage Tasks, then Import Task. Use the file path to locate the Task List (a text file) on the G: drive. To run the macro, go to Tools, Assigned Tasks, Currently Available Tasks.
- Go to File, Compile to MARC. This creates a new .mrc file that will replace the original .mrc file. Save the .mrk file in the bibloads\360Marc folder, along with the newly edited and compiled .mrc. file.
3. Loading the files
Large loads of >10,000 records, notify Monographs head and Lisa before loading. For more information on this and large files of 50,000 or more, see "General notes on reports" below.
In Symphony, open the Utility module (tab). Select MARC Import from left menu. Hit OK to the defaults screen (neither box is checked). Under the import tab, use the gadget to select the .mrc source file you just compiled from the appropriate /BibLoads folder. Name the destination file with a name that means something to you (Monodeletes). File extensions (.mrc) are unnecessary (file will eventually be deleted once it has been imported). Import, say No to: "Are there more files to upload?" Close. If errors in call number size are in the new records report, find the record using the ssj#, edit the call number in MarcEdit or Notepad, and reload the edited record.
4. Running the reports
Run reports in the following order: Deleted, New, and Changed for monographs (do monographs first) and repeat for serials. All reports are saved in g:\Acquisitions & Discovery\BibLoads\360Marc\SirsiErrorReports[year].
Open the Reports module. Schedule new reports, OK, select the appropriate report in the Templates tab. Set up and schedule. Under the Load tab, select the destination file you named when you imported the original .mrc file. Click Run now.
To check on the status of your report, go to Finished Reports, click on the bunny icon to refresh. View report , select Save As under blue arrow at top left and save in \SirsiErrorReports folder (within G:\Acquisitions & Discovery\BibLoads\360MARC). Note, there is a separate sub folder for Monographs and an 'm' is added to the title for monograph files. Beginning 2015 Monograph deletes files are saved. Note: you may find it more efficient to record stats during this process.
5. Record Stats
Open the Excel spreadsheet in g:\Acquisitions & Discovery\BibLoads\360MARC\Stats\360MARCloadstats2.xlsx and select the current FY sheet. Record the number of records read and the number of errors or errors/overlays for each report.
On the 15th of each month, get the totals calculated from the formulas in the spreadsheet (less the errors/overlays) for the current month for new monograph and serial records and deleted monograph and serial records to record in the Acquisitions & Discovery Monthly Stats Google form. The formulas will need to be edited each month first. The Monthly Stats Google form records the statistics under your username. To change your username to 360Marc, edit the shared Google spreadsheet, NCSU Acquisitions & Discovery statistics spreadsheet. Under the username column change your username to 360Marc for these entries. Alternatively, edit the Google spreadsheet linked to the form directly.
Email Yan and Lynn the stats using the following template for the monographs reports:
Changed records report --
Read : 5821
Errors : 0
New records report --
Read : 24231
Errors : 0
Deleted records report --
Read : 23977
Deleted : 23974
The changed, new and delete reports are saved in Acquisitions & Discovery/Bibloads/360Marc/ SirsiErrorReports[current year]/Monographs/360Marc-Changed, Delete or New yymmddm.txt.
Email Christenna any serials errors, i.e., records that could not be deleted due to orders or serial controls attached. Let her know if there are no errors in the serial delete report also. You can use the following template:
The serial delete files are found in Acquisitions & Discovery/Bibloads/360Marc/SirsiErrorReports[current year]/360Marc-Deleted yymmdd.txt.
There was 0 error(s) in the 6/15/2016 delete file.
There was 0 error(s) in the changed file.
The serial changed files are found in Acquisitions & Discovery/Bibloads/360Marc/SirsiErrorReports[current year]/360Marc-Changed yymmdd.txt.
General notes on reports
- If an error occurs while running a report, you can reopen MarcEdit, rename the .mrc file as another copy, and then rename the destination file to start fresh and keep your new files separate from the problem files.
- Overwriting a “new” report will return a huge error report because they will already have been loaded and all of the records will be kicked out; overwriting a “change” report should just overlay the records you loaded earlier and error out the non-match flexkeys (should be same amount).
- Large loads of >10,000 records, notify Monographs head and Lisa before loading. Monographs will alert you to new packages that have been turned on recently that will produce several new records. See note about splitting the file, if record number is over 50,000.
- If a file is unusually large (50,000 + records), use the MARCSplit utility in MarcEdit (under the Tools menu). Notify Dawn Pearce that you will be running large reports in SIRSI, since the general recommendation is to load about 50,000 records per day, including general cataloging records. Run each one separately in SIRSI (may have to schedule the reports to run during early morning hours).
- For “new” reports, if the error rate seems high, check some of the titles to see if they are protected. They were significantly higher in April 2012 but this could be due to the newly tracked titles from the NotTracked db project. New records were manually brought in and subsequently protected. Some of the titles may not have WaSeSS ssj#s in the 035, so check for this and add if necessary.
- The errors in a change report (top half) may contain ssib records that couldn’t load. These are just ssib records that have changed over time and don’t require attention. The titles will still be found in the Journal finder, even if not in Endeca via Sirsi.
- Finished reports remain in Symphony for approximately 3-4 months. After that, Dawn deletes them. Saved files/spreadsheets in G:\Acquisitions & Discovery\BibLoads\360Marc are retained for one year.
- If you don't want new records to go to Marcive (because the records had already been to Marcive previously), change the date cataloged to NEVER by modifying the General Information box in the Load tab of the report. Change Update date cataloged to NEVER and Update title control information from title information entry: 948. These records will not go to OCLC either.