DCam User Guide-Appendix C-Importing folders from a spreadsheet

October 18, 2024

The DCam Import Folders feature allows many folders to be created in a single step using data in a spreadsheet. This import process takes place as follows:

  1. Metadata describing documents to be imaged is obtained from an archive.
  2. The data is organized in a spreadsheet as a table of rows and columns.
  3. The table is saved to a .csv file.
  4. The .csv file is imported into DCam to create the folders.

Sections:

Getting Started
Dcam Listings Versus Folders
Creating an Import Folders Template File
Importing the Folders
Error Handling Error Log
Field ID (Column Header)
Creating The Dcam Listing
Opening and Saving .Csv Files with OpenOffice
Importing Images After Importing Folders

Getting Started

Spreadsheet Software

The recommended spreadsheet software is OpenOffice Calc. This tool can store spreadsheet data in a .csv form that preserves Unicode (UTF-8) extended characters.

Note: Microsoft Excel does not preserve extended characters. If the data does not contain extended characters, then Excel may be suitable. Other spreadsheet software that preserves extended characters may also work.

Data Organization

The data is organized as a table of rows and columns. Each row represents a folder. Each column represents a field of data for the folder.

The first row of the table contains a Field-ID for each column. The Field-ID uniquely identifies the data in that column and is also known as a Column Header. The order of columns in the table is arbitrary.

The following is a sample table:

Dcam Listings Versus Folders

DCam folders belong to a listing. For example, when creating a folder using the DCam Create option, the folder initially inherits values from the listing it is associated with. You can modify these values so that each folder is unique. (See “Creating the DCam Listing” that contains DCam Screenshots.)

Using the Import Folders feature, new folders are imported into a listing and belong to that listing.

Creating an Import Folders Template File

To create a template .csv file, from the More listings selection, select the New Import Folders Template option.

A Save As dialog allows the .csv file to be saved in a preferred location with a preferred name.

The default filename is: FoldersForImport_ProjectID_ListingID.csv

ProjectID—is a shortened project ID.

ListingID—is the listing ID (a number).

The file name may be changed. However, the ProjectID_ListingID and the.csv file extension should be preserved. The following is an acceptable name:

Xyz-Archive_ProjectID_ListingID_part1.csv

When you create a template .csv, an empty .csv file is generated with the first line containing Field-IDs.

Filling in the Table

The table can be populated with information to identify each folder. Not every Field-ID is required to be provided. The following is a simple example:

See “Field ID (Column Header)” for more detail on filling in the table. The steps to open and save a .csv file using OpenOffice Calc are found in “Opening and Saving .csv Files with OpenOffice”.

Importing the Folders

To start the process of importing folders, select the Import Folders option—this is one of the “More” listings options.

The Select a File Containing Folders to Be Imported dialog appears to allow selection of the .csv file.

The import folders process begins when you click Open.

The import process is as follows:

  • Check the .csv file for correctness.
  • Validate the data:
    • Make sure required fields are present.
    • Make sure fields are valid.
  • Ensure that the folders to be imported do not conflict with other folders. See Uniqueness constraints in “Unique Field Validations”.
  • If there are no errors, create the folders.

A progress dialog is displayed during import. If there are no errors, the Import Folders Results dialog displays, and the folders show up in the Folders section of the Manage Projects, Listings, and Folders screen.

Field-IDs

Note: A list of Field-IDs (column headers) and an explanation of their usage can be found in “Field ID (Column Header)”.

The first section of the table lists Basic Fields that exist for every folder. Some of these fields are required, and others are optional.

The second section lists Optional Fields often found in Listings. Optional Fields are not guaranteed to be the same from listing to listing.

Note: Optional Fields are displayed in DCam in the Record Description section (refer to Figure 4 in “Creating the DCam Listing”).

About Field-IDs

  • Every Field-ID is unique.
  • Only Field-IDs defined by a Listing’s FEX template are imported.
  • A Field-ID must only appear once in the table.
  • Not all Field-IDs are required.

Unknown columns are ignored. Unnamed columns are flagged as an error. User-defined Field-IDs are not supported.

Error Handling Error Log

When errors occur, they are displayed in a dialog. For example, the following table has several missing required fields:

An attempt to import this file results in the following errors:

This error information is also stored in a .log file. The log file has the same name as the .csv import file with “_nn” appended to the name and “.log” as an extension.

Note: nn is a number that keeps incrementing with each import attempt.

Field ID (Column Header)

Note: For convenience, a Field-ID may appear with or without spaces between the words that form the Field-ID. For example, “Begin Date” may also appear as “BeginDate:” This facilitates readability and wrapping of the column header to make narrower columns in the spreadsheet. 

Date Fields

Date fields are: BeginDate, EndDate, BeginCirca, and EndCirca

  • Only the BeginDate is required.
  • When EndDate is specified, then a date range is formed.

  • The BeginDate and EndDate must be text fields and not a spreadsheet date field.

    Note: Some spreadsheet applications limit date fields to the year 1900 and above.

  • The precision and format for BeginDate and EndDate follows the ISO 8601 standard:
    • YYYY—a year
    • YYYY-MM—a year and month
    • YYYY-MM-DD—a year, month, and day

    Note: YYYY, MM, and DD must be numeric. (Do not use language-dependent month names.)

  • BeginCirca and EndCirca indicate whether the date is exact or circa (approximate). “Yes” or “Y” indicates circa. “No,” “N,” or blank, indicates exact. These columns can be left blank or completely omitted if dates are exact.

Locality

For Locality, enter the RepID number of the locality that is listed in the top right portion of the locality box in DCam. To determine the correct RepID, use an existing DCam, search for the locality, and use the ID found for the desired location. For example:

RepID = 392685 is Montgomery, Indiana, United States

The spreadsheet will support a provisional place, which means a place under the listed RepID that has not been created yet. If you need to use a provisional place, use the following headers in the csv file:

  • ProvisionalPlace
  • ProvisionalPlaceLanguage
  • ProvisionalPlaceType

Record Type

For Record Type, enter the CVRecordTypeConceptID number—CV stands for Controlled Vocabulary. This ID is listed in the top right corner of the Record Type box in DCam.

To determine the correct CVRecordTypeConceptID, use DCam, search for the Record Type, and use the ID.

In the above example, the Record Type drop-down shows a list of choices:

Additional RecordTypes can be added if needed and can be specified in the CVRecordTypeConcept2 through CVRecordTypeConcept9.

Primary Language or Additional Languages

These fields specify languages associated with the folder’s contents. The Primary Language only allows 1 language to be specified, for example: Spanish. Secondary Languages can be zero or more languages separated by a comma, such as English, Italian, Portuguese.

Valid languages are found in the Primary Language drop-down of the DCam folder creation screen. 

Table Validations

  • Columns for required fields must be present.
    • RecordTitle
    • VolumeID
    • BeginDate
    • EndDate
    • ArchiveReferenceNumber
    • CVRecordTypeConceptID1
    • RepID
    • PrimaryLanguage
  • No Duplicate Field-IDs

Field Validations

  • Required fields must not be empty (or blank).
    • RecordTitle
    • VolumeID
    • BeginDate
    • EndDate
    • ArchiveReferenceNumber
    • CVRecordTypeConceptID1
    • PrimaryLanguage
  • Date Fields must be valid.
    • BeginDate and EndDate must be able to be parsed as an ISO-8601 date.
    • Must be a valid date. For example, 1950-14 and 1950-02-31 are not valid dates.
    • EndDate must be later or the same as the BeginDate and cannot be in the future.
    • BeginCirca and EndCirca must be blank or one of the defined values.
  • PrimaryLanguage or AdditionalLanguages must be valid.

Unique Field Validations

  • The combination of RecordTitle, VolumeID, and Date-Fields must be unique. Uniqueness is tested against all folders belonging to the Project in the DCam database.

    Note: The test for uniqueness occurs against (a) all folders about to be imported, (b) all folders already existing in the database.

  • ArchiveReferenceNumber
    • If provided, it must be unique.
    • If not provided or if a references number does not exist, click the No Archive Reference Number checkbox located below the text field.
    • Uniqueness is tested against all folders belonging to the project in the DCam database.

Creating The Dcam Listing

Record Title, Volume Designation and Dates are required fields. The Archive Reference Number, Locality, Language and Record Type fields (shown below) are also required.

Record type and record qualifier drop-downs are items associated with the record category.

Optional fields are shown above.

 

Opening and Saving .Csv Files with OpenOffice

Saving an OpenOffice Calc Worksheet as a csv File (Encoded in UTF8)

  1. Create a worksheet.
  2. From the File menu, click Save As.
  3. In the save file dialog, make sure the following options are set.
    1. Save as type = Text CSV (.csv) (*.csv)
    2. Edit filter settings = checked
  4. If the following dialog appears, select Keep Current Format to save the file as a csv file.
  5. In the Export of text files dialog, make sure the following options are set:
    1. Character set = Unicode (UTF-8)
    2. Field delimiter = , (comma)
    3. Text delimiter = " (double quote)
    4. Save cell content as shown = checked

Using OpenOffice Calc to Open or Edit an Existing csv File (Encoded in UTF8)

  1. From the File menu, click Open.
  2. Select the csv file.
  3. In the Text Import dialog, make sure the following options are set.
    • Character set = Unicode (UTF-8)
    • From row = 1
    • Separated by = selected
      • Comma = checked
      • Text delimiter = " (double quote)
      • Quoted field as text = checked
  4. Press OK.
  5. Make edits.
  6. From the File menu, click Save.
  7. If the following dialog appears, select Keep Current Format to save the file as a csv file.

Importing Images After Importing Folders

To import images after importing folders, create many folders from a spreadsheet, and then import images into the created folders.

Overview

The Import Folders feature allows many folders to be created in a single step using data in a spreadsheet. This feature has been enhanced to allow specifying a directory containing images for each folder. After creating the folders, DCam starts importing images into the newly created folders.

Specifying the Image Source

The following is a sample table representing the contents of an import folder .csv file. The ImageImportSource column indicates the directory containing images to be associated with the folder. This directory must exist and must contain images.

When this table is imported, each folder is created. Subsequently, for each folder with an ImageImportSource directory, an import action is created and placed in the DCam import queue.

After creating the folders and the import actions, the Import Folders Results dialog appears.

Image Import

The DCam importer begins importing images into folders immediately after the first import action is placed in the import queue. Each import action is processed one at a time until all have been processed.

Column Headers and Options

The following table describes the Column Headers and options associated with Image Import:

Example:

Tags
Camera Operators DCAM