Search Technologies Assessment · Web viewFor example, E-mail files in XML, MS-Word or MS-Excel documents, digital photographs from digital cameras, digital video from digital camcorders,

National Archives and Records Administration

National Archives Catalog (The Catalog)

NARA Catalog Internal API Design– Catalog Perspective –

Status-FinalVersion 1.1July 6, 2015

National Archives & Records Administration

NARA Catalog Internal API Design

Version 1.1

July 6, 2015

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=owCr1QpR6RHqlM&tbnid=Nzjz_YVq0miSxM:&ved=0CAUQjRw&url=http://blogs.archives.gov/aotus/?attachment_id=1236&ei=51H6Ub_GHdO64APu34H4Aw&bvm=bv.50165853,d.dmg&psig=AFQjCNEMljwv59HAB11EckpVc6CXxxOUkA&ust=1375445858498456

NARA Catalog API Reference Guide

Contents

1 Overview................................................................................................................31.1 Other Relevant Documents...............................................................................................3

1.2 NARA Catalog Content.......................................................................................................3

1.3 API Philosophy...................................................................................................................5

1.4 General API Structure........................................................................................................6

1.4.1 The NAID...................................................................................................................6

1.4.2 The OPA-ID................................................................................................................6

1.4.3 Other Identifiers........................................................................................................7

1.4.4 Examples...................................................................................................................8

1.5 API Responses....................................................................................................................8

1.5.1 API Response Formats...............................................................................................8

1.5.2 General Response Structure......................................................................................9

1.6 Authentication.................................................................................................................11

1.6.1 Login through Catalog.............................................................................................11

1.6.2 Login with Third Party Authentication [LOW PRIORITY – RELEASE 2]......................12

1.6.3 Login Error Codes....................................................................................................13

1.6.4 Acquiring Catalog Credentials.................................................................................13

1.6.5 Using Catalog Credentials........................................................................................13

1.7 Registration......................................................................................................................13

1.8 Configuration...................................................................................................................13

1.9 Storage.............................................................................................................................14

2 Access API.............................................................................................................162.1 The URL Path Map...........................................................................................................16

2.1.1 Why use the Access API?.........................................................................................17

2.1.2 Other paths in https://catalog.archives.gov............................................................17

2.2 Top-Level Path – /iapi/v1.................................................................................................18

2.3 Holdings – /iapi/v1/id......................................................................................................18

2.3.1 The NAID – /iapi/v1/id/<NAID>...............................................................................18

2.4 Authority Records – /iapi/v1/id.......................................................................................34

2.4.1 People – /iapi/v1/id/<person-id>............................................................................34


2.4.2 Organizations – /iapi/v1/id/<org-id>.......................................................................36

2.5 User Information – /iapi/v1/accounts.............................................................................39

3 Search API............................................................................................................403.1 Overview..........................................................................................................................41

3.1.1 Search Results Entries.............................................................................................41

3.1.2 Annotations.............................................................................................................42

3.1.3 Search Functions.....................................................................................................44

3.2 Search API Parameters.....................................................................................................45

3.3 Query Expressions...........................................................................................................53

3.3.1 Keyword Fields........................................................................................................53

3.3.2 String Fields.............................................................................................................54

3.3.3 Date and Date-Time Fields......................................................................................55

3.3.4 Integer Fields...........................................................................................................56

3.3.5 XML Fields...............................................................................................................56

3.4 Fields................................................................................................................................58

3.4.1 Identifiers................................................................................................................59

3.4.2 Types.......................................................................................................................60

3.4.3 Display Fields...........................................................................................................63

3.4.4 More Facet Fields....................................................................................................64

3.4.5 Archival Hierarchy Fields.........................................................................................65

3.4.6 Digital Object Information.......................................................................................65

3.4.7 Annotations.............................................................................................................66

3.4.8 XML Fields...............................................................................................................67

3.4.9 Authority Information.............................................................................................68

3.4.10 Dates and Times...............................................................................................69

3.5 Search Results Output.....................................................................................................71

3.5.1 Header Information.................................................................................................72

3.5.2 Search Results.........................................................................................................74

3.5.3 Web Page Results....................................................................................................82

3.5.4 Pdf Files Results.......................................................................................................84

3.5.5 Facet Values............................................................................................................85

3.5.6 Highlights................................................................................................................87


3.5.7 Thesaurus Expansions.............................................................................................88

3.5.8 Complete Example..................................................................................................91

3.6 Exports...........................................................................................................................100

3.6.1 Specifying What Results to Export........................................................................100

3.6.2 Parameters............................................................................................................100

3.6.3 Examples...............................................................................................................103

3.6.4 Response (standard export)..................................................................................104

3.6.5 Bulk Exports..........................................................................................................105

3.7 Search based Content Detail..........................................................................................106

3.7.1 Content Detail result.............................................................................................106

3.7.2 Jump to Page functionality....................................................................................108

4 Annotations API..................................................................................................1104.1 The Basics......................................................................................................................110

4.1.1 Annotations are attached to content....................................................................110

4.1.2 Modifying annotations requires login...................................................................111

4.1.3 Editing annotations...............................................................................................111

4.1.4 Contributed Annotations by user..........................................................................111

4.1.5 Annotation parameter validation..........................................................................111

4.2 Annotations Summary Information...............................................................................112

4.3 Comments API...............................................................................................................115

4.3.1 Fetching Comments and Replies...........................................................................115

4.3.2 Modifying Comments and Replies.........................................................................119

4.3.3 Bulk Import of Comments.....................................................................................122

4.4 Tagging API....................................................................................................................123

4.4.1 Fetching Tags.........................................................................................................123

4.4.2 Modifying Tags......................................................................................................125

4.4.3 Bulk Import of Tags...............................................................................................127

4.5 Transcription API............................................................................................................128

4.5.1 Fetching a Transcription........................................................................................128

4.5.2 Modifying Transcriptions.......................................................................................130

4.5.3 Bulk Import of Transcriptions................................................................................133

4.6 Translation API...............................................................................................................134


4.6.1 Fetching a List of all Translations...........................................................................134

4.6.2 Fetching a Translation...........................................................................................137

4.6.3 Modifying Translations..........................................................................................139

4.6.4 Bulk Import of Translations...................................................................................142

5 User Contributions and Activities API..................................................................1445.1 User Summary Information...........................................................................................144

5.2 User Contributions.........................................................................................................145

5.2.1 Contributions Summary........................................................................................146

5.2.2 Common Parameters for Contributions................................................................148

5.2.3 User Contributed Comments................................................................................149

5.2.4 User Contributed Tags...........................................................................................152

5.2.5 User Contributed Transcriptions...........................................................................155

5.2.6 User Contributed Translations..............................................................................156

5.3 My Lists..........................................................................................................................158

5.3.1 Managing the List of Lists......................................................................................158

5.3.2 Managing a Single List...........................................................................................160

5.3.3 Creating or Adding to a List using the Search API..................................................161

5.3.4 Managing List Results............................................................................................162

5.4 Bulk Exports...................................................................................................................169

5.4.1 Fetching the List of Bulk Exports...........................................................................169

5.4.2 Downloading a Bulk Export File.............................................................................171

5.4.3 Deleting a Bulk Export...........................................................................................171

5.5 Notifications..................................................................................................................171

5.5.1 Parameters............................................................................................................172

5.5.2 Response...............................................................................................................172

5.5.3 Clearing Notifications............................................................................................174

5.5.4 Error Codes...........................................................................................................174

5.6 Account Information......................................................................................................174

5.6.1 Viewing Account Information................................................................................175

5.6.2 Modifying Account Information............................................................................175

6 URL mapping to support selected OPA pilot URLs...............................................1786.1 Mapping legacy URLs to Catalog URLs...........................................................................178


7 Scheduled maintenance processes......................................................................1797.1 Account maintenance processes...................................................................................179

8 Appendices.........................................................................................................1808.1 Location IDs...................................................................................................................180

8.2 Technical Metadata.......................................................................................................181

8.3 Third-Party Authentication [LOW PRIORITY – RELEASE 2]............................................181

8.4 Languages......................................................................................................................181


Version Control

Version Date Reviewer Summary Description

0.1 2013-12-20 Elizabeth Hobbs Initial Draft

0.2 2014-01-12 Elizabeth Hobbs Added information to introductory and annotations sections.

0.3 2014-02-02 Elizabeth Hobbs Further information for annotations and user contributions.

0.4 2014-02-05 Paul Nelson Review and adjustments overall. Work through sections 1-3 and 4.3.

0.5 2014-02-10 Elizabeth Hobbs Section 5.

0.6 2014-02-17 Elizabeth Hobbs Revisit section 4 with global updates and reformatting. Complete most of it.

0.7 2014-02-18 Paul Nelson Another top-to bottom cleansing and polishing. Rework technical metadata section and file inventory. Add full NAID section. Rework some paths to map onto XML structures better.

0.8 2014-02-19 Paul Nelson Review and cleanup section 5.

0.9 2014-02-21 Paul Nelson Review and cleanup section 4.

0.10 2014-02-22 Paul Nelson Review and cleanup section 3. Complete search results section. Add nested objects. Add group=description. Cleanup other grouping types. Fields cleanup.

0.11 2014-02-23 Paul Nelson Add exports, and lists to section 3. Add method to fetch all translations. Rework export formats and export options. Cleanup of section 3. Add transcriptions/translations to inventory XML. Allow multiple tag formats. Allow for multiple errors in OPA response.

0.12 2014-02-25 Madhu Koneni Review

0.13 2014-02-25 Paul Nelson Implement last-minute comments from NARA.

0.14 2014-03-06 Kristy Martin Internal review

1


0.15 2014-03-09 Paul Nelson Further modifications based on customer comments received on 2/28 and 3/10.

0.16 2014-03-15 Paul Nelson Rename this guide the “Internal API Design Document”

0.17 2015-03-18 Luis Vargas Added UI support items for Congressional Fields.

0.18 2015-03-26 Luis Vargas Removed UI support items for Congressional fields.

1.0 2015-04-20 Luis VargasAlejandro Baltodano

Content detail changes, S3 implementation, configuration refactors, URL mapping.Sections:1.81.93.74.1.56

1.1 2015-07-06 Alejandro Baltodano

Update configuration sectionAdded API scheduler documentationSection 7Rebranding to NARA CatalogUpdated endpointsUpdated section 3.5.4

2


1 Overview

This document is the internal API design document to support the Catalog user interface.

Specifically, this document will cover:

Access API – Accessing and downloading content by URL from NARA Catalog

Search API – Searching for content using the search engine, returning brief results.

Annotations API – Accessing, adding, updating, and deleting tags, transcriptions, discussions, and translations.

User API – Anything which is user-related, such as information for the user’s home page, user lists, bulk downloads, and viewing user contributions.

1.1 Other Relevant DocumentsOther documents relevant to this document:

NARA Catalog User Interface Design

o Identifies all end-user functionality which must be supported by these APIs.

OPA Architecture Design

o Explains the data model for information in Catalog.

NARA Catalog Archival Descriptions DMD

o Identifies the metadata fields available for NARA archival descriptions and associated digital objects and what functions are possible for each field.

NARA Catalog Authority Records DMD

o Identifies metadata fields available for NARA authority records and what functions are possible for each field.

NARA Catalog Web Sites DMD

o Identifies metadata fields for results from archives.gov, blogs, and all presidential library web sites, and what functions are possible for each field.

1.2 NARA Catalog ContentContent in Catalog is a rich mixture of many types of information:

Archival Descriptions – contain descriptive metadata about the holdings.

o Example metadata includes title, dates covered, creator, author, etc.

3


o Archival descriptions will describe both on-line content as well as physical materials which can only be viewed by visiting one of the Archive’s facilities.

Digital Objects – are digital copies of the on-line holdings which can be downloaded and viewed through Catalog.

Technical Documentation – is information (PDFs, usually) about the content, how it was collected, for what purpose, the formatting, etc.

Authority Records – represent people, places, locations (i.e. geographic subjects), topics (i.e. topical subjects) and organizations.

o The primary purpose of authority records is to link archival descriptions to famous people, organizations, locations, topics, etc. in a uniform fashion.

o Currently, only people and organizations can be retrieved from Catalog.

Contributions / Annotations – are for information added to NARA holdings by users of Catalog. This can include:

o Transcriptions – Transcribed text from scanned documents, audio recordings, or video recordings.

o Translations – Translated versions of the holdings.

o Comments – Discussions about the holdings.

o Tags – User generated tags to aid in the search and categorization of the holdings.

Web Pages – Catalog will crawl the archives.gov web site and all of the presidential web sites. All web pages crawled by Catalog will be included in the Catalog search engine index.

The information in Catalog falls into several different categories. The point of identifying these categories is to note that not all OPA information is appropriate for all API calls.

Searchable Information : includes all things which are indexed into the Catalog search engine and which can be retrieved by searches. This includes:

o Archival Descriptions

o Authority Records

o Digital Objects (scanned images, videos, audio files, etc.)

o Web Pages

The content and fields of these items will be indexed when available.

Annotatable Information : includes everything which can be annotated by a user.

o Annotations consist of tags, comments, translations, and transcriptions.

o Not everything can have the same annotations:

4


Archival Descriptions – Tags and comments

Authority Records – Tags and comments

Digital Objects (scanned images, videos, audio files, etc.) – Tags, comments, transcriptions, translations

Web Pages – Web pages cannot be annotated via Catalog.

Accessible Information : includes everything which can be accessed directly with a URL, including:

o Archival Descriptions

o Authority Records

o Digital Objects

o Alternative renditions of content, including:

Thumbnails, tiled images (for now), XML representations of text content and content fields

o Downloadable packages, including:

Database files, reformatted database files, and pre-packaged ZIP files

o Dissemination Information Packages (DIPs) – [not expected for OPA release 1] are official representations of archival packages exported from Catalog that are self-descriptive so they can be imported into other systems.

o Annotations – Tags, comments, replies, transcriptions, translations

1.3 API PhilosophyCatalog APIs are RESTful, and support the GET, POST, PUT, and DELETE methods:

GET is recommended to access content with a simple URL (see section , the Access API) and for searches (see section 3)

POST is used when creating new things (e.g. creating new annotations)

PUT is used when updating something (e.g. saving annotations)

DELETE is used when removing things (e.g. deleting annotations)

The proper HTTP Request method to use for every function will be identified in the documentation.

In general, the goal of each API transaction is to return all the information needed so a user interface application can create an appropriate page from the response. Additional clicks required to display something (i.e. click on a tab, click on a link to see a pop-up) will typically require a second transaction.

5

http://en.wikipedia.org/wiki/Representational_state_transfer


When Using POST

In the documentation below, this documentation will specify example URLs as follows:

POST https://catalog.archives.gov/iapi/v1/id/7226539/objects/263/translations?language=SPA&action=saveAndUnlock&text=%22Ley de Parkinson%22 aplicado a las instituciones podría sugerir que a medida...

Note, however, when using POST, it is expected that all of the content after the “?” will be transmitted as request data (i.e. not in the URL itself, like it would be for a simple GET).

1.4 General API StructureAPIs consist of a URL path, an optional action, and one or more parameters with values:

URL Path – Identifies the scope of content which is affected

o This can be the item fetched or the scope of the content to be listed.

Other parameters – [optional] Additional parameters may be needed to provide additional information, control how the API is executed, or specify how the results should be returned.

1.4.1 The NAID

The primary identifier for OPA will be the “National Archives ID” (NAID), which is used to access all content. NAID in this context refers only to the identifier for Archival Descriptions.

All archival descriptions will have an NAID, and all content (digital objects, contributions, data files, etc.) will be grouped under the appropriate NAID.

As an example, the “Letter from George McGovern to Harry S. Truman, 08/19/1972” has the NAID 7226539. This means that the descriptive metadata for this item can be retrieved using:

GET

http://research.archives.gov/iapi/v1/holdings/7226539/description

1.4.2 The OPA-ID

The OPA-ID uniquely identifies each and every entry in the search engine index. The format of the OPA-ID depends on the type of data:

Archival descriptions

o Format: desc-{NARA ID}

o Example: desc-7226539

People authority records

o Format: person-{NARA ID}

6


o Example: person-8889

Organization authority records

o Format: org-{NARA ID}

o Example: org-15238

Geographic subject authority records

o Format: geo-{NARA ID}

o Example: geo-8889

Topical Subject authority records

o Format: top-{NARA ID}

o Example: top-15238

Special Record Types authority records

o Format: srt-{NARA ID}

o Example: srt-15238

Digital Objects

o Format: desc-{NARA ID}/{OBJECT-ID}

o Example: desc-7226539/32

Web pages

o Format: {URL}

o Example: http://www.archives.gov/veterans/

1.4.3 Other Identifiers

Most everything in NARA will also have an identifier, which will be used to fetch that item.

Authority Record ID – There are two different types of authority records, “people” and “organizations”:

o Every authority record which represents a person will have a person ID

o Every authority record which represents an organization will have an organization ID

Object IDs – For on-line content, identifies individual digital objects within an NAID.

o Object IDs will be the stable ID from DAS or the ID from the SEIP [TBD – Requires discussion, object IDs not in ARC XML].

Comment IDs – Every comment and reply will have an ID.

Tag – The tag text will serve as the unique identifier for a tag.

7


1.4.4 Examples

Fetch the Archival Description for NAID 7226539:

GET https://catalog.archives.gov/iapi/v1/id/7226539

Return the description in JSON:

GET https://catalog.archives.gov/iapi/v1/id/7226539?format=JSON

Fetch all description-level comments for 7226539:

GET https://catalog.archives.gov/iapi/v1/id/7226539/comments

To save a Spanish translation for the “263-a1-27-box-7-89-4-0002.jpg” object within the item with NAID 7226539:

POST https://catalog.archives.gov/iapi/v1/id/7226539/objects/263/translations?language=SPA&action=saveAndUnlock&text=%22Ley de Parkinson%22 aplicado a las instituciones podría sugerir que a medida...

A simple search request with some parameters:

GET https://catalog.archives.gov/iapi/v1?action=search&q=kennedy

1.5 API Responses

1.5.1 API Response Formats

API Search results, the content detail page (also known as full result display), information fetched from Catalog, and responses are available either as XML or JSON. The parameter format is used to specify which type:

&format=JSON

&format=XML

If not specified, JSON is the default.

Pretty Printing the Response

Add the special parameter:

&pretty=true

To pretty print the output with correct indentation, tags aligned, etc. The default is pretty=false to save network bandwidth on communications.

Note: All of the examples in this document show pretty printed output even though the pretty=true parameter is not shown. This is to make this documentation more readable.

8


1.5.2 General Response Structure

This section discusses the standard response structure returned by Catalog.

OPA will return the standard HTTP response code (e.g. 200, 400, 401, 404, etc.) in the response headers and will also return more detail on the response in the response data.

1.5.2.1 Response Codes

All APIs will return the following standard HTTP response codes:

200 – OK

400 – Bad Request (bad parameters specified)

401 – Unauthorized (attempted to perform a function without logging in first, OR attempted to modify someone else’s data)

404 – Not Found (when the URL path is poorly formed, or when one of the identifiers specified in the URL is not found)

500 – OPA internal server error

1.5.2.2 Response Data

Some API requests will simply return the content directly. For example, GET’ing a movie will simply stream the bytes of the movie back as the API response data.

But API methods which do something (save a transcription, for example) or list things (e.g. search) will return an <opa-response> tag with the following sub-elements:

header – Holds information about the request, including:

o @status – The HTTP status code

o @time – The time the system spent handling the request

o request – Request information including the following sub-elements:

@path – The path requested

Request parameters- These will be name/value pairs, one for every parameter included in

the request.

- Note that some parameters will be purposefully removed (such as password), or any parameter which is not a legal XML tag.

o error – Additional error information (if an error occurred), including the following sub-elements

@code – The Catalog error code.- This will be a string such as “INVALID_LIST” which can be used to

construct an appropriate error message to the end user.

9


- A list of error codes returned by each call is listed in the documentation.

Error parameters- These will be name/value pairs which provide additional data on the

error which occurred.

- Note: Should any parameters not be legal characters for XML tags, they will be removed from the output.

Results section [optional]

o What appears in this section varies based upon the API request. Some requests will not have results, only status.

o If there was an error, this section will not be present.

Outline

The following is a general outline of the header returned by most Catalog API calls:

<opa-response><header status="HTTP Status" time="Time in ms">

<request path="URL-path"><PARAM-NAME1>param-value1</PARAM-NAME1><PARAM-NAME2>param-value2</PARAM-NAME2><PARAM-NAME3>param-value3</PARAM-NAME3>

</request> <errors>

<error code="ERROR-CODE"> <PARAM-NAME1>param-value1</PARAM-NAME1>

<PARAM-NAME2>param-value2</PARAM-NAME2> <PARAM-NAME3>param-value3</PARAM-NAME3>

</error> </errors>

</header> . . Results specific tags as defined later in this document goes here .</opa-response>

XML Example

<opa-response><header status="200" time="132">

<request path="/iapi/v1"><action>search</action><q>kennedy</q><f.title>rose</f.title>

</request></header>

. . The search results would be returned here .</opa-response>

JSON Example

{ "header":{ "@status":"200", "@time":"132", "request":{

10


"@path":"/iapi/v1", "action":"search", "q":"kennedy", "f.title":"rose" } }, . . The search results would be returned here .}

1.6 AuthenticationSearch and viewing NARA content do not require authentication. However, any editing of annotations (tags, comments, transcriptions, and translations) will require a login.

1.6.1 Login through Catalog

All registered OPA users will be able to log in directly to Catalog with a username and password. The API method is straightforward:

GET https://catalog.archives.gov/iapi/v1/login?user=<user-id>&password=<password>

Note:

“https” is absolutely required. Logins through HTTP will be rejected.

The credentials will be returned as a cookie in the HTTP headers AND in the login response (in a nested <credentials> tag). See section 1.6.4 for more information on the cookie response.

For security reasons, the password parameter will not be returned in the response.

The example login responses below only show whether or not the login was successful. A “400” (BAD REQUEST) error will be returned if login failed.

XML Example Login Response

<opa-response><header status="200" time="132"/>

<request path="/iapi/v1/login"><user>pnelson</user>

</request></header><credentials>7F8A29ECAA4326AC81329AFBB26649ACE</credentials>

</opa-response>

JSON Example Login Response

{"header": {

"@status":"200","@time":"132","request": {

"@path":"/iapi/v1/login","user":"pnelson"

}

11


},"credentials":"7F8A29ECAA4326AC81329AFBB26649ACE"

}

1.6.2 Login with Third Party Authentication [LOW PRIORITY – RELEASE 2]

Login with a third-party credential will be done using the following URL:

GET https://catalog.archives.gov/iapi/v1/login?provider=<provider>&token=<auth-token>

The provider can be any of: “google”, “twitter”, or “facebook”. The token will be the login token returned by the provider’s authentication method.

See the appendix (section 8.3) for detailed instructions and links to documentation on how to fetch the appropriate authentication token for each provider.

Catalog will validate the login token and then will provide Catalog credentials which must be used for all future API calls.

Note:

“https” is absolutely required. Logins through HTTP will be rejected.

The credentials will be returned as a cookie in the HTTP headers (see section 1.6.4 for more information) as well as in the response data (in the <credentials> tag, see example below).

For security reasons, the authentication token parameter will not be returned in the response.

The example login responses below only show whether or not the login was successful.

XML Example Login Response

<opa-response><header status="200" time="132"/>

<request path="/iapi/v1/login"><provider>google</provider>

</request></header><credentials>7F8A29ECAA4326AC81329AFBB26649ACE</credentials>

</opa-response>

JSON Example Login Response

{"header": {

"@status":"200","@time":"132","request": {

"@path":"/iapi/v1/login""provider":"google"

}},"credentials":"7F8A29ECAA4326AC81329AFBB26649ACE"

}

12


1.6.3 Login Error Codes

Error Code Description Parameters

LOGIN_FAILED The username or password was specified incorrectly. None

BAD_PROVIDER The provider name was not one of “google”, “twitter” or “facebook”.

None

CORRUPTED_TOKEN

The token provided does not appear to be in the expected encoding or format.

None

INVALID_TOKEN The token provided for a third-party authentication could not be validated.

None

SYSTEM_ERROR An error occurred during login. Please try again later. None

1.6.4 Acquiring Catalog Credentials

The login credentials will be returned with the HTTP headers in response to a login request.

Set-Cookie: _lopa=<credentials>

The user of the Catalog APIs will need to save the cookie information so that it can be used for all future requests.

Note that the cookie has a lifetime of a single browser session. When using the APIs, you should only keep the cookie in RAM, it should never be written to disk and re-used.

1.6.5 Using Catalog Credentials

Credentials are reuired for any API method which modifies content (e.g. annotations) on Catalog. APIs requiring authentication are indicated with “Requires login” throughout this document.

1.6.5.1 /iapi/v1Using the Cookie: HTTP Header

The second method for specifying credentials is to send it as a “Cookie:” in the HTTP request headers:

Cookie: _lopa=< credentials>

Note that HTTP can be used (does not require HTTPS) after login.

1.7 RegistrationRegistration will be handled with JSP pages built on the server side, and not through API calls.

1.8 ConfigurationThe OpaAPI is a set of three maven projects which are:

13


OpaAPI which is the web application that will process the requests made to the OPA API.

OpaArchLib which is a library with the basic set of classes needed for OpaAPI and OpaExportServer. This is a dependency for both projects.

OpaExportServer which is an independent JAVA application in charge of scheduling and creating exports of huge size.

The OpaAPI and the OpaExportServer are projects that can be executed (in a server application or a java application), after the maven install command is run in both project, the follow file structure will be created:

Config directory: This contains the application properties and configuration file.

Resources directory: This contains the set of XSL file required for exports and content details, the files containing the whitelists and blacklist used for validating request parameters and the NaraDas file.

Output container file: This can be the war container file for a latter deployment on a server application or the jar file for running the export server.

In order to run these two components is necessary to set up the following variables as part of the JAVA variables passed to the application server or the java application.

-Dgov.nara.opa.api.data=<opastorage and export parent directory>-Dgov.nara.opa.api.config=<config directory>-Dgov.nara.opa.api.resources=<resources directory>

The data folder is used for pointing the parent directory where the opastorage and the output exports are located.

Besides those three variables there is another variable which needs to be set up only in the export server.

-Dgov.nara.opa.api.bin=<export server binary directory>

1.9 StorageFile API and Export Server use both local and S3 based storage.

Local storage in file system is used for the following:

Configuration and system resource files (read only)

Staging storage for export files (read/write)

S3 storageAPI is composed by three maven projects:

14


OpaAPI which is used for the following:

Media file and metadata storage (read only)

Finished export file storage (read/write)

For media file streaming performance API uses CloudFront as media server.

S3 and CloudFront require setting up connection and authorization values. These values are set in API and Export Server’sweb application.properties file. The settings are, in charge of handling the request sent to the following: OPA API.

useS3Storage: (true/false) specifies whether or not S3 storage is active. If false, normal file system storage is used.

s3BaseLocation: (path) specifies the S3 key prefix that all stored media and metadata elements will share.

s3ExportsLocation: (path) specifies the export files location in S3.

s3StorageBucketName: (string) the S3 bucket name.

s3StorageAccessKeyId: (string) the S3 access key.

s3StorageSecretKey: (string) the S3 secret key.

useCloudFront: (true/false) whether or not CloudFront should be used for media streaming.

cloudFrontDomainName: (url) the CloudFront server domain name

15


2 Access API

The Access API is used to download digital objects, get descriptive metadata, annotations and other information from OPA.

2.1 The URL Path MapThe following map shows all of the possible paths available for the APIs.

http://research.archives.gov/iapi/v1/authorities/ – all authority records (section 2.4)

organizations/ – all metadata and annotations on the organization (section 2.4.2)<organization-id>/ – all metadata and annotations on the organization

organization – just descriptive metadata on the organizationannotations/ – summary info for annotations on the org (sec. 4.1.5)

comments – comments on the organization (sec. 4.3)tags – tags on the organization (sec. 4.4)

people/ – all people authority records (section 2.4.1)<person-id>/ – all metadata and annotations on the person

person – just descriptive metadata on the specified personannotations/ – summary info for annotations on the person (sec. 4.1.5)

comments – comments on a particular person (sec. 4.3)tags – tags on a particular person (sec. 4.4)

holdings/ – all content and content related data (sec. 2.3)<NAID>/ – everything related to a single archival description (sec. 2.3.1)

annotations/ – summary info on all annotations (sec. 4.1.5)comments – comments on archival descriptions (sec. 4.3)tags – tags on archival descriptions (sec. 4.4)

content/ – all content files (sec. 2.3.1.4)description – descriptive metadata (sec. 2.3.1.1)objects/ – all information on digital objects in the NAID (sec. 2.3.1.2)

<object-id>/ – information on a specific objectannotations/ – summary info for annotations on the content file (sec. 4.1.5)

comments – comments on the content file (sec. 4.3)tags – tags on the content file (sec. 4.4)transcription – transcription of the content file (sec. 4.5)translations – all translated languages for the content file (sec. 4.6)

opa-renditions/ – renditions of content files for UIs (sec. 2.3.1.5)thumbnails/ – holds all thumbnails of content filesimage-tiles/ – holds tiled image representations for image browsers

16


users/ – information about registered users on OPA (sec. 5)<user-id>/ – summary information on a specific user ID (sec. 5.1)

account – profile information on the user (sec. 5.6)bulk-exports – information on bulk exports available or in progress (sec. 5.4)contributions/ – summary of all contributions by the user (sec. 5.2)

contributedComments – comments made by the user (sec. 5.2.3)contributedTags – comments made by the user (sec. 5.2.4)contributedTranscriptions – transcriptions made by the user (sec. 5.2.5)contributedTranslations – translations made by the user (sec. 5.2.6)

Note: The “contributed” prefixed on these paths is used to avoid confusion with the standard “comments”, “tags”, “transcriptions” and “translations” tags because the two sets of tags return different metadata structures.

lists – lists of documents (“My Lists”) by the user (sec. 5.3)notifications – user notifications (sec. 5.5)

web – allows for searching over web pages (sec. 3)

2.1.1 Why use the Access API?

We recommend using the Access API over using the Search API when you know the ID of a record to be fetched. The Access API will be faster than performing a search and will use fewer Catalog resources to complete the request.

2.1.2 Other paths in https://catalog.archives.gov

There will be other paths within https://catalog.archives.gov used by the end-user interface, including:

https://catalog.archives.gov/description

https://catalog.archives.gov/organization

https://catalog.archives.gov/person

https://catalog.archives.gov/

These paths are NOT part of the API and should NOT be used by users of the APIs to download content. Their functionality and content may change at any time without notice or an apology.

Note, however, these URLs can be used to construct links for web pages, for any user interface that wishes to link to web pages served by the official Catalog user interface.

17


2.2 Top-Level Path – /iapi/v1The “http://research.archives.gov/iapi/v1” path will be used for executing searches across all Catalog content (holdings, authorities, and web sites).

For more information on search features available, please see section 3.

2.3 Holdings – /iapi/v1/idDescriptions of physical holdings and actual copies of on-line holdings can be downloaded from Catalog using paths which start with “http://catalog.archives.gov/id”.

You may also execute searches on this path, if you wish to limit your results to only documents from the “holdings” source. See section 3 for more details.

2.3.1 The NAID – /iapi/v1/id/<NAID>

All Catalog holdings are accessed using the “NARA Identifier” (NAID). A unique NAID is assigned to every collection, record group, series, file unit, and item inside of NARA. All holdings receive an NAID, regardless of whether the holdings are on-line or only available by visiting a NARA facility.

Notes:

The NAID is an integer – For example “7226539”

o The number ranges from 1 to approximately 8 million.

The NAID is stable – Once assigned to a description it will never change

All descriptions at all levels have an NAID

o Record groups, collections, series, file units, and items

Examples:

“Letter from George McGovern to Harry S. Truman, 08/19/1972” (Item)

o NAID = 7226539

“Subject Files, 1953 – 1972” (Series)(from the Harry S. Truman Post-Presidential Papers, 1953 – 1973)

o NAID = 201502

“Harry S. Truman Post-Presidential Papers, 1953 – 1973” (Collection HST-PPP)

o NAID = 1204

18


How do I find the NAID?

There are two methods:

1. Search using the public Catalog web site: https://catalog.archives.gov/search .

a. When you find a record that you need, view the record and the “National Archives Identifier” (NAID) will be clearly identified in the search results and on the content detail page.

2. Perform a search with the Search API. See section 3.

a. All archival descriptions will have a “naid” field that you can use to locate and download content.

b. All objects will have a “parentNaid”.

i. This is also an NAID that can be used to locate and download content.

The NAID Contents

The contents of a NAID on Catalog are gathered together into a construct called an “OPA Information Package” (OPA-IP) which serves as a holder for all content and metadata related to the archival description identified by the NARA ID.

When the NAID can be fetched directly, using a path such as:

GET https://catalog.archives.gov/iapi/v1/id/7226539

This will return a summary of all of the information contained in the NAID, appropriate for presentation in a content-detail representation. This information will include:

<description> – The complete archival description (section 2.3.1.1)

<objects> – An inventory of the objects and files contained within the NAID (section 2.3.1.2)

<annotations> – A summary of the annotations contained within the NAID (section 4.1.5)

19

https://catalog.archives.gov/search


XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539

<opa-ip naid="7226539"> <description> . . The archival description goes here. See section 2.3.1.1 . </description> <objects version="OPA-OBJECTS-1.0" count="3" created="2014-03-14T13:44:34"> . . The archival description goes here. See section 2.3.1.2 . </objects> <annotations> . . A summary of the annotations on this description will go here (section 4.1.5) . </annotations></opa-ip>

JSON ExampleGET http://catalog.archives.gov/iapi/v1/id/7226539?format=json

{ "@naid":"7226539", "description":{ . . The archival description goes here. See section 2.3.1.1 . }, "objects":{ "@version":"OPA-OBJECTS-1.0", "@count":"3", "@created":"2014-03-14T13:44:34", . . The archival description goes here. See section 2.3.1.2 . }, "annotations":{ . . A summary of the annotations on this description will go here (section 4.1.5) . }}

2.3.1.1 description – /iapi/v1/id/<NAID>/description

Use this path to download the descriptive metadata for the archival description. The default representation will be XML. The format can be specified as a query parameter to download a JSON representation of the description like /iapi/v1/id/<NAID>/description?format=json.

For more information about the archival description XML provided by Catalog, please see [TBD – is there documentation on this XML format from NARA?].

Note: The <objects> tag will be removed from the <archival-description> document. Instead, use the <objects> from the objects inventory see section 2.3.1.2, which contains more information.

20


XML ExampleGET https://catalog.archives.gov/iapi/v1/id/863/description

<?xml version = '1.0'?><description> <arc-id>863</arc-id> <arc-id-desc>863</arc-id-desc> <title>Eunice B. Haden Collection</title> <title-only>Eunice B. Haden Collection</title-only> <collection-id>HADEN</collection-id> <sequence-number>1</sequence-number> <created-timestamp>11/9/2013 1:13:20</created-timestamp>

<edited-timestamp>[g_x16_795, g_x128_99, g_x2_6365, g_x64_198, g_x32_397, g_x8_1591, g_x4_3182, b_x1_12731]</edited-timestamp>

<level-of-desc level-id="COL"> <lod-display>Collection</lod-display> </level-of-desc> <series-count>1</series-count> <title-date>1865 - 1865</title-date> <inclusive-dates> <inc-start-date>1865</inc-start-date> <inc-end-date>1865</inc-end-date> </inclusive-dates> <donors> <donor num="1" donor-record-type="PER" donor-id="8380485" standard="Y"> <donor-display>Haden, Eunice B.</donor-display> </donor> </donors> <variant-control-numbers> <variant-control-number num="1" mlr="false"> <variant-number>HADEN</variant-number> <variant-number-desc>HADEN</variant-number-desc> <variant-type>NAIL Control Number</variant-type> </variant-control-number> </variant-control-numbers> <physical-occurrences> <physical-occurrence> <copy-status>Preservation-Reproduction-Reference</copy-status> <reference-units> <reference-unit num="1" summary="true"> <ref-id>31</ref-id> <name>National Archives at College Park - Still Pictures</name> <address1>National Archives at College Park</address1> <address2>8601 Adelphi Road</address2> <city>College Park</city> <state>MD</state> <postcode>20740-6001</postcode> <mailcode>RD-DC-S</mailcode> <phone>301-837-0561</phone> <fax>301-837-3621</fax> <email>[email protected]</email> </reference-unit> </reference-units> </physical-occurrence> </physical-occurrences> <indexable-dates> <date-range>[g_x16_116, g_x4_466, b_x1_1865, g_x64_29, g_x2_932, g_x128_14, g_x32_58,

g_x8_233]</date-range> </indexable-dates></description>

21


JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/863/description?format=json

{ "arc-id":"863", "arc-id-desc":"863", "title":"Eunice B. Haden Collection", "title-only":"Eunice B. Haden Collection", "collection-id":"HADEN", "sequence-number":"1", "created-timestamp":"11/9/2013 1:13:20", "edited-timestamp":"[g_x16_795, g_x128_99, g_x2_6365, g_x64_198, g_x32_397, g_x8_1591,

g_x4_3182, b_x1_12731]", "level-of-desc":{ "@level-id":"COL", "lod-display":"Collection" }, "series-count":"1", "title-date":"1865 - 1865", "inclusive-dates":{ "inc-start-date":"1865", "inc-end-date":"1865" }, "donors":{ "donor":{ "@num":"1", "@donor-record-type":"PER", "@donor-id":"8380485", "@standard":"Y", "donor-display":"Haden, Eunice B." } }, "variant-control-numbers":{ "variant-control-number":{ "@num":"1", "@mlr":"false", "variant-number":"HADEN", "variant-number-desc":"HADEN", "variant-type":"NAIL Control Number" } }, "physical-occurrences":{ "physical-occurrence":{ "copy-status":"Preservation-Reproduction-Reference", "reference-units":{ "reference-unit":{ "@num":"1", "@summary":"true", "ref-id":"31", "name":"National Archives at College Park - Still Pictures", "address1":"National Archives at College Park", "address2":"8601 Adelphi Road", "city":"College Park", "state":"MD", "postcode":"20740-6001", "mailcode":"RD-DC-S", "phone":"301-837-0561", "fax":"301-837-3621", "email":"[email protected]" } } } }, "indexable-dates":{ "date-range":"[g_x16_116, g_x4_466, b_x1_1865, g_x64_29, g_x2_932, g_x128_14, g_x32_58,

g_x8_233]" }}

22


2.3.1.2 objects – /iapi/v1/id/<NAID>/objects

Every NAID that contains digital objects will have an inventory which lists all digital objects and all of their various renditions and annotations. The list of objects can be fetched with:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects

The default listing will not return any nested annotations with the objects, only a @hasAnnotations attribute.

To return a summary of nested annotations (counts), use:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>/objects/<objectId>

To return all nested annotations and all text, use:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>/objects/<objectId>

Why not Objects inside of <description>?

The Objects XML is kept separate from the archival description for several reasons:

1. It contains files which are not available in the Lifecycle Data Requirements Guide (LCDRG), including:

a. Image tiles

b. Search Engine Data (se data)c. Transcriptions / Translations

2. It contains much more technical metadata than is available in the LCDRG.a. Image color models, resolutions, etc.

b. Video frames per second, audio channels, etc.c. Number of pages per PDF, other document metadata

3. It is built by Catalog during ingestion and is maintained as transcriptions and translations are updated.

Objects XML Format

The fields in the objects.xml are as follows:

@version – Specifies the type and version of the objects XML.

o The only version currently defined is “OPA-OBJECTS-1.0”

@created – Date-time this inventory.xml file was first created.

o Note: Must be maintained as the file is updated.

object – specifies a digital object

o This is a holder for multiple renditions and metadata for the object

23


Each <object> tag will have the following nested information:

@id – The identifier for this object, unique across all objects within the same NAID.

o Note: This will be the stable DAS object ID for the object, for objects from DAS.

o For objects from SEIP, this will be a unique number for the object within the SEIP.

@hasAnnotations – True if the object has annotations, false otherwise.

description – This is a copy of the object’s description from the archival description.

o From the LCDRG: “Provides information about the Digital Object that is not apparent from Object Designator or Object Identifier”

designator – This is a copy of the object’s designator from the archival description.

o From the LCDRG: “An identifier for each digital object when there is more than one digital object associated with an archival description.”

file – Specifies the file which holds a copy or alternative representation of the content of the digital object. Has the following sub-elements:

o @type – The type of the file (only “primary” supported in 1.0)

o @mime – The MIME type of the file

o @path – The path to the file, relative to the NAID URL.

o technicalMetadata – Holds extracted metadata for the file

The contents of the technical metadata section will vary based on the type of file. See the appendix (section 8.2) for detailed information on all of the technical metadata which is gathered for each different file type.

thumbnail – Specifies the location of the thumbnail for the object.

Note: This is only used for thumbnails which are an image of the actual content, and it may be missing. If missing, the mime type should be used to show an appropriate icon for the content in place of the thumbnail.

Note that thumbnails are Catalog -generated for presentation purposes only and will not have technical metadata.

Has the following sub-elements:

o @path – A path, relative to the NAID URL, for accessing the thumbnail.

o @mime – The mime type of the thumbnail.

imageTiles – If the content represents an image, <imageTiles> will specify the location of a tiled version of the document which can be accessed by Open Seadragon.

24

http://openseadragon.github.io/



o @path – The path, relative to the NAID URL, where the image tiles can be found. This is often a path to a directory where the individual tiled files can be accessed.

o @type – The format of the image tiles. Currently, only “DZI” is supported.

seData – Holds search engine data for the object. This is additional extracted metadata for the file such as from/to/cc/bcc for E-mail files. This metadata will be specified as XML in the NARA SEIP (version 1) SE data format.


o @path – Specifies the path to the Search Engine metadata, relative to the NAID URL.

o @mime – The mime type of the SE data. Always “text/xml”.

o all se-data fields – Note that the <seData> tag will contain the entire metadata from the SE data file for the specified object.

annotations – This is the parent tag for all annotations contained within the object. It contains the following sub-elements:

o comments – Lists all of the comments on the object, in the same format as described in section 4.3.1.

o tags – Lists the tags on the object, in the same format as described in section 4.4.1.

o transcription – If the object has a transcription, then the file name for the transcription will be specified, in the same format as described in section 4.5.1.Contains the following sub-elements:

o translations – Parent tag to hold the object translations, in the same format as described in section 4.6.1.

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects?annotations=summary

<objects version="OPA-OBJECTS-1.0" count="3" created="2014-03-14T13:44:34"> <object id="1" hasAnnotations="true"> <designator>1</designator> <file type="primary" path="./content/hst-ppp_93-1_18-01.jpg" mime="image/jpeg"> <technicalMetadata> <createDate>2012-10-22T03:26:34Z</createDate> <metadataDate>2013-02-20T02:55:27Z</metadataDate> <modifyDate>2013-02-20T02:55:27Z</modifyDate> <size>1142680</size> <mime>image/jpeg</mime> <dimensions width="2181" height="2781"/> <resolution x="300" y="300" units="pixels/inch"/> <bitsPerSample>8 8 8</bitsPerSample> <photometricInterpretation>RGB</photometricInterpretation> <orientation>horizontal</orientation> <samplesPerPixel>3</samplesPerPixel>

25


<planarConfiguration>chunky</planarConfiguration> <colorSpace>sRGB</colorSpace> <compression>uncompressed</compression> <software>Adobe Photoshop CS6 (Windows)</software> </technicalMetadata> </file> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-01-thumb.jpg" mime="image/jpeg"/> <imageTiles path="./opa-rendition/image-tiles/hst-ppp_93-1_18-01_jpg"/> <annotations> <comments comments="3" replies="2" total="5"> <comment id="19328" user="kmartin" created="2014-02-10T14:36:23" lastModified="2014-02-10T14:40:12"> <text>One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if

there was real affection between the two.</text> <replies> <reply id="45239" user="mstewart" fullName="Meredith Stewart" naraStaff="true" created="2014-02-10T15:00:03"> <text>True, but from different generations.</text> </reply> <reply id="45422" user="mkoneni" created="2014-03-08T04:12:52" lastModified="2014-03-08T04:18:02"> <text>I'm sure they ran into each other a bunch, you know at charity functions and

golf tournaments...</text> </reply> </replies> </comment> <comment id="19102" user="tjefferson" created="2014-03-08T04:12:52" isDeleted="true"/> <comment id="17024" user="rbarnes" created="2014-03-09T11:14:27"> <text>Seems rather formulaic to me. I bet a staff member wrote it.</text> </comment> </comments> <tags total="3"> <tag text="Truman Post President" user="jdoe" created="2014-02-21T16:22:58"/> <tag text="McGovern Letters" user="agullet" created="2014-02-27T18:23:47"/> <tag text="nara:president=Truman" user="mstewart" fullName="Meredith Stewart"

naraStaff="true" created="2014-03-19T07:07:42"/> </tags> <transcription lastModified="2014-02-21T16:22:58" isLocked="true" isauthoritative="false"

version="232"> <lockedBy id="gwashington" when="2014-02-22T10:08:41"/> <users total="3"> <user id="tjefferson" fullName="Thomas Jefferson" isNaraStaff="true" lastModified="2014-02-21T16:22:58" /> <user id="mstewart" fullName="Meredith Stewart" isNaraStaff="true" lastModified="2014-02-08T12:59:49"/> <user id="kmartin" lastModified="2014-02-02T08:44:12" /> </users> <text>GEORGE McGOVERN United States Senate Washington, D.C. 20510

August 19, 1972

The Honorable Harry S. Truman Independence, Missouri

My dear Mr. President:

It is an honor for me, as our party's candidate for President of the United States. . .</text> </transcription> <translations total="3"> <translation code="ita" lastModified="2014-02-21T16:22:58" isLocked="true"

isauthoritative="false" version="232"> <lockedBy id="gwashington" when="2014-02-22T10:08:41"/> <users total="3"> <user id="tjefferson" fullName="Tomás Jefferson" isNaraStaff="true" lastModified="2014-02-21T16:22:58" /> <user id="gwashington" fullName="Giorgio Washington" isNaraStaff="true" lastModified="2014-02-08T12:59:49"/> </users> <text>George McGovern

26


Senato degli Stati Uniti Washington, D.C. 20510

19 ago 1972

L'Onorevole Harry S. Truman Independence, Missouri

Mio caro Signor Presidente:

E 'un onore per me, come il nostro partito di candidato alla Presidenza degli Stati Uniti. .</text> </translation> <translation code="spa"> . . Full information on the Spanish translation goes here . </translation> <translation code="deu"> . . Full information on the German translation goes here . </translation> </translations> </annotations> </object>

<object id="2"> <designator>2</designator> <file type="primary" path="./content/hst-ppp_93-1_18-02.jpg" mime="image/jpeg"> <technicalMetadata> . . Technical metadata for the primary file for the second object will go here . </technicalMetadata> </file> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-02-thumb.jpg" mime="image/jpeg"/> <imageTiles path="./opa-rendition/image-tiles/hst-ppp_93-1_18-02_jpg"/> </object>

<object id="3"> <designator>3</designator> <file type="primary" path="./content/hst-ppp_93-1_18-03.jpg" mime="image/jpeg"> <technicalMetadata> . . Technical metadata for the primary file for the third object will go here . </technicalMetadata> </file> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-03-thumb.jpg" mime="image/jpeg"/> <imageTiles path="./opa-rendition/image-tiles/hst-ppp_93-1_18-03_jpg" type="DZI"/> </object></objects>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects?annotations=summary&

format=json

{ "@version":"OPA-OBJECTS-1.0", "object":[ { "@id":"1", "@hasAnnotations":"true", "designator":"1", "file":{ "@type":"primary", "@path":"./content/hst-ppp_93-1_18-01.jpg",

27


"@mime":"image/jpeg", "technicalMetadata":{ "createDate":"2012-10-22T03:26:34Z", "metadataDate":"2013-02-20T02:55:27Z", "modifyDate":"2013-02-20T02:55:27Z", "size":"1142680", "mime":"image/jpeg", "dimensions":{ "@width":"2181", "@height":"2781" }, "resolution":{ "@x":"300", "@y":"300", "@units":"pixels/inch" }, "bitsPerSample":"8 8 8", "photometricInterpretation":"RGB", "orientation":"horizontal", "samplesPerPixel":"3", "planarConfiguration":"chunky", "colorSpace":"sRGB", "compression":"uncompressed", "software":"Adobe Photoshop CS6 (Windows)" } }, "thumbnail":{ "@path":"./opa-rendition/thumbnails/hst-ppp_93-1_18-01-thumb.jpg", "@mime":"image/jpeg" }, "imageTiles":{ "@path":"./opa-rendition/image-tiles/hst-ppp_93-1_18-01_jpg" }, "annotations":{ "comments":{ "@comments":"3", "@replies":"2", "@total":"5", "comment":[ { "@id":"19328", "@user":"kmartin", "@created":"2014-02-10T14:36:23", "@lastModified":"2014-02-10T14:40:12", "text":"One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or

if there was real affection between the two.", "replies":{ "reply":[ { "@id":"45239", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-02-10T15:00:03", "text":"True, but from different generations." }, { "@id":"45422", "@user":"mkoneni", "@created":"2014-03-08T04:12:52", "@lastModified":"2014-03-08T04:18:02", "text":"I'm sure they ran into each other a bunch, you know at charity

functions and golf tournaments..." } ] } }, { "@id":"19102", "@user":"tjefferson", "@created":"2014-03-08T04:12:52", "@isDeleted":"true"

28


}, { "@id":"17024", "@user":"rbarnes", "@created":"2014-03-09T11:14:27", "text":"Seems rather formulaic to me. I bet a staff member wrote it." } ] }, "tags":{ "@total":"3", "tag":[ { "@text":"Truman Post President", "@user":"jdoe", "@created":"2014-02-21T16:22:58" }, { "@text":"McGovern Letters", "@user":"agullet", "@created":"2014-02-27T18:23:47" }, { "@text":"nara:president=Truman", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-03-19T07:07:42" } ] }, "transcription":{ "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", "lockedBy":{ "@id":"gwashington", "@when":"2014-02-22T10:08:41" }, "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Thomas Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"mstewart", "@fullName":"Meredith Stewart", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" }, { "@id":"kmartin", "@lastModified":"2014-02-02T08:44:12" } ] }, "text":"GEORGE McGOVERN\n United States Senate\n Washington, D.C. 20510\n\n

August 19, 1972\n\n The Honorable Harry S. Truman\n Independence, Missouri\n\n My dear Mr. President:\n\n It is an honor for me, as our party's\n candidate for President of the United States. . ."

}, "translations":{ "@total":"3", "translation":[ { "@code":"ita", "@lastModified":"2014-02-21T16:22:58",

29


"@isLocked":"true", "@isauthoritative":"false", "@version":"232", "lockedBy":{ "@id":"gwashington", "@when":"2014-02-22T10:08:41" }, "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Tomßs Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"gwashington", "@fullName":"Giorgio Washington", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" } ] }, "text":"George McGovern \n\n Senato degli Stati Uniti \n

Washington, D.C. 20510 \n\n 19 ago 1972 \n\n L'Onorevole Harry S. Truman \n Independence,Missouri \n\n Mio caro Signor Presidente: \n\n E 'un onore per me, come il nostro partito di \n candidato alla Presidenza degli Stati Uniti. ."

}, { "@code":"spa", . . Full information on the Spanish translation goes here . }, { "@code":"deu", . . Full information on the Spanish translation goes here . } ] } } }, { "@id":"2", "designator":"2", "file":{ "@type":"primary", "@path":"./content/hst-ppp_93-1_18-02.jpg", "@mime":"image/jpeg", "technicalMetadata":{ . . Technical metadata for the primary file for the second object will go here . } }, "thumbnail":{ "@path":"./opa-rendition/thumbnails/hst-ppp_93-1_18-02-thumb.jpg", "@mime":"image/jpeg" }, "imageTiles":{ "@path":"./opa-rendition/image-tiles/hst-ppp_93-1_18-02_jpg" } }, { "@id":"3", "designator":"3", "file":{ "@type":"primary",

30


"@path":"./content/hst-ppp_93-1_18-03.jpg", "@mime":"image/jpeg", "technicalMetadata":{ . . Technical metadata for the primary file for the second object will go here . } }, "thumbnail":{ "@path":"./opa-rendition/thumbnails/hst-ppp_93-1_18-03-thumb.jpg", "@mime":"image/jpeg" }, "imageTiles":{ "@path":"./opa-rendition/image-tiles/hst-ppp_93-1_18-03_jpg", "@type":"DZI" } } ]}

2.3.1.3 annotations – /iapi/v1/contributions/summary/id/<NAID>

Use this URL to get summary information on all of the annotations for the specified NAID.

This will include all of the tags (with metadata) and all of the comments (with metadata), plus all summary information on the existence of transcriptions and translations.

See section 4.1.5 for more details.

2.3.1.3.1 comments – /iapi/v1/id/<NAID>/comments

Use this URL to access information on comments attached to an archival description. Within this path, you can access three types of data:

comments – The list of all comments and replies attached to the description

o This includes all comment and reply metadata (creator, creation date, etc.)

comments/<comment-id> - A single comment with all replies

o This includes all comment and reply metadata (creator, creation date, etc.)

comments/<reply-id> - A single reply to a single comment

o This includes all reply metadata (creator, creation date, etc.)

More information on comments can be found in section 4.3.

2.3.1.3.2 tags – /iapi/v1/id/<NAID>/tags

Use this URL to access tags attached to an archival description. Within this path you can access two types of information:

tags – The list of all tags attached to the description

o These are sorted alphabetically

o Each tag is returned with metadata such as who created the tag, the date of the tag, etc.

31


tags?text=<tag-text> – Retrieves information about a single tag

o This includes all tag metadata (creation date, creator, etc.)

More information about tags can be found in section 4.4.

2.3.1.4 content – /iapi/v1/media/<NAID>/

The content directory holds content files which can be downloaded. Content files include:

Scanned images of physical items

o For example, scans of documents, posters, maps, etc.

Digital renditions of analog content

o For example, digital video of news reels, digital audio of content on reel-to-reel analog audio tape

Copies or renditions of born-digital content

o For example, E-mail files in XML, MS-Word or MS-Excel documents, digital photographs from digital cameras, digital video from digital camcorders, digital audio

Copies or renditions of database files

o Much content in NARA is in the form of large tables of information from Government computer systems.

o These tables (sometimes in various formats) can be downloaded in bulk for off-line analysis.

Technical documentation

o Usually PDFs, often from paper documents, which describe the content

How do I get the Full Path to a Content File?

After the content path, you must specify a particular file or path to a file in order to fetch a file from Catalog. For example:

GET https://catalog.archives.gov/iapi/v1/media/7226539/content/hst-ppp_93-1_18-03.jpg

Although rare, it’s possible for files to be embedded inside sub-directories within a content directory:

GET https://catalog.archives.gov/iapi/v1/media/7226539/content/images/hst-ppp_93-1_18-03.jpg

The list of available content files can be found by accessing the NAID inventory of objects. For example:

32


GET https://catalog.archives.gov/iapi/v1/id/7226539/objects

Will return a list of all of the digital objects in the “72265639” package, with all of the various renditions for each. See section 2.3.1.2 above for more information.

Alternatively, you can directly search for all digital objects in NARA using https://catalog.archives.gov/iapi/v1?action=search. Search API results will contain URLs to the content files, thumbnails, etc.

See section 3 below for more information on the search API.

2.3.1.4.1 Content File Annotations

Every file in Catalog can be annotated with tags, comments, transcriptions or translations. While not all annotations are appropriate for every digital object, the API makes no distinction (any type of diogital object can be annotated – note: this may change in a future version).

Digital object annotations can be downloaded, created, and updated by adding the appropriate suffix to the end of the content file path:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>/objects/<obj-id>

See section 4.1.5 for more information on annotations summary information.

GET https://catalog.archives.gov/iapi/v1/id/<naid>/objects/<obj-id>/comments

See section 4.3 for more information on comments.

GET https://catalog.archives.gov/iapi/v1/id/<naid>/objects/<obj-id>/tags

See section 4.4 for more information on tags.

GET https://catalog.archives.gov/iapi/v1/id/<naid>/objects/<obj-id>/transcriptions

See section 4.5 for more information on transcriptions.

GET https://catalog.archives.gov/iapi/v1/id/<naid>/objects/<obj-id>/translations

See section 4.6 for more information on translations.

2.3.1.5 opa-renditions – /iapi/v1/media/<NAID>/opa-renditions

The “opa-renditions” path contains additional renditions of the content for use by Catalog user interfaces (and also available for other interfaces as well). These renditions are created by Catalog content processing system from the original content.

The renditions include:

GET https://catalog.archives.gov/iapi/v1/media/<NAID>/opa-renditions/thumbnails/...

Holds thumbnail images of image files in the content directory

Later releases may contain thumbnail images of PDFs and video files as well.

33


GET https://catalog.archives.gov/iapi/v1/media/<NAID>/opa-renditions/image-tiles/...

Holds image tiles for lower-bandwidth browsing of very-high resolution photographs. Image tiles can be used by the OpenSeadragon tool.

How do I get the Full Path to an OPA Rendition?

The full paths for OPA renditions of content files can be acquired in two ways:

Get the list of all objects for an OPA package (section 2.3.1.2).

Perform a search with the search API for content files (section 3).

o The thumbnail and tiles filenames are available as fields in the search results.

2.4 Authority Records – /iapi/v1/idAuthority records contain authoritative information about people and organizations. “Authoritative information” usually means the official name, alternative names, dates, and description of a person or an organization.

Authority records can be accessed by type (“people”, “organizations”) and by ID.

How do I Get a Person ID or Organization ID?

There are two methods for getting a person or organization ID:

Authority Record IDs are available in the archival description (section 2.3.1.1)

o For example, in section 2.3.1.1 the @donor-id attribute references a person ID.

You can use the Search API to search for authority records (section 3)

o The search fields will contain authority record ID and authority record type

2.4.1 People – /iapi/v1/id/<person-id>

Authority record metadata and annotations for people can be directly accessed from Catalog using the person ID.

Accessing this URL directly will fetch the following for the person:

<person> – The person metadata (section 2.4.1.1 below)

<annotations> – All annotations (comments and tags) on the record (section 2.4.1.2 below).


<authority type="person" id="1090294"> <person>

34

http://openseadragon.github.io/


. . Authority record metadata on the person goes here, see section 2.4.1.1 . </person> <annotations> . . Annotations attached to the person go here, see section 4.1.5. . </annotations></authority>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/1090294&format=json

{ "@type":"person", "@id":"1090294", "person":{ . . Authority record metadata on the person goes here, see section 2.4.1.1 . }, "annotations":{ . . Annotations attached to the person go here, see section 4.1.5. . }}

2.4.1.1 Person Metadata

To fetch just the person metadata, use <person-id>/person to access the information.

For more information about the metadata returned for people authority records returned by Catalog, please see [TBD – is there documentation on this XML format from NARA?].

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/1090294/person

<person> <person-id>1090294</person-id> <display-name>Seward, William Henry, 1801-1872</display-name> <preferred>Y</preferred> <date-of-birth>1801</date-of-birth> <date-of-death>1872</date-of-death> <source-note>His Alaska Speech ... Aug. 1869NUCMC data from NJ Hist. Soc. for Congar, H.N. Papers, 1863-1905 (William H. Seward, Sec. of

State)WwWA, 1607-1896 (Seward, William Henry, 1801-1872; U.S. Sen., Sec. State; s. Dr. Samuel & Mary

(Jennings) Seward; m. Frances Miller; law practice & res., Auburn, N.Y.; gov. N.Y., 1838-42; memb. U.S.Sen., 1849-1868; sec. state under Lincoln; negot. Alaska purchase, 1867)

Dict. of U.S. history, 1931 (Seward, William Henry, 1801-1872; gov. N.Y. 1839-1843)</source-note> <created-timestamp>11/3/2013 23:18:48</created-timestamp>

<index-character>s</index-character> <related>11</related> <donations>0</donations> <creations>0</creations> <contributions>4</contributions> <subject-of>7</subject-of> <staff-member>1</staff-member></person>

35


JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/1090294/person?format=json

{ "person-id":"1090294", "display-name":"Seward, William Henry, 1801-1872", "preferred":"Y", "date-of-birth":"1801", "date-of-death":"1872", "source-note":"His Alaska Speech ... Aug. 1869\nNUCMC data from NJ Hist. Soc. for Congar, H.N.

Papers, 1863-1905 (William H. Seward, Sec. of State)\nWwWA, 1607-1896 (Seward, William Henry, 1801-1872; U.S. Sen., Sec. State; s. Dr. Samuel & Mary (Jennings) Seward; m. Frances Miller; law practice & res., Auburn, N.Y.; gov. N.Y., 1838-42; memb. U.S.Sen., 1849-1868; sec. state under Lincoln; negot. Alaska purchase, 1867)\nDict. of U.S. history, 1931 (Seward, William Henry, 1801-1872; gov. N.Y. 1839-1843)",

"created-timestamp":"11/3/2013 23:18:48", "index-character":"s", "related":"11", "donations":"0", "creations":"0", "contributions":"4", "subject-of":"7", "staff-member":"1"}

2.4.1.2 People Annotations

Tags and comments can be accessed, added, removed, or updated to any person authority record, using the following paths:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<person-id>


GET https://catalog.archives.gov/iapi/v1/id/<person-id>/comments

See section 4.3 for more information.

GET https://catalog.archives.gov/iapi/v1/id/<person-id>/tags

See section 4.4 for more information.

2.4.2 Organizations – /iapi/v1/id/<org-id>

Authority records for organizations can be accessed directly from Catalog with the specified path.

Accessing this URL directly will fetch the following for the organization:

<organization> – The organization metadata (section 2.4.2.1 below)

<annotations> – All annotations (comments and tags) on the record (section 2.4.2.2 below).


<authority type="organization" id="142876">

36


<organization> . . Authority record metadata on the organization goes here, see section 2.4.2.1 . </organization> <annotations> . . Annotations attached to the organization go here, see section 4.1.5. . </annotations></authority>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/1090294&format=json

{ "@type":"organization", "@id":"1090294", "organization":{ . . Authority record metadata on the organization goes here, see section 2.4.2.1 . }, "annotations":{ . . Annotations attached to the organization go here, see section 4.1.5. . }}

2.4.2.1 Organization Metadata – /iapi/v1/id/<org-id>/organization

When you need just the organization metadata (and not the annotations), you can fetch it with “id/<org id>”.

For more information about the metadata returned for organization authority records returned by OPA, please see [TBD – is there documentation on this XML format from NARA?].

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/142876/organization

<organization> <organization-id>142876</organization-id> <display-name>Canadian Meteorological Service.</display-name> <preferred>Y</preferred> <source-note>CREATED FROM LCNAF CONVERSION</source-note> <created-timestamp>11/4/2013 17:24:57</created-timestamp>

<index-character>c</index-character> <related>0</related> <organization-names> <organization-name org-name-id="142876"> <name>Canadian Meteorological Service.</name> <related>0</related> <donations>0</donations> <creations>0</creations> <contributions>0</contributions> <subject-of>0</subject-of> <predecessors> <predecessor num="1" predecessor-id="1055883" standard="Y"> <predecessor-display-name>Canada. Meteorological Branch</predecessor-display-name> </predecessor> </predecessors>

37


<successors> <successor num="1" successor-id="1055884" standard="Y"> <successor-display-name>Canada. Atmospheric Environment Service</successor-display-name> </successor> </successors> </organization-name> </organization-names></organization>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/142876/organization?format=json

{ "organization-id":"142876", "display-name":"Canadian Meteorological Service.", "preferred":"Y", "source-note":"CREATED FROM LCNAF CONVERSION", "created-timestamp":"11/4/2013 17:24:57", "index-character":"c", "related":"0", "organization-names":{ "organization-name":{ "@org-name-id":"142876", "name":"Canadian Meteorological Service.", "related":"0", "donations":"0", "creations":"0", "contributions":"0", "subject-of":"0", "predecessors":{ "predecessor":{ "@num":"1", "@predecessor-id":"1055883", "@standard":"Y", "predecessor-display-name":"Canada. Meteorological Branch" } }, "successors":{ "successor":{ "@num":"1", "@successor-id":"1055884", "@standard":"Y", "successor-display-name":"Canada. Atmospheric Environment Service" } } } }}

2.4.2.2 Organization Annotations – /iapi/v1/id/<org-id>/annotations

Tags and comments can be accessed, added, removed, or updated to any organization authority record, using the following paths:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<org-id>


GET https://catalog.archives.gov/iapi/v1/id/<org-id>/comments

See section 4.3 for more information

GET https://catalog.archives.gov/iapi/v1/id /<org-id>/tags

See section 4.4 for more information

38


2.5 User Information – /iapi/v1/accountsInformation on users can be downloaded directly from Catalog with the user login ID (for example “kmartin”).

Note: Much of what is available within the user URLs will vary depending on who is logged in.

Users who are logged will have more complete access to their own user information than they will for other users.

The following URLs are available to acquire user information:

GET https://catalog.archives.gov/iapi/v1/accounts/profile/<username>

User account information. See section 5.6 for details.

GET https://catalog.archives.gov/iapi/v1/exports/auth

Information on bulk exports either in progress or being created for the user. See section 5.4 for details.

GET https://catalog.archives.gov/iapi/v1/contributions/summary

Information on contributions made by the user. See section 5.2 for details.

GET https://catalog.archives.gov/iapi/v1/contributions/comments?username=<username>

Information on contributions made by the user. See section 5.2.3 for details.

GET https://catalog.archives.gov/iapi/v1/contributions/tags?username=<username>


GET https://catalog.archives.gov/iapi/v1/contributions/transcriptions?username=<username>


GET https://catalog.archives.gov/iapi/v1/ contributions/translations?username=<username>


GET https://catalog.archives.gov/iapi/v1/lists/view

Information on lists of search results saved by the user. Section 5.3 for details.

GET https://catalog.archives.gov/iapi/v1/accounts/notifications

Information on notifications for the user. Currently, notifications include events such as replies, edits to transcriptions, translations, and moderator actions which concern the user. Section 5.5 for details.

39


3 Search API

The heart of the OPA system is a powerful search engine capable of searching over all of the archival descriptions, on-line content and authority records which are available from Catalog.

Searching

The Search API can be used to search over different data sources:

To search over everything:

GET https://catalog.archives.gov/iapi/v1?action=search&parameters...

To search over holdings: (archival descriptions and digital objects)

GET https://catalog.archives.gov/iapi/v1?action=search&source=holdings&parameters...

To search over authority records:

GET https://catalog.archives.gov/iapi/v1?action=search&source=authorities&parameters...

To search over web pages:

GET https://catalog.archives.gov/iapi/v1?action=search&source=web&parameters...

Exports

The Search API can also be used to create exports of search results:

POST https://catalog.archives.gov/iapi/v1?action=export&parameters...

Building Lists

Finally, the Search API can also be used to save results to lists.

To save results to a new list:

POST https://catalog.archives.gov/iapi/v1/lists/create?listname=<list-name>&parameters...

To add results to an existing list:

POST https://catalog.archives.gov/iapi/v1/lists/add/<list-name>?parameters...

40


3.1 OverviewWith the search API, you specify query terms, filters, and the scope of the search as parameters, and the search engine will return results. Each result will contain metadata fields about the item.

In addition to results, the search API can also do the following:

Highlights – These are highlighted search keywords found in the field content.

Facets – These are histograms of field metadata values (for certain fields only) which can be used to sub-set the results (also called “guided navigation”).

Save results for bulk export – This allows you to bulk export metadata and content from Catalog into compressed files (tar.gz) which can be efficiently downloaded.

o Bulk exports are processed in the background.

Save results to list – Results can be saved to a list to be individually examined later.

One step search and process – Allows you to perform any of the access or annotation functions on the first result returned by a search

3.1.1 Search Results Entries

The following can be returned as individual entries in the search results:

Archival Descriptions – Descriptive metadata about every collection, record group, series, file unit, and item available from NARA.

Authority Records – Authoritative information about people and organizations which may be referenced by archival descriptions.

Digital Objects – Files which represent the on-line content (and technical documentation about the on-line content) can be individually retrieved using the Search API.

How Do I: Search for Archival Descriptions?

Filter on type = “description”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description

How Do I: Search for Digital Objects?

Filter on type = “objects”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=object

41


How Do I: Search for Both?

Filter on type = “description,object”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description,object

OR, filter on source = “holdings”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&source=holdings

OR, restrict your scope to holdings:

GET https://catalog.archives.gov/iapi/v1/holdings?action=search&q=truman

How Do I: Retrieve all Digital Objects for a Description?

Filter on naid=<ID of the description> and type=object:

GET https://catalog.archives.gov/iapi/v1?action=search&type=object&naid=7283924

How Do I: Search for Authority Records?

Filter on source = “authorities”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&source=authorities

OR, restrict your scope to “authorities”:

GET https://catalog.archives.gov/iapi/v1/authorities?action=search&q=truman

How Do I: Search for everything?

Have no filter at all:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman

How Do I: Search for everything, but return web pages separately?

Have no filter, but group by descriptions and web pages:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&tabType=web

3.1.2 Annotations

Individual annotations are not indexed as separate entries in the search engine index, but instead are indexed with the item (Description, Authoriy Record or an object) to which they are attached.

Since the search engine can return descriptions, digital objects, authority records, and web pages as search results, annotations will be returned as fields inside each search result.

42


The search API can:

Locate all descriptions, digital objects, or authority records which contain:

o A particular tag

o Specified query terms in any tag or tags

o Any tag created by a specified user ID

o Specified query terms in any comment or reply

o Comments or replies by a specific user ID

Locate all digital objects which contain:

o Specified query terms in a transcription or translation

o Transcriptions or translations originated by a particular user ID

o Transcriptions or translations edited by a particular user ID

The search API cannot:

List all annotations by a particular user ID

o Use the contributed comments API for comments by user, section 5.2.3.

o Use the contributed tags API for tags by user, section 5.2.4.

o Use the contributed transcriptions API for transcriptions by user, section 5.2.5.

o Use the contributed translations API for translations by user, section 5.2.6.

o However, the search API can return a list of descriptions, digital objects, or authority records which contain a contribution by a particular user ID.

List all comments or replies which contain specified query words

o However, the search API can return a list of descriptions, digital objects, or authority records which contain the specified query words in their comments.

List all tags which contain specified query words

o However, the search API can return a list of descriptions, digital objects, or authority records which contain the specified query words in their tags.

List all unique tags

o Tags can be returned by person, description, digital object, or authority record.

Use the tagging API to get tags for descriptions, digital objects, or authority records, see section 4.4.

o A single list of all available tags is not anticipated for Catalog production (first release)

43


List all comments

o Comments can be returned by person, description, digital object, or authority record.

o A single list of all comments is not anticipated for Catalog production

3.1.3 Search Functions

The Catalog search API provides a number of capabilities:

High-quality relevancy ranked keyword text search of content and selected metadata fields

Keyword text search of any public, indexed, metadata field

o Some fields which are highly transient in nature (e.g. counts) may not be indexed and searchable [TBD]

Computing a ‘relevancy score’ for all search results, based on how well the document matches the query

Boolean queries with AND, OR, and NOT

Range searches such as locating documents with a specified date or integer range

Filtering results based on keywords found in metadata or content fields

Specifying fields to return, in addition to default fields

o Fields may also contain URLs to other pages, such as content detail pages, content, thumbnails, etc. as appropriate

Faceting over metadata fields

o Faceting provides a histogram of the count of search results which contain specified field values

o Faceting is available over Data Source, Level of Description, Type of Materials, File Format, Location, and Date

Sorting of search results

o The default sort order is by relevancy in descending order

o Some fields (as specified in the DMD) may also be used for sorting

o Sorting can be done over multiple fields as well as relevancy with the same API call (e.g., primary sort, secondary sort, etc.)

Grouping of search results

o Several different methods are available

Suggested query terms from a Catalog thesaurus

Highlighting of query term matches within the content

44


Return of “best bets” or “featured holdings” which match a specified query

o These special results are also referred to as “search optimization”

Search results list pagination, including returning a specified page or range of results

Result set management (aka “lists”) - requires login

o Save a search results list (up to a system defined maximum) to a named list in the user’s account.

o For more information, see section 5.3

Bulk export of search results (requires login)

o May include digital objects associated with the search results as well

One-step “Search and Process”

o To download or annotate the first result returned by a search

3.2 Search API ParametersAll searches provided by the search API begin with one of the following URLs:

GET https://catalog.archives.gov/iapi/v1?action=search&search parameters. . .

GET https://catalog.archives.gov/iapi/v1/holdings?action=search&search parameters. . .

GET https://catalog.archives.gov/iapi/v1/authorities?action=search&search parameters. . .

GET https://catalog.archives.gov/iapi/v1/web?action=search&search parameters. . .

Parameters for the search API are given in the following table.

Parameter Description Examplesaction=search|

searchTag|searchWithin|contentDetail

Specifies the type of search to do and what to do with the search results. search – Perform a standard search and return

the results searchTag – Perform a search with the supplied

tag searchWithin – Perform a search for all records

which are descendants of a description. contentDetail – Perform a standard search and

return the results and the content detail xsl for the first result.

action=search

action= searchTag

action= searchWithin

45


Parameter Description Examplessource=source1,

source2,source3, . . .

Specifies what major sources to search. Sources can be any of: authorities – Authority records for people and

organizations. holdings – Descriptions and digital objects. web – Archives.gov web pages and presidential

library web pages.If no “source” parameter is specified, Catalog will search over all sources.

source=holdingssource=authorities,

holdings

type=type1,type2,type3, . . .

Specifies a list of result types to search. Types can be any of: description – All archival descriptions object – Digital objects (files) person – People authority records organization – Organization authority records archivesWeb – Web pages from archives.gov. presidentialWeb – Web pages from presidential

libraries.If no “type” parameter is specified, Catalog will search over all types.

type=description

type=description,object,archivesWeb

naIds=naId1, naId2, naId3 . . .

Specifies a list of NAIDs to search over. naIds=7238423,536932,4562889,3552324

opaIds=opaId1, opaId2, opaId3. . .

Specifies a list of OPA-IDs to search over. opaIds=desc-7238423,person-98824,org-889242,desc-7238423/32

format=json|xml Specify the format of the results, either “json” or “xml”. Default is “xml”

format=jsonformat=xml

Simple Keyword Queries

q=query expression Keyword search over the full-text content (and selected full-text fields such as title).Full query expressions can be used as the value for ‘q’. See section 3.3 for more details.The ‘q’ parameter is optional. When missing, the search results will be based entirely on the filter parameters below.The exact list of fields which will be searched by the ‘q’ parameter will be different for each different type of content source, but usually includes the full content of the item plus high-quality full text fields such as ‘title’, ‘description’, etc.See the appropriate Data Model Definition (DMD) document for each source to identify exactly what fields are searched by ‘q’.

q=Navy

q=Kennedy and NASA

q=kennedy missile crisis

q= johnson and space and (nasa or "space agency")

46


Parameter Description ExamplesParametric Searches / Filter Queries

filter=query expression

Filters will restrict the scope of the search results to data which is in the specified fields.The bare “filter” parameter allows you to enter any arbitrary filter query expression including field: operators.The full query expression language can be found in section 3.3.

filter=title:kennedy and level:(series or fileUnit)

f.keywordField=query expression

f.keywordField is used to restrict results for keyword fields. It can be used for any type of “keyword” field specified in section 3.4.The query expression can be any expression specified in section 3.3.1.

f.title=JFK

f.title=campaign and (JFK or LBJ)

f.stringField=query expression

f.stringField is used to restrict results for string fields. It can be used for any type of “string” field specified in section 3.4.“string” fields differ from keyword fields in that individual words within the field cannot be searched. The entire string value must be specified. String fields are typically for identifiers and when searching for exact tag values.The query expression can be any expression specified in section 3.3.2.

f.opaId=desc-72232422

f.opaId=desc-72232422 or desc-83343533

f.dateField=date expression

f.dateField is used to restrict results based on a date or date range. It can be used for any type of date field specified in section 3.4.See section 3.3.3 for date expressions available to search date fields.

f.inclusiveStart= range(1900,1999)

f.inclusiveStart=2000

f.inclusiveStart= range(1980-01-01, 1980-06-30)

f.intField=int expression

f.intField is used to restrict results based on an integer or integer range. It can be used for any type of integer field specified in section 3.4.Integer expressions can use operators specified in section 3.3.4.

f.inclusiveStartYear= range(1900,1999)

f.inclusiveStartYear=2000

f.boolField=true/false

f.boolField is used to restrict results for boolean fields. It can be used for any type of boolean field specified in section 3.4.Only “true” or “false” can be specified for boolean filter fields.

f.hasOnlineContent=true

f.hasChildren=false

47


Parameter Description Examplesf.fieldXml.tags=

query expressionf.fieldXML.tags is a special expression used to search XML content. In this parameter, replace “fieldXml” with an XML field (see sections 3.4.7 or 3.4.8) and then replace “tags” with one or more XML tags separated by periods.For example: f.description.variant-number

o Searches all <variant-number> tags in the description.

f.description.creator.creator-typeo Searches all <creator-type> tags found

within <creator> tags in the description

f.description.variant-number="NWDNS-179-WP-1565"

Controlling the Output

resultFields=field1, field2,field3, . . .

List of fields to return in the result set.If a field is given that cannot be put in the return (such as a field that does not exist), an error is returned.Content within XML fields can be individually returned by specifying the tag (or sub-tag or attribute) with “field.tag.tag” notation.XML fields will be returned either as JSON or XML, depending on the setting of the “format” parameter.If no fields are specified, the following fields are returned automatically: source, type, naId, opaId, url, accessPath, localId, hmsEntryNumbers, containerId, iconType, title, titleDate, parentLevel, parentTitle, creators, and thumbnailFile.Note: If resultFields is missing, all fields in the index entry are returned.

resultFields=title,url

resultFields=description

resultFields=description.subject-reference.display-name

resultFields=title,url,objects.object.annotations.transcription

highlight=true/false Turn on highlighting of query term matches. highlight=true

highlight.fields=field1, field2, . . .

If highlighting is on, this parameter identifies which fields to highlight.If an invalid field name is used, an error is returned.If no fields are specified and highlight=true, highlights will be returned for “content”, “title”, and “creator”.Only fields identified as “keyword” in section 3.4 can be specified as a highlight field.

highlight.fields=title

highlight.fields=title, content, creators

48


Parameter Description Examplesgroup=groupType Indicates if the results should be grouped or not,

and how the results should be grouped. Values allowed are: web – Group web results. Leave all other results

as un-grouped.If not specified, a single flat list of results is returned.

group=web

sort=field1 [asc|desc],field2 [asc|desc],. . .

Sorts results based on field values. Specify the field to sort by followed by “asc” for ascending or “desc” for descending. Use the special field “score” (only available in sorting) to sort by relevancy.If not specified, the default sort is “score desc”.If a field name is given that cannot be sorted, an error is returned. If an invalid sort type is given, an error is returned.See section 3.4 for fields which can be used for sorting.

sort=localId asc

sort=localId asc, score desc

rows=# Indicates the number of search results to fetch. The default is 25.Note that different maximum values for “rows” may apply depending on what is done with the search results. [All values below TBD] For simple searches, “rows” is limited to 200. For saving search results to a list (see section

5.3), “rows” is limited to 1000. For exporting bulk results, “rows” is limited to

10000.

rows=50

offset=# The zero-based offset into the search results of the first result to be returned. This parameter is used in combination with ‘rows’ to paginate the results. If missing, the default offset is 0.If an offset less than 0 or greater than the maximum [Maximum TBD – based on type of user] is specified, an error is returned.

offset=100

facet=true/false Enable faceting so that counts are returned. Facets are essentially histograms of the values which occur in specified fields.

facet=true

49


Parameter Description Examplesfacet.fields=field Specifies the list of fields for which facet values are

returned.If not specified and facet=true, then facets will be automatically returned for source, type, level, fileFormat, materialsType, dateRangeFacet, and locationIds.If a field is given that cannot be used for faceting, an error is returned. See section 3.4 for fields which can be used for faceting.

facet.fields=level,fileFormat

facet.stringField=query expression

facet.stringField is for filtering results by facet values. It operates exactly the same as f.stringField, but is provided with a separate parameter name to help clients distinguish between facet filters and advanced search filters.

facet.location=Lyndon%20Baines%20Johnson%20Library

Query Expansions

thesaurus=true/false

Turn on the use of the system thesaurus for query term expansion.Note that term expansions are only applied to keywords specified with the “q” parameter.

thesaurus=true

thesaurus.terms=cat1, cat2, cat3, . . .

Add all terms from the specified categories into the query. Allowed category values are: all, related, narrower, broader.

thesaurus.terms=related, narrower

thesaurus.terms.term=cat1, cat2, cat3, . . ., term1, term2, term3 . . .

Specifies both categories and individual terms to add for the specified query term.Allowed category values are: all, related, narrower, broader. Individual terms can be any keyword, but are typically those returned by the system thesaurus in response to a previous query.If the “term” does not exist as a keyword in the “q” parameter, an error will be returned.

thesaurus.terms.ship=related, narrower

thesaurus,terms.ship=transport,steamboat,tanker,warship,yacht,anchor

Saved Result Lists (only available for registered users after login.)

list=listName

Specifies the name of the list to which results are saved (if action=newList) or added (if action=addList).If action=newList and the name already exists, an error is returned.Note: Only available for registered users after login.[TBD – to preserve system resources, the maximum number of lists is limited to XXX for registered users and YYY for ‘power’ users]See section 5.3 for more details.

list=MyList2

Export ParametersSee section 3.6 for all export parameters and more information.

50


Examples

Simple search:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman

Boolean query search:

GET https://catalog.archives.gov/iapi/v1?action=search&q=(bess or harry) and truman

Boolean query search with phrases:

GET https://catalog.archives.gov/iapi/v1?action=search&q="president truman" or "harry s truman"

Search over everything from a particular location:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&f.location="Lyndon Baines Johnson Library"

Find all titles with “truman”:

GET https://catalog.archives.gov/iapi/v1?action=search&f.title=truman

Search for “truman”, filter by inclusive-start date between 1980 and 1990:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&f.inclusiveStartDate=range(1980,1990)

Search over descriptions returning just the archival description XML:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&resultFields=description

Search over posters (subject-reference from the archival description contains “poster”):

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&f.description.subject-reference.display-name=poster&resultFields=description

Find all descriptions tagged with “Kennedy Vietnam”

GET https://catalog.archives.gov/iapi/v1?action=search&f.tagsExact="Kennedy Vietnam"&type=description

Find all digital objects tagged with any tag that contains “kennedy”:

GET https://catalog.archives.gov/iapi/v1?action=search&f.tagsKeywords=kennedy&type=object

Note: Note that there are 2 different fields to search for tags. When an exact match is desired, use “tagsExact” and use “tagsKeywords” field otherwise.

Find documents which contain the key word “truman” and facet on locations (return a histogram of location IDs in the search results) and description level:

51


GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&facet=true&facet.field=locationIds,level

Find documents which contain the key word “truman” and highlight matches on “truman” in the title and content fields:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&highlight=true&highlight.fields=title,content

Find documents which contain the key word “truman” and sort by inclusive start date, descending and then sub-sort on the score (descending):

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&sort=inclusiveStartDate desc,score desc

Return the first 10 results which contain the key word “truman”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&rows=10

Return the results 11-20 which contain the key word “truman”:

GET https://catalog.archives.gov/iapi/v1?action=search&q=truman&rows=10&offset=10

Return the results for “ship” and “dock” and return thesaurus expansions for the words:

GET http://catalog.archives.gov/iapi/v1?action=search&q=ship dock&thesaurus=true

Return the results for “ship” and “dock” with “ship” expanded to include “steamship” and “warship” and “dock” expanded to include braoder terms:

GET https://catalog.archives.gov/iapi/v1?action=search&q=ship dock&thesaurus.terms.ship=steamship,warship&thesaurus.terms.dock=broader

Save the first 100 results to a list called “Mary’s List”:

POST https://catalog.archives.gov/iapi/v1/lists/add/Mary's%20List?action= addToListFromSearch&q=truman&offset=0&rows=100

Add results 10 to 20 to the list called “Mary’s List”:

POST

https://catalog.archives.gov/iapi/v1/lists/add/Mary's%20List?action= addToListFromSearch&q=truman&offset=10&rows=10

Search all children within the series with NAID = 1523922:

POST https://catalog.archives.gov/iapi/v1?action=search&q=truman&ancestorNaids=1523922

3.3 Query ExpressionsThis section covers the operators which are available to construct query expressions in Catalog.

52


3.3.1 Keyword Fields

The following operators are available for searching all keyword fields:

Query Operator Description Examplesimplied and Any time an operator is missing, it is an

implied and. Both of the words or sub-expressions specified must exist in the returned results.

Katyn massacre

(john or jack) kennedy

and Return results that have both of the words or sub-expressions specified.Can be upper or lower case “AND”.

katyn and massacre

(john or jack) AND kennedy

or Return results that have either of the words or sub-expressions specified.Can be upper or lower case “OR”.

kennedy or Johnson

(john kennedy) OR (lyndon johnson)

not Return results which do not have the word or sub-expression specified.Can be upper or lower case “NOT”.Expressions without a positive clause are not accepted in OPA at this time.

kennedy not Jacqueline

johnson NOT "lady bird"

not source:web

( expression ) Parenthetical expressions are allowed for more carefully specifying the boolean logic desired.

(john kennedy) OR (lyndon johnson)

(john or jack) AND kennedy

"term term term . . ." Phrases can be expressed with double-quotes. All terms must exist in the specified order before the item is returned.Note: Phrases work differently for string fields. See note below in section 3.3.2.

"lady bird johnson"

"world war ii"

field:expression The “field:” operator allows the construction of complex query expressions of multiple fields.The “field” specified can be any of the fields (except those identified as “no-search”) from section 3.4. The allowed “expression” is based on the field type.Note 1: Only available with the “filter” parameter. Not available for any “f.field” parameter or the “q” parameter.Note 2: See section 3.3.5 below about how to use field:expression for XML fields.

title:kennedy and level:(collection or recordGroup or series)

53


Tokenization of Keyword Fields

Text in keyword searches are automatically split into separate terms on punctuation and white space. For example, the query:

diesel-electric tank/armored-vehicle

Will be searched as:

diesel AND electric AND tank AND armored AND vehicle

3.3.2 String Fields

All operators specified above (section 3.3.1) are also available for string fields.

Not Separated by Punctuation

However, unlike keyword fields, string fields are only split on white-space. Punctuation will keep string fields together. For example, the query:

desc-132423 OR person-24239

Will be queried exactly as specified, as two terms (not 4) separated by “or”.

Strings with Embedded White Space

Some string fields may have embedded white space. In order to search on these fields, enclose the value in double-quotes:

location:"Center for Legislative Archives"

3.3.3 Date and Date-Time Fields

The following operators and options can be used to search over both date and date-time fields:

Query Operator Description ExamplesYYYY Automatically search on the entire range

specified by the year.This includes all seconds from midnight January 1 to 11:59:59 on December 31, inclusive.

recordCreatedDateTime:2014

releaseDate:1985

YYYY-MM Automatically search on the entire range specified by the year and month.This includes all seconds from midnight on the first of the month to 11:59:59pm on the last day of the month, inclusive.

recordCreatedDateTime:2014-04

releaseDate:2001-09

54


Query Operator Description ExamplesYYYY-MM-DD Automatically search on the entire range

specified by the day, from 0:00:00 to 23:59:59.recordCreatedDateTime:

2014-04-01

releaseDate:2001-09-11

YYYY-MM-DDTHH For date-time fields only, automatically search on the entire range specified by the hour.

recordCreatedDateTime:2014-04-01T10

YYYY-MM-DDTHH:MM For date-time fields only, automatically search on all seconds within the specified minute.

recordCreatedDateTime:2014-04-01T10:30

YYYY-MM-DDTHH:MM:SS

For date-time fields only, search just for the specified second.

recordCreatedDateTime:2014-04-01T10:30:45

range(date1,date2) For both date and date-time fields, specify the range of values (inclusive) over which to search.If any date value is truncated to just the Year or Year-month, the range will be expanded appropriately to cover the entire inclusive range.For example, for range(2001, 2002) the search will include all days from January 1, 2001 to December 31, 2002, inclusive.For range(2001-09, 2001-12), the search will include all days from the September 1, 2001 to December 31, 2001.

recordCreatedDateTime:range(2014-01-01, 2014-04-30)

releaseDate:range(2001, 2002)

releaseDate:range(2001-09, 2001-12)

range(dateTime1,dateTime2)

For date-time fields only, search over the specified range, inclusive.If any date value is truncated to just the Year or Year-month, the range will be expanded appropriately to cover the entire inclusive range.For example, for range(2001, 2002) the search will include all days from January 1, 2001 to December 31, 2002, inclusive.For range(2001-09, 2001-12), the search will include all days from the September 1, 2001 to December 31, 2001.

recordCreatedDateTime:range(2014-04-01T05:00:00, 2014-04-01T06:59:59)

recordCreatedDateTime:range(2014-04-01T05:30, 2014-04-01T05:59)

3.3.4 Integer Fields

Note that NAID and other identifiers are indexed as string fields by Catalog and not integer fields. (see section 3.3.2 for more information).

Integer fields can be searched either for a specific number, or for a range.

Query Operator Description Examples# Search for the number specified. fileSize:1024

55


Query Operator Description Examplesrange(#,#) Search for all numbers within the specified

range (inclusive).fileSize:range(0,1048575)

3.3.5 XML Fields

The XML fields specified in section 3.4.8 support methods for searching within the XML structure contained within those fields. This allows complex filtering of search results based on XML tag structure values.

Note: These searches are slower and use up more resources than searches on standard fields. Therefore standard fields should be used whenever possible.

Due to technical limitations, integer and date range searches are not allowed inside XML fields. Only standard fields support integer and date range searches. This limitation may be lifted in a future Catalog release.

Query Operator Description ExamplesxmlField:(expression...) Search on keywords within an XML field.

This search will find the key word inside any tag within the XML data.

description:poster

xmlField.tag:(expression…)

Search on keywords within a specific XML tag.Note 1: Any tag at any level of nesting can be specified.Note 2: The entire expression specified by (expression…) must match within the same tag. Search results will not be returned if parts of the expression match in one instance of the tag, and other parts of the expression match elsewhere.Note 3: The nested expression can contain the following operators: “and”, “or”, phrases, parenthesis, and nested field: operators for XML tags nested within the parent tag specified.Note that the NOT operator is not supported in XML fields. [TBD – restriction may be lifted in a future release]

description.creator-display:(emergency and management)

xmlField.tag.tag:(expression…)

Specify multiple tags to require nesting.For example, withdescription.reference-unit.name:(park)

only results with the keyword “park” inside the <name> tag inside the <reference-unit> tag will be returned.

description.reference-unit.name:(college park)

56


Query Operator Description ExamplesxmlField.tag.@attribute:

(terms…) Attributes can be specified using the “@” symbol for the XML tag.

description.subject-reference.@subject-id:4173619

Example of Nested Field: Operators

XML fields support complex nested field: operators. For example, to search only those variant control numbers where the specified variant type is “Search Identifier”, you might use the following expression:

f.description.variant-control-number=(variant-type:"Search Identifier" AND variant-number:(learning center))

This expression will match only descriptions which contain the specified <variant-type> and <variant-number> inside the same <variant-control-number> tag. If the type and number are within two different <variant-control-number> tags, then it will not be matched.

For example, given the above search expression, the following will be matched:

<variant-control-number mlr="false" num="3"> <variant-number>si Learning Center</variant-number> <variant-number-desc>si Learning Center</variant-number-desc> <variant-type>Search Identifier</variant-type> </variant-control-number>

But the following will not:

<variant-control-number mlr="false" num="3"> <variant-number>si Learning Center</variant-number> <variant-number-desc>si Learning Center</variant-number-desc> <variant-type>Learning Center Type</variant-type> </variant-control-number> <variant-control-number mlr="false" num="4"> <variant-number>si National History Standard Era 8</variant-number> <variant-number-desc>si National History Standard Era 8</variant-number-desc> <variant-type>Search Identifier</variant-type> </variant-control-number>

3.4 Fields This section lists fields available for search, and what functions are available for each field. The “non-XML” fields are indexed for performance and to allow certain features (such as faceting, sorting, etc.) not available with XML fields.

For more complex search use cases, you may need to use the “XML Fields” (see section 3.4.8 below), which provide search over all fields in the archival description XML, authority record XML, technical metadata XML, or content XML.

57


Field Codes

Each field is coded to identify what sorts of functions that field supports:

keyword – A keyword search field.

o Individual keywords (separated by punctuation or spaces) can be searched within the field.

o See section 3.3.1 for search expressions that this field supports.

string – A string search field.

o For these fields, only the entire field can be searched, and not individual words within the field. Usually used for identifiers, types, and for exact-tag searching.

o See section 3.3.2 for search expressions that this field supports.

date – A date search field.

o Supports date range and date search expressions from section 3.3.3.

date-time – A date-time search field.

o Supports time-based search expressions from section 3.3.3.

int – An integer field.

o Supports integer range and integer search expressions from section 3.3.4.

xml – The field contains embedded XML data.

o These fields support the parameters and search expressions specified in section 3.3.5.

sortable – The field can be used with the “sort” parameter.

no-search – The field cannot be searched. It is only used to provide information in the search results.

multi-valued – The field can have multiple values. Often used for type or category-type fields where an index entry can be any of a number of different things.

o All other fields are single-valued.

no-results – The field can only be used for search. The content of the field cannot be returned with the search results.

o All other fields return can return their content in the search results.

facet – The field can be used for the “facet” parameter.

58


3.4.1 Identifiers

The Catalog system will track the following identifiers:

naId – [string, sortable] The “NARA ID”, this is the integer identifier for all archival descriptions.

o NAID is managed by DAS and tagged on all DAS descriptions.

o Formerly known as the “ARC-ID”.

o See section 1.4.1 for more details.

personId – [string, sortable] The ID for the person authority record. This is the integer identifier that spans all person records. ((NEW))

o The person ID is managed by DAS and tagged on all DAS descriptions.

orgId – [string, sortable] The ID for the organization authority record. This is the integer identifier that spans all organization records. ((NEW))

o The organization ID is managed by DAS and tagged on all DAS descriptions.

opaId – [string, sortable] This is the identifier for all entries in the search engine index.

o A unique OPA_ID will be assigned to everything indexed into the search engine. For example: desc-5132492, person-1432, org-1342234

o This ID must be unique across all archival descriptions, and authority records.

o See section 1.4.2 for more details.

hmsEntryNumbers – [keyword, multi-valued] The “Holdings Management System” entry numbers for the record, a method for locating the physical materials from Archives II. Not available for all records.

hmsEntryNumberSort – [string, sortable] A sort-able version of the HMS Entry Number.

o TBD: This will likely be a list of all HMS numbers assigned to the item, sorted and concatenated together into a primary / secondary sortable representations.

localId – [string, sortable] The identifier that a NARA custodial unit specifies to be used to request archival materials in the unit's custody. Not available for all records.

containerId – [string] The identifier or number for the individual container storing each specific media type.

url – [keyword] Specifies the URL in the Catalog user interface where the item can be viewed. This will be different depending on the type of item:

o Archival descriptions – The content detail page for the description.

For example: https://catalog.archives.gov/id/7226539

o Authority records – The content detail page for the authority record.

For example: https://catalog.archives.gov/id/1307945

59


o Digital objects – The content detail page for the description with ?object=N to specify which object should be displayed.

For example: https://catalog.archives.gov/id/7226539?object=2

o Web pages – The URL to the web page.

For example: http://blogs.archives.gov/aotus/?p=5334

accessPath – [no-search] Specifies a relative path to directly access this item from Catalog.

o Prefix this path with “https://catalog.archives.gov/iapi/v1/” to provide the full path to the item.

o For archival descriptions, this will be: holdings/<NAID>

For example: holdings/7229423

o For authority records:

For people: authorities/people/<authorityId>- For example: authorities/people/142345

For organizations:- For example: authorities/organizations/9328

o For digital objects: The access path to the content itself.

For example: holdings/7229423/content/hst-ppp_93-1_18-02.jpg

3.4.2 Types

This section specifies various fields used for determining document types.

source – [string, facet] The scope to which this index entry belongs. Can be one of:

o authorities – For all authority records.

o holdings – For all archival descriptions.

o web – For all web pages.

type – [string, facet] Identifies type of result to return. Can be one of:

o description – For all archival descriptions.

o object – For all digital objects.

o person – For all people authority records.

o organization – For all organization authority records.

o archivesWeb – For all web pages from archives.gov.

o presidentialWeb – For all web pages from presidential libraries.

oldScope – [string, facet] Specifies the scope of the item according to the old OPA Pilot system.

60


o Can be any of: archives.gov, descriptions, online, authority, presidential

level – [string, facet] The level of archival description.

o Only available if source = holdings.

o Can be any of the following: recordGroup, collection, series, fileUnit, item, object.

parentLevel – [string] The level of description for the parent of this index entry in the archival hierarchy. For objects, this will be the parent of the description to which they belong.

o Can be any of the following: recordGroup, collection, series, fileUnit, item.

iconType – [string] Specifies a type for displaying the correct icon for this index entry. This will be specified in “mime type” format.

o Digital objects will use the official IANA mime type for the record, normalized. See http://www.iana.org/assignments/media-types/media-types.xhtml for the official list.

Examples include: application/pdf, application/vnd.ms-powerpoint, audio/mp3, video/mpeg, etc.

o Some mime types will be normalized to a simpler type.

For example, there are many different PowerPoint types. All will be normalized to “application/vnd.ms-powerpoint”.

This is done to simplify the display of the appropriate icon representing the type of digital object.

o Custom NARA types will use mime types which start with “nara/”, for example:

nara/record-group , nara/collection, nara/series, nara/item, nara/file - unit , nara/person, nara/organization

o Catalog has icons for all types specified by “iconType”. Use the following path to get the image:

https://catalog.archives.gov/images/<iconType>.svg

For example, if the icon type is “collection”, the image for it can be accessed using:

https://catalog.archives.gov/images/collection.svg

fileFormat – [string, facet, multi-valued] Specifies the file format for the content represented by or contained within the index-entry.

o For digital objects, this will be the normalized mime type for the object.

o For descriptions, this will be a list of the unique iconType’s for all of the digital objects contained within the description.

61

http://www.iana.org/assignments/media-types/media-types.xhtml


Note that mime types stored in “fileFormat” will be normalized.

o For example, there are many different PowerPoint types. All will be normalized to “application/vnd.ms-powerpoint” to simplify the presentation to the user.

originalMimeType – [keyword] For digital objects and web pages, this will be the original mime type of the file. This field will be missing for all other index entries.

tabType – [string, multi-valued] Specifies what tabs the Catalog index entry will show up on. Values can be one or more of the following:

o all – For everything which can show up on the “All” tab.

All descriptions (with or without on-line content) and authority records.

All digital objects as separately retrievable items (these will be grouped into the description to which they belong).

o online – Archival descriptions which contain on-line content, and web pages.

All digital objects as separately retrievable items, grouped into the description to which they belong.

o web – Only web pages.

o document – Descriptions which contain on-line content where “materialsType” (see below) is “text”.

o image – Descriptions which contain on-line content where “materialsType” (see below) is “photographsAndGraphics”.

o video – Descriptions which contain on-line content where “materialsType” (see below) is “movingImages”.

materialsType – [string, facet, multi-valued] The type of materials for descriptions. From from the “general-records-type” field of the archival description.

o drawings – “Architectural and Engineering Drawings”

o artifacts – “Artifacts”

o dataFiles – “Data Files”

o mapsAndCharts – “Maps and Charts”

o movingImages – “Moving Images”

o photographsAndGraphics – “Photographs and other Graphic Materials”

o sound – “Sound Recordings”

o text – “Textual Records”

o web – “Web Pages”

62


3.4.3 Display Fields

These are standard fields for display.

title – [keyword] Holds the title to be displayed to the user for the index entry.

titleSort – [no-search, sortable] Holds a sort-able version of the title.

o This version of the title will have leading articles (“a”, “an”, “the”, etc.) and prepositions (“of”, “by”, “in”, etc.) removed.

o This field may also be truncated, for very long titles.

parentTitle – [keyword] Holds the title of the parent archival description to this record. For objects, this will be the parent of the description to which they belong.

o Will be missing for record groups and collections.

o Not applicable for the authorities and web sources.

allTitles – [keyword, multi-valued] Holds all available titles for the index-entry, including subtitle, series-title, other-titles, and series-sub-titles.

webArea – [keyword] Holds the area of the web site to which a web page belongs.

o This is displayed to users to give them an idea of where within the archives.gov web site the search result is from. For presidential libraries, this will be the title of the presidential library web site.

webAreaUrl – [keyword] Holds the URL which corresponds to the webArea field.

o For example, if the webArea is “Archives.gov: Press Releases” the webAreaUrl will be “http://www.archives.gov/press/press-releases/”.

o For presidential libraries, this will be the home page for the library.

content – [keyword] This is the main content field used for generating dynamic teasers.

o Note that, while all of the content is indexed for keyword search, only the first [TBD – How much?] 100K bytes of the content will be stored to generate dynamic teasers.

creators – [keyword, multi-valued] Holds a list of creators of the content from the archival description.

teaser – [keyword] A representative summary of the document, typically the first 500 characters of the content.

o This field should be displayed for any document which doesn’t otherwise have highlighted content to show.

isOnline – [boolean] Specifies if the index entry represents something which is on-line.

o This will be true for all digital objects and any archival description which contains a digital object.

63


hasOnline – [boolean] specifies if the index entry represents a hierarchical archival description which has within it on-line content.

o This will be true for any archival description which has a descendent description that contains digital objects.

For example, for a series which contains an item that has digital objects, the series will be flagged as “hasOnline” of “true”.

thumbnailFile – [no-search] Holds the filename of the thumbnail for the item within the package to which this item belongs.

o The full URL of the thumbnail can be constructed with the following formula:

https://catalog.archives.gov/iapi/v1/media/<naId>/opa-renditions/thumbnails/<thumbnailFile>

o When not present, the iconType (see above) should be used to display the appropriate icon for the document.

titleDate – [no-search] This is fully formatted date string for the item. Can be a date range such as “1892 – 1972”.

3.4.4 More Facet Fields

Other fields for faceting include level, fileFormat, and materialsType. See the sections above for more information on those fields.

location – [string, multi-valued, facet] Holds the physical location of the archival materials represented by this index-entry.

o This information comes from the <reference-units> section of the archival description. Only the primary name of the location is stored.

o Note that copies of the content may be available from multiple locations. Therefore, this field is multi-valued.

o This field can be used for doing exact searches only on a location. For keyword searches on a location name, use the field below.

locationKeywords – [keyword, multi-valued, no-results] Used to search for keywords within the physical locations.

o Note: Use “location” to return the location data in the search results.

locationIds – [string, facet, multi-valued] Holds the reference IDs of the physical location of the archival materials represented by this index-entry.

o Location Ids will be used by standard interfaces for location facets.

o See the appendix (section 8.1) for a list of location IDs.

dateRangeFacet – [string, facet, multi-valued] Holds a list of the date ranges (e.g. 1600-1699, 1700-1749, 1750-1799, 1800-1809, 1810-1819, etc.) appropriate for this index entry.

64


3.4.5 Archival Hierarchy Fields

Also see section 3.4.3 for parentTitle and section 3.4.2 for parentLevel.

parentNaId – [string] The NAID of the parent of this item in the archival descriptions hierarchy. For objects, this will be the parent of the description to which they belong.

o Note that this field will have the single token “root” for record groups and collections.

o Use this field to search for all directl children for any archival description.

ancestorNaIds – [string, multi-valued] Holds a space-separated list of all ancestor NAIDs in the archival descriptions hierarchy for this Catalog index entry.

o Use this field to search for any descendent for any archival description.

hasChildren – [boolean] “true” if this record is an archival description with child archival descriptions. “false” otherwise.

3.4.6 Digital Object Information

This section contains fields specifically for digital objects. These fields will be missing for other types of records. See also originalMimeType above for the mime type of the digital object.

objectId – [string] A unique identifier for the digital object.

o Note: This will be the DAS stable object ID for the digital object, or a unique (and stable) object ID within the SEIP, as appropriate.

objectSortNum – [int, sortable] For all digital objects, this is the object number used to sort the object amongst its peers.

o Note that images will be sorted first (as they are displayed in Catalog), followed by other documents.

objectFile – [string] The file name of the object.

fileSize – [int] The file size of the digital object, in bytes.

3.4.7 Annotations

This section covers annotations which can be attached to digital objects, archival descriptions, and authority records. These fields are separated from the “objects” field (see section 3.4.8) to allow for date range searching, facets on tags, and faster searching for contributors.

In the text below, index entry refers to any archival description or authority record (for comments and tags) or digital object (for any sort of annotation).

allContributors – [string, multi-valued] – A list of the user IDs for everyone who has contributed in any way to this index entry.

65


allContributionsFirstDateTime – [date-time] The date-time when the first contribution (of any type) was made to this index entry.

allContributionsLatestDateTime – [date-time] The date-time when the most recent contribution (of any type) was created or modified.

3.4.7.1 Tags

tagsKeywords – [keyword, multi-valued, no-result] Holds a list of all tags indexed as keywords for flexible searching.

o Use tagsExact to see the exact values of the tags placed on a Catalog index entry.

tagsExact – [string, multi-valued,facet] Holds a list of all tags indexed as strings, so exact tags can be matched.

tagsContributors – [string, multi-valued] Holds a list of all contributors (by registered user ID) who have contributed tags to this index entry. Multiple tags contributed by a single contributor will be shown only once.

tagFirstDateTime – [date-time] Holds the date-time when the first tag was placed on this index entry.

tagLatestDateTime – [date-time] Holds the date-time of the most recent tag to be placed on this index entry.

3.4.7.2 Comments

commentFirstDateTime – [date-time] Holds the date-time when the first comment was placed on this index entry.

commentLatestDateTime – [date-time] Holds the date-time of the most comment or reply to be added to this index entry.

commentsContributors – [string, multi-valued] Holds a list of all contributors (by registered user ID) who have contributed comments or replies to this index entry. Multiple comments or replies contributed by a single contributor will be shown only once.

3.4.7.3 Transcriptions

transcriptionFirstDateTime – [date-time] Holds the date-time when the transcription was first created.

transcriptionLatestDateTime – [date-time] Holds the date-time of the most recent edit to this transcription.

transcriptionContributors – [string, multi-valued] Holds the list of everyone who has contributed to this transcription.

66


3.4.7.4 Translations

translationFirstDateTime – [date-time] Holds the date-time when the first translation was added to this index entry.

translationLatestDateTime – [date-time] Holds the date-time of the most recent edit to any translation on this index entry occurred.

translationContributors – [string, multi-valued] Holds the list of everyone who has contributed any translation to this index entry.

3.4.8 XML Fields

This section covers XML fields available from Catalog. XML fields contain full XML content which can be searched using XML tags and attributes as described above in section 3.3.5.

Note that XML fields will be automatically converted to JSON if JSON results are requested.

description – [xml] Contains the XML for the archival description.

o Only available for archival descriptions and digital objects(i.e., type=description or object).

o For digital objects, this will hold the xml of the archival description to which the object belongs.

o For purposes of scalability, and since digital objects can be independently retrieved, digital object information (the <objects> tag and its contents) is not included with this XML.

o TBD: Currently this is the <archival-description> XML (the old ARC export format). Should this be updated?

o TBD: Need reference where the user can find a full description of this XML format.

authority – [xml] Contains the complete XML for the authority record.

o Only available for authority records (i.e., source=authority).

o TBD: Currently this is the <person> and <organization> XML (the old ARC export format). Should this be updated?

o TBD: Need reference where the user can find a full description of this XML format.

objects – [xml] Holds the complete object XML for all digital objects (if a description) OR just the one <object> tag (for digital objects).

o Note: The content of objects will be as shown in section 2.3.1.2 with “annotations=full”. This means that the full content of all annotations (including all transcriptions and translations) will be copied into this XML field.

67


3.4.9 Authority Information

This section holds authority identifiers used to locate descriptions which reference a particular authority record.

Note: The following fields are only tagged on archival descriptions and digital objects. All of these fields will be missing for all other types of Catalog index entries.

TBD: Are the different types of authority relationships (creator, donor, etc.) still valid? The following values do not match up exactly to the LCDRG.

All authority IDs in the fields below will be specified as OPA ID’s for the authority record, either: person-<number> or org-<number>, as appropriate. For example, “person-3825288”.

allAuthorityIds – [string, multi-valued] holds a list of all authority IDs linked to this archival description.

o This field is used to limit the search results to only descriptions which contain any link to the specified authority record.

creatorIds – [string, multi-valued] holds the authority IDs for all authorities tagged as “creator” for the content described by the archival description.

subjectIds – [string, multi-valued] holds the authority IDs for all authorities tagged as a “subject” for the content described by the archival description.

donorIds – [string, multi-valued] holds the authority IDs for all authorities tagged as a “donor” for the content described by the archival description.

contributorIds – [string, multi-valued] holds the authority IDs for all authorities tagged as a “contributor” for the content described by the archival description.

personalReferenceIds – [string, multi-valued] holds the authority IDs for all authorities tagged as a “personal reference” for the content described by the archival description.

3.4.10 Dates and Times

This section specifies date and date-time fields which can be used for search and sorting within Catalog. See also “titleDate” above in section 3.4.3.

TBD: Are these the appropriate dates and date-times to be included in the index? Need more information from NARA.

recordCreatedDateTime – [date-time, sortable] The date/time the record was created.

o This is the time that the catalog record was created in NARA (not the creation date of the original content).

recordUpdatedDateTime – [date-time, sortable] The date/time the record was last updated.

68


o This is the time that the catalog record was updated by NARA (not the last update date of the original content).

firstIngestedDateTime – [date-time, sortable] The date/time the index-entry was first ingested into Catalog.

o This date/time is useful for locating items which are “new to Catalog”, for example as an RSS feed of “what’s new”.

o Note: This date may reset when a new version of Catalog is released.

ingestedDateTime – [date-time, sortable] The date/time the index-entry was ingested into Catalog.

o This date/time is updated whenever the index-entry is re-loaded or re-processed into Catalog.

productionDate – [date, sortable] The date on which an item was produced or created.

productionDateQualifier - [no-search] Qualifier for productionDate (section 3.4.10.1).

broadcastDate – [date, sortable] The date on which the item was first broadcast or another known broadcast date, if the first date is unknown.

broadcastDateQualifier - [no-search] Qualifier for broadcastDate (section 3.4.10.1).

releaseDate – [date, sortable] The date on which the audiovisual item was released for distribution.

releaseDateQualifier - [no-search] Qualifier for releaseDate (section 3.4.10.1).

coverageStartDate – [date, sortable] The starting date of the time period covered by the subject(s) of the record group, collection, or archival materials.

coverageStartDateQualifier - [no-search] Qualifier for coverageStartDate (section 3.4.10.1).

coverageEndDate – [date, sortable] The ending date of the time period covered by the subject(s) of the record group, collection, or archival materials.

coverageEndDateQualifier - [no-search] Qualifier for coverageEndDate (section 3.4.10.1).

inclusiveStartDate – [date, sortable] The beginning date on which the record group, collection, or series was created, maintained, or accumulated by the creator. Serves as a primary access point to allow users to retrieve or sort by time period.

inclusiveStartDateQualifier - [no-search] Qualifier for inclusiveStartDate (section 3.4.10.1).

inclusiveEndDate – [date, sortable] The last date on which the record group, collection, or series was created, maintained, or accumulated by the creator.

inclusiveEndDateQualifier - [no-search] Qualifier for inclusiveEndDate (section 3.4.10.1).

69


authorityStartDate – [date, sortable] The starting date for the person or organization.

o For people, this will be the birth date.

o For organizations, this will be the date the organization was established.

authorityStartDateQualifier - [no-search] Qualifier for authorityStartDate (section 3.4.10.1).

authorityEndDate – [date, sortable] The ending date for the person or organization.

o For people, this will be the death date.

o For organizations, this will be the date the organization was abolished.

o Will be missing if the authority still exists.

authorityEndDateQualifier - [no-search] Qualifier for inclusiveStartDate (section 3.4.10.1).

3.4.10.1 Date Qualifiers in Catalog

Due to technical reasons, dates in Catalog must be stored as full dates (YYYY-MM-DD). Partial dates (such as YYYY) are not allowed in date fields. Therefore, every date field which may be indeterminate has a “dateQualifier” field which identifies how to qualify the date in the date field.

In Catalog, dateQualifier fields can have the following values:

Y – Year only, value is known

Y? – Year only, value is uncertain

Yca – Year only, date is approximate (e.g., “circa”)

YM – Year and month, value is known

YM? – Year and month, value is uncertain

YMca – Year and month, date is approximate (e.g., “circa”)

YMD – Full date: year, month and day, value is known

YMD? – Full date: year, month and day, value is uncertain

YMDca – Full date: year, month and day, date is approximate (e.g., “circa”)

For example: if productionDate = “1948-01-01” and productionDateQualifier is “Yca”, then the date should be displayed as “ca. 1948”.

3.5 Search Results OutputIn response to a search request, the search API will return the following information, either structured as XML or JSON as requested:

70


Header information

o Status of the request

o Any error messages which occurred

o The time it took to satisfy the request

o All of the search parameters specified with the request

Search results

o The main list of search results, including the total number of documents which match the query

o Each result will contain the score (aka relevancy) of the result and a list of all of the fields requested to be included with the result.

Grouped web page results

o If group=web is specified, then web site results will be returned as a separate section.

Facet values

o If facet=true, all facet fields and facet results are returned.

Highlights

o If highlight=true, then specified highlighted fields are returned.

Thesaurus expansions

o If thesaurus=true, then thesaurus expansion for each keyword are returned.

Each of these sections is discussed individually. Some complete examples are found at the end of this section.

3.5.1 Header Information

The standard headers specified in section 1.5.2 are returned by the search API. For example:

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman


<request path="/iapi/v1"><action>search</action><q>truman</q>

</request></header>

. . search results will be returned here .</opa-response>

71


JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&format=json

{ "header":{ "@status":"200", "@time":"231", "request":{ "@path":"/iapi/v1", "action":"search" "q":"truman" } },

. . search results will be returned here .}

3.5.1.1 Search API Error Codes

The following error codes are currently defined as part of the Search API response:


EMBEDDED_HTML The query or filter parameters contain embedded HTML which implies a potential cross-site scripting attack.

param: The parameter where the embedded HTML was detected.

INVALID_PARAM_VALUE The value specified to a parameter was invalid. For example, an incorrect source value, incorrect group type, etc.

param: The parameter which contained the invalid value.

INVALID_FIELD A request was made to search over an invalid field.

param: The parameter which contained the invalid field.field: The invalid field name.

QUERY_PARSE_ERROR An error occurred parsing the query.Note: This should rarely occur. Default behavior is defined for most invalid expressions.

param: The parameter which contained the invalid search expression.

OFFSET_LIMIT_EXCEEDED The maximum search result request has exceeded the limit. See the “offset” parameter for more details.

request: The result number requested.max: The maximum result number allowed.

ROWS_LIMIT_EXCEEDED The maximum number of rows to be returned with the search results was reached.

request: The number of rows requested.max: The maximum number of rows allowed.

INVALID_LIST An invalid list name was specified when saving results to a list.This typically occurs when using the “action=addList” parameter.

list: The list name requested.

LIST_LIMIT_EXCEEDED The maximum number of lists that a single user can create was exceeded.

list: The name for the new list requested.

72


LIST_SIZE_EXCEEDED The maximum number of results which can be added to a list was exceeded.

request: The number of results requested to be added to the list.max: The maximum number of results allowed.

QUERY_TIMEOUT Query execution took too long and was canceled.

max: The maximum number of seconds allowed for a query to execute.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&rows=10000


<request path="/iapi/v1"><action>search</action><q>truman</q><rows>10000</rows>

</request></header>

<errors> <error code="ROW_LIMIT_EXCEEDED">

<request>10000</request><max>200</max>

</error> </errors> . . search results will be returned here .</opa-response>

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&rows=10000&format=json

{ "header":{ "@status":"400", "@time":"8", "request":{ "@path":"/iapi/v1", "action":"search", "q":"truman", "rows":"10000" } }, "errors":{ "error":{ "@code":"ROW_LIMIT_EXCEEDED", "request":"10000", "max":"200" } }, . . search results will be returned here .}

3.5.2 Search Results

The following is a general outline of the search results section:

73


<results total="--total-matching-results --" offset="--first-result-num--" rows="-- num-results--">

<result num="--result-number--" score="--relevancy-score--"> <FIELD-NAME1>FIELD-VALUE1</FIELD-NAME1> <FIELD-NAME2>FIELD-VALUE2</FIELD-NAME2> <FIELD-NAME3>FIELD-VALUE3</FIELD-NAME3> . . MORE FIELDS GO HERE . </result> . . MORE RESULTS GO HERE .</results>

In the above outline, the various values are:

total-matching-results – The total number of results across all of Catalog which match the query. This number may be thousands or millions large.

first-result-num – The offset number of the first result returned.

rows – The actual number of results returned.

result-number – The result number (where 0 is the first result) within the list of all possible results (not just results returned by this query).

relevancy-score – The relevancy score returned by the search engine.

o The number doesn’t actually mean anything. It is only useful for sorting results by relevancy.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description

<results total="6695" offset="0" rows="25"> <result num="0" score="2.34"> <source>holdings</source> <type>description</type> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title> <parentTitle>Subject Files, 1953 - 1972</parentTitle> <titleDate>1972-08-19</titleDate> <url>http://research.archives.gov/description/7226539</url> <accessPath>holdings/7226539</accessPath> <localId>hst-ppp_93-1_18</localId> <iconType>nara/item</iconType> <creators><val>Truman, Harry S., 1884-1972</val></creators> <thumbnailFile>hst-ppp_93-1_18-01.jpg</thumbnailFile> </result> <result num="1" score="2.14"> <source>holdings</source> <type>description</type> <naId>7283002</naId> <opaId>desc-7283002</opaId> <title>Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy</title> <parentTitle>Articles from "Studies in Intelligence", 1955 - 1992</parentTitle> <parentLevel>series</parentLevel> <url>http://research.archives.gov/description/7283002</url> <accessPath>holdings/7283002</accessPath>

74


<hmsEntryNumbers><val>A1 27</val></hmsEntryNumbers> <iconType>nara/fileUnit</iconType> <creators><val>Central Intelligence Agency. (12/04/1981 - )</val></creators> <thumbnailFile>263-a1-27-box-7-69-1-0001.jpg</thumbnailFile> </result> . . MORE RESULTS GO HERE .</results>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&format=json

"results":{ "@total":"6695", "@offset":"0", "@rows":"25", "result":[ { "@num":"0", "@score":"2.34", "source":"holdings", "type":"description", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "parentTitle":"Subject Files, 1953 - 1972", "titleDate":"1972-08-19", "url":"http://research.archives.gov/description/7226539", "accessPath":"holdings/7226539", "localId":"hst-ppp_93-1_18", "iconType":"nara/item", "creators":{ "val":"Truman, Harry S., 1884-1972" }, "thumbnailFile":"hst-ppp_93-1_18-01.jpg" }, { "@num":"1", "@score":"2.14", "source":"holdings", "type":"description", "naId":"7283002", "opaId":"desc-7283002", "title":"Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy", "parentTitle":"Articles from \"Studies in Intelligence\", 1955 - 1992", "parentLevel":"series", "url":"http://research.archives.gov/description/7283002", "accessPath":"holdings/7283002", "hmsEntryNumbers":{ "val":"A1 27" }, "iconType":"nara/fileUnit", "creators":{ "val":"Central Intelligence Agency. (12/04/1981 - )" }, "thumbnailFile":"263-a1-27-box-7-69-1-0001.jpg" } . . MORE RESULTS GO HERE . ],}

75


3.5.2.1 Optimized Results

“Optimized Results” are results which have been chosen by NARA as the best results for specified queries. When that query is entered, the optimized results will be “elevated” to the top of the results set. These are often called “best bets” or “featured results” in other systems.

When a result is one of the “optimized results” chosen by NARA, then it will have the@optimized attribute attached to it:

<result num="0" score="2.34" optimized="true"> . . . </result>

TBD: Can we change this attribute to “featured” and change all of the text to say “featured Results”? It would be more recognizable in the industry.

3.5.2.2 Search results with XML Fields

When search results contain XML fields (see section 3.4.8), the XML data will be included inside the top-level field.

For example, when returning the “description” field, the entire <archival-description> XML will be provided in the search results:

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&

resultFields=source,type,naId,opaId,title,description

<results total="6695" offset="0" rows="25"> <result num="0" score="2.34"> <source>holdings</source> <type>description</type> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title> <description> <arc-id>7226539</arc-id> <arc-id-desc>7226539</arc-id-desc> <title>Letter from George McGovern to Harry S. Truman</title> <title-only>Letter from George McGovern to Harry S. Truman</title-only> <local-identifier>hst-ppp_93-1_18</local-identifier> <local-identifier-desc>hst-ppp_93-1_18</local-identifier-desc> <created-timestamp>9/21/2013 14:39:46</created-timestamp> <level-of-desc level-id="NAVI"> <lod-display>Item</lod-display> </level-of-desc> . . More description fields go here . </description> </result> . . MORE RESULTS GO HERE .</results>

76


JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&

resultFields=source,type,naId,opaId,title,description&format=json

"results":{ "@total":"6695", "@offset":"0", "@rows":"25", "result":{ "@num":"0", "@score":"2.34", "source":"holdings", "type":"description", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "parentTitle":"Subject Files, 1953 - 1972", "titleDate":"1972-08-19", "url":"http://research.archives.gov/description/7226539", "accessPath":"holdings/7226539", "localId":"hst-ppp_93-1_18", "iconType":"nara/item", "creators":{ "val":"Truman, Harry S., 1884-1972" }, "thumbnailFile":"hst-ppp_93-1_18-01.jpg", "description":{ "arc-id":"7226539", "arc-id-desc":"7226539", "title":"Letter from George McGovern to Harry S. Truman", "title-only":"Letter from George McGovern to Harry S. Truman", "local-identifier":"hst-ppp_93-1_18", "local-identifier-desc":"hst-ppp_93-1_18", "created-timestamp":"9/21/2013 14:39:46", "level-of-desc":{ "@level-id":"NAVI", "lod-display":"Item" }, . . More description fields go here . } }, . . MORE RESULTS GO HERE .}

3.5.2.3 Search results with portions of XML Fields

In addition to selecting an entire XML field, portions of the XML field can be selected:

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&

resultFields=source,type,naId,opaId,title,description.local-identifier,description.subject-references.subject-reference

<results total="6695" offset="0" rows="25"> <result num="0" score="2.34"> <source>holdings</source> <type>description</type> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title>

77


<parentTitle>Subject Files, 1953 - 1972</parentTitle> <titleDate>1972-08-19</titleDate> <url>http://research.archives.gov/description/7226539</url> <accessPath>holdings/7226539</accessPath> <localId>hst-ppp_93-1_18</localId> <iconType>nara/item</iconType> <creators><val>Truman, Harry S., 1884-1972</val></creators> <thumbnailFile>hst-ppp_93-1_18-01.jpg</thumbnailFile> <description> <local-identifier>hst-ppp_93-1_18</local-identifier> <subject-references> <subject-reference num="1" standard="Y" subject-id="1300242" subject-type="PER"> <display-name>Dewey, Thomas E. (Thomas Edmund), 1902-1971</display-name> </subject-reference> <subject-reference num="2" standard="Y" subject-id="3868518" subject-type="SUBJ"> <display-name>Presidential campaign, 1972</display-name> </subject-reference> <subject-reference num="3" standard="Y" subject-id="2396882" subject-type="PER"> <display-name>Shriver, Sargent, 1915-2011</display-name> </subject-reference> </subject-references> </description> </result> . . MORE RESULTS GO HERE .</results>

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&type=description&

resultFields=source,type,naId,opaId,title,description.local-identifier,description.subject-references.subject-reference&format=json

"results":{ "@total":"6695", "@offset":"0", "@rows":"25", "result":[ { "@num":"0", "@score":"2.34", "source":"holdings", "type":"description", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "parentTitle":"Subject Files, 1953 - 1972", "titleDate":"1972-08-19", "url":"http://research.archives.gov/description/7226539", "accessPath":"holdings/7226539", "localId":"hst-ppp_93-1_18", "iconType":"nara/item", "creators":{ "val":"Truman, Harry S., 1884-1972" }, "thumbnailFile":"hst-ppp_93-1_18-01.jpg", "description":{ "local-identifier":"hst-ppp_93-1_18", "subject-references":{ "subject-reference":[ { "@num":"1", "@standard":"Y", "@subject-id":"1300242", "@subject-type":"PER", "display-name":"Dewey, Thomas E. (Thomas Edmund), 1902-1971" }, {

78


"@num":"2", "@standard":"Y", "@subject-id":"3868518", "@subject-type":"SUBJ", "display-name":"Presidential campaign, 1972" }, { "@num":"3", "@standard":"Y", "@subject-id":"2396882", "@subject-type":"PER", "display-name":"Shriver, Sargent, 1915-2011" } ] } } }, . . MORE RESULTS GO HERE . ]}

3.5.2.4 Archival descriptions with digital objects

You can search over both archival descriptions with specified digital objects, but return only a list of archival descriptions.

XML ExampleGET

https://catalog.archives.gov/iapi/v1?action=search&q=truman&resultFields=source,type,naId,opaId,title,objects.object.file.@path,objects.object.thumbnailFile,objects.object.annotations.transcription

<results total="6695" offset="0" rows="25"> <result num="0" score="2.34"> <source>holdings</source> <type>description</type> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title> <objects> <object> <file path="./content/hst-ppp_93-1_18-01.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-01-thumb.jpg"

mime="image/jpeg"/> <annotations> <transcription lastModified="2014-02-21T16:22:58" isauthoritative="false" version="232"> <users total="3"> <user id="tjefferson" fullName="Thomas Jefferson" isNaraStaff="true" lastModified="2014-02-21T16:22:58" /> <user id="mstewart" fullName="Meredith Stewart" isNaraStaff="true" lastModified="2014-02-08T12:59:49"/> <user id="kmartin" lastModified="2014-02-02T08:44:12" /> </users> <text>GEORGE McGOVERN United States Senate Washington, D.C. 20510

August 19, 1972



79


It is an honor for me, as our party's candidate for President of the United States. . .</text> </transcription> </annotations> </object> <object> <file path="./content/hst-ppp_93-1_18-02.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-02-thumb.jpg"

mime="image/jpeg"/> </object> <object> <file path="./content/hst-ppp_93-1_18-03.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-03-thumb.jpg"

mime="image/jpeg"/> </object> </objects> </result> . . MORE RESULTS GO HERE .</results>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1?action=search&q=truman&resultFields=source,type,naId,opaId,title,objects.object.file.@path,objects.object.thumbnailFile,objects.object.annotations.transcription&format=json

{ "@total":"6695", "@offset":"0", "@rows":"25", "result":[ { "@num":"0", "@score":"2.34", "source":"holdings", "type":"description", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "objects":{ "object":[ { "file":{ "@path":".\/content\/hst-ppp_93-1_18-01.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-01-thumb.jpg", "@mime":"image\/jpeg" }, "annotations":{ "transcription":{ "@lastModified":"2014-02-21T16:22:58", "@isauthoritative":"false", "@version":"232", "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Thomas Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, {

80


"@id":"mstewart", "@fullName":"Meredith Stewart", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" }, { "@id":"kmartin", "@lastModified":"2014-02-02T08:44:12" } ] }, "text":"GEORGE McGOVERN\n United States Senate\n Washington,

D.C. 20510\n\n August 19, 1972\n\n The Honorable Harry S. Truman\n Independence, Missouri\n\n My dear Mr. President:\n\n It is an honor for me, as our party's\n candidate for President of the United States. . ."

} } }, { "file":{ "@path":".\/content\/hst-ppp_93-1_18-02.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-02-thumb.jpg", "@mime":"image\/jpeg" } }, { "file":{ "@path":".\/content\/hst-ppp_93-1_18-03.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-03-thumb.jpg", "@mime":"image\/jpeg" } } ] } }, . . MORE RESULTS GO HERE . ]}

3.5.3 Web Page Results

When the group=web parameter is specified, a set of 3 web results will be returned separate from the standard results, so they can be styled as a “call out” inside the standard (aka “organic”) results. Further, the web results will contain a fixed set of fields.

To get more web results, or download additional fields with your web results, submit separate queries for web results (using the source=web parameter) and the standard results without grouping.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&group=web

<webPages total="312" offset="0" rows="3"> <result num="0" score="1.54"> <opaId>http://www.trumanlibrary.org/</opaId> <title>Harry S. Truman Library and Museum</title> <webArea>Harry S. Truman Library and Museum</webArea> <webAreaUrl>http://www.trumanlibrary.org/</webAreaUrl>

81


<url>http://www.trumanlibrary.org/</url> <iconType>text/html</iconType> </result> <result num="1" score="1.55"> <opaId>http://www.archives.gov/press/press-releases/2009/nr09-74.html</opaId> <title>Spring Prologue Magazine Highlights Truman’s 125th Birthday</title> <webArea>Archives.gov: Press Releases</webArea> <webAreaUrl>http://www.archives.gov/press/press-releases/</webAreaUrl> <url>http://www.archives.gov/press/press-releases/2009/nr09-74.html</url> <iconType>text/html</iconType> </result> <result num="2" score="1.53"> <opaId>http://www.trumanlibrary.org/hst-bio.htm</opaId> <title>Truman: HST Biography</title> <url>http://www.trumanlibrary.org/hst-bio.htm</url> <webArea>Harry S. Truman Library and Museum</webArea> <webAreaUrl>http://www.trumanlibrary.org/</webAreaUrl> <iconType>text/html</iconType> </result></webPages>

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&group=web&format=json

"webPages":{ "@total":"312", "@offset":"0", "@rows":"3", "result":[ { "@num":"0", "@score":"1.54", "opaId":"http://www.trumanlibrary.org/", "title":"Harry S. Truman Library and Museum", "webArea":"Harry S. Truman Library and Museum", "webAreaUrl":"http://www.trumanlibrary.org/", "url":"http://www.trumanlibrary.org/", "iconType":"text/html" }, { "@num":"1", "@score":"1.55", "opaId":"http://www.archives.gov/press/press-releases/2009/nr09-74.html", "title":"Spring Prologue Magazine Highlights Truman's 125th Birthday", "webArea":"Archives.gov: Press Releases", "webAreaUrl":"http://www.archives.gov/press/press-releases/", "url":"http://www.archives.gov/press/press-releases/2009/nr09-74.html", "iconType":"text/html" }, { "@num":"2", "@score":"1.53", "opaId":"http://www.trumanlibrary.org/hst-bio.htm", "title":"Truman: HST Biography", "url":"http://www.trumanlibrary.org/hst-bio.htm", "webArea":"Harry S. Truman Library and Museum", "webAreaUrl":"http://www.trumanlibrary.org/", "iconType":"text/html" } ]}

82


3.5.4 Pdf Files Results

When the result contain pdf documents, one more field will be added in addition to the regular fields (pageCount, CreationDate, LastModified, Title), that is the extractedText, containing the text layer of the document itself.

For PDF objects, a field called extractedText will be present on the objects node.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&rows=10&q=12019827&format=xml

JSON Example

83

<result><num>1</num><score>0.14369197</score><type>object</type><naId>12019827</naId><objects created="2015-04-21T05:33:05Z" version="OPA-OBJECTS-1.0"> <object id="15951168" objectSortNum="1"><file mime="application/pdf" name="1069084.pdf" path="content/library/document/0003/1069084.pdf" type="primary"url="https://uat.research.archives.gov/OpaAPI/media/12019827/content/library/document/0003/1069084.pdf" /><thumbnail mime="image/jpeg" path="opa-renditions/thumbnails/1069084.pdf-thumb.jpg" url="https://uat.research.archives.gov/OpaAPI/media/12019827/opa-renditions/thumbnails/1069084.pdf-thumb.jpg" /> <technicalMetadata><size>1272102</size><mime>


GET https://catalog.archives.gov/ iapi/v1?action=search&rows=10&q=12019827

3.5.5 Facet Values

Facets are grouped under the <facets> tag, which contains a list of <field> tags, each with a list of facet values.

Use the parameter facet=true to turn on facets. Use facet.fields to specify the list of fields for which facet values are computed. See the complete list of fields (section 3.4) to see what fields are eligible for being used as facets.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&facet=true&

facet.fields=type,fileFormat

<facets> <field name="type"> <v name="description" count="72"/> <v name="archivesWeb" count="13"/> <v name="organization" count="8"/> <v name="person" count="4"/> <v name="presidentialWeb" count="3"/> </field> <field name="fileFormat"> <v name="application/pdf" count="50"/> <v name="application/vnd.ms-powerpoint" count="17"/> <v name="video/mpeg4" count="3"/> <v name="audio/mp3" count="1"/> <v name="image/gif" count="1"/> </field></facets>

84

object: {

@id: "15951168"

@objectSortNum: "1"

file: {

@mime: "application/pdf"

@name: "1069084.pdf"

@path: "content/library/document/0003/1069084.pdf"

@type: "primary"

@url: "https://uat.research.archives.gov/OpaAPI/media/12019827/content/library/document/0003/1069084.pdf"

}-

thumbnail: {

@mime: "image/jpeg"

@path: "opa-renditions/thumbnails/1069084.pdf-thumb.jpg"

@url: "https://uat.research.archives.gov/OpaAPI/media/12019827/opa-renditions/thumbnails/1069084.pdf-thumb.jpg"

}-

technicalMetadata: {


JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&facet=true&

facet.fields=type,fileFormat&format=json

"facets":{ "field":[ { "@name":"type", "v":[ { "@name":"description", "@count":"72" }, { "@name":"archivesWeb", "@count":"13" }, { "@name":"organization", "@count":"8" }, { "@name":"person", "@count":"4" }, { "@name":"presidentialWeb", "@count":"3" } ] }, { "@name":"fileFormat", "v":[ { "@name":"application/pdf", "@count":"50" }, { "@name":"application/vnd.ms-powerpoint", "@count":"17" }, { "@name":"video/mpeg4", "@count":"3" }, { "@name":"audio/mp3", "@count":"1" }, { "@name":"image/gif", "@count":"1" } ] }, ],}

3.5.6 Highlights

Document highlights are stored in a separate section and referenced to the document by OPA-ID. The <highlights> section will contain a list of <result> tags, each of which references the OPA-ID of the result that they are associated with.

85


Turn on highlights with the highlight=true param. See section 3.4 for additional parameters to specify what fields are highlighted.

Only fields which actually contain keywords that match the query are returned in the highlights section. In the example below, the second result (desc-7283002) contains no <creators> tag, because none of the creators had the keyword “truman” in the text.

The highlights themselves are indicated with embedded <em> tags. Note that the angles are escaped in XML:

Letter from George McGovern to Harry S. <em>Truman</em>

But not in JSON:

Letter from George McGovern to Harry S. <em>Truman</em>

At most 3 highlighted snippets will be created for each field. Only words specified in the “q” parameter will be highlighted in search results.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&highlight=true

<highlights> <result opaId="desc-7226539"> <title> <val>Letter from George McGovern to Harry S. <em>Truman</em></val> </title> <content> <val>The Honorable Harry ·s. <em>Truman</em> Independence, Missouri My dear Mr.

President:...</val> <val>GEORGE McGOVERN WASHINGTON, D.C. 20510 AIR MAIL SPECIAL DELIVERY The Honorable Harry s.

<em>Truman</em> Independence, Missouri AIR MAIL SPECIAL DELIVERY...</val> </content> <creators><val><em>Truman</em>, Harry S., 1884-1972</val></creators> </result>

<result opaId="desc-7283002"> <title> <val>Spring 1976: 7-69-1: <em>Truman</em> on CIA (Examining President

</em>Truman</em>'s Role in the Establishment of the Agency), by Thomas F. Troy</val>

</title> <content> <val>Examining President Truman's role in the establishment of the Agency TRUMAN ON CIA

Thomas F. Troy...</val> <val>President Harry S. Truman had his own version of his role in the establishment of the

Central Intelligence Agency. He once summed it up this way: '' I got a couple of...</val>

<val>In Truman's most extended account, in his Memoirs, he related how he discovered the lack of coordinated intelligence in Washington, asked what was being done about it, solicited advice</val>

</content> </result> . . Highlights for more results go here .</highlights>

86


JSON ExampleGET

https://catalog.archives.gov/iapi/v1?action=search&q=truman&highlight=true&format=json

"highlights":{ "result":[ { "@opaId":"desc-7226539", "title":[ "Letter from George McGovern to Harry S. <em>Truman</em>" ], "content":[ "The Honorable Harry s. <em>Truman</em> Independence, Missouri My dear Mr.

President:...", "GEORGE McGOVERN WASHINGTON, D.C. 20510 AIR MAIL SPECIAL DELIVERY The Honorable Harry s.

<em>Truman</em> Independence, Missouri AIR MAIL SPECIAL DELIVERY..." ], "creators":[ "<em>Truman</em>, Harry S., 1884-1972" ] }, { "@opaId":"desc-7283002", "title":[ "Spring 1976: 7-69-1: <em>Truman</em> on CIA (Examining President </em>Truman</em>'s Role

in the Establishment of the Agency), by Thomas F. Troy" ], "content":[ "Examining President Truman's role in the establishment of the Agency TRUMAN ON CIA

Thomas F. Troy...", "President Harry S. Truman had his own version of his role in the establishment of the

Central Intelligence Agency. He once summed it up this way: '' I got a couple of...", "In Truman's most extended account, in his Memoirs, he related how he discovered the

lack of coordinated intelligence in Washington, asked what was being done about it, solicited advice"

] }, . . Highlights for more results go here . ],}

3.5.7 Thesaurus Expansions

When thesaurus=true is added to the query, thesaurus expansions will be added to the search results output. These expansions can be used to expand the scope of the query to include words and word variations the user may not have thought of on their own.

Thesaurus expansion only applies to keywords specified in the “q” parameter.

For each keyword, different types of expansions will be provided:

broader terms – These are more general terms than the one provided by the user.

o For example: ship transportation

narrower terms – These are more specific examples of the term entered by the user.

o For example: ship tanker

87


related terms – These are any other terms which have any other sort of relationship to the term entered by the user.

o For example: ship navigation

Thesaurus Expansion Process

The steps in using thesaurus expansions are as follows:

1. Submit a query with thesaurus=true.a. See XML output below.

2. Provide a user interface for users to choose thesaurus expansion to be added back to the query.

a. Include selections for categories, such as “all related terms” and “all related terms for term X”.

3. Submit the chosen expansions back to the search engine with a new query.

This is done with the “thesaurus.terms” parameters specified in section 3.2, for example:thesaurus.terms.ship=relatedthesasurus.terms.ship=tankers,yachts,warships,steamboats

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=ship dock&thesaurus=true

<thesaurus> <term name="ship"> <broader> <val>transportation</val> <val>transport</val> <val>transporting</val> </broader> <narrower> <val>ships bridges</val> <val>steamboats</val> <val>hijacking of ships</val> <val>tankers</val> <val>warships</val> <val>seamanship</val> <val>yachts</val> <val>yachting</val> <val>armored vessels</val> <val>landing craft</val> <val>tank landing ships</val> <val>anchors</val> <val>anchoring</val> <val>cutters</val> <val>hospital ships</val> <val>ice-breaking vessels</val> <val>icebreakers</val> </narrower> <related> <val>freight transportation</val> <val>boats and boating</val> <val>navigation</val> <val>navigate</val> <val>shipbuilding</val> <val>shipbuilder</val> <val>steam power</val>

88


</related> </term> <term name="dock"> <broader> <val>harbors</val> <val>hydraulic structures</val> </broader> <narrower> <val>dry docks</val> <val>drydock</val> <val>drydocking</val> </narrower> </term></thesaurus>

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=ship dock&thesaurus=true&format=json

"thesaurus":{ "term":[ { "@name":"ship", "broader":[ "transportation", "transport", "transporting" ], "narrower":[ "ships bridges", "steamboats", "hijacking of ships", "tankers", "warships", "seamanship", "yachts", "yachting", "armored vessels", "landing craft", "tank landing ships", "anchors", "anchoring", "cutters", "hospital ships", "ice-breaking vessels", "icebreakers" ], "related":[ "freight transportation", "boats and boating", "navigation", "navigate", "shipbuilding", "shipbuilder", "steam power" ] }, { "@name":"dock", "broader":[ "harbors", "hydraulic structures" ], "narrower":[ "dry docks", "drydock", "drydocking" ] } ]

89


}

3.5.8 Complete Example

The following is a complete Search API response example.

Note that the content is for documentation purposes only – the actual content returned by the given search will likely be different.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&highlight=true&thesaurus=true&

resultFields=naId,opaId,title,parentTitle,titleDate,url,localId,iconType,creators,thumbnailFile,description,objects.object.file.@path,objects.object.thumbnailFile,objects.object.annotations.transcription&facet=true&facet.fields=type,fileFormat&group=web

<opa-response> <header status="200" time="231"> <request path="/iapi/v1"> <action>search</action> <q>truman</q> <highlight>true</highlight> <thesaurus>true</thesaurus> <facet>true</facet> <facet.fields>subScore,fileFormat</facet.fields> <group>web</group> </request> </header>

<results total="6695" offset="0" rows="25"> <result num="0" score="2.34"> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title> <parentTitle>Subject Files, 1953 - 1972</parentTitle> <titleDate>1972-08-19</titleDate> <url>http://research.archives.gov/description/7226539</url> <localId>hst-ppp_93-1_18</localId> <iconType>nara/item</iconType> <creators> <val>Truman, Harry S., 1884-1972</val> </creators> <thumbnailFile>hst-ppp_93-1_18-01.jpg</thumbnailFile> <description> <arc-id>7226539</arc-id> <arc-id-desc>7226539</arc-id-desc> <title>Letter from George McGovern to Harry S. Truman</title> <title-only>Letter from George McGovern to Harry S. Truman</title-only> <local-identifier>hst-ppp_93-1_18</local-identifier> <local-identifier-desc>hst-ppp_93-1_18</local-identifier-desc> <created-timestamp>9/21/2013 14:39:46</created-timestamp> <level-of-desc level-id="NAVI"> <lod-display>Item</lod-display> </level-of-desc> . . More description fields goes here . </description> <objects> <object> <file path="./content/hst-ppp_93-1_18-01.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-01-thumb.jpg"

mime="image/jpeg"/> <annotations>

90


<transcription lastModified="2014-02-21T16:22:58" isauthoritative="false" version="232">

<users total="3"> <user id="tjefferson" fullName="Thomas Jefferson" isNaraStaff="true" lastModified="2014-02-21T16:22:58" /> <user id="mstewart" fullName="Meredith Stewart" isNaraStaff="true" lastModified="2014-02-08T12:59:49"/> <user id="kmartin" lastModified="2014-02-02T08:44:12" /> </users> <text>GEORGE McGOVERN United States Senate Washington, D.C. 20510

August 19, 1972



It is an honor for me, as our party's candidate for President of the United States. . .</text> </transcription> </annotations> </object> <object> <file path="./content/hst-ppp_93-1_18-02.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-02-thumb.jpg"

mime="image/jpeg"/> </object> <object> <file path="./content/hst-ppp_93-1_18-03.jpg"/> <thumbnail path="./opa-rendition/thumbnails/hst-ppp_93-1_18-03-thumb.jpg"

mime="image/jpeg"/> </object> </objects> </result> <result num="1" score="2.14"> <naId>7283002</naId> <opaId>desc-7283002</opaId> <title>Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy</title> <parentTitle>Articles from "Studies in Intelligence", 1955 - 1992</parentTitle> <parentLevel>series</parentLevel> <url>http://research.archives.gov/description/7283002</url> <hmsEntryNumbers> <val>A1 27</val> </hmsEntryNumbers> <iconType>nara/fileUnit</iconType> <creators> <val>Central Intelligence Agency. (12/04/1981 - )</val> </creators> <thumbnailFile>263-a1-27-box-7-69-1-0001.jpg</thumbnailFile> <description> <arc-id>7283002</arc-id> <arc-id-desc>7283002</arc-id-desc> <title>Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy</title> <title-only>Truman on CIA (Examining President Truman's Role in the Establishment of the

Agency), by Thomas F. Troy</title-only> <created-timestamp>1/3/2010 10:12:33</created-timestamp> <level-of-desc level-id="FU"> <lod-display>FileUnit</lod-display> </level-of-desc> . . More description fields goes here . </description> <objects> . . More objects go here .

91


</objects> </result> . . MORE RESULTS GO HERE . </results>

<webPages total="6695" offset="0" rows="25"> <result num="0" score="1.54"> <opaId>http://www.trumanlibrary.org/</opaId> <title>Harry S. Truman Library and Museum</title> <webArea>Harry S. Truman Library and Museum</webArea> <webAreaUrl>http://www.trumanlibrary.org/</webAreaUrl> <url>http://www.trumanlibrary.org/</url> <iconType>text/html</iconType> </result> <result num="1" score="1.55"> <opaId>http://www.archives.gov/press/press-releases/2009/nr09-74.html</opaId> <title>Spring Prologue Magazine Highlights Truman's 125th Birthday</title> <webArea>Archives.gov: Press Releases</webArea> <webAreaUrl>http://www.archives.gov/press/press-releases/</webAreaUrl> <url>http://www.archives.gov/press/press-releases/2009/nr09-74.html</url> <iconType>text/html</iconType> </result> <result num="2" score="1.53"> <opaId>http://www.trumanlibrary.org/hst-bio.htm</opaId> <title>Truman: HST Biography</title> <url>http://www.trumanlibrary.org/hst-bio.htm</url> <webArea>Harry S. Truman Library and Museum</webArea> <webAreaUrl>http://www.trumanlibrary.org/</webAreaUrl> <iconType>text/html</iconType> </result> </webPages>

<facets> <field name="type"> <v name="description" count="72"/> <v name="archivesWeb" count="13"/> <v name="organization" count="8"/> <v name="person" count="4"/> <v name="presidentialWeb" count="3"/> </field> <field name="fileFormat"> <v name="application/pdf" count="50"/> <v name="application/vnd.ms-powerpoint" count="17"/> <v name="video/mpeg4" count="3"/> <v name="audio/mp3" count="1"/> <v name="image/gif" count="1"/> </field> . . More facet fields go here . </facets>

<highlights> <result opaId="desc-7226539"> <title>Letter from George McGovern to Harry S. <em>Truman</em></title> <content> <val>The Honorable Harry s. <em>Truman</em> Independence, Missouri My dear Mr.

President:...</val> <val>GEORGE McGOVERN WASHINGTON, D.C. 20510 AIR MAIL SPECIAL DELIVERY The Honorable Harry

s. <em>Truman</em> Independence, Missouri AIR MAIL SPECIAL DELIVERY...</val>

</content> <creators> <val><em>Truman</em>, Harry S., 1884-1972</val> </creators> </result>

<result opaId="desc-7283002"> <title>

92


<val>Spring 1976: 7-69-1: <em>Truman</em> on CIA (Examining President </em>Truman</em>'s Role in the Establishment of the Agency), by Thomas F. Troy</val>

</title> <content> <val>Examining President Truman's role in the establishment of the Agency TRUMAN ON CIA

Thomas F. Troy...</val> <val>President Harry S. Truman had his own version of his role in the establishment of the

Central Intelligence Agency. He once summed it up this way: '' I got a couple of . . .</val>

<val>In Truman's most extended account, in his Memoirs, he related how he discovered the lack of coordinated intelligence in Washington, asked what was being done about it, solicited advice</val>

</content> </result> . . More highlight results go here . </highlights>

<thesaurus> <term name="ship"> <broader> <val>transportation</val> <val>transport</val> <val>transporting</val> </broader> <narrower> <val>ships bridges</val> <val>steamboats</val> <val>hijacking of ships</val> <val>tankers</val> <val>warships</val> <val>seamanship</val> <val>yachts</val> <val>yachting</val> <val>armored vessels</val> <val>landing craft</val> <val>tank landing ships</val> <val>anchors</val> <val>anchoring</val> <val>cutters</val> <val>hospital ships</val> <val>ice-breaking vessels</val> <val>icebreakers</val> </narrower> <related> <val>freight transportation</val> <val>boats and boating</val> <val>navigation</val> <val>navigate</val> <val>shipbuilding</val> <val>shipbuilder</val> <val>steam power</val> </related> </term> <term name="dock"> <broader> <val>harbors</val> <val>hydraulic structures</val> </broader> <narrower> <val>dry docks</val> <val>drydock</val> <val>drydocking</val> </narrower> </term> </thesaurus></opa-response>

93


JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=search&q=truman&highlight=true&thesaurus=true&

resultFields=naId,opaId,title,parentTitle,titleDate,url,localId,iconType,creators,thumbnailFile,description,objects.object.file.@path,objects.object.thumbnailFile,objects.object.annotations.transcription&facet=true&facet.fields=type,fileFormat&group=web&format=json

"opa-response":{ "header":{ "@status":"200", "@time":"231", "request":{ "@path":"/iapi/v1", "action":"search", "q":"truman", "highlight":"true", "thesaurus":"true", "facet":"true", "facet.fields":"subScore,fileFormat", "group":"web" } }, "results":{ "@total":"6695", "@offset":"0", "@rows":"25", "result":[ { "@num":"0", "@score":"2.34", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "parentTitle":"Subject Files, 1953 - 1972", "titleDate":"1972-08-19", "url":"http://research.archives.gov/description/7226539", "localId":"hst-ppp_93-1_18", "iconType":"nara/item", "creators":{ "val":"Truman, Harry S., 1884-1972" }, "thumbnailFile":"hst-ppp_93-1_18-01.jpg", "description":{ "arc-id":"7226539", "arc-id-desc":"7226539", "title":"Letter from George McGovern to Harry S. Truman", "title-only":"Letter from George McGovern to Harry S. Truman", "local-identifier":"hst-ppp_93-1_18", "local-identifier-desc":"hst-ppp_93-1_18", "created-timestamp":"9/21/2013 14:39:46", "level-of-desc":{ "@level-id":"NAVI", "lod-display":"Item" }, . . More description fields goes here . }, "objects":{ "object":[ { "file":{ "@path":".\/content\/hst-ppp_93-1_18-01.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-01-thumb.jpg", "@mime":"image\/jpeg" },

94


"annotations":{ "transcription":{ "@lastModified":"2014-02-21T16:22:58", "@isauthoritative":"false", "@version":"232", "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Thomas Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"mstewart", "@fullName":"Meredith Stewart", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" }, { "@id":"kmartin", "@lastModified":"2014-02-02T08:44:12" } ] }, "text":"GEORGE McGOVERN\n United States Senate\n Washington,

D.C. 20510\n\n August 19, 1972\n\n The Honorable Harry S. Truman\n Independence, Missouri\n\n My dear Mr. President:\n\n It is an honor for me, as our party's\n candidate for President of the United States. . ."

} } }, { "file":{ "@path":".\/content\/hst-ppp_93-1_18-02.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-02-thumb.jpg", "@mime":"image\/jpeg" } }, { "file":{ "@path":".\/content\/hst-ppp_93-1_18-03.jpg" }, "thumbnail":{ "@path":".\/opa-rendition\/thumbnails\/hst-ppp_93-1_18-03-thumb.jpg", "@mime":"image\/jpeg" } } ] } }, { "@num":"1", "@score":"2.14", "naId":"7283002", "opaId":"desc-7283002", "title":"Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy", "parentTitle":"Articles from \"Studies in Intelligence\", 1955 - 1992", "parentLevel":"series", "url":"http://research.archives.gov/description/7283002", "hmsEntryNumbers":{ "val":"A1 27" }, "iconType":"nara/fileUnit", "creators":{ "val":"Central Intelligence Agency. (12/04/1981 - )" }, "thumbnailFile":"263-a1-27-box-7-69-1-0001.jpg",

95


"description":{ "arc-id":"7283002", "arc-id-desc":"7283002", "title":"Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy", "title-only":"Truman on CIA (Examining President Truman's Role in the Establishment of

the Agency), by Thomas F. Troy", "created-timestamp":"1/3/2010 10:12:33", "level-of-desc":{ "@level-id":"FU", "lod-display":"FileUnit" }, . . More description fields go here . }, "objects":{ "object":[ . . More objects go here . ] } } . . MORE RESULTS GO HERE . ], }, "webPages":{ "@total":"6695", "@offset":"0", "@rows":"25", "result":[ { "@num":"0", "@score":"1.54", "opaId":"http://www.trumanlibrary.org/", "title":"Harry S. Truman Library and Museum", "webArea":"Harry S. Truman Library and Museum", "webAreaUrl":"http://www.trumanlibrary.org/", "url":"http://www.trumanlibrary.org/", "iconType":"text/html" }, { "@num":"1", "@score":"1.55", "opaId":"http://www.archives.gov/press/press-releases/2009/nr09-74.html", "title":"Spring Prologue Magazine Highlights Truman's 125th Birthday", "webArea":"Archives.gov: Press Releases", "webAreaUrl":"http://www.archives.gov/press/press-releases/", "url":"http://www.archives.gov/press/press-releases/2009/nr09-74.html", "iconType":"text/html" }, { "@num":"2", "@score":"1.53", "opaId":"http://www.trumanlibrary.org/hst-bio.htm", "title":"Truman: HST Biography", "url":"http://www.trumanlibrary.org/hst-bio.htm", "webArea":"Harry S. Truman Library and Museum", "webAreaUrl":"http://www.trumanlibrary.org/", "iconType":"text/html" } ] }, "facets":{ "field":[ { "@name":"type", "v":[ {

96


"@name":"description", "@count":"72" }, { "@name":"archivesWeb", "@count":"13" }, { "@name":"organization", "@count":"8" }, { "@name":"person", "@count":"4" }, { "@name":"presidentialWeb", "@count":"3" } ] }, { "@name":"fileFormat", "v":[ { "@name":"application/pdf", "@count":"50" }, { "@name":"application/vnd.ms-powerpoint", "@count":"17" }, { "@name":"video/mpeg4", "@count":"3" }, { "@name":"audio/mp3", "@count":"1" }, { "@name":"image/gif", "@count":"1" } ] } . . More facet fields go here . ], }, "highlights":{ "result":[ { "@opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. <em>Truman</em>", "content":{ "val":[ "The Honorable Harry s. <em>Truman</em> Independence, Missouri My dear Mr.

President:...", "GEORGE McGOVERN WASHINGTON, D.C. 20510 AIR MAIL SPECIAL DELIVERY The Honorable Harry

s. <em>Truman</em> Independence, Missouri AIR MAIL SPECIAL DELIVERY..." ] }, "creators":{ "val":"<em>Truman</em>, Harry S., 1884-1972" } }, { "@opaId":"desc-7283002", "title":{

97


"val":"Spring 1976: 7-69-1: <em>Truman</em> on CIA (Examining President </em>Truman</em>'s Role in the Establishment of the Agency), by Thomas F. Troy"

}, "content":{ "val":[ "Examining President Truman's role in the establishment of the Agency TRUMAN ON CIA

Thomas F. Troy...", "President Harry S. Truman had his own version of his role in the establishment of the

Central Intelligence Agency. He once summed it up this way: '' I got a couple of . . .",

"In Truman's most extended account, in his Memoirs, he related how he discovered the lack of coordinated intelligence in Washington, asked what was being done about it, solicited advice"

] } } . . More highlighted fields go here . ], }, "thesaurus":{ "term":[ { "@name":"ship", "broader":{ "val":[ "transportation", "transport", "transporting" ] }, "narrower":{ "val":[ "ships bridges", "steamboats", "hijacking of ships", "tankers", "warships", "seamanship", "yachts", "yachting", "armored vessels", "landing craft", "tank landing ships", "anchors", "anchoring", "cutters", "hospital ships", "ice-breaking vessels", "icebreakers" ] }, "related":{ "val":[ "freight transportation", "boats and boating", "navigation", "navigate", "shipbuilding", "shipbuilder", "steam power" ] } }, { "@name":"dock", "broader":{ "val":[ "harbors", "hydraulic structures" ]

98


}, "narrower":{ "val":[ "dry docks", "drydock", "drydocking" ] } } ] }}

3.6 ExportsCatalog can export search results and content in a variety of different formats with a variety of different content. Output can be viewed, printed, or downloaded for off-line processing.

3.6.1 Specifying What Results to Export

Results can be specified in three different ways:

Results from a search:

POST https://catalog.archives.gov/iapi/v1/exports/auth?search-params...&export-params...

See section 3.2 for search parameters.

Results from a stored “my list”:

POST https://catalog.archives.gov/iapi/v1/exports/auth?listName=<list-name>&export params...

See section 5.3.4.2 for more information on lists.

A list of OPA-ID values:

POST https://catalog.archives.gov/iapi/v1/exports/auth?export.ids=<list-of-id’s-to-export>&more export params...

See below for more information on export parameters.

3.6.2 Parameters

Parameter Description Examplesaction=export Save the search results with additional information

as a bulk export file.action=export

99


Parameter Description Examplesexport.type=brief |

full | fieldsSpecifies what to export. One of: brief – All of the fields in the brief results

(standard search engine results). full – All of the fields in the full content-detail

page for each result. fields – Export selected fields. Specify the exact

fields to export with the resultFields parameter (see below).

export.type=brief

export.type=full

export.what=list of optional items to export

When exporting information, the following additional items may be exported with each result: thumbnails – Thumbnails.

o For export.type=brief, shows only the primary thumbnail for the item.

o For export.type=full, exports thumbnails for digital objects.

tags – Export tags attached to the record.o For export.type=brief, exports tags on the

description.o For export.type=full, exports the tags on

the description and on each object. comments – All comments attached to each

record.o For export.type=brief, exports the

comments on the description.o For export.type=full, exports the comments

on the description and on each object. transcriptions – Transcriptions for each digital

object. [Only available for export.type=full.] translations – Translations for each digital object.

[Only available for export.type=full.]

export.what=thumbnails,tags,comments

export.languages=list of language codes

If export.options=translations, then list the IS0 639-3 (three letter) language codes to export.

export.languages=spa,ita,deu

export.languages.all=true/false

If export.options=translations, then use this parameter to specify that all languages should be exported.

export.languages.all=true

resultFields=list of fields…

If “export.type=fields”, the list of fields to export is specified with a “resultFields” parameter. See section 3.4 for the list of fields which can be exported.

resultFields=title,naId,opaId,description

100


Parameter Description Examplesexport.ids=list of

opa-ids…If you wish to a selected list of results (to include in the results from a search), specify a comma-separated list of OPA_IDs to export.See section 1.4.2 for more information on OPA_IDs.

export.ids=desc-154232,desc-24242, person-4289,org-98242,desc-24242/9232.jpg

export.format=xml|json|csv|txt|pdf|html

Specify the format for the search results. xml – Export all metadata in XML format. json – Export all metadata in XML format. csv – Export metadata in CSV format. Not

compatible with the following “export.type” options:

o TBD – Uncertain how to export CSV formats for complex metadata structures and multi-valued fields. Requires discussion with NARA.

txt – Export metadata in text format.o Incompatible with export.type=fields. Will

return an error if specified.o Incompatible with

“export.options=thumbnails” (option will be ignored if present).

pdf – Export a PDF rendition of the results.o Incompatible with export.type=fields. Will

return an error if specified. html – Export an HTML rendition of the results.

o Incompatible with export.type=fields. Will return an error if specified.

export.results.format=json

export.bulk=true/false

If exporting a very large number of records, records with a large number of objects, or exporting content files, then you will likely need to specify “export.bulk=true”.When export.bulk=true, a bulk export will be queued for background processing and a bulk export ID will be returned.See section 5.4 for getting the status of the bulk export and getting a download URL when the export file is complete.

export.bulk=true

101


Parameter Description Examplesexport.bulk.content

=objects|thumbnails|transcriptions|translations

When export.bulk=true, additional content files can also be exported. Use this parameter to specify what content files to export. objects – Digital object files attached to each

result. thumbnails – Thumbnail files attached to each

result.Note: objects.xml files (with annotations=full, see section 2.3.1.2) will be automatically exported for all results.

export.bulk.content=objects,thumbnails

3.6.3 Examples

Export the brief results of a simple search as HTML:

POST https://catalog.archives.gov/iapi/v1/exports/auth?&q=truman&type=description&export.type=brief&export.format=html

Export the brief results of a simple search as HTML, with thumbnails:

POST https://catalog.archives.gov/iapi/ v1/exports/auth ?q=truman&type=description&export.type=brief&export.format=html&export.what=thumbnails

Export the full results of a simple search as PDF, with thumbnails, tags, and comments:

POST https://catalog.archives.gov/iapi/v1/exports/auth?q=truman&type=description&export.type=full&export.format=pdf&export.what=thumbnails,tags,comments

Export the full results of selected items as text, with transcript:

POST https://catalog.archives.gov/iapi/v1/exports/auth?export.ids=desc-72889324,desc-6232422,desc-5523412,desc-6872412,desc-7114423&type=description&export.type=full&export.format=txt&export.what=transcriptions

Export the full results of a simple search as XML, with everything possible including all language translations:

POST https://catalog.archives.gov/iapi/v1/exports/auth?q=truman&type=description&export.type=full&export.format=pdf&export.what=thumbnails,tags,comments,transcriptions,translations&export.languages.all=true

Export selected fields, including the full archival description and content metadata, from mstewart’s “HarrySTrumanResearch” list:

POST https://catalog.archives.gov/iapi/v1/exports/?listName=HarrySTrumanResearch&rows=250&export.type=fields&resultFields=naId,opaId,title,localId,url,level,fileFormat,creators,description,tagsExact,comments,transcription,translations,contentMetadata

102


Export all of the fields above, plus the actual content files and thumbnail files as a bulk export:

POST https://catalog.archives.gov/iapi/v1/exports/auth?listName=HarrySTrumanResearch&rows=250&export.type=fields&resultFields=naId,opaId,title,localId,url,level,fileFormat,creators,description,tagsExact,comments,transcription,translations,contentMetadata&export.bulk=true&export.bulk.content=objects,thumbnails

3.6.4 Response (standard export)

When exporting results which are not bulk exports, OPA will directly respond with the content.

For example, if executing the following URL:

POST https://catalog.archives.gov/iapi/v1/exports/auth ?q=truman&type=description&export.type=brief&export.format=html

The data in the response will be HTML data.

Error Responses

If there is an error in one of the parameters, Catalog will respond with an HTTP “400” error (bad request). The content of the error will be a standard <opa-response> XML (see section 1.5.2) or JSON. Possible error codes included in the <opa-response> are shown below:


INVALID_PARAM_VALUE A value for a parameter is invalid. param: The parameter with the invalid value.value: The invalid value.

INCOMPATIBLE_OPTIONS The options specified are incompatible with the format specified or with the type of export requested (brief, full, or fields).

what: The incompatible option.format: The incompatible format.type: The incompatible type of export.

TOO_MANY_RESULTS Too many results were requested to be exported. [TBD-Maximum number of results?]Note that the maximum number of results will be different for bulk exports than simple exports.

request: The number of results requested to be exported.max: The maximum number of results allowed to be exported.

INVALID_FIELD One of the fields specified in the resultFields parameter is invalid.


103


EXTRANEOUS_PARAM One of the parameters is extraneous. For example, export.bulk.content when export.bulk is false. Or export.languages when translations are not exported.

param: The parameter which is extraneous.

3.6.5 Bulk Exports

Bulk exports from Catalog are performed using the following steps:

1. Choose the set of results to export (either using a search query, a list of IDs, or a “My List”, see above).

2. Initiate the bulk-export with the parameters above.

a. Set export.bulk=true.

3. Save the export ID number returned by Catalog, see below.

a. This number will be used to get status on the bulk export and to download the resulting compressed file when it is ready.

4. Get the status of the bulk download by going to:

GET https://catalog.archives.gov/iapi/v1/exports/auth/status//<export-id>

5. Once the bulk-download is ready, fetch the bulk download file from:

GET https://catalog.archives.gov/iapi/v1/exports/auth/files/<export-id>

and then download the contents.

See section 5.4 for more information on bulk export status.

XML ExamplePOST https://catalog.archives.gov/iapi/v1/exports/auth?q=truman&export.type=fields&

resultFields=description&export.format=xml&export.bulk=true&export.bulk.content=objects,thumbnails,transcriptions

<opa-response> <header status="200" time="231"> <request path="/iapi/v1"> <action>export</action> <q>truman</q> <export.type>fields</export.type> <resultFields>description</resultFields> <export.format>xml</export.format> <export.bulk>true</export.bulk> <export.bulk.content>objects,thumbnails,transcriptions</export.bulk.content> </request> </header> <bulk-export exportId="42"/></opa-response>

104


JSON ExamplePOST https://catalog.archives.gov/iapi/v1/exports/auth?q=truman&export.type=fields&

resultFields=description&export.format=json&export.bulk=true&export.bulk.content=objects,thumbnails,transcriptions&format=json

{ "header":{ "@status":"200", "@time":"231", "request":{ "@path":"/iapi/v1", "action":"export", "q":"truman", "export.type":"fields", "resultFields":"description", "export.format":"json", "export.bulk":"true", "export.bulk.content":"objects,thumbnails,transcriptions" } }, "bulk-export":{ "@exportId":"42" }}

3.7 Search based Content DetailBeing able to page through thousands of Content Detail pages required the existence of an API method that uses the original search results query but returns the current page only, plus the ids of the previous and next ones.

See sections 3.3, 3.4 and 3.5 for search request and response reference.

3.7.1 Content Detail result

The following is the base query that is executed to retrieve content detail results:

GET https://catalog.archives.gov/iapi/v1?action=contentDetail&search parameters. . .

It will contain all the search parameters that were specified in a previous brief results search to ensure the paging is done over the same result set, excluding web results.

The search parameters that will be carried over are the following:

q : The search query

SearchType : Used only for advanced search cases

Facet filters : Filters in facets will also be carried across pages

sort : Specifies the value to sort on, default value is relevance

105


Result fields must not be specified as the expected response will always have the same structure.

XML ExampleGET https://catalog.archives.gov/iapi/v1?action=contentDetail&q=truman&offset=0

&rows=1&format=xml

<opaResponse naId="2668751" nextNaId="6037881" opaId="desc-2668751" prevNaId="" title="Truman Doctrine" total="71035">

<header status="200">...</header><content>

<description>...</description><objects>...</objects>

</content></opa-response>

JSON ExampleGET https://catalog.archives.gov/iapi/v1? contentDetail&q=truman&offset=0 &rows=1

"opaResponse":{ "header":{ "@status":"200", "time":"2015-04-16T16:11:12.253Z", "request":{ "@action":"contentDetail", "@path":"\/iapi\/v1", "format":"json", "pretty":true } }, "@total":"71035", "@naId":"2668751", "@prevNaId":"", "@nextNaId":"6037881", "@opaId":"desc-2668751", "@title":"Truman Doctrine", "content":{ "description":"...", "objects":{...} } }}

JSON Example 2GET https:// catalog.archives.gov/OpaAPI/iapi/v1?

SearchType=advanced&action=contentDetail&f.level=item&f.materialsType=photographsandgraphics&f.oldScope=(online+or+descriptions)&offset=1&q=truman&rows=1&sort=titleSort+asc

"opaResponse":{ "header":{ "@status":"200", "time":"2015-04-16T16:11:12.253Z", "request":{ "@action":"contentDetail", "@path":"\/iapi\/v1", "format":"json",

106


"pretty":true } }, "@total":"71035", "@naId":"2668751", "@prevNaId":"", "@nextNaId":"6037881", "@opaId":"desc-2668751", "@title":"Truman Doctrine", "content":{ "description":"...", "objects":{...} } }}

As part of the S3 implementation the following parameter was added to videos stored in S3, it’s necessary to download the videos through the API instead of S3 or cloudfront to avoid security issues.

downloadURL.

Here is an extract of an example response containing a video:

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=contentDetail&q=truman&offset=0 &rows=1

.

.

."file":{ "@mime":"video \/mpeg ", "@name":"09026_2010_01_a.mpg", "@path”:"content\/rediscovery\/09026_2010_01_a.mpg", "@stream":" https://d2mbal85fggx1u.cloudfront.net/

content\/rediscovery\/09026_2010_01_a.mpg", "@type":"primary" }...

3.7.2 Jump to Page functionality

Jump to page is achieved by specifying the page number in the offset parameter of the search. Both offset and rows parameters must always be provided and the rows parameter must always have a value of 1.

JSON ExampleGET https://catalog.archives.gov/iapi/v1?action=contentDetail&q=truman&offset=30 &rows=1

The previous example will retrieve the 31st result page in a 0 based results list.

107


The Jump to Page functionality will validate that the user doesn’t go beyond the limits set for it’s user type, thus if for example a maximum of 2000 search results is set for a public user, the last offset that user will be able to access is 1999.

108


4 Annotations API

The Catalog system allows users to annotate Catalog content. Annotations include transcriptions, translations, comments, and tags.

4.1 The BasicsThis section describes basic information required to access and edit annotations.

4.1.1 Annotations are attached to content

Annotations, as the name implies, are attached to content in Catalog. Specifically:

Tags and comments – Can be attached to archival descriptions, authority records, and digital objects.

Transcriptions and translations – Can be attached to digital objects.

Therefore, to access or modify annotations, you will most likely need to first determine what you want to annotate, and then you can access the annotations from there.

To access annotations:

Authority records:

https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>

https://catalog.archives.gov/iapi/v1/id/<NAID>/tags

https://catalog.archives.gov/iapi/v1/id/<NAID>/comments

Archival Descriptions:

https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>

https://catalog.archives.gov/iapi/v1/id/<NAID> /comments

https://catalog.archives.gov/iapi/v1/id/<NAID> /tags

Digital Objects:

https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>/objects/<obj-id>

https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /tags

https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /comments

https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id>/transcriptions

https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /translations

See a map of all of the ways that annotations can be accessed in section 2.1.

109


4.1.2 Modifying annotations requires login

In order to make or modify an annotation, the user will need to have a registered user account and will need to log in. This will allow all contributions created or modified to be traced back to a username for the person that made the modification.

See section 1.6 for API methods to login and maintaining credentials for API calls.

4.1.3 Editing annotations

People can edit each other’s transcriptions and translations. This in contrast to tags and comments, where only the author can edit or delete their own comment, reply, or tag.

Note that NARA moderators have privileged access and can delete or restore any annotation.

If two users are accessing the same transcription or translation at the same time, one user will be allowed to do the edit, and the other will be locked out. The API will return the lock status of transcriptions and translations, and will return an error if an attempt is made to edit a locked annotation.

4.1.4 Contributed Annotations by user

Each user’s contributions are publically available. The User API (see section 5) provides a means to view all annotations made by a particular user.

4.1.5 Annotation parameter validation

Annotations always refer to a specific NaId and may sometimes refer to an ObjectId. Due to the nature of the implementation of adds and updates it becomes necessary to validate those two values for the following cases:

Adding and updating transcriptions

Adding tags

Adding and updating comments

Adding and updating translations

In the past, NaId and ObjectId were validated against the Solr server index for every operation that required these two parameters, even for read only operations such as annotation retrieval. This created a heavy load on the Solr server and it was determined that:

1. For read operations it’s not necessary to validate these parameters. Parameter validation can be bypassed and API would respond with a NOT_FOUND error if a specific item wasn’t present.

2. For write operations the validation would be performed against file system.

110


NaId is performed by validating the existence of the directory for the NaId in file system. Ingestion creates a directory for each NaID that contains all the media files and metadata for both the NaId item and its child objects.

ObjectId is validated by checking for its existence within the objects.xml file that is inside the NaId directory in file system as initialized by the ingestion process.

4.2 Annotations Summary InformationFor authority records, archival descriptions, and digital objects, the API provides a means for obtaining a summary of its annotations. Typically the summary is simply the counts of annotations available for the specified path.

URL Paths:

Annotation summaries can be accessed for any authority record, archival description, or digital object using the following URLs:

Authority records:

GET https://catalog.archives.gov/iapi/v1/contributions/summaryid/<NAID>


GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>

Digital Objects:

GET https://catalog.archives.gov/iapi/v1/contributions/summary/id/<NAID>/objects/<obj-id>

See a map of all of the ways that annotations can be accessed in section 2.1.

Parameters

The summary can be further customized to include detailed information any (or all) annotations, using the “include” parameter:

Parameter Description Examplesinclude=[tags],

[comments],[transcription],[translations],[all]

A comma-separated list of annotations to fetch as part of the summary.(default) if not specified, only counts for all annotations are provided.

include=tags,comments

include=all

111


Response

The summary response will have four sections:

<comments>

o Default:

Only the counts (the @comments, @replies, and @total attributes)

o If include=comments:

All of the same comment XML as described in section 4.3.1.

<tags>

o Default:

Only the counts (the @total attribute)

o If include=tags:

All of the same tag XML as described in section 4.4.1.

<transcriptions>

o Default:

Only the attribute @hasTranscription which is “true” if a transcription exists and “false” otherwise.

o If include=transcript:

All of the same XML as described in section 4.5.1.

<translations>

o Default:

Has a total count of translations, plus nested tags for each language which is currently translated.

o If include=translations:

All of the same XML as described in section 4.6.1.

XML ExampleGET https://catalog.archives.gov/iapi/v1/contributions/summary/id/7226539?include=comments,tags

<annotations> <comments comments="3" replies="2" total="5"> <comment id="19328" user="kmartin" created="2014-02-10T14:36:23"

lastModified="2014-02-10T14:40:12"> <text>One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if there

was real affection between the two.</text> <replies> <reply id="45239" user="mstewart" fullName="Meredith Stewart" naraStaff="true"

created="2014-02-10T15:00:03"> <text>True, but from different generations.</text> </reply> <reply id="45422" user="mkoneni" created="2014-03-08T04:12:52"

lastModified="2014-03-08T04:18:02">

112


<text>I'm sure they ran into each other a bunch, you know at charity functions and golf tournaments...</text>

</reply> </replies> </comment> <comment id="19102" user="tjefferson" created="2014-03-08T04:12:52" isDeleted="true"/> <comment id="17024" user="rbarnes" created="2014-03-09T11:14:27"> <text>Seems rather formulaic to me. I bet a staff member wrote it.</text> </comment> </comments> <tags total="3"> <tag text="Truman Post President" user="jdoe" created="2014-02-21T16:22:58"/> <tag text="McGovern Letters" user="agullet" created="2014-02-27T18:23:47"/> <tag text="nara:president=Truman" user="mstewart" fullName="Meredith Stewart"

naraStaff="true" created="2014-03-19T07:07:42"/> </tags> <transcription hasTranscription="true"/> <translations total="3"> <language code="ENG"/> <language code="ITA"/> <language code="DEU"/> </translations></annotations>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/summary/id/7226539?include=comments,tags&format=json

{ "comments":{ "@comments":"3", "@replies":"2", "@total":"5", "comment":[ { "@id":"19328", "@user":"kmartin", "@created":"2014-02-10T14:36:23", "@lastModified":"2014-02-10T14:40:12", "text":"One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if

there was real affection between the two.", "replies":{ "reply":[ { "@id":"45239", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-02-10T15:00:03", "text":"True, but from different generations." }, { "@id":"45422", "@user":"mkoneni", "@created":"2014-03-08T04:12:52", "@lastModified":"2014-03-08T04:18:02", "text":"I'm sure they ran into each other a bunch, you know at charity functions and

golf tournaments..." } ] } }, { "@id":"19102", "@user":"tjefferson", "@created":"2014-03-08T04:12:52", "@isDeleted":"true"

113


}, { "@id":"17024", "@user":"rbarnes", "@created":"2014-03-09T11:14:27", "text":"Seems rather formulaic to me. I bet a staff member wrote it." } ] }, "tags":{ "@total":"3", "tag":[ { "@text":"Truman Post President", "@user":"jdoe", "@created":"2014-02-21T16:22:58" }, { "@text":"McGovern Letters", "@user":"agullet", "@created":"2014-02-27T18:23:47" }, { "@text":"nara:president=Truman", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-03-19T07:07:42" } ] }, "transcription":{ "@hasTranscription":"true" }, "translations":{ "@total":"3", "language":[ { "@code":"ENG" }, { "@code":"ITA" }, { "@code":"DEU" } ] }}

4.3 Comments APIUsers can make comments on archival descriptions, authority records, objects, documents, audiovisual files, and images, and they can reply to others’ comments. Users can edit their own comments and replies, and delete them. Moderators can also delete comments.

4.3.1 Fetching Comments and Replies

Comments, with their replies, can be fetched using the following URLs:

Authority records:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/ comments

114



GET https://catalog.archives.gov/iapi/v1/id/<NAID> /comments

Digital Objects:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /comments

Response

The <comments> returned by these URLs will contain the following information:

@total – Total count of all comments and replies on the item.

comment – Contains the text and replies for a comment. Has the following sub-elements:

o @id – The unique identifier of the comment.

This can be used to permalink the comment.

o @isEditable – “true” if the user can modify the text of the comment, “false” otherwise.

o @user – The user ID of the person who created the comment.

o @fullName – The full name of the user.

The @fullName is only included if the user wishes for the full name to be shown in place of their user ID.

o @naraStaff – “true” if the user is NARA staff. “false” otherwise.

o @lastModified – The date-time when the comment was created or last edited.

o @isDeleted – “true” if the comment has been deleted. “false” otherwise.

o text – The actual text of the comment.

o replies – Container for the replies of the comment. Has the following sub-elements:

@total – Total count of the replies for a comment.

reply – Holds the information for a single reply. Has the following sub-elements:

- @id – The unique identifier of the reply.

- @isEditable – “true” if the user can modify the text of the reply, “false” otherwise.

- @user – The user ID of the person who created the reply.

- @fullName – The full name of the replying user.o The @fullName is only included if the user wishes for the full

name to be shown in place of their user ID.- @naraStaff – “true” if the replying user is NARA staff. “false”

otherwise.

115


- @lastModified – The date-time when the reply was created or last edited.

- @isDeleted – “true” if the comment has been deleted. “false” otherwise.

- text – The actual text of a reply to the comment.

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539 /comments

<comments comments="3" replies="2" total="5"> <comment id="19328" user="kmartin" created="2014-02-10T14:36:23"

lastModified="2014-02-10T14:40:12"> <text>One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if there

was real affection between the two.</text> <replies> <reply id="45239" user="mstewart" fullName="Meredith Stewart" naraStaff="true"

created="2014-02-10T15:00:03"> <text>True, but from different generations.</text> </reply> <reply id="45422" user="mkoneni" created="2014-03-08T04:12:52"

lastModified="2014-03-08T04:18:02"> <text>I'm sure they ran into each other a bunch, you know at charity functions and golf

tournaments...</text> </reply> </replies> </comment> <comment id="19102" user="tjefferson" created="2014-03-08T04:12:52" isDeleted="true"/> <comment id="17024" user="rbarnes" created="2014-03-09T11:14:27"> <text>Seems rather formulaic to me. I bet a staff member wrote it.</text> </comment></comments>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1/id/7226539 /comments&format=json

"comments":{ "@comments":"3", "@replies":"2", "@total":"5", "comment":[ { "@id":"19328", "@user":"kmartin", "@created":"2014-02-10T14:36:23", "@lastModified":"2014-02-10T14:40:12", "text":"One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if

there was real affection between the two.", "replies":{ "reply":[ { "@id":"45239", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-02-10T15:00:03", "text":"True, but from different generations." }, { "@id":"45422", "@user":"mkoneni", "@created":"2014-03-08T04:12:52", "@lastModified":"2014-03-08T04:18:02",

116


"text":"I'm sure they ran into each other a bunch, you know at charity functions and golf tournaments..."

} ] } }, { "@id":"19102", "@user":"tjefferson", "@created":"2014-03-08T04:12:52", "@isDeleted":"true" }, { "@id":"17024", "@user":"rbarnes", "@created":"2014-03-09T11:14:27", "text":"Seems rather formulaic to me. I bet a staff member wrote it." } ]}

4.3.1.1 Fetching a Single Comment or Reply

To fetch a single comment and its associated replies add the comment ID to the end of the appropriate “comments” URL:

Authority Records:

GET https://catalog.archives.gov/iapi/v1/id /<NAID> /comments/<comment-id>

GET https://catalog.archives.gov/iapi/v1/id /<NAID> /comments/<comment-id>


GET https://catalog.archives.gov/iapi/v1/id/<NAID> /comments/<comment-id>

Digital Objects:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id>/comments/<comment-id>

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539 /comments/19328

<comment id="19328" user="kmartin" created="2014-02-10T14:36:23" lastModified="2014-02-10T14:40:12"> <text>One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if there

was real affection between the two.</text> <replies> <reply id="45239" user="mstewart" fullName="Meredith Stewart" naraStaff="true" created="2014-02-10T15:00:03"> <text>True, but from different generations.</text> </reply> <reply id="45422" user="mkoneni" created="2014-03-08T04:12:52" lastModified="2014-03-08T04:18:02"> <text>I'm sure they ran into each other a bunch, you know at charity functions and golf

tournaments...</text> </reply> </replies></comment>

117


JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539 /comments/19328?

format=json

{ "@id":"19328", "@user":"kmartin", "@created":"2014-02-10T14:36:23", "@lastModified":"2014-02-10T14:40:12", "text":"One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if there

was real affection between the two.", "replies":{ "reply":[ { "@id":"45239", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-02-10T15:00:03", "text":"True, but from different generations." }, { "@id":"45422", "@user":"mkoneni", "@created":"2014-03-08T04:12:52", "@lastModified":"2014-03-08T04:18:02", "text":"I'm sure they ran into each other a bunch, you know at charity functions and golf

tournaments..." } ] }}

4.3.2 Modifying Comments and Replies

This section deals with creating, editing, and deleting comments and/or replies. If a comment or a reply is to be edited or deleted, the comment-id (for comments) or both the comment ID and the reply ID (for replies) are given in the URL. For example:

Creating comments:

POST . . . /comments?text=<new comment text here>

Updating comments:

PUT . . . /comments/<comment-id>?text=<new comment text here>

Deleting comments:

DELETE . . . /comments/<comment-id>

Creating replies:

POST . . . /comments/<comment-id>?text=<new reply text here>

Updating replies:

PUT . . . /comments/<comment-id>/<reply-id>?text=<new reply text here>

Deleting replies:

118


DELETE . . . /comments/<comment-id>/<reply-id>

4.3.2.1 Parameters

The following table shows the Comments API parameters.

Parameter Description Examplestext=comment-or-

reply-textSpecifies the text of the comment or reply to be created or used for the update (edit).

text=In my opinion the reason for the change had to do with the prevailing political opinion. . .

Examples:

Creating a new comment on a description:

POST https://catalog.archives.gov/iapi/v1/id/7226539 /comments?text=Comment text goes here . . .

Creating a new comment on a digital object:

POST https://catalog.archives.gov/iapi/v1/id/7226539/objects/263/comments?text=Comment text goes here . . .

Creating a new comment on an authority record:

POST https://catalog.archives.gov/iapi/v1/id /1090294 /comments?text=Comment text goes here . . .

Updating a comment:

PUT https://catalog.archives.gov/iapi/v1/id/7226539 /comments/19328?text=Comment text goes here . . .

Deleting a comment:

DELETE https://catalog.archives.gov/iapi/v1/id/7226539/comments/19328

Updating a reply:

PUT https://catalog.archives.gov/iapi/v1/id/7226539 /comments/19328/152?text=Reply text goes here . . .

Deleting a reply:

DELETE https://catalog.archives.gov/iapi/v1/id/7226539 /comments/19328/152

4.3.2.2 Response

A "200" response indicates that the comment or reply was created/updated/deleted successfully. See section 1.5.2 for more information on the general response structure.

When creating replies or comments, the new comment ID / reply ID will be returned in the response.

119


XML Example (create comment)POST https://catalog.archives.gov/iapi/v1/id/7226539/comments?

text=Comment text goes here . . .

<opa-response> <header status="200" time="231"> <request path="/iapi/v1/holdings/7226539/comments"> <action>create</action> <text>Comment text goes here . . .</text> </request> </header>

<comment id="19423"/></opa-response>

JSON Example (create comment)POST https://catalog.archives.gov/iapi/v1/id/7226539/comments?format=json&

text=Comment text goes here . . .

{ "header":{ "@status":"200", "@time":"231", "request":{ "@path":"/iapi/v1/holdings/7226539/comments", "action":"create", "text":"Comment text goes here . . ." } }, "comment":{ "@id":"19423" }}

XML Example (create a reply to a comment)POST https://catalog.archives.gov/iapi/v1/id/7226539/comments/19423?

text=Reply text goes here. . .

<opa-response> <header status="200" time="231"> <request path="/iapi/v1/holdings/7226539/comments/19423"> <action>create</action> <text>Reply text goes here. . .</comment> </request> </header> <comment id="19423"> <reply id="15823"/> </comment></opa-response>

JSON Example (create a reply to a comment)POST https://catalog.archives.gov/iapi/v1/id/7226539/comments/19423?format=json&

text=Reply text goes here. . .

"opa-response":{ "header":{ "@status":"200", "@time":"231", "request":{ "@path":"/iapi/v1/holdings/7226539/comments/19423", "action":"create",

120


"text":"Reply text goes here. . ." } }, "comment":{ "@id":"19423", "reply":{ "@id":"15823" } }}

4.3.2.3 Error Codes

The following error codes are currently defined as part of the API response:


EMBEDDED_HTML The comment or reply text contains embedded HTML characters which implies a potential cross-site scripting attack.


COMMENT_SIZE_EXCEEDED

The size of the comment or reply (number of characters) is too large. Comments and replies cannot be larger than [TBD comment size limit] UTF-8 characters long.

request: The size of the comment or reply requested.max: The maximum number of characters allowed in a comment or reply.

COMMENT_NOT_FOUND The specified comment or reply doesn’t exist and cannot be deleted, edited or replied to.

comment: The identifier value for the comment or reply that does not exist.

REPLY_NOT_FOUND The specified reply doesn’t exist (or doesn’t exist within the specified comment ID) and cannot be deleted, edited or replied to.

comment: The identifier of the comment.reply: The identifier of the reply.

ILLEGAL_CHARACTERS Comment or reply rejected because it contains illegal (for example, non-printable) characters.

none

NOT_LOGGED_IN Attempted to create or modify a comment or reply without logging in.

none

NOT_OWNER Attempted to edit or delete someone else’s comment or reply.

request: The user ID who requested to delete the comment or reply.owner: The user ID who owns the comment or reply.

4.3.3 Bulk Import of Comments

When bulk importing comments, the following elements are expected:

comments – Identifies that this is a list of comments.

naId – Specify the NARA ID for the comment.

121


o NOTE: No “type” is required (e.g. “person”, “organization”, “description”, etc.).

objectId – [OPTIONAL] Specify the object ID (within the specified NAID) of the object to which the comment is attached.

text – Specifies the text of the comment.

JSON Example:

{ "comments":[ { "naId":"19328", "text":"One wonders if this letter to Frm Pres. Truman was felt to be obligatory, or if

there was real affection between the two." }, { "naId":"17024", "objectId":"32", "text":"Seems rather formulaic to me. I bet a staff member wrote it." } ]}

4.4 Tagging APIUsers can download tags for archival descriptions, authority records, and digital objects.

A logged-in user can delete their own tags. There is no facility for updating a tag, instead you would need to delete the tag and add a new one.

Users can also create machine tags, which look like the following:

geo:latitude=70.232

Which will be encoded in the URL as follows:

text=geo%3Alatitude%3D70.232

4.4.1 Fetching Tags

Tags can be fetched using the following URLs:

Authority Records:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/tags


GET https://catalog.archives.gov/iapi/v1/id/<NAID>/tags

Digital Objects:

GET https://catalog.archives.gov/iapi/v1/id/<naid>/objects/<obj-id> /tags

122


The content returned by these URLs will contain the following information inside <tags>:

@total – Total count of all tags on the item.

tag – Detailed information on an individual tag. This contains the following sub-elements:

o @text – The actual text of the tag.

Tag text is considered to be case insensitive when matching tags and all whitespace is normalized.

o @user – The user ID of the person who created the tag.

o @fullName – The full name of the user.

The @fullName tag is only included if the user wishes for the full name to be shown in place of their user ID.

o @isNaraStaff – “true” if the user is NARA staff. “false” otherwise.

o @created – The date-time when the tag was created.

4.4.1.1 Fetching a list of tags

To fetch all of the tags on an archival description, authority record, or digital object, fetch the contents of the appropriate “tags” URL.

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/tags

<tags total="3"> <tag text="Truman Post President" user="jdoe" created="2014-02-21T16:22:58"/> <tag text="McGovern Letters" user="agullet" created="2014-02-27T18:23:47"/> <tag text="nara:president=Truman" user="mstewart" fullName="Meredith Stewart" naraStaff="true"

created="2014-03-19T07:07:42"/></tags>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/tags&format=json

{ "@total":"3", "tag":[ { "@text":"Truman Post President", "@user":"jdoe", "@created":"2014-02-21T16:22:58" }, { "@text":"McGovern Letters", "@user":"agullet", "@created":"2014-02-27T18:23:47" }, { "@text":"nara:president=Truman", "@user":"mstewart", "@fullName":"Meredith Stewart", "@naraStaff":"true", "@created":"2014-03-19T07:07:42"

123


} ]},

4.4.1.2 Fetching a single tag

To fetch a single tag, add the tag to the end of the URL:

Authority Records:

GET https://catalog.archives.gov/iapi/v1/id /<NAID>/tags?text=<tag>


GET https://catalog.archives.gov/iapi/v1/id/<NAID>/tags?text=<tag>

Digital Objects:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /tags?text=<tag>

Note that special characters (such as spaces and ‘#’) will need to be URL encoded.

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/tags?text=McGovern%20Letters

<tag text="McGovern Letters" user="agullet" created="2014-02-27T18:23:47"/>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/tags?text=McGovern%20Letters&format=json

{ "@text":"McGovern Letters", "@user":"agullet", "@created":"2014-02-27T18:23:47"}

4.4.2 Modifying Tags

Tags can be modified using the appropriate HTTP method and specifying the text of the tag:

Creating tags:

POST . . . /tags?text=<new tag text here>

Deleting tags:

DELETE . . . /tags?text=<tag>

124


Parameters

The following table shows the Tagging API parameters.

Parameter Description Examplestext=tag-text Specify the text of the tag to be added or

deleted.Multiple text= parameters can be specified to write multiple tags.

text=McGovern%20Letters

text=nara%3Alatitude%3d47.22(note encoding of special URL

characters)

Examples:

Creating a new tag on a description:

POST https://catalog.archives.gov/iapi/v1/id/7226539/tags?text=VP%20Candidate%20Shriver

Creating two new tags on a description:

POST https://catalog.archives.gov/iapi/v1/id/7226539/tags?text=VP%20Candidate%20Shriver&text=Kennedy%20Politics

Creating a new tag on a digital object:

POST https://catalog.archives.gov/iapi/v1/id/7226539/objects/263/tags?text=VP%20Candidate%20Shriver

Creating a new tag on an authority record:

POST https://catalog.archives.gov/iapi/v1/id /1090294/tags?text=VP%20Candidate%20Shriver

Deleting a tag:

DELETE https://catalog.archives.gov/iapi/v1/id/7226539/tags?text=VP%20Candidate%20Shriver

Response

The response will the standard Catalog response (see section 1.5.2 for more information).

4.4.2.1 Error Codes

The following error codes are currently defined as part of the API response.

Note that if multiple tags are specified (i.e. with multiple “text” parameters) then there may be multiple errors in the response (for every tag which was in error).

No tag will be added unless all tags are error-free.

125



EMBEDDED_HTML The tag text contains embedded HTML characters which implies a potential cross-site scripting attack.


TAG_SIZE_EXCEEDED The size of the tag (number of characters) is too large. Tags cannot be larger than [TBD tag size limit] UTF-8 characters long.

tag: The tag value which exceeds the size limit.request: The size of the tag requested.max: The maximum number of characters allowed in a tag.

TAG_ALREADY_EXISTS The specified tag has already been created by someone else.

tag: The tag value which already exists.

ILLEGAL_CHARACTERS Tag rejected because it contains illegal (for example, non-printable) characters.

none

NOT_LOGGED_IN This will be returned for users who attempt to create a tag without logging in.

none

NOT_OWNER This will be returned for users who attempt to delete someone else’s tag.

request: The user ID who requested to delete the tag.owner: The user ID who owns the tag.

4.4.3 Bulk Import of Tags

When bulk importing tags, the following elements are expected:

tags – Identifies that this is a list of tags.

naId – Specify the NARA ID for the tag.


objectId – [OPTIONAL] Specify the object ID (within the specified NAID) of the object to which the tag is attached.

text – Specifies the text of the tag.

JSON Example

{ "tags":[ { "naId":"19328", "text":"Truman Post President" }, { "naId":"19328", "text":"McGovern Letters" }, { "naId":"7205115", "objectId":"1936360", "text":"nara:president=Truman"

126


} ]}

4.5 Transcription APIUsers can create transcriptions for digital objects stored in Catalog. Each object can have at most one transcription. Users can also edit transcriptions – both those they create and those created by others, with one exception. Only authorized users can modify an authoritative transcription.

Notes:

You must be logged in to create or modify transcriptions.

Anyone can edit anyone else’s transcription.

Only moderators can delete transcriptions.

4.5.1 Fetching a Transcription

Note that there is only one transcription per digital object. A transcription can be fetched using the following URL:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /transcriptions

ResponseThe URL will return the following information in the <transcription> tag:

@lastModified – The date-time when the transcription was last modified.

@isLocked – “true” if the transcription is currently locked by another user, “false” otherwise .

@isAuthoritative – “true” if this is an authoritative transcription, “false” otherwise.

@version – The version number of the latest transcription.

o This number will increment for every modification (including deletes and restores) made to the transcription.

lockedBy – Holds information on the user who has locked the transcription. Contains the following sub-elements:

o @id – The user ID of the user (for example, “kmartin”).

o @fullName – The full name of the user. Attribute is only present if the user wishes to make their full name public.


o @when – The date-time when the lock was initiated.

127


users – Parent tag which contains a list of the users who have contributed to this transcription. Contains the following sub-elements:

o @total – Total number of unique users who have contributed to the transcription.

o user – Information on an individual user who helped contribute to the transcription. Contains the following sub-elements:

@id – The user ID of the user (for example, “kmartin”).

@fullName – The full name of the user. Attribute is only present if the user wishes to make their full name public.

@isNaraStaff – “true” if the user is NARA staff. “false” otherwise.

@lastModified – The date-time when this user last modified the transcription.

text – The actual text of the transcription.

Note that users will be automatically sorted by @lastModified date (most recent modifier first).

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects/263/transcriptions

<transcription lastModified="2014-02-21T16:22:58" isLocked="true" isauthoritative="false" version="232">

<lockedBy id="gwashington" when="2014-02-22T10:08:41"/> <users total="3"> <user id="tjefferson" fullName="Thomas Jefferson" isNaraStaff="true"

lastModified="2014-02-21T16:22:58" /> <user id="mstewart" fullName="Meredith Stewart" isNaraStaff="true"

lastModified="2014-02-08T12:59:49"/> <user id="kmartin" lastModified="2014-02-02T08:44:12" /> </users> <text>GEORGE McGOVERNUnited States SenateWashington, D.C. 20510

August 19, 1972

The Honorable Harry S. TrumanIndependence, Missouri


It is an honor for me, as our party'scandidate for President of the United States. . .</text></transcription>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /transcriptions?

format=json

{ "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", "lockedBy":{

128


"@id":"gwashington", "@when":"2014-02-22T10:08:41" }, "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Thomas Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"mstewart", "@fullName":"Meredith Stewart", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" }, { "@id":"kmartin", "@lastModified":"2014-02-02T08:44:12" } ] }, "text":"GEORGE McGOVERN\nUnited States Senate\nWashington, D.C. 20510\n\nAugust 19, 1972\n\nThe

Honorable Harry S. Truman\nIndependence, Missouri\n\nMy dear Mr. President:\n\n It is an honor for me, as our party's\ncandidate for President of the United States. . ."

}

4.5.2 Modifying Transcriptions

The APIs provide a method to “lock” a transcription so that it cannot be edited by other users during the current user’s session.

When a transcription is fetched information about its lock status – @isLocked – at that moment is returned. However, the transcription could be locked by another user before the current user begins editing. If it is locked, the API will return an error, and indicate what username currently holds the lock.

Use the following URL to lock, unlock, or save the transcription:

PUT https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /transcriptions?params…

Notes:

The user must be logged in to modify transcriptions.

Only authorized users (NARA Staff) can modify or create authoritative transcriptions

You must obtain a lock before you can save a transcription.

129


4.5.2.1 Parameters

Parameter Description Examplesaction=

lock|saveAndRelock|saveAndUnlock|unlock

Indicates what to do with the transcription. Choices are: lock – obtain a lock for the transcription,

if not locked, so the transcription can be edited.

o Locks expire after 30 minutes.o To refresh the lock, use

action=lock again.o action=lock will return a fresh copy

of the current transcription. saveAndRelock – save the transcription

and keep the lock.o The lock is refreshed so that the

user has 30 more minutes before the lock expires.

saveAndUnlock – save the transcription and release the lock.

unlock – cancel the editing session and release the lock.

action=lock

action=saveAndRelock

action=saveAndUnlock

action=unlock

text=text-of-transcript

Specifies the text of the transcript to be created or used for the replacement (saving).

text=This%20document%20is%20the%20reference%20guide%20for%20the%20OPA%20APIs

(note encoding of special URL characters)

isAuthoritative=true | false

Set the “isAuthoritative” flag for the transcription. Can only be done by authorized users (NARA Staff).

isAuthoritative=true

Examples:

Locking a transcription for edit:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /transcriptions?action=lock

Saving a transcription on a digital object and releasing the lock:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /transcriptions?action=saveAndUnlock&text=The text of the transcription goes here . . .

Saving a transcription on a digital object and keeping the lock:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /transcriptions?action=saveAndRelock&text=The text of the transcription goes here . . .

130


Releasing the lock without saving anything:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /transcriptions?action=unlock

4.5.2.2 Response for action=lock

To absolutely ensure that the user is editing the latest version of a transcription, the response for action=lock will return a fresh copy of the <transcription> as specified above (section 4.5.1).

Note that, like all actions, the transcription returned will be nested inside of an <opa-response> tag (see examples below). If the lock can not be obtained, an error is returned.

XML ExamplePUT https://catalog.archives.gov/iapi/v1/id/7223928/objects/263 /transcriptions?action=lock

<opa-response> <header status="200" time="132">

<request path="/iapi/v1/holdings/7223928/objects/263/annotations/transcription"> <action>lock</action>

</request> </header> <transcription lastModified="2014-02-21T16:22:58"

isLocked="true" isauthoritative="false" version="232"> . . Transcription users and text will be here (see section 4.5.1) . This is the text that should be provided to the user to edit. . </transcription></opa-response>

JSON ExamplePUT https://catalog.archives.gov/iapi/v1/id/7223928/objects/263 /transcriptions?action=lock

{ "header":{ "@status":"200", "@time":"132", "request":{ "@path":"/iapi/v1/holdings/7223928/objects/263/annotations/transcription", "action":"lock" } }, "transcription":{ "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", . . Transcription users and text will be here (see section 4.5.1) . This is the text that should be provided to the user to edit. .}

4.5.2.3 Response for Other Actions


131


Error Codes



EMBEDDED_HTML The transcription text contains embedded HTML characters which imply a potential cross-site scripting attack.


NO_LOCK Tried to save a transcription to a digital object without first acquiring a lock.

none

LOCKED_BY_ANOTHER

Transcription currently locked by someone else. user: The username of the user currently editing the transcription.when: When the other user initiated the lock.

ILLEGAL_CHARACTERS Transcription rejected because it contains illegal (for example, non-printable) characters.

none

NOT_LOGGED_IN This will be returned for users who attempt to save or lock a transcription without logging in.

none

NOT_AUTHORIZED This will be returned for unauthorized users who attempt to edit an authoritative transcription, OR for unauthorized users who attempt to set the “isAuthoritative” flag.

request: The user ID who requested to edit the authoritative transcript.

4.5.3 Bulk Import of Transcriptions

When bulk importing transcriptions, the following elements are expected:

transcriptions – Identifies that this is a list of transcriptions.

naId – Specify the NARA ID for the transcription.


o This is optional for objects. Objects just need objectId

objectId – Specify the object ID (within the specified NAID) of the object to which the transcription is attached. Note: For transcriptions, the objectId is required.

text – Specifies the text of the transcription.

JSON Example

{ "transcriptions":[ { "naId":"7205115", "objectId":"1936351", "text":"GEORGE McGOVERN\n United States Senate\n Washington, D.C. 20510\n\n August

19, 1972\n\n The Honorable Harry S. Truman\n Independence, Missouri\n\n My dear

132


Mr. President:\n\n It is an honor forme, as our party's\n candidate for President of the United States. . ."

}, { "naId":"2668751", "objectId":"203578", "text":"GEORGE McGOVERN\n United States Senate\n Washington, D.C. 20510\n AIRMAIL" } ]}

4.6 Translation APIUsers can translate the text in digital objects and have those translations stored in Catalog. Users can choose to translate into almost any language (see section 8.4 for a list of all languages). A single digital object can hold multiple translations for multiple languages, but there can be only one translation for a particular language.

Users can also edit translations – both those they create and those created by others, with one exception. Only authorized users can modify an authoritative translation.

Notes:

You must be logged in to create or modify translations.

Anyone can edit anyone else’s translation.

Only moderators can delete translations.

4.6.1 Fetching a List of all Translations

A list of all available translations can be fetched from a digital object using the following URL:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /translations

Response

Parameter Description Examplesfull=true|false Specify if the full text and metadata should

be retrieved for all translations.Without full=true, a summary of the translations will be returned. See section 4.6.1.1.

full=true

Response

The data returned by this call includes the following inside the <translations> tag:

@total – The total number of available translations for the object.

133


translation/@code – The language code for an available translation.

o This is the ISO 639-3 (three-letter) code for the language.

o See section 8.4 for a list of the languages supported by NARA.

4.6.1.1 Fetching a summary of translations

The default method for fetching translations fetches a list of the languages which have been translated for the digital object:

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7223928/objects/293 /translations

<translations total="3"> <translation code="eng"/> <translation code="ita"/> <translation code="deu"/></translations>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7223928/objects/293 /translations?

format=json

{ "@total":"3", "translation":[ { "@code":"eng" }, { "@code":"ita" }, { "@code":"deu" } ]}

4.6.1.2 Fetching a list of full translations

Add the “full=true” parameter to fetch the complete translation text and metadata for all translations. See section 4.6.2 below for a complete description of all of the metadata fields found in a full translation.

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7223928/objects/293 /translations?

full=true

<translations total="3"> <translation code="ita" lastModified="2014-02-21T16:22:58" isLocked="true"

isauthoritative="false" version="232"> <lockedBy id="gwashington" when="2014-02-22T10:08:41"/> <users total="3"> <user id="tjefferson" fullName="Tomás Jefferson" isNaraStaff="true"

lastModified="2014-02-21T16:22:58" /> <user id="gwashington" fullName="Giorgio Washington" isNaraStaff="true"

lastModified="2014-02-08T12:59:49"/>

134


</users> <text>George McGovern


19 ago 1972



E 'un onore per me, come il nostro partito di candidato alla Presidenza degli Stati Uniti. .</text> </translation> <translation code="spa"> . . Full information on the Spanish translation goes here . </translation> <translation code="deu"> . . Full information on the German translation goes here . </translation></translations>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7223928/objects/293 /translations?

full=true&format=json

"translations":{ "@total":"3", "translation":[ { "@code":"ita", "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", "lockedBy":{ "@id":"gwashington", "@when":"2014-02-22T10:08:41" }, "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Tomßs Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"gwashington", "@fullName":"Giorgio Washington", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" } ] }, "text":"George McGovern \n\n Senato degli Stati Uniti \n Washington, D.C. 20510 \

n\n 19 ago 1972 \n\n L'Onorevole Harry S. Truman \n Independence, Missouri \n\n Mio caro Signor Presidente: \n\n E 'un onore per me, come il nostro partito di \n candidato alla Presidenza degli Stati Uniti. ."

}, { "@code":"spa",

135


. . Full information on the Spanish translation goes here . }, { "@code":"deu", . . Full information on the German translation goes here . } ]}

4.6.2 Fetching a Translation

Note that there is only one translation per digital object. A translation can be fetched using the following URL:

GET https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /translations/<lang-code>

The <lang-code> in the above example will be the ISO 639-3 (three-letter) code for the language. See section 8.4 for a list of all languages supported by NARA.

ResponseThe URL will return the following information in the <translation> tag:

@code – The language code of the language.

o This is the ISO 639-3 (three-letter) code for the language.

o See section 8.4 for a list of the languages supported by NARA.

@lastModified – The date-time when the translation was last modified.

@isLocked – “true” if the translation is currently locked by another user, “false” otherwise.

@isAuthoritative – “true” if this is an authoritative translation, “false” otherwise.

@version – The version number of the latest translation.

o This number will increment for every modification (including deletes and restores) made to the translation.

lockedBy – Holds information on the user who has locked the translation. Contains the following sub-elements:

o @id – The user ID of the user (for example, “kmartin”).

o @fullName – The full name of the user. Attribute is only present if the user wishes to make their full name public.


o @when – The date-time when the lock was initiated.

136


users – Parent tag which contains a list of the users who have contributed to this translation. Contains the following sub-elements:

o @total – Total number of unique users who have contributed to the translation.

o user – Information on an individual user who helped contribute to the translation. Contains the following sub-elements:

@id – The user ID of the user (for example, “kmartin”).

@fullName – The full name of the user. Attribute is only present if the user wishes to make their full name public.

@isNaraStaff – “true” if the user is NARA staff. “false” otherwise.

@lastModified – The date-time when this user last modified the translation.

text – The actual text of the translation.

Note that users will be automatically sorted by @lastModified date (most recent modifier first).

XML ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita

<translation code="ita" lastModified="2014-02-21T16:22:58"isLocked="true" isauthoritative="false" version="232">

<lockedBy id="gwashington" when="2014-02-22T10:08:41"/> <users total="3"> <user id="tjefferson" fullName="Tomás Jefferson" isNaraStaff="true"

lastModified="2014-02-21T16:22:58" /> <user id="gwashington" fullName="Giorgio Washington" isNaraStaff="true"

lastModified="2014-02-08T12:59:49"/> </users> <text>George McGovern


19 ago 1972



E 'un onore per me, come il nostro partito di candidato alla Presidenza degli Stati Uniti. .</text></translation>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita&

format=json

{ "@code":"ita", "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", "lockedBy":{

137


"@id":"gwashington", "@when":"2014-02-22T10:08:41" }, "users":{ "@total":"3", "user":[ { "@id":"tjefferson", "@fullName":"Tomás Jefferson", "@isNaraStaff":"true", "@lastModified":"2014-02-21T16:22:58" }, { "@id":"gwashington", "@fullName":"Giorgio Washington", "@isNaraStaff":"true", "@lastModified":"2014-02-08T12:59:49" } ] }, "text":"George McGovern \n\nSenato degli Stati Uniti \nWashington, D.C. 20510 \n\n19 ago 1972 \

n\nL'Onorevole Harry S. Truman \nIndependence, Missouri \n\nMio caro Signor Presidente: \n\n E 'un onore per me, come il nostro partito di \ncandidato alla Presidenza degli Stati Uniti. ."

}

4.6.3 Modifying Translations

The APIs provide a method to “lock” a translation so that it cannot be edited by other users during the current user’s session.

When a translation is fetched information about its lock status – @isLocked – at that moment is returned. However, the translation could be locked by another user before the current user begins editing. If it is locked, the API will return an error, and indicate what username currently holds the lock.

Use the following URL to modify the translation for a digital object:

PUT https://catalog.archives.gov/iapi/v1/id/<NAID>/objects/<obj-id> /translation/<lang-code>?params…

Notes:

The user must be logged in to modify translations.

Only authorized users (NARA Staff) can modify or create authoritative translations.

You must obtain a lock before you can save a translation.

138


4.6.3.1 Parameters

Parameters for modifying translations are given in the following table.


lock|saveAndRelock|saveAndUnlock|unlock

Indicates what to do with the translation. Choices are: lock – obtain a lock for the translation, if

not locked, so the translation can be edited.

o Locks expire after 30 minutes.o To refresh the lock, use

action=lock again.o action=lock will return a fresh copy

of the current translation. saveAndRelock – save the translation

and keep the lock.o The lock is refreshed so that the

user has 30 more minutes before the lock expires.

saveAndUnlock – save the translation and release the lock.

unlock – cancel the editing session and release the lock.

action=lock

action=saveAndRelock

action=saveAndUnlock

action=unlock

text=text-of-translation

Specifies the text of the translation to be created or used for the replacement (saving).

text=Il%20testo%20della%20traduzione%20va%20qui

(note encoding of special URL characters)

isAuthoritative=true | false

Set the “isAuthoritative” flag for the translation. Can only be done by authorized users (NARA Staff).

isAuthoritative=true

Examples:

Locking the Italian translation for edit:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?action=lock

Saving an Italian translation on a digital object and releasing the lock:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?action=saveAndUnlock&text=The text of the translation goes here . . .

139


Saving an Italian translation on a digital object and keeping the lock:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?action=saveAndRelock&text=Il testo della traduzione va qui. . .

Releasing the lock without saving anything:

PUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?action=unlock

4.6.3.2 Response for action=lock

To absolutely ensure that the user is editing the latest version of a translation, the response for action=lock will return a fresh copy of the <translation> as specified above (section 4.6.1).

The translation returned will be nested inside of an <opa-response> tag:

XML ExamplePUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?

action=lock

<opa-response> <header status="200" time="132">

<request path="/iapi/v1/holdings/7226539/content/263.jpg#translation"> <action>lock</action>

</request> </header> <translation code="ita" lastModified="2014-02-21T16:22:58"

isLocked="true" isauthoritative="false" version="232"> . . Translation users and text will be here (see section 4.5.1) . This is the text that should be provided to the user to edit. . </translation></opa-response>

JSON ExamplePUT https://catalog.archives.gov/iapi/v1/id/7226539/objects/263 /translations/ita?

action=lock&format=json

{ "header":{ "@status":"200", "@time":"132", "request":{ "@path":"/iapi/v1/holdings/7226539/content/263.jpg#translation", "action":"lock" } }, "translation":{ "@lastModified":"2014-02-21T16:22:58", "@isLocked":"true", "@isauthoritative":"false", "@version":"232", . . Translation users and text will be here (see section 4.5.1) . This is the text that should be provided to the user to edit. .}

140


4.6.3.3 Response for Other Actions

All other translation actions will the standard Catalog response (see section 1.5.2 for more information).

Error Codes



EMBEDDED_HTML The translation text contains embedded HTML characters which imply a potential cross-site scripting attack.


NO_LOCK Tried to save a translation to a digital object without first acquiring a lock.

none

LOCKED_BY_ANOTHER

Translation currently locked by someone else. user: The username of the user currently editing the translation.when: When the other user initiated the lock.

ILLEGAL_CHARACTERS Translation rejected because it contains illegal (for example, non-printable) characters.

none

NOT_LOGGED_IN This will be returned for users who attempt to save or lock a translation without logging in.

none

NOT_AUTHORIZED This will be returned for unauthorized users who attempt to edit an authoritative translation, OR for unauthorized users who attempt to set the “isAuthoritative” flag.

request: The user ID who requested to edit the authoritative translation.

4.6.4 Bulk Import of Translations

When bulk importing translations, the following elements are expected:

translations – Identifies that this is a list of translations.

naId – Specify the NARA ID for the translation.


o This is optional for objects. Objects just need objectId

objectId – Specify the object ID (within the specified NAID) of the object to which the translation is attached. Note: For translations, the objectId is required.

langCode – Specify the ISO-639-3 (three-letter) language code of the language of the translation.

text – Specifies the text of the translation.

141


JSON Example

{ "translations":[ { "naId":"19328", "objectId":"1", "langCode":"ita", "text":"George McGovern \n\n Senato degli Stati Uniti \n Washington, D.C.

20510 \n\n 19 ago 1972 \n\n L'Onorevole Harry S. Truman \n Independence, Missouri \n\n Mio caro Signor Presidente: \n\n E 'un onore per me, come il nostro partito di \n candidato alla Presidenza degli Stati Uniti. ."

}, { "naId":"19328", "objectId":"1", "langCode":"spa", "text":"George McGovern \n Senado de Estados Unidos \n Washington, DC 20510 \n\n

19 de agosto 1972 \n\n El Honorable Harry S. Truman \n Independence, Missouri \n\n Mi querido Señor Presidente: \n\n Es un honor para mí, como nuestro grupo de \n candidato a Presidente de los Estados Unidos. . ."

} ]}

142


5 User Contributions and Activities API

Once a user creates an account, he or she can annotate content, save search results to lists, and track downloads. This API provides capabilities to list annotations and activity-related information from the user perspective.

This section discusses fetching the contributions for a user, including summary information, but does not deal with creating or modifying annotations (see section 3.7).

5.1 User Summary InformationA summary of user information is available by accessing the user’s URLs:

GET https://catalog.archives.gov/iapi/v1/contributions/summary?fullstats=true&username=<user-name>


GET https://catalog.archives.gov/iapi/v1/exports/auth/summarystatus

GET https://catalog.archives.gov/iapi/v1/accounts/notifications

GET https://catalog.archives.gov/iapi/v1/accounts/profile/<user-name>

These will return a summary of all of the contributions made by the user, and (if the user requested is the same as the logged-in user) the number of lists, notifications, bulk-exports, and account information.

The information returned is shown below for all users:

id – The user ID of the user.

fullName – The user’s full name. Only shown if the user has chosen to display their full name to the public.

isNaraStaff – “true” if the user is a NARA staff member.

contributions – As summary of the contributions made by the user.

o See section 5.2 below for more details.

The information returned is shown below only of the logged-in user is the requested user:

notifications/total – A count of the unread notifications the user has.

o See section 5.5 for more information on notifications.

userLists/total – The total number of lists saved by the user.

o See section 5.3 for more information on lists.

143


accountExportsStatusSummary/Pending – The number of bulk exports in process for the user.

o See section 5.4 for more information on bulk exports.

accountExportsStatusSummary/Ccompleted – The number of bulk exports completed and waiting to be downloaded.

user – Detailed account information.

o See section 5.6 for more information on account information.

XML ExampleGET https://catalog.archives.gov/iapi/v1/accounts/profile/tjefferson

<user> <id>tjefferson</id> <type>registeredUser</type> <fullName>Thomas Jefferson</fullName> <displayFullname>true</displayFullname> <email>[email protected]</email></user>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/accounts/profile/tjefferson?format=json

{ "user":{ "id":"tjefferson", "type":"registeredUser", "fullName":"Thomas Jefferson", "displayFullname":"true", "email":"[email protected]" }}

5.2 User ContributionsAnnotations made by an individual user can be obtained by supplying the username of the user that created or modified them. For example, a user interface may wish to allow a user to view just their own annotations (“My Contributions”). It is also possible for a user to view other users’ contributions.

If multiple users have contributed to an annotation, the annotation can be obtained with any of those usernames.

Specific categories of user contributions can be fetched using the following URLs:

GET https://catalog.archives.gov/iapi/v1/contributions/summary?username=<user-name>

A summary of all of the user’s contributions. Can either be a simple count of contributions by type of annotation, or can contain more statistical information.

GET https://catalog.archives.gov/iapi/v1/contributions/comments?username=<user-name>

144


List of comments contributed by the user.

GET https://catalog.archives.gov/iapi/v1/contributions/tags?username=<user-name>

List of all unique tags contributed by the user.

GET https://catalog.archives.gov/iapi/v1/contributions/tags/titles?username=<user-name>&tagText=<tag>

List of everything tagged by the user with the specified tag.

GET https://catalog.archives.gov/iapi/v1/ contributions/transcriptions?username=<user-name>

List of transcriptions created or modified by the user.

GET https://catalog.archives.gov/iapi/v1 contributions/translations?username=<user-name>

List of translations created or modified by the user.

Note 1: The listings of entities that have transcriptions, translations, comments, and a particular tag can all be paginated. The listings are returned by default in reverse date order (newest first). Pagination and sorting are controllable by parameters given below in section 5.2.2 (common parameters for contributions).

Note 2: The “contributed” prefixed on these paths is used to avoid confusion with the standard “comments”, “tags”, “transcriptions” and “translations” tags because the two sets of tags return different metadata structures.

5.2.1 Contributions Summary

The contribution summary can be fetched using the following URL:

GET https://catalog.archives.gov/iapi/v1/contributions/summary?fullstats=true&username=<user-name>

Note that there are two versions of this URL. One provides the full statistical information, and a second (faster and more efficient) version only provides the total counts.

The simple statistics include the following:

@total – Total contributions made over all time.

tags/@total – Total tags created over all time.

comments/@total – Total comments (+ replies) contributed over all time.

transcriptions/@total – Total transcriptions contributed over all time.

translations/@total – Total translations contributed over all time.

Adding “fullStats=true” to the URL will return the following:

145


@totalMonth – Total contributions made this month.

@totalYear – Total contributions made over the previous year.

tags/@totalMonth – Total tags created this month.

tags/@totalYear – Total tags created this year.

comments/@totalMonth – Total comments (+ replies) contributed over the past month.

comments/@totalYear – Total comments (+ replies) contributed over the past year.

transcriptions/@totalMonth – Total transcriptions contributed in the past month.

transcriptions/@totalYear – Total transcriptions contributed in the past year.

translations/@totalMonth – Total translations contributed over the past month.

translations/@totalYear – Total translations contributed over the past year.

XML Example (simple counts)GET https://catalog.archives.gov/iapi/v1/contributions/summary?fullstats=false&username=tjefferson

<contributions DisplayFullNameFlag="true" isNaraStaff="true" total="74" userFullName="Thomas Jefferson"> <tags total="27"/> <transcriptions total="27"/> <comments total="10"/> <translations total="10"/> </contributions>

JSON Example (simple counts)GET https://catalog.archives.gov/iapi/v1/contributions/summary?fullstats=false?format=json

"contributions":{ "@userFullName":"Thomas Jefferson", "@DisplayFullNameFlag":"true", "@isNaraStaff":"true", "@total":"74", "tags":{ "@total":"27" }, "transcriptions":{ "@total":"27" }, "comments":{ "@total":"10" }, "translations” :{ "@total”:”10” } }

XML Example (full statistics)GET https://catalog.archives.gov/iapi/v1/contributions/summary?fullstats=true&username=tjefferson

<contributions DisplayFullNameFlag="true"

146


isNaraStaff="true" total="74" totalMonth="26" totalYear="74" userFullName="Thomas Jefferson"> <tags total="27" totalMonth="14" totalYear="27"/> <transcriptions total="27" totalMonth="10" totalYear="27"/> <comments total="10" totalMonth="1" totalYear="10"/> <translations total="10" totalMonth="1" totalYear="10"/> </contributions>

JSON Example (full statistics)GET https://catalog.archives.gov/iapi/v1/contributions/summary?

fullstats=true&username=tjefferson&format=json

"contributions":{ "@userFullName":"Thomas Jefferson", "@DisplayFullNameFlag":"true", "@isNaraStaff":"true", "@total":"74", "@totalMonth":"26", "@totalYear":"74", "tags":{ "@total":"27", "@totalMonth":"14", "@totalYear":"27" }, "transcriptions":{ "@total":"27", "@totalMonth":"10", "@totalYear":"27" }, "comments":{ "@total":"10", "@totalMonth":"1", "@totalYear":"10" }, "translations":{ "@total":"10", "@totalMonth":"1", "@totalYear":"10" }}

5.2.2 Common Parameters for Contributions

All of the user contribution methods support several common parameters for paging and sorting.

Parameter Description Examplesoffset=# The zero-based offset into the list of the first

result to be returned. This parameter is used in combination with ‘rows’ to paginate the list. If missing, the default offset is 0.If an offset less than 0 or greater than the total is specified, an error is returned.

offset=100

147


Parameter Description Examplesrows=# Indicates the number of entries in the list to

fetch. The default is 25.The maximum value for rows is 200. [TBD]

rows=50

sort=field [asc|desc] Sorts listing based on a field value. Specify the field to sort by followed by “asc” for ascending or “desc” for descending. The current fields are defined: lastModified – [all annotations] The date

the annotation was last modified. tag – [tag list only] The text of the tag. count – [tag list only] The number of

occurrences of the tag.

sort=lastModified asc

5.2.2.1 Error Codes

If the paging parameters are incorrect, then a standard error header will be returned (see section 1.5.2) with one of the following error codes:


INVALID_OFFSET An incorrect offset was requested. offset: The requested offset.avail: The number of rows available.

INVALID_ROWS A “rows” number less than zero was requested.

rows: The requested rows.

TOO_MANY_ROWS Too many rows were requested. rows: The requested rows.max: The maximum number of rows allowed.

INVALID_SORT Invalid sort field or sort direction requested. none

5.2.3 User Contributed Comments

User contributed comments can be fetched using the following URL:

GET https://catalog.archives.gov/iapi/v1/contributions/comments?username=<user-name>

The URL will accept the common parameters specified above in section 5.2.2.

Response

When the listing of entities (digital objects, authority records, and descriptions) annotated by comments for a particular user are requested, the following information is returned.

total – Total count of all comments made by the user.

comments/comment/@id – The ID for this comment.

148


o Uniquely identifies this ID within Catalog. Can be used to perma-link the comment.

comments/comment/@lastModified – Date this comment was last modified, or created (if no modifications have been made).

comments/comment/@replies – The number of replies for this comment.

comments/comment/@title – A display title for the contribution. Usually the title of the associated description or authority record.

For archival descriptions: (all contained within the <comments/comment> tag)

naId – The NAID of the description on which the comment was made.

For digital objects:

naId – The NAID of the description which contains the object.

objectId – The object ID on which the comment was made.

pageNum – The page number for the object.

totalPages – Total of pages for this description.

For person authority records:

naId – The person ID number.

For organization authority records:

naId – The organization ID number.

149


XML ExampleGET https://catalog.archives.gov/iapi/v1/contributions/comments?username=mstewart&offset=0

<comments> <total>3</total> <comment id="9012" lastModified="2015-06-29T16:54:50.000Z" parentId="0" replies="0" title="[Scotland - Dundee - Consulate] Register of Correspondence Received and

Sent"> <naId>1307945</naId> <totalPages>0</totalPages> <pageNum>0</pageNum> </comment> <comment id="9222" lastModified="2015-06-29T16:54:15.000Z" parentId="0" replies="2" title="Fall 1981: 8-91-1 An Interview with Richard Helms, by David Frost"> <naId>7283099</naId> <totalPages>31</totalPages> <pageNum>31</pageNum> </comment> <comment id="8889" lastModified="2015-06-29T16:53:40.000Z" parentId="0" replies="1" title="Spring 1981: 7-89-4: Harry S. Truman on CIA Covert Operations, by Hayden B.

Peake"> <naId>7283080</naId> <objectId>15692924</objectId> <totalPages>12</totalPages> <pageNum>2</pageNum> </comment> </comments>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/contributions/comments?

username=mstewart&offset=0&format=json

"comments":{ "total":3, "comment":[ { "@id":"9012", "@parentId":"0", "@title":"[Scotland - Dundee - Consulate] Register of Correspondence Received and Sent", "naId":"1307945", "totalPages":0, "pageNum":0, "@replies":"0", "@lastModified":"2015-06-29T16:54:50.000Z" }, { "@id":"9222", "@parentId":"0", "@title":"Fall 1981: 8-91-1 An Interview with Richard Helms, by David Frost", "naId":"7283099", "totalPages":31, "pageNum":31, "@replies":"2", "@lastModified":"2015-06-29T16:54:15.000Z" }, { "@id":"8889", "@parentId":"0",

150


"@title":"Spring 1981: 7-89-4: Harry S. Truman on CIA Covert Operations, by Hayden B. Peake",

"naId":"7283080", "objectId":"15692924", "totalPages":12, "pageNum":2, "@replies":"1", "@lastModified":"2015-06-29T16:53:40.000Z" } ] }

5.2.4 User Contributed Tags

5.2.4.1 Fetching all unique tags

The list of all unique contributed tags can be fetched using the following URL:

GET https://catalog.archives.gov/iapi/v1/contributions/tags?username=<user-name>


Sorting

The list of all unique tags for the user can be sorted by:

sort=tag

o Sort by the tag text.

sort=count

o Sort by the number of times the user has contributed the tag.

Default sort is by tag text.

Response

When the listing of tags created by a user is requested, the following information is returned.

tags/total – Total count of all tags made by the user..tags/tag/count – The number of entities tagged with this tag.

tags/tag/tag – The actual text of the tag.

XML ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/tags?username=mstewart&sort=tag%20DESC&offset=0

<tags> <total>2</total> <tag> <count>6</count> <tag>Cuba</tag> </tag> <tag>

151


<count>3</count> <tag>Bay of Pigs</tag> </tag> </tags>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/tags?username=mstewart&sort=tag%20DESC&offset=0&format=json

"tags":{ "total":2, "tag":[ { "count":6, "tag":"Cuba" }, { "count":3, "tag":"Bay of Pigs" } ] }

5.2.4.2 Fetching all contributions for a contributed tag

To fetch the list of contributions for a single contributed tag, use:

GET https://catalog.archives.gov/iapi/v1/ contributions/tags/titles?tagtext=<tag>&username=<user-name>


Response

When the listing of entities (digital objects, authority records, and descriptions) annotated by comments for a particular user are requested, the following information is returned.

titles/total – Total count of all contributions made by the user for the tag.

titles/title/addedTs – The date the tag was placed.

titles/title/opaTitle – A display title for the contribution. Usually the title of the associated description or authority record.

titles/title/naId – The NAID of the description on which the comment was made.


titles/title/objectId – The object ID on which the tag was placed.

titles/title/pageNum– A display value which helps identify the object to the user. Typically a page number.

152


XML ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/tags/titles?username=mstewart&tagtext=Bay%20of%20Pigs?offset=0

<titles> <title> <element> <addedTs>2015-06-29T19:59:57.000Z</addedTs> <naId>6920522</naId> <opaTitle>Cuba</opaTitle> <opaType>itemAv</opaType> <pageNum>0</pageNum> <totalPages>0</totalPages> </element> <element> <addedTs>2015-06-29T19:59:54.000Z</addedTs> <naId>7412730</naId> <opaTitle>633) Bay of Pigs</opaTitle> <opaType>fileUnit</opaType> <pageNum>0</pageNum> <totalPages>0</totalPages> </element> <element> <addedTs>2015-06-29T19:59:51.000Z</addedTs> <naId>7284003</naId> <objectId>15744634</objectId> <opaTitle>Winter 1979: 19-58-4: Book Reviews: Bay of Pigs, by Peter Wyden</opaTitle> <opaType>fileUnit</opaType> <pageNum>1</pageNum> <totalPages>4</totalPages> </element> </title> <total>3</total> </titles>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/tags/titles?username=mstewart&Bay%20of%20Pigs?offset=0&format=json

"titles":{ "total":3, "title":[ { "naId":"6920522", "opaTitle":"Cuba", "opaType":"itemAv", "pageNum":"0", "totalPages":"0", "addedTs":"2015-06-29T19:59:57.000Z" }, { "naId":"7412730", "opaTitle":"633) Bay of Pigs", "opaType":"fileUnit", "pageNum":"0", "totalPages":"0", "addedTs":"2015-06-29T19:59:54.000Z" }, { "naId":"7284003", "opaTitle":"Winter 1979: 19-58-4: Book Reviews: Bay of Pigs, by Peter Wyden", "opaType":"fileUnit", "objectId":"15744634",

153


"pageNum":"1", "totalPages":"4", "addedTs":"2015-06-29T19:59:51.000Z" } ] }

5.2.5 User Contributed Transcriptions

All user contributed transcriptions can be accessed through the following URL:

GET https://catalog.archives.gov/iapi/v1/contributions/transcriptions/titles?username=<user-name>


Response

When the listing of digital objects that have transcriptions by a particular user is requested, the following information is returned.

titles/total – Total count of all transcriptions made by the user.

titles/title – Information on a transcription contributed by the user.

titles/title/LastModifiedByUserTs – Date the transcription was last modified by or created (if no modifications have been made) the user.

titles/title/versionAddedTs – Date the transcription was last modified by someone else.

o Attribute may be missing of no one else has modified the transcription, or if the user was the last one to modify the transcription.

titles/title/ModifiedBy – The user ID of the other user who transcribed the object.

titles/title/ModifiedByDisplayName – The full name or username of the other user.

titles/title/opaTitle – A display title for the contribution. Usually the title of the associated description.

titles/title/naId – The NAID of the description on which the transcription was made.


titles/title/objectId – The object ID on which the transcription was made.

titles/title/pageNum – A display value which helps identify the object to the user. Typically a page number.

XML ExampleGET https://catalog.archives.gov/iapi/v1/contributions/transcriptions/titles?username=mstewart

<titles> <total>1</total> <title> <naId>2668751</naId> <opaTitle>Truman Doctrine</opaTitle>

154


<opaType>item</opaType> <objectId>14293115</objectId> <pageNum>1</pageNum> <totalPages>3</totalPages> <versionAddedTs>2015-06-29T20:55:36.000Z</versionAddedTs> <createdTs>2015-02-03T19:43:52.000Z</createdTs> <LastModifiedByUserTs>2015-06-29T20:47:44.000Z</LastModifiedByUserTs> <UserDisplayName>Mary Stewart</UserDisplayName> <LastModifiedByOtherTs>2015-06-29T20:55:36.000Z</LastModifiedByOtherTs> <ModifiedBy>tjefferson</ModifiedBy> <ModifiedByDisplayName>Thomas Jefferson</ModifiedByDisplayName> <CreatedBy>tjefferson</CreatedBy> <CreatedByDisplayName>Thomas Jefferson</CreatedByDisplayName> </title></titles>

JSON ExampleGET

https://catalog.archives.gov/iapi/v1/contributions/transcriptions/titles?username=mstewart&format=json

"titles":{ "total":1, "title":[ { "naId":"2668751", "opaTitle":"Truman Doctrine", "opaType":"item", "objectId":"14293115", "pageNum":"1", "totalPages":"3", "versionAddedTs":"2015-06-29T20:55:36.000Z", "createdTs":"2015-02-03T19:43:52.000Z", "LastModifiedByUserTs":"2015-06-29T20:47:44.000Z", "UserDisplayName":"Mary Stewart", "LastModifiedByOtherTs":"2015-06-29T20:55:36.000Z", "ModifiedBy":"tjefferson", "ModifiedByDisplayName":"Thomas Jefferson", "CreatedBy":"tjefferson", "CreatedByDisplayName":"Thomas Jefferson" } ] }

5.2.6 User Contributed Translations

All user contributed translations can be accessed through the following URL:

GET https://catalog.archives.gov/iapi/v1/contributions/translations?username=<user-name>


Response

When the listing of digital objects that have translations by a particular user is requested, the following information is returned.

translations/total – Total count of all translations made by the user.

translations/translation – Information on a translation contributed by the user.

155


translations/translation/LastModifiedByUserTs – Date the translation was last modified by or created (if no modifications have been made) the user.

translations/translation/versionAddedTs – Date the translation was last modified by someone else.

o Attribute may be missing of no one else has modified the translation, or if the user was the last one to modify the translation.

translations/translation/ModifiedBy – The user ID of the other user who translated the object.

translations/translation/ModifiedByDisplayName – The full name or username of the other user.

translations/translation/opaTitle – A display title for the contribution. Usually the title of the associated description.

translations/translation/naId – The NAID of the description on which the translation was made.

translations/translation/objectId – The object ID on which the translation was made.

translations/translation/pageNum – A display value which helps identify the object to the user. Typically a page number.

translations/translation/language – The full English name for the language.

translations/translation /languageCode – The ISO 639-3 3-letter code for the language.

XML ExampleGET https://catalog.archives.gov/iapi/v1/contributions/translations?username=mstewart

<translations> <total>1</total> <translation> <naId>2668751</naId> <opaTitle>Truman Doctrine</opaTitle> <opaType>item</opaType> <objectId>14293115</objectId> <pageNum>1</pageNum> <totalPages>3</totalPages> <language>Spanish</language> <languageCode>SPA</languageCode> <versionAddedTs>2015-06-29T20:55:36.000Z</versionAddedTs> <createdTs>2015-02-03T19:43:52.000Z</createdTs> <LastModifiedByUserTs>2015-06-29T20:47:44.000Z</LastModifiedByUserTs> <UserDisplayName>Mary Stewart</UserDisplayName> <LastModifiedByOtherTs>2015-06-29T20:55:36.000Z</LastModifiedByOtherTs> <ModifiedBy>tjefferson</ModifiedBy> <ModifiedByDisplayName>Thomas Jefferson</ModifiedByDisplayName> <CreatedBy>tjefferson</CreatedBy> <CreatedByDisplayName>Thomas Jefferson</CreatedByDisplayName> </translation></translations>

156


JSON ExampleGET https://catalog.archives.gov/iapi/v1/contributions/translations?username=mstewart&format=json

"translations":{ "total":1, "translation":[ { "naId":"2668751", "opaTitle":"Truman Doctrine", "opaType":"item", "objectId":"14293115", "pageNum":"1", "totalPages":"3", "language":"Spanish", "languageCode":"SPA", "versionAddedTs":"2015-06-29T20:55:36.000Z", "createdTs":"2015-02-03T19:43:52.000Z", "LastModifiedByUserTs":"2015-06-29T20:47:44.000Z", "UserDisplayName":"Mary Stewart", "LastModifiedByOtherTs":"2015-06-29T20:55:36.000Z", "ModifiedBy":"tjefferson", "ModifiedByDisplayName":"Thomas Jefferson", "CreatedBy":"tjefferson", "CreatedByDisplayName":"Thomas Jefferson" } ] }

5.3 My ListsUsers can manage their saved search results lists from their account. They can rename and delete lists, delete all lists, and delete specific entries from a list. Note that creating a new list and adding results to a list occurs as part of a search.

Information about a user’s lists can be accessed using the following URLs:


Fetch a list of all lists for a specific user

GET https://catalog.archives.gov/iapi/v1/lists/viewentries/<list-name>?username=<user-name>

Fetch information for a specific list

Note:

Lists are only available when logged in (see section 1.6).

o Except share lists which have their persistent url.

The logged in user can only fetch lists for themselves (not for others)

5.3.1 Managing the List of Lists

5.3.1.1 Fetching the Lists of Lists

The following URL can be used to fetch the list of all of the user’s saved lists:

GET https://catalog.archives.gov/iapi/v1//lists/view

157


Note that all lists are always retrieved. There is no pagination of the list of lists.

Response

The following attributes are provided for the list of the user’s lists:

userLists/total – The total number of lists the user has.

userLists/userList/@name – The list name. This information is repeated for each of the user’s lists.

userLists/userList/total – The total number of results in the list.

userLists/userList/@created – The date time the list was created.

userLists/userList/@lastModified – The date time the list was last modified.

The following examples show how to obtain summary information about all lists for a specific user.

XML ExampleGET https://catalog.archives.gov/iapi/v1/lists/view

<userLists> <total>2</total> <userList name="Truman Info" total="66" lastModified="2015-06-29 21:52:16.0"

created="2015-06-29 21:52:16.0"/> <userList name="Kennedy search" total="150" lastModified="2015-06-29 21:52:16.0"

created="2015-06-29 21:52:16.0"/></userLists>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/lists/view?format=json

"userLists":{ "total":2, "userList":[ { "@name":"Truman Info", "total":66, "@lastModified":"2015-06-29 21:52:16.0", "@created":"2015-06-29 21:52:16.0" }, { "@name":"Kennedy search", "total":150, "@lastModified":"2015-06-29 21:52:16.0", "@created":"2015-06-29 21:52:16.0" } ] }

5.3.1.2 Modifying the List of Lists

The list of lists can be modified. Single actions are provided to delete all lists or create a new, empty list.

158


To delete all lists:

DELETE https://catalog.archives.gov/iapi/v1/lists/deleteall

To create a new list:

POST http://research.archives.gov/iapi/v1/lists/create?listname=<list-name>

Parameters

Parameter Description Exampleslistname=listName Indicates the name of the list.

In this context, only used when action=create.

listname=listname

Examples

Delete all lists:


Create a new list called “Hello World”:

POST https://catalog.archives.gov/iapi/v1/lists/create?listname=Hello+World

Response


Error Codes

Creating a new list could return the following error code:


DUPLICATE_LIST The list name specified already exists. list: The list name requested.

NOT_LOGGED_IN Attempted to modify a list when not logged in.

none

NOT_OWNER Attempted to modify someone else’s list. request: The user ID attempting to modify the list.

5.3.2 Managing a Single List

Single lists can be renamed and deleted. This is done with the following URLs:

Rename a list:

PUT https://catalog.archives.gov/iapi/v1//lists/rename?listname=<list-name>&newname=<new-name>

159


Delete a list:

DELETE https://catalog.archives.gov/iapi/v1/lists/delete/<listname>

Parameters

Parameter Description Exampleslistname=

oldListNameSpecifies the name for the list listname=Truman%20List

newname=newListName

Specifies the new name for the list, when action=rename

newName=Truman%20Info

Examples

Delete the “Truman Old” list:

DELETE https://catalog.archives.gov/iapi/v1/lists/delete/Truman%20Old

Rename the list “Truman Old” to “Truman New” in the “mstewart” account:

PUT https://catalog.archives.gov/iapi/v1/lists/rename?listname=Truman%20Old&newname=Truman%20New

Response


Error Codes




NOT_LOGGED_IN Attempted to modify a list when not logged in. none


5.3.3 Creating or Adding to a List using the Search API

Lists can be created and built using the search API, with the following URLs:

To save search results to a new list:

POST https://catalog.archives.gov/iapi/v1?action=newList&search-params . . &list=<list-name>

To add results to an existing list:

POST https://catalog.archives.gov/iapi/v1?action=addList&search-params . . &list=<list-name>

160


Examples

Save the first 10 descriptions which mention the word “truman” to a new list called “HTList”:

POST https://catalog.archives.gov/iapi/v1?action=newList&q=truman&type=description&rows=10&list=HTList

Add results 11-20 to “HTList”:

POST https://catalog.archives.gov/iapi/v1?action=addList&q=truman&type=description&rows=10&offset=10&list=HTList

Response


Error Codes




MISSING_LIST The “list” parameter is missing when action=addList or action=newList.

none

INVALID_LIST An invalid list name was specified when saving results to a list with action=addList.

list: The list name requested.


none


5.3.4 Managing List Results

This section covers the methods for fetching the results from a saved list and manipulating those results (creating, adding).

5.3.4.1 Fetching List Results

Fetching information from a saved list shares some of the same parameters as for performing a standard search with the Search API (section 3). Fetching list results is done with the following URL:

GET https://catalog.archives.gov/iapi/v1/lists/viewentries/<list-name>?username=<user-name>&parameters . . .

161


Parameters

Parameter Description ExamplesControlling the Output

resultFields=field1, field2,field3, . . .

List of fields to return for results from the list.If a field is given that cannot be put in the return (such as a field that does not exist), an error is returned.If no fields are specified, the following fields are returned automatically: naId, opaId, url, localId, hmsEntryNumbers, containerId, iconType, title, titleDate, parentLevel, parentTitle, creators, thumbnailFile.If “*” is specified, then all standard fields are returned except “content” and the XML fields specified in section 3.4.7. Those fields would need to be specified individually, and limits may be placed on their use [future, TBD].

resultFields=title, url

resultFields=*, description

offset=# The zero-based offset into the list of the first result to be returned. This parameter is used in combination with ‘rows’ to paginate the results. If missing, the default offset is 0.If an offset less than 0 or greater than the maximum [Maximum TBD – based on type of user] is specified, an error is returned.

offset=100

rows=# Indicates the number of results to fetch from the list. The default is 25.The maximum value for rows is 200. [TBD]

rows=50

Response

The following is a general outline of what is returned when fetching results from a list:

<results total="--total-matching-results --" offset="--first-result-num--" rows="-- num-results--">

<result num="--result-number--"> <FIELD-NAME1>FIELD-VALUE1</FIELD-NAME1> <FIELD-NAME2>FIELD-VALUE2</FIELD-NAME2> <FIELD-NAME3>FIELD-VALUE3</FIELD-NAME3> . . MORE FIELDS GO HERE . </result> . . MORE RESULTS GO HERE .</results>

In the above outline, the various values are:

162


@total – The total number of results across all of Catalog which match the query. This number may be thousands or millions large.

@offset – The offset number of the first result returned.

rows – The actual number of results returned.

result – Holds the selected metadata returned for each result.

result/@num – The result number (where 0 is the first result) within the list of all possible results (not just results returned by this query).

XML ExampleGET https://catalog.archives.gov/iapi/v1/lists/viewentries/Truman%20Misc?offset=0

<results total="6695" offset="0”> <rows>25</rows> <result num="0"> <naId>7226539</naId> <opaId>desc-7226539</opaId> <title>Letter from George McGovern to Harry S. Truman</title> <parentTitle>Subject Files, 1953 - 1972</parentTitle> <titleDate>1972-08-19</titleDate> <url>http://research.archives.gov/description/7226539</url> <localId>hst-ppp_93-1_18</localId> <iconType>nara/item</iconType> <creators><val>Truman, Harry S., 1884-1972</val></creators> <thumbnailFile>hst-ppp_93-1_18-01.jpg</thumbnailFile> </result> <result num="1"> <naId>7283002</naId> <opaId>desc-7283002</opaId> <title>Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy</title> <parentTitle>Articles from "Studies in Intelligence", 1955 - 1992</parentTitle> <parentLevel>series</parentLevel> <url>http://research.archives.gov/description/7283002</url> <hmsEntryNumbers><val>A1 27</val></hmsEntryNumbers> <iconType>nara/fileUnit</iconType> <creators><val>Central Intelligence Agency. (12/04/1981 - )</val></creators> <thumbnailFile>263-a1-27-box-7-69-1-0001.jpg</thumbnailFile> </result> . . MORE RESULTS GO HERE .</results>

JSON ExampleGET https://catalog.archives.gov/iapi/v1 lists/viewentries/Truman%20Misc?offset=0&format=json

"results":{ "@total":"6695", "@offset":"0", "rows":"25", "result":[ { "@num":"0", "naId":"7226539", "opaId":"desc-7226539", "title":"Letter from George McGovern to Harry S. Truman", "parentTitle":"Subject Files, 1953 - 1972", "titleDate":"1972-08-19", "url":"http://research.archives.gov/description/7226539",

163


"localId":"hst-ppp_93-1_18", "iconType":"nara/item", "creators":["Truman, Harry S., 1884-1972"], "thumbnailFile":"hst-ppp_93-1_18-01.jpg" }, { "@num":"1", "naId":"7283002", "opaId":"desc-7283002", "title":"Spring 1976: 7-69-1: Truman on CIA (Examining President Truman's Role in the

Establishment of the Agency), by Thomas F. Troy", "parentTitle":"Articles from \"Studies in Intelligence\", 1955 - 1992", "parentLevel":"series", "url":"http://research.archives.gov/description/7283002", "hmsEntryNumbers":["A1 27"], "iconType":"nara/fileUnit", "creators":["Central Intelligence Agency. (12/04/1981 - )"], "thumbnailFile":"263-a1-27-box-7-69-1-0001.jpg" } . . MORE RESULTS GO HERE . ]}

Error Codes

The following error codes are currently defined when fetching the contents of a list:


EMBEDDED_HTML The query or filter parameters contain embedded HTML which implies a potential cross-site scripting attack.


INVALID_PARAM_VALUE The value specified to a parameter was invalid. For example, an incorrect source value, incorrect group type, etc.

param: The parameter which contained the invalid value.

INVALID_FIELD A request was made to search over an invalid field.


OFFSET_LIMIT_EXCEEDED The maximum search result request has exceeded the limit. See the “offset” parameter for more details.

request: The result number requested.max: The maximum result number allowed.

ROWS_LIMIT_EXCEEDED The maximum number of rows to be returned with the search results was reached.

request: The number of rows requested.max: The maximum number of rows allowed.


none


164


5.3.4.2 Exporting List Results

Results in a list can be exported in a variety of different formats (PDF, HTML, etc) with different types of metadata. See section 3.6 for all export parameters.

Use the list URL to export list results:

GET https://catalog.archives.gov/iapi/v1?action=export&listName=<list-name>&username=<user-name>&export parameters ...

For example the following call will export brief results for the first 1000 records in the list as PDF with embedded thumbnails:

GET

https://catalog.archives.gov/iapi/v1?action=export&listName=TrumanRecords&username=mstewart&export.what=brief&export.options=thumbnails&export.format=pdf&offset=0&rows=1000

5.3.4.3 Modifying List Results

The contents of a list can be modified by individually adding or deleting entries from the list using the following URLs:

Deleting all list entries:


Deleting a list entry:

DELETE https://catalog.archives.gov/iapi/v1/lists/delete/<list-name>?what=...

Adding a list entry:

POST https://catalog.archives.gov/iapi/v1/lists/add/<list-name>?parameters...

165


Parameters

Parameter Description Exampleswhat=opaId|

description|person|organization|object|webpage

Specifies what type of item to add to the search results. opaId – Add results based on OPA-ID

values description – an archival description person – a person authority record organization – an organization authority

record object – a digital object webPage – a web page from archives.gov

or a presidential library

what=description

what=object

For use when what=opaId

opaIds=ID-value Specify the OPA-ID to add or delete from the list. See section 1.4.2 for OPA-ID formats.Can be a comma separated list of OPA-ID values.

opaIds=desc-7228234

opaIds=desc-7228234,person-2223

For use when what=description or object

naid=ID-value Specify the NAID value of the new entry to add to or delete from the list.

naid=7228234

For use when what=object

objectId=Object ID

Specify the object ID of the object to add or delete the digital object from the list.Note: Also requires a naid parameter.

objectId=93

For use when what=person or organization

authId=ID-value Either the person ID or the organization ID of the authority record to add or delete from the list.

authId=8889

For use when what=webpage

url=web-page-url The URL of the web page to add or delete from the list.

url=http://archives.gov/veterans/

Examples

Delete the entry with OPA-ID “desc-7228924” from mstewart’s “Truman Presidency” list:

DELETE https://catalog.archives.gov/iapi/v1/lists/delete/Truman%20Presidency?what=desc-7228924

Add the description 7228924 to mstewart’s “Truman Presidency” list:

POST https://catalog.archives.gov/iapi/v1/lists/add/Truman%20Presidency?what=desc-7228924

Add three OPA-ID’s to mstewart’s “Truman Presidency” list:

166


POST https://catalog.archives.gov/iapi/v1/lists/add/Truman%20Presidency?what=desc-7228924,desc-247232,desc-5589241

Response

The response will the standard OPA response (see section 1.5.2 for more information).

Error Codes

The following error codes are currently defined as part of the Search API response:


MISSING_PARAM A necessary parameter is missing. Either the “what” parameter, or one of the other parameters necessary to identify the entry to delete or add.

param: The parameter which is missing.

UNKNOWN_WHAT The value specified for the “what” parameter is not one of the allowed values.

what: The value of the what parameter specified.

DELETE_ENTRY_NOT_IN_LIST The item requested to be deleted is not currently in the list.

The following params specify what was requested:what: The type of itemopaId: The OPA-IDnaid: The NAIDobjectId: The object IDauthId: The person or org IDurl: The web page URL

ADD_ENTRY_NOT_IN_OPA The item requested to be added to the list is not found in the Catalog search engine index.All items added to the list must be in the search engine index.

The following params specify what was requested:what: The type of itemopaId: The OPA-IDnaid: The NAIDobjectId: The object IDauthId: The person or org IDurl: The web page URL


none


5.4 Bulk ExportsInformation about bulk downloads for a user can be obtained using the Catalog APIs. Information about bulk downloads can be accessed using the following URLs:

167


Fetch a list of all available or in-progress bulk exports for a specific user:


Fetch information on a specific bulk export:

GET https://catalog.archives.gov/iapi/v1/exports/auth/status/<bulkExportId>

Fetch a single bulk export file:

GET https://catalog.archives.gov/iapi/v1/exports/noauth/files/<exportId>

Note:

Bulk exports are only available when logged in.

The logged in user can only fetch the bulk exports for themselves (not for others).

However, the download file is available to anyone.

5.4.1 Fetching the List of Bulk Exports

Use the following URL to fetch a list of bulk exports for the specified user:


Response

In the download output, the following attributes are provided for each bulk export:

accountExports/@total – Total of bulk exports requested by this user.

accountExports/accountExport – Contains information on each bulk export.

accountExports/accountExport/exportId – The ID number for this bulk export. This will be the same ID number reported when the bulk export was first created.

accountExports/accountExport/exportName– A computer generated description about the bulk export from the query used to generate the items.

accountExports/accountExport/status– The current status of the bulk export. It could be “Queued”, “Scheduled”, “Processing”, “Completed”.

accountExports/accountExport/percentageComplete – The percentage complete. An integer from 1 to 100.

accountExports/accountExport/totalProcesedRecords – The total count of everything processed to be included in the download.

o For bulk exports from searches, this will be the number of search results.

accountExports/accountExport/exportFormat – The search results format requested for the bulk-export.

168


accountExports/accountExport/requestTs – The date-time the request was made and the bulk-export was queued.

accountExports/accountExport/expiresTs – The date-time when the export file will be removed from the system.

accountExports/accountExport/downloadUrl – The URL where the export file can be downloaded.

XML ExampleGET https://catalog.archives.gov/iapi/v1/exports/auth

<accountExports total=1> <accountExport> <exportId>1396</exportId> <exportName>tjefferson - 2015-06-30T14:49:40.413Z - 1396</exportName> <status>Completed</status> <exportFormat>PDF</exportFormat> <requestTs>2015-06-30T14:49:40.000Z</requestTs> <expiresTs>2015-08-29T14:49:40.000Z</expiresTs> <downloadUrl>/OpaAPI/iapi/v1/exports/auth/files/1396</downloadUrl> <percentageComplete>100</percentageComplete> <bulkExport>true</bulkExport> <fileSize>5790688</fileSize> <totalProcesedRecords>25000</totalProcesedRecords> </accountExport></accountExports>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/exports/auth?format=json

"accountExports":{ "@total":"1", "accountExport":[ { "exportId":1396, "exportName":"tjefferson - 2015-06-30T14:49:40.413Z - 1396", "status":"Completed", "exportFormat":"PDF", "requestTs":"2015-06-30T14:49:40.000Z", "expiresTs":"2015-08-29T14:49:40.000Z", "downloadUrl":"\/OpaAPI\/iapi\/v1\/exports\/auth\/files\/1396", "percentageComplete":100, "bulkExport":true, "fileSize":5790688, "totalProcesedRecords":25000 } ] }

5.4.2 Downloading a Bulk Export File

Once the download is complete, the download file will be returned in the bulk exports list. The file name will be based on the download ID:

GET https://catalog.archives.gov/iapi/v1/exports/auth/files/<bulk-export-id>

For example:

169


GET https://catalog.archives.gov/iapi/v1/exports/auth/files/43

5.4.3 Deleting a Bulk Export

Users may cancel (or delete) a bulk export from the list. This is done with the following URL:

DELETE https://catalog.archives.gov/iapi/v1/exports/auth?exportId=<export-id>

Example

Deleting a bulk export:

DELETE https://catalog.archives.gov/iapi/v1/exports/auth?exportId=43

Response


5.4.3.1 Error Codes



NO_BULK_EXPORT An invalid or non-existing bulk export identifier was used in the delete request.

id: The identifier used in the request.

NOT_LOGGED_IN This will be returned for users who attempt to create or edit a translation without logging in.

none

NOT_OWNER This will be returned if trying to delete a bulk download or viewing the list of bulk downloads for another user account.


5.5 NotificationsEvents which may be of interest to the user will be included in the user’s notifications. Currently, this includes:

Modifications to any of the user’s transcriptions or translations

Deletion of any of the user’s annotations by a moderator

Replies to the user’s comment

Notifications can be accessed through the following URL:

GET https://catalog.archives.gov/iapi/v1/accounts/notifications?parameters...

Note:

170


Notifications are only available when logged in.

The logged in user can only fetch the notifications for themselves (not for others).

5.5.1 Parameters

The following parameters are supported for fetching or clearing notifications:

Parameter Description Examplesoffset=# The zero-based offset into the list

notifications to be returned. This parameter is used in combination with ‘rows’ to paginate the list. If missing, the default offset is 0.If an offset less than 0 or greater than the total is specified, an error is returned.

offset=100

rows=# Indicates the number of notifications to fetch. The default is 25.The maximum value for rows is 200. [TBD]

rows=50

5.5.2 Response

When fetching notifications, the following data will be returned:

notification – Detailed information on an individual notification, with the following sub-elements:

o @on – Specifies what was transcribed, one of “description”, “object”, “person”, or “organization”.

o title – A display title for the contribution associated with the event. Usually the title of the associated description or authority record.

o description – Detailed information on the description associated with the notification, with the following sub-elements:

@naid – The NAID of the description on which the comment was made.

@objectId – The object ID on which the comment was made.

@pageNum – A display value which helps identify the object to the user. Typically a page number.

event – The event which created the notification, with the following sub-elements:

o @type – The type of event. One of: “edit”, “delete”, or “reply”.

o @on – Specifies the type of annotation associated with the event. Can be one of “tag”, “transcription”, “translation”, or “comment”.

o @when – Date-time which the event occurred.

o @otherUser – The user ID of the other user who initiated the event.

171


o @otherFullName – The full name of the other user.

o @otherIsNaraStaff – “true” if the other user is a NARA staff member.

o comment – Details on the comment associated with the event (if on=”comment”). Contains the following sub-element:

@id – The comment ID. Can be used to permalink to the comment.

XML ExampleGET https://catalog.archives.gov/iapi/v1/accounts/notifications?offset=0

<notifications> <total>1</total> <totalNew>1</totalNew> <notification on="item"> <event type="UPDATE" on="transcriptions" when="2015-06-29T20:55:36.000Z"

otherUser="kmartin" otherFullName="Kristy Martin" isNaraStaff=”false” displayNameFlag=”false”>

</event> <title>Truman Doctrine</title> <item naId="2668751" objectId="14293115" pageNum=1 totalPages=3> </item> </notification></notifications>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/accounts/notifications?offset=0&format=json

"notifications":{ "total":1, "totalNew":1, "notification":[ { "@on":"item", "event":{ "@type":"UPDATE", "@on":"transcriptions", "@when":"2015-06-29T20:55:36.000Z", "@otherUser":"kmartin", "@otherFullName":"Kristy Martin", "@isNaraStaff":"false", "@displayNameFlag":"false" }, "title":"Truman Doctrine", "item":{ "@naId":"2668751", "@objectId":"14293115", "@pageNum":"1", "@totalPages":"3" } } ] }

5.5.3 Clearing Notifications

Notifications can be cleared using the following URL:

DELETE https://catalog.archives.gov/iapi/v1/accounts/notifications

For example:

172


DELETE https://catalog.archives.gov/iapi/v1/accounts/notifications

Notes:

Users must be logged in to change notifications.

Users can only clear their own notifications.

Response


5.5.4 Error Codes



INVALID_OFFSET An incorrect offset was requested. offset: The requested offset.avail: The number of rows available.

INVALID_ROWS A “rows” number less than zero was requested.

rows: The requested rows.

TOO_MANY_ROWS Too many rows were requested. rows: The requested rows.max: The maximum number of rows allowed.

NOT_LOGGED_IN This will be returned for users who attempt to clear or view notifications without logging in.

none

NOT_OWNER This will be returned if trying clear or view someone else’s notifications.


5.6 Account InformationThe Catalog User APIs can be used to change account information such as the user’s full name, e-mail address, active account status, etc.

5.6.1 Viewing Account Information

Use the following URL to view the user’s account information:

GET https://catalog.archives.gov/iapi/v1/accounts/profile/<user-name>

Notes:

You must be logged into to view account information (see section 1.6).

Users can only view their own account information.

173


XML ExampleGET https://catalog.archives.gov/iapi/v1/accounts/profile/tjefferson

<account> <userId>tjefferson</userId> <userType>registeredUser</userType> <fullName>Thomas Jefferson</fullName> <isFullNamePublic>true</isFullNamePublic> <email>[email protected]</email></account>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/accounts/profile/tjefferson?format=json

{ "userId":"tjefferson", "userType":"registeredUser", "fullName":"Thomas Jefferson", "isFullNamePublic":"true", "email":"[email protected]"}

5.6.2 Modifying Account Information

Account information can be viewed and modified using the following URL:

POST https://catalog.archives.gov/iapi/v1/accounts/modify/<user-name>?parameters...

Note:

HTTPS (SSL) protocol is absolutely required whenever modifying account information.

o Standard HTTP requests will be rejected.

Parameters


change|link|deactivate

Indicates what to do with the user’s account information. change – change one or more fields in

the user’s account information.o Current password is required.

link – Link the user to a social media account (Facebook, Twitter, or Google).

o Current password is required.

action=change

action=link

password=password The user’s current password. This must be provided when making any change to the user’s account information.

password=abcd1234

For use when action=change

newPassword=password

Specify the new password. The password must contain a minimum of 8 characters.

newPassword=wxyz9876

174


Parameter Description Examplesemail=email Specify the new e-mail address for the user. [email protected]

fullName=full-name Specify the new full name for the user. fullName=Thomas%20Jefferson

displayFullName=true|false

Set to “true” if the full name for the user should be displayed to the public, “false” otherwise.

displayFullName=true

For use when action=link [LOW PRIORITY – RELEASE 2]

provider=google|twitterIfacebook

The provider can be any of: “google”, “twitter”, or “facebook”.

provider=facebook

token=token-data The token will be the login token returned by the provider’s authentication method.

token=82AB20CA238241DF.13289AG8A9EF21090.FEACBA92. . .

Examples

Change the mstewart’s password:

POST

https://catalog.archives.gov/iapi/v1/accounts/modify/mstewart?password=abcd1234&newPassword=zyxw9876

Change mstewart’s full name & display full name preference:

POST

https://catalog.archives.gov/iapi/v1/accounts/modify/mstewart?fullName=Meredith%20Stewart&displayFullName=true

Change mstewart’s E-mail Address:

POST https://catalog.archives.gov/iapi/v1/accounts/modify/[email protected]

Link the mstewart account to a google account: [LOW PRIORITY – RELEASE 2]

POST

https://catalog.archives.gov/iapi/v1/accounts/modify/mstewart?action=link&provider=google&token=82AB20CA238241DF.13289AG8A9EF21090.FEACBA92. . .

Response


175


Error Codes



NOT_LOGGED_IN Attempt to view or modify account information without logging in.

none

NOT_OWNER Attempt to modify the account information for other users.

request: The user ID who made the request.

BAD_PROVIDER The provider name was not one of “google”, “twitter” or “facebook”.

none

CORRUPTED_TOKEN The token provided does not appear to be in the expected encoding or format.

none

INVALID_TOKEN The token provided for a third-party authentication could not be validated.

none

BAD_PASSWORD The password provided does not match the user’s current password.

none

MISSING_PASSWORD All account modification requests require the user’s current password.

none

176


6 URL mapping to support selected OPA pilot URLs

The OPA URL mapping API is used to support selected OPA pilot URLs.

6.1 Mapping legacy URLs to Catalog URLsUse the following URL to map a legacy URL to a new Catalog URL:

GET https://catalog.archives.gov/iapi/v1/urlmapping/<record-type>/<arc-id>

XML ExampleGET https://catalog.archives.gov/iapi/v1/urlmapping/all/3859926

<naIds total="2"><naId arcId="3859926" naId="10658335" recordType="topical-subject"/><naId arcId="3859926" naId="10574162" recordType="person"/>

</naIds>

JSON ExampleGET https://catalog.archives.gov/iapi/v1/urlmapping/all/3859926

"naIds":{ "@total":"2", "naId":[ { "@arcId":"3859926", "@recordType":"topical-subject", "@naId":"10658335" }, { "@arcId":"3859926", "@recordType":"person", "@naId":"10574162" } ] }

Response


Error Codes



NO_URL_MAPPINGS_FOUND

Attempt to map an invalid arcID. none

177


7 Scheduled maintenance processes

A Catalog task scheduler was created in order to execute configured periodic tasks. These tasks are calls to the Catalog API, so API can run some maintenance actions on their side, on a daily basis. Tasks should be specified via the scheduler configuration file where a task can be created with its time to be execute and the endpoint the scheduler needs to call.

Here is an XML example used to configure some periodic tasks on the scheduler:<tasks>

<task name="Clean up unverified accounts"><endpoint>/administrator/accounts/remove-

unverified</endpoint><time>14:00:00</time>

</task><task name="Disable idle accounts">

<endpoint>/administrator/accounts/auto-disable</endpoint><time>14:20:00</time>

</task><task name="Cancel unverified email changes">

<endpoint>/administrator/accounts/cancel-email-modifications</endpoint>

<time>14:40:00</time></task>

</tasks>

7.1 Account maintenance processesThe following three tasks are part of the set of periodic maintenance tasks that can be executed on the Catalog task scheduler:

Deactivate idle users: An idle account user is a user who hasn’t interacted with the Catalog system for a specific amount of time defined in the API configuration properties. A task was created for deactiving all the idle users whose their last interaction with the system has exceeded the allowed threshold set in the system.

Delete unverified accounts: An unverified account user is a user who hasn’t verified his/her registration. Once a user is registered in the system an email with an activation link is sent to the email the user provided while registration, so users can verify their accounts. A task was created for deleting all the accounts who haven’t been verified for a specific period of time set on the API configuration.

Delete email change requests: When a user changes his/her email address in Catalog system, an email with a verification link is sent to the new provided email address. Once the user verifies this new email, the email change is done by the API, otherwise no changes occurred on API side. A task was created for deleting all the email change

178


requests which haven’t been verified for the user for a specific period of time set on the API configuration.

179


8 Appendices

8.1 Location IDsThe following is the list of all location ID values.

Location V Search Value(ref-id)

William J. Clinton Library 1

Dwight D. Eisenhower Library 2

Franklin D. Roosevelt Library 3

George Bush Library 4

Gerald R. Ford Library 5

Gerald R. Ford Museum 6

Herbert Hoover Library 7

Harry S. Truman Library 8

Jimmy Carter Library 9

John F. Kennedy Library 10

Lyndon Baines Johnson Library 11

Richard Nixon Library - College Park 12

Ronald Reagan Library 13

National Archives at Boston 14

National Archives at New York 15

National Archives at Philadelphia 17

National Archives at Atlanta 18

National Archives at Chicago 19

National Archives at Kansas City 20

National Archives at Fort Worth 21

National Archives at Denver 22

National Archives at Riverside 23

National Archives at San Francisco 24

National Archives at Anchorage 25

National Archives at Seattle 26

National Personnel Records Center - Civilian Personnel Records 27

National Personnel Records Center - Military Personnel Records 28

National Archives - Washington, DC – Cartographic 29

National Archives - Washington, DC - Motion Pictures 30

180


Location V Search Value(ref-id)

National Archives - Washington, DC - Still Pictures 31

National Archives - Washington, DC - Archives I Textual Reference 32

National Archives - Washington, DC - Archives II Textual Reference (Civilian) 33

National Archives - Washington, DC – FOIA 34

National Archives - Washington, DC - Archives II Textual Reference (Military) 35

Center for Legislative Archives 36

National Archives - Washington, DC – Electronic 37

Library of Congress, Prints and Photographs Division (an affiliated archives) 38

National Park Service, Yellowstone National Park Archives (an affiliated archives) 39

New Mexico Commission of Public Records, State Records Center and Archives (an affiliated archives)

40

Oklahoma Historical Society (an affiliated archives) 41

Pennsylvania Historical and Museum Commission, State Archives (an affiliated archives)

42

United States Military Academy Archives (an affiliated archives) 43

United States Naval Academy, William W. Jeffries Memorial Archives (an affiliated archives)

44

Presidential Materials Division 48

National Archives at St. Louis 50

Richard Nixon Library 51

George W. Bush Library 53

U.S. Government Printing Office (an affiliated archives) 54

University of North Texas Libraries (an affiliated archives) 57

8.2 Technical Metadata[Under construction – complete technical metadata provided for all different types of files]

8.3 Third-Party Authentication [LOW PRIORITY – RELEASE 2][Under construction – exact methods for acquiring the authentication token for all third-party systems: google, twitter, and facebook]

181


8.4 Languages[Under construction – ISO 639-3 three letter codes need to be added]

List of languages supported by Catalog with the numeric NARA language code and the ISO-639-3 (three-letter) language code.

1 [ABK] Abkhaz 2 [ACE] Achinese 3 [ACH] Acoli 4 [ADA] Adangme

5 Afar 6 Afrihili (Artificial language)

7 Afrikaans 8 Afroasiatic (Other)

9 Akan 10 Akkadian 11 Albanian 12 Aleut

13 Algonquian (Other)

14 Aljamma 15 Altaic (Other) 16 Amharic

17 Apache languages 18 Arabic 19 Aramaic 20 Arapaho

21 Arawak 22 Armenian 23 Artificial (Other) 24 Assamese

25 Athapascan (Other)

26 Australian languages

27 Austronesian (Other)

28 Avaric

29 Avestan 30 Awadhi 31 Aymara 32 Azerbaijani

33 Balinese 34 Baltic (Other) 35 Baluchi 36 Bambara

37 Bamileke languages

38 Banda 39 Bantu (Other) 40 Basa

41 Bashkir 42 Basque 43 Batak 44 Beja

45 Belarusian 46 Bemba 47 Bengali 48 Berber (Other)

49 Bhojpuri 50 Bihari 51 Bikol 52 Bini

53 Bislama 54 Bosnian 55 Braj 56 Breton

57 Bugis 58 Bulgarian 59 Buriat 60 Burmese

61 Caddo 62 Carib 63 Catalan 64 Caucasian (Other)

65 Cebuano 66 Celtic (Other) 67 Central American Indian (Other)

68 Chagatai

69 Chamic languages 70 Chamorro 71 Chechen 72 Cherokee

73 Cheyenne 74 Chibcha 75 Chinese 76 Chinook jargon

77 Chipewyan 78 Choctaw 79 Church Slavic 80 Chuvash

81 Coptic 82 Cornish 83 Corsican 84 Cree

85 Creek 86 Creoles and Pidgins (Other)

87 Creoles and Pidgins, English-based (Other)

88 Creoles and Pidgins, French-based (Other)

89 Creoles and Pidgins, Portuguese-based (Other)

90 Croatian 91 Cushitic (Other) 92 Czech

93 Dakota 94 Danish 95 Dayak 96 Delaware

97 Dinka 98 Divehi 99 Dogri

182


100 Dogrib 101Dravidian (Other) 102 Duala 103 Dutch

104 Dutch, Middle (ca. 1050-1350)

105Dyula 106 Dzongkha 107 Efik

108 Egyptian 109Ekajuk 110 Elamite 111 English, Middle (1100-1500)

112 English, Old (ca. 450-1100)

113Eskimo languages 114 Esperanto 115 Estonian

116 Ethiopic 117Ewe 118 Ewondo 119 Fang

120 Fanti 121Faroese 122 Fijian 123 Filipino

124 Finnish 125Finno-Ugrian (Other)

126 Fon 127 French

128 French, Middle (ca. 1400-1600)

129French, Old (ca. 842-1400)

130 Frisian 131 Friulian

132 Fula 133Galician 134 Ganda 135 Gayo

136 Gbaya 137Gc 138 Georgian 139 German

140 German, Middle High (ca. 1050-1500)

141German, Old High (ca. 750-1050)

142 Germanic (Other) 143 Gilbertese

144 Gondi 145Gorontalo 146 Gothic 147 Grebo

148 Greek, Ancient (to 1453)

149Greek, Modern (1453- )

150 Guarani 151 Gujarati

152 Gwich'in 153Haida 154 Hausa 155 Hawaiian

156 Hebrew 157Herero 158 Hiligaynon 159 Himachali

160 Hindi 161Hiri Motu 162 Hittite 163 Hmong

164 Hungarian 165Hupa 166 Iban 167 Icelandic

168 Igbo 169Ijo 170 Iloko 171 Indian

172 Indic (Other) 173Indo-European (Other)

174 Indonesian 175 Interlingua (International Auxiliary Language Association)

176 Interlingue 177Inuktitut 178 Inupiaq 179 Iranian (Other)

180 Irish 181Irish, Middle (ca. 1100-1550)

182 Irish, Old (to 1100) 183 Iroquoian (Other)

184 Italian 185Japanese 186 Javanese 187 Judeo-Arabic

188 Judeo-Persian 189Kabyle 190 Kachin 191 Kalbtdlisut

192 Kamba 193Kannada 194 Kanuri 195 Kara-Kalpak

196 Karen 197Kashmiri 198 Kawi 199 Kazakh

200 Khasi 201Khmer 202 Khoisan (Other) 203 Khotanese

204 Kikuyu 205Kimbundu 206 Kinyarwanda 207 Kiswahili

208 Komi 209Kongo 210 Konkani 211 Korean

212 Kpelle 213Kru 214 Kuanyama 215 Kumyk

216 Kurdish 217Kurukh 218 Kusaie 219 Kutenai

183


220 Kyrgyz 221Ladino 222 Lahnda 223 Lakota

224 Lamba 225Lao 226 Latin 227 Latvian

228 Letzeburgesch 229Lezgian 230 Lingala 231 Lithuanian

232 Lozi 233Luba-Katanga 234 Luba-Lulua 235 Luiseqo

236 Lunda 237Luo (Kenya and Tanzania)

238 Lushai 239 Macedonian

240 Madurese 241Magahi 242 Maithili 243 Makasar

244 Malagasy 245Malay 246 Malayalam 247 Maltese

248 Manchu 249Mandar 250 Mandingo 251 Manipuri

252 Manobo languages 253Manx 254 Maori 255 Mapuche

256 Marathi 257Mari 258 Marshall 259 Marwari

260 Masai 261Mayan languages 262 Mende 263 Micmac

264 Minangkabau 265Miscellaneous languages

266 Mohawk 267 Mohlam

268 Moldavian 269Mon-Khmer (Other)

270 Mongo-Nkundu 271 Mongolian

272 Moori 273Multiple languages 274 Munda (Other) 275 Nahuatl

276 Nauru 277Navajo 278 Ndebele (South Africa)

279 Ndebele (Zimbabwe)

280 Ndonga 281Nepali 282 Newari 283 Nias

284 Niger-Kordofanian (Other)

285Nilo-Saharan (Other)

286 Niuean 287 North American Indian (Other)

288 Northern Sami 289Northern Sotho 290 Norwegian 291 Nubian languages

292 Nyamwezi 293Nyanja 294 Nyankole 295 Nyoro

296 Nzima 297Occitan (post-1500)

298 Ojibwa 299 Old Norse

300 Old Persian (ca. 600-400 B.C.)

301Oriya 302 Oromo 303 Osage

304 Ossetic 305Otomian languages 306 Pahlavi 307 Palauan

308 Pali 309Pampanga 310 Pangasinan 311 Panjabi

312 Papiamento 313Papuan (Other) 314 Persian 315 Philippine (Other)

316 Phoenician 317Polish 318 Ponape 319 Portuguese

320 Prakrit languages 321Provengal (to 1500) 322 Pushto 323 Quechua

324 Raeto-Romance 325Rajasthani 326 Rapanui 327 Rarotongan

328 Romance (Other) 329Romanian 330 Romany 331 Rundi

332 Russian 333Salishan languages 334 Samaritan Aramaic 335 Sami

336 Samoan 337Sandawe 338 Sango 339 Sanskrit

340 Santali 341Sardinian 342 Sasak 343 Scots

344 Scottish Gaelic 345Selkup 346 Semitic (Other) 347 Serbian

184


348 Serer 349Shan 350 Shona 351 Sidamo

352 Sign languages 353Siksika 354 Sindhi 355 Sinhalese

356 Sino-Tibetan (Other)

357Siouan (Other) 358 Slave 359 Slavic (Other)

360 Slovak 361Slovenian 362 Sogdian 363 Somali

364 Songhai 365Soninke 366 Sorbian languages 367 Sotho

368 South American Indian (Other)

369Spanish 370 Sukuma 371 Sumerian

372 Sundanese 373Susu 374 Swahili 375 Swazi

376 Swedish 377Syriac 378 Tagalog 379 Tahitian

380 Tai (Other) 381Tajik 382 Tamashek 383 Tamil

384 Tatar 385Telugu 386 Temne 387 Terena

388 Tetum 389Thai 390 Tibetan 391 Tigri

392 Tigrinya 393Tiv 394 Tlingit 395 Tok Pisin

396 Tokelauan 397Tonga (Nyasa) 398 Tongan 399 Truk

400 Tsimshian 401Tsonga 402 Tswana 403 Tumbuka

405 Turkish Ottoman 406 Turkmen 407 Tuvaluan 408 Tuvinian

409 Twi 410 Ugaritic 411 Uighur 412 Ukrainian

413 Umbundu 414 Undetermined 415 Urdu 416 Uzbek

417 Vai 418 Venda 419 Vietnamese 420 Volapjk

421 Votic 422 Wakashan languages

423 Walamo 424 Waray

425 Washo 426 Welsh 427 Wolof 428 Xhosa

429 Yakut 430 Yao 431 Yapese 432 Yiddish

433 Yoruba 434 Yupik languages 435 Zande 436 Zapotec

437 Zenaga 438 Zhuang 439 Zulu 440 Zuni

185