article level TEI

2011/09/23 in TEI | Permalink

The newest release of Best Practices for TEI in Libraries is anticipated in October, but the draft is now available and very useful. Oddly, keeping article level vs. issue level metadata from getting mixed up is somewhat mind-bending. A workable draft TEI header has been completed and the article level text has been added for a sample set of two issues. The next step is import into Greenstone for trial run of Greenstone METS profile for article level entry. The current TEI template is not complete and the trial run of Greenstone import will likely be flawed, but, this is simply a result of more work on proof of concept. Documentation of the work on semantic markup and structured metadata has been moved to a Digress.It blog. The URL for that blog will be available once the TEI template is complete.

review: TEI and Greenstone METS

2011/09/17 in metadata, METS & TEI, project review | Permalink

As a result of the editorial work completed by Karan in 2007 on the single file of OCR from scans done by Internet Archive, the single individual issues of (MIR) Moving Image Review were broken into structured text files with matching “article” opening and closing tags. Article metadata was added in an attribute of “type” and repeated typos/misreadings created during the OCR process were corrected (details available but not published yet).
e.g.:

<article type="Preservation">
Two Decades Of
TV Film To Be Preserved:
Maine's largest and oldest broadcast
collection.

WABI-TV, the Bangor Historical Society
and NHF are cooperating to save and
make accessible to the public an estimated
300 hours (roughly 650,000 feet) of uni-
que 16mm film containing news, sports
and commercials. The film was shot by
Maine's first TV broadcaster, WABI-TV in
Bangor, between 1953 and 1974.

The footage had not been seen since it
was put onto reels after airing on nightly
news broadcasts. It has recently been

(continued on pg. 2)
</article>

Karan converted at least one issue into individual article files that were to become .xml files for future testing, but the discrete documents of the 2007 version of MIR Online were MIR issues.

While prefacing each issues with a corpus TEI header was effective for bibliographic control of MIR issues for import into Greenstone, discerning the “type” of each article so that “article type” could become a browse term required a messy workaround. In Greenstone, documents are the building block of each Digital Library and the appropriate granularity for Moving Image Review is more clearly article as document instead of issue as document.

A test six-issue sub-set of MIR issues have been chosen for this iteration of MIR Online:

  • Winter: 1988; 1989; & 2003
  • Summer: 1998; 1999; & 2003

Currently, each issue is being broken down into a directory of articles. Articles on more than one page are being moved into the same file. BBEdit, Perl scripts, and regular expressions are being used to remove end-of-line hyphenation and line returns within paragraphs, a TEI header prepended, and the article title and text wrapped in TEI tags. Once this work on these six issues is completed, these source documents will be imported into Greenstone and the resulting source documents will be converted into the Greenstone METS profile.

metadata structure

2011/09/12 in METS & TEI, project review | Permalink

Almost all details of the 2007 diagram of the metadata structure for Moving Image Review Online are still valid. But, the resources that were to be published in the Digital Library are more distributed than anticipated in 2007. In particular, moving image files stream from numerous sites, such as The Maine Memory Network and Windows on Maine, in addition to portions of the Collections, such as clips now served by NHF.

MIR Metadata Structure[download as .pdf]

This will mean that metadata considerations will not be as broad as the above diagram indicates, but, will focus primarily on the articles of Moving Image Review. First draft of this Digital Library will encode each issue as a METS file and each article of an issue as TEI. XSLT will transform TEI/XML files to HTML5.

PDFs and still images may be added to this Digital Library, if time permits, at which point encoding will be MODS and MIX, respectively. But, because the primary task of this project is establishing Moving Image Review Online, text will be the only focus until all issues are available at movingimagereview.org.

The Greenstone METS profile and the encoding of the draft: January, 2008 corpus TEI header are now under review.