In 2008, the State of Oregon opened a waiting list for enrollment in the Oregon Health Plan, Oregon’s public health insurance program for low-income adults. Over 85,000 people put their names on the list – many more than the state could afford to insure at the time. In these circumstances, the state decided that the fairest procedure was a random one: 35,000 individuals were randomly selected from the list to receive applications for the health plan.

Several researchers around the country, including Amy Finkelstein (MIT), Katherine Baicker (Harvard School of Public Health), and Bill Wright (Providence Health & Services in Portland, OR) realized that Oregon’s random selection procedure could serve as the basis for a randomized, controlled study of the effects of health insurance on a variety of outcomes –- health, access to healthcare, financial status, etc. Randomized, controlled experiments are considered the “gold standard” in medicine and the physical sciences, but are generally difficult to arrange in the social sciences. The events in Oregon were an unprecedented chance to apply these rigorous methods to the study of health insurance.

Read more about how DDD helped these researchers process close to 70,000 surveys.

Everyone involved was so thrilled with the success of the project that we threw an international, virtual party to celebrate, with our team in Cambodia, and CCS in Germany and Romania. When the project is completed, eight million pages of historical newspapers will have been converted from dusty hard copies in storage to searchable and accessible articles online.

Our European Sales Team Representative Ed McLean gave some perspective on our historical newspapers projects: “Digitized papers once buried deep in paper archives reveal incredible things – this process opens them up to be discovered not just today but also in many years’, decades’ or centuries’ time. It is a window into how we used to live.”

Learn more about DDD’s newspaper digitization services.

1:  A poorly chosen conversion process increases costs, time frame or prevalence of errors

Optical Character Recognition (OCR) technology can be faster and cheaper than relatively expensive keying (re-typing). However OCR is unsuitable for some projects, with results so inaccurate that they cannot be used for the intended purpose. Without testing your conversion process it is not always possible to predict the quality of the output.  OCR can often convert characters (numbers, letters, etc.) correctly, but often merges fields and cell positions.  We also find academics frequently scripting OCR data into structured data bases. These scripts usually rely on punctuation or line break – exactly the two things that OCR is the weakest at converting.

Solutions

• - Before embarking on conversion consider the intended purpose of your digitized data and decide on the acceptable number of errors per thousand characters.  Consider both character erros and cell position errors.
• - Evaluate the quality (clarity of type, page condition, marks, stains, legibility, etc) and complexity (tables, equations, diagrams, etc) of your source materials and be realistic about the potential for accurate conversion.
• - Run a small trial to test your chosen conversion process produces the results you expect.
• - Bear in mind that different parts of your source materials might require different conversion processes (OCR the text, key the tables? Key the handwritten, OCR the type-set?).
• - Run tests and evaluate whether your chosen method(s) return the accuracy and productivity you require and whether you can complete the project within budget.
• - Consider asking colleagues or experts who have used similar source materials.

2: The scale and complexity of the project is inaccurately estimated because of the sheer quantity of the source materials, resulting in poor quality output, delays and increased costs

Consider the following scenario; a project of three hundred thousand pages is deemed after a brief inspection to be suitable for OCR. Budget is secured and timescales are set. However, in the depths of the project it is discovered that 10% of the pages have a typeset font that cannot be read by the OCR. These pages must be identified and then reprocessed, using double-keying. The project is 25% over-budget and several weeks late.

Solution

Add a scoping stage to your project during which you take representative samples and then thoroughly audit your primary materials, understanding the different conditions present throughout. Appropriate decisions can then be taken on the correct conversion process and how much time and budget will be required.

3: Poorly scanned, photographed or photocopied images make data entry expensive and inaccurate

Solution

Professional scanning can improve accuracy and even enable OCR to work properly, cutting costs considerably.  Many academic libraries provide scanning services. Also, professional scanning vendors can scan from books (even fragile or rare), loose paper and microfilm. DDD works with professional scanning firms around the world and can recommend a partner.

4: Poor decisions on resourcing projects cause delays, low data quality and high overall cost

Some researchers report that with course work serving as a distraction, the repetitive, undemanding nature of data entry work and the lack of obligation to produce perfect results, the use of student workers can result in high costs, delays and low quality data even though their cost per hour to your budget may be small. Furthermore the scale and flexibility of student “market” is limited and the weight of managing the project and coordination the students rests on the shoulders of the academic commissioning the work.

Solution

Consider carefully whether students can fulfill your needs and investigate alternative resources, should you require them.

5: Within the source material the same data are entered in different ways, making collation for analysis difficult, especially where more than one person is entering data. For example;

• - Some numbers have a decimal (10.5) and some have fractions (10 ½).  Some have apostrophes (10,000) and some do not (10000).
• - Some dates are DD-MM-YYYY (e.g. 04-07-2009) and others are in long form (4^th July 2009).
• - Cities, counties and locations abide by different naming rules (Hertfordshire, Herts).

Solution

Before embarking on conversion, review representative samples from the source materials to identify which differences may occur. Establish written rules for the correct way to enter these differences and train those inputting the data in how to follow them. Make them aware of the procedure to follow when a new difference (without a rule) is discovered. Ideally a review of the final data should be made to correct any instances that “slip through”.

Related to this issue and #6, below, is character “encoding.” Different software applications store text characters differently. Spreadsheet software often applies data masks for dates and fractions that can change the underlying text data. Be sure to check the data output in the final application you will be using for data analysis.

6: An inappropriate choice for the output format makes data hard to interrogate and/or expensive to reconvert

Excel or Word can seem the obvious choice, but other formats might be more effective for current and/or long-term use (XML, Access, SPSS, etc.)

Solution

Consider the current and future potential uses of your data. Is your chosen output format flexible enough to be easily converted in to other potentially required formats? In most cases it is advisable to avoid proprietary formats (e.g. Word 2007).

7: Inconsistent file-naming means data is lost or hard to find

Solution

Output file names should carry descriptive information, for example, “the_times_22_07_1976_p35.xls” so that they can be quickly and easily related back to the source materials. Any digital input files (such as JPEG or TIFF scans) should ideally have the same names.

8: Absence of rules for handling exceptions results in data inconsistency

Exceptions can impact final data analysis, and should be recorded and coded for reference.  E.g. a table which spans multiple pages, a chart that has different titles inside a data collection, or special characters from a foreign language (i.e. a Greek “delta” character in a financial table).

Solution

Establish your processing rules (“tables /text /titles /authors /captions /pictures /headings, etc are handled in the following way”). Consider also your process for exceptions, however simple: “Any pages or page items which fall outside the processing rules should be stored in folder X and the corresponding entry in the output spreadsheet should be marked with an X.”

Michael has served as a consultant on philanthropy and international development to private foundations and individual donors and also helped start a non-profit to bring internet access and training to more than 5,000 schools around the world. Michael came full circle when he recently left his job at the Bill & Melinda Gates Foundation and returned to DDD to help more people learn new skills, find good jobs, increase their income and improve the quality of their lives.

Michael’s new role reflects DDD’s commitment to exploring ways to continue to strengthen our model and create an impact beyond Cambodia and Laos.  He will be coordinating DDD’s efforts to consider expansion to other parts of the world. We are very happy to welcome Michael to this new role in the DDD family and look forward to going far together.

Contact us for more information
Date: February 22-24, 2010
Location: Marriott Marquis Times Square, New York City

Digital Divide Data is excited to share information at this conference on how we serve the XML conversion, eBook creation and digital preservation needs of publishers, libraries, and content hosts world-wide. We deliver these services on the foundation of a socially responsible model that creates opportunity for the world’s poorest citizens to earn competitive wages, complete their education, and achieve upward mobility through working in our business.

http://www.dlib.org/dlib/july09/munoz/07munoz.html

By: Simon Tanner (King’s Digital Consultancy Services), Trevor Muñoz (Centre for Computing in the Humanities), Pich Hemy Ros (Digital Divide Data)

This article will discuss how to measure the accuracy of Optical Character Recognition (OCR) output in a way that is relevant to the needs of the end users of digital resources. A case study measuring the OCR accuracy of the British Library’s 19th Century Newspapers Database provides a clear example of the benefits to be gained from measuring not just character accuracy but also word and significant word accuracy. As OCR primarily facilitates searching, indexing and other means of structuring the user experience of online newspaper archives, measuring the word and significant word accuracy of the OCR output is very revealing of a resource’s likely performance for these functions. Having such data is therefore extremely helpful for planning and quality assurance assessment. After briefly discussing the role of OCR in the text capture process and how OCR works, we give a detailed description of the methodology, statistical data gathering techniques and analysis used in this study. Our conclusions point the way forward with suggested actions to assist other mass digitization projects in applying these techniques.

We are proud to report that we increased earned revenues from clients to $2.2 million for the year ending June 30, 2009. This was up 50% from the previous year of $1.5 million.

For the fourth straight year DDD covered its business costs through earned revenue. We then used generous support from our donors to support our social mission related expenses, particularly the recruiting and training of disadvantaged young people and educational benefits.

Congratulations to the whole DDD family for continuing to improve our financial sustainability while improving our social impact.

DDD provides GoMobo with a customized solution for creating and maintaining online data of menus and restaurants. Using GoMobo’s quality assurance process, DDD digitizes menus and configures menus for online access in GoMobo’s unique system. The service ensures that GoMobo’s data is continually updated and accurate.

GoMobo and DDD’s relationship was founded out of a commitment to providing exemplary service to clients and making a positive impact on the world. Both led by young, innovative entrepreneurs, the partnership continues as an exemplary model of social entrepreneurship.

Read the full article: “City Critic – Much Ado About Lunch: Fast Food A Little Faster” by Ariel Kaminer.

Learn more about GoMobo’s partnership with DDD.