Redaction counts in the ARC

[First posted on January 14, 2019, at rgr-cyt.org.]

In my recent posts on the JFK Assassination Records Collection (ARC), I have been looking at some of the fine (even trivial) details of the most recent release of redacted texts in the Collection. This post steps back to look at totals for all seven releases of documents in 2017-2018. Surprisingly, NARA’s totals for released in full documents and documents still redacted show significant discrepancies.

This post starts from NARA 18, a spreadsheet NARA posted with its 26 April releases of ARC documents. NARA 18 is a cumulative list of all ARC documents released from July 2017 to April 2018. Documents are identified on the spreadsheet by a column listing their RIF numbers, a unique fifteen-character number given on the Reader Information Form attached to each document. Each row on the spreadsheet is supposed to be linked to a document posted as a file on NARA’s public server, and since there are 54,637 rows (the first row is headers for each column), one could say there are 54,636 documents released.

For reasons discussed in earlier posts on the ARC, there are many, many documents listed multiple times on NARA 18. Adjusting for these, there were a total of 36576 unique RIF numbers listed on NARA 18.1

The 2017-2018 releases were mandated by the JFK Assassination Records Collection Act of 1992, which required all documents with content withheld to be released in full by October 27, 2017. This deadline was pushed back six months by an executive order from President Trump, so that 26 April 2018 was the deadline for releasing this material. However, to make a long story short, President Trump agreed to continue postpone release of some withheld text in a number of documents, subject to further review by October 26, 2021.

The question for researchers now is which documents still contain redacted material and which have been released in full. Unfortunately, this has turned out to be a very difficult question to answer. NARA should be able to answer it, but the information it has given so far has been surprisingly contradictory.

The ARPP figures

One answer is given on NARA’s JFK Assassination Records Processing Project webpage (ARPP), a general FAQ posted following the 26 April release. The ARPP gives much valuable information, but its document totals have serious problems.

According to the ARPP, “Since July 2017, NARA has released in full 13,371 documents.” Note that this a cumulative total. In the same section, the ARPP also notes that “15,834 documents are still redacted.” The problem with this accounting is that the two figures add up to 29,205, 8731 short of the number number of documents listed on NARA 18.

Unless NARA released a large number of documents that were already released in full, I don’t understand why NARA’s total of released in full and remaining redacted documents should be thousands less than the total number of documents released.

The IG report figures

Another way to see the problem is to go back to an earlier report from the NARA Inspector-General (previously discussed here). The IG report predates the 26 April release, so it is no longer current, but it offers totals that come much closer to the actual releases than the ARPP does. Before we look at these, however, we should note a problem with the IG figures. The IG report gave a total of 34,873 ARC documents released in 2017. The actual figure should be 35,436.2

Despite this error, the IG report gives more consistent figures for released in full documents and redacted documents than the ARPP webpage does. As discussed in an earlier post on the IG report, the IG relies on NF18, a January 2018 list of redacted documents in the ARC, to provide a figure of 18,980 redacted documents remaining. It also gives a total figure of approximately 16,000 documents released in full during 2017. It does not give a source for this figure, but by comparing NF18 against the list of releases in 2017, I was able to get a similar result.

Explaining the IG and ARPP differences

The difference between the IG report total for documents released in full and the ARPP total is striking. Despite the fact that hundreds, more likely thousands of documents were released in full on 26 April 2018, ARPP claims a cumulative total of only 13,371 records released in full. In contrast, the IG report gives a figure of 16,000 for the 2017 releases alone.

One of these figures is clearly wrong, and after running through the 2017 releases, the ARPP figure seems much more likely to have a large error in it somewhere. The total number of ARC documents released in full should now be far in excess of 16,000. perhaps as much 20,000. An update from NARA on this odd problem is needed.

  1. One early release was omitted from NARA 18, so the full count should be 36577. 10 rows have bad file links, but this does not affect the total count of unique documents.
  2. For total number of documents released in 2017, the IG used the figures NARA gave in its press releases for each of the six releases in 2017. This is not correct, first because these releases also had duplicate documents, and second because the 2017-12-15 release used a different way of counting documents than all the other releases.

    In this release, a single file on NARA’s server often contained many ARC documents. Rather than give a figure for ARC documents released, however, NARA simply listed the number of files released, which was 3539. To make things even more confusing, this number seems to also be off by two, or at least I was only able to find 3537 files to download.

    If one uses the same method of counting document releases for all six 2017 releases, counting unique document numbers, NARA released 35,436 ARC documents were released during this period, rather than the IG’s figure of 34,873.

Leave a Reply

Your email address will not be published. Required fields are marked *