International. IBM secretly used footage from NYPD CCTV cameras to develop surveillance technology that could search for people based on bodily characteristics, such as age and skin tone. This is according to a report by The Intercept that cites confidential company documents, as well as interviews with former IBM researchers and NYPD officials.
Although IBM announces the ability of its video analytics software to search for people who use traits such as age and ethnicity, the company has not previously disclosed its use of Surveillance Footage of New York to develop this technology. The extent to which IBM worked with the NYPD, and the secrecy with which it did so, is a troubling example of technology companies partnering with governments to build surveillance systems without public oversight.
The ability of new technology, such as artificial intelligence to power surveillance, has worried privacy advocates for years. Not only do we know that such systems may have a racial bias (meaning they are prone to making more mistakes when identifying individuals with specific skin colors), but there is no clear legislation to regulate their use.
According to The Intercept's story, the NYPD acquired IBM's video analytics software through subsidiary Vexcel in 2007. The technology began testing at New York's counterterrorism command center, which is part of the Lower Manhattan Security Initiative, in March 2010. NYPD spokesman Peter Donald told The Intercept that the software was running on "less than fifty" of the center's 512 cameras at the time.
By 2012, IBM was reportedly "testing video analytics software on the bodies and faces of New Yorkers, capturing and archiving their physical data as they walked in public." The company then used these images to improve its search functionality, allowing operators to build features to find people based on age, gender, hair color, facial hair, and skin tone.
The NYPD says he never wore these features, hoping to avoid the "suggestion of the emergence of any kind of technological racial profiling." A spokesperson told The Intercept, "While tools that featured racial or skin tone search capabilities were offered to the NYPD, they were explicitly rejected by the NYPD."
However, former IBM researcher Rick Kjeldsen, who said he worked on the NYPD project between 2009 and 2013, claims this is misleading. "We wouldn't have explored [search functionality] if the NYPD had told us 'we don't want to do that,'" Kjeldsen told The Intercept. "No company is going to spend money where there's no customer interest."
Kjeldsen says several IBM researchers involved in the project were concerned about the potential misuse of the technology. He adds that one of his main objections was the lack of public disclosure about how these potentially invasive tools could be implemented. "That's where we need the conversation," Kjeldsen said. "That's exactly why knowledge of this should be more widely available, so we can figure it out."
The NYPD says it stopped using IBM's technology in 2016. When contacted by The Verge, IBM did not answer specific questions, but gave a statement saying it "remains absolutely committed to the responsible advancement of new technologies."
