Paper accepted for HCOMP 2019

HCOMP 2019 logo

The Crowd Lab had a paper, titled, “Second Opinion: Supporting last-mile person identification with crowdsourcing and face recognition,” accepted for the upcoming AAAI Human Computation and Crowdsourcing (HCOMP 2019) conference at the Skamania Lodge in Stevenson, WA, USA, October 28-30, 2019. The conference had a 25% acceptance rate.

Ph.D. student and lead author Vikram Mohanty will present the paper, co-authored with Dr. Luther and Crowd Lab undergraduate researchers Kareem Abdol-Hamid and Courtney Ebersohl. Here’s the paper’s abstract:

As AI-based face recognition technologies are increasingly adopted for high-stakes applications like locating suspected criminals, public concerns about the accuracy of these technologies have grown as well. These technologies often present a human expert with a shortlist of high-confidence candidate faces from which the expert must select correct match(es) while avoiding false positives, which we term the “last-mile problem.” We propose Second Opinion, a web-based software tool that employs a novel crowdsourcing workflow inspired by cognitive psychology, seed-gather-analyze, to assist experts in solving the last-mile problem. We evaluated Second Opinion with a mixed-methods lab study involving 10 experts and 300 crowd workers who collaborate to identify people in historical photos. We found that crowds can eliminate 75% of false positives from the highest-confidence candidates suggested by face recognition, and that experts were enthusiastic about using Second Opinion in their work. We also discuss broader implications for crowd–AI interaction and crowdsourced person identification.

Presented at ACWM grand opening

Credit Penelope M. Carrington / The American Civil War Museum

Dr. Luther gave an invited presentation on Civil War Photo Sleuth at the grand opening celebrations of the American Civil War Museum in Richmond, VA, on May 4. He was one of eight Emerging Scholars invited to speak. The museum described the event and program as follows:

On Saturday, May 4, 2019, the American Civil War Museum celebrates the grand opening of its new museum building and exhibits. As part of that program, the ACWM will highlight some of the most interesting work of the next generation of writers, communicators, and thinkers of Civil War era history/public history with a series lightning talks by emerging professionals in their field. Over the winter, ACWM staff reviewed many applications and selected eight individuals in the early phases of their careers who represented a blend of compelling scholarship and communication skills.

You can read more about the grand opening of the museum here.

More press for Civil War Photo Sleuth

Courtesy: Matt Gentry/Roanoke Times

Our Civil War Photo Sleuth project continues to attract press and media attention. Here is a roundup of additional articles since the last post:

Best Paper Award at IUI 2019

Dr. Kurt Luther (center) receives the IUI 2019 Best Paper Award from PC co-chairs Oliver Brdiczka (left) and Polo Chau (right). Photo courtesy John Wenskovitch.

Our paper, “Photo Sleuth: Combining Human Expertise and Face Recognition to Identify Historical Portraits,” received the Best Paper Award at IUI 2019 in Los Angeles, CA. This award recognized the best paper among 282 submissions. Congratulations to lead author Vikram Mohanty (CS Ph.D. student), David Thames (CS undergraduate), and Sneha Mehta (CS Ph.D. student). A video of the talk (presented by Dr. Luther is embedded below.

Dr. Kurt Luther presents Photo Sleuth at IUI 2019.

Presented at Civil War Photo Talks

Courtesy: Kerone Wetter

Dr. Luther gave an invited presentation, titled “Civil War Photo Sleuthing: Past, Present, and Future” at Civil War Photo Talks in Arlington, VA, co-sponsored by Military Images Magazine and Civil War Faces. Other invited speakers included Ann Shumard, National Portrait Gallery; Micah Messenheimer, Library of Congress; Bryan Cheeseboro, National Archives; and Rick Brown, Military Images. The abstract for Dr. Luther’s talk was as follows:

People have struggled to identify unknown soldiers and sailors in Civil War photos since even before the war ended. In this talk, I trace the 150-year history of photo sleuthing, showing how the passage of time has magnified some challenges, but also unlocked exciting new possibilities. I show how technologies like social media, face recognition, and digital archives allow us to solve photo mysteries that have eluded families and researchers for a century and a half.

Dr. Luther named ACWM Emerging Scholar

Dr. Luther was selected as one of eight Emerging Scholars by the American Civil War Museum in Richmond, VA. He will give an invited presentation on Civil War Photo Sleuth to audiences at the grand opening of the newly expanded museum on May 4. The goal of the program is to “highlight some of the most interesting work of the next generation of writers, communicators, and thinkers of Civil War era history/public history.”

Two papers accepted for IUI 2019

Two members of the Crowd Lab each had a paper accepted for presentation at the upcoming IUI 2019 conference in Los Angeles, CA. The acceptance rate for this conference, which focuses on the intersection of human-computer interaction and artificial intelligence, was 25%.

Crowd Lab Ph.D. student Vikram Mohanty will present “Photo Sleuth: Combining Human Expertise and Face Recognition to Identify Historical Portraits“, co-authored with undergraduate David Thames and Ph.D. student Sneha Mehta. Here is the paper’s abstract:

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861- 65), the first widely-photographed conflict. Many thousands of these portraits survive, but only 10–20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluation of Photo Sleuth one month after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.

Crowd Lab Ph.D. student Tianyi Li will present “What Data Should I Protect? A Recommender and Impact Analysis Design to Assist Decision Making“, co-authored with Informatica colleagues Gregorio Convertino, Ranjeet Kumar Tayi, and Shima Kazerooni. Here is the paper’s abstract:

Major breaches of sensitive company data, as for Facebook’s 50 million user accounts in 2018 or Equifax’s 143 million user accounts in 2017, are showing the limitations of reactive data security technologies. Companies and government organizations are turning to proactive data security technologies that secure sensitive data at source. However, data security analysts still face two fundamental challenges in data protection decisions: 1) the information overload from the growing number of data repositories and protection techniques to consider; 2) the optimization of protection plans given the current goals and available resources in the organization. In this work, we propose an intelligent user interface for security analysts that recommends what data to protect, visualizes simulated protection impact, and helps build protection plans. In a domain with limited access to expert users and practices, we elicited user requirements from security analysts in industry and modeled data risks based on architectural and conceptual attributes. Our preliminary evaluation suggests that the design improves the understanding and trust of the recommended protections and helps convert risk information in protection plans.

Congratulations to Vikram, David, Sneha, Tianyi, and their collaborators!

Photo Sleuth press in Slate, Smithsonian, and more

Our Civil War Photo Sleuth project got a burst of publicity in recent weeks, leading to hundreds of new site registrations and contributions. Here is a round-up of some highlights:

New Photo Research Tool in Civil War Times magazine

Thanks to these media outlets for the great publicity!

Presented at Image of War Seminar

Dr. Luther and his frequent collaborator Ron Coddington, editor and publisher of Military Images magazine, gave an invited presentation on Civil War photo sleuthing at the 18th annual Image of War Seminar in Alexandria, VA, hosted by the Center for Civil War Photography. The presentation included a brief history of American Civil War photography and a live demonstration of the Civil War Photo Sleuth website.

Public launch of Civil War Photo Sleuth website

Assistant Surgeon Francis Marion Eveleth, identified by Dr. Luther via Civil War Photo Sleuth website

On August 1, we held our public launch party for the Civil War Photo Sleuth website at the National Archives Building in Washington, DC. Our team spent the day helping new users (in person and online) get signed up and contributing to the site. Dr. Luther and Military Images editor Ron Coddington gave brief remarks, and we were joined by many distinguished guests, including Library of Congress and National Archives staff. The National Archives’ Innovation Hub provided the perfect setting for the event. We were also grateful for VT Computer Science and Civil War Times for event photography and social media coverage (more photos are available here).

A highlight of the event was sharing a new identification — made via the website — of a previously unknown Civil War soldier tintype from the Library of Congress collection. The donor of the photo, Tom Liljenquist, was present to receive the identification.