Author: Kurt Luther

Dr. Kurt Luther is an associate professor of computer science and (by courtesy) history at Virginia Tech, where he directs the Crowd Intelligence Lab.

More press for Civil War Photo Sleuth

Courtesy: Matt Gentry/Roanoke Times

Our Civil War Photo Sleuth project continues to attract press and media attention. Here is a roundup of additional articles since the last post:

Best Paper Award at IUI 2019

Dr. Kurt Luther (center) receives the IUI 2019 Best Paper Award from PC co-chairs Oliver Brdiczka (left) and Polo Chau (right). Photo courtesy John Wenskovitch.

Our paper, “Photo Sleuth: Combining Human Expertise and Face Recognition to Identify Historical Portraits,” received the Best Paper Award at IUI 2019 in Los Angeles, CA. This award recognized the best paper among 282 submissions. Congratulations to lead author Vikram Mohanty (CS Ph.D. student), David Thames (CS undergraduate), and Sneha Mehta (CS Ph.D. student). A video of the talk (presented by Dr. Luther is embedded below.

Dr. Kurt Luther presents Photo Sleuth at IUI 2019.

Presented at Civil War Photo Talks

Courtesy: Kerone Wetter

Dr. Luther gave an invited presentation, titled “Civil War Photo Sleuthing: Past, Present, and Future” at Civil War Photo Talks in Arlington, VA, co-sponsored by Military Images Magazine and Civil War Faces. Other invited speakers included Ann Shumard, National Portrait Gallery; Micah Messenheimer, Library of Congress; Bryan Cheeseboro, National Archives; and Rick Brown, Military Images. The abstract for Dr. Luther’s talk was as follows:

People have struggled to identify unknown soldiers and sailors in Civil War photos since even before the war ended. In this talk, I trace the 150-year history of photo sleuthing, showing how the passage of time has magnified some challenges, but also unlocked exciting new possibilities. I show how technologies like social media, face recognition, and digital archives allow us to solve photo mysteries that have eluded families and researchers for a century and a half.

Paper accepted to CHI 2019 HCI + AI workshop

Our paper, “Flud: a hybrid crowd-algorithm approach for visualizing biological networks,” was accepted to the CHI 2019 workshop titled, Where is the Human? Bridging the Gap Between AI and HCI, in Glasgow, Scotland. Congratulations to Crowd Lab co-authors Aditya Bharadwaj (Ph.D. student) and David Gwizdala (undergraduate researcher), as well as Yoonjin Kim and Aditya’s co-advisor, Dr. T.M. Murali.

Presented at University of Washington DUB Seminar

Dr. Luther gave an invited presentation, titled “Solving Photo Mysteries with Expert-Led Crowdsourcing,” at the University of Washington’s DUB (Design, Use, Build) Seminar on February 27. Here is the abstract for the presentation:

Investigators in domains such as journalism, military intelligence, and human rights advocacy frequently analyze photographs of questionable or unknown provenance. These photos can provide invaluable leads and evidence, but even experts must invest significant time in each analysis, with no guarantee of success. Crowdsourcing, with its affordances for scalability and parallelization, has great potential to augment expert performance, but little is known about how crowds might fit into photo analysts’ complex workflows. In this talk, I present my group’s research with two communities: open-source investigators who geolocate and verify social media photos, and antiquarians who identify unknown persons in 19th-century portrait photography. Informed by qualitative studies of current practice, we developed a novel approach, expert-led crowdsourcing, that combines the complementary strengths of experts and crowds to solve photo mysteries. We built two software tools based on this approach, GroundTruth and Photo Sleuth, and evaluated them with real experts. I conclude by discussing some broader takeaways for crowdsourced investigations, sensemaking, and image analysis.

Dr. Luther named ACWM Emerging Scholar

Dr. Luther was selected as one of eight Emerging Scholars by the American Civil War Museum in Richmond, VA. He will give an invited presentation on Civil War Photo Sleuth to audiences at the grand opening of the newly expanded museum on May 4. The goal of the program is to “highlight some of the most interesting work of the next generation of writers, communicators, and thinkers of Civil War era history/public history.”

Guest editing a special issue of Transactions on Social Computing

Cover of Transactions on Social Computing journal

Dr. Luther is the lead guest editor for an upcoming special issue of the journal ACM Transactions on Social Computing. The theme of the special issue, “Negotiating Truth and Trust in Socio-Technical Systems“, emerged from the Designing Socio-Technical Systems of Truth workshop that Dr. Luther led at Virginia Tech in March 2018. The special issue co-editors are Crowd Lab postdoc Jacob Thebault-Spieker, Andrea Kavanaugh (Virginia Tech), and Judd Antin (AirBnb).

Paper accepted for CHI 2019

CHI 2019 logo

Congrats to Crowd Lab Ph.D. student Aditya Bharadwaj for his accepted paper at the upcoming CHI 2019 conference in Glasgow, Scotland, in May. The acceptance rate for this top-tier human-computer interaction conference is 24%. The paper, titled “Critter: Augmenting Creative Work with Dynamic Checklists, Automated Quality Assurance, and Contextual Reviewer Feedback“, was co-authored with colleagues Pao Siangliulue and Adam Marcus at the New York-based startup B12, where Aditya interned last summer. The paper’s abstract is as follows:

Checklists and guidelines have played an increasingly important role in complex tasks ranging from the cockpit to the operating theater. Their role in creative tasks like design is less explored. In a needfinding study with expert web designers, we identified designers’ challenges in adhering to a checklist of design guidelines. We built Critter, which addressed these challenges with three components: Dynamic Checklists that progressively disclose guideline complexity with a self-pruning hierarchical view, AutoQA to automate common quality assurance checks, and guideline-specific feedback provided by a reviewer to highlight mistakes as they appear. In an observational study, we found that the more engaged a designer was with Critter, the fewer mistakes they made in following design guidelines. Designers rated the AutoQA and contextual feedback experience highly, and provided feedback on the tradeoffs of the hierarchical Dynamic Checklists. We additionally found that a majority of designers rated the AutoQA experience as excellent and felt that it increased the quality of their work. Finally, we discuss broader implications for supporting complex creative tasks.

Two papers accepted for IUI 2019

Two members of the Crowd Lab each had a paper accepted for presentation at the upcoming IUI 2019 conference in Los Angeles, CA. The acceptance rate for this conference, which focuses on the intersection of human-computer interaction and artificial intelligence, was 25%.

Crowd Lab Ph.D. student Vikram Mohanty will present “Photo Sleuth: Combining Human Expertise and Face Recognition to Identify Historical Portraits“, co-authored with undergraduate David Thames and Ph.D. student Sneha Mehta. Here is the paper’s abstract:

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861- 65), the first widely-photographed conflict. Many thousands of these portraits survive, but only 10–20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluation of Photo Sleuth one month after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.

Crowd Lab Ph.D. student Tianyi Li will present “What Data Should I Protect? A Recommender and Impact Analysis Design to Assist Decision Making“, co-authored with Informatica colleagues Gregorio Convertino, Ranjeet Kumar Tayi, and Shima Kazerooni. Here is the paper’s abstract:

Major breaches of sensitive company data, as for Facebook’s 50 million user accounts in 2018 or Equifax’s 143 million user accounts in 2017, are showing the limitations of reactive data security technologies. Companies and government organizations are turning to proactive data security technologies that secure sensitive data at source. However, data security analysts still face two fundamental challenges in data protection decisions: 1) the information overload from the growing number of data repositories and protection techniques to consider; 2) the optimization of protection plans given the current goals and available resources in the organization. In this work, we propose an intelligent user interface for security analysts that recommends what data to protect, visualizes simulated protection impact, and helps build protection plans. In a domain with limited access to expert users and practices, we elicited user requirements from security analysts in industry and modeled data risks based on architectural and conceptual attributes. Our preliminary evaluation suggests that the design improves the understanding and trust of the recommended protections and helps convert risk information in protection plans.

Congratulations to Vikram, David, Sneha, Tianyi, and their collaborators!

Nai-Ching Wang successfully defends PhD dissertation

Courtesy: Vikram Mohanty

Nai-Ching Wang, a Ph.D. student advised by Dr. Luther, successfully defended his dissertation today. His dissertation is titled, “Supporting Historical Research and Education with Crowdsourced Analysis of Primary Sources”, and his committee members were Dr. Luther (chair), Ed Fox, Gang Wang, and Paul Quigley, with Matt Lease (UT Austin School of Information) as the external member. Here is the abstract for his dissertation:

Historians, like many types of scholars, are often researchers and educators, and both roles involve significant interaction with primary sources. Primary sources are not only direct evidence for historical arguments but also important materials for teaching historical thinking skills to students in classrooms, and engaging the broader public. However, finding high quality primary sources that are relevant to a historian’s specialized topics of interest remains a significant challenge. Automated approaches to text analysis struggle to provide relevant results for these “long tail” searches with long semantic distances from the source material. Consequently, historians are often frustrated at spending so much time on manually the relevance of the contents of these archives other than writing and analysis. To overcome these challenges, my dissertation explores the use of crowdsourcing to support historians in analysis of primary sources. In four studies, I first proposed a class-sourcing model where historians outsource historical analysis to students as a teaching method and students learn historical thinking and gain authentic research experience while doing these analysis tasks. Incite, a realization of this model, deployed in 15 classrooms with positive feedback. Second, I expanded the class-sourcing model to a broader audience, novice (paid) crowds and developed the Read-agree-predict (RAP) technique to accurately evaluate relevance between primary sources and research topics. Third, I presented a set of design principles for crowdsourcing complex historical documents via the American Soldier project on Zooniverse. Finally, I developed CrowdSCIM to help crowds learn historical thinking and evaluated the tradeoffs between quality, learning and efficiency. The outcomes of the studies provide systems, techniques and design guidelines to 1) support historians in their research and teaching practices, 2) help crowd workers learn historical thinking and 3) suggest implications for the design of future crowdsourcing systems.

Congratulations Dr. Wang!