Rescued History

University of Illinois Group Uncovers Black Women’s Experience through Massive Data Analysis

Why It’s Important

We say, “History is written by the victors,” but it’s probably more true to say it’s written by the people who have the opportunity to write. It can be very difficult for us to capture what life was like for people who either didn’t or couldn’t record their experiences—most of humanity throughout history. Those who did write often didn’t think about these people. One example of this is the study of Black women, their lives and their experiences. Many documents don’t mention them; those that do are often historically obscure, hidden away in vast library collections and unintentionally misleadingly titled or cataloged. The computational methods of the digital humanities hold great promise in finding and analyzing these works, rescuing the “lost” experience of Black women and other people whose lives weren’t typically examined by the writers of their day. Among other techniques, digitial analysis can examine the work of authors who weren’t Black women to identify the ways in which these documents provide overlapping information that “fills in” the blind spots of each individual author.

“We’re generally interested in Black women and their life experience. But we also see this as a tool that social scientists and people in the humanities can use to study many topics.”

—Ruby Mendenhall, University of Illinois at Urbana-Champaign

How PSC and XSEDE Helped

A collaboration led by Ruby Mendenhall of the University of Illinois Urbana-Champaign has been using the large-memory XSEDE resources Blacklight and Greenfield at PSC to study documents from the 18th through 20th centuries to glean data about Black women and help understand that information via graphic display. With help from XSEDE Extended Collaborative Support Service and Novel and Innovative Projects staff at PSC, the National Center for Supercomputing Applications and the National Institute for Computational Sciences, they used 20,000 documents known to contain information about Black women in the HathiTrust and JSTOR databases to create a computational model that they’re now using to study the entire 800,000 documents in both databases. One initial finding confirmed the prediction that many of the same documents referenced the post-World-War-I Black Women's Club and New Negro movements. The next step for the project, using Greenfield and upcoming XSEDE resource Bridges at PSC, will be to rescue and recover documents by and about Black women that may not be viewed as part of the historical record about the lived experiences of Black women.

“The beauty of computation and big data lies in how it complements the traditional ‘close reading,’ the two methods complementing each other to give you a full picture of what’s going on.”

—Nicole M. Brown, University of Illinois at Urbana-Champaign

Understanding Choice

Researchers Use Blacklight to Make Sense of Thousands of Decision-Making Theories

Why It's Important: Understanding how human beings make decisions is critical in fields like cybersecurity, public health, elections and governance, and economics. How often do people make rational choices, weighing all the options? How often do they use mental shortcuts, short-circuiting good choices?

Read more: Understanding Choice

Spotting the Signal

U of Illinois Researchers Use Blacklight to Discover Sparse Clues for Stock Performance

Wednesday, Dec. 2, 2015

Why it's important: From the smallest retirement investors to the highest-tech hedge fund manager on Wall Street, we're all trying to anticipate the market. Should I buy stock A? If I do, when should I sell it? Finance researchers have long suspected that the most talented traders use their experience and intuition to pick out market details—performance of key stocks, for example--that help them guess whether a particular stock of interest is likely to go up or down. But how do they identify such "predictor" stocks, and how few predictors do you need? Or could it be that the idea of predictor stocks is a myth?

"Our intuition is that a lot of these signals are appearing and experienced traders know what they mean, but they're too fleeting to get picked up by most traditional statistical methods. Where the supercomputer comes in is that we were able to analyze the last 30 minutes of data among some 7,600 variables relatively quickly."

—Adam Daniel Clark-Joseph, University of Illinois

How PSC Helped: University of Illinois finance professors Adam D. Clark-Joseph, Alex Chinco and Mao Ye teamed up to study whether "sparse signals"—small numbers of predictor stocks—actually exist in the markets. With help from XSEDE Extended Collaborative Support Service staff at PSC, Clark-Joseph and his co-authors leveraged the large memory and numbers of processors of now-retired XSEDE resource Blacklight at PSC to test this idea using nine months of minute-by-minute 2010 New York Stock Exchange returns for thousands of stocks. Out of the roughly 2,200 stocks they analyzed each month, the researchers could statistically identify about 12 predictor stocks at a time that helped forecast the performance of a given "target" stock. But these predictors were only briefly relevant to the target. The sparse signals shifted quickly, with 90 percent of predictor stocks remaining relevant for four minutes or less. Adding the fleeting predictor-stock information improved forecasts by a factor of nearly 1.5 compared with standard forecasting methods that target a stock's historical performance alone. Even more interesting, more or less the same group of predictor stocks was relevant to numerous other stocks at a given moment. Subsequent work using XSEDE resource Gordon at the San Diego Supercompter Center (analyzing data between 2005 and 2013) showed that the sparse structure found in the 2010 data by Blacklight has been a pervasive feature of the market for quite some time. Future work using systems like PSC's upcoming Bridges will explore the deeper structure that could underlie these fleeting relationships between seemingly unrelated stocks.

"We find that this pattern holds for pretty much every stock—there are only 13 to 14 other predictors that matter for forecasting the returns in the next minute, and they matter a lot. You improve on the standard benchmarks by a factor of almost 1.5. For a big enough portfolio, the benefit of trading based on the improved predictions would exceed the trading costs. That's not a hedge-fund level of profit, but could potentially improve long-term investment strategies."

—Adam Daniel Clark-Joseph, University of Illinois

People. Science. Collaboration.

People. Science. Collaboration. is a biannual summary of the research, education and workforce development projects being conducted and supported by PSC programs, staff and computational resources. Produced in the fall and spring, People. Science. Collaboration. highlights the center’s accomplishments over the previous six months in addressing questions of broad social impact, such as:

  • Understanding how life processes in health and disease work by simulating protein movements

  • Modeling disease spread and the distribution of treatments to better respond to epidemics

  • Improving computer networking to make exchange of information more rapid and reliable

  • Managing today’s vast datasets and extracting information valuable to health care, industry and individuals



30th Anniversary Edition

View | Download PDF (coming soon)

Fall2015 0cover

Fall 2015 - View |

Download PDF

Spring2015 cover

Spring 2015 - View |

Download PDF

PSC Fall2014-cover

Fall 2014 - View |

Download PDF

PSC Spring2014c

Spring 2014

Download PDF 


In addition to the "People. Science. Collaboration." biannual editions, you can view the full publication as well as individual articles of PSC's former publications "Projects in Scientific Computing" by clicking on the report image or View button below. You can also search the archives at PSC.EDU/science:

PSC2013 covers web

Download PDF

Download PDF


Download PDF


Download PDF


Download PDF

Download PDF


View Full Archives 

CIREN to the Rescue

Blacklight simulations give new insight to crash injuries, better prevention

Why it’s important: Over 33,000 Americans die in motor vehicle crashes annually, according to the U.S. Centers for Disease Control. Modern restraint systems have decreased deaths but some deaths and injuries remain – and restraints themselves can cause some injuries. “Crash-test dummies” can help engineers design safer cars, but provide only limited information about the forces experienced by the body in an impact. Improved computer models of vehicle crashes could predict how restraints and other safety systems could work better. They could also help researchers to duplicate the effects of thousands of changes that would be far too slow to test in physical crash tests.

Read more: Blacklight simulations power crash injury models