Museum of Jewish Heritage Is Using A.I. to Preserve Holocaust Survivo…

The Museum of Jewish Heritage in Lower Manhattan, like other institutions that memorialize the Holocaust, has long relied on survivors to provide firsthand accounts of the cruelties imposed by the Nazis and the various paths people took to endure.

But with the ranks of survivors thinning — almost all are in their 80s or 90s — the museum has been working to find the most effective ways to convey to future generations how easily a civilized society can descend into almost incomprehensible barbarism and systematic mass slaughter.

“What I care about is what your grandchildren’s grandchildren will know,” said Jack Kliger, the museum’s president and chief executive, a son of survivors.

In planning for what Kliger calls “a post-survivor world,” the museum could have simply offered taped videos of individuals recounting their painful experiences. But museum officials worried that such an approach risked putting forth a fragmented and misleading sense of what happened; that someone viewing, say, the testimony of a prisoner of a concentration camp might think that all Holocaust survivors spent World War II in concentration camps.

Instead, in an effort to depict the widest range of experiences and context, the museum has decided to use a technology equipped for that task — artificial intelligence.

When the museum’s new A.I.-powered installation is completed in the fall, visitors will be able to pose questions by voice or text and an algorithm will find the most relevant prerecorded video clips among the testimonies of 10 survivors selected by the museum from its speakers’ bureau.

If, for example, the visitor asks, “What was concentration camp life like?” she might see a clip of Alice Ginsburg, one of several survivors who volunteered to detail their experiences for posterity. As a 13-year-old in Czechoslovakia, Ginsburg, now 93, was deported to Auschwitz-Birkenau with her family and never saw her mother and sister again. After several months, she was transferred to a labor camp to turn out ammunition for the Nazi war machine and then went on to endure a 100-mile “death march,” watching sick and malnourished fellow inmates drop dead along the way.

But other survivors were never sent to concentration camps, though they lost countless relatives and suffered other atrocities. Toby Levy, 90, was hidden in a barn for two years by a Polish woman who had been a customer in her father’s fabric store. Mark Schonwetter, 90, remembers being penned as a child with his mother and sister inside a Jewish ghetto encircled by barbed wire. They lived on two daily helpings of thin soup and stale bread, without functioning toilets or water to wash. Escaping, they spent three years hiding in dense forests during the warm months and in farm lofts and haystacks when it was cold.

Another question, such as “Were you hungry?,” might prompt responses from several of the survivors in succession recounting their variety of struggles.

One of the experts shepherding the project, Mike Jones, the director of automation technologies at University of Southern California Libraries, said the algorithm had been tweaked to match certain phrases in questions — like “close call” and “in danger” — to the most appropriate answers. Given the power of A.I., the matches are expected to improve as the machine “learns” from previous successes and errors.

Details of the installation are not fully set, but it is likely to include a large-screen monitor that displays the video responses and a touch screen for visitors to ask questions. The hope, museum officials said, is that the interactions will resemble the intimacy and engagement of actual conversations.

The urgency that drives this effort is a reflection of the dwindling number of survivors, now estimated to be roughly 245,000 worldwide, with 30,000 of them in the New York metropolitan area. There is also a sense among experts that, amid rising antisemitism, the witness accounts provide the most compelling evidence of the Nazi effort to exterminate the Jews of Europe, which led to 6 million deaths, roughly one third of the world’s Jewish population at the time.

“I want my story to go on long after I’m gone,” Ginsburg said in an interview last week. “I do speaking engagements but after a while I won’t be able to. I want to educate the public for years to come because after a while it will be forgotten just like the Inquisition has been forgotten — nobody talks about it anymore. And the persecution of Jews is an ongoing thing.”

Supported by $200,000 in grants, largely from the Myron and Alayne Meilman Family Foundation, the project is being shaped by technology experts from the USC Shoah Foundation in addition to the USC Libraries. The Shoah Foundation was started 30 years ago by Steven Spielberg and has collected the testimonies of over 57,000 survivors and other witnesses in 75 countries, according to its website. USC officials have also provided expertise for four museums — in Dallas, Buenos Aires, St. Petersburg, Fla., and Terre Haute, Ind. — that also use A.I. to answer questions about the Holocaust in a program called Dimensions in Testimony.

Jones said the existing installations allow visitors to asks questions of only one survivor at a time; the Jewish Heritage museum’s exhibit is intended to allow users to engage collectively with several survivors in answering a single question.

Ten were interviewed in July at the museum's Battery Park City building by a team from the museum and the Shoah Foundation and USC Libraries. Over a week, each survivor was filmed while sitting in a chair against a stark white background. Markings on the floor directed where they should place their feet.

These survivors are accustomed to unspooling their stories at length because the museum often dispatches them to schools in New York and other states to provide firsthand accounts of their experiences. But in these interviews, the survivors were asked to tightly focus and simplify their answers, keeping their responses to 30 seconds or less, a limit that momentarily distressed some of the survivors.

When Dan Rosenbaum, a project volunteer, asked Levy to recall the hide-outs she her family had used during the war, she at first launched into a lengthy tale. She described hiding with eight relatives in a barn and all the help provided by a Polish farm woman and the woman’s 16-year-old son to make sure they were fed and warm — all at great risk to their own lives. As poignant as the story was, it was too long and sinuous to work as an answer to a visitor’s question at an automated installation.

“Can we do it very brief?” Rosenbaum asked. Levy nodded.

In her second take she tightened her response: “The first hiding place was a cellar. The second place was a barn.” Levy, a grandmother of five who for years operated a New York City jewelry store on 47th Street with her husband, was then asked to talk about close calls she might have had where she was almost discovered. Later she was asked what revenge she has sought for her German oppressors.

“My revenge is I’m here,” she said.

Given the number of Holocaust deniers spreading disinformation on the internet, the creators are careful to avoid any embellishments or fabrications. The installation will only use clips of actual survivors, not animated bots generated by A.I. like the one of Vincent van Gogh created for an exhibit at the Musée D’Orsay in Paris.

“In no way are these testimonies artificial,” Schonwetter, one of the survivors, said. “These are real people and real testimonies.”

And though their answers may be short and to the point, each of the survivors is given the opportunity to present the full gamut of the horrors they faced. For Levy, there are 114 distinct clips of her answers to potential questions.

“Each one of us has a different survivor’s story,” she said. “Each one of us is a miracle.”