As Major General Robert Ross and his 4,000 British troops closed in on Washington, with orders to set fire to the city’s public buildings, Dolley Madison stood her ground at the White House. One of the most powerful first ladies in history, she maintained enough composure to gather some of the nation’s treasures before making her escape.

That fateful day, August 24, 1814, Dolley famously arranged for servants to bust the frame of Gilbert Stuart’s portrait of George Washington hanging in the state dining room and cart it off to safety. She also saved some silver, china and, of all things, red velvet draperies from the Oval Drawing Room.

At the National Portrait Gallery, a fiery red velvet dress steals the attention of visitors to “1812: A Nation Emerges,” a new exhibition commemorating the bicentennial of the War of 1812. Could the empire-style gown, which Dolley Madison owned until her death in 1849, have been made from the curtains she salvaged from the White House? Some historians and curators suspect so.

Piecing together the story of the dress requires, first, a consideration of the history of the draperies. In 1809, Congress appropriated $14,000 for architect Benjamin Latrobe to redecorate the White House. For the Oval Drawing Room (now called the Blue Room) Latrobe envisioned grand window treatments made of silk damask. But he wrote to Dolley, on March 22, 1809, with disappointing news: “There is no silk damask to be had in either New York of Philadelphia, and I am therefore forced to give you crimson velvet curtains.”

When Latrobe received the velvet, he found it garish. “The curtains! Oh the terrible velvet curtains! Their effect will ruin me entirely, so brilliant will they be,” he wrote in an April letter to the First Lady. Dolley, on the other hand, known for having bold tastes, liked the fabric.

“She gets her way, of course,” says Sid Hart, the National Portrait Gallery’s senior historian and curator of the exhibition.

A letter Dolley wrote to Latrobe’s wife, Mary, shortly after the burning of the White House, is often cited as evidence that she did, in fact, grab the curtains. “Two hours before the enemy entered the city…I sent out the silver (nearly all) and velvet curtains and General Washington’s picture.” She saw to it that only a few cherished items were saved, so why include the curtains?

“She had a special affection for the drapes,” says Hart. “Maybe they somehow represented in her mind her efforts to make the White House a center of social activity.”

At the outbreak of the War of 1812, the nation was about as polarized as it would be nearly 50 years later, at the start of the Civil War. Democratic-Republicans, like President Madison, supported the war, while Federalists opposed it. “There needed to be a cohesive force in Washington,” says Hart. Vivacious as she was, Dolley served that role.

During her husband’s term as president, Dolley hosted parties every Wednesday night, attended by people of all different views. Quite purposefully, she brought factions together in hopes that agreements could be struck. The gatherings, often held in the Oval Drawing Room, where the velvet curtains hung, were called “squeezes,” Hart explains, because “everybody wanted to squeeze in.”

Late in life, as a widow, Dolley was rather poor. When she died, most of her remaining possessions were sold at public auction. At an auction in 1852, Dolley’s niece Anna Payne purchased the red velvet dress, a portrait of Dolley, a few of her trademark silk turbans and other items, which Payne’s daughter and grandson later inherited. In 1956, a trunk containing the belongings was discovered in the attic of a rural Pennsylvania home, where the grandson’s widow had lived. The Dolley Madison Memorial Association invested in the collection and then donated it to the Greensboro Historical Museum in 1963. (Dolley was born in Greensboro.)

Once in the hands of the museum, researchers started to talk about how Dolley’s red dress seemed to be made of drapery-weight velvet. The dress was featured in a 1977 exhibition, titled “Dolley and the ‘Great Little Madison,’” at the Octagon House in Washington, where the Madisons lived after the burning of the White House. In an accompanying book, the show’s curator Conover Hunt-Jones noted that the gown was made “not of the light velvets ordinarily used for clothing.” The observation was enough to feed the imaginations of historians, and many since have entertained the idea that Dolley may have repurposed the curtains.

“It seems to be in character,” says Susan Webster, curator of costumes and textiles at the Greensboro Historical Museum. “Why let this go to waste, and won’t this be a great piece to talk about when we are having dinner with folks? Maybe it is her practicality as a Quaker. I think she treasured things. She understood their value.”

Documents found with the red dress tie it, unquestionably, to Dolley. It was likely made sometime between 1810 and 1820. Yet, no record, be it a letter of Dolley’s or an order for a dress, has ever been found linking the dress to Latrobe’s draperies. “It is a 20th century folklore,” says Webster.

In the stir of publicity for the National Portrait Gallery’s exhibition, Diane Dunkley, director and chief curator of the Daughters of the American Revolution (DAR) Museum, also in Washington, D.C., read about the dress—most likely on display for the last time given its fragile condition. Her ears perked. The DAR Museum has in its collection a swatch of fabric purportedly from the red velvet draperies.

Plans quickly formulated. The DAR Museum and the Greensboro Historical Museum sent clippings of the alleged curtains and the dress to the National Museum of American History, for costumes conservator Sunae Park Evans to compare them using a new digital microscope.

“You can’t absolutely prove that the history is true just from a comparison,” explains Alden O’Brien, curator of costumes and textiles at the DAR Museum. Only through oral history, after all, does the DAR Museum know that their swatch comes from the curtains. “But if the fabrics match, it strengthens the likelihood that there’s truth to the shared histories,” she says.

In a brightly lit lab in the basement of the American History Museum, accompanied by a few half-built Styrofoam mannequin bodices, I watch as Evans and O’Brien analyze a tiny piece of the DAR’s remnant. The microscope’s magnified view is transposed on a computer screen. Based on the fabric’s weave, they quickly realize that it is satin, not velvet. Somewhat disappointingly, O’Brien concludes that the swatch couldn’t possibly be from the red draperies in the Oval Room Drawing Room, as the DAR thought, since all references to the curtains specify that they are velvet.

Evans then places a small snippet of the dress, taken from an inside seam, under the lens. “Oh, very different weave structure,” O’Brien exclaims. “Totally different.” In fact, the color is too. This piece is more pinkish than the previous swatch. Based on the way the fibers are woven, Evans says with certainty that this one is velvet. Whether it is the velvet from the draperies, though, no one can say.

Hart, of the National Portrait Gallery, likes to believe in the tale. “It seems reasonable to me,” says the historian. Dolley did keep the dress until her dying day. “But there is no way that I can see that this can ever really be proved one way or another,” he says.