Wading into the controversy surrounding an Islamic center planned for a site near New York City’s Ground Zero memorial this past August, President Obama declared: “This is America. And our commitment to religious freedom must be unshakeable. The principle that people of all faiths are welcome in this country and that they will not be treated differently by their government is essential to who we are.” In doing so, he paid homage to a vision that politicians and preachers have extolled for more than two centuries—that America historically has been a place of religious tolerance. It was a sentiment George Washington voiced shortly after taking the oath of office just a few blocks from Ground Zero.

But is it so?

In the storybook version most of us learned in school, the Pilgrims came to America aboard the Mayflower in search of religious freedom in 1620. The Puritans soon followed, for the same reason. Ever since these religious dissidents arrived at their shining “city upon a hill,” as their governor John Winthrop called it, millions from around the world have done the same, coming to an America where they found a welcome melting pot in which everyone was free to practice his or her own faith.

The problem is that this tidy narrative is an American myth. The real story of religion in America’s past is an often awkward, frequently embarrassing and occasionally bloody tale that most civics books and high-school texts either paper over or shunt to the side. And much of the recent conversation about America’s ideal of religious freedom has paid lip service to this comforting tableau.

From the earliest arrival of Europeans on America’s shores, religion has often been a cudgel, used to discriminate, suppress and even kill the foreign, the “heretic” and the “unbeliever”—including the “heathen” natives already here. Moreover, while it is true that the vast majority of early-generation Americans were Christian, the pitched battles between various Protestant sects and, more explosively, between Protestants and Catholics, present an unavoidable contradiction to the widely held notion that America is a “Christian nation.”

First, a little overlooked history: the initial encounter between Europeans in the future United States came with the establishment of a Huguenot (French Protestant) colony in 1564 at Fort Caroline (near modern Jacksonville, Florida). More than half a century before the Mayflower set sail, French pilgrims had come to America in search of religious freedom.

The Spanish had other ideas. In 1565, they established a forward operating base at St. Augustine and proceeded to wipe out the Fort Caroline colony. The Spanish commander, Pedro Menéndez de Avilés, wrote to the Spanish King Philip II that he had “hanged all those we had found in [Fort Caroline] because...they were scattering the odious Lutheran doctrine in these Provinces.” When hundreds of survivors of a shipwrecked French fleet washed up on the beaches of Florida, they were put to the sword, beside a river the Spanish called Matanzas (“slaughters”). In other words, the first encounter between European Christians in America ended in a blood bath.

The much-ballyhooed arrival of the Pilgrims and Puritans in New England in the early 1600s was indeed a response to persecution that these religious dissenters had experienced in England. But the Puritan fathers of the Massachusetts Bay Colony did not countenance tolerance of opposing religious views. Their “city upon a hill” was a theocracy that brooked no dissent, religious or political.

The most famous dissidents within the Puritan community, Roger Williams and Anne Hutchinson, were banished following disagreements over theology and policy. From Puritan Boston’s earliest days, Catholics (“Papists”) were anathema and were banned from the colonies, along with other non-Puritans. Four Quakers were hanged in Boston between 1659 and 1661 for persistently returning to the city to stand up for their beliefs.

Throughout the colonial era, Anglo-American antipathy toward Catholics—especially French and Spanish Catholics—was pronounced and often reflected in the sermons of such famous clerics as Cotton Mather and in statutes that discriminated against Catholics in matters of property and voting. Anti-Catholic feelings even contributed to the revolutionary mood in America after King George III extended an olive branch to French Catholics in Canada with the Quebec Act of 1774, which recognized their religion.

When George Washington dispatched Benedict Arnold on a mission to court French Canadians’ support for the American Revolution in 1775, he cautioned Arnold not to let their religion get in the way. “Prudence, policy and a true Christian Spirit,” Washington advised, “will lead us to look with compassion upon their errors, without insulting them.” (After Arnold betrayed the American cause, he publicly cited America’s alliance with Catholic France as one of his reasons for doing so.)

In newly independent America, there was a crazy quilt of state laws regarding religion. In Massachusetts, only Christians were allowed to hold public office, and Catholics were allowed to do so only after renouncing papal authority. In 1777, New York State’s constitution banned Catholics from public office (and would do so until 1806). In Maryland, Catholics had full civil rights, but Jews did not. Delaware required an oath affirming belief in the Trinity. Several states, including Massachusetts and South Carolina, had official, state-supported churches.

In 1779, as Virginia’s governor, Thomas Jefferson had drafted a bill that guaranteed legal equality for citizens of all religions—including those of no religion—in the state. It was around then that Jefferson famously wrote, “But it does me no injury for my neighbor to say there are twenty gods or no God. It neither picks my pocket nor breaks my leg.” But Jefferson’s plan did not advance—until after Patrick (“Give Me Liberty or Give Me Death”) Henry introduced a bill in 1784 calling for state support for “teachers of the Christian religion.”

Future President James Madison stepped into the breach. In a carefully argued essay titled “Memorial and Remonstrance Against Religious Assessments,” the soon-to-be father of the Constitution eloquently laid out reasons why the state had no business supporting Christian instruction. Signed by some 2,000 Virginians, Madison’s argument became a fundamental piece of American political philosophy, a ringing endorsement of the secular state that “should be as familiar to students of American history as the Declaration of Independence and the Constitution,” as Susan Jacoby has written in Freethinkers, her excellent history of American secularism.

Among Madison’s 15 points was his declaration that “the Religion then of every man must be left to the conviction and conscience of every...man to exercise it as these may dictate. This right is in its nature an inalienable right.”

Madison also made a point that any believer of any religion should understand: that the government sanction of a religion was, in essence, a threat to religion. “Who does not see,” he wrote, “that the same authority which can establish Christianity, in exclusion of all other Religions, may establish with the same ease any particular sect of Christians, in exclusion of all other Sects?” Madison was writing from his memory of Baptist ministers being arrested in his native Virginia.

As a Christian, Madison also noted that Christianity had spread in the face of persecution from worldly powers, not with their help. Christianity, he contended, “disavows a dependence on the powers of this world...for it is known that this Religion both existed and flourished, not only without the support of human laws, but in spite of every opposition from them.”

Recognizing the idea of America as a refuge for the protester or rebel, Madison also argued that Henry’s proposal was “a departure from that generous policy, which offering an Asylum to the persecuted and oppressed of every Nation and Religion, promised a lustre to our country.”

After long debate, Patrick Henry’s bill was defeated, with the opposition outnumbering supporters 12 to 1. Instead, the Virginia legislature took up Jefferson’s plan for the separation of church and state. In 1786, the Virginia Act for Establishing Religious Freedom, modified somewhat from Jefferson’s original draft, became law. The act is one of three accomplishments Jefferson included on his tombstone, along with writing the Declaration and founding the University of Virginia. (He omitted his presidency of the United States.) After the bill was passed, Jefferson proudly wrote that the law “meant to comprehend, within the mantle of its protection, the Jew, the Gentile, the Christian and the Mahometan, the Hindoo and Infidel of every denomination.”

Madison wanted Jefferson’s view to become the law of the land when he went to the Constitutional Convention in Philadelphia in 1787. And as framed in Philadelphia that year, the U.S. Constitution clearly stated in Article VI that federal elective and appointed officials “shall be bound by Oath or Affirmation, to support this Constitution, but no religious Test shall ever be required as a Qualification to any Office or public Trust under the United States.”

This passage—along with the facts that the Constitution does not mention God or a deity (except for a pro forma “year of our Lord” date) and that its very first amendment forbids Congress from making laws that would infringe of the free exercise of religion—attests to the founders’ resolve that America be a secular republic. The men who fought the Revolution may have thanked Providence and attended church regularly—or not. But they also fought a war against a country in which the head of state was the head of the church. Knowing well the history of religious warfare that led to America’s settlement, they clearly understood both the dangers of that system and of sectarian conflict.

It was the recognition of that divisive past by the founders—notably Washington, Jefferson, Adams and Madison—that secured America as a secular republic. As president, Washington wrote in 1790: “All possess alike liberty of conscience and immunity of citizenship. ...For happily the Government of the United States, which gives to bigotry no sanction, to persecution no assistance requires only that they who live under its protection should demean themselves as good citizens.”

He was addressing the members of America’s oldest synagogue, the Touro Synagogue in Newport, Rhode Island (where his letter is read aloud every August). In closing, he wrote specifically to the Jews a phrase that applies to Muslims as well: “May the children of the Stock of Abraham, who dwell in this land, continue to merit and enjoy the good will of the other inhabitants, while every one shall sit in safety under his own vine and figtree, and there shall be none to make him afraid.”

As for Adams and Jefferson, they would disagree vehemently over policy, but on the question of religious freedom they were united. “In their seventies,” Jacoby writes, “with a friendship that had survived serious political conflicts, Adams and Jefferson could look back with satisfaction on what they both considered their greatest achievement—their role in establishing a secular government whose legislators would never be required, or permitted, to rule on the legality of theological views.”

Late in his life, James Madison wrote a letter summarizing his views: “And I have no doubt that every new example, will succeed, as every past one has done, in shewing that religion & Govt. will both exist in greater purity, the less they are mixed together.”

While some of America’s early leaders were models of virtuous tolerance, American attitudes were slow to change. The anti-Catholicism of America’s Calvinist past found new voice in the 19th century. The belief widely held and preached by some of the most prominent ministers in America was that Catholics would, if permitted, turn America over to the pope. Anti-Catholic venom was part of the typical American school day, along with Bible readings. In Massachusetts, a convent—coincidentally near the site of the Bunker Hill Monument—was burned to the ground in 1834 by an anti-Catholic mob incited by reports that young women were being abused in the convent school. In Philadelphia, the City of Brotherly Love, anti-Catholic sentiment, combined with the country’s anti-immigrant mood, fueled the Bible Riots of 1844, in which houses were torched, two Catholic churches were destroyed and at least 20 people were killed.

At about the same time, Joseph Smith founded a new American religion—and soon met with the wrath of the mainstream Protestant majority. In 1832, a mob tarred and feathered him, marking the beginning of a long battle between Christian America and Smith’s Mormonism. In October 1838, after a series of conflicts over land and religious tension, Missouri Governor Lilburn Boggs ordered that all Mormons be expelled from his state. Three days later, rogue militiamen massacred 17 church members, including children, at the Mormon settlement of Haun’s Mill. In 1844, a mob murdered Joseph Smith and his brother Hyrum while they were jailed in Carthage, Illinois. No one was ever convicted of the crime.

Even as late as 1960, Catholic presidential candidate John F. Kennedy felt compelled to make a major speech declaring that his loyalty was to America, not the pope. (And as recently as the 2008 Republican primary campaign, Mormon candidate Mitt Romney felt compelled to address the suspicions still directed toward the Church of Jesus Christ of Latter-day Saints.) Of course, America’s anti-Semitism was practiced institutionally as well as socially for decades. With the great threat of “godless” Communism looming in the 1950s, the country’s fear of atheism also reached new heights.

America can still be, as Madison perceived the nation in 1785, “an Asylum to the persecuted and oppressed of every Nation and Religion.” But recognizing that deep religious discord has been part of America’s social DNA is a healthy and necessary step. When we acknowledge that dark past, perhaps the nation will return to that “promised...lustre” of which Madison so grandiloquently wrote.

Kenneth C. Davis is the author of Don’t Know Much About History and A Nation Rising, among other books.