Even as he battles economic woes at home and security threats abroad, President Barack Obama is advancing a bold strategy for improving the nation’s schools. His new budget aims to recast Washington’s role in education, shedding the regulatory minutiae of the No Child Left Behind Act put in place by George W. Bush.

Instead, Obama is banking on economic incentives to alter the behavior of educators. Armed with $4.3 billion in stimulus funding, Secretary of Education Arne Duncan is pushing states to award bonuses to effective teachers, curtail the power of teachers unions, and seed a more robust generation of charter schools.

This competition- and incentive-based approach to reform is partly drawn from the playbook of moderate Republicans a generation ago. But it’s Obama’s faith in charter schools—which are financed by taxpayers but operated outside the strictures of the normal public-school system—that’s proving most controversial. To some, charters are the key to reforming public education by increasing flexibility and accountability at the local level. Others feel the promotion of charters diverts attention from public-school systems’ most intractable problems. And while the number of charter schools nationwide continues to grow, the evidence of their success remains inconclusive. Obama’s approach ensures that more charters will be established. But it can’t guarantee that they’ll work.

Enthusiasts see charters as a fix for schools suffocated by government rules and limited by labor contracts that protect teachers at students’ expense. Charter principals can hire (and fire) their own teachers and experiment with inventive classroom techniques. And charter managers respond to parents’ preferences, from requiring uniforms and drilling basic skills to emphasizing themes of social justice or ethnic identity. Some struggling Catholic dioceses have turned their school properties over to charter firms, hoping to preserve at least some of the benefits of a parochial education for their students. (See Paul Moses, “The Public Option,” Commonweal, December 4, 2009.) Mayors from New York to Los Angeles, impatient with foot-dragging by union leaders, are galvanizing support for charters among neighborhood leaders, working-class families, and wealthy pro-education donors.

 “Charters...force the kind of experimentation and innovation that helps to drive excellence in every other aspect of life,” Obama told the Washington Post last summer. He admires the accomplishments of Secretary Duncan, who expanded charter schools in Chicago during his seven-year tenure there as superintendent. But Duncan acknowledges it’s not clear that charter-school students outperform peers in regular public schools. “I am not a fan of charters,” he told me last fall. “I’m a fan of good charters. Bad charters are part of the problem.”

Still, before school is out for the summer, Duncan will award large grants from the $4.3 billion Race to the Top fund to governors who lift caps on the number of charter schools in their states and promise to hand over failing schools to charter companies, which increasingly dominate a market once led by dissident teachers and community activists. Critics of Obama’s faith in charter schools see charters as a costly distraction from deeper problems threatening all public schools: low pay for entering teachers, a scarcity of inspiring principals, the negative effects of poverty on children’s development. These arguments, voiced most vehemently by union leaders, convinced New York legislators to retain the state’s cap on authorized charter schools, risking the loss of Race to the Top funds.

The idea of liberating public educators from top-down regulation was first advanced by Albert Shanker, the late New York union leader. He wanted to charter individual teachers in the 1980s to allow them to depart from government standards and experiment with new approaches in the classroom. But Shanker’s union colleagues worried that his proposal would undercut seniority and teacher protections. In 1991, Minnesota’s Republican governor, Arne Carlson, signed the nation’s first charter law, granting public dollars to parents or teachers who created their own schools. California followed suit one year later. Initially skeptical of market fixes, Democratic legislators soon came to side with moderate Republicans, heading off the call for school vouchers from a growing chorus of conservatives. President Bill Clinton boosted federal funding for charters. The movement spread quickly, and today almost 1.5 million students attend about forty-two hundred charter schools nationwide, according to the U.S. Department of Education.

In the nation’s urban centers, low test scores in public schools contribute to parents’ enthusiasm for charter schools—often seen as safer, smaller, and staffed with more attentive teachers and administrators. For Alicia Ortiz, a Los Angeles mother of five, charter schools mean a welcome return to tradition. “Uniforms are important, there’s more discipline,” she said. She worries about kids getting lost in the shuffle in overcrowded public schools. Garfield High School senior Jazmin Casas echoed Ortiz’s concerns. “You’ll go into a class, like physiology, and we have forty-seven students,” she said. “Some of the teachers don’t even pay attention to you—it’s like, ‘What’s your name?’” Public schools in Los Angeles are beset by declining enrollment—except for the city’s 152 charters, where overall registration has jumped by one-fifth since last year.

Political support for charters also grows as civic leaders watch middle-class parents (and their employers) flee to the suburbs in search of better public schools. But when big-city mayors, like Antonio Villaraigosa of Los Angeles, attempt to innovate, they run into union leaders who have long opposed reforms that move strong teachers to weak schools and grant principals more power over staffing decisions. Villaraigosa actually began his career as a union organizer, agitating in densely packed schools downtown. Today, he says, “I’m public enemy number one within the United Teachers of Los Angeles.” That’s after raising $3 million to unseat pro-union members of the local school board and replace them with his own allies. He then led a palace revolt, convincing the board to hand over up to 251 campuses to charter firms and other nonprofits. “This is a war, but it’s a war I’m willing to fight,” Villaraigosa told me.

Union leaders see this realignment as a promarket nightmare. “It’s a retreat from the New Deal, the civil-rights politics of the 1960s,” said Josh Pechthalt, vice president of the L.A. teachers union. “It’s the notion that marketplace dynamics are the best way to shape public institutions.” Randi Weingarten, head of the American Federation of Teachers, calls Obama’s education initiative “Bush III.” Still, the threat of charter growth has moved unions in Boston and L.A. to support independent “pilot schools,” similarly innovative campuses that operate under thin labor contracts, granting principals stronger authority and defining more flexible roles for teachers, who are encouraged to stay after school to tutor kids or supervise internships in the community.

Charter schools are now forcing a dramatic realignment of powerful reform groups. Corporate activists and leaders of private foundations—often progressive Democrats who have done well in the markets—believe competition and incentives can work to buoy schools as well. Mayor Villaraigosa relies on the largesse of billionaire developer Eli Broad for campaign contributions and political firepower against combative union leaders. Broad supports California’s charter-school lobby, as does Netflix founder Reed Hastings. Hollywood mogul Casey Wasserman recently tossed in $4.3 million to help implement Villaraigosa’s charter-and-choice plan. Bill Gates added $60 million to experiment with new ways of evaluating and promoting charter-school teachers in L.A. Two former Gates Foundation officials, now inside Obama’s education department, helped design Race to the Top.

Obama insisted to the Washington Post, “I just don’t think about these things ideologically. Federal dollars need to be spent in ways that actually improve student achievement.” But do charter schools really work? Can they improve results across states and districts, or do they simply sap resources—gifted students, effective teachers, taxpayer dollars—from nearby public schools, further isolating the poorest students and least-engaged parents?

Comparing the learning curves of kids in charters and regular schools across sixteen states, Stanford University economist Margaret Raymond found that in some states, charter schools offer their students significant advantages—but quality varies so much from state to state that in the aggregate there are no discernible achievement differences. This confirms an earlier study by the National Center for Education Statistics. In fact, taking all sixteen states together, Raymond found that students attending charter high schools actually did slightly worse than their counterparts in regular schools. In some cities, like L.A., charters enroll smaller percentages of special-education students and children from non-English-speaking homes. Charter-school advocates often tout higher test scores without mentioning that their schools are filtering out more difficult students.

Encouraging results have been measured at well-supported charter schools like the Knowledge Is Power Program (KIPP) Academy in the South Bronx and Green Dot charters in L.A., which have boosted student attendance and graduation rates. But these boutique charters benefit from ample foundation support; they don’t live on public funds alone. New York City charters are aided by nonprofit reform groups, whose oversight contributes to their success. But where governors license many charter schools at once and then require little public accountability, as in Arizona, the “choice” for parents is from among a wider variety of mediocre schools. The Obama strategy of pressuring governors to approve new charter schools may stretch the states’ capacity to exercise quality control.

Secretary Duncan told me, “This is not about a thousand flowers blossoming. You have to hold [charter schools] accountable.” Both skeptics and fans of the charter-school movement are prepared to do just that. The White House needs labor on board to steer its reform agenda through an increasingly cautious Congress, and wary union leaders will be keeping a close eye on Obama’s education initiatives. Parents and educators will be watching to see whether the push to reward teachers with merit pay and nurture local competition through charter schools motivates teachers and elevates student achievement. If not, charters could go from being a promising innovation to being just another part of the public-schools problem.

Related: Course Correction and The Public Option:
Will Catholic Schools Become Charter Schools? by Paul Moses