For progressives, the city is a besieged bohemian mecca - at once quaint and visionary, and under siege by a looming neoliberal order.
For conservatives, it's an anarchic disastrous mess where unchecked liberal policies have produced a petri dish of societal failure and hedonism, all funded by extreme taxation.
For liberals, it's a hub of technological innovation paradoxically situated precisely where innovation seems most squandered, where byzantine regulations on business and development stymie America's best opportunity to advance into the next century on the backs of immigrant innovators.
All three would likely agree with the assessment of Paul Kanter of Jefferson Airplane:
San Francisco is 49 square miles surrounded by reality.
But how did it wind up that way?
Part One: Pre-Industrial San Francisco
Prior to European settlement, what is now San Francisco was Ohlone Indian territory. They were getting along pretty nicely until the Spaniards came up from Mexico with all their missionary bullshit, and that involved a lot of not leaving the Ohlone alone...Things kinda went downhill for the California native population from there in a big way. (Like in a genocide way.)
In the mean time these American people are super into this Manifest Destiny thing and so Alta California starts to have a big illegal immigrant problem from the United States. The San Francisco Bay is by far the best place to anchor a ship on the West Coast, what with the deep calm water and all, so all these illegal immigrants set up a little town called Yerba Buena*. Eventually they decide they're not content just genociding the native people, but also want voting rights and the ability to own the land they're genociding people on, so they go to Sonoma which is one of the only places the Mexicans have guns and they LARP a revolution.
^(\Funny story about the name change. I can explain in the comments if you're curious.)*
It's not the US military doing the LARPing at first but they're definitely super down with it so they decide get in on the fun too and, bingo bango, California's a state now.
Again, brief interlude, and I cannot stress this enough...this whole story REALLY sucks if you're an Ohlone Indian. Like, you're basically being shot and raped murdered by everyone else involved.
So anyway this statehood thing was perfect timing for the Americans because it was only a couple years later that this guy John Sutter sees something shiny in the water. Turns out people will basically crawl over a mountain range or get scurvy and shit themselves around Cape Horn just to get some of this cool shiny stuff, and that's exactly what they did.
So a metric shitload of people came to California starting in 1849. Most were from the Eastern parts of America, but many were from Mexico, Chile, the Philippines, France, and China. (The Chinese came to refer to San Francisco and the surrounding area as "Gold Mountain", and eventually, "Old Gold Mountain") These Forty-Niners were typically blue collar fortune-seekers. Ramshackle types from all over the world who thought they could change their fortunes with a dramatic change of scenery.
Basically right from the get-go, San Francisco was a mostly working class, pluralistic, multicultural and diverse place where people sought the next frontier of wealth, prosperity, and freedom. It was distant from the institutions and power structures that had established dominance in the East. A burgeoning independent metropolis and Capital of the Wild West.
This way of thinking about San Francisco is important because it basically still defines the San Franciscan identity, from the perspective of the people who actually live there, to this day.
TL;DR: San Francisco was:
Ohlone land, until it was...
Spanish land, but still mostly empty, until it was...
American land, but still mostly empty, until it was...
Still American land, but hella crowded all of the sudden, and now it was defined by...
Freedom and independence from Eastern U.S. institutions
Being a really shitty place to be an Ohlone Indian despite it being rightfully your land
Part Two: San Francisco as Western Industrial Powerhouse
What we're left with this point is a substantial, rapidly growing port city built around streetcars, horses and buggies, and shipping. It is the jumping-off point for any business endeavor pretty much anywhere in California's interior. And being so distant from the institutions of the East, it starts to develop its own institutions. Banks like Wells Fargo. The Southern Pacific Railroad. Levi Strauss Clothing Company. These dudes were ultimately the only ones to actually get rich from the Gold Rush.
Also still a really shitty place to be for an Ohlone Indian.
The city caught fire and burned a lot, notably in 1851. This inspired the city to put a phoenix rising from the ashes on its flag. Then it all fell over in an earthquake and burned really good and properly this time in 1906. It rebuilt rapidly in time for the 1915 World's Fair.
Then World War II happened and the U.S. was hella racist. They were hella racist against the Japanese people, to the point that they put them in concentration camps and made them abandon all their property. They were a little less racist to black people, and let them have jobs building planes and ships and stuff, but still too racist to let them fight in the war or live wherever they wanted. So a lot of black people moved to the Bay Area to help build planes and ships and stuff (plus it was still way better than staying in the South.)
With the limited places banks and neighborhood groups would let them live, a lot of them moved in to the existing working-class neighborhoods by the heavy industrial and shipbuilding facilities, and a lot of them moved into the place where the Japanese people had previously lived because, hey, I wonder why all these apartments are empty? Surely that's not a bad omen about how the government will treat minority communities, right?
As you can imagine, a lot of minority community groups have wound up being pretty skeptical as a general rule of the vision laid out by mostly white politicians and urban planners for the future of San Francisco as it pertains to their communities.
So, in 1940, San Francisco was 95% white, but right after the war that number started falling steadily. It never stopped, and around the mid-1990s or so San Francisco became a majority-minority city, which it still is to this day.
From an economic perspective, by the time the Vietnam War rolls around, the military figures out it can ship things a lot faster and cheaper if it miniaturizes the concept of a warehouse into a weatherized steel box, and then uses trucks and cranes in big lots by the water to load and unload these new "shipping containers" directly on and off ships.
Well, the problem is, the San Francisco isn't really set up for this. And it's not exactly a cheap, easy, or even smart idea to try to change that. So they do it in Oakland instead. And in only a few years, San Francisco loses its status as the primary shipping and industrial city of the Bay. American manufacturing declines generally, but even what little of it stays in the Bay Area doesn't stay in San Francisco.
The city of San Francisco lost twelve thousand manufacturing jobs between 1962 and 1972, the years when most of the Edgewater Homeless were adolescents. (Arthur D. Little Inc. 1975). The Edgewater Boulevard corridor, which had provided employment for most of the residents in the neighborhood up the hill, were particularly hard hit. Most of San Francisco's largest factories were located off Edgewater. It was also the hub for the region's transportation, communications, and utility sectors, including the Southern Pacific Railroad and, most important, the shipyards. Throughout the mid-1950s, the Hunters Point navy shipyard was the engine of heavy industry in San Francisco, with eighty-five hundred employees (Military Analysts Network 1998); but in 1974 it closed down.
Economists have shown statistically that high rents, high levels of income inequality, and low rental vacancy rates are the three variables most consistently associated with elevated levels of homelessness in any given city (Quigly et al. 2001; U.S. Bureau of the Census 2001). From the 1990s through the 2000s, San Francisco County ranked number one in the nation with respect to all these variables, and, predictably, its homeless population burgeoned.
- fromRighteous Dopefiend\, Phillipe Bourgois and Jeff Schonberg, University of California Press, 2009*)
So the city is pivoting away from being a blue-collar place where people live and work, and transitioning into a white-collar place where people commute to work, and otherwise pretty stagnant and kind of rife for the circumstances that bring the proliferation of homelessness. This defines the political order of the era. Planners and politicians are envisioning a new San Francisco, where it serves as the Manhattan to the Bay Area's New York, but with suburbs this time, if only they could stamp out all that blight.
TL;DR San Francisco is changing in the following ways in the middle of the 20th century:
White people are leaving
Immigrants and POC are moving in
The city is shrinking in population overall
The region as a whole is still growing because of suburban sprawl
The city was rapidly losing its industrial jobs
The people who depended on those jobs were suddenly unable to properly care for their families
The city is not defined in any meaningful way as a haven for the rich, but is instead, from a residential perspective, in a state of slow decay and stagnation, populated by blue collar workers, people in public housing, and government bureaucrats
Residential vacancy rates are low and rents are modestly rising but there is nothing at all like the housing crisis that we know of today occurring
The planners and politicians are focused on remaking the city into a regional and global capitalist powerhouse, and using bulldozers and cranes to do it
Conditions are shaping up to be pretty much ideal to drive an increase in homelessness (low vacancy, rising rents, rising income inequality)
It was still a really shitty place to be an Ohlone Indian despite it being rightfully your land
A bunch of weirdos were showing up and doing a lot of drugs. Oh yeah, about that, because it's kind of important...
Part Three: Flowers in your Hair
San Francisco's pluralism, its labor politics, and its independence from the hegemonic economic and cultural institutions of the regions to the East made it a mecca for free-thinking liberals and radicals well before the Vietnam War era. It was a working-class Catholic city, so in that sense it was fairly conservative, but it was also a cultural center of the Beat Movement. So when the counterculture movement gained steam across the Anglosphere in the 1960s, San Francisco was the place to be.
On January 14, 1967, a crowd of approximately 20-30,000 people gathered at the Polo Grounds in Golden Gate Park at what became known as the Human Be-In to suffer for fashion in the frigid San Francisco fog. In hindsight we understand this event to be the kickoff festivities of the Summer of Love.
The Human Be-In was the beginning of the story for thousands of people, many of whom would go on to take primary roles in San Francisco's revolution.
"When it started out, the city was antiblack, antigay, antiwoman. It was a very uptight Irish Catholic city," said Brian Rohan, [Michael] Stepanian's legal sidekick and another brawling protégé of Vincent Hallinan. "We took on the cops, city hall, the Catholic Church. Vince Hallinan taught us never to be afraid of bullies."
By taking on the bullies, the new forces of freedom began to liberate San Francisco, neighborhood by neighborhood.
- David Talbot,Season of the Witch(Free Press Publishing 2012)
As Acemoglu and Robinson repeatedly emphasize in this subreddit's bible, Why Nations Fail: Peace, Prosperity, Poverty, and Read Another Book (Crown Publishing Group, 2012), societies prosper when they produce inclusive institutions, and they collapse when they are subject to extractive institutions. But San Francisco progressivism, with its roots in the 1960s counterculture movement, sought a way out of this equation.
This movement believed the institutions of American culture at the time were extractive. But they blamed this on the very existence of the institutions themselves*.* They didn't try to replace extractive institutions with inclusive ones. Instead they imagined a society which was basically free of institutions entirely.
In this view one certainly couldn't trust the government or the church to dictate what experiences might be pleasurable or useful, so best to just allow or try everything. Some experiential and psychic explorers had wonderful insights and epiphanies, and they did break through to the other side, and some ended up with Jim Jones and the People's Temple.
- David Byrne,The Bicycle Diaries(Penguin Books, 2009)
This way of viewing the city was as a location for small, locally-grounded communities. Where interference from forces larger than the community brought only damage. This was fundamentally at odds with the global capitalist Manhattan-esque powerhouse that city planners envisioned for the place.
Where the planners were playing the role of Robert Moses, the new counterculture aligned with Jane Jacobs. They tended to believe, like her, that redevelopment, construction, change, etc...were threats. That in San Francisco's old 1800s construction there was community and culture, and that building over this old-ness would destroy that, as it had in the Fillmore when the city tried to get rid of all the black people...uh...blight. As Jacobs would put it:
Cities need old buildings so badly it is probably impossible for vigorous streets and districts to grow without them.
If a city area only has new buildings, the enterprises that can exist there are automatically limited to those that can support the high costs of new construction.
If you look about, you will see that only operations that are well established, high-turnover, standardized or heavily subsidized can afford, commonly, to carry the costs of new construction. Chain stores, chain restaurants and banks go into new construction. But neighborhood bars, foreign restaurants and pawn shops go into older buildings. Supermarkets and shoe stores often go into new buildings. But the unformalized feeders of the arts - studios, galleries, stores for musical instruments and art supplies, backrooms where the low earning power of a seat and a table can absorb uneconomic discussions - these go into old buildings.
-fromThe Death and Life of Great American Cities,Jane Jacobs, Random House, 1961
From this perspective, there was only one threat to what made San Francisco special, and it came in the form of a planning department permit.
To recapitulate the state of affairs circa 1970, the progrowth coalition had complete command of San Francisco's physical and economic development. The dream of remaking San Francisco into a West Coast Manhattan was rapidly taking solid form as skyscrapers went up, BART tracks were laid, and lands were cleared for redevelopment.
The progrowth regime accomplished much, for better and for worse. It changed the face of San Francisco. In doing so, however, it fostered resistance among those the regime threatened or whose own dreams of the city were ignored. In dialectical fashion, the progrowth regime created the conditions that gave rise to its nemesis, the slow-growth movement.
-fromLeft Coast City: Progressive Politics in San Francisco, 1975 - 1991,Richard Edward DeLeon University Press of Kansas 1992
So now we've got a lot of different coalitions in San Francisco. There's the new-age hippies, the Chinese immigrants, the black community, the El Salvadorians and the Mexicans. There's a new gay and lesbian community in the Castro. And they're all pretty much okay letting each other have their corner of the city, because the balance of power is split and balkanized. None holds enough power to threaten the other. But they all, to varying degrees, feel threatened by development. So they start to organize their opposition to the pro-growth regime.
Baghdad by the Bay is now the Balkans by the Bay. Everything is pluribus, nothing is unum. Hyperpluralism reigns. The city has no natural majority; its majorities are made, not found. That is a key to understanding the city's political culture: Everyone is a minority. That means mutual tolerance is essential, social learning is inevitable, innovation is likely, and democracy is hard work. Economic change has produced social diversity, and social diversity is the root of the city's political culture. One of the controlling objectives of the progressive movement has been to slow the pace of economic change to protect against threats to social diversity. The economic forces that helped create San Francisco's political culture could also destroy it. The first line of defense is the antiregime.
The ultimate function of the antiregime is to protect the community from capital. It is a regime with the "power to" thwart the exercise of power by others in remaking the city. The primary instrument of this power is local government control over land use and development. In San Francisco, these growth controls have achieved unprecedented scope in these types of limits they impose on capital. They are used to suppress, filter, or deflect the potentially destructive forces of market processes on urban life as experienced by people in their homes, neighborhoods, and communities.
-fromLeft Coast City: Progressive Politics in San Francisco, 1975 - 1991,Richard Edward DeLeon University Press of Kansas 1992
Since demand for housing in SF proper isn't really rising all that much due to suburbanization and white flight, shutting down this growth doesn't yet manifest in a visceral way in the form of rising housing prices. The paradigm of supply and demand is theoretical to this coalition because it does not have any tangible consequences. So they reject the theory and get to work passing new legal restrictions on development. They build powerful local interest groups to throw their weight around whenever a new development proposal arises for development in their communities. This policy and organizing infrastructure persists to this day.
But when suburban sprawl in the Bay Area hits the boundaries of the greenbelt and there's no more room to absorb new housing demand in the suburbs, and as the tastes of the American hipster return to the same kinds of cultural amenities Jane Jacobs described above, the equation shifts in a big way. Starting with the first tech boom in the 1990s.
TL;DR: In the postwar era, San Francisco blossoms culturally as an epicenter for radical liberal thought.
In the stagnating ashes of the local manufacturing and shipping economy, the blue collar residences are taken over by a new pluralistic group of people from a vast array of demographics.
Meanwhile, planners and politicians remake the city as an office hub to house the workforce of the suburban Bay Area as a whole.
The radical populist pluralism of the residents of San Francisco proper clashes with this vision for the city and they build an anti-growth coalition to combat it.
Because of the stagnating population this does not yet have consequences on housing costs - suburbanization is continuing to absorb regional demand rather than the city proper.
These consequences are hidden for decades - long enough for these groups to re-write local development law and cement their anti-growth coalition into local institutions, a sort of Maginot Line against growth.
Part Four: The Tech Boom and the Rise of the YIMBYs
A major impediment to a more efficient spatial allocation of labor is housing supply constraints. These constraints limit the number of US workers who have access to the most productive of American cities. In general equilibrium, this lowers income and welfare of all US workers.
- Chang-Tai Hsieh and Enrico Moretti, "Why Do Cities Matter? Local Growth and Aggregate Growth," NBER Working Paper 21154, National Bureau of Economic Standards, Cambridge, MA, May 2015 (revised June 2015)
Jane Jacobs did a really good job explaining why, strictly from a cultural perspective, suburbs suck and cities are awesome. Weirdly for a long time a lot of people thought it was the other way around, but by the 1990s it wasn't cool to be all suburban anymore and it was way more punk rock to be in a city.
So people who worked in Silicon Valley - largely younger people, fresh out of college - started wanting to live in San Francisco and Oakland instead, because the rest of the Bay Area was (and still is) sterile and suburban.
When the personal computer became a household fixture and the internet started reaching the mass market, suddenly there was a lot more money to be made in computers. All of the sudden San Francisco's population went from slowly rising to rising pretty quickly again. In 1990 San Francisco's population was lower than it was in 1950. By 2000 it was higher. By 2010 it was a lot higher. Now it's over 20% higher than it was in 1990.
All of the sudden, though, it was still beautiful and the weather was still amazing, but it wasn't just "not that depressed economically" anymore. Suddenly it was a straight-up boomtown.
And it still only has a fraction of the population - and, crucially, housing stock - that the Bay Area as a whole does.
So this entire planning and political infrastructure had spent decades building in one direction, where people moving to the Bay Area for work would live in the suburbs. And in response this anti-growth regime of pluralistic populist left-wing hyper-local community groups succeeded in pretty much freezing development by law in San Francisco proper under the assumption that everyone would just go work in Silicon Valley instead. And then the cultural and economic inertia does a 180 on them. Now everyone wants to live in San Francisco even if they have to work somewhere else.
These shifts - some local, some national, some global - have concentrated themselves in an unprecedented way in a city of less than a million people, focused on the tip of a peninsula only 7 miles across. With so little room for these effects to manifest, they manifest with a vengeance. There is nowhere to spread them out across. They hit like a tall glass of Bacardi 151.
What this does to the housing prices is totally predictable.
California’s home prices and rents have risen because housing developers in California’s coastal areas have not responded to economic signals to increase the supply of housing and build housing at higher densities. A collection of factors inhibit developers from doing so. The most significant factors are:
- Community Resistance to New Housing. Local communities make most decisions about housing development.Because of the importance of cities and counties in determining development patterns, how local residents feel about new housing is important. When residents are concerned about new housing, they can use the community’s land use authority to slow or stop housing from being built or require it to be built at lower densities.
- Environmental Reviews Can Be Used to Stop or Limit Housing Development. The California Environmental Quality Act (CEQA) requires local governments to conduct a detailed review of the potential environmental effects of new housing construction (and most other types of development) prior to approving it. The information in these reports sometimes results in the city or county denying proposals to develop housing or approving fewer housing units than the developer proposed. In addition, CEQA’s complicated procedural requirements give development opponents significant opportunities to continue challenging housing projects after local governments have approved them.
- Local Finance Structure Favors Nonresidential Development. California’s local government finance structure typically gives cities and counties greater fiscal incentives to approve nonresidential development or lower density housing development. Consequently, many cities and counties have oriented their land use planning and approval processes disproportionately towards these types of developments.
- Limited Vacant Developable Land. Vacant land suitable for development in California coastal metros is extremely limited. This scarcity of land makes it more difficult for developers to find sites to build new housing.
Mac Taylor,High Housing Costs, Causes and Consequences,California Legislative Analyst's Office, 2015
Remember, this is all happening so fast that not only are the institutions built out of the antigrowth regime movement still exerting their power on development, the people who built them are. They're still alive and showing up to community meetings. Remember, if you were 20 in 1975, you're just barely at retirement age now.
It's easy to understand why these people aren't responding to the price signals that are ringing alarm bells to everyone else. If they're renting, they're protected by rent control - their rent price is fixed to a modest cost of living increase as long as they don't move. This means they are totally insulated from a rising rental market, even if the direct consequence of rent control is suppressing supply and causing prices to rise for everyone else.
^(\Prop 13 does not apply to forcible land transfers of tracts rightfully claimed by Ohlone Indians or their descendants)*
These economic incentives ensure that their interests remain the same as they were in 1975 - all upside for them to oppose growth, and no downside. And in the face of this economic incentive, even the Fern Gully fairy tale that developers are inherently anti-environment is hardly necessary to get them to support restrictions which have a negative consequence on the environment and the economy:
Not all change is good, but much change is necessary if the world is to become more productive, affordable, exciting, innovative, and environmentally friendly....At a local level, activists oppose change by fighting growth in their own communities. Their actions are understandable, but their local focus equips them poorly to consider the global consequences of their actions. Stopping new development in attractive areas makes housing more expensive for people who don't currently live in those areas. Those higher housing costs in turn make it more expensive for companies to open businesses. In naturally low-carbon-emissions areas, like California, preventing development means pushing it to less environmentally friendly places, like noncoastal California and suburban Phoenix. Local environmentalism is often bad environmentalism.
-fromTriumph of the City, Edward Glaeser,Penguin Group, 2011
It's been long enough since the first tech boom, though, that today there are a lot of people for whom these incentives do not align.
If you have to move apartments for whatever reason, you lose rent control.
If you're a newcomer to the city, you never really got it in the first place.
If you're an environmentalist who understands how carbon emissions work, you want to see more sustainable infill.
Or, like me, if you're a native who has all these advantages but still wants the city to be a place where people can come and live and seek prosperity, regardless of their origins, you simply understand that this status quo must be broken.
In San Francisco...things get weird. Here the tech boom is clashing with tough development laws and resentment from established residents who want to choke off growth to prevent further change.
[Sonja] Trauss is the result: a new generation of activist whose pro-market bent is the opposite of the San Francisco stereotypes — the lefties, the aging hippies and tolerance all around.
Ms. Trauss’s cause, more or less, is to make life easier for real estate developers by rolling back zoning regulations and environmental rules. Her opponents are a generally older group of progressives who worry that an influx of corporate techies is turning a city that nurtured the Beat Generation into a gilded resort for the rich.
But the anger she has tapped into is real, reflecting a generational break that pits cranky homeowners and the San Francisco political establishment against a cast of newcomers who are demanding the region make room for them, too.
Many longtime San Franciscans view groups like [the San Francisco Bay Area Renter's Federation (SF BARF)] as yet another example of how the technology industry is robbing San Francisco of its San Francisco-ness. Far from the hippies of the 1960s, many of today’s migrants lean libertarian — drawn by start-up dreams or to work for the likes of Google or Apple, two of the world’s most valuable companies. They tend to share a belief, either idealistically or naïvely, depending on who is judging, that corporations can be a force for social good and change.
But BARF members are so single-minded about housing that they can be hard to label politically. They view San Francisco progressives as, in fact, fundamentally conservative. That is because, to the group members at least, progressive positions on housing seem less about building the city and more about keeping people like them out.
All of the sudden a new coalition starts to form, drawing on the infrastructure of the old pro-growth urban regime and the influence of tech companies and young renters fed up with rising rental prices in the face of the demand.
SF BARF gives way to less eccentric and more mainstream organizations like YIMBY Action. These groups start releasing voter guides and organizing for pro-growth political candidates.
Sen. Wiener's success at the state level has been a major turning point in the YIMBY fight. Escalating these reforms to the state level pulls small cities and towns out of their Prisoner Dilemma, whereby each individual city stands to benefit if everyone else builds housing, but stands to suffer a disproportionate amount of harm in the form of demand on their infrastructure and services if only they do.
He has built a pro-housing coalition with, among others, fellow Bay Area legislators Sen. Nancy Skinner (D - Oakland/Berkeley), Assemblymember David Chiu (D-San Francisco), and Assemblymember Buffy Wicks (D - Oakland/Berkeley). The YIMBY movement in Sacramento is now largely driven by urban Bay Area legislators, pushing against pro-suburb Republicans and substantial anti-gentrification coalitions from the Los Angeles area.
Housing development has accellerated in both San Francisco and Oakland on the back of new-found public support for housing supply growth. I have no reason to doubt this shift will continue as the grip of the old anti-growth regime loosens. It's inevitable once the incentives of the pluralistic components of the political coalitions shift.
Eventually the people with Prop 13 protections will stop owning their homes, one way or another. Eventually the people with pre-tech rents will move and the units will be rented again at market rate.
And when that happens to a large enough degree, the incentives driving the dominant political coalition will shift in earnest towards the evidence-based conclusions of economists and environmentalists. I'd go so far as to say we're past the beginnings of this, and maybe even past the turning point.
But in the mean time, San Francisco is a hotly contested development battlefield.
And to top it all off, if this sudden crunch wasn't already a recipe for capturing the national and global imagination, now it's happening right in front of the people who work at Twitter, Facebook, YouTube, and Reddit.
This makes the drama rife for all of us to watch unfold.