Contract and Grant Data at the Federal Level

fedspending.org, a project funded by the Sunlight foundation, provides quick and easy access to all of the contracts and grants of the federal government, data similar to what the SF controller’s office has on a local level in San Francisco.  It beat a project mandated by the federal government by a year and spent just a fraction of the cost, as reported by Sunlight’s Larry Mackinson.

Next step is mashing the data with campaign funding and other related information. We’ve started this for San Francisco– at whosfundingwhom.org, one can now see a list of entities who appear in both the campaign finance data and the contracts data. But this is only the start of what is possible in helping researchers do jobs like find organizations that have broken the law by contributing in the same year they were awarded a contract.

We mashed by downloading the entire SF contracts data and campaign finance data. What is needed, of course, at both the local and federal levels, are web service APIs, such as what exists for funding data at the state level. These would eliminate the need for mashup applications to download entire databases, and keep data as up-to-date as possible.

Sunlight Foundation’s Collaborative Muckraking

Ellen Miller and Sunlight have yet another innovative project. For this one, they set up a collaborative muckraking project whereby any technical politico (or political techie) could participate by researching nepotism in the congress. Its a great example of how the ‘masses’, with the help of collaborative software, can do important work that would take a single journalists weeks. This project, focusing on spousal relationships of the 435 congress members, took 2 days!

Jay Rosen, NYU journalism professor and founder of the innovative journalism experiment PressThink has a nice article about the project.

followthemoney.org has an API for State Campaign Funding Data

The Institute on Money in Politics site, followthemoney.org is great for finding campaign funding data for any state in the US. They have real people that collect data for each state and even standardize contributor names, something which I know is a problem from my work with the SF site whosfundingwhom.org.

They also have an API! The first I’ve seen at any level of government for this type of data. I’m already making plans for a California version of opencampaigns.com which mashes up funding data from the followthemoney API. Funding data and news articles for each politician, all in one place!

Finally, they have a cut and paste widget that you can create at their site with a form. You say what candidate you want, and it dumps html code which gives you a ‘live’ feed. Here’s a sample web page where I pasted in the generated code. It makes something that looks like this:

Thanks for this link goes to John Musser and his ProgrammmableWeb site which is the place to go to find interesting APIs and mashups about anything, even politics.

Who are the ‘business interests’ in SF? Part III

When the media refers to San Francisco ‘business interests’ , about whom do they speak? This article is the fourth in a series exploring this question . We tackle the problem using the campaign finance data that SF political committees must file with the San Francisco Ethics Commission. The data shows the contributions and expenditures made and received and includes committees for individual candidates and for propositions, as well as data about individuals, corporations, and the vendors that spend the money, typically political consultants and lobbyist firms. This data is perhaps best viewed at whosfundingwhom.org, a site developed by some University of San Francisco students.

First, who is raising money? The top ten fundraisers consisted of three ‘proposition’ committees, four board of supervisor candidate committees, two business groups, and Newsom for Mayor.

The three committees designed to advocate for or against specific propositions were:

1. Yes on A, Let’s Rebuild San Francisco Schools
2. Save Laguna Honda Hospital: No on D.
3. No on E: The Parking Tax

These committees all raised over $200,000, with Yes on A topping the list at $338,000. Though such committees target specific propositions, it has been speculated that some have more general goals or even target specific candidates. For instance, BeyondChron reported speculation that No on E really had a goal of getting Chris Daly out of office. In 2004, the Chronicle report, “CAMPAIGN 2004 – SAN FRANCISCO Contribution limits don’t crimp spending: Business interests funnel cash to aid certain candidates“, detailed how Donald Fisher’s ‘Yes on Citizenship, No on F’ committee spearheaded a smear campaign of supervisor Gerardo Sandoval when he ran for re-election. So in trying to figure out the landscape, one must consider these ‘proposition’ committees, with their huge bankrolls, to be major players, and perhaps playing for more than their name implies.

What people and companies are behind these efforts, monetarily? Businessman Warren Hellman was the biggest contributor to ‘Yes on A’, putting nearly $50,000 into it personally. Other contributors to Yes on A include Wells Fargo Bank, Bechtel Corporation, and Shorenstein Company LLC.

The top contributor to The No on D effort came from labor, the SEIU United Health Care Workers, who dropped $63,000 into the effort. They also received funds from Donald Fisher ($20,000) and the Turner Construction Company ($15,000) amongst others. No on E was supported by various parking companies as well as the Teamsters Union.

Proposition committees are also funded by business group committees, including the Committee on Jobs, the Building Owners and Managers Association (BOMA), and the Golden Gate Restaurant Association. The Committee on Jobs with both Fisher and Hellman as board members, gave $10,000 to Yes on A as well as over $40,000 to the No on C: Save Transbay committee that was targeting Chris Daly. The source of these Committee on Jobs funds? Top contributors were Charles Schwab, Inc., Hellman ($35,000), and PG&E and AT&T ($25,000 each).

Now one cannot jump to conclusions when following these money trails, but one might speculate that Schwab, Hellman, and the telephone companies don’t care much for Chris Daly. Similar trails exist with the BOMA Builder committees, who are funded by numerous smaller donations (TST Mission Street LLC was the biggest at $10,000) but, like the Committee on Jobs, also funnel money to No on C, No on E, and No on D, and even some directly to candidates such as Rob Black, Daly’s competition in District 6.

The Golden Gate Restaurant Association, which is also funded by smaller donations, contributed directly to campaigns, including those of Black, Michela Alioto-Pier, Doug Chan, and Phil Ting.

So we’ve got ‘proposition’ committees and candidate committees which are funded by business interest committees, which are themselves funded by big mega-donors, big corporations, and smaller businesses. The next question: how do they all spend their money? A quick glance at the top payees at whosfundingwhom.org suggests that the majority of the money goes to political consultant firms. For instance, , $225,000 of the $275,000 the Yes on A committee spent went to BMWL Consulting.

And Yes on A is not alone. BMWL is the top payee for campaign funds in the city, raking in over $600,000. One might say they are the heart of the influence network, taking in a pool of money and dispensing out mailings, flyers, advertisements and whatever else is needed for their clients. Besides the Yes on A committee, their clients include No on C and the aforementioned more general business partnerships BOMA and the Golden Gate Restaraunt Association (GGRA).

What does it all mean? Were ‘business interests’ really out to get Chris Daly? Was Rob Black an innocent as these business interests thrashed his opponent? Are Hellman and Fisher running San Francisco?

Not sure, but hopefully this installment in our little study will at least shed some light on the political landscape. For more information, one can check the whosfundingwhom.org site, as well as opencampaigns.com, a wikipedia-like directory of San Francisco politicos.

Money can and can’t buy love: SF Supervisor candidate spends $250,000 and loses, top spending BOE candidates win

Running for office is expensive in SF. A board of supervisor candidate (dist.4), Doug Chan, spent over $250,000 and lost. Other supervisor candidates are in the six figures as well. Two Board of Education candidates spent over $30,000 and won. Note that these figures don’t include money spent by ‘independent’ committes, such as SOS, who reportedly have spent thousands to get Chris Daly out of office (SOS has reported only $7000 in contributions, to ‘Citizen’s For Reform Leadership’ #1,#2,#3,#4,#5 and #6.).

Chan’s figure is quite interesting given that:

1. he lost,
2. A few weeks ago, the Chronicle published an article, ‘$100,000 grubsteak’, detailing the big spending of his opponent Janryn Mak.
3. the Chronicle endorsed Mr. Chan, and
4. Mak ended up spending over $100,000 less than Chan and still lost (perhaps due to the Chronicle article). Ron Dudum, the projected winner, spent less than either by a large margin.

Here’s the totals that have been reported, as shown on whosfundingwhom.org today and verified at the San Francisco Ethics Commission. Projected winners have a *.

District 2:
Alioto-Pier $138,550 *
District 4:
Chan $261,586
Mak $141,572
Dudum $80,924 *
Jew $84,769
District 6:
Daly $150,450 *
Black $116,653

District 8
Dufty $110,851 *
Rossenthal $45,226
District 10
Maxwell $77,805 *

Board of Education

Mendoza $36,054 *
Jane Kim $33,804 *
Bayard Fong, $33,413
Bob Twomey $21,294
Mauricio Vela $8,145
Dan Kelly $5,105
Kim-Shree Maufus (no reports) *

PEOPLE ARE NOT TEXT: Adding People-Context to On-line Articles

Daniela Kirshenbaum recently wrote an article at Fog City Journal describing how some of the business interests in San Francisco effect the city’s politics. It’s a great article based on a lot of research and connecting the dots between a lot of people and organizations. I’m going to use it as an example of how on-line articles have the potential for adding a lot more context to a story than they do now. About how people should be linkable just like web pages.

Here’s how things work today: A writer, blogger, emailer, or journalist like Daniela writes something. In it, she refers to people with plain text, e.g.,

Donald Fisher of the Gap is a strong voice in San Francisco politics.

The readers of the article then go through the following process: they see names and organizations in the article, they’d like to know more about them, e.g., ‘who is this Donald Fisher?’. So they google him, and Google shows them some pertinent articles, but also old stuff, and stuff about other Donald Fishers, etc. The readers sift and filter through it. If the reader is a journalist or some other serious researcher, perhaps they bookmark the articles pertinent to Donald Fisher the Gap owner. But all the readers do this same ‘filtering’ work, this same Googling. Over and over. They don’t share their research work.

A Better Solution

Wouldn’t it be better if the writer of an article could help her readers out by making the ‘Donald Fisher’ reference in the article point to more information about thee Donald Fisher being talked about, i.e., put a link to a page with the important articles about Donald Fisher. Then the readers wouldn’t have to all Google him to see more, and all that individual filtering work could be eliminated. Plus they could just click on a link as opposed to typing in a name at Google or in a google box. Context for the article would be at their fingertips.

Of course the question arises: to where should the author link the text ‘Donald Fisher’. His wikipedia page is one possibility. So she could change her sentence to:

Donald Fisher of the Gap is a strong voice in San Francisco politics.

This solution is pretty good. The readers, if they want to learn more about this person, can click on the link and get to at least a starting point about Donald Fisher.

But this wikipedia-based solution has some limitations. Not everyone is in wikipedia. Its a world encyclopedia and is at least somewhat thought of as about universally famous people. Fisher is in it, but his page is pretty bare and generic. There’s certainly not any links to articles giving context to how he effects SF politics.

Now a writer could go to wikipedia and add more, and wikipedia will certainly grow as time rolls on. But editing a wikipedia page, though accessible to most computer users, is still a task. Most of us don’t do it. So generally speaking, most people on this earth aren’t in there or, if they are, there isn’t much.

Peoplicious is a suite of web applications with a goal of providing a better solution. Peoplicious sites are domain-specific. The one pertinent to this discussion is about Bay Area politicos and can be reached at http://opencampaigns.com. Its domain-specific nature allows for focus, e.g., the articles concerning Donald Fisher at opencampaigns.com pertain to his effect on San Francisco politics.

Like wikipedia, it is also a site where anyone can add information. However, submitting information to the system is as easy as bookmarking– a user can add an article to a person’s page with about three clicks. Besides articles, one can also add a person’s homepage, their blog if they have one, an image– all the basic information that make up a person’s virtual persona. The idea is that, because it is so easy to add information about a person and anyone can do it, the pages for each person will stay relatively up-to-date. People can share their googling work and make use of the work done by others.

Take for instance Fisher’s page. Users (well, me) have added a number of articles to it. If Kirshenbaum used Fisher’s opencampaigns page as the link, our sample sentence would appear as:

Donald Fisher of the Gap is a strong voice in San Francisco politics.

Click on the above link. The page that appears has recent articles concerning Fisher and pertaining to Bay Area politics.

O.K., great, you say. Let’s say enough users get involved so that opencampaigns grows and stays up-to-date. Isn’t it still laborious for journalists to add links in their articles to the people they reference? The answer is yes. In fact, I did this for Kirshenbaum’s article, you can find the version with links at http://www.cs.usfca.edu/~wolber/politics/kirshenbaum_061015.shtml. It took me ten minutes and it was boring. The last thing a writer will want to do after completing an article is go through and insert these links.

If you’re a journalist, you might be thinking ‘intern’ or ‘secretary’. I’m a computer scientist, so as I was doing the laborious work I thought: hey, software can do this for us! My plan is to write some software that will take an html verison of an article, cross reference the names in it with the opencampaigns database, and automatically add links to the article. Of course the writer will have to check the links for pseudonym problems, but given the domain-specific nature of the data, there should be none or just a few. So adding people-context links to an article can be pretty much effortless.

Summary

I’m a USF computer science professor, and here’s my vision. opencampaigns.com becomes a place where, as a community, we collect articles and categorize them under the people they are about. Journalists then use software (which I will write) to automatically add links to people pages in their on-line articles. In this manner, the articles at sfgate, the examiner, beyondchron, etc. are transformed from independent textual islands to context-rich sources of knowledge.

The public becomes more knowledgeable, politics become more transparent, and maybe, just maybe, our politicians and ‘business leaders’ behave a little better.

You can contact me at wolber@usfca.edu

A del.icio.us, wikipedia-ish People Directory for San Francisco Politicos

The web is full of articles concerning the politicans, candidates, and other politicos in San Francisco, but they are so scattered that researching even a single race would take an individual until November 7th. Most of us google our brains out, working the midnight hours alone in a vain effort to understand the wacky San Francisco political landscape.When we finally give our weary eyes a break the filtering work we’ve performed is lost and the knowledge we’ve gained exists only in our head and our desktops. Maybe we mention a highlight to Joe and Suzy at the coffee shop but that is the extent of our sharing.

opencampaigns.com is a collaborative website with a goal of allowing the San Francisco community to share their research work. Like wikipedia, anyone can add information to the site. In this case, contributors add links about people involved in politics– their homepage, their blog, and any articles about or by the person found on the web. Contributing is as easy as bookmarking, but instead of bookmarking for just yourself, your bookmarks are shared with the community and organized under the people they concern.

As more and more articles are added, the opencampaigns pages become a one-stop shop for information about each person in the system. For instance, the Chris Daly page lists the various articles that have appeared about Chris in this election year. You could find these articles by googling ‘Chris Daly’, but you’d have to do a bunch of filtering work that has already been performed.

Contributors can also add lists and put people in lists. For instance, there are lists for the board of supervisor races including the very entertaining District 6 soap opera, and a list for politicos in the media. There’s even a list for business leaders involved in politics which includes such notables as the Gap’s Donald Fisher and the head of the SOS political committee Wade Randlett

opencampaigns.com is part of an internet movement towards collaborative tools sometimes referred to as Web 2.0. Techies are quite familiar with such software including the social bookmarking system del.icio.us, the photo-sharing site flickr, and the community publishing site digg. opencampaigns.com is an experiment in applying such collaborative ideas to the goal of transparency in politics.

Currently, the system has basically one SUPER-user, the young and handsome professor who developed the site. But the goal is to involve as many community members as possible, so please try it out at http://opencampaigns.com.You can “consume” information without even registering, and contribute after a simple ten second registration.

Who are the city business interests, part II

As election 2006 nears, the talk about money and politics is heating up. In today’s chronicle, Patrick Hoge exposes the money trail behind Prop 90 and the kingpen, libertarian Howie Rich.

Hoge’s article is interesting and one appreciates the hours of research work that went into it. At whosfundingwhom.org, we’re tying to reduce the research work for such reports by providing graphs and other associative views for money trails in SF politics. Check out this automatically generated graph, showing the Golden Gate Restraunt Association at the center:

With a quick glance, one can see some of the key relationships that form the “business interests” in San Francisco. Big players include the so-called Committee on Jobs, the Save Laguna Hospital Committee, and assembly candidate Fiona Ma.

Who are the city “business/real-estate interests” part I

When the press speaks of San Francisco real estate interests, who do they mean? Well, one organization is BOMA– The Building Owners and Managers Association. They are very active in politics, as this search from whosfundingwhom.org shows.

What is BOMA doing to effect the policital process? What politicians do they support? They include Michella Alioto-Pier, Carole Midgen, Tom Hsieh, and Rob Black, as shown here. Wonder if BOMA s also supporting Black’s competitor in District 6, Chris Daly? For the answer, see Chris Daly’s rant about business interests at the Fog City Journal.

San Francisco to Bank of New York: 634 mil!

The top city contractor for the city of san francisco is the Bank of New York— they have city contracts for over 634 million bucks since 2004, almost all with the Airport commission…don’t know what this means exactly, and after all what could be wrong with a little cross-costal friendliness.
Why do I know this? I’m integrating the city contracts database, available at the Controller’s office, with the campaign finance database we have at http://whosfundingwhom.org. The first thing I did was download the raw data from the controllers and run some database operations to sort by total amounts. You can also see which city departments are involved in big contracts, and with whom.

My next step is to do a name match on the databases to find out the top campaign contributors who have also been awarded city contracts. What fun we’re having at http://whosfundingwhom.org!

Here’s the top 10:

Vendor	Amount
B N Y WESTERN TRUST	$634,763,741.00
REGENTS OF THE UNIVE	$177,036,142.00
STATE OF CALIFORNIA	$130,532,260.00
CCSF HRD WORKERS COM	$68,921,845.00
MCKESSON	$60,257,539.00