How open OpenAI?

ai
Tags: #<Tag:0x00007f08e5d72ba8>

#1

Introducing OpenAI (emphasis added):

Because of AI’s surprising history, it’s hard to predict when human-level AI might come within reach. When it does, it’ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest.

We’re hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We’ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies.

Potentially very good, and with very significant backing:

Sam [Altman], Greg [Brockman], Elon [Musk], Reid Hoffman, Jessica Livingston, Peter Thiel, Amazon Web Services (AWS), Infosys, and YC Research are donating to support OpenAI. In total, these funders have committed $1 billion, although we expect to only spend a tiny fraction of this in the next few years.

See previous commentary on YC President Altman’s Technology and wealth inequality and especially The Software Revolution. OpenAI is the first initiative of YC Research.

Musk says in an interview:

we came to the conclusion that having a 501c3, a non-profit, with no obligation to maximize profitability, would probably be a good thing to do.

(Pointers to confirmation that OpenAI is actually a 501©(3) charity welcome.)

How will the commons-favoring policy bolded above be implemented? Hopefully in ways at least up to date with contemporary full open access, open data, free/open source software practices. One trivial signal in this direction that OpenAI could’ve sent would’ve been to make their initial two page website openly licensed. Hopefully they will fix this. Another place to watch for early signals might be github.com/openai though I don’t know whether that is them. Will OpenAI also advocate for commons-favoring public policy around software, perhaps starting by bringing other funders of AI research around to mandating openness?

If it turns out that OpenAI is not informed by and does not implement free/open best practices, I’ll take it as an indicator of how far free/open movements have to progress and a critique of their (well, our) arguments and relevance, as much as I will take it as a disappointing indicator that OpenAI will not meet its potential on the predictable (Open) part of its name, whatever it accomplishes toward the unpredictable (AI) part.


#2

The current answer is not bad but rather mushy.

Ask Me Anything: the OpenAI Research Team:

We’ll constantly re-evaluate the best strategy…the “Open” in “OpenAI” means we want everyone to benefit from the fruits of AI as much as possible.

The one goal we consider immutable is our mission to advance digital intelligence in the way that is most likely to benefit humanity as a whole. Everything else is a tactic that helps us achieve that goal.

Today the best impact comes from being quite open: publishing, open-sourcing code, working with universities and with companies to deploy AI systems, etc… But even today, we could imagine some cases where positive impact comes at the expense of openness: for example, where an important collaboration requires us to produce proprietary code for a company. We’ll be willing to do these, though only as very rare exceptions and to effect exceptional benefit outside of that company.

We’ll post code on Github (https://github.com/openai), and link data from our site (https://openai.com) and/or Twitter (https://twitter.com/open_ai).

We intend to conduct most of our research using publicly available datasets. However, if we find ourselves making significant use of proprietary data for our research, then we will either try to convince the company to release an appropriately anonymized version or the dataset, or simply minimize our usage of such data.

We’ll use others’ software and hardware where possible, and only invent our own if we have to. In terms of deep learning toolkit, we expect to primarily use TensorFlow for the near future. Later we may need to develop new tools for large-scale learning and optimization (which we’ll open-source wherever possible!).

One unanswered question: “Do you have a vision for how you will deal with IP?”

Another “Just how open will OpenAI be? I.e. With results, techniques, code, etc…i.e. Is the Open in OpenAI meant for Open Source?” obtained no response from the team but a commenter pointed out an interview with one of the team:

We are not obligated to share everything — in that sense the name of the company is a misnomer — but the spirit of the company is that we do by default.

In the end, if there is a small chance of something crazy happening in AI research, everything else being equal, do you want these advances to be made inside a commercial company, especially one that has monopoly on the research, or do you want this to happen within a non-profit?


#3

Interview with OpenAI co-founder and CTO Greg Brockman:

From the perspective of this organization, we view our goal as to advance the state of the art in a way that is positive and ensure that it is something that is good for the world. In the short term a lot of that is about advancing the capabilities and ensuring that everyone has access to those, so we are not going to be patenting our work. In the long term, we think about the question of how to make sure that this happens in the best possible way.

I think that one important thing structurally is that we are a non-profit. We are not operating for financial return. We focus on this one singular goal to do the best thing. That best thing is adaptive. You cannot hope to know what the world will look like once you have self-driving cars, home robots, and adaptive systems that understand a lot more about what we are doing. It is going to be an ongoing debate in society around what is the best way for these things to unfold and how do we make sure that it is positive. We know that there are tradeoffs. Job displacement is a obvious tradeoff and as time goes on there are going to be a lot questions around surveillance and how these technologies are applied by humans. In the long term there will be a question of what are the emergent effects? If you have a capable system, how do you make sure that it is acting in a way that is good for the world? All these questions need to be answered. It is not necessarily the case that we want Open AI to be the one to answer all those questions, but we want to help. We want to be an organization that is clear that we are focused on one thing: the benefit for humanity. Let us make sure that the actions we take are the things that we think will maximize that.

I wonder how they are evaluating potential actions against the objective of maximizing benefit for humanity?

I would say there is not much of a direct difference between the way that I think about the work with Open AI as a nonprofit and the work with a for profit from the perspective of thinking about the long-term and thinking about impact.

I say that the big difference is the question of what happens with the things that we produce. As a company, when you produce new inventions or intellectual property you monetize them, you patent them because you just invested quite a lot in making those happen. Think about any pharmaceutical company. They spend so much money developing all these drugs and that is why they have to sell them for the price that they do. With Open AI, we have been put in this fortunate position where we are able to focus just on developing this technology in the way that seems to be best for humanity, and we do not have to think about bringing in a revenue stream that makes this sustainable. That is not to say that we will not. There is a decent chance that at some point we will make this self-sustaining and build a product. But again, that should all be subject to the question of what do we think will most advance the cause, what do we think will help with the research, will make us sustainable in the long run, and make the kind of impact we want in the world.

The above reinforces for me that they see “open” as a strategy rather than a necessity.

I predict that unless open (as in commons-based production) dominates the field of AI, including such directly used by the masses, WPIO is the outcome.


#4

Inside OpenAI, Elon Musk’s Wild Plan to Set Artificial Intelligence Free has a section titled ‘The Limits of Openness’. A few excerpts:

But for all of OpenAI’s idealism, the researchers may find themselves facing some of the same compromises they had to make at their old jobs. Openness has its limits. And the long-term vision for AI isn’t the only interest in play. OpenAI is not a charity. Musk’s companies that could benefit greatly the startup’s work, and so could many of the companies backed by Altman’s Y Combinator. “There are certainly some competing objectives,” LeCun says. “It’s a non-profit, but then there is a very close link with Y Combinator. And people are paid as if they are working in the industry.”

I wonder whether “not a charity” means that OpenAI is a trade association, despite Musk’s mention of 501©3 in an interview quoted above?

According to Brockman, the lab doesn’t pay the same astronomical salaries that AI researchers are now getting at places like Google and Facebook. But he says the lab does want to “pay them well,” and it’s offering to compensate researchers with stock options, first in Y Combinator and perhaps later in SpaceX (which, unlike Tesla, is still a private company).

A curious practice for a non-profit. To the extent stock options work as incentives, receiving such options would align OpenAI employees with the companies of the options received.

Regarding “astronomical salaries”, earlier the article links to http://www.bloomberg.com/news/articles/2014-01-27/the-race-to-buy-the-human-brains-behind-deep-learning-machines which quotes a Microsoft person as saying “Last year [2013], the cost of a top, world-class deep learning expert was about the same as a top NFL quarterback prospect. The cost of that talent is pretty remarkable.” I searched for what that might mean and found http://www.spotrac.com/nfl/draft/2013/quarterback/ – not sure I understand, but the contracts seem to be for 4 years and have a total value, so it might be reasonable to divide total value by 4 to get yearly compensation – that ranges from $2.2m for the top quarterback pick to $0.5m for the lowest (other recent years, the top quarterback pick seems to have gotten around $5m/year). Is this a curiosity or harbinger of both knowledge (software (AI)) as the commanding heights and yet another data point on winner-take-most for knowledge production (which needs to be mitigated by making the knowledge not subject to property regimes)?

Nonetheless, Brockman insists that OpenAI won’t give special treatment to its sister companies. OpenAI is a research outfit, he says, not a consulting firm. But when pressed, he acknowledges that OpenAI’s idealistic vision has its limits. The company may not open source everything it produces, though it will aim to share most of its research eventually, either through research papers or Internet services. “Doing all your research in the open is not necessarily the best way to go. You want to nurture an idea, see where it goes, and then publish it,” Brockman says. “We will produce lot of open source code. But we will also have a lot of stuff that we are not quite ready to release.”

If the research papers and internet services include complete source code under open licenses, great. Though even then, treating open source as a verb (we’ll-open-source-it-when-its-ready) does not inspire confidence. This is just another indication that “open” is not very foundational to OpenAI.

Both Sutskever and Brockman also add that OpenAI could go so far as to patent some of its work. “We won’t patent anything in the near term,” Brockman says. “But we’re open to changing tactics in the long term, if we find it’s the best thing for the world.” For instance, he says, OpenAI could engage in pre-emptive patenting, a tactic that seeks to prevent others from securing patents.

But to some, patents suggest a profit motive—or at least a weaker commitment to open source than OpenAI’s founders have espoused. “That’s what the patent system is about,” says Oren Etzioni, head of the Allen Institute for Artificial Intelligence. “This makes me wonder where they’re really going.”

It’s possible to come up with a patenting strategy which would promote the commons – it would involve aggressively asserting patents against proprietary entities – but nobody has implemented, and surely OpenAI won’t. Publishing early, often, and preferably continuously, and filing defensive publications is probably the best that could be hoped for, but as above, OpenAI isn’t doing all its work in the open. Potential patenting activity raises additional suspicion, given questionable commitment to openness generally.

There’s additional background story on OpenAI in the linked article, and a recent personal blog post by their CTO at https://blog.gregbrockman.com/my-path-to-openai

There’s a link in the article to Strategic Implications of Openness in AI Development, a working paper by Nick Bostrom; brief comments on earlier Bostrom work.


#5

I’ve not been following OpenAI work, but noticed that a recent decision to not release the full model behind a recent paper has provoked a large backlash https://thegradient.pub/openai-please-open-source-your-language-model/