The important moves in artificial intelligence.

CLIP: Connecting text and images

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of GPT-2 and GPT-3.

Organizational update from OpenAI

It’s been a year of dramatic change and growth at OpenAI.

Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models

Porting fairseq wmt19 translation system to transformers

Hyperparameter Search with Transformers and Ray Tune

Transformer-based Encoder-Decoder Models

OpenAI licenses GPT-3 technology to Microsoft

OpenAI has agreed to license GPT-3 to Microsoft for their own products and services.

Block Sparse Matrices for Smaller and Faster Language Models

Generative language modeling for automated theorem proving

Learning to summarize with human feedback

We’ve applied reinforcement learning from human feedback to train language models that are better at summarization.

OpenAI Scholars 2020: Final projects

Our third class of OpenAI Scholars presented their final projects at virtual Demo Day, showcasing their research results from over the past five months.

The Reformer - Pushing the limits of language modeling