The important moves in artificial intelligence.

Fine-Tune Wav2Vec2 for English ASR in Hugging Face with 🤗 Transformers

Hugging Face Reads, Feb. 2021 - Long-range Transformers

Multimodal neurons in artificial neural networks

We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.

Simple considerations for simple people building fancy neural networks

Retrieval Augmented Generation with Huggingface Transformers and Ray

Hugging Face on PyTorch / XLA TPUs

Understanding the capabilities, limitations, and societal impact of large language models

Faster TensorFlow models in Hugging Face Transformers

Scaling Kubernetes to 7,500 nodes

We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.

Fit More and Train Faster With ZeRO via DeepSpeed and FairScale

How we sped up transformer inference 100x for 🤗 API customers

DALL·E: Creating images from text

We’ve trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language.