The important moves in artificial intelligence.

Efficient Table Pre-training without Real Data: An Introduction to TAPEX

Accelerated Inference with Optimum and Transformers Pipelines

Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel

Text and code embeddings by contrastive pre-training

Training CodeParrot 🦜 from Scratch

Scaling up BERT-like model Inference on modern CPU - Part 2

Train a Sentence Embedding Model with 1B Training Pairs

Deep Learning over the Internet: Training Language Models Collaboratively

Improving language model behavior by training on a curated dataset

Our latest research finds we can improve language model behavior with respect to specific behavioral values by fine-tuning on a small, curated dataset.

Scaling-up BERT Inference on CPU (Part 1)

Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker

Scaling Kubernetes to 7,500 nodes

We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.