Skip to content
AI Lehel Briefing
← Back to latest
Infrastructure AWS Machine Learning

Mixture-of-Experts kernel engineering on Amazon SageMaker AI

AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.

Excerpt supplied by the publisher’s feed

Read the original article at AWS Machine Learning Opens in a new tab

More in Infrastructure

Infrastructure AWS Machine Learning

Manage private pricing offers for Amazon Bedrock with managed entitlements

Managing AI model access across dozens or hundreds of AWS accounts creates friction when each account must accept AWS Marketplace Private Pricing Offers individually. This post shows two ways to activate MPPO entitlements in AWS License Manager: a lightweight script for a one-time fix, and CDK automation for ongoing event-driven processing.

Read brief Original at AWS Machine Learning (opens in a new tab) ↗
Infrastructure AWS Machine Learning

How ShipStation Global built an AI IT help desk on Amazon Bedrock AgentCore

ShipStation Global's engineering team built Deckhand, an AI IT help desk agent for Slack on Amazon Bedrock AgentCore. It answers routine IT questions from a curated knowledge base, reads error-message screenshots, and files Jira tickets. In its first three months it resolved 97.5% of channel queries with a 96% positive feedback rating.

Read brief Original at AWS Machine Learning (opens in a new tab) ↗