Submit Blog

Sign up Sign in

Search Articles

Filter by Tag

Sort By

Popular Tags

Mixtral 8x7b Articles

Page 1 of 1 (2 articles)

Deploy Mixtral 8x7B on AWS Inferentia2 with Hugging Face Optimum

6/18/2024 • EN

Deploy Mixtral 8x7B on AWS Inferentia2 with Hugging Face Optimum

A technical guide on deploying the Mixtral 8x7B LLM on AWS Inferentia2 using Hugging Face Optimum and Amazon SageMaker.

Amazon Sagemaker AWS Inferentia2 Hugging Face Optimum LLM Deployment Mixtral 8x7b

Accelerate Mixtral 8x7B with Speculative Decoding and Quantization on Amazon SageMaker

4/2/2024 • EN

Accelerate Mixtral 8x7B with Speculative Decoding and Quantization on Amazon SageMaker

A technical guide on accelerating the Mixtral 8x7B LLM using speculative decoding (Medusa) and quantization (AWQ) for deployment on Amazon SageMaker.

Amazon Sagemaker LLM Inference Mixtral 8x7b Quantization Speculative Decoding