⭐ Repository → 💼 AWS Well-Architected → 💼 Sustainability → 💼 Hardware and services

💼 SUS05-BP04 Optimize your use of hardware-based compute accelerators

• ID: /frameworks/aws-well-architected/sustainability/hardware-and-services/bp04

Description

Optimize your use of accelerated computing instances to reduce the physical infrastructure demands of your workload.

Common anti-patterns

• You are not monitoring GPU usage.
• You are using a general-purpose instance for workload while a purpose-built instance can deliver higher performance, lower cost, and better performance per watt.
• You are using hardware-based compute accelerators for tasks where they’re more efficient using CPU-based alternatives.

Benefits of establishing this best practice

By optimizing the use of hardware-based accelerators, you can reduce the physical-infrastructure demands of your workload.

Level of risk exposed if this best practice is not established: Medium

Implementation guidance

If you require high processing capability, you can benefit from using accelerated computing instances, which provide access to hardware-based compute accelerators such as graphics processing units (GPUs) and field programmable gate arrays (FPGAs). These hardware accelerators perform certain functions like graphic processing or data pattern matching more efficiently than CPU-based alternatives. Many accelerated workloads, such as rendering, transcoding, and machine learning, are highly variable in terms of resource usage. Only run this hardware for the time needed, and decommission them with automation when not required to minimize resources consumed.

Implementation steps

1. Explore compute accelerators: Identify which accelerated computing instances can address your requirements.

2. Use purpose-built hardware: For machine learning workloads, take advantage of purpose-built hardware that is specific to your workload, such as AWS Trainium, AWS Inferentia, and Amazon EC2 DL1. AWS Inferentia instances such as Inf2 instances offer up to 50% better performance per watt over comparable Amazon EC2 instances.

3. Monitor usage metrics: Collect usage metrics for your accelerated computing instances. For example, you can use CloudWatch agent to collect metrics such as utilization_gpu and utilization_memory for your GPUs as shown in Collect NVIDIA GPU metrics with Amazon CloudWatch.

4. Rightsize: Optimize the code, network operation, and settings of hardware accelerators to make sure that underlying hardware is fully utilized.

   • Optimize GPU settings
   • GPU Monitoring and Optimization in the Deep Learning AMI
   • Optimizing I/O for GPU performance tuning of deep learning training in Amazon SageMaker AI

5. Keep up to date: Use the latest high performant libraries and GPU drivers.

6. Release unneeded instances: Use automation to release GPU instances when not in use.

Similar

Sub Sections

Section	Sub Sections	Internal Rules	Policies	Flags	Compliance