In early July, AWS raised EC2 Capacity Block prices 20% across its main GPU instance families. That is the second hike in six months, following a 15% increase on January 4. If you are on H100-based P5 instances, you are now paying $5.19 per GPU hour. B300 slots hit $14.04 per GPU hour. For a team running a p5e.48xlarge continuously, the January hike alone pushed the bill from $34.61 to $39.80 per hour. The July hike adds another layer on top of that.
This matters because the compute story in 2026 has been running in two directions, and those directions are diverging fast.
API Token Costs Fell. Reserved GPU Costs Did Not.
If you are calling inference APIs, you are paying a fraction of what you paid two years ago. Per-token prices have dropped roughly 80% in 2026 alone and around 280-fold since 2022. Claude, Gemini Flash, and the latest DeepSeek models are all priced in a range that would have seemed impossible eighteen months ago.
But that is API inference. If you are reserving GPU capacity for training runs, fine-tuning, or heavy batch processing, you are in a different market. That market is going up. Demand for training capacity has not softened. Amazon committed roughly $200 billion in AI infrastructure capex this year to keep pace. They are not subsidizing that from goodwill.
The mistake I see teams make is treating these two markets as one question: "should we use cloud GPUs or APIs?" That framing is too coarse. The real question is what kind of compute you are actually buying, and which market it belongs to. AWS announced the July 1 increase in late June, affecting P6-B300, P6-B200, P5, P5e, P5en, and P4de families. If your team missed that, check your bill.
Training and Fine-Tuning: You Are on the Capacity Block Side
Anything that involves gradient updates lives in the capacity block world. You need reserved GPU time in large blocks, and the price just went up again.
A lot of teams I talk to are revisiting whether the fine-tuning step is necessary at all, or whether a well-crafted system prompt with retrieval does 90% of the job. In many cases, it does. Before you book more capacity blocks, run that experiment. The gap between "fine-tuned model" and "strong base model with good retrieval and a detailed prompt" has narrowed, and the compute cost of closing it has widened.
For smaller models (under 13B parameters), the math on fine-tuning has also shifted. With LORA and QLORA, you can fine-tune on a much smaller GPU footprint. The new G7e instances (NVIDIA RTX PRO 6000 Blackwell, 96 GB GDDR7) fit a 70B-parameter model in FP8 on a single card, and they are priced as inference-tier compute, not training-tier. That is worth knowing if your fine-tuning jobs are small enough to run on one GPU.
Inference: The Case for APIs Has Never Been Stronger
If you are doing inference rather than training, the calculus has flipped. AWS launched G7 instances in US East on July 10, powered by NVIDIA RTX PRO 4500 Blackwell GPUs, claiming 4.6x AI inference throughput over G6. Benchmarks for Llama 3.1 8B on G7 come in around $0.70 per million tokens. That is competitive with managed APIs for mid-size models.
But that $0.70 is the compute cost only. It does not include the ops burden of running your own vLLM cluster, handling model updates, managing batching configurations, monitoring GPU health, or the ML engineer hours it takes to keep that running. Below roughly 50 million tokens per day of sustained load, a managed API almost always wins once you add those real costs. I have not run this at every possible scale, but the pattern holds across every team I have seen try it.
Unless you have a specific reason to own inference compute (strict data residency requirements, extremely low latency needs at very high volume, or a model that simply is not available via any API), default to managed APIs and let the price competition work in your favor.
The Practical Adjustment for 2026
A few things worth changing in how your team thinks about this:
- Stop bundling training cost and inference cost into one "AI compute" budget line. They are in different markets with opposite price trajectories.
- Audit your capacity block reservations. If you are holding reserved GPU time for workloads that run less than 50% of the time, you are probably overpaying significantly.
- Before your next fine-tuning run, benchmark whether full fine-tuning actually beats retrieval-augmented generation plus good prompting for your specific use case. The compute cost gap has widened enough that the answer is less obvious than it was twelve months ago.
- If you are self-hosting inference today, benchmark your real per-token cost (including ops overhead) against current API prices. The gap has closed considerably since early 2025.
AWS will almost certainly raise capacity block prices again. The demand side has not cooled and $200 billion in capex has to get paid back. The teams that come out ahead are the ones who stop treating all GPU spend as one category and start routing workloads to the market that actually fits them.