Memory-Efficient Attention Kernels

Questions to ask kernel authors or vendors:

• What hardware and batch configurations achieved the advertised speedups?
• How does performance scale with sequence length and head count?
• Are there known precision or stability caveats?
• How is memory fragmentation handled under multi-tenant loads?
• Is there a roadmap for new architectures (next-gen GPUs, accelerators)?