About this role
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Performance Modeling Engineer, AI Inference Deployments (Senior / Principal) based in the United States.
In this role, you will develop advanced performance models and deployment-planning tools that help customers understand how large-scale AI inference workloads will perform on next-generation hardware.
You will work at the intersection of silicon architecture, systems engineering, and customer solutions to translate complex deployment requirements into accurate, actionable performance estimates.
Your work will cover everything from chip-level simulation to large-scale deployment configurations, including throughput, latency, scalability, and hardware requirements.
You will collaborate directly with customers, product teams, and chip modeling engineers to evaluate deployment strategies and identify performance optimization opportunities.
A key part of your mission will be developing analytical models and customer-facing tools that enable organizations to independently estimate and plan their AI infrastructure needs.
You will also validate predictions against real hardware measurements, ensuring that models remain accurate as hardware, software, and workloads evolve.
This is an opportunity to influence major AI infrastructure decisions and help shape how frontier-scale inference deployments are designed, evaluated, and scaled.
Accountabilities:
• Partner with customers and product teams to translate deployment requirements into structured performance studies, defining assumptions, scenarios, metrics, and actionable recommendations.
• Conduct objective, apples-to-apples performance comparisons between competing hardware platforms, including GPUs and other AI accelerators, identifying the factors driving differences in performance.
• Design, develop, and deliver a standalone performance-estimation tool that customers can use to evaluate their own AI inference workloads and deployment configurations.
• Build and maintain deployment-level performance models for large-scale inference systems, accounting for tensor, pipeline, expert, and data parallelism strategies.
• Evaluate the impact of batch sizes, context lengths, workload characteristics, and mixed hardware configurations involving different accelerator platforms.
• Model network and system-level performance, including interconnect latency, bandwidth, topology, and redundancy, to understand how these factors influence throughput and scalability as the number of chips increases.
• Develop and extend analytical and simulation-based deployment models that leverage existing chip performance simulators, ensuring that all deployment estimates remain consistent with a shared hardware performance model.
• Analyze system configurations and identify performance bottlenecks, scalability limitations, and opportunities to optimize deployment efficiency.
• Validate performance projections against actual hardware measurements and benchmark results, continuously calibrating models as hardware architectures, software environments, and workloads evolve.
• Collaborate closely with chip modeling and engineering teams to incorporate new hardware capabilities and improve the accuracy of deployment predictions.
• Communicate technical findings clearly to customers and internal stakeholders, explaining performance estimates, key assumptions, influencing factors, and confidence levels.
• Depending on experience and seniority, take ownership of either the overall modeling architecture or individual model deployment studies and the development of customer-facing performance tools.
Requirements:
• At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
• Demonstrated experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
• Deep understanding of large language model (LLM) inference at the systems level, including prefill and decode phases, KV cache management, parallelism strategies, mixture-of-experts (MoE) routing, batching, time-to-first-token (TTFT), and tokens-per-second performance metrics.
• Strong knowledge of large-scale deployment architectures and the factors affecting inference throughput, latency, hardware utilization, and scalability.
• Solid understanding of data-center networking concepts, including interconnect latency, bandwidth, network topology, and their impact on distributed system performance.
• Good knowledge of GPU architecture and performance characteristics, with the ability to evaluate and compare different accelerator platforms.
• Experience building analytical models, conducting performance experiments, interpreting benchmark results, and translating technical findings into practical deployment recommendations.
• Ability to work with incomplete technical information, establish and document assumptions, and deliver useful performance estimates with clearly communicated uncertainty and error margins.
• Strong programming skills in Rust, or expert-level C/C++ proficiency combined with a willingness to work extensively in Rust.
• Proficiency in Python for performance analysis, experimentation, data processing, and visualization.
• Excellent analytical and problem-solving abilities, with a strong understanding of the relationship between hardware architecture, workload characteristics, and system-level performance.
• Excellent written and verbal communication skills, including the ability to explain deployment sizing recommendations, performance drivers, and confidence levels to both technical teams and customers.
• Hands-on experience sizing, benchmarking, or optimizing production inference deployments across multiple accelerators
Description from Jobgether's public careers feed, reproduced so you can read the role here. Apply on the company's own site; RealAnalystJobs never submits anything for you.
