# Machine Learning Engineer - Inference

Source: https://realanalystjobs.com/job/machine-learning-engineer-inference-together-ai-7543b91b
Machine Learning Engineer - Inference at Together AI, an on-site role based in San Francisco, California, USA. Experience level: 3+ Years. Pay stated in the posting: $200,000 to $300,000. Posted on 6 June 2024, read from Together AI's own careers page (Greenhouse), which RealAnalystJobs re-checks daily and removes the role when it closes.

Updated 20 November 2026.

## Key facts

- Company: [Together AI](https://realanalystjobs.com/companies/together-ai)
- Location: San Francisco, California, USA
- Work mode: On-site
- Pay stated in the posting: $200,000 to $300,000
- Posted: Jun 6, 2024
- Employment type: Full time
- Experience level: 3+ Years
- Source: Together AI's careers page (Greenhouse), re-checked daily

Skills: Fine-tuning, LLM, PyTorch, CUDA, [Python](https://realanalystjobs.com/skills/python), Rust

## About this role

About the Role

Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale. If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI!

Responsibilities

• Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.

• Develop and optimize runtime inference services for large-scale AI applications.

• Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.

• Conduct design and code reviews to ensure high standards of quality.

• Create services, tools, and developer documentation to support the inference engine.

• Implement robust and fault-tolerant systems for data ingestion and processing.

Requirements

• 3+ years of experience writing high-performance, well-tested, production-quality code.

• Proficiency with Python and PyTorch.

• Demonstrated experience in building high performance libraries and tooling.

• Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.

• Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum

• Preferred: Knowledge of AI inference techniques such as speculative decoding.

• Preferred: Knowledge of CUDA/Triton programming.

• Nice to have: Knowledge of Rust, Cython and compilers.

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Compensation

We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $200,000 - $300,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Application Policy

Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy

Description from Together AI's public careers feed, reproduced so you can read the role here. Apply on the company's own site; Real Analyst Jobs never submits anything for you.

## How to apply

Open [this role on Real Analyst Jobs](https://realanalystjobs.com/job/machine-learning-engineer-inference-together-ai-7543b91b) and use Apply now there to reach the employer's application. The listing is re-checked against Together AI's careers page daily and removed when it closes.

## Together AI

Analyst roles on Real Analyst Jobs. [More roles at Together AI](https://realanalystjobs.com/companies/together-ai)

## More like this

- [Data analyst jobs in the USA](https://realanalystjobs.com/jobs/data-analyst-jobs-in-usa)
- [Analyst job market report](https://realanalystjobs.com/analyst-job-market)
- [What analyst roles pay](https://realanalystjobs.com/blog/data-analyst-salaries)

## Sources and links

- [This role on Real Analyst Jobs](https://realanalystjobs.com/job/machine-learning-engineer-inference-together-ai-7543b91b)
- [Together AI on Real Analyst Jobs](https://realanalystjobs.com/companies/together-ai)
- [All analyst jobs](https://realanalystjobs.com/jobs)
