Inference Performance Engineer
1 week ago
Tel Aviv District, TA, Israel
Stealth AI Startup
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
*
About the Company
* We are an early-stage, still-in-stealth team building an AI platform that gives organizations control of their AI by delivering special models that are faster, more cost-efficient, and more secure than one-size-fits-all alternatives, and that customers can own and run on their own terms. If you are passionate about high-scale infrastructure, hardware-software co-optimization, and shaping the foundation of the modern AI stack, we would love to talk. *Role Overview* As an Inference Performance Engineer , you will own and evolve our core execution platform and LLM serving infrastructure. You will design, deploy, and optimize highly available, resilient multi-cloud GPU and Kubernetes systems. Working closely with systems engineers and research teams, you will tackle bare-metal, network, and framework bottlenecks to keep models serving at maximum throughput and ultra-low latency. *Core Responsibilities*
- Inference & Performance Optimization: Profile, benchmark, and optimize high-throughput, latency-sensitive LLM workloads like vLLM, smart routing layers, and KV-cache orchestration.
- Systems Architecture: Design, build, and operate resilient, k8s-native platforms and custom automation across multi-cloud environments.
- Low-Level & Hardware Efficiency: Locate and eliminate system-level bottlenecks to squeeze maximum performance out of compute and GPU resources.
- Developer Platform & Tooling: Build robust, self-service platform tools to enable rapid experimentation, deployment, and scalability. *Requirements*
- 5+ years of software engineering, systems backend, or performance engineering experience.
- Deep expertise in Kubernetes managing production workloads including CRDs, Helm, Karpenter, affinity/anti-affinity, and custom scheduling.
- Strong background in Performance Engineering with proven experience profiling and squeezing efficiency out of high-throughput systems handling thousands of requests per second.
- Proficiency in Go or Python .
- Hands-on experience with major cloud platforms like AWS or GCP and production observability stacks like Prometheus and Grafana. *Preferred Qualifications*
- Experience with GPU orchestration on Kubernetes using NVIDIA device plugins, DCGM, or GPU scheduling.
- Deep familiarity with LLM inference mechanics and procedures such as prefill/decode phases, TTFT, ITL, and custom engine tweaks.
- Experience with multi-cloud or hybrid-cloud architectures.
About the Company
* We are an early-stage, still-in-stealth team building an AI platform that gives organizations control of their AI by delivering special models that are faster, more cost-efficient, and more secure than one-size-fits-all alternatives, and that customers can own and run on their own terms. If you are passionate about high-scale infrastructure, hardware-software co-optimization, and shaping the foundation of the modern AI stack, we would love to talk. *Role Overview* As an Inference Performance Engineer , you will own and evolve our core execution platform and LLM serving infrastructure. You will design, deploy, and optimize highly available, resilient multi-cloud GPU and Kubernetes systems. Working closely with systems engineers and research teams, you will tackle bare-metal, network, and framework bottlenecks to keep models serving at maximum throughput and ultra-low latency. *Core Responsibilities*
- Inference & Performance Optimization: Profile, benchmark, and optimize high-throughput, latency-sensitive LLM workloads like vLLM, smart routing layers, and KV-cache orchestration.
- Systems Architecture: Design, build, and operate resilient, k8s-native platforms and custom automation across multi-cloud environments.
- Low-Level & Hardware Efficiency: Locate and eliminate system-level bottlenecks to squeeze maximum performance out of compute and GPU resources.
- Developer Platform & Tooling: Build robust, self-service platform tools to enable rapid experimentation, deployment, and scalability. *Requirements*
- 5+ years of software engineering, systems backend, or performance engineering experience.
- Deep expertise in Kubernetes managing production workloads including CRDs, Helm, Karpenter, affinity/anti-affinity, and custom scheduling.
- Strong background in Performance Engineering with proven experience profiling and squeezing efficiency out of high-throughput systems handling thousands of requests per second.
- Proficiency in Go or Python .
- Hands-on experience with major cloud platforms like AWS or GCP and production observability stacks like Prometheus and Grafana. *Preferred Qualifications*
- Experience with GPU orchestration on Kubernetes using NVIDIA device plugins, DCGM, or GPU scheduling.
- Deep familiarity with LLM inference mechanics and procedures such as prefill/decode phases, TTFT, ITL, and custom engine tweaks.
- Experience with multi-cloud or hybrid-cloud architectures.