Skip to content
Artificialanalysis

Forward Deployed Engineer - Language Models

San Francisco (On-site); Melbourne (On-site)Onsite · United States
FullTime · Platform & Forward Deployed Engineering
POSTED 29 JUL
Apply Ask
Role
116 days opentrue age
Posted 29 Jul
Pay
$160k–$240kno pay posted
Market p25–p75
$160k–$240k
SF Bay Area · n=2331 · 365d to 19 Nov
CompanyArtificialanalysis →
20open roles
Opened · 30d
+2
Closed · 30d
−0
Signal
● Steady+2 net
19d of snapshots · 30d window
Market
37,577open · Software engineering
Opened · 7d
+3,64510% of open
All software engineering
75,020open
All tracked boards · live 22 Nov · n=37,577
Python

Job Description – Forward Deployed Engineer – Language Models

Location: San Francisco, CA (on-site, 5 days a week, in our San Francisco office)

Or

Melbourne, Australia (on-site, 3 days a week, in our Melbourne office)

About Artificial Analysis

Artificial Analysis is the leading independent AI benchmarking company. We support labs, engineers and enterprises to understand AI capabilities and make critical decisions about their AI strategies. We are the go-to authority for understanding AI, from AI labs and enterprises to media, investors, and policymakers. Our benchmarks don’t just measure the cutting edge of AI, they are actively shaping the frontier.

Our benchmarks and analysis are trusted by hundreds of thousands of users and are the go-to reference for leading AI labs including OpenAI, Google, Meta, NVIDIA and Anthropic, and major publications including the Wall Street Journal, Bloomberg, the Financial Times and The Economist.

We're a fast-growing team backed by Nat Friedman (GitHub, Meta), Daniel Gross (SSI, Meta), Andrew Ng (Google Brain, DeepLearning.ai, Amazon), Adam D’Angelo (Quora, Poe, OpenAI), Clem Delangue (Hugging Face) and other industry leaders.

The Opportunity

Artificial Analysis maintains one of the most comprehensive language model benchmarking suites in the industry. The leading AI labs use our results to understand how their models perform, where they fall short, and how they compare to the frontier.

We’re hiring a Forward Deployed Engineer to own technical delivery of that work and be the expert our lab partners turn to. You’ll take new models from early access through to validated, published results and investigate what is driving their performance. You’ll work directly with research and engineering teams at the labs as their technical point of contact, on everything from endpoint issues to methodology to what a result means.

Doing this well means becoming an expert in language model evaluation, and we’ll expect you to. You’ll work closely with our eval research team on benchmarks, how results are communicated, and the future roadmap, and you’ll bring what you learn in the field back into our platform and the wider business. The role suits an engineer who wants to own outcomes end-to-end, go deep on how frontier models behave, and help shape what the industry measures next.

What You’ll Do

Own technical delivery across multiple AI lab partnerships, from first access to a new model through to published results

Act as the expert technical point of contact for research and engineering teams at the leading labs, building trusted relationships with their technical leads

Work closely with the eval research team on benchmarks, communication, and roadmap, feeding back what you hear from labs

Investigate and monitor benchmarking results, guiding partners on what drives a score and where their models are strong or weak

Build deep expertise in language model evals and contribute it elsewhere across the business. This includes contributing to our Python-based benchmarking platform, shipping fixes and building reusable tooling, agents and playbooks

Stay current with new model releases, API changes, and developments across the language model ecosystem

What We’re Looking For

Required

3+ years of experience in a client-facing technical role - solutions engineering, support engineering, technical consulting, or similar (companies like Stripe, Vercel, Cloudflare, Datadog, Palantir, Accenture, or comparable)

Strong Python proficiency and comfort working with complex codebases you didn’t write

Hands-on experience working with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.)

Excellent debugging skills; you can trace issues across APIs, data pipelines, and code

Strong written and verbal English communication skills, with the ability to explain technical concepts and results clearly to research and engineering teams

Highly responsive and reliable; you take ownership of partner issues and follow through

You want to own outcomes end-to-end, and you take pride in delivering results that are accurate, reliable and on time

Source · ArtificialanalysisFirst seen 4 Nov1 editsOriginal ↗