Skip to content
Kong

Senior Software Engineer, AI Gateway - Shanghai

Shanghai, ChinaOnsite · China
Senior · FullTime · All Cost Center
POSTED 23 OCTVERIFIED 10H AGO
Apply Ask
Role
24 days opentrue age
Closes in ~3d at the median (27d)
Posted 23 Oct · n=21
CompanyKong →
116open roles
Opened · 30d
+27
Closed · 30d
−17
Signal
● Steady+10 net
14d of snapshots · 30d window
Market
17,135open · Senior Software engineering
Opened · 7d
+1,4068% of open
All software engineering
75,371open
All tracked boards · live 16 Nov · n=17,135
Visa
8H-1B LCAs · FY26
Median wage
$206k
LCAs FY26 · n=8 · to 2026-06-16
RustLLMsDistributed systems

Are you ready to unlock intelligence?

If you don’t think you meet all of the criteria below but are still interested in the job, please apply. Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others.

About the Role

As Kong advances into the Agentic Era, the AI Gateway team is developing the vital infrastructure that links enterprise systems with generative AI technologies. In this position, you will focus on engineering the core system powering live AI traffic. You will take ownership of the runtime and data plane responsible for routing, translating, securing, and analyzing every request, ensuring performance, security, and scalability as AI workloads expand.

What You'll Do

Responsible for designing, building and maintaining Kong’s AI Gateway runtime and its underlying data plane stack.

Architecture, design, and implement new innovative features including proxying capabilities, semantic/intelligent routing, load balancing, guardrails and compression, and identity, that extend Kong’s AI Gateway offering.

Build and maintain security, performance, and compliance foundations, including token cost management, access control, and low-level optimizations for production AI workloads.

Engage with the open source community and extend Kong core to support the latest industry trends.

Fill the gaps between Kong AI Gateway and third-party AI/ML providers, deployment platforms, and tooling.

And any additional tasks required by the manager.

What You'll Bring

8+ years of programming in server-side applications and components.

Experience designing, prototyping, building and debugging applications that are highly scalable, distributed, reliable and resilient.

Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling.

Experience in distributed systems, Linux networking stack and computer systems programming, Kong works with APIs on both L7 and L4 transports and protocols, so this expertise and knowledge are required.

Comfortable working with a remotely distributed team.

Strong sense of ownership and excellent verbal and written communication skills.

Familiarity with AI LLM concepts and applications development.

Bonus Points

Hands-on experience with NGINX/OpenResty/Kong/LuaJIT.

Hands-on experience with Envoy or Istio.

Hands-on experience with Rust.

Hands-on experience leveraging AI agents to deliver intricate software systems efficiently and with superior quality.

#LI-CL1

About Kong

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Trusted by the Fortune 500® and AI-native startups alike, Kong’s unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic — on any model, any cloud. For more information, visit www.konghq.com.

Source · KongFirst seen 2 NovOriginal ↗