💼

SDE2 - API Platform at Sarvam

NEW
Location
Bengaluru East, Karnataka, India
Job type
Full time
Posted 1 day ago
Description

About The Role


Sarvam's model APIs (LLM, ASR, TTS, Vision, Translate) are how developers and enterprises ship products on our foundation models. Our OpenAI-compatible LLM APIs route every request across multiple model deployments, with streaming, failover, metering, billing, and rate-limiting all part of the same stack. Today it's FastAPI and Python; we're in the middle of rewriting it in Go.


We're hiring a Software Engineer (SDE 2) to own core parts of this platform. The features. The reliability. The performance. This is an IC role.


What You'll Do


  • Building and evolving the LLM API platform, from the moment a request hits the edge to the response that goes back out
  • The Python→Go rewrite: building services in Go alongside the existing Python stack, without compromising reliability
  • Streaming infrastructure: SSE for Chat Completions and Responses, long-lived connections, clean handling of cancellations and client disconnects
  • Routing and failover: weighted provider chains, retries with backoff, and latency-driven admission control across model deployments
  • Protocol compatibility: keeping the API faithful to the OpenAI spec across inference servers (vLLM, SGLang, Dynamo) that each behave slightly differently
  • The commercial layer: metering, billing, and rate-limiting, engineered to the same bar as the model APIs themselves
  • Performance and reliability for a multi-tenant platform serving long-context agentic and coding workloads
  • Observability (logging, metrics, tracing), so the team can debug production confidently and quickly
  • Integration and load testing: the harness that lets us ship fast without breaking customer-facing APIs
  • Close partnership with the Inference and MLOps teams on production readiness for new model launches
  • Design-first, documentation-first engineering culture within the team: RFCs before code, decisions written down, no tribal knowledge


What We're Looking For


  • 3+ years building production backend systems
  • Strong Python in production: asyncio, FastAPI or similar, and services you've built and operated under real load
  • Working knowledge of Go, or a strong willingness to learn it: you'll work in both languages through the rewrite
  • Experience building low-latency, multi-tenant backend services
  • Hands-on experience with streaming systems: SSE or WebSockets, long-lived connections, backpressure
  • Strong PostgreSQL and Redis fundamentals: schema design, query performance, caching, and rate limiting
  • Comfort running services on Kubernetes in production
  • An ownership mindset: you test before you ship, debug production with metrics, logs, and traces, and write the RCA afterwards


Bonus Points


  • Experience serving or integrating LLMs in production, especially OpenAI-compatible APIs
  • Familiarity with inference servers (vLLM, SGLang, Dynamo) and latency metrics such as time to first token and inter-token latency
  • Experience building metering, billing, or usage-based pricing infrastructure
  • Time at an early- or growth-stage startup: comfort with ambiguity, speed, and ownership

Why Sarvam?


Sarvam is a fast-moving, high talent-density team building full-stack AI for India, working on problems that push the frontiers of AI with real population-scale impact.


  • Work alongside researchers, engineers, builders, and business leaders who move fast and hold each other to a very high bar
  • High ownership and high impact, from day one
  • Everything we do is AI-first, from the way we build and ship to the way we think about problems
  • You can work on problems that could change how an entire country learns, works, and communicates
101-500

‘Apply now’ takes you to the job application page after a quick sign-in

Master Backend Engineering
AI-First Software Engineering
Next batch starts on 07 Nov
Always stay in the know!
Join our community and stay notified of latest job opportunities.
101-500

‘Apply now’ takes you to the job application page after a quick sign-in

Tap to Apply

Frequently Asked Questions

What is the SDE2 - API Platform at Sarvam role?
Chevron down
Where is the SDE2 - API Platform at Sarvam job located?
Chevron down
Is the SDE2 - API Platform at Sarvam position full-time?
Chevron down
What experience is required for SDE2 - API Platform at Sarvam?
Chevron down
What is the salary for SDE2 - API Platform at Sarvam?
Chevron down
How can I apply for SDE2 - API Platform at Sarvam?
Chevron down

Why work at Sarvam

The SDE2 - API Platform at Sarvam role may appeal to software engineers who want to work on the systems that make products and services accessible through reliable APIs. An API platform is more than a collection of endpoints: it can provide the foundation for authentication, request handling, integrations, monitoring, and consistent developer experiences. Working on this layer gives an engineer the chance to solve problems that can affect multiple products and teams.

Sarvam’s engineering opportunity is based in Bengaluru East, Karnataka, and is listed as a full-time position. For a candidate interested in platform engineering, this location offers access to Bengaluru’s broad technology community while providing an opportunity to contribute to a specific engineering area: API infrastructure.

A platform role can offer varied technical challenges. Depending on the team’s needs, those challenges may involve designing interfaces, improving service reliability, handling traffic safely, integrating with internal systems, or making APIs easier for developers to use. The job listing does not specify the exact technologies, product scope, or team practices, so candidates should confirm these details during the hiring process. Still, the SDE2 level suggests that applicants should be ready to discuss how they have taken ownership of engineering work, made sound implementation decisions, and collaborated with others to deliver maintainable software.

Another reason to consider the SDE2 - API Platform at Sarvam position is the opportunity to build skills that transfer across many areas of software engineering. Experience with API design, distributed systems, data handling, testing, observability, and production operations is useful in product companies of many sizes. Engineers who enjoy balancing technical quality with practical delivery may find platform work particularly engaging.

What’s it like to work at Sarvam

The available listing identifies Sarvam as the employer and describes the position as full-time, in the Engineering department, and based in Bengaluru East. It does not provide detailed information about work culture, benefits, reporting structure, or day-to-day team rituals. Candidates should use the interview process to learn how the engineering team works rather than assume that every company follows the same approach.

When evaluating a role at Sarvam, ask how platform engineers partner with product and application teams. API platform work often involves understanding the needs of several users: developers who build on the platform, teams that operate services, and customers who depend on predictable behavior. A strong working environment for this kind of role typically encourages clear technical communication, careful review of changes, and shared responsibility for reliability. These are useful areas to explore with the hiring team.

You can also ask how the team decides what to build, how it balances new capabilities with operational work, and how engineers receive feedback. For a platform team, questions about service ownership, incident response, documentation, and engineering standards can help you understand the real working experience. If the role involves supporting production APIs, it is useful to learn what monitoring and escalation processes are in place.

The SDE2 - API Platform at Sarvam opportunity is based in Bengaluru East, India. Before applying or proceeding through interviews, confirm the expected work arrangement, team location, and any role-specific requirements directly with the employer. The listing does not include compensation details or a minimum experience requirement, so those are also appropriate topics to clarify with the recruiter.

What’s it like to work as an SDE2 - API Platform at Sarvam

An SDE2 working on an API platform would generally be expected to contribute beyond writing isolated pieces of code. The work may include understanding a requirement, identifying the right interface or service boundary, implementing a solution, testing it, and considering how it behaves after deployment. The precise responsibilities for this Sarvam position are not included in the listing, so the points below describe common aspects of API platform engineering rather than confirmed duties.

API design is often central to the role. Engineers need to think about how clients will use an endpoint, what inputs and outputs are appropriate, how errors should be represented, and how changes can be introduced without unexpectedly breaking existing integrations. Good API design also considers documentation and consistency: developers should be able to understand what an interface does and how to use it safely.

Platform engineers may also work on reliability and performance. That can involve examining latency, handling retries and timeouts, preventing duplicate operations, and understanding how services behave when a dependency is unavailable. Security is another important part of API work. Authentication, authorization, input validation, rate limiting, and protection of sensitive data should be considered as part of system design, not added as an afterthought.

At the SDE2 level, interviewers commonly look for independent execution alongside sound collaboration. You may be expected to explain trade-offs, seek feedback, write tests, participate in code reviews, and communicate risks. A strong candidate can describe not only what they built, but why they chose a particular design and how they measured or verified its results.

The day-to-day experience can vary depending on the team’s roadmap. It may include feature development, design discussions, debugging, testing, documentation, and operational improvements. During the process, ask the hiring team which of these areas are most important for the SDE2 - API Platform at Sarvam role.

SDE2 - API Platform interview questions at Sarvam

The questions below are examples for preparation. They are not confirmed Sarvam interview questions, and the exact process may vary by team and candidate. Use them to practise explaining your reasoning clearly.

API design and backend fundamentals

  1. How would you design an API endpoint for creating and retrieving a resource?
  2. What makes an API backward compatible? How would you introduce a breaking change?
  3. How would you decide between REST and another communication approach for a particular service?
  4. How should an API represent validation errors and unexpected server errors?
  5. What is idempotency, and how would you support it for a request that might be retried?
  6. How would you implement pagination for a large collection?
  7. What considerations matter when designing authentication and authorization for an API?
  8. How would you prevent one client from consuming an unfair share of a service’s capacity?

System design and reliability

  1. Design a service that receives requests from multiple clients and routes them to backend services.
  2. How would you approach rate limiting when the system runs across several application instances?
  3. What would you do if a downstream service became slow or unavailable?
  4. How would you diagnose an increase in API latency?
  5. What metrics, logs, and traces would you use to monitor a production API?
  6. How would you make a long-running operation available through an API?
  7. How would you handle retries without creating duplicate work?
  8. How might you roll out a change safely when many clients depend on an endpoint?

Coding and engineering practices

  1. Write code to validate a request and return useful error information.
  2. How would you test a service that depends on an external API?
  3. What is your approach to code reviews?
  4. Describe a bug you investigated and how you narrowed down its cause.
  5. How do you decide whether to refactor existing code while implementing a feature?
  6. Explain a technical trade-off you made in a recent project.

Collaboration and ownership

  1. Tell us about a project you owned from initial requirements through delivery.
  2. Describe a time you disagreed with a proposed technical design. How did you handle it?
  3. How do you communicate delivery risks or changing estimates?
  4. What do you do when requirements are unclear?
  5. How have you improved a system after observing a production issue?

For each answer, use a clear structure: explain the context, outline the constraints, describe your decision, and share the result. If a question involves a hypothetical design, state your assumptions before presenting a solution.

SDE2 - API Platform interview preparation at Sarvam

Begin by reviewing your own backend and platform experience. Prepare two or three examples that demonstrate technical ownership: a service you designed or improved, a difficult reliability issue you diagnosed, and a project where you coordinated with other engineers. Be specific about your contribution. Explain the problem, the options you considered, the implementation, and what you learned. Avoid presenting team outcomes as individual achievements unless you can clearly distinguish your role.

Refresh core API concepts, including HTTP methods and status codes, request validation, pagination, versioning, idempotency, authentication, authorization, and error handling. Be ready to discuss how clients experience an API, not just how the server implements it. For example, consider what happens when a request is duplicated, a client times out, or a response format needs to evolve.

For system design practice, focus on dependable services. Think through request flow, service boundaries, storage choices, scaling, failure handling, observability, and security. A good design discussion is not just a diagram: explain the assumptions behind your choices and identify where you would add safeguards. Consider how you would detect and investigate errors after launch.

Practise coding in the language you are most comfortable using, especially tasks involving data structures, validation, concurrency, and clean error handling. Write readable code and explain edge cases as you work. If you do not know a detail, state what you would verify rather than guessing.

Finally, prepare questions for the hiring team. Ask about the API platform’s users, the responsibilities of the role, the team’s current engineering priorities, and how success is evaluated for an SDE2. Since the listing does not describe the interview format, you can also ask about the stages, expected technical areas, and whether there will be a coding or system design round.

SDE2 - API Platform interview tips at Sarvam

  • Clarify assumptions first. In a system design question, establish expected traffic, clients, security needs, and reliability requirements before proposing a solution.
  • Explain trade-offs. There may be several valid approaches. Compare them in terms of complexity, reliability, performance, and maintainability.
  • Think about API consumers. Discuss clear contracts, consistent errors, documentation, and compatibility—not only server-side implementation.
  • Include failure scenarios. Consider timeouts, retries, unavailable dependencies, invalid input, and partial failures.
  • Connect design to operations. Mention how you would monitor the service, investigate problems, and introduce changes safely.
  • Use concrete examples. When answering behavioral questions, describe the situation, your actions, and the outcome without overstating your ownership.
  • Be transparent about experience. If you have not used a particular tool or architecture, explain related experience and how you would approach learning it.
  • Confirm role details. The listing does not specify the technology stack, minimum experience, compensation, or interview stages. Ask the recruiter or interviewer for current information.

Thoughtful preparation can help you show the skills relevant to the SDE2 - API Platform at Sarvam role: backend engineering fundamentals, careful API design, practical system thinking, and the ability to take responsibility for delivering dependable software.