AI Engineer
- Benchmark IT Solutions
- Pune, India
- INR 1,800,000 – INR 3,000,000
The role
You will build GenAI-powered features that ship to real users: retrieval-augmented generation (RAG) services, structured data extraction from documents, LLM-backed APIs, and agent-style workflows with human review. You will also write the ordinary engineering that surrounds a model — validation, APIs, data access, tests and monitoring — because that is what makes an AI feature dependable. You will work in a team with a Technical Lead who reviews your designs and code.
This is a hands-on engineering role, not a research or prompt-writing role.
What you will do
- Build RAG pipelines end to end: document ingestion, chunking, embedding, vector search, re-ranking, grounded answers with citations.
- Implement structured extraction from documents and semi-structured text into validated JSON, with confidence handling and fallback.
- Design and consume REST APIs around LLM features; integrate with existing applications and workflow pipelines.
- Implement tool calling and Agentic workflows with clear guardrails and human-in-the-loop steps where needed.
- Write evaluation sets and tests for LLM features — measure retrieval quality and answer accuracy, and catch regressions before release.
- Track cost, latency and failures in production; debug why a request went wrong.
- Follow good engineering practice: version control, code review, secrets kept out of code, readable and tested code.
- Use AI coding assistants productively while remaining accountable for what you commit.
What you must bring
- 1–2 years of hands-on experience building LLM or GenAI applications, with at least one feature that real users have used — not only a notebook or a hackathon demo.
- Strong Python fundamentals; comfortable with FastAPI, Flask or Django for building APIs.
- Experience with at least one LLM API (OpenAI, Azure OpenAI, Anthropic, Gemini or open-weight models) and at least one vector store (pgvector, Qdrant, Pinecone, Chroma, Azure AI Search, MongoDB Atlas Vector Search, FAISS).
- Working understanding of RAG concepts and failure modes: chunking trade-offs, retrieval misses, hallucination, prompt injection, context limits.
- Structured outputs: function or tool calling, JSON schema, Pydantic validation.
- Git, basic Docker, and the habit of writing tests.
- Ability to explain, with reasons, when a rule or a simple lookup is a better choice than an LLM.
Good to have
- LangChain, LangGraph, LlamaIndex, Semantic Kernel or a similar framework — and the judgment to use them sparingly.
- OCR and document AI tools (Azure Document Intelligence, Tesseract, PaddleOCR).
- Evaluation tooling (Ragas, DeepEval, promptfoo) or a harness you built yourself.
- Cloud experience, Azure preferred; workflow orchestrators (Temporal, Airflow, Prefect).
- Exposure to enterprise or regulated-industry projects; any of AI-102, AZ-204 or equivalent certifications.
Skills
- Python
- RAG
- LLM
- Vector Databases
- REST APIs
- Prompt Engineering
- Testing



