Vernacular.ai Prompt Engineer Interview: Rounds, Questions and Preparation

What the job involves

A prompt engineer in an AI and deeptech company designs, tests, and refines prompts and evaluation pipelines that make large language models and speech models produce reliable, safe, and useful outputs. Day to day you work with product and ML teams to translate user needs into prompt templates, run A/B tests, measure quality with human and automated evals, and iterate on guardrails for Indian language and voice use cases.

Role and technical knowledge

1. 1

   What is prompt engineering, and how does it differ from fine-tuning a model?

   Explain that prompting guides a frozen model with instructions and examples, while fine-tuning updates weights. Mention cost, latency, and data needs as trade-offs.

2. 2

   How would you design a prompt for a customer support bot that handles Hindi, English, and Hinglish queries?

   Cover language detection or explicit language instruction, few-shot examples in each language, and fallback behaviour for mixed scripts.

3. 3

   What metrics would you use to evaluate a prompt for a voice assistant in a noisy Indian environment?

   Include word error rate for ASR, intent accuracy, task completion rate, and human ratings for naturalness and cultural fit.

4. 4

   What are common failure modes of LLMs, and how can prompts reduce them?

   Discuss hallucination, bias, prompt injection, and verbosity. Suggest constraints, role setting, output schemas, and refusal instructions.

Practical scenarios

1. 5

   A prompt works well in testing but fails on 30 percent of real user queries. How do you debug it?

   Propose sampling failures, clustering error types, checking for distribution shift, and adding targeted examples or guardrails.

2. 6

   You need to reduce the cost of an LLM-powered feature by half without hurting quality. What do you try?

   Suggest shorter prompts, caching, smaller models for easy cases, routing, and measuring quality before and after each change.

3. 7

   How would you build a golden dataset for evaluating prompts in a low-resource Indian language?

   Mention sourcing real user utterances, native speaker annotation, clear guidelines, inter-annotator agreement, and keeping a held-out test set.

4. 8

   A stakeholder wants a prompt that never says 'I don't know'. How do you respond?

   Explain the risk of hallucination, propose a safe fallback like asking a clarifying question or escalating, and show evaluation data.

Behavioural and teamwork

1. 9

   Tell me about a time you had to learn a new tool or concept quickly for a project.

   Use STAR format. Show curiosity, a concrete learning method, and how you applied it to deliver a result.

2. 10

   Describe a situation where you disagreed with a teammate on the best approach.

   Focus on listening, using data or a small experiment to decide, and maintaining respect even if your idea was not chosen.

3. 11

   How do you handle feedback on your work, especially when it is critical?

   Show that you separate ego from work, ask clarifying questions, and follow up with visible improvements.

4. 12

   Why do you want to work on Indian language AI and voice technology?

   Connect personal or academic exposure to India's language diversity, and show awareness of the social and business impact.

HR and career fit

1. 13

   What do you know about our company and the problems we solve?

   Mention the sector (AI and deeptech), the focus on vernacular or voice AI, and one or two product use cases without claiming insider knowledge.

2. 14

   Where do you see yourself in two years as a prompt engineer?

   Show a path from executing prompt tests to owning evaluation strategy or a product area, with continued learning in ML and NLP.

3. 15

   How do you prioritise when you have multiple prompt experiments to run and limited time?

   Talk about impact, effort, and dependencies. Mention quick wins versus long-term bets, and communicating trade-offs to the team.

4. 16

   Do you have any questions for us?

   Ask about evaluation culture, tools used, how prompts are versioned, and how success is measured for the role.