Runway India Prompt Engineer Interview: Rounds, Questions and Preparation

What the job involves

Prompt Engineers in AI and deeptech design, test, and refine prompts that make large language models and multimodal models produce reliable outputs for products and internal tools. Day to day they run experiments, evaluate model responses, write reusable prompt templates, and work with product, data, and engineering teams to turn model behaviour into a shippable feature. At smaller or fast-growing AI companies, they often own evaluation datasets, cost and latency trade-offs, and safety guardrails for prompts.

Role and technical knowledge

1. 1

   What is the difference between zero-shot, few-shot, and chain-of-thought prompting, and when would you use each?

   A strong answer explains that zero-shot relies on clear instructions, few-shot adds examples to shape format or style, and chain-of-thought asks for intermediate reasoning for multi-step tasks. Mention that chain-of-thought can increase latency and token cost, so use it only when accuracy on reasoning tasks matters.

2. 2

   How do temperature and top-p affect model outputs, and how would you set them for a customer support bot versus a creative writing tool?

   Explain that lower temperature and top-p make outputs more deterministic and factual, while higher values increase diversity. A support bot should use low values for consistency; a creative tool can use higher values, but still needs guardrails against unsafe or off-brand text.

3. 3

   What is retrieval-augmented generation, and why is it useful for an AI product company?

   Cover that RAG retrieves relevant documents or chunks and adds them to the prompt so the model answers from fresh or private data. Mention benefits like reduced hallucination, easier updates without retraining, and the need to evaluate retrieval quality separately from generation quality.

4. 4

   How would you force a model to return valid JSON for a downstream API, and what can go wrong?

   Mention schema instructions, JSON mode or function calling where available, few-shot examples, and validation with a parser plus retries. Risks include missing fields, wrong types, extra text, and token limits, so you should always validate and log failures.

Practical scenarios

1. 5

   A prompt works well in testing but fails on real user inputs. How do you debug and fix it?

   A strong answer starts by collecting failing examples, grouping them by failure type, and checking whether the issue is ambiguous instructions, missing context, or model limits. Then iterate with clearer constraints, add examples for the failing cases, and re-run an evaluation set before shipping.

2. 6

   You need to reduce the cost of an LLM feature by 50 percent without hurting quality. What do you try?

   Discuss shorter prompts, trimming retrieved context, caching frequent responses, choosing a smaller or cheaper model for easy cases, and routing hard cases to a stronger model. Emphasise measuring quality on a fixed eval set before and after each change.

3. 7

   How would you test a prompt for safety against prompt injection or jailbreak attempts?

   Explain creating a red-team set of adversarial inputs, testing whether the model leaks system instructions or follows malicious user content, and adding defences like input filtering, instruction hierarchy, output checks, and refusal patterns. Mention logging and monitoring after deployment.

4. 8

   A product manager asks for a summarisation feature that must handle long documents. How do you design the prompt and pipeline?

   Cover chunking or map-reduce summarisation, handling context window limits, preserving key facts, and evaluating summaries for coverage and hallucination. Mention testing with different document types and deciding whether to use a single long-context model or a multi-step pipeline.

Evaluation and experimentation

1. 9

   How do you measure whether one prompt is better than another for a generative task?

   A strong answer mentions a fixed test set, clear rubric or metrics such as accuracy, relevance, format compliance, and hallucination rate, plus human review for subjective quality. Also mention tracking variance across multiple runs and using pairwise comparison when absolute scores are hard.

2. 10

   What is an evaluation dataset, and how would you build one for a new prompt engineering project?

   Explain that it is a representative set of inputs with expected outputs or scoring criteria. Describe sourcing real examples, covering edge cases, keeping a held-out test set, and updating the dataset as new failure modes appear.

3. 11

   How do you decide between improving the prompt and fine-tuning a model?

   Discuss starting with prompt engineering because it is faster and cheaper, and fine-tuning when you need consistent style, domain-specific behaviour, or lower latency with a smaller model. Mention that fine-tuning needs good data and still requires evaluation and prompt guardrails.

4. 12

   What metrics would you track after a prompt goes live in production?

   Cover quality metrics from user feedback or automated checks, plus operational metrics like latency, token usage, cost per request, error rate, and refusal rate. Mention monitoring for drift and setting alerts when quality drops.

Behavioural and HR

1. 13

   Why do you want to work as a Prompt Engineer at an early-stage AI company rather than a large tech firm?

   Show genuine interest in fast iteration, broad ownership, and working close to the model and product. Mention that you enjoy ambiguous problems and want to learn across evaluation, safety, and deployment, not just one narrow task.

2. 14

   Tell me about a time you had to learn a new AI tool or concept quickly for a project.

   Use a specific example, such as learning an LLM API or a new evaluation method. Describe what you did, how you verified your understanding, and what you delivered, keeping the focus on action and result.

3. 15

   Describe a situation where your first attempt at a solution did not work and you had to iterate.

   Pick a project where you tested, found a failure, and improved based on evidence. Show that you are comfortable with iteration and that you use data or feedback rather than guessing.

4. 16

   How would you explain a complex prompt engineering decision to a non-technical teammate?

   Demonstrate clear communication by avoiding jargon, using an analogy or simple example, and focusing on the trade-off that matters to them, such as quality, cost, or user experience.