Cohere India Prompt Engineer Interview: Rounds, Questions and Preparation

What the job involves

Prompt Engineers design, test, and refine prompts to make large language models produce reliable, safe, and useful outputs for products and customers. In AI and deeptech companies, they work closely with product, research, and engineering teams to translate use cases into prompt patterns, evaluate model behaviour, and document best practices. Day to day involves iterating on prompts, running evaluations, analysing failure cases, and building reusable prompt libraries.

Role and technical knowledge

1. 1

   What is a prompt engineer and how does the role differ from a machine learning engineer?

   A strong answer explains that prompt engineers focus on designing and iterating natural language instructions and evaluation, while ML engineers train, fine-tune, or deploy models. Mention collaboration between the two.

2. 2

   Explain zero-shot, one-shot, and few-shot prompting with examples relevant to a customer support chatbot.

   Define each and give a concrete example where the number of examples changes output quality. Note trade-offs like token cost and latency.

3. 3

   What is chain-of-thought prompting and when would you use it?

   Describe how it encourages step-by-step reasoning and improves accuracy on multi-step tasks. Mention that it can increase token usage and may not help simple tasks.

4. 4

   How do context windows and tokenisation affect prompt design?

   Explain that context windows limit input and output length, and tokenisation affects cost and truncation. Give a strategy like summarising or chunking long documents.

Practical scenarios and evaluation

1. 5

   You are building a prompt for a RAG system that answers questions from company policy documents. How would you design and test the prompt?

   Cover instructions to use only retrieved context, citation requirements, fallback for unknown answers, and a test set with expected answers. Mention measuring faithfulness and answer relevance.

2. 6

   A model keeps producing hallucinated facts in a summarisation task. How would you debug and fix this with prompt changes?

   Suggest adding constraints like 'only use information from the provided text', asking for quotes, and using a verification step. Mention evaluating with a factuality metric and iterating.

3. 7

   How would you create an evaluation rubric for a prompt that generates marketing copy?

   List criteria such as tone, brand alignment, clarity, and call to action. Explain how to score outputs, use human review, and track improvements over versions.

4. 8

   Describe how you would use Python to run a batch of prompts and compare outputs across model versions.

   Mention using an API, storing prompts and responses in a structured format, and computing metrics like exact match or similarity. Highlight reproducibility and logging.

Safety, ethics, and product thinking

1. 9

   What are common safety risks in LLM applications and how can prompt engineering reduce them?

   Discuss prompt injection, jailbreaks, bias, and harmful content. Suggest system prompts, input sanitisation, output filters, and refusal instructions.

2. 10

   How would you handle a user prompt that tries to extract confidential information from a RAG system?

   Explain access control, prompt instructions to ignore such requests, and not including sensitive data in context. Mention monitoring and escalation.

3. 11

   How do you balance creativity and factual accuracy when designing prompts for a product?

   Describe adjusting temperature, adding constraints, and using retrieval for facts. Give an example like a creative writing tool versus a legal assistant.

4. 12

   Why is documentation important for prompt engineering in a product team?

   Explain that prompts are assets that need versioning, rationale, and test results. Mention enabling collaboration and reducing regressions.

Behavioural and HR questions

1. 13

   Tell me about a time you had to learn a new AI tool or concept quickly for a project.

   Use STAR format. Focus on the learning process, resources used, and how you applied it to deliver a result.

2. 14

   Describe a situation where you received critical feedback on your work and how you responded.

   Show openness to feedback, specific changes you made, and the improved outcome. Avoid defensiveness.

3. 15

   How would you explain a complex prompt engineering concept to a non-technical stakeholder?

   Demonstrate clear communication with an analogy or simple example. Emphasise listening to their needs and avoiding jargon.

4. 16

   Why do you want to work in AI and deeptech, and why this role?

   Connect personal interest in language models to the company's mission. Mention specific projects or learning you have done in prompt engineering.