Rephrase.ai Computer Vision Engineer Interview: Rounds, Questions and Preparation

What the job involves

A Computer Vision Engineer in AI and deeptech builds and deploys models that extract information from images and videos, often for generative media applications like synthetic video or avatars. Day to day, you will work with large visual datasets, train deep learning models (e.g., for face reenactment, lip sync, or 3D reconstruction), optimise inference for real-time performance, and collaborate with backend and product teams to integrate vision pipelines. You will also evaluate model quality, handle edge cases, and iterate on architecture choices.

Role and technical knowledge

1. 1

   Explain how a convolutional neural network works. What are the key layers and their purposes?

   Cover convolution, pooling, activation (ReLU), and fully connected layers. Mention parameter sharing, local receptive fields, and how depth increases feature abstraction.

2. 2

   What is the difference between object detection, semantic segmentation, and instance segmentation? Give an example use case for each.

   Define each task clearly: detection gives bounding boxes, semantic segmentation labels every pixel by class, instance segmentation separates individual objects. Use examples like face detection, background removal, and counting people.

3. 3

   How would you handle a dataset with very few labelled images for a face landmark detection task?

   Discuss data augmentation (flips, rotations, colour jitter), transfer learning from a pre-trained face model, semi-supervised learning, and synthetic data generation if applicable.

4. 4

   What are GANs and how are they used in image or video generation? What challenges do they pose?

   Explain generator-discriminator adversarial training. Mention applications like face reenactment or style transfer. Challenges include mode collapse, unstable training, and high compute requirements.

Practical scenarios and problem solving

1. 5

   You need to deploy a real-time face swapping model on a mobile device. What steps would you take to optimise it?

   Cover model compression (pruning, quantisation), using lightweight architectures (MobileNet, EfficientNet), converting to TensorFlow Lite or ONNX, and reducing input resolution while maintaining quality.

2. 6

   Your model performs well on the validation set but poorly on real-world videos with varied lighting and occlusion. How do you diagnose and fix this?

   Suggest checking for domain shift, adding more diverse training data, using augmentation that mimics real conditions, and possibly fine-tuning on a small real-world set. Also consider test-time augmentation.

3. 7

   How would you build a pipeline to generate a talking-head video from an audio clip and a single portrait image?

   Outline stages: face detection and alignment, landmark extraction, audio-to-landmark mapping (e.g., using a seq2seq model), and image generation (e.g., using a GAN or neural rendering). Mention temporal consistency and lip-sync evaluation.

4. 8

   You have a large video dataset but limited storage. How would you design an efficient data loading and training pipeline?

   Discuss on-the-fly decoding and augmentation, using TFRecord or WebDataset formats, prefetching, and distributed training. Mention caching strategies and memory-mapped files.

Coding and implementation

1. 9

   Write a Python function using OpenCV to detect faces in an image and draw bounding boxes.

   Use cv2.CascadeClassifier or a pre-trained DNN face detector. Load image, convert to grayscale, detect faces, and draw rectangles. Mention handling multiple faces and adjusting parameters.

2. 10

   Implement a 2D convolution operation in NumPy without using deep learning libraries.

   Show sliding window approach with padding and stride. Use nested loops or im2col for efficiency. Discuss computational complexity and vectorisation.

3. 11

   Given a list of image paths and labels, write a PyTorch Dataset class for a classification task with augmentation.

   Subclass torch.utils.data.Dataset, implement __len__ and __getitem__, apply transforms from torchvision.transforms. Mention normalisation and converting to tensors.

4. 12

   How would you compute the Intersection over Union (IoU) between two bounding boxes? Provide a code snippet.

   Calculate intersection area and union area, then divide. Handle cases with no overlap. Show vectorised implementation for multiple boxes.

Behavioural and HR questions

1. 13

   Tell me about a computer vision project you worked on. What was your role and what challenges did you face?

   Use STAR method: describe the project, your specific contributions, technical hurdles (e.g., data scarcity, model convergence), and how you overcame them. Highlight learning outcomes.

2. 14

   Why do you want to work in generative AI and computer vision, specifically at a company like ours?

   Show genuine interest in synthetic media and its applications. Connect your skills to the company's domain (e.g., video generation, avatars). Mention specific projects or technologies that excite you.

3. 15

   Describe a time you had to learn a new technology quickly to complete a task. How did you approach it?

   Give a concrete example, e.g., learning a new framework or paper implementation. Emphasise self-study, online resources, and applying it under deadline. Mention the result.

4. 16

   How do you handle feedback on your work, especially when a model you built is not performing well?

   Show openness to critique, focus on data-driven debugging, and willingness to iterate. Mention collaborating with mentors or peers to improve.