Question 1

What is Vertex AI?

Accepted Answer

Vertex AI is GCP's unified machine-learning platform, covering the whole lifecycle: data labelling, feature management, AutoML and custom training, hyperparameter tuning, a model registry, batch and online prediction endpoints, pipelines for orchestration, and model monitoring. It also hosts Google's foundation models such as Gemini and provides the tooling around them. The word "unified" is the point — it replaced separate AI Platform and AutoML products that had different APIs and artefacts. Framing it as one platform with consistent metadata and lineage across training and serving is what the product is actually for.

Question 2

What is the difference between AutoML and custom training on Vertex AI?

Accepted Answer

AutoML trains a model from your labelled data with no model code — you choose the objective, supply data, and Google searches architectures and hyperparameters. Custom training runs your own code in a container with your chosen framework, giving full control over architecture, loss and training loop, at the cost of doing the work yourself. The decision rule is a baseline argument: start with AutoML to establish what accuracy is achievable and how quickly, then move to custom training only if you can beat it meaningfully. Teams that begin with custom training often spend weeks matching what AutoML produced in an afternoon.

Question 3

What is the difference between online and batch prediction?

Accepted Answer

Online prediction serves a deployed model behind a low-latency endpoint for one request at a time, with autoscaling instances you pay for while they exist. Batch prediction runs a job over a large input set in Cloud Storage or BigQuery, writes results back, and costs nothing when idle because there is no persistent endpoint. The cost consequence is the practical part: an online endpoint with a minimum replica count bills continuously, so a model only used for a nightly scoring run should use batch prediction. That is a very common and expensive misconfiguration.

Question 4

What is Vertex AI Pipelines?

Accepted Answer

Vertex AI Pipelines runs machine-learning workflows defined with Kubeflow Pipelines or TFX as a serverless managed service. Each step is a containerised component, inputs and outputs are tracked as artefacts, and the platform records lineage, caches unchanged steps and lets you schedule and parameterise runs. Step caching is the feature to name because it changes the economics of iteration: re-running a pipeline after changing only the training step skips the expensive data preparation. Lineage — knowing exactly which data and code produced a deployed model — is the governance argument.

Question 5

What is Vertex AI Feature Store and why does it exist?

Accepted Answer

A feature store centralises the computation, storage and serving of machine-learning features so the same definition is used for training and for online inference. It provides low-latency online serving for real-time prediction and consistent historical values for training, along with sharing and discovery across teams. The problem it solves is training-serving skew: a feature computed one way in a training SQL query and another way in production code produces a model that performs worse in production than in evaluation. Naming training-serving skew explicitly is what makes this answer land.

Question 6

What is training-serving skew and how do you prevent it?

Accepted Answer

It is a mismatch between the data distribution or the feature computation at training time and at serving time, causing the deployed model to perform worse than evaluation suggested. Prevent it with a feature store or shared transformation code used by both paths, by validating serving inputs against the training schema, and by monitoring the live feature distribution against the training baseline. The subtle version is timing: computing a feature from data that would not have been available at prediction time — target leakage — makes the model look excellent in evaluation and useless in production. Naming leakage as a related failure shows depth.

Question 7

What is model monitoring in Vertex AI?

Accepted Answer

Vertex AI Model Monitoring watches deployed endpoints for training-serving skew and for prediction drift — comparing the distribution of incoming features against a training baseline or a previous window — and alerts when a statistical distance exceeds a threshold. It can also monitor prediction output distributions. The distinction to make is between skew (serving data differs from training data) and drift (serving data changes over time). Both degrade a model silently, because accuracy is not observable in production until labels arrive, which may be weeks later. That delayed-label problem is why distribution monitoring is the practical proxy.

Question 8

What is the Vertex AI Model Registry?

Accepted Answer

The Model Registry is the central catalogue of model versions with their metadata, evaluation metrics, lineage back to the training run and data, and deployment state. It gives you versioning, aliasing, and a controlled path from a trained artefact to a deployed endpoint. The governance value is auditability — for a regulated use case you must be able to say which data and code produced the model that made a specific decision. Aliases such as "production" and "champion" also let you swap versions without changing client configuration.

Question 9

How do you deploy a model to a Vertex AI endpoint with zero downtime?

Accepted Answer

Deploy the new model version to the same endpoint alongside the existing one and split traffic by percentage, increasing gradually while monitoring prediction latency, error rate and output distribution. Roll back by shifting traffic back to the previous version, which is still deployed. Traffic splitting at the endpoint is the mechanism, and the discipline is having a rollback criterion defined before you start. For a model, the criterion should include an output-distribution check, not just latency and errors, because a bad model returns valid responses that are wrong.

Question 10

What is Vertex AI Workbench?

Accepted Answer

Workbench provides managed Jupyter notebook environments integrated with GCP — pre-installed frameworks, access to BigQuery and Cloud Storage without credential handling, idle shutdown, and the ability to scale to a GPU instance. It is the development surface for data scientists. The operational controls worth naming are idle shutdown, which prevents a forgotten GPU notebook running all month, and the ability to enforce no-public-IP and use of a service account, which is what makes notebooks acceptable in a governed environment.

Question 11

What is hyperparameter tuning on Vertex AI?

Accepted Answer

Vertex AI Vizier-backed hyperparameter tuning runs many training trials with different parameter values, using Bayesian optimisation to choose promising combinations rather than exhaustive grid search, and reports the best trial by the metric you optimise. Trials run in parallel within a configured limit. The efficiency argument is that Bayesian search reaches a good configuration in far fewer trials than grid or random search, which matters because each trial costs GPU time. Setting a sensible parallel-trial count is a trade-off: more parallelism is faster but gives the optimiser less information per round.

Question 12

What is Gemini and how do you use it on GCP?

Accepted Answer

Gemini is Google's family of multimodal foundation models, available through the Vertex AI API for text, image, audio and video understanding, code generation and function calling. On Vertex AI you get enterprise controls — data residency, VPC Service Controls, IAM, customer-managed encryption and no use of your data for training — alongside the model. The enterprise-controls point is the reason a company uses Vertex AI rather than a consumer API, and it is what an interviewer for a GCP role is listening for. Naming grounding, safety filters and provisioned throughput as additional platform features rounds it out.

Question 13

What is retrieval-augmented generation and how would you build it on GCP?

Accepted Answer

RAG grounds a language model in your own data: you embed documents into vectors, store them in a vector index, retrieve the most relevant chunks for a user query, and include them in the prompt so the model answers from that context. On GCP you would use an embedding model on Vertex AI, Vertex AI Vector Search or a pgvector-enabled AlloyDB or BigQuery vector index for retrieval, and Gemini for generation — or Vertex AI Search for a managed end-to-end version. The design decisions that matter are chunking strategy, embedding model choice, and whether you rerank retrieved results before prompting. Naming evaluation — measuring retrieval recall separately from answer quality — is what distinguishes someone who has shipped RAG from someone who has read about it.

Question 14

What is Vertex AI Vector Search?

Accepted Answer

Vector Search is a managed approximate nearest neighbour service, built on Google's ScaNN research, that indexes high-dimensional embeddings and returns the most similar vectors at very low latency across billions of items. It supports filtering by metadata alongside the similarity search. The trade-off inherent in ANN is recall versus latency and cost — an approximate index does not guarantee the true nearest neighbours, and the tuning parameters control that balance. Acknowledging approximation rather than describing it as exact search is the correctness marker.

Question 15

What is an embedding and why does it matter?

Accepted Answer

An embedding is a dense numeric vector representing the meaning of text, an image or another item, such that semantically similar items are close together in the vector space. It matters because it turns semantic similarity into a distance calculation, enabling search, recommendation, clustering and deduplication that keyword matching cannot do. The practical caution is that embeddings from different models are not comparable, so changing the embedding model requires re-embedding the entire corpus. That migration cost is a real operational consideration people discover late.

Question 16

What is prompt engineering versus fine-tuning versus RAG?

Accepted Answer

Prompt engineering shapes behaviour through instructions and examples with no training — cheapest and fastest to iterate. RAG supplies external knowledge at inference time, which is right when the model needs facts it was not trained on and those facts change. Fine-tuning adjusts model weights on your examples, which is right for teaching a consistent style, format or task behaviour that prompting cannot reliably achieve. The rule to state is that RAG solves knowledge problems and fine-tuning solves behaviour problems, and you try prompting first because it costs nothing to change. Candidates who propose fine-tuning to add facts have the mental model backwards, and interviewers probe exactly that.

Question 17

What is Vertex AI Agent Builder / Vertex AI Search?

Accepted Answer

Vertex AI Search provides managed enterprise search and RAG over your own data with connectors, ingestion, chunking, retrieval and grounded generation handled for you. Agent Builder extends that to conversational agents that can call tools and follow multi-step flows without you assembling the retrieval stack yourself. The build-versus-buy framing is what interviewers want: managed search gets a grounded assistant working quickly with less control over chunking and ranking, while a custom stack on Vector Search gives full control at higher engineering cost. Naming the criterion — how much retrieval tuning the use case needs — is the answer.

Question 18

How do you evaluate a machine learning model before deploying it?

Accepted Answer

Hold out a test set that the model never saw, evaluate with metrics matched to the business problem — precision and recall or AUC for imbalanced classification rather than accuracy — check performance across important slices such as region or customer segment, compare against a simple baseline, and validate on the most recent data to catch temporal drift. The slice evaluation is what separates a thoughtful answer: an overall accuracy of 95% can hide a model that fails completely for a minority segment. For generative models, evaluation shifts to human review, LLM-as-judge scoring and task-specific benchmarks, which is worth mentioning if the role is generative-AI focused.

Question 19

What is MLOps and what does a mature GCP MLOps setup look like?

Accepted Answer

MLOps applies engineering discipline to machine learning: version-controlled data, code and models; automated, reproducible training pipelines; automated evaluation gates before promotion; a model registry with lineage; controlled deployment with traffic splitting; and continuous monitoring with retraining triggers. On GCP that is Vertex AI Pipelines, Model Registry, Feature Store, Model Monitoring and Cloud Build, tied together with Terraform-managed infrastructure. The maturity ladder is a good framing — manual notebooks, then automated training, then automated retraining triggered by monitoring. Naming the trigger for retraining, and insisting that a retrained model must pass the same evaluation gate as a manual one, is the senior detail.

Question 20

When would you retrain a model, and how do you decide?

Accepted Answer

On a schedule appropriate to how fast the domain changes; on a drift or skew alert from model monitoring; or on a measured performance drop once ground-truth labels arrive. The decision should be automatic to trigger and gated to deploy — a retrained model is promoted only if it beats the incumbent on the evaluation set. The gate is essential: automatic retraining without an evaluation gate can deploy a model trained on corrupted or drifted data, making things worse. Describing champion-challenger evaluation before promotion is what makes this a production answer.

Question 21

What is BigQuery ML and when would you use it instead of Vertex AI?

Accepted Answer

BigQuery ML trains and serves models with SQL directly on warehouse data, supporting regression, classification, clustering, time series and imported models. Use it when the data is already in BigQuery, the problem fits its model types, and the team is SQL-fluent. Use Vertex AI for custom architectures, deep learning, sophisticated experiment tracking or serving requirements it cannot meet. The integration point worth naming is that BigQuery ML models can be registered in Vertex AI Model Registry and deployed to a Vertex endpoint, so the two are not mutually exclusive — you can prototype in SQL and productionise on Vertex.

Question 22

How do you serve a model with low latency and control cost?

Accepted Answer

Deploy to a Vertex AI endpoint sized to the traffic with autoscaling and a sensible minimum replica count, choose an accelerator only if the model genuinely needs it, batch requests where the client can tolerate it, cache repeated predictions, and consider a smaller distilled model if the accuracy trade-off is acceptable. For non-interactive scoring, use batch prediction instead of an endpoint. The distillation and model-size lever is the one candidates least often mention, and it is frequently the largest saving — serving a model a tenth of the size at nearly the same accuracy beats any infrastructure tuning.

Question 23

What are TPUs and when would you use them over GPUs?

Accepted Answer

TPUs are Google's custom accelerators designed for large-scale tensor operations, particularly effective for training and serving large neural networks with high throughput and good performance per rupee at scale. GPUs are more flexible, support a wider range of frameworks and custom operations, and are easier to obtain for smaller workloads. The practical criterion is workload shape and framework support: TPUs excel with large, regular matrix workloads in supported frameworks, while custom kernels or unusual operations favour GPUs. Recommending TPUs without checking framework compatibility is a mistake an interviewer will probe.

Question 24

How do you handle sensitive data in a machine learning pipeline on GCP?

Accepted Answer

Use Cloud Data Loss Prevention to discover, classify and de-identify PII before it reaches training data — tokenisation, masking or format-preserving encryption. Apply column-level policy tags in BigQuery, keep the pipeline inside a VPC Service Controls perimeter, use CMEK, and ensure prediction logging does not capture raw sensitive inputs. Prediction request logging is the trap: enabling it for debugging can silently persist sensitive payloads in logs that have much weaker access controls than the source data. Naming that specific leak is a strong signal of practical security awareness.

Question 25

What is Cloud Data Loss Prevention (Sensitive Data Protection)?

Accepted Answer

It is a service that inspects text, images and structured data for over a hundred built-in sensitive information types — names, national ID numbers, card numbers, health identifiers — and can de-identify them by redaction, masking, tokenisation or format-preserving encryption, including reversible tokenisation with a KMS-wrapped key. The reversible tokenisation capability is the interesting one: you can de-identify data for analysts while retaining the ability to re-identify for authorised purposes, which is what makes it usable in real workflows rather than only for redaction.

Question 26

What is responsible AI and what does GCP provide for it?

Accepted Answer

Responsible AI covers fairness, transparency, safety and accountability. GCP provides safety filters and configurable thresholds on generative models, Explainable AI with feature attributions for tabular and image models, Model Cards for documentation, and evaluation tooling for bias across slices. The organisational half — review processes and defined acceptable use — is not a product. The strongest answer acknowledges that tooling supports but does not deliver responsible AI: deciding what fairness means for a specific decision is a human judgement. Interviewers for senior roles look for that distinction rather than a product list.

Question 27

What is Explainable AI on Vertex?

Accepted Answer

Explainable AI provides feature attributions — which inputs most influenced a prediction — using methods such as sampled Shapley, integrated gradients and XRAI for images. It works for both batch and online predictions and helps with debugging, stakeholder trust and regulatory requirements to explain automated decisions. The caution to include is that attributions explain the model, not the world — a high attribution on a proxy variable reveals what the model uses, not causation. Overstating explanations as causal is a common and consequential mistake.

Question 28

What is a Vertex AI custom container and when do you need one?

Accepted Answer

A custom container packages your training or serving code with its exact dependencies, so you control the framework version, system libraries and entry point. You need one when the prebuilt containers do not have your framework version, you need native dependencies, or your serving logic includes preprocessing that must run alongside the model. The serving-side contract is the part to know: the container must implement the expected health and predict routes and respond within the timeout. Getting that contract wrong is the usual cause of a deployment that builds fine and then fails health checks.

Question 29

How would you build a recommendation system on GCP?

Accepted Answer

For a managed route, Vertex AI Search for commerce provides retail recommendations with minimal modelling. For a custom route: build user and item embeddings from interaction data with a two-tower model or matrix factorisation in BigQuery ML, index item embeddings in Vector Search for candidate retrieval, then rank candidates with a model that includes context features, serving features from Feature Store and the final model from a Vertex endpoint. The two-stage retrieval-then-ranking architecture is the substance of the answer — you cannot score millions of items per request, so you retrieve a few hundred cheaply and rank them expensively. Naming that structure demonstrates real recommender knowledge.

Question 30

What is function calling in a generative model and why does it matter?

Accepted Answer

Function calling lets you describe available tools to the model, and instead of answering directly the model returns a structured request to call one of them with arguments. Your code executes the call and returns the result, which the model uses to compose its answer. It is how a language model interacts with live data and takes actions. The security point to raise unprompted is that the model chooses which function to call based on untrusted input, so your code must validate arguments and enforce authorisation independently. Treating a model's function-call request as authorised is a serious vulnerability.

Question 31

What is grounding in generative AI on Vertex?

Accepted Answer

Grounding attaches the model's response to a verifiable source — your own data through Vertex AI Search, or Google Search for public information — and returns citations. It substantially reduces hallucination by forcing answers to be supported by retrieved content rather than generated from parametric memory. The honest qualification is that grounding reduces but does not eliminate hallucination, and that a model can still misread retrieved context. For high-stakes use cases, citations that a user can check are as important as the accuracy improvement itself.

Question 32

How do you control the cost of generative AI workloads on GCP?

Accepted Answer

Choose the smallest model that meets quality requirements — Flash-class models are dramatically cheaper than Pro-class for many tasks. Cap output tokens, trim prompts and avoid resending unchanged context, use context caching for repeated large prompts, cache responses for repeated queries, batch where latency permits, and consider provisioned throughput if usage is high and steady. Context caching is the lever most teams miss: a RAG system that resends the same large system prompt on every request pays for those input tokens every time. Naming it, along with model right-sizing, shows genuine cost engineering rather than generic advice.

Question 33

What is the difference between supervised, unsupervised and reinforcement learning?

Accepted Answer

Supervised learning trains on labelled examples to predict a target — classification and regression. Unsupervised learning finds structure in unlabelled data — clustering, dimensionality reduction, anomaly detection. Reinforcement learning learns a policy by taking actions in an environment and receiving rewards, without labelled examples. A GCP-flavoured close is naming a service for each: AutoML or BigQuery ML for supervised, k-means in BigQuery ML for unsupervised, and reinforcement learning as a custom training workload since there is no managed product for it. That grounds the theory in the platform.

Question 34

What is overfitting and how do you detect and prevent it?

Accepted Answer

Overfitting is when a model learns noise specific to the training data and therefore performs much worse on unseen data. You detect it as a large gap between training and validation performance. You prevent it with more or more varied data, regularisation, simpler models, early stopping, dropout for neural networks, and proper cross-validation. The GCP-specific addition is that Vertex AI training supports early stopping and hyperparameter tuning can optimise regularisation strength directly. Also worth naming: a validation set contaminated by leakage will hide overfitting entirely, which is the failure behind many models that look excellent and fail in production.

Question 35

What is the difference between precision and recall, and when do you optimise for each?

Accepted Answer

Precision is the fraction of positive predictions that are correct; recall is the fraction of actual positives that were found. Optimise precision when a false positive is costly — flagging a legitimate transaction as fraud and blocking a customer. Optimise recall when a false negative is costly — missing a disease diagnosis or a security breach. The framing to give is that the threshold is a business decision, not a modelling one: the same model produces different precision and recall at different thresholds, so the right question is what each error costs. That reframing is what interviewers look for at mid level and above.

Question 36

What is a confusion matrix?

Accepted Answer

A table of true positives, false positives, true negatives and false negatives for a classifier, from which precision, recall, specificity, F1 and accuracy are all derived. It shows exactly which kinds of mistakes the model makes rather than collapsing performance into one number. The value to emphasise is with imbalanced data: a model predicting the majority class for everything can show 99% accuracy while the confusion matrix immediately reveals it never predicts the minority class at all. That is why accuracy alone is a misleading metric.

Question 37

What is feature engineering and where does it happen on GCP?

Accepted Answer

Feature engineering transforms raw data into inputs a model can learn from — aggregations, ratios, time-based windows, encodings and normalisation. On GCP it typically happens in BigQuery SQL for tabular data, in Dataflow for streaming features, or in TensorFlow Transform within a pipeline, with the results stored in Feature Store for consistent reuse. The rule to state is that whatever computes the feature for training must also compute it for serving, which is exactly why the feature store exists. Framing feature engineering as a consistency problem rather than only a modelling one is what connects it to production reality.

Question 38

What is a Vertex AI endpoint versus a model?

Accepted Answer

A model is the trained artefact registered in the Model Registry. An endpoint is the serving resource with a stable URL and compute behind it. You deploy one or more model versions to an endpoint with a traffic split, so the endpoint is the stable address while the models behind it change. This separation is what makes zero-downtime model updates possible: clients call the endpoint and never learn which version served them. Understanding that a model can be deployed to several endpoints, and an endpoint can host several models, is the full picture.

Question 39

How would you debug a model that performs well in evaluation but poorly in production?

Accepted Answer

Check for training-serving skew in feature computation; compare the live feature distribution against the training baseline with model monitoring; look for target leakage in the training data that would not be available at prediction time; verify the serving preprocessing matches the training preprocessing exactly; and check whether the production population differs from the training population in a way the evaluation split hid. The ordering matters: skew and leakage explain the great majority of such cases, and both are data problems rather than model problems. A candidate who immediately proposes trying a different architecture is looking in the wrong place, which is precisely what the question tests.

Question 40

Design an end-to-end ML platform on GCP for a team of ten data scientists.

Accepted Answer

Data lands in BigQuery and Cloud Storage with governed access through policy tags. Features are defined once and served from Vertex AI Feature Store. Experimentation happens in Vertex AI Workbench with no public IPs and idle shutdown. Training runs as Vertex AI Pipelines, version-controlled and triggered by Cloud Build, writing to the Model Registry with lineage. Promotion is gated on automated evaluation including slice metrics. Deployment is to Vertex endpoints with traffic splitting, or batch prediction where interactive latency is not needed. Model Monitoring watches skew and drift and triggers retraining, and everything is provisioned with Terraform inside a VPC Service Controls perimeter. The closing scenario. The senior markers are the evaluation gate before promotion, the feature store solving skew rather than being a nice-to-have, and treating monitoring as a retraining trigger rather than a dashboard nobody reads.

GCP Vertex AI & Machine Learning Interview Questions and Answers

How to use this set