TensorFlow Interview Questions and Answers

Answer: Training across multiple GPUs or machines

Distributed training trains models across multiple GPUs/machines. Speeds up training for large models/datasets. TensorFlow provides tf.distribute strategies.

Answer: Flexible API for building complex models with branching

Functional API builds models as graphs of layers. More flexible than Sequential. Supports multi-input/output, shared layers, branching.

Answer: Open-source machine learning framework by Google

TensorFlow is an open-source end-to-end machine learning platform developed by Google Brain team. It provides tools for building and deploying ML models.

Answer: Multi-dimensional array, fundamental data structure

A tensor is a multi-dimensional array with a uniform type. It is the fundamental data structure in TensorFlow, generalizing scalars, vectors, and matrices.

Answer: Synchronous training on multiple GPUs on single machine

MirroredStrategy performs synchronous training across multiple GPUs on single machine. Replicates model on each GPU, aggregates gradients. Simplest multi-GPU strategy.

Answer: API for building efficient input pipelines

tf.data.Dataset provides efficient data pipeline API. Supports lazy evaluation, parallel processing, prefetching. Essential for large datasets.

Answer: Synchronous training across multiple machines

MultiWorkerMirroredStrategy extends MirroredStrategy to multiple machines. Each machine can have multiple GPUs. Uses collective communication (all-reduce).

Answer: tf.constant([1, 2, 3])

Use tf.constant() to create a constant tensor. Example: tf.constant([1, 2, 3]) creates a 1D tensor with immutable values.

Answer: Combines consecutive elements into batches

dataset.batch(batch_size) combines consecutive elements into batches. Essential for mini-batch gradient descent training.

Answer: Strategy for training on Tensor Processing Units

TPUStrategy enables training on Google TPUs. TPUs are specialized hardware for ML. Extremely fast for large models. Use with TPU-optimized code.

Answer: Prefetches data while training to reduce waiting

dataset.prefetch() overlaps data preprocessing and model execution. Fetches next batch while training on current. Improves performance.

Answer: tf.constant is immutable, tf.Variable is mutable

tf.constant creates immutable tensors. tf.Variable creates mutable tensors that can be updated during training (weights, biases).

Answer: Artificially increases dataset through transformations

Data augmentation artificially expands training data through random transformations (rotation, flip, crop, etc.). Reduces overfitting, improves generalization.

Answer: High-level neural networks API integrated into TensorFlow

Keras is a high-level neural networks API. Since TensorFlow 2.0, it is tightly integrated as tf.keras, the recommended way to build models.

Answer: Accumulates gradients over multiple batches before updating

Gradient accumulation accumulates gradients over multiple mini-batches before updating weights. Simulates larger batch size with limited memory.

Answer: Automatically uses float16/float32 for optimal performance

AMP automatically chooses float16/float32 per operation for optimal performance and stability. Use tf.keras.mixed_precision.Policy. Requires gradient scaling.

Answer: Sequential model stacking layers linearly

Sequential is a linear stack of layers. Most straightforward way to build models. Each layer has one input and one output tensor.

Answer: Downsampling operation reducing spatial dimensions

Pooling reduces spatial dimensions of feature maps. MaxPooling takes maximum, AvgPooling takes average. Reduces parameters and computation.

Answer: Compiler optimizing TensorFlow computations

XLA compiles TensorFlow computations to optimized code. Fuses operations, reduces memory, improves speed. Enable with jit_compile=True or tf.function(jit_compile=True).

Answer: model.add(layer)

Use model.add(layer) to add layers to Sequential model. Example: model.add(tf.keras.layers.Dense(64, activation='relu')).

Answer: Removes unimportant weights to reduce model size

Pruning removes unimportant weights/neurons. Reduces model size and inference time. May require fine-tuning. Use TensorFlow Model Optimization toolkit.

Answer: Normalizes layer inputs for each mini-batch

Batch normalization normalizes layer inputs across mini-batch. Stabilizes and speeds up training. Reduces internal covariate shift.

Answer: Fully connected layer where each neuron connects to all inputs

Dense layer is a fully connected layer. Each output is computed from all inputs: output = activation(dot(input, weights) + bias).

Answer: Function called at training stages for custom behavior

Callbacks are functions called at specific training stages. Examples: EarlyStopping, ModelCheckpoint, ReduceLROnPlateau, TensorBoard.