22 Questions on Gen AI

Understanding Generative AI

1. Generative vs Discriminative AI Models

Q1: What distinguishes generative AI models from discriminative ones?

A1: Generative models (e.g., VAEs, GANs) learn the joint probability distribution $$ P(X,Y) $$ to generate new data resembling training data. Discriminative models (e.g., logistic regression, SVMs) learn $$ P(Y|X) $$ to classify or predict labels. Generative models excel in creativity (e.g., art generation), while discriminative models prioritize accuracy in classification tasks.

---

2. Rule-based vs Probabilistic Approaches

Q2: How do rule-based systems differ from probabilistic models like VAEs?

A2: Rule-based systems use predefined if-then logic (deterministic), ideal for structured tasks like grading. Probabilistic models (e.g., VAEs) leverage statistical distributions to handle uncertainty, generating data via latent space sampling. VAEs merge deep learning with probabilistic graphical models for flexible, noise-resistant outputs.

---

3. Evolutionary Algorithms & Fitness Functions

Q3: What role do fitness functions play in evolutionary algorithms?

A3: Fitness functions evaluate candidate solutions in evolutionary algorithms, mimicking natural selection. They guide mutation and crossover operations to optimize outcomes (e.g., route planning). High fitness solutions propagate, enabling adaptation to complex problems.

---

4. Multi-Modal Model Architecture

Q4: Describe the architecture of multi-modal models.

A4: Multi-modal models (e.g., GPT-4, DALL-E) use separate encoders for each data type (text, image, audio) and a fusion module to integrate embeddings. Cross-modal attention mechanisms align representations, enabling tasks like text-to-image generation.

---

5. Large Datasets & Bayesian Networks

Q5: Why do generative models require large datasets?

A5: Large datasets capture diverse patterns for robust generation (e.g., GPT-3 trained on 45TB text). Bayesian networks model dependencies via directed acyclic graphs, using probabilistic inference to handle missing data.

---

6. Scalability & Robustness

Q6: How are scalability and robustness addressed in generative AI?

A6: Scalability: Distributed training and model parallelism manage computational load. Robustness: Adversarial training and noise injection improve resilience to input variations.

---

7. Preprocessing Importance

Q7: Why is preprocessing critical for generative models?

A7: Preprocessing (normalization, tokenization) reduces noise and standardizes inputs, enhancing training stability. For images, resizing and augmentation improve model generalization.

---

8. Loss Functions

Q8: Compare MSE, cross-entropy, and KL divergence.

A8:
- MSE: Used in regression (e.g., image reconstruction).
- Cross-entropy: For classification, measuring predicted vs true label distributions.
- KL divergence: Quantifies divergence between distributions, key in VAEs.

---

9. Regularization Techniques

Q9: How do L1/L2 regularization prevent overfitting?

A9: L1 adds absolute weight penalties (sparsity), L2 squares weights (smoothing). Both constrain model complexity, reducing overfitting.

---

10. Deep Learning Architectures

Q10: Contrast CNNs, RNNs, GANs, and VAEs.

A10:
- CNNs: Image processing via convolutional layers.
- RNNs: Sequence modeling with memory cells (e.g., LSTM).
- GANs: Adversarial training for realistic generation.
- VAEs: Probabilistic latent space for diverse outputs.

---

11. Metrics

Q11: Define F1, AUC-ROC, Inception Score.

A11:
- F1: Balances precision/recall.
- AUC-ROC: Classifier performance across thresholds.
- Inception Score: Image quality/diversity via classifier entropy.

---

12. Complexity vs Interpretability

Q12: Why balance model complexity and interpretability?

A12: Complex models (e.g., GANs) may lack transparency, hindering trust. Simpler models (rule-based) offer explainability but lower accuracy.

---

13. Adversarial Networks

Q13: How do adversarial networks enhance generative AI?

A13: GANs use a generator-discriminator duel to refine outputs, achieving photorealistic images/text. The discriminator’s feedback improves generator realism.

---

14. Reinforcement Learning (RL)

Q14: How is RL expanding in generative AI?

A14: RL optimizes generation via reward signals (e.g., text style adherence). Applications include game strategy and robotic control.

---

15. Graph Neural Networks (GNNs)

Q15: What tasks use GNNs?

A15: GNNs process graph-structured data (social networks, molecules) for node classification or link prediction.

---

16. Unsupervised Learning

Q16: How does unsupervised learning aid generative AI?

A16: Techniques like clustering and VAEs discover patterns without labels, enabling anomaly detection and data generation.

---

17. Federated Learning

Q17: How does federated learning scale generative AI?

A17: Trains models across decentralized devices, preserving privacy (e.g., healthcare). Reduces central data storage needs.

---

18. Recent Breakthroughs

Q18: Highlight recent generative AI advancements.

A18: Graph NNs for molecule design, RL with Q-learning, evolutionary algorithms for optimization, and gradient descent variants (Adam).

---

19. Text-Audio Metrics

Q19: Define WER, BLEU, perplexity.

A19 :
- WER: Speech recognition accuracy.
- BLEU: Text translation quality.
- Perplexity: Language model confidence.

---

20. Text-Image Metrics

Q20: What metrics evaluate text-to-image models?

A20: Inception Score (quality/diversity), FID (realism), SSIM (structural similarity).

---

21. Limitations

Q21: What are key generative AI limitations?

A21:
- Data hunger: Requires massive datasets.
- Bias: Training data biases propagate.
- GAN instability: Mode collapse risks.

---

22. Practical Bottlenecks

Q22: Identify deployment challenges.

A22:
- Computational: High GPU costs for training.
- Ethical: Deepfakes, copyright issues. Mitigation includes watermarking and ethical guidelines.