Generative AI Comprehensive Quiz & Projects
30 questions on Generative AI Tutorial.
Question 1: Why do standard Large Language Models (like GPT-4) utilize a Decoder-only Transformer architecture for text generation?
- A. Decoders are cheaper to compute during the initial tokenization step.
- B. Decoder-only models generate text autoregressively by predicting the next token based on previous tokens using causal masking. β (correct answer)
- C. Decoders prevent models from outputting hallucinations.
- D. Decoder-only architectures do not require GPU hardware.
Explanation: Causal masking in decoder-only architectures ensures that token predictions rely only on past context, perfect for generative tasks.
Question 2: How does the Temperature parameter affect Large Language Model outputs?
- A. High temperature increases hardware temperature during processing.
- B. Low temperature makes outputs more creative, while high temperature restricts formatting.
- C. Temperature controls probability distribution sharpness; higher values increase output randomness and diversity. β (correct answer)
- D. Temperature controls the rate of prompt token consumption.
Explanation: A higher temperature flattens token probability distribution, allowing less likely words to be selected, boosting randomness.
Question 3: What is the core architecture of Retrieval-Augmented Generation (RAG)?
- A. Fine-tuning model weights using proprietary PDFs.
- B. Querying an external data source (like a vector database) using prompt embeddings to inject relevant context before the LLM generates a response. β (correct answer)
- C. Compiling multiple LLMs to vote on the best generated output.
- D. Hardcoding standard answers into a lookup dictionary.
Explanation: RAG dynamic context injection provides the model with factual reference data at query time, bypassing the need for model fine-tuning.
Question 4: When is Fine-Tuning preferred over Prompt Engineering / In-Context Learning?
- A. When you want the model to learn a completely new format, tone, style, or specific internal database schemas. β (correct answer)
- B. When you need to update the model with real-time sports score data.
- C. When you want to decrease the model size.
- D. When you do not have access to any training data.
Explanation: Fine-tuning modifies neural weights, adapting model behavior, formatting rules, or stylistic guidelines.
Question 5: What is the primary cause of 'Hallucinations' in autoregressive language models?
- A. The GPU running out of memory mid-generation.
- B. The model attempting to write binary code.
- C. The statistical nature of predicting the next most probable token without an inherent mechanism for factual verification. β (correct answer)
- D. The system prompt containing grammatical errors.
Explanation: LLMs are next-token probability predictors. They generate linguistically coherent paths, not verified facts.
Question 6: What is Generative AI?
- A. A system that translates SQL code to Python files.
- B. A category of artificial intelligence designed to create new, original content (text, images, audio, code) based on training patterns. β (correct answer)
- C. A database backup manager.
- D. A server load balancer.
Explanation: Generative AI uses deep learning architectures to generate new outputs mimicking training data.
Question 7: In Generative Adversarial Networks (GANs), what are the roles of the Generator and the Discriminator?
- A. The Generator compiles the code, and the Discriminator runs test units.
- B. The Generator creates fake data samples, and the Discriminator attempts to distinguish between real data and fake generator samples. β (correct answer)
- C. The Generator logs errors, and the Discriminator encrypts files.
- D. They are identical.
Explanation: GANs use an adversarial training loop where both models improve through competition.
Question 8: What mathematical process defines Diffusion Models (like Stable Diffusion) for image generation?
- A. Multiplying pixel matrices by random integers.
- B. Iteratively removing added Gaussian noise from a random noise tensor to recover a clean, coherent image guided by text embeddings. β (correct answer)
- C. Compressing image file bytes into base64.
- D. Scanning images with Sobel kernels.
Explanation: Diffusion models learn to reverse a forward noise process, starting from pure static and generating images step-by-step.
Question 9: Which model class is designed for language translation by mapping sequences to sequences?
- A. CNN
- B. RNN
- C. Encoder-Decoder Transformer β (correct answer)
- D. GAN
Explanation: Encoder-Decoder architectures (like T5) encode source sequences and decode target translated sequences.
Question 10: What does the 'context window' of a Large Language Model represent?
- A. The size of the browser window.
- B. The maximum number of tokens (prompt + output combined) the model can process in a single inference call. β (correct answer)
- C. The speed of token output generation.
- D. The directory size of database tables.
Explanation: Context windows limit the text history models remember in a single chat session.
Question 11: What is RLHF (Reinforcement Learning from Human Feedback) primarily used for?
- A. Speeding up GPU compute times.
- B. Aligning LLM behavior with human values, safety guidelines, and preference styles using reward models. β (correct answer)
- C. Creating vector database index tables.
- D. Translating text between programming languages.
Explanation: RLHF refines pre-trained models, minimizing toxic outputs by training on human preference ratings.
Question 12: What is a 'Token' in the context of LLM processing?
- A. A security access key.
- B. A word, subword, or character chunk parsed by model tokenizers before calculating probabilities. β (correct answer)
- C. A database column key.
- D. An HTML element wrapper.
Explanation: Tokens are the basic string units models read; e.g., 100 words is roughly 130 tokens.
Question 13: What does a 'Vector Database' accomplish in RAG pipelines?
- A. It compresses PDF files.
- B. It stores and index high-dimensional vector embeddings, enabling fast semantic similarity lookups of document chunks. β (correct answer)
- C. It compiles Python modules.
- D. It acts as an email server.
Explanation: Vector DBs (e.g. Pinecone) search text chunks by meaning (vector angles) rather than exact word matches.
Question 14: What is the difference between Parameter-Efficient Fine-Tuning (PEFT) and Full Fine-Tuning?
- A. PEFT is slower and requires more hardware RAM.
- B. PEFT modifies only a small subset of parameters (e.g. LoRA adapters) while keeping base weights frozen, saving memory and compute resources. β (correct answer)
- C. Full Fine-Tuning does not require training data.
- D. PEFT is deprecated.
Explanation: PEFT techniques like LoRA allow training models on consumer GPUs by freezing base model weights.
Question 15: What does the Top-P (Nucleus Sampling) parameter control?
- A. The processor thread counts.
- B. The token pool considered for selection, limiting choices to candidates whose cumulative probability mass exceeds threshold P. β (correct answer)
- C. The context window token size.
- D. The system prompt length.
Explanation: Top-P dynamically adjusts the word selection pool, filtering out low-probability choices.
Question 16: What is a 'Prompt' in generative applications?
- A. A system command running backend migrations.
- B. The input text instruction provided to a model to guide its output generation. β (correct answer)
- C. A local network configuration.
- D. A class definition.
Explanation: Prompts instruct the model on what to write, format, or calculate.
Question 17: What is a Variational Autoencoder (VAE)?
- A. A database schema checking tool.
- B. A generative model that encodes inputs into a continuous latent space representation, allowing generation of new samples by sampling from this space. β (correct answer)
- C. A template compiler.
- D. An image edge detection filter.
Explanation: VAEs generate content by decoding points sampled from a probabilistic latent vector distribution.
Question 18: What does a high 'hallucination rate' indicate about an LLM's output?
- A. The output contains a high number of coding syntax errors.
- B. The output contains a high frequency of plausible-sounding but factually incorrect assertions. β (correct answer)
- C. The generation speed is slow.
- D. The model is running out of context memory.
Explanation: Minimizing hallucinations requires techniques like RAG, fact checking, and prompt constraints.
Question 19: How does LoRA (Low-Rank Adaptation) optimize model fine-tuning?
- A. By decreasing input token lengths.
- B. By decomposing weight update matrices into low-rank matrices, drastically reducing trainable parameter counts during training. β (correct answer)
- C. By storing weights in SQL database tables.
- D. By running training loops in parallel servers.
Explanation: LoRA adapters are small files, avoiding the overhead of saving copies of massive base weights.
Question 20: Which model class is commonly used to generate human faces or text-to-image prompts?
- A. RNN
- B. CNN
- C. Diffusion Models (and GANs) β (correct answer)
- D. LSTM
Explanation: Diffusion models and GANs dominate modern image generation applications.
Question 21: What is the function of the encoder block in a Transformer?
- A. It compiles styles.
- B. It processes input text tokens to generate context-rich vector representations of the input sequence. β (correct answer)
- C. It autoregressively outputs text tokens.
- D. It connects to the vector database.
Explanation: Encoders map input meaning; decoders read these representations to generate responses.
Question 22: What is the purpose of RLHF's reward model?
- A. To reward developers with points.
- B. To evaluate model outputs, returning a score that reflects human preference alignment to guide reinforcement learning updates. β (correct answer)
- C. To count token costs.
- D. To clean training datasets.
Explanation: Reward models mimic human rating decisions, acting as loss targets in policy updates.
Question 23: Which parameter limits token selection to a static count of the most probable candidate words?
- A. Temperature
- B. Top-K β (correct answer)
- C. Top-P
- D. Max Tokens
Explanation: Top-K limits choices to a fixed number (K) of candidates with the highest probabilities.
Question 24: What is the function of a 'System Message' in chat APIs?
- A. To display warnings to database administrators.
- B. To define the persona, bounds, and instructions that govern the LLM's behavior across subsequent user messages. β (correct answer)
- C. To handle routing configuration.
- D. To log chat histories to text files.
Explanation: System messages set the rules and guards the model must obey during conversations.
Question 25: What is the main limitation of auto-regressive decoding during inference?
- A. It uses too much GPU memory.
- B. It must generate tokens sequentially, one-by-one, which restricts parallelization and limits inference speed. β (correct answer)
- C. It requires fine-tuning.
- D. It does not support temperature scaling.
Explanation: Because each token depends on all previous tokens, generation is bounded by sequential latency.
Question 26: What does 'LLM' stand for?
- A. Low-Level Machine
- B. Large Language Model β (correct answer)
- C. Logic Loop Manager
- D. Linear Learning Matrix
Explanation: Large Language Models are neural networks trained on vast text corpora to predict and generate language.
Question 27: What is 'prompt hacking' or 'prompt engineering'?
- A. Writing assembly code files.
- B. The practice of designing and refining prompt inputs to elicit high-quality, targeted outputs from language models. β (correct answer)
- C. Editing model weight matrices.
- D. Accessing cloud server directories.
Explanation: Prompt engineering structures instructions to guide models toward accurate formats.
Question 28: What is the difference between semantic search and keyword search?
- A. Semantic search matches words exactly, while keyword search handles definitions.
- B. Semantic search analyzes the conceptual meaning (intent/context) of queries, while keyword search looks for literal character string matches. β (correct answer)
- C. Keyword search requires vector databases.
- D. There is no difference.
Explanation: Semantic search uses vector space embeddings to retrieve concepts related in meaning.
Question 29: What is the purpose of 'few-shot' prompting?
- A. Limiting the output to a few words.
- B. Providing a few examples of target input-output pairs in the prompt to guide the model's response formatting. β (correct answer)
- C. Running the model on multiple GPUs.
- D. Training the model weights on a small dataset.
Explanation: Few-shot prompting demonstrates structural formats, aligning the output style dynamically.
Question 30: Which company developed the Transformer architecture in 2017?
- A. OpenAI
- B. Google (Google Brain team) β (correct answer)
- C. Meta
- D. Microsoft
Explanation: The paper 'Attention Is All You Need' by Google Brain introduced the Transformer architecture.