CT-GenAI ISTQB Certified Tester Testing with Generative AI (CT-GenAI) v1.0 Questions and Answers

The environmental impact and sustainability of AI are increasingly important considerations in software engineering. The overall energy consumption of an LLM during inference (when the model is actually being used by a tester) is most directly influenced by thenumber of tokens processed. Every token generated or analyzed requires a massive amount of floating-point operations within the GPU clusters of a data center. Therefore, the "length" of the input prompt and the "length" of the AI’s response are the primary drivers of the power draw and, consequently, the carbon intensity of the query. This is a crucial concept for "Green AI" initiatives in testing; more efficient prompting—such as avoiding unnecessarily verbose context or limiting output lengths—can lead to more sustainable testing practices. While data center location (Option B) affects thetypeof energy used (renewable vs. fossil fuel), it does not determine the model's accuracy. Similarly, while cloud platforms (Option D) and session durations (Option C) play roles in operational logistics, the mathematical workload tied to token count remains the fundamental unit of energy expenditure in Generative AI.

In the context of LLM inference,Temperatureis a hyperparameter that controls the randomness or "creativity" of the model's output. When the temperature is set high, the model's probability distribution is "flattened," meaning it is more likely to select less-probable tokens, leading to more diverse and sometimes unpredictable text. For software testing, where precision and repeatability are paramount,lowering the temperature(Option C) is the standard practice. A temperature of 0.0 makes the model "deterministic," meaning it will consistently choose the token with the highest probability. This narrows the sampling distribution and significantly reduces variability between runs. While a larger context window (Option D) allows the model to process more information, it does not directly control the randomness of token selection. Similarly, the "learning rate" (Option B) is a parameter used during thetrainingorfine-tuningphase, not during inference. For generating test cases or scripts that must follow strict logic, a lower temperature ensures that the model remains focused and produces consistent results.

Generative AI is exceptionally capable of creating structured and unstructured data, but its role is limited to "generation" and "transformation," not infrastructure management or direct database administration. Creating production database backups (Option A) is a physical data management task involving the copying of actual stateful data from a server to storage; this is handled by database management systems (DBMS) and DevOps pipelines, not LLMs. Conversely, LLMs excel at the logic-based tasks listed in the other options. They can analyze requirements to identify and set boundary values (Option B) for input validation. They are also highly effective at creating combinatorial data (Option C), such as pairwise or all-combinations tables, by understanding the relationships between variables. Finally, one of the most powerful uses of GenAI in testing is generating synthetic datasets (Option D)—creating "fake" but realistically structured data that mimics production patterns without exposing Sensitive Personally Identifiable Information (SPII), thereby supporting privacy-compliant testing.

While a basic chatbot is primarily designed for textual interaction and information retrieval, anLLM-powered agent(or AI Agent) is characterized by itsagency—the ability to use tools and trigger actions in the external world. In a software testing context, an agent does not just "talk" about testing; it can actually perform testing tasks. For example, an agent could be given the goal to "verify the login module," and it would independently decide to call an API, generate a test script, execute it against a test environment, and then analyze the results to report a bug in Jira. This ability totrigger automated actions(Option C) through "function calling" or tool integration is what makes agents far more powerful than simple conversational interfaces (Option D). Agents can reason about "how" to achieve a goal, selecting the appropriate tools (like Selenium, Postman, or specialized internal utilities) to complete the task. This moves the AI from being a passive advisor to an active participant in the test automation ecosystem, requiring testers to focus more on goal definition and result validation.

The statement that fine-tuning "replaces the model's general knowledge entirely" isincorrect. Fine-tuning is a process of "incremental learning" where a pre-trained model (which already possesses vast general knowledge) is further trained on a smaller, domain-specific dataset—such as an organization's internal API documentation or historical test scripts. The goal is to adjust the model's internal weights so that it becomes more proficient in a specific area (Option A) and adheres better to local terminology and formatting standards (Option C). It doesnoterase the foundational language capabilities of the model. Furthermore, fine-tuning is a common strategy for Small Language Models (SLMs) to allow them to punch above their weight class in specific tasks while remaining computationally efficient (Option D). However, if done poorly, fine-tuning can actuallycauseoverfitting (where the model becomes too rigid and loses its ability to generalize), rather than preventing it. Therefore, fine-tuning should be viewed as a "specialization" layer rather than a total replacement of the model's base intelligence.

Tokenizationis the foundational process by which an LLM breaks down raw text into smaller, manageable units called "tokens." These tokens can represent individual words, parts of words (sub-words), or even punctuation marks. This is a critical step because LLMs do not "read" words like humans do; they process numerical representations of these tokens. The way text is tokenized directly impacts the model's efficiency and its ability to understand complex technical terminology used in software testing. For example, a rare technical term might be broken into several sub-word tokens. This process is closely linked to theContext Window(Option C), which is the maximum number of tokens a model can "remember" or process at one time. WhileEmbeddings(Option B) are the numerical vectors that represent the meaning of these tokens, and theTransformer(Option A) is the underlying architecture that processes them, tokenization is the specific mechanism for initial text decomposition. Understanding tokenization is vital for testers when managing long requirement documents to ensure they do not exceed the model's limits.

Few-shot promptingis the technique of providing a few examples (exemplars) within the prompt to demonstrate the desired task and output format to the LLM. In this scenario, providing 10 existing, high-quality test cases acts as a "pattern" for the model to follow. This is significantly more effective than "Zero-shot prompting" (Option D), where the model is given a task without examples and may deviate from the specific organizational format required (e.g., specific JSON structures or assertion styles). While "Prompt chaining" (Option A) is useful for breaking down complex tasks into sub-tasks, the primary need here is pattern recognition and replication, which is the core strength of Few-shot learning. "Meta prompting" (Option C) involves having the AI write the prompt itself, which is unnecessary when the team already has clear examples. By using Few-shot prompting, the tester "conditions" the model's latent space to prioritize the provided format, ensuring that all 500 generated test cases maintain consistency with the HTTP methods, headers, and assertion logic defined in the exemplars.

ISO/IEC 42001:2023is the international standard for an AI Management System (AIMS). It is designed to help organizations develop, provide, or use AI systems responsibly by providing a certifiable framework of requirements and controls. In a software testing context, this standard is vital for establishing governance, ensuring that GenAI tools are used consistently and ethically across the lifecycle.NIST AI RMF 1.0(Option B) is a highly respected framework, but it is a set of voluntary guidelines for managing risk, not a "requirement standard" for a management system.ISO/IEC 23053:2022(Option C) provides a general framework for AI using machine learning but lacks the comprehensive "management system" scope found in 42001. Finally, theEU AI Act(Option D) is a regulation (law), not a technical standard. For a test organization looking to align its GenAI strategy with international best practices and achieve formal certification, ISO/IEC 42001 is the definitive standard to follow, as it covers the organizational processes, data handling, and risk management necessary for high-quality AI operations.

This scenario describes a form ofRequest Manipulation, specifically a type of "Prompt Injection" or "Adversarial Prompting." In this attack vector, the user (or an external attacker) provides malicious or deceptive input—in this case, via an image in a multimodal LLM—to bypass the model's intended constraints or to steer its logic toward an unintended outcome. By crafting an image that tricks the LLM into seeing "acceptance criteria" that aren't actually there, the attacker manipulates the model’s request processing to generate false validation results. This is different fromData Poisoning(Option A), which involves corrupting the training data before the model is even built. It is also distinct fromData Exfiltration(Option B), which aims to steal data from the model. In a testing environment, request manipulation is a significant risk because it can lead to "Silent Failures," where the AI reports that tests have passed or requirements are met based on deceptive input, thereby compromising the integrity of the entire Quality Assurance process.