AIP-210 CertNexus Certified Artificial Intelligence Practitioner (CAIP) Questions and Answers

Model validation is the process of checking whether the model meets the specified requirements and quality standards. It involves testing the model on a validation dataset, which is different from the training and testing datasets, and evaluating the model performance using appropriate metrics. References: Overview of ML Pipelines | Machine Learning, MLOps: Continuous delivery and automation pipelines in machine learning

Latency is the time delay between a request and a response. Latency can affect the performance and user experience of an application, especially when real-time or near-real-time responses are required. Deploying a deep learning model as an embedded model on edge devices can reduce latency, as the model can run locally on the device without relying on network connectivity or cloud servers. Edge devices are devices that are located at the edge of a network, such as smartphones, tablets, laptops, sensors, cameras, or drones.

Caffe is a deep learning framework that is designed for speed and modularity. It can process large datasets of images efficiently and supports various types of neural networks. It also has a large and active developer community that contributes to its code base and documentation. Caffe is suitable for image processing tasks such as classification, segmentation, detection, and recognition

Semi-supervised learning is an approach that is best if a limited portion of your training data is labeled. Semi-supervised learning is a type of machine learning that uses both labeled and unlabeled data to train a model. Semi-supervised learning can leverage the large amount of unlabeled data that is easier and cheaper to obtain and use it to improve the model’s performance. Semi-supervised learning can use various techniques, such as self-training, co-training, or generative models, to incorporate unlabeled data into the learning process.

A handover is the process of transferring the ownership and responsibility of an ML system from one party to another, such as from the developers to the end users. A handover should include all the necessary information and resources that enable the end users to use and run a trained model on their own system. Some of the items that should be included in a handover are:

Link to a GitHub repository of the codebase: A GitHub repository is an online platform that hosts the source code and version control of an ML system. A link to a GitHub repository can provide the end users with access to the latest and most updated version of the codebase, as well as the history and documentation of the changes made to the code.

README document: A README document is a text file that provides an overview and instructions for an ML system. A README document can include information such as the purpose, features, requirements, installation, usage, testing, troubleshooting, and license of the system.

Sample input and output data files: Sample input and output data files are data files that contain examples of valid inputs and expected outputs for an ML system. Sample input and output data files can help the end users understand how to use and run the system, as well as verify its functionality and performance.

Stratification is not a valid cross-validation method, but a technique to ensure that each subset of data has the same proportion of classes or labels as the original data. Stratification can be used in conjunction with cross-validation methods such as k-fold or leave-one-out to preserve the class distribution and reduce bias or variance in the validation results. Bootstrapping, k-fold, and leave-one-out are all valid cross-validation methods that use different ways of splitting and resampling the data to estimate the performance of a machine learning model.

Performance testing is a type of testing that should be performed at the production level before deploying a newly retrained model. Performance testing measures how well the model meets the non-functional requirements, such as speed, scalability, reliability, availability, and resource consumption. Performance testing can help identify any bottlenecks or issues that may affect the user experience or satisfaction with the model. References: [Performance Testing Tutorial: What is, Types, Metrics and Example], [Performance Testing for Machine Learning Systems | by David Talby | Towards Data Science]

Lasso regression is a type of linear regression that adds a regularization term to the loss function to reduce overfitting and improve generalization. Lasso regression uses an L1 norm as the regularization term, which is the sum of the absolute values of the coefficients. Lasso regression can shrink some of the coefficients to zero, which effectively eliminates some of the features from the model. Lasso regression is preferable over ridge regression when there are many features with no association with the dependent variable, as it can perform feature selection and reduce the complexity and noise of the model.

The Adversarial ML Threat Matrix is an open framework designed to help detect, respond to, and remediate threats in ML systems. The Adversarial ML Threat Matrix is inspired by the MITRE ATTandCK® Matrix1, which is a framework for describing cyberattacks across various stages of an attack lifecycle. The Adversarial ML Threat Matrix adapts this framework to address specific threats and vulnerabilities in ML systems, such as data poisoning, model stealing, model evasion, or model inversion2. The Adversarial ML Threat Matrix provides a structured way to organize and classify adversarial techniques, tactics, procedures, examples, and mitigations for ML systems2.

Data privacy is the right of individuals to control how their personal data is collected, used, shared, and protected. It also involves complying with relevant laws and regulations that govern the handling of personal data. Data privacy is especially important when extracting business intelligence from primary data captured from the public, as it may contain sensitive or confidential information that could harm the individuals if misused or breached .

 Transparency is an ethical principle that describes the degree to which an AI system can provide clear and understandable information about its inputs, outputs, processes, and decisions. Transparency can help increase trust and confidence among users and stakeholders, as well as enable accountability and responsibility for the system’s actions and outcomes. Logging and storing historical predictions for potential, future access requests is an example of transparency, as it can help provide evidence and explanation for the system’s recommendations, as well as facilitate auditing and feedback.

Cloud models are service models that provide different levels of abstraction and control over computing resources in a cloud environment. Some of the common cloud models for machine learning pipelines are:

Software as a Service (SaaS): SaaS provides ready-to-use applications that run on the cloud provider’s infrastructure and are accessible through a web browser or an API. SaaS can provide AI practitioner data science services such as Jupyter notebooks, which are web-based interactive environments that allow users to create and share documents that contain code, text, visualizations, and more.

Platform as a Service (PaaS): PaaS provides a platform that allows users to develop, run, and manage applications without worrying about the underlying infrastructure. PaaS can provide some services within an application such as payment applications to create efficient results.

Infrastructure as a Service (IaaS): IaaS provides access to fundamental computing resources such as servers, storage, networks, and operating systems. IaaS can provide CPU, memory, disk, network and GPU resources that can be used to run machine learning models and applications.

Data as a Service (DaaS): DaaS provides access to data sources that can be consumed by applications or users on demand. DaaS can host the databases providing backups, clustering, and high availability.

Data skews happen in the ML pipeline when the distribution or characteristics of the live input data differ from those of the offline data used for training and testing the model. This can lead to a degradation of the model performance and accuracy, as the model is not able to generalize well to new data. Data skews can be caused by various factors, such as changes in user behavior, data collection methods, data quality issues, or external events. References: What is training-serving skew in Machine Learning?, Data preprocessing for ML: options and recommendations

Personal data is any information that relates to an identified or identifiable individual, such as name, address, email, phone number, or biometric data. Personal data should not be disclosed, made available, or otherwise used for purposes other than specified, except with:

The consent of the person it is collected from: Consent is a clear and voluntary indication of agreement by the person to the processing of their personal data for a specific purpose. Consent can be given by a statement or a clear affirmative action, such as ticking a box or clicking a button.

The authority of law: The authority of law is a legal basis or obligation that requires or permits the processing of personal data for a legitimate purpose. For example, the authority of law could be a court order, a subpoena, a warrant, or a statute.

The transform-design pattern for ML pipelines aims to separate inputs from features, encapsulate the processing steps of ML pipelines, and represent steps in the pipeline with a DAG. These goals help to make the pipeline modular, reusable, and easy to understand. The transform-design pattern does not seek to isolate individual steps of ML pipelines, as this would create entanglement and dependency issues. It also does not transform the output data after production, as this would violate the principle of separation of concerns. 

Underfitting is a problem that occurs when a model learns too little from the training data and fails to capture the underlying complexity or structure of the data. Underfitting can result from using insufficient or irrelevant features, a low complexity of the model, or a lack of training data. Underfitting can reduce the accuracy and generalization of the model, as it may produce oversimplified or inaccurate predictions. Some of the ways to fix underfitting are:

Add features to training data: Adding more features or variables to the training data can help increase the information and diversity of the data, which can help the model learn more complex patterns and relationships.

Increase the complexity of the model: Increasing the complexity of the model can help increase its expressive power and flexibility, which can help it fit better to the data. For example, adding more layers or nodes to a neural network can increase its complexity.

Train the model for more epochs: Training the model for more epochs can help increase its learning ability and convergence, which can help it optimize its parameters and reduce its error.

Getting more training data will not work to fix underfitting, as it will not change the complexity or structure of the data or the model. Getting more training data may help with overfitting, which is when a model learns too much from the training data and fails to generalize well to new or unseen data.

Entanglement in ML pipelines occurs when a change in one step affects other steps that depend on it. Changing the normalization function in the feature engineering step would affect the model training and evaluation steps, as they rely on the features generated by the feature engineering step. Therefore, this scenario is an example of entanglement in ML pipelines. The other scenarios are not examples of entanglement, as they do not affect other steps in the pipeline.

SVM models can use kernel functions to map the input data into higher-dimensional feature spaces, where linear separation is possible. This allows SVM models to handle non-linear problems effectively. References: CertNexus Certified Artificial Intelligence Practitioner, Support vector machine - Wikipedia

Harms modeling is a technique that helps product managers design AI solutions responsibly by evaluating both positive and negative outcomes. Harms modeling involves creating a shared taxonomy of potential negative impacts and conducting an assessment along vectors such as severity, impact, frequency, and likelihood. Harms modeling can help identify and mitigate any risks or harms that may arise from using AI solutions. References: [Harms Modeling for Responsible AI | by Google Developers | Google Developers], [Harms Modeling for Responsible AI - YouTube]

Data lineage is the process of tracking the origin, transformation, and usage of data throughout its lifecycle. It helps to ensure data quality, integrity, and provenance. Data lineage also supports the Privacy by Design methodology, which is a framework that aims to embed privacy principles into the design and operation of systems, processes, and products that involve personal data. By understanding, documenting, and displaying data lineage, an ML system can demonstrate how it collects, processes, stores, and deletes personal data in a transparent and accountable manner3 .

The optimal value of K is the one that minimizes the inertia or within-cluster sum of squares, while avoiding too many clusters that may overfit the data. The elbow plot shows a sharp decrease in inertia from K = 1 to K = 2, and then a more gradual decrease from K = 2 to K = 3. After K = 3, the inertia does not change much as K increases. Therefore, the elbow point is at K = 3, which is the optimal value of K for this data. References: How to Run K-Means Clustering in Python, K-means clustering - Wikipedia

Unsupervised learning is a type of machine learning that involves finding patterns or structures in unlabeled data without any predefined outcome or feedback. Unsupervised learning can be used for various tasks, such as clustering, dimensionality reduction, anomaly detection, or association rule mining. Some of the common algorithms for unsupervised learning are:

Principal components analysis: Principal components analysis (PCA) is a method that reduces the dimensionality of data by transforming it into a new set of orthogonal variables (principal components) that capture the maximum amount of variance in the data. PCA can help simplify and visualize high-dimensional data, as well as remove noise or redundancy from the data.

K-means clustering: K-means clustering is a method that partitions data into k groups (clusters) based on their similarity or distance. K-means clustering can help discover natural or hidden groups in the data, as well as identify outliers or anomalies in the data.

Apriori algorithm: Apriori algorithm is a method that finds frequent itemsets (sets of items that occur together frequently) and association rules (rules that describe how items are related or correlated) in transactional data. Apriori algorithm can help discover patterns or insights in the data, such as customer behavior, preferences, or recommendations.

Normalization is a method that can help us to solve the problem of features being on different scales when using the k-nearest neighbors algorithm. Normalization is a technique that rescales the values of features to a common range, such as [0, 1] or [-1, 1]. Normalization can help reduce the influence or dominance of some features over others, as well as improve the accuracy and performance of the algorithm2.

Federated learning is a distributed approach to machine learning that allows multiple parties to collaboratively train a model without sharing their data with each other or a central server. This protects the privacy of the user’s data while still enabling well-trained models that can benefit from diverse and large-scale datasets. References: [Federated Learning - Wikipedia], [Federated Learning for Mobile Keyboard Prediction - Google AI Blog]