AI Engineer Test

You are developing a recommendation system for a book platform. Below is the video showing the actions of a new user interacting with the site: Based on user behavior, which type of recommendation filtering method would be the most appropriate to use?

A law firm wants to use an LLM to work with its proprietary documents, which analysts will use for their queries.

An AI consultancy has proposed RAG to complete the requirements.

Select the correct pipeline steps corresponding to labels A, B, C, and D in the RAG pipeline below:

A) ______________________________

B) ______________________________

C) ______________________________

D) ______________________________

The CEO of your company understands that sometimes things go wrong. No system is perfect, but she wants to ensure that the company is improving.

Which metrics could be used to track the company's incident management?

A platform that schedules and records video game matches has an on-premise database of several terabytes in size. The database contains match results, players, recordings of best plays, and scheduled matches.

Analytic queries are taking too long to execute, so the CTO wants to change the data architecture of the platform and has asked you to provide suggestions.

Considering that:

• Platform users scheduling matches generate a high volume of INSERT operations.
• Analytic reports don't need real time data, and they can tolerate significant latency.
• Videos are generated automatically for each match, and are stored as BLOBs in the database.

Where should data be stored to satisfy the points above while having good performance for user and analytics operations?

The following .csv file contains the data from a classifier model that predicts if an image contains a dog: predictions.csv 

The first column contains information if the dog is in the image or not. The second column contains the classifier prediction, which is in the interval 0-100, with higher values meaning that the classifier is more confident that image contains a dog.

What is the value of the decision boundary that will maximize the accuracy of the model? Values greater than or equal to the decision boundary will be treated as positive.

The marketing team of the telecommunications company launched an SMS campaign to encourage customers to make more countrywide calls. The manager of the team asked you to analyze the results of the campaign.  Your main goal is to predict which customers are most likely to increase their number of calls as a result of similar SMS campaigns in the future. You have created a Random Forest model but it has problems with fitting the actual training dataset.

Which of the following steps can be taken to reduce underfitting and improve the model’s performance?

Your company has been working to increase its frequency of deployment for new features by optimizing its full development and operating lifecycle. The CEO wants to cut unnecessary phases to deliver new features to the customer.

Which phases are not part of the core DevOps lifecycle?

As a data engineer at an electronic components factory, you have been tasked with improving the performance of the factory's information database. The database is experiencing slow read operations, and the factory plans to add several new servers to help with the problem.

The database includes several critical tables, including the ones mentioned below along with their related issues:

• Orders: The table is read-heavy and it's where recent orders are accessed most frequently, and has become a bottleneck due to its size.
• QualityControl: The table has numerous columns, many of which are rarely queried causing unnecessary data processing during reads.

What can be done to improve the mentioned issues while taking advantage of new servers?

A fintech company uses a non-reasoning LLM to support faster responses and lower costs for high-volume fraud analysis.

Analyst's input to the LLM:

• Transaction metadata (amount, geodata, device, etc.)
• User history

Output from the LLM:

• Short explanation
• Decision: Approve / Block / Escalate

During peak traffic, analysts refreshing the review screen often trigger a second evaluation of the same transaction ID, which returns a different recommendation.

Which changes would most directly ensure deterministic outputs for the same transaction?

A SAAS company reported a 20% churn rate (the rate at which customers stop subscribing to a service ) in the last quarter, which is much larger than the expected 12%. Your manager asked you to create a model that can predict the churners of the next 12 months in advance. With the help of this model, the marketing team will launch churn prevention campaigns to reduce the churn rate.

You decided to use a Gradient Boosting model. The parameters used in the model are the following:

• Learning rate = 0.3
• Maximum depth of a tree = 6
• Minimum sum of instance weight needed in a child = 1
• Subsample = 1
• L1 regularization term on weights = 0
• L2 regularization term on weights = 1

Currently the model has a problem, the training set has more than 95% accuracy but the test set has only 60% accuracy.

Which of the following steps can help solve the issue?

A classifier that predicts if an image contains only a cat, a dog, or a llama produced the following confusion matrix:

What is the accuracy of the model, in percentages?