Lead Software Engineer @ Viruslab Systems

Our team invests countless hours in the exploration, discovery and investigation of emerging trends such as AI and futuristic technology. Stay informed about the evolving landscape of cyberthreats through our diligent efforts and insights.

Security Evangelist at Gen

Threat Operations Analyst at Gen

Research

A fake invoice in your email is easy to ignore. A fake invoice inside your order history feels different.&nbsp;Norton customers have reported fake Norton invoices appearing inside the Shop app, the shopping and order-tracking app from Shopify. Public reports suggest the same technique is not limited to Norton. Similar suspicious Shop app notifications have used McAfee, Apple gift cards, iPhones, PayPal-style payment claims and other high-value purchases as bait. The impersonated brand may change, but the mechanics are familiar: make the user believe they have been charged, then give them a phone number to call.&nbsp;That phone number is where the scam really begins.&nbsp;<h4>Why the Shop app makes this more believable&nbsp;</h4>Shop is a legitimate app. People use it to track packages, check receipts and receive order updates. According to Shop’s own documentation, the app can automatically track orders from several sources, including Gmail, Outlook, the email address linked to the user’s Shop account and orders paid through Shop Pay. If Gmail or Outlook is connected, Shop says it scans recent emails for keywords such as “tracking number” and “track your package” to populate deliveries in the Orders tab.&nbsp;

Shop also supports push notifications for delivery updates, payments and favorite products. So the first alert the victim sees may not be an email, an SMS or a calendar invite. It may be a notification from an app they already use for real purchases.&nbsp;That context matters. The scam text is not especially polished. Some of the examples we reviewed contain awkward wording and obvious signs of fraud. But when the message appears inside an order-tracking app, surrounded by real receipts and shipping updates, users may give it more credibility than it deserves.&nbsp;The screenshots reported to us show fake orders under generic seller names such as “My Store.” The receipts claim that a Norton or Norton LifeLock subscription has been activated or renewed, usually for several hundred dollars. The phone number appears in the product description, the receipt body or even the shipping address field.&nbsp;That is not how a normal receipt should look.&nbsp;<h4>The brand is interchangeable&nbsp;</h4>Norton is the example we saw through customer reports. Norton support now warns that scammers may abuse Android and iOS shopping platforms or applications to generate fake order notifications that appear to be legitimate Norton purchases. Those notifications may include fake order confirmations, fake charges, trusted third-party app delivery and fake support phone numbers embedded in order details.&nbsp;Public user reports point to a wider pattern. One Reddit user described receiving a Shop.app message about a transaction they did not recognize, with no follow-up email confirmation, and a message telling them to call support if the order was not placed by them. Another thread includes reports of unrecognized iPhone purchases under “My Store,” suspicious contact emails and users saying they saw no matching charges on their bank cards. Other reports mention fictitious Apple gift card purchases and iPhone orders using the same kind of wording: “If Order Not Placed By You,” followed by a phone number.&nbsp;The choice of brand is part of the social engineering. A fake security subscription charge plays on two fears at once: being billed for something you did not buy, and being left unprotected if you ignore it. Apple products and gift cards create high-value purchase panic. PayPal-style wording makes users worry about payment account abuse. In each case, the fake order is there to push the victim toward a call.&nbsp;<h4>What we know, and what we do not&nbsp;</h4>We have not found any evidence that any of the impersonated brands have been compromised.&nbsp;&nbsp;We also should not claim that Shop or Shopify was breached based on the evidence we have. The exact abuse path still needs confirmation. The fake order may be entering the app through a merchant/order workflow, through email parsing, through account association, or through another legitimate mechanism that scammers have found a way to misuse. Shop’s own documentation shows that orders can populate from multiple sources, so assuming one route would be premature.&nbsp;&nbsp;What we can say is narrower and stronger: scammers are placing fake purchase claims inside shopping app order experiences, where users are used to seeing real receipts and order updates. Norton’s own support page now describes this class of shopping app scam, and public reports show similar lures using other brands and products.&nbsp;&nbsp;That is enough to warn users without overstating the technical evidence.&nbsp;<h4>How the scam works&nbsp;</h4>The user receives a notification or sees a new order in the Shop app. The order looks like a receipt for something expensive: a security subscription, an Apple gift card, an iPhone, a MacBook or another high-value purchase.

The receipt says the charge has been processed, the subscription has renewed, or the order is being prepared. Somewhere in the order details, the scammer inserts a phone number, email address or support instruction.&nbsp;The wording is often clumsy. We have seen phrases like “If Order Not Place By You” and “If Need Help.” Those errors may look obvious when reviewing a screenshot. They are easier to miss when a phone notification tells you that hundreds of dollars have just been charged.&nbsp;If the victim calls, the scam moves off-platform. The person answering may claim to be billing support, Norton support, PayPal support or a cancellation department. The script can vary, but the request usually moves toward credentials, payment data, one-time codes or remote access software.&nbsp;By that point, the fake receipt has done its job. It has created enough urgency for the victim to enter a conversation controlled by the scammer.&nbsp;<h4>Why this is harder for users to judge&nbsp;</h4>Most people understand that email can be spam. They may still fall for phishing, but at least the inbox is a place where scams are expected.&nbsp;Order-tracking apps are different. Their purpose is to collect receipts, shipping updates and purchase information in one place. A notification from that environment can feel as if it has already passed through some layer of trust, even when the content inside the order is fraudulent.&nbsp;We have seen the same logic with calendar invite scams. The message itself is not always convincing, but the delivery channel changes how people read it. A fake invoice in an inbox is one thing. A fake invoice inside a calendar reminder or shopping app receipt can feel closer to a real event.&nbsp;For security products, this also creates a detection problem. The first visible alert may not be an email, SMS or malicious website. It may be a legitimate app notification carrying fraudulent text. The scam still depends on social engineering, but the delivery surface is harder to classify with the usual rules.&nbsp;<h4>What to do if you see one&nbsp;</h4>Do not call the number in the receipt.&nbsp;Check the charge directly. Open your bank or credit card app. Open the official account for the brand being impersonated. If it claims to be Norton, go to your Norton account or official Norton support. If it claims to involve PayPal, open PayPal directly. Do not use the phone number, email address or link inside the suspicious order.&nbsp;If there is no matching charge, treat the receipt as a scam attempt.&nbsp;Report the suspicious store or message in the Shop app if the option is available. Shop’s help pages say suspicious Shop stores can be reported from the store page through the app, and Shopify also tells users to forward phishing messages to its phishing inbox.&nbsp;&nbsp;Norton users can forward suspicious messages to Norton and should verify unexpected charges only through their Norton account or official Norton support channels.&nbsp;&nbsp;Opening the receipt does not automatically mean your device was compromised. The risk increases if you call the number, share information, install software or follow instructions from the person on the phone.&nbsp;<h4>What to do if you have already called&nbsp;</h4>End the call.&nbsp;If you shared payment details, contact your bank or card provider immediately using the number on your card or inside your banking app. If you gave away a password, change it from a different device if possible. Start with your email account, Apple ID or Google account, banking account and any account connected to the fake invoice.&nbsp;If you installed remote access software, disconnect the device from the internet and remove the software. Run a security scan before using the device again for banking, email or shopping.&nbsp;If you gave a one-time code to the caller, treat the related account as compromised. Change the password, review recovery options and check recent login activity.&nbsp;<h4>The lesson&nbsp;</h4>Scammers keep testing new places to put old tricks. For years, fake invoices were mostly an email problem. Then similar subscription notices appeared in calendar invites. Now we are seeing fake receipts inside shopping app order histories and push notification flows.&nbsp;The safest response is to separate the alert from the action. If an app, email, text message or calendar invite says you have been charged, do not use the contact details inside that alert. Go directly to the account or payment provider yourself. Check whether the charge exists. Use official support channels.&nbsp;The fake receipt, the fake support number and the fake emergency are all part of the same controlled environment. The moment you verify the charge somewhere else, the scam loses most of its power.&nbsp;

Scammers are using order-tracking apps to place fake receipts where users expect to see real purchases, then pushing them to call fake support numbers. 

Fake invoices are moving from inboxes to shopping apps 

Threat Researcher at Gen

<h2>Introduction&nbsp;</h2>Infostealers are constantly evolving, and so are the techniques they use to bypass <a href="https://security.googleblog.com/2024/07/improving-security-of-chrome-cookies-on.html" target="_blank" rel="noreferrer noopener">Application-Bound Encryption (ABE)</a>. In recent weeks, Vidar has been among the most actively developed stealers and, apart from <a href="https://x.com/GenThreatLabs/status/2064836539952361976" target="_blank" rel="noreferrer noopener">multiple updates to its string obfuscation</a> and <a href="https://x.com/GenThreatLabs/status/2061462524604649840" target="_blank" rel="noreferrer noopener">a reworked approach to protecting its configuration</a>, it has also introduced a novel technique for bypassing ABE. And while there have been many other changes in Vidar lately, with new versions dropping every week, in this blog post we focus solely on the ABE bypass and its technical aspects.&nbsp; &nbsp;<h2>The ABE Bypass&nbsp;</h2>From a high-level perspective, Vidar’s approach to bypassing ABE is somewhat similar to that of Remus/Lumma, in that both try to extract the <code>v20_master_key</code> directly from the browser’s memory. What sets them apart, however, is how they achieve it. Since we’ve already covered the inner workings of ABE in earlier posts (<a href="https://www.gendigital.com/blog/insights/research/remus-64bit-variant-of-lumma-stealer" target="_blank" rel="noreferrer noopener">Remus</a>, <a href="https://www.gendigital.com/blog/insights/research/voidstealer-abe-bypass" target="_blank" rel="noreferrer noopener">VoidStealer</a>), we don’t revisit them here and focus instead on the bypass itself. For readers interested in the broader background, we encourage checking those earlier posts.&nbsp;&nbsp;In short, the end goal of infostealers is to obtain the <code>v20_master_key</code>, as it alone is sufficient for decrypting any ABE-protected data tied to a specific application. This key can be obtained in one of two ways: either from disk, where it is protected by two layers of <code>CryptProtectData</code> (one applied in the context of the logged-in user and one as <code>NT AUTHORITY\SYSTEM</code>), or from the browser’s memory – the direction Vidar took.&nbsp;In memory, however, the key is kept protected with <code>CryptProtectMemory</code> using the <code>CRYPTPROTECTMEMORY_SAME_PROCESS</code> flag. This ensures that even if an attacker manages to read the encrypted form of the key from memory, they cannot simply decrypt it (they would need to invoke the complementary <code>CryptUnprotectMemory</code> function directly from within the browser process). That leaves attackers with two problems to solve: how to find the encrypted key in memory, and how to get it decrypted.&nbsp;<h2>Locating the v20_master_key in Browser’s Memory&nbsp;</h2>Vidar’s approach to locating the encrypted <code>v20_master_key</code> in memory involves scanning the entire memory of a running browser for a specific pattern. But that first requires a browser process to scan.&nbsp;When the target browser is already running, Vidar reuses the existing process. However, it doesn’t read its memory directly. Instead, it creates a fork of it using <code>NtCreateProcessEx</code> with <code>SectionHandle</code> set to <code>NULL</code>, instructing the kernel to clone the address space of the process specified by the <code>ParentProcess</code> as a copy-on-write mapping. The resulting process has no threads and is never resumed – it exists purely as a static memory snapshot.&nbsp;&nbsp;To perform the fork, Vidar first opens a handle to the target browser via <code>OpenProcess</code> with <code>PROCESS_QUERY_INFORMATION | PROCESS_CREATE_PROCESS | PROCESS_VM_READ</code>, falling back to <code>PROCESS_CREATE_PROCESS</code> if the initial call fails. It then passes this handle as the <code>ParentProcess</code> argument to <code>NtCreateProcessEx</code>, requesting <code>PROCESS_ALL_ACCESS</code> on the resulting child. If the fork fails, Vidar falls back to scanning the live browser process directly.&nbsp;

If no target browser process is found running, Vidar creates a new isolated desktop via <code>CreateDesktopA</code>, assigns it a name of the form <code>v20_%d</code> (where <code>%d</code> is replaced with the browser’s index in Vidar’s configuration), and launches the target browser in there. In the case of Vidar version 2.1, the browser is launched with the command line <code>--no-first-run --disable-gpu about:blank</code>, although the exact command line varies across versions. Since modern browsers use a multi-process architecture, there are typically multiple processes matching a given browser. Vidar enumerates all of them (up to 64) and applies the fork-and-scan procedure to each one independently.&nbsp;

With a handle to the forked process in hand, Vidar proceeds to enumerate its virtual memory regions using <code>NtQueryVirtualMemory</code>. It iterates through the entire browser’s address space, collecting regions that match a specific profile: committed (<code>MEM_COMMIT</code>), private (<code>MEM_PRIVATE</code>), and either readable (<code>PAGE_READONLY</code>) or read-write (<code>PAGE_READWRITE</code>). Vidar records up to 4096 memory regions matching these requirements, along with their base addresses and sizes. The actual pattern scanning is then performed in parallel with up to 64 worker threads, depending on the system.&nbsp;&nbsp;

Each thread scans its assigned regions using <code>NtReadVirtualMemory</code>, searching for a 32-byte pattern. When a match is found, the thread reads 8 bytes at offset +32 from the match, interprets them as a pointer, and validates that it points to committed, private, and readable memory (the expected memory state for the encrypted <code>v20_master_key</code>).&nbsp;&nbsp;

This naturally raises the question of what pattern Vidar searches for. The answer is a 32-byte signature: <code>76 32 30 00 ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? 03 00 00 00 00 01 ?? ?? ??</code>, where <code>??</code> denotes a wildcard (Vidar uses the byte <code>AA</code> as the wildcard marker in its internal representation). Notably, the first four bytes correspond to the ASCII <code>v20\x00</code>, which is a common prefix used for ABE-related data.&nbsp;&nbsp;

We assess with high confidence that this pattern is intended to match a node structure within Chromium’s <code>Encryptor::KeyRing</code>, the structure that holds the keys used by the <code>Encryptor</code> class. Now, this part is admittedly a bit tricky as it relies on several assumptions we made about the internal memory layout, as well as on implementation-specific representations of various C++ standard library types <code>(std::map</code>, <code>std::string</code>, <code>std::vector</code>, <code>std::optional</code>), which is why we frame this as a high-confidence rather than a certainty.&nbsp;&nbsp;Our reasoning is as follows: <code>KeyRing</code> is <a href="https://github.com/chromium/chromium/blob/cf5d502bcaedb22927e3829a18be1ff6d8b6f598/components/os_crypt/async/common/encryptor.h#L132" target="_blank" rel="noreferrer noopener">defined</a> as <code>std::map&lt;/*tag=*/std::string, std::optional&lt;Key&gt;&gt;</code>, and <code>std::map</code> is typically implemented as a red-black tree, where each tree node stores three pointers (left child, parent, and right child) followed by the key-value pair. In this case, the pair is in the form of <code>std::pair&lt;/*tag=*/std::string, std::optional&lt;Key&gt;&gt;</code>, meaning that the <code>std::string</code> and the <code>Key</code> object might be laid out consecutively in memory within each node. For a <code>v20_master_key</code> entry, the tag corresponds to the string <code>v20</code>, which at 4 bytes including the null terminator fits within the small string optimization (SSO) buffer and is therefore stored inline rather than heap-allocated. The <code>Key</code> class that immediately follows <a href="https://github.com/chromium/chromium/blob/cf5d502bcaedb22927e3829a18be1ff6d8b6f598/components/os_crypt/async/common/encryptor.h#L93-L98" target="_blank" rel="noreferrer noopener">contains three member variables</a>: <code>std::optional&lt;mojom::Algorithm&gt; algorithm_</code>, <code>std::vector&lt;uint8_t&gt; key_</code>, and <code>bool encrypted_ = false</code>.&nbsp;Putting it all together and applying it to a concrete example from a browser memory dump, we get the following layout:&nbsp;

Please note that while the exact layout may contain minor inaccuracies on our part, the overall structure should be roughly correct. For easier orientation, the following diagram illustrates how the individual structures are nested within each other:&nbsp;

At this point, it should be clear how the scan pattern maps onto the searched structure. The 32-byte match covers the <code>std::string</code> tag <code>v20\x00</code> and the beginning of the <code>Key</code> object up to and including <code>algorithm_</code>. The pointer at offset +32 from the start of the match then lands precisely on <code>key_.begin</code>, the first member of <code>std::vector&lt;uint8_t&gt;</code> that holds the encrypted <code>v20_master_key</code>.&nbsp;Matches from all threads are collected into a shared buffer and subjected to a majority-voting scheme: candidates consisting mostly of zeroes are discarded, the rest are pairwise-compared, and each is scored by the number of identical copies found. This voting is performed independently for each browser process, producing one best candidate per process.&nbsp;<h2>Decrypting the v20_master_key via APC Injections&nbsp;</h2>For each browser process that produced a match, Vidar now holds a candidate address pointing to a potential encrypted <code>v20_master_key</code>. However, as already noted, since the key is protected with <code>CryptProtectMemory</code> using <code>CRYPTPROTECTMEMORY_SAME_PROCESS</code>, it can only be decrypted by invoking <code>CryptUnprotectMemory</code> from within the browser process itself. Vidar solves this by injecting an Asynchronous Procedure Call (APC) into the live browser process.&nbsp;Vidar supports two APC injection methods, selected based on whether ESET or Bitdefender is detected on the system. If either of these products is present, Vidar uses the first method, which it refers to as “classic” (based on decrypted strings) – it creates a suspended thread in the target browser via <code>CreateRemoteThread</code> (with <code>NtTestAlert</code> as the thread start address and the <code>CREATE_SUSPENDED</code> flag), queues an APC using <code>NtQueueApcThread</code>, resumes the thread, and then waits for it to complete via <code>WaitForSingleObject</code> with a randomized timeout. When the thread resumes, it calls <code>NtTestAlert</code>, which flushes the APC queue and executes the injected routine.&nbsp;

If neither product is detected, it uses the “special” method – it finds an existing thread in the target browser using a combination of <code>CreateToolhelp32Snapshot</code> and <code>Thread32First/Thread32Next</code>, opens it via <code>NtOpenThread</code>, and calls <code>NtQueueApcThreadEx2</code> with the <code>QUEUE_USER_APC_FLAGS_SPECIAL_USER_APC</code> flag. Unlike regular APCs, special user APCs execute immediately. Therefore, the target thread does not need to be in an alertable wait state.&nbsp;

In both cases, the APC routine to be executed is <code>CryptUnprotectMemory</code>, called with the following three arguments: the address of the encrypted <code>v20_master_key</code> candidate, its size (32 bytes), and <code>CRYPTPROTECTMEMORY_SAME_PROCESS</code> for the <code>flags</code> parameter.&nbsp;&nbsp;Since <code>CryptUnprotectMemory</code> decrypts the data in place, once the APC executes, the key is decrypted directly in the live browser's memory. To read it, Vidar creates a second fork of the browser process and reads the key from the same address using <code>NtReadVirtualMemory</code>. It then compares the result against a snapshot of the encrypted key taken from the first fork before the APC call – if the data is unchanged, decryption failed, and Vidar moves on to the next browser process’s candidate.&nbsp;If the data differs, Vidar uses the same second fork to perform an additional verification: it scans the forked process’s memory for the byte sequence <code>v20</code> and, treating each match as a potentially ABE-encrypted entry (cookie, password), attempts AES-256-GCM decryption with the key candidate using <code>BCryptDecrypt</code> (successful GCM tag verification confirms that the key is correct).&nbsp;Whenever the data has changed (indicating that decryption of the key took place), Vidar re-encrypts the key in the browser’s memory by injecting another APC with <code>CryptProtectMemory</code> as the routine, preserving the browser’s state. If no candidate produced a successful decryption (i.e., the data remained unchanged after every APC call), Vidar takes a more aggressive approach: it terminates all browser instances, restarts the browser, and repeats the entire scan-decrypt-verify cycle.&nbsp;By abusing APC injections instead of the more traditional approach of injecting code via <code>NtWriteVirtualMemory</code> that performs the decryption, Vidar trades one suspicious behavior for another. APC injections are arguably less common and may thus slip past some detection systems, though they are still well-known techniques monitored by AV and EDR solutions. But that is the nature of ABE bypasses – each has its own advantages and disadvantages, and that’s exactly why infostealers keep inventing new ones. &nbsp;<h2>Indicator of Compromise (IoC)&nbsp;</h2>459daa809751e73f60fbbe4384a7d1653c36bb06945e4eb3635270924241100a (Vidar 2.1)&nbsp;

A Technical Walkthrough of How Vidar Defeats Application-Bound Encryption

Inside Vidar’s ABE Bypass: From Memory Scanning to APC Injections

You have a summer flight booked. You have seen the headlines about jet fuel shortages, cancelled routes and higher travel costs. Then a text message arrives: your flight has been affected, and you need to click a link to confirm your refund or rebook before the offer expires.&nbsp;That message feels plausible because the situation around it is real. In April, <a href="https://www.iata.org/en/pressroom/2026-releases/2026-04-17-01/" target="_blank" rel="noreferrer noopener">IATA warned</a> that Europe could start seeing flight cancellations by the end of May if jet fuel shortages materialized. Airlines and governments have since been trying to reassure travelers, while also publishing guidance on passenger rights, refunds and re-routing.&nbsp;&nbsp;That is exactly the kind of confusion scammers like. They do not need to cause the disruption, they only need to arrive at the right moment, with the right brand logo and the right sense of urgency.&nbsp;<h4>Why travel disruption works so well as a scam hook&nbsp;</h4>Most scam messages fail because they arrive out of context. A random refund email from an airline you never use is easy to ignore. A refund message from the airline you are flying with next week, during a period of public travel disruption, is different.&nbsp;The scammer’s job is to make you think: “This could be real.”&nbsp;That can happen in several ways. You might receive a fake cancellation email that links to a refund form. You might search for an airline’s customer service number and land on a fake support page. You might post publicly on social media asking an airline for help, then receive a reply from an account that looks official. Or you might get a WhatsApp message that knows your hotel name, travel dates or booking reference.&nbsp;Each version uses the same pressure point: you are worried about losing money, losing your flight or being stranded.&nbsp;Criminals are already using disruption this way. In March, <a href="https://www.theguardian.com/money/2026/mar/15/travel-scam-airline-accounts-fake-refunds-iran-war-flight-disruption" target="_blank" rel="noreferrer noopener">The Guardian reported</a> that fake airline support accounts were targeting travelers affected by Middle East flight disruption. Some victims were looking for help or refunds, then received replies from accounts pretending to represent airlines. The scam moved into direct messages, where victims were asked for details and sent links that led to money being taken instead of refunded.&nbsp;&nbsp;The jet fuel story gives scammers another pretext. A message can claim that your flight was cancelled, your fare changed, your fuel surcharge failed, or your refund must be claimed within a few hours. None of those claims needs to be true to be effective.&nbsp;<h4>The scam may start before the message arrives&nbsp;</h4>A fake travel message becomes much harder to spot when it includes real information.&nbsp;This is the idea behind <a href="https://www.gendigital.com/blog/insights/research/reservation-hijack-scam" target="_blank" rel="noreferrer noopener">Reservation Hijack scams</a>, a form of targeted phishing where cybercriminals use real booking details to make a message look legitimate. In some cases, attackers may use information exposed through breaches or compromised travel systems. In other cases, they collect details from emails, social media posts, previous leaks or fake customer service interactions. Gen has previously described Reservation Hijack scams as scams that use real hotel booking details to make fraudulent payment or verification requests feel authentic.&nbsp;&nbsp;That distinction matters. A scam message does not need your credit card number to fool you. Knowing your name, your destination, your travel dates and your booking platform may be enough.&nbsp;<h4>What these scams can look like&nbsp;</h4>A travel disruption scam usually does not begin with a strange offer. It begins with something boring and administrative. That is part of the trick.&nbsp;You may see messages like:&nbsp;“Your flight has been cancelled due to fuel supply issues. Claim your refund here.”&nbsp;“Your rebooking is pending. Confirm passenger details within two hours.”&nbsp;“Additional airport tax or fuel surcharge is required to keep your booking.”&nbsp;“Your hotel reservation will be cancelled unless payment is verified.”&nbsp;“Due to high call volume, please continue this request on WhatsApp.”&nbsp;The exact wording changes, but the structure is familiar. There is a problem, a deadline and a link. The message may also move you away from the official channel into SMS, WhatsApp, Telegram, direct messages or a phone call. Once that happens, the scammer controls the conversation.&nbsp;There is another detail worth watching: some messages do not ask for money immediately. They start by asking for your booking reference, last name, phone number or one-time password. That can still be dangerous. Those details can help scammers access a booking, impersonate you to customer support, reset an account or make the next message more convincing.&nbsp;<h4>Your real passenger rights can be used against you&nbsp;</h4>If an airline cancels your flight, you may have rights to a refund, re-routing or assistance, depending on where you are travelling and the rules that apply. The European Commission said passengers affected by cancellations continue to have rights to reimbursement, re-routing or return, airport assistance and compensation for last-minute cancellations in relevant circumstances. It also said airlines cannot retroactively add fuel surcharges to already published final ticket prices.&nbsp;&nbsp;That information is useful for travelers, but it is also useful for scammers.&nbsp;A fake refund message becomes more believable when people already know refunds are possible. A fake rebooking link becomes more believable when airlines are genuinely changing schedules. A fake “fuel surcharge” demand becomes more dangerous when fuel prices are in the news, even though legitimate retroactive charges may not be allowed in many cases.&nbsp;This is why the safest route is not to judge the message only by whether the story sounds plausible. During travel disruption, many scam stories will sound plausible. Judge the channel instead.&nbsp;<h4>How to check before you click&nbsp;</h4>Start from the airline or booking platform’s official app or website. Do not use the link in the message. Open the app yourself, type the website address yourself, or use the contact details listed on your original booking confirmation.&nbsp;Check whether the same cancellation, refund or payment request appears inside your account. If the message is real, the airline or hotel should be able to confirm it through an official channel.&nbsp;Be careful with social media support. If an account replies to your public complaint, check whether it is linked from the airline’s official website. Look at the handle carefully. Fake support accounts often copy logos and names, but have small differences in spelling, low follower counts or recently created profiles. The Guardian reported that fraudsters often scan social media for passengers trying to contact legitimate airline accounts and reply to them first, which makes the scam harder to spot.&nbsp;&nbsp;Do not share one-time passwords, payment card details or login credentials with anyone who contacts you unexpectedly. A real airline or hotel does not need your banking OTP to send you a refund.&nbsp;Be especially suspicious of pressure. “You have 30 minutes to claim your refund” is not customer service. It is a tactic.&nbsp;<h4>What to do if you already clicked&nbsp;</h4>If you entered payment details, contact your bank or card provider immediately. Ask whether the transaction can be blocked or reversed, and request a new card if needed.&nbsp;If you shared login details for an airline, hotel or booking platform, change the password from the official site or app. Use a unique password, not one you use elsewhere. Turn on two-factor authentication where it is available.&nbsp;If you shared a one-time password or authentication code, assume the account may have been accessed. Check your booking details, payment methods and recent account activity.&nbsp;Report the scam to the airline, hotel or booking platform being impersonated. If the scam came through social media, report the account as impersonation. If money was taken, report it to the relevant fraud authority in your country.&nbsp;<h4>The real lesson for summer travel&nbsp;</h4>Travel disruption changes how people behave. They search for support numbers in a hurry. They post publicly because customer service queues are long. They accept strange instructions because they are worried about losing a flight or refund.&nbsp;Scammers build around that behavior.&nbsp;The safest assumption is simple: if a message about your flight, hotel or refund creates urgency and gives you a link, pause. Go back to the official source. A real cancellation will still be there when you check through the airline or booking platform. A scammer needs you to act before you do.&nbsp;

Travel disruption gives scammers the one thing they need most: a believable reason to rush you

Your flight was cancelled. Is the refund message real? 

A fake job interview page is easier to trust than a fake ticket shop.In samples reviewed by Gen threat researchers, scammers used branded hiring pages that looked like ordinary recruitment flows: a company logo, a recruiter profile, a 30-minute meeting, and a button to continue with Google or Facebook. There was no attachment to open and no software to install. The credential theft attempt sat inside a step many job applicants already expect: signing in to schedule a call.FIFA is an obvious lure ahead of the 2026 World Cup. On May 22, ESET reported fake FIFA-themed websites that copied ticket and merchandise flows to steal money and personal data. Our researchers later found another FIFA-themed domain, hxxps://fifa-mg[.]shop, which followed a naming pattern not mentioned in that report. Domains like these can be short-lived, often appearing for hours or days before they are blocked, abandoned, or taken down.Our researchers also found a different kind of FIFA lure: hxxps://fifahiring[.]com, a fake hiring page that encouraged users to schedule a call with an alleged FIFA recruiter. When a user tried to register with a personal Gmail address, the page asked for a business email instead. The page was steering victims toward credentials with more value: accounts that could provide access to a company environment.

<h4>The FIFA hiring page did not point to FIFA</h4>The fake page looked believable at first glance.

It used FIFA branding, presented a recruiter identity, and copied parts of the experience users see on legitimate recruiting and scheduling platforms. The recruiter’s name and photo appeared to match a real LinkedIn profile, which made the page feel more credible. For publication, we should keep that person’s identity redacted.The domain did not match FIFA’s real hiring infrastructure. FIFA’s legitimate careers pages are jobs.fifa.com and fifa.pinpointhq.com. The fake page used neither. It also loaded the favicon from the real FIFA recruitment site hosted at pinpointhq.com, giving the page a small visual cue borrowed from the legitimate service.Other signals pointed in the same direction. The domain and certificate were newly created, from May 23. The site used commodity hosting infrastructure, including Vercel and Render, rather than a corporate environment. Those signs are not enough on their own, but they fit the rest of the evidence.The login flow was the clearest part. Credentials were posted to:fifeq2026eqbackeq[.]onrender[.]com/api/loginThe Google sign-in prompt was not a real Google authentication window. It was rendered inside the page itself. The visible controls and links did nothing, and the page did not send the user into a genuine Google sign-in flow. According to the analysis, the only Google asset loaded was the favicon, apparently to make the fake window look familiar.A page can look like it opened a Google login without ever handing control to Google.Researchers also observed hxxps://careers-fifahiring[.]com, which later became inactive, and hxxps://fifajobs[.]com, which showed the same content and was indexed by Google.&nbsp;<h4>The recruitment lure went beyond FIFA</h4>The FIFA page was one version of a wider recruitment phishing pattern. The brand can change. The flow stays familiar: recruiter profile, meeting prompt, business email, fake sign-in.Researchers found similar fake hiring pages abusing other major brands, including Heineken, Hilton, Coca-Cola, Netflix, PepsiCo, Delta,&nbsp;and Spotify. In the Heineken case, the page appeared to use the identity of someone who, according to LinkedIn, really works as a recruiter at the company.

Additional hunting found two other recruitment-themed kits:hxxps://aquent-talent-jobs[.]com hxxps://hays-talent-opportunities[.]comThe Hays-themed version targeted Facebook credentials instead of Google credentials. Researchers assessed that the Aquent and Hays pages shared code, although they differed from the earlier FIFA examples.

This flexibility makes the campaign harder to frame as a single World Cup scam. FIFA provides the timely hook, but the mechanism is recruitment phishing. A scammer can swap the logo, recruiter name, and sign-in provider while keeping the same basic path to the victim’s credentials.<h4>Why fake recruitment pages work</h4>Recruitment gives attackers a clean excuse to ask for things that might otherwise raise suspicion.A candidate expects to click a scheduling link. They expect a recruiter profile. They expect a calendar prompt. In some cases, they may also expect to use a work email, especially for corporate, contractor, or partner-facing roles.The fake login window turns that ordinary flow into credential theft. Users are used to “Continue with Google” and “Continue with Facebook” buttons, but the button itself proves nothing. A genuine authentication flow should take the user to the provider’s domain and controls. A fake page can draw a convincing copy while keeping the victim inside the attacker-controlled site.The business email prompt is also a warning sign. A page that rejects a personal email and asks for a work address may be trying to raise the value of the stolen account.<h4>What job seekers should check before signing in</h4>Start from the company’s official careers page, not from a link in a message, ad, search result, or forwarded post. For FIFA, that means checking jobs.fifa.com or fifa.pinpointhq.com, not trusting a domain because it contains “fifa,” “career,” “hiring,” or “jobs.”Look closely at the domain before entering credentials. Scam pages often add plausible hiring words around a brand name: career, talent, jobs, hiring, portal, or connect. A polished page on a new domain is still a new domain.Treat the sign-in step as the real checkpoint. If a page offers Google or Facebook login, check whether the flow actually opens a real provider-controlled authentication page. A fake window embedded inside the original site, broken controls, inactive links, or a provider logo used only as decoration should stop the process.Use security software that includes web and phishing protection. These tools can block known malicious pages before they load, which is especially useful when scam domains look polished, appear in search results, or are shared through otherwise legitimate channels.<h4>What companies can do</h4>Recruitment pages are public by design, which makes them easy to copy.Security teams should monitor for newly registered domains combining their brand with hiring-related terms. They should also watch for pages that reuse favicons, recruiter photos, job descriptions, and calendar-booking language from legitimate hiring flows.HR and recruiting teams should make the official hiring path easy to verify. Candidates should not have to guess whether a domain is legitimate. A clear careers page, consistent recruiter communication, and warnings about unofficial scheduling links can remove some of the ambiguity attackers rely on.<h4>Conclusion</h4>The fake FIFA hiring page worked because it did not feel unusual. It looked like a normal step in a hiring process.Attackers do not need to invent new behavior when they can copy one people already trust. A branded scheduling page, a recruiter profile, and a familiar sign-in button can be enough to move a victim from interest to credential entry.The decision point is the login. Before entering a work email or password, check the domain, check where the authentication flow really goes, and verify the role through the company’s official careers site. Security software with web and phishing protection adds another layer, especially against short-lived scam pages that may only stay online for hours or days.IOCs:hxxps://fifahiring[.]com hxxps://careers-fifahiring[.]com hxxps://fifajobs[.]com hxxps://fifa-mg[.]shop hxxps://heineken-careers[.]com hxxps://aquent-talent-jobs[.]com hxxps://hays-talent-opportunities[.]com fifeq2026eqbackeq[.]onrender[.]com/api/login

Scammers are copying recruitment and calendar-booking flows to make fake Google and Facebook sign-ins look routine. 

Fake hiring pages abuse FIFA and other major brands to steal work credentials 

Threat Research Team Lead

Since January 2026, we have detected 7.4 million malicious incidents tied to residential proxy traffic, affecting 572,000 users in our telemetry. In this model, the household whose connection is being used is often not the buyer of the traffic. It is the exit point.That is the uncomfortable part of residential proxy networks. A home router, laptop, phone, smart TV, or other always-on device can quietly become part of a commercial network that routes strangers’ internet traffic through ordinary consumer IP addresses. To the websites on the other end, that traffic looks as if it came from a real household, because it did.The technology itself is not inherently malicious. Businesses use residential proxies for market research, ad verification, brand protection, security testing, and services that need to see the web as a real user would. But the same feature that makes these networks useful also makes them attractive for abuse: traffic from a household IP address is hard to block without risking false positives against legitimate users.The problem for consumers is simpler. Many people do not realize their device or connection can be enrolled into this kind of network at all. Some users opt in through pay-for-bandwidth apps. Others are enrolled through “free” VPNs, bundled software development kits inside unrelated apps, or unwanted software that installs proxy components without meaningful consent.Here is how the market works, who pays for it, and what it can cost the household that unknowingly hosts it.<h2>What is a residential proxy</h2>A residential proxy is an internet connection – typically a home laptop, phone, smart TV, or router – that has been enrolled into a third-party network and is being used to route someone else's traffic. To the rest of the internet, that traffic looks like it is coming from an ordinary household IP address assigned by a consumer ISP, not from a datacenter or&nbsp;VPN provider.Residential proxy networks are sold commercially. The model is two-sided: some people pay to tunnel their traffic out through someone else's connection (much like a VPN, except the exit point is a real home), while others get paid to provide those exit endpoints. Some are structured as paid peer-to-peer networks, where any participant's connection can serve as an exit for any other.Buyers pay for access to pools of millions of these "real user" IPs because they are extremely difficult to block, or rather, blocking them carries real consequences: the traffic originates from genuine households, so any block risks shutting out a legitimate user – effectively a false positive.Common buyers include market research firms, price-comparison services, brand-protection vendors, security teams testing their own defenses, and consumer privacy tools. At the abusive end of the market, the same infrastructure used by operators running automated purchasing bots, scrapping campaigns, ad fraud, credential stuffing, phishing and other attacks where hiding the true origin of the traffic matters. Demand is also being shaped by large-scale web scraping, including&nbsp;<a href="https://blog.includesecurity.com/2026/06/the-smart-tv-in-your-livingroom-is-a-node-in-the-aiscraping-economy/">scraping for AI training data</a>. As anti-bot systems become better at identifying traffic from datacenters and known VPN providers, residential IP addresses become more attractive because they look like ordinary users. That does not make every residential proxy user malicious. It does mean consumers may be carrying traffic for businesses and actors they have never heard of.<h2>How consumer devices end up in these networks</h2>There are several ways a home device can become part of a residential proxy network. Some are explicit. Others are easy to miss.The most obvious route is the pay-for-bandwidth model. Apps such as Honeygain, IPRoyal Pawns, and PacketStream pay users to share their internet connection. The user opts in, but they may not understand who will buy access to that connection later, what kind of traffic will pass through it, or what happens if that traffic triggers abuse reports.A second route is the “free” VPN or proxy app. Instead of charging a subscription, the app monetizes the user’s connection by reselling bandwidth. In practice, the user may think they are getting a privacy tool while also becoming infrastructure for other people’s traffic.A third route is through bundled software development kits, or SDKs. A developer adds a third-party component to an otherwise unrelated app, such as a flashlight, wallpaper, game, or utility. The developer gets paid when the app is active. The user sees the host app, not the proxy network behind it.The fourth route is malware or potentially unwanted software. In those cases, the proxy component is installed silently or deceptively, with no meaningful user consent at all.<h2>Why the "consent" is rarely meaningful</h2>Even when consent technically exists, it is often not the kind of consent most people would recognize.The disclosure may be buried in a long terms-of-service document. The wording may be vague, with phrases such as “share your unused bandwidth,” “support the app,” or “join our network.” These phrases do not clearly tell the user that arbitrary third parties may route traffic through the user’s home IP address.There is also the question of downstream use. Users are rarely told who buys access to the network or what those buyers do with it. Their connection could be used for benign market research. It could also be used for scraping, automated account creation, ad fraud, credential stuffing, or phishing infrastructure.The problem gets worse when the proxy component sits inside another app. The host app’s privacy policy may mention a partner SDK by brand name, but that still does not make it clear to a normal user that their device can become a proxy endpoint.Uninstalling the visible app may not always resolve the issue cleanly either, especially when proxy components are installed separately or bundled with other unwanted software.<h2>The risks consumers actually carry</h2>When a household device becomes a residential proxy node, the user takes on risks created by someone else’s traffic.The first risk is attribution. Abusive, fraudulent, or illegal activity routed through the device appears to come from the household’s IP address. ISP abuse notices, platform bans, fraud checks, and law enforcement requests point first to the subscriber, not to the person who bought the proxy traffic.The second risk is performance. Continuous third-party traffic can consume bandwidth, slow the home network, hit data caps, or violate ISP terms of service. The user may only notice that the internet feels slower, a bill is higher, or a service suddenly blocks access.The third risk is security exposure. Proxy clients run persistent background processes and maintain network connections so they can relay traffic. That adds always-on networking software to the device, often with limited visibility for the user.The fourth risk is reputation. A home IP address used for abuse can end up on blocklists used by banks, retailers, streaming services, and online platforms. The result can be account friction, failed logins, extra verification steps, or outright blocking from services the household actually uses.There is also a family angle. A child or another household member may install a free game, VPN, or utility without understanding the consequences. The legal, financial, and reputational risk still lands on the account holder.<h2>What our telemetry shows</h2>Since the beginning of 2026, we have detected 7.4 million malicious traffic incidents associated with residential proxy traffic, affecting 572,000 unique users in our user base.&nbsp; This is not a measurement of the entire residential proxy market. It reflects what we observe in our telemetry: malicious traffic, detections, and related activity affecting protected users. The real footprint is far larger, because residential proxy networks span major consumer platforms and many devices outside our visibility.The activity is global, but it is concentrated in a smaller set of countries. India leads by a wide margin, with roughly 83,000 affected users, followed by Vietnam with 61,000 and Brazil with 39,000. The Philippines (23,000), Poland (18,000), Indonesia (18,000) and Thailand (14,000) form a second tier, with Egypt, Spain, Ukraine, Argentina, Mexico and Italy each accounting for 10,000 to 13,000. The spread across South and Southeast Asia, Latin America, and Eastern and Southern Europe reflects where the host apps that carry this software are most widely installed.That geographic spread likely reflects a combination of app distribution, local install patterns, user appetite for free or pay-for-bandwidth apps, and the economics of bandwidth resale. It should not be read as a simple ranking of where residential proxy abuse originates.The chart below breaks down the threats we observed in this proxy-related traffic. Two categories dominate: phishing and malicious advertising. Behind them sits a long tail of consumer-facing fraud, including e-shop scams, generic scams, financial scams, and fake-tutorial pages, alongside outright malware such as trojans and other malicious traffic.&nbsp;

In the malicious proxy traffic we observed, scams and malware dominate. The important point for consumers is not only that their connection may be used by someone else. It is that, when abuse happens, their household IP address can be the return address.<h2>What consumers can do</h2>Most people will not identify a residential proxy component by name. But there are practical warning signs.Review “free” VPNs, proxy apps, bandwidth-sharing apps, and utilities from unknown publishers. Check whether they mention bandwidth sharing, peer-to-peer networking, traffic routing, or network participation in their terms. If an app offers payment for leaving your device online, assume your connection may be resold unless the provider explains exactly how traffic is vetted and who can buy access.On phones and computers, remove apps you do not recognize or no longer use. On home routers, check the list of connected devices and look for anything unfamiliar. If your ISP sends abuse notices, websites start blocking you, or your home IP address suddenly fails fraud checks, a proxy component is one possible explanation.Security software can also help by detecting unwanted proxy clients, bundled components, and related malicious activity. But this should not be a guessing game for consumers. Software that turns a household connection into a commercial exit node should be named clearly and made easy to remove.<h2>The takeaway&nbsp;</h2>With residential proxiesthe user’s connection becomes an infrastructure for someone else’s business.The issue is not that residential proxy technology exists. It has legitimate uses. The issue is whether people know when their devices are part of it, understand what kind of traffic may pass through their connection, and have a real choice to opt out.The fix is not to pretend consumers will read every EULA. This software should be named clearly, shown plainly, and made easy to remove before a household connection becomes someone else’s infrastructure.

Your home connection could be routing traffic for strangers. Here's how residential proxy networks work, how devices get enrolled and what our telemetry reveals about the risks for consumers.

Who's Really Using Your Home Internet Connection?

<a href="https://github.com/fthomas/scala-steward">Scala Steward</a> is a bot that helps you keep your project’s dependencies and SBT plugins always up-to-date. It is a very useful tool because the world of Scala is evolving rapidly and it can become quite hard to keep up with all the updates. And most of the updates are just small so it is worth the effort to automate this process.We have been using Scala Steward in <a href="https://github.com/avast/scala-server-toolkit">our open source projects</a> and had a very good experience with it. However we lacked a tool like that for our internal projects (in our GitHub Enterprise instance). So we decided to make it run for internal projects and this article describes how we’ve done it.There is <a href="https://github.com/fthomas/scala-steward/blob/master/docs/running.md">some documentation</a> on how to run Steward which is a great start, but not everything is well-documented and we needed to make some tweaks.<h2>Custom Docker Image</h2>Steward can be run either directly via SBT or as Docker image. I do not think that build tools should be used to run applications so I chose the latter option. There is no official Scala Steward image so the first thing you need to do is build one.

We made some changes to the built Docker image. We use a different base image (internal one used for most JVM projects) and we install <a href="https://aws.amazon.com/cli/">awscli</a> (I will talk about that later). These changes can be made inside build.sbt so you need to make an internal fork of the Scala Steward repository.

Once you have the Docker image you need some place to run Steward periodically. There are many options but we run Steward in our internal Kubernetes cluster as a cron job.<h2>Deployment to Kubernetes</h2>You need to create an YAML file describing how Steward should be deployed. This is our configuration:

As you can see it is a definition of a Kubernetes CronJob resource scheduled to run every hour. It takes the Docker image we built internally and runs a series of commands to achieve what we set out to do in the beginning.<h2>Persistence</h2>The first thing that is a bit complicated is persistence. Steward is able to run without any storage but certain features do not work without it (e.g. <a href="https://github.com/fthomas/scala-steward/blob/master/docs/repo-specific-configuration.md">choosing PR frequency</a>). Our Kubernetes cluster does not support persistent volumes so we needed some remote storage. Thankfully we have an internal S3-like storage that we could use. That is the reason for installing awscli into our Docker image.Basically Steward’s workspace archive is downloaded before the run and uploaded back after Steward finishes. It does not support parallel run of multiple Steward instances but we do not need that so it is absolutely fine for us.The run command seems to be complex mostly because we need to do workspace synchronization. It would be much simpler without it.<h2>Configuration</h2>The last missing piece to describe is how Steward is configured.We have a repository with a list of internal repositories that should be updated by Steward. It looks the same as the <a href="https://github.com/scala-steward-org/repos/blob/master/repos-github.md">public variant</a>.This repository is cloned at the beginning of the update process and it also contains the git-ask-pass script:

We are using the following command-line options to run Steward: &nbsp;

We are setting where the Steward’s workspace is, where the repos.md file is and several options to configure Git (GitHub) access using a service account and its personal access token.The OSS version of Scala Steward does not have direct access to the repositories it updates so it needs to create forks. We have the luxury that we can give Steward the proper access so the –do-not-fork option is enabled. We also don’t need to run Steward in a sandbox so it is disabled.<h2>Logging</h2>Logs from standard out are pushed to Kibana so we can always see how Steward is doing. This is quite important because we have encountered several issues in the past which we could not resolve without some information from logs (the issues were mostly related to the workspace synchronization).<h2>Conclusion</h2>We have been running Scala Steward since February 2020. It has created around 750 PRs since then. It is a huge amount of work we would have to do manually otherwise and it helps us keep up with the rapid pace of development – yes, we do also evolve our internal libraries quite fast.It can sometimes be a bit overwhelming to receive so many PRs from Steward. First, it gets better over time as you update most of your dependencies. Second, some developers use the PR frequency setting to make it more manageable. Overall I think most Scala developers are happy that we have Steward running inside our company.

Automating Scala Dependency Updates On-premise

Running Scala Steward on-premise

In the previous post I introduced and explained the rationale behind Scala Server Toolkit. Today I would like to continue and get into more detail of some of the offered modules.<h2>Getting Started</h2>Our goal is to initialize and start a simple HTTP server using <a href="https://http4s.org/">http4s</a>. There are four basic things you will need in order to achieve that:<ol><li>Configuration,</li><li>thread pool (execution context),</li><li>HTTP handler,</li><li>and router (specification of which URL should call which handler).</li></ol>Let’s cover these one by one.<h2>Configuration</h2>Every application requires some configuration. It is common in Scala world to use Lightbend Config which is a great library to load <a href="https://github.com/lightbend/config/blob/master/HOCON.md">HOCON</a> configuration files and retrieve the configuration values by their name and type. The loaded <code>Config</code> is usually passed around your application components so that each one can retrieve its configuration values. This works well for small applications but it is not robust for large applications and has its downsides.The downsides are that loading of configuration values is scattered around all your codebase and it can fail at any time if you make a mistake in the name or the expected type of the value (it can even fail much later in the lifetime of your app if some components are loaded lazily). Your code is also tied with the Config library and you cannot easily change it for something else (e.g. in tests).There is a better approach. Use case classes to model your configuration. Case classes are pure Scala solution (no extra dependency needed), everyone understands them well, they can model the configuration with specific types (e.g. <code>Port</code>) and they are easy to use from tests. What you need to solve then is how to fill them with values from a configuration file because in practice you do not want to do that manually. Luckily there are many libraries for that and the one I would recommend is <a href="https://pureconfig.github.io/">PureConfig</a>.Scala Server Toolkit already contains a configuration case class for the server: <code>com.avast.sst.http4s.server.Http4sBlazeServerConfig</code> in subproject <code>sst-http4s-server-blaze</code> which contains all the possible configuration options for the blaze implementation of http4s server. There is subproject <code>sst-http4s-server-blaze-pureconfig</code> which provides implicit <code>ConfigReader</code> instances which you need to import so that PureConfig knows how to fill the case class with values from configuration file. And finally there is <code>sst-pureconfig</code> which allows you to put these together and actually load the configuration:

The import <code>com.avast.sst.http4s.server.pureconfig.implicits._</code> is important because it gives you the <code>ConfigReader</code> instances for the <code>Http4sBlazeServerConfig</code> case class. The default naming convention is kebab-case which means that field named <code>configuration-value</code> in the file is filled into field named <code>configurationValue</code> in the case class. It is also possible to import <code>com.avast.sst.http4s.server.pureconfig.implicits.CamelCase._</code> in which case both the case class field and field in the file have to match.The import <code>zio.interop.catz._</code> is important because it gives you instances for cats-effect typeclasses such as <code>ContextShift</code> and <code>Sync</code> for the <code>Task</code> effect type of ZIO.<h2>Thread Pool</h2>Every application also requires some thread pools to execute on. It is <a href="https://gist.github.com/djspiewak/46b543800958cf61af6efa8e072bfd5c">recommended</a> to have at least two pools – one bounded for CPU-bound operations and one unbounded for blocking (IO) operations. Scala Server Toolkit offers subproject <code>sst-jvm</code> which contains <code>ExecutorModule</code> which gives you exactly that out of the box. You can also wrap already existing thread pool if you want to reuse it from your “runtime” (e.g. <a href="https://zio.dev/">ZIO</a>). There are more methods which allow you to create more thread pools of different kinds and with different configuration.

<h2>HTTP Handler</h2>The next thing you need is an HTTP handler. Simply put, it is just a function from HTTP request to HTTP response. In practice it is a bit more complicated because such handler usually does side effects so in http4s HTTP handler has the following type signature: <code>Request[Task] =&gt; Task[Response[Task]]</code>. http4s provides DSL to easily create a response. See the <a href="https://http4s.org/v0.20/dsl">documentation</a> for more information.

<h2>Router</h2>And the last piece is a router. We need to tell http4s server which URLs should be routed to which handlers. It is customary to add a default route which return <code>HTTP 404 Not Found</code>.

<code>HttpRoutes</code> is an object from http4s which allows you to define the actual routing using <code>Http4sDsl</code>. You can see that any <code>GET</code> request at endpoint <code>/hello</code> will call our HTTP handler.Scala Server Toolkit gives you <code>Http4sRouting</code> object which allows you to combine multiple <code>HttpRoutes</code> objects into one and also adds the default <code>HTTP 404 Not Found</code> route. The result is <code>HttpApp</code> which can be passed directly to http4s server.<h2>Conclusion</h2>Now we have everything we need to start a simple HTTP server from scratch. Note that we need to wrap everything in <code>cats.effect.Resource</code> to achieve resource-safety which is very important. Some of the modules return <code>Resource</code> directly (e.g. <code>Http4sBlazeServerModule</code>), some do not and then you need to use <code>Resource.liftF</code> to make the types align.

If you run this application you should see the following output: &nbsp;

and if you call the endpoint (using <a href="https://httpie.org/">HTTPie</a>) we created you get the expected HTTP response. &nbsp;

And that’s it!It might be hard to see the benefits of this approach in such a simple example. You could probably write it in less number of lines of code but the point of all of this is that usually our applications are more complex and are extended over time and only then you start seeing the benefits of principled approach to type safe configuration, proper resource management and unified approach to component initialization. This approach scales very well and all the components and their relationships are directly visible. This is what we have found to work best for us at Avast.

Building a Scalable HTTP Server

Scala Server Toolkit – creating HTTP server

Today I would like to introduce a new project called <a href="https://github.com/avast/scala-server-toolkit">Scala Server Toolkit</a>. In short it is culmination of years of Scala server development at Avast representing the best practices we have gained. We decided we want to share it with everyone because we believe it could be very useful not just to us. Let me describe to you the genesis of this project, the rationale behind it and show you what it offers.<h2>Genesis &amp; Rationale</h2>Avast backend developers have been using Scala since around 2011. Over the years they have went through the usual stages of a Scala developer:<ul><li>Using Scala as better Java: imperative code full of mutability and side effects.</li><li>Idiomatic Scala: expression-oriented programming, full potential of Scala collections, convenient mix of OOP and FP.</li><li>FP Scala: taming side effects, monads and algebraic laws, all about referential transparency, composability and separation of concerns.</li></ul>The evolution process from imperative Scala towards FP Scala was very natural. As we have been learning Scala more and more it lead us to the more functional side of the language. Now we are in a phase in which we are dedicated to go “full FP”.However not everything was rose-colored. The different styles of programming we have used in the past mean that we have very diverse codebases and teams/people are used to do things differently. Functional programming is very popular nowadays and the amount of innovation in that space is incredible but we are still lacking some kind of library that would give us unified approach to programming server applications. Thus we have decided to write our own. It should help us unify our programming style, promote best practices, functional programming and we believe it could solve the same problems for others too.Basically the problem we are trying to solve is that there are lots of great open source libraries which we want to use but putting them all together in unified way and teaching all of that to newcomers is quite some task. And we think this should not be so much difficult.It should be pointed out that we do not intend to write some full-blown framework that would try to solve all your server-side problems. We are actually trying to do quite the opposite. Reuse as most open-source libraries as we can and provide only the missing integration pieces to simplify everyday programming. Most of the code revolves around initialization and integration of existing OSS libraries in unified way.<h2>Design</h2>There are certain design decisions and constraints that we put in place to guide the development of this library.<ul><li>Modular design: small, cohesive, orthogonal and composable components.<ul><li>The project is split into completely separate modules mostly based on dependencies. Every module provides some functionality, usually something small. The idea is to compose multiple modules together to get a working application but be able to replace any module with different implementation in case it is needed.</li></ul></li><li>Keep the number of dependencies as low as possible.<ul><li>Adding a dependency seems like a great idea to avoid having to implement some piece of existing logic however every dependency is also a burden that can cause a lot of problems. Therefore every dependency should be considered well and only very important and well-maintained libraries should be added to our core.</li></ul></li><li>Functional programming.<ul><li>We believe in functional programming so this library also utilizes its concepts.</li><li>We do not want to force anyone into any specific effect data type so the code is written in so-called tagless final style.</li></ul></li><li>Type safe configuration and resource lifecycle.<ul><li>Application initialization is often overlooked. By using type safe configuration and proper resource management (to never leak resources) we can make it correct without much hassle.</li></ul></li><li>No need for dependency injection.<ul><li>Dependency injection is not needed in most cases – plain constructors and good application architecture usually suffice.</li><li>Use a dependency injection framework if your application is huge and it justifies its cost.</li></ul></li><li>Strive for <a href="https://slides.yowconference.com/yowwest2014/Morris-ParametricityTypesDocumentationCodeReadability.pdf">Scalazzi Safe Scala Subset</a>.</li></ul><h2>Why Choose It?</h2>I assume you are a Scala developer doing functional programming who wants to build a server application using <a href="https://http4s.org/">http4s</a> and <a href="https://zio.dev/">ZIO</a>. But not only that you want to load your configuration into case classes (<a href="https://pureconfig.github.io/">PureConfig</a>), you want to <a href="https://typelevel.org/cats-effect/datatypes/resource.html">manage the resources</a> and of course you want to <a href="https://micrometer.io/">monitor the health of your application via metrics</a>.As you can see from the links in the above paragraph there are great tools for all of this but how do you combine them? There is a huge space for error because each library does things a bit differently. For example Http4s server cannot be configured via case class so you have to use the provided builder and fetch the configuration values from somewhere manually. Loading of configuration via PureConfig is side-effectful so you have to wrap it in <code>F</code> manually. Micrometer is a Java library so it does not provide Resource for initialization.All of these things are small warts that need to be handled in each and every server application which is tedious. Scala Server Toolkit tries to abstract away the differences and gives you unified interface to initialization of components. Interface that is pure, resource-safe and composable.<h2>Example</h2>How does it look in practice? I am going to show you the minimal example on how to start some HTTP server. Keep in mind that “minimal” is not about the number of lines. You do have to write some code to put everything together but that is fine because you only have to write that code once in the lifetime of your application and all the dependencies between components are explicitly visible in your code – there is no hidden magic.

This is just a short example. There will be follow-up articles explaining the individual parts of the library in more detail. If you want to find out more now you can look at the <a href="https://github.com/avast/scala-server-toolkit/blob/master/docs/index.md">documentation</a> (it is compiled so it is always up-to-date) or an <a href="https://github.com/avast/scala-server-toolkit/tree/master/example/src/main/scala/com/avast/sst/example">example</a> server application in the repository.<h2>Conclusion</h2>Scala Server Toolkit is currently going through rapid development phase since we are at the beginning of the project. That being said it is already being used in some of our projects. It still requires some work before we can release the first stable version but I encourage everyone to try it out and let us know what they think. The first stable version is coming in the following months.

Jakub Janeček

Posts by Jakub Janeček

Running Scala Steward on-premise

Scala Server Toolkit – creating HTTP server

Introducing Scala Server Toolkit