The global advertising market is growing and forecasted to exceed $700 billion soon. Much of that growth is attributed to digital advertising aimed at people who are spending more time online, looking at screens, streaming ad-supported music and entertainment, and connecting through social networks. Companies’ spending on digital advertising experienced double digit growth in 2020, despite the pandemic.  Adtech companies are responding to this growing demand with fast-paced programmatic advertising, which utilizes data insights and algorithms to automatically serve ads to the right user, at the right time, on the right platform, and at the right price. The Importance of Data in Programmatic Advertising As the world of digital advertising becomes more dependent on this type of programmatic media buying, data is transforming how the adtech industry operates. The data – cost per impression, cost per click, page views, bid response times, number of timeouts, number of transactions per client, etc. –is as important as the money spent on those impressions. The data shows how effective the ad buys really are, thus proving whether or not they are worth the money spent on them. This is one reason that data must be continuously monitored. The vast array of moving parts in online advertising means that adtech companies need to collect, analyze, interpret, and act upon immense datasets instantaneously, every single day. The insights that come from this massive onslaught of data can create a competitive advantage for those who are prepared to act upon those observations quickly. Traditional business intelligence tools can’t scale to fully support adtech needs Addressing the current data analytics needs in adtech can be challenging. With billions of daily transactions, the sheer volume, velocity, and complexity of the data can easily overwhelm conventional business intelligence tools. While traditional BI tools such as dashboards and email alerts offer some support, in general, their capacity in the context of adtech analytics is severely limited. Among the most common problems are: Lack of data correlation – Traditional tools may show only one problem, like server latency, but will not show or correlate multiple issues in the same alert, for example, server latency and a dip in conversions due to time-out issues. This can make it difficult to uncover technical anomalies that can dramatically affect revenues. Alert fatigue – An overly sensitive monitoring solution can generate large volumes of alerts for even small incidents. The more alerts, the greater likelihood of false positives and the chance that staff will ignore alerts due to lack of time to investigate them all. Seasonality issues – Traditional BI tools based on thresholds don’t consider seasonal patterns in data and often end up capturing too many samples that are falsely identified as anomalies.  BI tools work from hindsight – It can take hours, days or even weeks to find issues and apply remediation when using traditional BI and monitoring tools, making them unsuitable for the fast pace of programmatic advertising. Minor undetected issues can cause major losses – Adtech companies can lose hundreds of thousands, if not millions, of dollars due to the passage of time between a business incident and its discovery. If undetected for too long, even minor issues can cause a detrimental disruption to service. AI in adtech delivers actionable insights Real-time analysis is the only way for adtech companies to determine whether key indicators are under or over performing. To ensure these companies always have their finger on the pulse of every consequential metric or data anomaly, executives, data scientists, and analysts are turning to real-time machine learning, artificial intelligence (AI/ML), and predictive analytics to help them identify and resolve issues immediately. With data accumulating at an exponential rate, it's simply impossible for data analysts to extract relevant and timely business insights without autonomous AI analytics. Adtech companies need a scalable, real-time BI and analytics solution like Anodot, which can handle any number of data variables, intelligently correlating related anomalies that may not be apparent to a human observer.  Best results are achieved with machine learning, which does not require any manual configuration, data selection, or threshold settings, along with algorithms that can handle complex data such as click rates, impressions, and bid duration for every combination of campaign, publisher, advertiser, and ad exchange. Anodot’s AI-powered business monitoring for adtech  Where traditional BI tools fail due to time delays, data constraints, and complexity, Anodot’s predictive analysis learns data patterns for a variety of KPIs and dimensions and delivers actionable insight through automated anomaly detection.  Anodot’s big data ML algorithms are specifically designed to detect outliers, preemptively identifying trends as well as issues before they become problems, and facilitating optimization and operational maintenance. In real time, Anodot can detect anomalous behavior, correlate multiple anomalies, and then alert the proper teams in order to get a fix in place. Anodot Helps Xandr Resolve Issues Quickly Xandr is a massive-scale marketplace that connects the demand side to the supply side in the advertising ecosystem. Ben John, CTO at Xandr, describes what his company went through in trying to solve their data monitoring challenges before engaging Anodot.  “We had to install these agents and run hundreds if not thousands of servers and applications across our global data centers. When a business-critical incident happened, people had to look at the logs, at some of the monitors, and at alerts. They would try to correlate it all to understand the business incident or business impact. That is really hard, and every minute we were losing revenue, and also our customers were losing revenue, so time is of the essence.” Xandr needed an automated solution that could scale to the company’s rigorous demands, and yet could detect anomalies happening for a single customer in a single region of a global business. They chose Anodot’s cloud-based solution to identify and resolve incidents before they can impact business. “We reduced the time to detection of root causes from up to a week to less than a day. The complexity of our platform makes manual detection incredibly difficult,” says John. “Before Anodot, it could take up to a week because our platform integrates with so many partners. Now, this data helps us find so many incidents within a few hours or within a day, compared to multiple days and weeks.” Anodot caught events that resulted in savings of thousands of dollars per event. “Each campaign going through the Xandr platform configures hundreds of thousands, if not millions of ads, and if things go wrong, it can have a significant financial impact. We were able to save lots of money for both Xandr and our customers,” according to John. Anodot helps keep the Magnite ad exchange working smoothly With 2.5 times more transactions than NASDAQ, Magnite (formerly Rubicon Project) is one of the largest ad exchanges in the world. More than 90% of people browsing the Internet will see an ad that goes through the Magnite exchange. Using this service, the world’s leading publishers and advertising applications can reach more than a billion consumers. With 13 trillion monthly bid requests, 55,000 CPUs, and 7 data centers, Magnite’s BI needs were well beyond the scope of what humans could monitor, analyze, and control. Magnite turned to Anodot to track their data in real time to aid in the creation of a fair and healthy ad marketplace. Anodot’s advanced machine learning-based BI and analytics solution allows the Magnite team to identify trends and correlations in real time. Recently, Magnite was able to instantly correlate a drop in one customer’s bidding activity to system time-outs. Magnite immediately contacted the client and alerted them. The customer identified a bug in a recent software release as the culprit for the time-outs and resolved it quickly to get back in the game. Magnite also benefited from the added ability to pull existing business intelligence solutions into the Anodot system. Magnite used an open source monitoring tool, so Anodot simply extracted data from the tool, allowing Magnite to streamline and automate data analytics.

Explore how this ride-share leader uses Anodot to identify business risks in real time

Immediately address data quality problems and save weeks of dealing with inaccurately reported data. AI Analytics and Anomaly Detection puts renewed trust in the quality of the data that directly impacts business priorities.

Fintech Metrics: Critical Transaction KPIs to Monitor In a previous post about payment transaction monitoring, we learned how AI-based payment monitoring can protect revenue and improve customer experience for merchants, acquirers and payment service providers. In this post, we’ll highlight the critical transaction metrics that should be monitored in order to achieve these goals. When most organizations think about ‘transaction metrics’ they probably think the KPIs are only relevant to BI or analytics teams. Measuring and monitoring payment metrics and other data doesn’t take priority in running the daily affairs of Fintech operations. Or does it? What if we told you that the opposite is true. If Fintech companies want to protect revenue, payment operations teams must be able to find and fix transaction issues as they’re happening. In an increasingly digitized and competitive environment, no one can afford to wait for periodic reports to provide the necessary insights to run and optimize their daily operations. It’s time for data to be approachable and understandable to all business units, and we’ll explain why in this post. Read on to discover how to improve transaction metrics monitoring to meet the challenges that lie ahead - or on your table right now. Using transaction data proactively Transaction processing metrics are significantly more complex to monitor than most digital metrics like web traffic or engagement. On top of the financial responsibility and risks, teams are dealing with heightened operational complexity. Just think how many steps are necessary for a single transaction on your site and how many parties are entangled. Many stages require verification, authentication, and approval. It’s never just a click. With so many intersections and points of friction, there’s a lot that can potentially go wrong. A glitch in any of the related algorithms, APIs, or other functionalities can cause chain reactions in a whole series of processes and immediately lead to reduced customer satisfaction and eventually to a loss of revenue. It also means there are many opportunities to optimize processes and increase efficiency. At each link in the chain, there’s something to improve. To make both possible - detect failures and opportunities - it’s critical to monitor the entire set of digital payment metrics. Currently, that’s in the hands of the BI or IT teams. Operational teams depend on standardized reports of historical data after it passes through the relevance filters of the data analysts. You may be missing specific transaction metrics that could provide a valuable understanding of how consumers behave or point towards weaknesses in the operational processes. You are definitely losing time when it comes to identifying failures. Why organizations need more granularity for payment metrics The amount of data and metrics to monitor has become overwhelming even for the dedicated business units. There are only so many dashboards a human being can observe. To remain efficient, they currently focus on critical business metrics and generalized data. Alert systems notify about irregularities based on manually set static thresholds, causing alert storms when there are natural fluctuations. Let’s imagine transaction performance metrics show a decrease, and the data you receive helps you identify a reduced payment approval rate. That’s still a pretty general observation that creates more questions than answers. A more granular view of the data, such as by location, vendor, payment method, device, and so on, could deliver insights that point you towards the cause. The same is true for optimization efforts. With a deeper level of granularity, companies can pinpoint weaknesses and strengths more precisely and act upon them with a higher chance of success. You can easily identify your highest-performing affiliates or discover the geographical locations you are most popular in.   Revenue-critical KPIs to monitor Because there are so many metrics and dimensions to measure across the payment ecosystem, it’s important to focus on the most critical KPIs. Fintech operations teams should make sure they have accurate and timely insight into the following metrics: Payment approval - compare payment requests vs. payments approved. With Anodot you can identify discrepancies on the spot and reduce the time to identify and fix issues. Merchant behavior - measure the number of transactions, financial amounts, and more. Anodot lets you analyze merchant behavior and uncover ways to optimize marketing and business. Vendor routing - evaluate your payment providers. Anodot helps you focus your efforts on the strongest vendors. APIs - nothing goes without functioning APIs in fintech. With Anodot you can easily monitor the functionality and ensure smooth processes. Deposits and settlements - monitor the two layers for payment. Use Anodot to stay on top of the entire payment process and increase efficiency. Processing intervals - keep an eye on the time it takes for payments to go through. With Anodot you’ll know right away when there’s a delay somewhere in the system and can avoid customers being disappointed and abandoning your site. The benefits of real-time payment metrics The problem with the current method of analyzing transaction metrics analysis is that data is historical, too generalized, and not effectively prioritized. In other words, by the time the information reaches you, it already belongs to the past. Strictly speaking, decisions are based on outdated information. Real-time data enables you to see and react to what’s happening right now. That may not sound all that beneficial at first. Some people even find the thought of having to respond in real-time stressful. But monitoring real-time data doesn’t mean you sit around watching your data monitor like a flight supervisor. Back to the payment approval issue; The tool correlates out-of-the-ordinary data behavior and finds related incidents in real-time. Instead of you - or a data person - digging up possible related metrics and creating reports to see what caused the drop, the tool points you towards the cause and the solution. How AI makes data accessible to more business units Anodot’s AI-driven business monitoring tool learns normal data behavior patterns, taking seasonal changes and fluctuations into consideration to identify anomalies that impact business. Anodot monitors all your business data and learns the behavioral patterns of every single metric. The monitoring goes on even when you are not looking, distilling billions of data events into single relevant alerts. Anodot also correlates irregularities in transaction metrics with other data and notifies the relevant business units. This means, when you receive an alert, it contains maximum information to help you get to the bottom of what’s happening and how things are connected. Let’s say you detect a drop in deposits. Anodot correlates all related metrics and identifies that all activities with a specific vendor are down, so the failure is with that particular vendor. You are a huge step closer to the next phase of problem-solving. Anodot also prioritizes and scores the severity of an alert based on its financial impact. You only get notified about the metrics that are relevant and need immediate attention. Autonomous payment metrics monitoring for higher efficiency Only an AI/ML-based solution that autonomously monitors all metrics, correlating and prioritizing data, can ensure that each business units receive the insights they need when they need it. The days where data was the sole domain of a chosen few are over. In today’s digitalized business environments, data is everywhere and needs to be accessible to those who need it most. Monitoring data is part of a daily routine, just like keeping an eye on the fuel gauge in your car to know when you need to refill.

Companies produce, collect and manage massive amounts of data Recently in TechBullion, Anodot’s CEO, David Drai, addressed the question, ‘Why Every Company Needs DataOps’ With DevOps, IT was finally recognized as the strategic advantage that the business needed to beat the pants off their competition. Companies now deploy code tens, hundreds or even thousands of times per day, while still delivering unsurpassed stability, reliability and security. DevOps isn’t foolproof Drai expands by saying: “I can cite hundreds of major and expensive incidents that even DevOps couldn’t protect businesses from facing.” More and more, organizations have come to the realization that DevOps is just a part of the solution for maintaining reliable business performance. Where does DevOps Fall Short? While DevOps plays a key role in minimizing the friction between development and production, BI teams see a similar struggle with between backroom and front room developers. The challenge is in closing the gap between these two areas. Drai wrote, “Devops understands monitoring without a holistic understanding of the business and its granular data.  On the other end of the spectrum are BI and data teams that do have a nuanced understanding of business data, but are lacking in tools for around-the-clock monitoring and alerting to abnormal behavior of the data.” What is DataOps? Companies rely on data from a variety of different sources, helping them to gain a better understanding of customers, products, and markets. Explained Drai, “an entirely new role is needed: DataOps.  Because of the dynamic nature of data and the constant new services, partnerships, and products entering the market every quarter, the DataOps role is ongoing and should comprehensively understand and use the proper tools to monitor the ebb and flow of company data including business anomalies, trend changes, changes in predictions, etc.” Why not traditional BI role? How does DataOps differ? The skills gap will not be found in traditional BI strategies. The DataOps role will fill growing gap by working with data across the organization and uncovered a better ways to develop and deliver analytics. “As the focus of DataOps is to monitor and understand all company data, there is a strong existing link between this role and existing company roles like BI analysts and data engineers,” Drai emphasized, “Each role is unique enough to stand on its own, and all three should be reporting to a Chief Data Officer, a position that is becoming increasingly prevalent in data-driven companies.” Next Step See the full article on TechBullion: From DevOps to DataOps : Why Every Company Needs DataOps  

This new report from Blue Hill Research takes a closer look at how enterprises deploy DataOps models to establish the free flow of data within their organization. It includes real-world case studies which demonstrate how organizations in various industries from retail and ecommerce to education are leveraging new technologies to break down silos.

Learn the key components and processes of automated forecasting, as well as business use cases, in this 3-part series on time series forecasting.

Unchecked Vulnerability Leaks Information on Millions The headline was almost too big to believe. On Sept 7, The New York Times announced, “Equifax Says Cyberattack May Have Affected 143 Million in the U.S.” This meant that personal credentials, like Security numbers and other data, for almost half the population of the United States was leaked to hackers. The Verge added, “It has been marked as the worst data breach in US history.” As the picture becomes clearer, the issue at stake was a vulnerability in one of the plugins in the Apache Struts framework. Former Equifax CEO Richard Smith said. “It was this unpatched vulnerability that allowed hackers to access personal identifying information.” This week, in Congressional testimony, The Guardian reported, “It’s like the guards at Fort Knox forgot to lock the doors and failed to notice the thieves were emptying the vaults,” Greg Walden, the chairman of the House energy and commerce committee, told Smith. “How does this happen when so much is at stake?” Walden said. “I don’t think we can pass a law that fixes stupid.” The question arises, if Equifax had an AI-powered analytics solution that tracked anomalies in real time,  would this have surfaced the hack immediately, giving the company plenty of time to respond, and thwart any damage? What happened with Equifax? Equifax is one of the three major consumer credit reporting agencies. The company reported on September 7th that hackers had gained access to company data that potentially compromised sensitive information for 143 million American consumers, including Social Security numbers and driver’s license numbers, posing serious repercussions for identity theft. Dan Goodin reported in Ars Technica “The breach Equifax reported Thursday, however, very possibly is the most severe of all for a simple reason: the breath-taking amount of highly sensitive data it handed over to criminals. By providing full names, Social Security numbers, birth dates, addresses, and, in some cases, driver license numbers, it provided most of the information banks, insurance companies, and other businesses use to confirm consumers are who they claim to be.” While still unclear who was behind the attack, with some conjecturing that this was a state-sponsored attack, the data could now be in the hands of hostile governments, criminal gangs, or both and will stay there indefinitely. This leaves over half the population of the US’s vital identifying information exposed. Even worse, while the leak occurred in the spring, the company only went public in September. “The fallout has been swift, with government agencies looking into the incident, class action lawsuits being filed, and consumers demanding free credit freezes.” Why did so much personal data get leaked from Equifax? Cybercriminals exploited a security flaw on the Equifax website. Brian Krebs reported on KrebsonSecurity how the criminals did it. "It took almost no time for them to discover that an online portal designed to let Equifax employees in Argentina manage credit report disputes from consumers in that country was wide open, protected by perhaps the most easy-to-guess password combination ever: “admin/admin.” Looking deeper into the hack, Blogger and admin for SPUZ said, “I asked the hackers one last request before disconnecting. I asked, "How did you manage to get the passwords to some of the databases?" Surely the panels had really bad security but what about the other sections to them? Surely there was encrypted data stored within these large archives no? Yes. There was. But guess where they decided to keep the private keys? Embedded within the panels themselves.” Equifax has confirmed that a web server vulnerability in Apache Struts that it failed to patch months ago was to blame for the data breach. DZone explains how this framework functions. “The Struts framework is typically used to implement HTTP APIs, either supporting RESTful-type APIs or supporting dynamic server-side HTML UI generation. The flaw occurred in the way Struts maps submitted HTML forms to Struts-based, server-side actions/endpoint controllers. These key/value string pairs are mapped Java objects using the OGNL Jakarta framework, which is a dependent library used by the Struts framework. OGNL is a reflection-based library that allows Java objects to be manipulated using string commands.” How could AI-powered Analytics have made an impact? This situation could have been reacted to much faster had the right real-time business intelligence services been integrated into their systems. Approaches like this, such as Anodot’s AI-powered Analytics solution, correlate a company’s raw data to quickly identify anomalous behavior and discover suspicious events in real time, before they become crises. Once an issue is detected technical teams are alerted, so they can resolve issues before they unravel. Companies need to know what their data can tell them right away in order to fix costly problems. Working at the scale of actively monitoring thousands or even millions of metrics, you need an AI-powered analytics solution, with automated real time anomaly detection. Had Anodot’s AI-powered analytics been in place, it could have tracked the number of API Get Requests for user data, and noticed an anomalous spike in requests, catching the breach instantly, regardless of the existing vulnerabilities.

Learn the key components and processes of automated forecasting, as well as business use cases, in this 3-part series on time series forecasting.