The “AI” label is everywhere, from apps to startups and even on smart fridges. But while some systems rely on real sophisticated models, others use AI as little more than a buzzword.

AI Hype Meme

That gap between hype and real understanding is what motivated this series. Instead of vague promises about AI magic, we’ll explore the foundations of deep learning through a concrete security challenge: detecting malicious URLs. No black-box, no shortcuts, just a practical journey from AI basics to building and deploying a working model.

Why Deep Learning for Security?

If you’ve ever written YARA rules, built detection logic or tuned SIEM queries, you’ve done feature engineering: manually extracting meaningful patterns from raw data. You know the pain: every new attack variant breaks your carefully crafted rules, every evasion technique requires another regex, every false positive demands another exception.

Deep learning offers a different approach: teach the system to recognize underlying patterns rather than enumerating every possible variation. Instead of writing rules for each threat, you provide examples and let the model discover what distinguishes malicious URLs from benign ones. This doesn’t replace your expertise, it amplifies it. Your knowledge still guides feature selection, model architecture and evaluation, but the model handles learning complex patterns from thousands of real examples.

⚠️ Disclaimer
This series is not a substitute for a full and rigorous deep learning course. Its goal is to introduce key concepts in an accessible and practical way, particularly for readers with a security background and to provide a solid foundation for the more advanced blog articles that will follow.

If you’re looking for a deeper, textbook-level treatment of the subject, I highly recommend Dive into Deep Learning: a free, open-source book with hands-on examples and theoretical depth.

So… What Is AI, Really?

At its core, Artificial Intelligence (AI) is any technique that enables machines to mimic or approximate human intelligence, such as reasoning, decision-making, planning or learning.

That could mean a neural network… or just a clever graph search algorithm. Yes, even a navigation app choosing the fastest route using A* graph search or a chess engine calculating the best move using minimax falls under the umbrella of AI. These systems don’t learn from data: they make decisions by reasoning through a problem space.

This highlights an important distinction at the heart of AI. Just like humans have two modes of thinking, intuitive and analytical, AI also has two fundamental approaches: learning from data and inference through logic and planning.

  • Learning corresponds to our intuitive mind: recognizing patterns from experience, making fast decisions based on past exposure. This is where machine learning and deep learning fit in.
  • Inference mirrors our analytical mind: step-by-step reasoning, solving problems with structured rules, searching through possibilities.

Understanding the Two Approaches

The “analytical mind” of AI is methodical, logic-based problem solving. Think of it as the computer scientist’s approach: if you can define the problem clearly enough, you can write an algorithm to solve it. These systems excel at explainability (you know exactly why they made a decision) and consistency (same input always produces same output). However, they struggle with novel scenarios not covered by their programmed rules. Intelligence in reasoning-based systems emerges from their ability to systematically explore complex problem spaces and find optimal solutions. A chess engine doesn’t just follow simple rules but demonstrates intelligence by evaluating millions of possible future positions and understanding strategic concepts like piece value and positional advantage and adapting its strategy based on the opponent’s moves.

While the AI’s “intuitive mind” focuses on recognizing patterns from past experience. Instead of being told what to look for, these systems learn what matters by seeing thousands or millions of examples. These systems excel at adaptability (can handle new, unseen scenarios) and pattern recognition (finding subtle relationships humans might miss). However, they can be unpredictable and difficult to explain. Intelligence in learning-based systems manifests as emergent understanding that goes beyond memorization. A model that distinguishes photos of dogs from those of cats develops an internal representation of what the two animals look like, allowing it to predict between the two which is more likely.

Most of the current AI buzz focuses on the learning side, but true intelligence often combines both. In fact, many real-world systems still rely heavily on inference, especially where planning, optimization or safety is critical.

Agents, Models and Goals

Now that we understand how AI systems “think”, let’s explore how they act in the real world. Whether using reasoning or learning or both, AI systems need a framework to operate effectively: the agent. An agent is any system that perceives its environment and takes actions to achieve a specific goal.

The key insight is that agents exist at many levels of complexity:

  • Simple agents: A firewall that blocks or allows network traffic based on rules.
  • Complex agents: A threat intelligence platform that continuously analyzes new malware samples, correlates threat data and updates detection rules.

What makes them all agents? Each one:

  • Perceives its environment (network traffic, malware samples).
  • Processes that information using its internal world model.
  • Takes actions toward a goal (block threats, update defenses).

The goal is the “why” or the agent’s purpose. The agent’s intelligence, the “how” of its operation, comes from its internal world model. A world model is simply the agent’s internal representation of how its environment works, whether that’s traffic rules for navigation, piece movements for chess or threat patterns for security. This model is the agent’s brain, a representation of its environment and the rules that govern it.

In this classic approach, the agent’s world model is explicitly built by human developers with rules and logic. Think of it as a hand-crafted brain:

  • For a navigation app, the model is a map combined with explicit rules about speed limits and one-way streets. For a chess engine, the model is the set of rules for how pieces move and the conditions for winning or losing the game.
  • This agent achieves its goal through inference and planning. It uses its hardcoded model to reason step by step, searching through all the possibilities to find the optimal path or the best move. Its intelligence lies in the cleverness of its search algorithms and the accuracy of its pre-programmed world. It does not learn from experience, it reasons through a problem it already understands.

In modern approach, the agent’s world model is not programmed but learned automatically from data. It builds its own brain through exposure to countless examples and experiences.

  • Consider a self-driving car navigating a busy city. It is impossible for a developer to write a rule for every conceivable event. Instead, the agent learns an internal model of the world by processing millions of hours of driving data and simulations. It learns to identify pedestrians, predict the behavior of other drivers and understand complex traffic flows on its own.
  • This agent achieves its goal through pattern recognition and prediction. Its actions, like steering or braking, are not based on a simple checklist. They are the result of continuous predictions based on its learned model.

Although these two types of agents use different strategies, both are united by the same principle: using a model of the world to achieve a goal. The first reasons with a map it was given, the second learns to draw its own.

Learning-Based Systems

As the name suggests, this blog will focus on deep learning, so it is essential to explore the entire category of models based on learning.

Machine Learning (ML)

Whenever a model improves through exposure to examples, it falls under the umbrella term machine learning: a subset of AI that focuses on building systems that learn patterns from data, rather than being explicitly programmed with rules. For example, instead of writing code to define what a phishing URL looks like, you can train a model using thousands of labeled URLs and it will learn to recognize phishing attempts on its own.

To understand how this works in practice, imagine you want to build a system that classifies animals as cats or dogs based on their weight and height. In a traditional programming approach, you might write rules like “if weight > 20kg then dog” but this would fail for small dogs and large cats. Instead, machine learning lets the system discover the patterns by itself. You start by collecting training examples: hundreds of data points where each animal has known measurements (weight and height) and a correct response (cat or dog). The ML algorithm examines all these examples and gradually learns that while there’s overlap between the groups, dogs tend to cluster in one region of the weight-height space and cats in another. It discovers boundaries that best separate the two groups. When you later show the trained model a new animal with weight 8kg and height 30cm, it can predict cat based on the patterns it learned from the training data.

Machine learning problems usually fall into one of a few task types, each describing what you want the model to predict:

  • Classification: Predict one or more categories from a fixed set.

    Is this URL benign or malicious? What malware family is this: ransomware, trojan, adware or spyware?

  • Regression: Predict a numeric value.

    What’s the potential ransom demand from the cybercriminals who targeted us?

  • Clustering: Group similar entities or detect unusual patterns without predefined labels.

    Cluster users with similar behavior or flag traffic that looks nothing like the usual baseline.

To solve these tasks, ML models follow different learning strategies, depending on the kind of data available, especially whether labels (the correct answer) are provided:

  • Supervised learning: Each example in the dataset comes with a known label.

    For instance, a file labeled malicious or benign. The model learns to map inputs (features) to outputs (labels) and generalize this mapping to new data.

  • Unsupervised learning: No labels are provided.

    The model explores the structure of the data to find groupings or patterns. For example, clustering new types of malware without knowing their names.

  • Semi-supervised and self-supervised methods also exist, combining elements of the two, particularly useful when labels are scarce or expensive to obtain.

Regardless of the strategy, the goal is always the same: use data to train a model that can make useful predictions or discoveries. How that training happens and how we evaluate it, is the focus of the next articles.

Deep Learning (DL)

When patterns are too subtle or too intricate for classical algorithms, we turn to deep learning. This machine learning approach uses structures called artificial neural networks: they basically consist of multiple layers of processing, which is where the word deep comes from.

Each layer takes the input, transforms it a little and passes it on to the next. Step by step, the network learns to extract increasingly meaningful features from the raw data: like going from pixels to shapes to objects in an image or from characters to words to intent in a text. At the end of the network, there’s an output layer, this produces the final prediction: phishing vs. benign for a URL or cat vs. dog for an image. The network learns by comparing its predictions to the correct answers and adjusting itself to improve over time.
These architectures are loosely inspired by how biological neurons work, but make no mistake: they’re not tiny brains. They’re mathematical models made of simple building blocks (like weighted sums and non-linear functions), stacked together to solve complex problems by learning patterns in data.

Think of a deep network like LEGO: individual bricks (neural network layers) are simple, but by putting them together in different ways you can build anything. This flexibility allow us to address very different problems with one extensible toolbox.

Deep learning is at its best when data are unstructured and highly dimensional: images, audio, text. In security, this translates to:

  • Binary analysis: Learning malicious patterns from raw executable bytes.
  • Log analysis: Capturing complex event sequences indicating malicious behavior.
  • URL analysis: Recognizing phishing patterns across character sequences, not just blacklists.

Unlike classical ML models, where analysts must manually design features (e.g. extracting URL length, subdomain count, etc.), deep networks can learn hierarchical representations directly from raw or minimally processed data (e.g. character sequences in URLs). This ability to automatically learn features makes deep learning particularly powerful for complex tasks where relevant patterns are not easily defined by humans. Of course, this power has a price. Deep models are data-hungry (you need thousands of labeled examples), need heavy computation (typically GPUs, not just CPUs) and can be opaque (hard to explain why a model flagged something).

One of the most visible applications of deep learning is Generative AI: models like GPT, Banana or Veo that can generate realistic text, images or video. These models are trained on massive datasets with billions of parameters using advanced architectures like transformers. Generative AI excels at certain tasks, particularly those requiring zero-shot capabilities (handling tasks without specific training) or processing complex text-based data. A large language model can summarize documents, answer questions or assist with coding without being explicitly trained for each specific use case. This flexibility makes generative AI powerful for general-purpose applications.

However, for specialized production systems processing large volumes of data, custom models trained on high-quality domain-specific data often outperform generative AI. Consider a URL classifier: a generative model could classify URLs, but it would be slower, more expensive to run and potentially less accurate than a dedicated neural network. When processing thousands of URLs per day with sub-100ms latency requirements, even small differences matter: a few percentage points of false positives translate to hundreds of legitimate sites wrongly blocked, while extra seconds of latency break real-time response requirements. In these scenarios, building and training your own deep learning model tailored to the specific task and data often yields better performance, efficiency and cost-effectiveness.

The Data Scientist’s Mindset

Data science isn’t just math and code: it’s a way of thinking.

Now that we understand how AI agents operate and how machine learning discovers patterns in data, let’s explore how to apply these concepts systematically to solve real security problems. This is where the data scientist’s mindset becomes crucial: it’s not about throwing algorithms at data and hoping for the best but about approaching problems with a structured and scientific method.

The Key Components

At its core, data science is about using data to answer real-world questions. That requires a blend of three key pillars:

Pillar Why it matters in cybersecurity
Mathematics & Statistics Understand uncertainty, compare models, spot bias and choose the right metrics.
Computer Science & Engineering Collect, process and model large-scale data using efficient and scalable tools.
Domain Knowledge Ask meaningful questions, interpret patterns and know what matters in context.

That third pillar, domain knowledge, is often underrated, but it’s where security researchers shine. You don’t need a PhD in math to build useful models. If you understand attacks, behaviors and patterns of abuse, you already hold critical intuition. Machine learning can sharpen that intuition and statistics can help you validate it.

Once that core idea is in place everything else (e.g. modeling, metrics, tuning) can follow. That’s the essence of data science: combining real-world insight with analytical rigor. You use data to test your assumptions, refine your thinking and build solutions that actually work. And when it’s time to optimize, a data scientist or statistician can help push your model further.

A Practical Roadmap

The data science process is inherently iterative. You rarely get the right answer on the first try and that’s perfectly normal. Instead you cycle through phases and refine your approach based on what you learn. In this series, we’ll use that mindset to build a working deep learning model: from raw URLs to threat detection. Not by randomly throwing models at data but by following a structured and scientific process:

  1. Understand the Problem
  2. Explore and Prepare the Data
  3. Build and Train the Model
  4. Evaluate the Results
  5. Improve and Iterate

This structured approach isn’t just for deep learning: it applies to most applied data science projects. By the end of the series, you’ll have more than just a working model: you’ll have a method for tackling new problems with a repeatable, scientific mindset.

1. Understand the Problem

First, we define what we’re trying to solve. In our case: Can we automatically detect malicious URLs based on their structure?

To do this well, we also need to think carefully about a few key points:

  • What data do we have (or can collect)?
    In this case, we’ll use a Kaggle dataset containing over 650,000 labeled URLs, grouped into four categories: phishing, malware, defacement and benign.
  • What kind of output do we expect from the model?
    A simple yes/no? A risk score? A category like phishing, malware, defacement or benign?
  • How will we measure success?
    Do we care more about catching every threat? Or avoiding false alarms? How do we quantify “good enough”?

Getting this stage right is critical. It doesn’t matter how advanced the model is: if the goal is unclear or if we optimize for the wrong thing, the results won’t be useful in practice.

2. Explore and Prepare the Data

Once the goal is clear, the next step is to understand the dataset and make it ready for training. Before building any model, it’s important to explore the data: this helps catch obvious issues like missing values, duplicates or labeling mistakes and gives you a feel for the patterns (or lack of them) in the dataset. Exploration also reveals hidden biases that can sabotage your model. For instance, if benign URLs in your dataset rarely include http:// while malicious ones always do, the model will learn this spurious correlation instead of actual threat indicators. Catching these problems early through visualization and statistical analysis saves you from building a model that performs well on test data but fails in production.

Next comes preparation: cleaning the data, removing noise and structuring it in a way that models can use. This step is often underestimated, but data issues here will always show up later as poor performance. Deep learning models only understand numbers: not raw text, labels or protocols. So every input must be converted into a numeric format: labels like malware or benign become integers, text is tokenized and embedded, sequences (like system calls or network packets) are turned into numeric arrays and so on.

Finally, the dataset is usually split into separate parts:

  • A training set: used to train the model.
  • A validation set: used during training to tune and evaluate.
  • A test set: used only at the end to assess real-world performance.

Keeping these sets separate is critical to prevent data leakage: when entries from the test set accidentally influences training, leading to overly optimistic performance estimates that don’t hold up in the real world.

3. Build and Train the Model

Now it’s time to turn data into predictions. We feed the model a sequence of numeric features extracted from each URL as input and expect it to produce a numeric label as output (e.g. 0 for benign, 1 for malicious). Training means adjusting the model’s internal parameters so that, given these inputs, it consistently outputs the correct labels.

Choosing the right architecture and matching its complexity to the data, matters:

  • A simple model may not capture enough detail, leading to underfitting: it fails to learn the underlying patterns and performs poorly even on training data.
  • A complex model may learn too many details, leading to overfitting: it memorizes the training data but fails on new examples. Overfitting is like memorizing practice questions instead of learning the basic concept, so when the real test comes, you’re lost.

4. Evaluate the Results

Once trained, we test the model on data it hasn’t seen before. This tells us not just how accurate the model is but also how it fails:

  • Too many false positives (flagging safe URLs as malicious)?
    You’ll block legitimate users and hurt trust.
  • Too many false negatives (missing real threats)?
    You’ll leave the door open to attacks.

Depending on the context, one type of error can be far more costly than the other: in an XDR a false negatives can mean real damage. Also, be careful with accuracy as a metric. Accuracy is just the percentage of predictions the model gets right overall. Suppose 90% of the URLs in your dataset are benign and only 10% are malicious. A model that blindly labels every URL as “benign” would boast 90% accuracy (yet it would still miss all of the malicious URLs).

Good evaluation means looking beyond the overall score and understanding the model’s behavior where it matters most.

5. Improve and Iterate

Machine learning is rarely perfect on the first try, here’s where we refine: testing new features, adjusting architectures, cleaning data, tuning parameters.
Even small changes can make a big difference!

Throughout this series, we’ll walk through the full pipeline, step by step, from raw data to a production-ready deep learning model.

References