David García

X-ray of a model Let’s take a look at a model from the inside, physically. For that, we’ll use this one: https://huggingface.co/distilbert/distilbert-base-uncased It’s a model (a distilled version of BERT) with a modest 67 million parameters. A bit old (in internet terms), but perfect for illustration purposes. With a short script using the 'transformers' library (also from HuggingFace), we can download the model into the local directory: It downloads two files: One is the model itself, model.safetensors (253 megabytes). The other is config.json, which contains the model’s architecture—required for loading the model correctly with the library we’ll use (PyTorch). This config.json file also provides us with a lot of valuable information. Let’s focus on some of the key data in that file. Disk space required by the model’s vector space The vocabulary size (vocab_size) the model 'understands' is 30,522 tokens: Each of those tokens is associated with a vector (dim) of 768 'slots' or values. This vector is the same one we introduced at the beginning of the article, as a reminder: TEF model: [1,1,1,1,1] The piano: [0, 0.2, 0.8, 0, 1] In other words, a position within a 768-dimensional vector space (keeping in mind that a Cartesian plane only has two). Now, there’s another important point: each of those 768 data points in the vector space has float32 precision (torch_dtype), defining a unique token among the 30,522. That float32 format means 4 bytes (or 32 bits). So, on disk, just the tokens require: 30,522 * 768 * 4 bytes = approximately 90 megabytes. By the way, if you're curious to see the vocabulary of this model (the tokens), it’s available here. Layers and parameters Let’s not forget a key characteristic of neural networks: layers. From the input layer, through the hidden layers, and out to the output layer, data flows and bounces across all the layers to produce the final result. Our model under analysis has roughly 67 million parameters. Just the embeddings layer (which holds the tokens and their associated vectors) contains: 30,522 * 768 = 23,440,896, around 23 million parameters. We also need to add the parameters for each of the model’s layers. According to the "config.json" file, this model has six internal layers (the transformer model), indicated by the "n_layers" field. In addition to those layer parameters, the model has others used to model bias, normalize values, and apply various tricks that help generate the final result: the model’s answer. When you sum it all up, you reach that 67 million parameter count. What does a model look like on the inside? It’s a physical file (or sometimes a set of files). It includes a metadata field describing the internal structure, and the rest is binary data. Let’s look at the metadata header: The rest of the file consists mainly of the weights of the different layers, in binary forma; naturally, unintelligible: Using a small script with the safetensors library (also from HuggingFace), we can visualize what we discussed regarding the layers: Take a look: we see the embeddings layer we examined earlier. With certain distributions (which may vary from one model to another), represented as tensors: word_embeddings, easy to identify by its shape (30,522, 768), is a matrix (or 2D tensor) of 30,522 rows corresponding to the tokens, each with a 768-float32 vector; as indicated by the 'dtype'. position_embeddings is another matrix, sized (512, 768). Note how important this is. When a phrase or paragraph is passed to the model as tokens, the model has no 'awareness' of their order. That’s why this space is used to organize the input. That 512 value isn’t arbitrary: it’s the model’s maximum context length. Any input beyond 512 tokens will be truncated and not processed. It’s like the model stops listening after 512 tokens. Of course, it can process inputs in chunks (chunking), but… well, it’s not the same. ■ To give you an idea of modern context sizes: a model like Llama 3.2 1B or 3B has a context window of 128,000 tokens. Even Llama 4 Scout (though with more demanding requirements) can handle a context of 10 million tokens. Context is critical for a model to understand what we’re asking it. The larger the context window, the higher the fidelity and quality of the response. The elements shown as LayerNorm bias and weight are two vectors (1D tensors) with values learned during training. These are used to fine-tune (scale and shift) the output from the combination of word_embeddings and position_embeddings, just before the information enters the network layers. Next, we see the actual layers—six in total, indexed from zero to five, all with the same types of tensors: The first thing to understand is that all the tensors in the model are, essentially, either weights (weight) or biases (bias). Weights are the major parameters the network learns during training. They act as the 'wiring' that transforms data from one layer to the next. Bias is a type of 'fine-tuning' that adjusts each layer’s output ensuring, for instance, that even if the input is all zeros, the output can still be meaningful. It’s a subtle yet essential component. So, each layer usually comes in pairs: a weight matrix and a bias vector to ensure proper performance in all situations. We won’t go into every distinction, but one thing stands out: the label 'attention'. You’ve probably come across the phrase Attention is all you need. These are the parts of each layer that belong to the attention sub-layer or module. Instead of processing text linearly (or sequentially), the attention mechanism—the core of the Transformer model—lets the model analyze the input (what we say) by interrelating all words simultaneously. It assigns each word a “score” based on the context, determining which to focus on more than others. A true revolution. ■ FFN is a familiar type of neural network: "Feed-Forward Network", which complements the attention mechanism. Thanks to the FFN, the model can focus (this time on a single token) and further refine the result coming from attention. In fact, if we look at the iconic diagram from the famous paper, we’ll see FFNs play a key role as a model component: Illustration 1 Transformer Model Without this network, the model wouldn’t be able to detect complex relationships or nuances. The rest consists of normalization layers that shape each layer’s output. Bonus track: what happens if we progressively remove layers from a model? Alright, let’s try it out. This is a model that replaces [MASK] with a word that fits the context. We ask it: > Madrid is located in: First, let’s see it in action with all six of its layers: It works correctly, guessing the word 'spain'. Now we create a function that calls the model the same way, but progressively removes its layers... ■ As we can see, the output starts to go off track, since removing layers also reduces the model’s response accuracy. In upcoming articles, we’ll dive deeper into technical and practical aspects of large language models. ■ PARTE I Telefónica Tech The incredible inner world of LLMs (I) June 10, 2025

The concept of a 'model' Have you ever heard or read phrases like “a model student” or “a role model”? A model is a name tied to a set of defined characteristics. If we define a 'TEF' as something that should weigh around a kilo, be made of Bakelite, shiny, have a dial with ten numbers, and a copper system inside... we're assigning a set of attributes to the name 'TEF'. We’ve just named a model. Hooray! Now let's take a real-world object. For example, a piano. Pianos have some copper inside their metal framework. Their keys or plastic components might contain Bakelite. Generally, they don't have a ten-digit dial (though some modern models may have a keypad). But they weigh more than a kilo — much more. Shine... yes, our piano has that classic piano-black gloss... So how similar is the piano to our 'TEF' model? There are many ways to find out. One of them is to place the mentioned attributes in a vector space and use a function to determine how close those attributes are to the model's. Let’s see: Now we have two vectors: TEF model: [1,1,1,1,1] The piano: [0, 0.2, 0.8, 0, 1] There are various ways to calculate similarity: Manhattan distance, cosine similarity, etc... Intuitively, the piano is (roughly) 40% a TEF model. In other words, it's still quite far from being a proper TEF. Now we head to the attic and find an old Bakelite telephone, but instead of a rotary dial, it has buttons (a Heraldo model with a keypad). Let’s calculate the similarity: The similarity is... 80%! Even more so if we were more flexible with the “Ten-digit rotary dial” attribute. If instead we used “Has ten numbers” as one category, and “Has rotary dial” as another, the similarity to the 'TEF' model would reach nearly 90%. So, with this simple example, you now have a rough idea of what a model is in machine learning. The neural network model A neural network is trained with input data: photos of pianos. Thousands, hundreds of thousands... no — millions of piano photos! These are broken down into vectors, and after seeing many photos, a set of attributes is discovered that can represent most pianos in images. A neural network is just that: thousands to millions or even billions of vectors with specific values after being trained on piano photos. These adjustments are what we call the weights of the neural network. And the trained network that detects pianos in photos is called a model. Now you feed it ten photos of cats, and among them, one of a cat playing a piano. The network will convert the photos into vectors, pass them through its layers (like bouncing around in a pinball machine), and return a score indicating whether a piano is present. If the training was successful, only the photo of the cat playing the piano will score high enough to be a match. What’s more, if we only trained it on black pianos, our model might hesitate if shown a piano of another color. It would recognize the shape, score it well, but dismiss it as a piano because the weights say a piano can only be black. ■ This is what's known in machine learning as overtraining or overfitting. Without variety in the dataset, other attributes may stray too far from the model. And beware — if we provide too much variety and randomness, the opposite occurs: underfitting. The network ends up asking, “Okay, lots of photos, but what exactly do you want me to learn?” In this case, it might mistake a piano for a helicopter. But what exactly is a model when we talk about LLMs? Great. Now we have a conceptual idea of what a model is. Let’s get specific about LLMs (language-specialized models). ■ LLM stands for Large Language Model. It's a model that is large and designed to handle human language — take your pick of definition: “large-scale,” “extensive,” “massive.” So why do we call them large? Remember those attributes? Copper, shiny, Bakelite... How many did we count? Five? That’s a tiny micromodel. Now imagine a model with billions (US billions) of parameters. Massive, right? That's why it’s called large — the interconnection between weights (attributes) is so enormous it’s impossible to manually calculate even the distance between two words. But today we have computers with processors and memory capabilities that were once unimaginable. Add distributed computing on top of that. It’s all massive. ■ That’s why AI has advanced so quickly in recent years, emerging from one of its winters straight into a blazing 40º summer in the shade. Can we train models on a modest laptop? Of course you can, but... they won’t have billions of parameters. Training a model involves processing (tokenizing and vectorizing) millions upon millions of documents: all the knowledge you can get. It’s a Herculean task. Training a current LLM can take months. The cost runs into the millions of euros. You need lots of power and extremely powerful processors working in sync for long periods. But if we scale down our ambitions, we can train a small model with just a few million parameters. For example, processing all of Wikipedia might only take a few days on an average GPU. Just don’t expect philosophical sunset chats in white robes, of course. ■ The alternative is to take an existing, solid, reliable model and retrain it for a specific purpose. This is called fine-tuning, which we’ll explore practically in future articles. Where are models hosted and how do we evaluate or classify them? Well, you’ve likely used several models like ChatGPT, Claude, etc. These are private models. You can use them, but you can’t download or run them locally (and even if you could, their hardware requirements are beyond the average user). The good news is that many models are available for download and use. We’ve already discussed how to do it in these articles, and here. Now let’s put on our technical glasses and look at the types of models out there — and where to find them: the land of HuggingFace. This community is packed with models, datasets for training or fine-tuning, papers, docs, libraries, forums, etc. It’s a true bazaar — and fortunately, most of it is open-source or under permissive licenses. In fact, there’s so much available that you can easily get lost in its bustling “streets,” where new models appear at a pace rivaling the JavaScript ecosystem. Take a look: Currently — and still growing — there are nearly 1.7 million models. Easy to get lost... Also (see red box on the left of the image), we have different types of models based on their purpose. That is, models specialized in computer vision: image classification (hello again, pianos!), video classification, image/video description, generation, etc. Besides media-focused models, there are the classic ones specialized in text or natural language processing: text classification, generation, translation, and more. Then there are the multitaskers: multimodal models that combine multiple capabilities under one interface. —For example, you upload an image and “chat” with the model about it. One model processes the image and generates a description; another “head” then chats with you about that description. It looks like a single AI, but it’s actually several working together. 1.7 million models, but there's a catch... Before anything else — from that staggering number, 1.7 million, we need to do some filtering. Many models are already “old”, surpassed by newer versions, and their training data cutoff is now considered obsolete. Beyond that, we also need to divide the remaining number by model variations based on fine-tuning: You take LLaMA 3.2, fine-tune it for a specific task, and publish it on HuggingFace — you now have a “new” model. —For example, Facebook’s flagship model LLaMA spawns from this version: adaptations, fine-tunings, merges with other models, different quantizations… A whole family of siblings and cousins. Take a look at the numbers: You may also have heard of “distill” or distilled models. This is essentially a technique for transferring the knowledge or behavior of a large model into a smaller one — saving training time (and costs) and producing a more practical model in terms of size and requirements. Also, a single model can have multiple versions. Check out this chart for the PHI2-GGUF model: What you see — 2-bit, 3-bit, 4-bit — refers to the model’s quantization. What is quantization? Remember when we talked about vectors and such? We won’t go into math, but this is essentially the “precision” of the neural network layer weights in the model. Let’s look at an example: A person steps on a scale and it reads 74.365777543423 kg. That’s quite precise, right? Now let’s say we don’t want that much precision, so we quantize to 8-bit: The weight would read 74.366 kg. Still too precise. Let’s lower it further, say to 4-bit: Now it reads 74.4 kg. We can go even lower, 2-bit: Now it stores the person’s weight as: 70 kg. What just happened? → As we lowered the quantization, we lost precision in the weight measurement, but gained storage efficiency. In short: We’ve compressed the model’s memory footprint at the cost of precision. Parameters Another important factor to consider is the model’s number of parameters. No matter how well-quantized a model is, if it has too many billions (US) of parameters, it simply won’t fit into an average laptop’s RAM. The model I showed earlier has 2.78 billion parameters. Quantized to 8-bit, its disk size is 2.96 GB; with 2-bit quantization, it drops to 1.17 GB. But if we try this with a model that has, say, 70 billion parameters... this happens: As you can see, my machine (16 GB of RAM) can’t handle even the most aggressive 3-bit quantization — the model size exceeds the total RAM (let alone the available RAM...) In short, no matter how much we quantize, there’s a limit — you can’t load a 70-billion-parameter model on modest RAM. ■ PART II Telefónica Tech The incredible inner world of LLMs (II) June 11, 2025

Working with generative AI has been part of our work since it began to be seen as one of the new revolutions that is going to change everything and... You will have heard the rest many times by now. The truth is that having a deep neural network model that crunches hundreds of gigabytes and learns to do things that used to take us a good handful of hours in a given job and give us a hand, results in a boost to productivity. After all, programming and computers in general never stopped being an accelerator for tasks that we could do “by hand”. In fact, the first computers were created precisely to speed up calculations that were done and reviewed by people in the context of World War II. One thing that was clear from the outset was that to use AI, good computational power was needed. To this end, online services are offered to use it. From free use to subscription, the supply of AI-based services has produced a veritable compost field whose rate of emergence makes it impossible to keep up with (significantly faster even than the rate of emergence of JavaScript frameworks). LLMs have been democratized to the point of providing models for mobile devices and even much more modest in terms of hardware features. However, is it really necessary to have a high-end machine to run a model for chatting or generating images? Nothing could be further from the truth. LLMs have been democratized to the point of providing models for mobile devices and even much more modest in terms of hardware features. In addition to lowering the requirements, the deployment (installation) is increasingly simple and its interface more pleasant for the person with little technical knowledge. We are going to present several alternatives, both proprietary and open-source applications that we can install on our computers (of course, we will not use that laptop that has been in the storage room for 12 years) and experiment with generative AI and "on premise", without resorting to online services. LM-Studio, easy and fast The first of all, the most “user-friendly” is LM-Studio. Extremely easy to install and comes with a relatively intuitive user interface. Link: https://lmstudio.ai/ Once installed the first thing it asks us is to choose and download a model for local use: We choose the default one: Llama 3.2 of 3 billion parameters (about 2Gb). An undemanding model, although of course, will have its limitations while giving a lot of play. On the machine we used for testing (Apple M1 Pro), Llama 3.2 3B gave a rate of almost 60 tokens per second. Very good numbers for local execution. In addition, the response at all times is consistent. All in all, it is an option to quickly and smoothly experiment LLM on your machine without requiring large requirements, which is the aim of this article. Telefónica Tech IA & Data You can run Generative AI models on your computer: Step-by-step instructions to install LM Studio January 8, 2025 Ollama, open-source and with a lot of options An open-source project made in Go that allows us to run a large number of models. You only need 8Gb of RAM to run models like Llama 3, Phi 3, Mistral or Gemma 2 among others as long as you don't use the versions with more than 7 billion parameters. Ollama is the core, which is used from the command line. However, it has numerous options to attach a user interface if we are not used to dealing with the terminal. Below you can see the various user interface options: https://github.com/ollama/ollama?tab=readme-ov-file#web--desktop Its installation leaves us with only a terminal command, 📍“ollama” and an application that raises a background service. By doing an 📍“ollama run phi3” from the command line, the Phi3 model will be downloaded, and we can start chatting with the model. If you are used to the terminal and don't need to attach a graphical interface, you can have it running and you can always access it quickly for any query. We even asked it to do a simple Python function using a well-known framework for web applications. It comes out quickly, albeit with its bugs. Ollama is flexibility and above all the confidence of being open-source. Nothing you ask or talk to the model will leave your machine. llama.cpp, Georgi Gerganov's revolution Open-source project started by the brilliant Bulgarian engineer Georgi Gerganov. Inspired by previous projects (whisper.cpp) and shocked by the appearance of Meta's Llama model for free use, Georgi realized that to run the model required the use of PyTorch-type framework, which was an additional burden on the machine. As with some of his other projects, he started an implementation in C (and C++) that had efficiency as its flagship. The challenge, which was not a small one, was to make a platform to load the model (Llama) on computers with modest specifications. What started as a personal challenge exploded and is now surrounded by a large community that contributes to all aspects: adaptation to other models, creation of an interface, server mode, etc. and even a new model format adapted to the internal architecture of llama.cpp. In fact, the project is used by many services that are just a mere layer on top of llama.cpp. Interestingly, although the original name is llama.cpp, as mentioned above, it currently supports numerous models. Thanks to the contributors of the project, its installation is simple. Just download the precompiled file from the publication page. Once installed, it presents us with a series of command line tools. Although the appearance is reminiscent of an environment exclusively for advanced users, setting up the server and its web interface is a relatively simple process. Of course, we will have to download the model (something tremendously simple, just explore the models with format converted to GGUF.) Once downloaded and from the installation folder where the 📍"llama-server" command is, this is all we have to do: Once the server is up, we visit 📍“localhost:8080” in the browser and we will see an interface that will remind us of ChatGPT through which we will be able to “chat” with it. Conclusions LLMs are not only here to stay but to become a big part of our lives. This is just the beginning. We will foreseeably see how they will increase their context, their affinity with our questions and conversations or their knowledge of the medium and the context of the conversation. We have not talked about models for generating media formats: photos, videos, etc. That would be something else for another article. But it is possible to load models that take care of these tasks. The purpose of this article is simply to show that we do not need either a big machine to use models or online services to take advantage of them and start experimenting with AI at home. Working with generative AI has been a part of our daily work since it began to emerge as one of the new revolutions. Telefónica Tech IA & Data How to install and use DeepSeek on your PC February 1, 2025

A very important (better said, fundamental) part of threat intelligence is the acquisition of signatures that help identify components of a campaign's infrastructure. They are “fixed points” that help us to identify and coordinate defensive actions. Without them, it would be very difficult to target a target that moves stealthily and quickly. They are called IOCs (Indicators of Commitment). If you find an IOC in your systems (usually by triggering one or more rules) it is time to examine to rule out the false positive, the intrusion or its attempt. Without them, the only thing left to do would be to strengthen all the commitment to behavioral detection and new technologies that are still heuristics, even if they come with the artificial intelligence seal. We need IOCs, at least in the short and medium term. An IOC in isolation does not tell us much, but if we pull the thread and perform a correlation exercise and “paint” the directed graph, we will get a complete picture. We go from having isolated data to information. With the latter we will be able to draw our plans better and if we base them on previous “successes” (intelligence) we will know what and how to face in the best possible way. We know several of them, the classics: IP, domain, hashes, and even the mutexes used by malware to, among other things, avoid duplicating its execution-infection on a system. However, other new ones are relatively new, such as JA3 and JA3S or JARM. JA3, a signature to identify the fingerprint of encrypted channels Just as the statement reads. The idea and implementation (which originated with three Salesforce engineers (and for the sake of curiosity, the acronyms of the names of the three are J.A.) is based on the hash of a string that is the concatenation of several fields from the handshake or negotiation between client and server to establish an encrypted connection. Let's take a breath to explain more concretely what we have just read. An encrypted connection is negotiated between the two parties to agree on what encryption and configuration they will use. If all goes well, they will exchange keys (Diffie-Hellman, for example) and the payload of that conversation will be encrypted. So, before the encryption starts, there are a series of handshakes in which both parties inform about the possible encryption and configuration they support. The idea is that as many clients and servers have a presentation of this “fixed” configuration, we can extract the fields that do not vary and produce a hash with them. And that's it, if this hash appears we know which customer or server (JA3 or JA3S) we are dealing with. An image of the fields used for the calculation: Source. From this information, a string is extracted with the values that can be inspected and a text string is formed, which is then “hashed”. In fact, the hash function used is the classic MD5 (yes, it is considered insecure, but for the JA3 mission it has no negative impact implications). In JA3S (the server identification part) some fields vary but the result is identical. A practical use of JA3 Ok, but how do we use this? It is easy and simple. Imagine we have a rule that detects a certain trojan with a hash, for example, a TrickBot. But now, the creators of that malware have released a new version that slightly changes the configuration. This small change means that the rule is no longer valid for detecting new versions. However, since the creators continue to use the same cryptographic libraries, even if the hash has changed, the TLS negotiation will remain the same and we will continue to identify the Trojan unless they make a change in the libraries, they use to encrypt communications. Let's look at a JA3 determined on the "TrickBot" family (8916410db85077a5460817142dcbc8de): Source: Own capture. Consulting this hash in the ABUSE database we see that it identifies about 60,000 samples of that family. And now the question that many of you may be asking: Doesn't that hash collide with software that uses that same library in that particular way? Yes, it is possible. So JA3 is not a silver bullet but a good starting point for associating with families (if it crosses with SSDEEP, for example) and gives more weight to a detection by other factors. In other words, it is a “handle with care” but gives us a very good edge in the relationship graph. JA3 problems grow Despite its relatively short rollout (JA3 left the nest in 2017), from its implementation to today it has been found to have certain major limitations. In a post by CloudFlare's engineering team they point them out and list them: Browsers have started to output in random order the TLS extensions of connections. This in itself shatters the accumulated signatures when it comes to identifying more current browsers (clients). There are notable differences in the ecosystem of tools and utilities that compute JA3 hashes. This means that there are several hashes for the same TLS stack. That is, we can have a detection rule for a certain JA3 hash while the detection component calculates a completely different hash for the same object. Finally, they criticize (rightly) that JA3 focuses only on certain fields of the TLS negotiation, the “ClientHello”; it does not take advantage of the other fields that can help us to better “nail” the identification. JA3 is still a very good idea, despite these limitations due to use and experience. Thus, in 2023 a new iteration or twist was presented to solve some of the shortcomings already mentioned. JA4, the next generation JA4 was presented by FoxIO engineers in September 2023. The first thing that stands out is that there are no longer only two types of signature: JA3 and JA3S, but a complete suite of signatures specific to each layer using encryption and specific applications or protocols: Source. As we can see, JA4 is a complete family of hashes oriented to extract typical and differentiated information from each protocol. Moreover, if we read a JA4 hash, we can make a reading of it, since it carries information that is to some extent interpretable to the naked eye by a human. Let us look at the case of a JA4, which would be the equivalent of a JA3: Source. If we take a look, we see the first fragment of the hash which is interpretable on a read. Next, we see two fragments that are parts of the SHA256 hash of the supported cipher sets and the advertised extensions... but instead of calculating them in the order they appear, they do it by sorting them before applying the hash (which allows to avoid the randomness we were talking about before). We have already solved most of the shortcomings of JA3 with this ordering of extensions and ciphers and the “upgrade” from MD5 to SHA256 to avoid collisions. It is also a kind of fuzzy hash because, even if only one extension or cipher suite changes, we can continue to group the fragments that have not changed. Now we will look at a completely different animal. Instead of focusing on TLS connections we will focus on the HTTP protocol: Source. JA4H is specialized in extracting signatures from an HTTPS client connection. As we can see, it is organized in four fragments (a, b, c, d). The first of them has readable information, which we can understand in one reading. The next three fragments are parts of three SHA256 hashes. Each of these three hashes are calculated from the order of appearance of the HTTP headers, the hash of the cookies but sorted alphabetically and the third one is the same as the second one but with the values of the cookies. The complete hash is not unique in its parts. For example, in two connections we can obtain the same first three fragments and only the fourth fragment, which carries the cookie values that can potentially change in each client connection, varies. We therefore have a hash that leaves the context of the TLS ClientHello fields and is applied to that of the HTTP protocol. Conclusions As we said at the beginning, information is everything, but if we can create intelligence with it, then it is everything and more. And since the ingredients of information are data, the better the data, the better the dish will be cooked. A path has been trodden with JA3 that gives good results, but with the JA4 family of hashes we now have a well paved road. It is only a matter of time before it is adopted and we see more JA4 hashes databases on which we can dive to enrich our information. Image: 8photo / Freepik.

As we discussed before, malware creators have a primary objective when developing a new piece of software: to avoid detection—or at least delay it for as long as possible. To achieve this, they try to remain unnoticed, but in a large-scale infection attempt targeting hundreds of thousands of devices, this is practically impossible. Eventually, the malware sample will end up on a malware analyst's desk, where all its secrets will be laid bare. Given that the released specimen will have a short lifespan, efforts are made to equip it with mechanisms that complicate the task of extracting its functionalities and communications. In other words, to make the analysts sweat before emerging victorious with the necessary data to create blocking and detection rules. ____ In previous entries we saw that they used new programming languages to make this task harder. The reason, which we reiterate here, is simple: there is not much information and few tools for these binaries, and until these gaps are filled, they exploit this window of opportunity. Specifically, we know of Go and Rust (among others). These languages also enable faster development times due to their abstraction mechanisms and conveniences, which are absent in other native languages (such as C and C++). Today, we're bringing a "dark horse" to the forefront. We’ve chosen this term because calling it "unknown" would be unfair. We’re referring to Dart, an object-oriented programming language designed by Google for the web. However, Dart isn't as popular as Flutter, the platform where Dart plays a starring role. Flutter is widely used as a cross-platform application development framework, particularly in the mobile world, where developers can use Flutter to deploy the same version on both Android and iOS, drastically reducing development times. Flutter is widely used as a cross-platform application development framework, particularly in the mobile world. In any case, malware creators do not primarily use Dart because of Flutter’s portability (although they might, such as in the case of the MoneyMonger malware) but rather due to a feature of its compiler, which we’ll explore further below. Fluhorse: an example of malware written in Flutter to hinder analysis In May 2023, a new Android malware family named Fluhorse emerged. Although malware using Flutter had already been detected and studied, the novelty of Fluhorse was that its malicious routines were written specifically for this framework. While Flutter's use was previously considered mainly for its cross-platform portability, this time it was leveraged for its anti-analysis capabilities, using concepts we will discuss shortly. It’s worth reading the static analysis report from Fortinet, which details the "struggle" to reverse-engineer the Flutter binary and how a slip-up by the malware creators, by including the x86-64 binary, facilitated its analysis. The post delves into the intricate layers of protection surrounding the malware, from the packaging of the resulting APK, which makes the package content structure challenging to interpret "humanly": Source: Fortinet. Additionally, the payload part (the set of actions in the code meant to be hidden from analysts) is encrypted (by the same packer) within the classic "classes.dex." Once decrypted, it unfolds another "classes.dex" containing malicious functionality. This final object is the Flutter "snapshot," and here they erect another (almost) insurmountable barrier. This is where we’ll explore why Flutter is used as an anti-analysis mechanism. Flutter as an anti-analytical measure Firstly, Flutter's creators did not hinder analysis, but rather produced a final product optimized for size and performance. This is especially important for mobile development frameworks, where every byte and cycle taxes battery life. That said, like other frameworks that allow reverse-engineering obfuscation to protect intellectual property, Flutter has optionsto obfuscate the code. This essentially means changing the names of variables, methods, and classes to random strings (see the image above). However, this is not the feature malware creators are most interested in. Dart, the language used to program Flutter, allows the application to be compiled into different formats, providing flexibility and adaptability to meet developers' requirements for the context in which the application will be deployed. Among the various compilation output formats, there are two main groups: Dart for the web and Native Dart. The latter has two modes: JIT and AOT: JIT (Just In Time execution) mode compiles the code on-the-fly during program execution. In the AOT (Ahead Of Time compilation) mode the code is precompiled by the application creator and injected into the Dart virtual machine. Source: Dart. In summary, AOT saves the hassle of compiling for the platform but, in exchange, requires pre-compiling the application for the different architectures on which it will run. Remember, mobile devices have several types of architectures, which means a Dart AOT binary carries extra weight because it must contain all the code for each architecture. Recall what happened to the Fortinet lab: "they stumbled upon the x86-64 version code..." AOT Snapshots Here lies the crux of this discussion. This format is a serialization of the Dart virtual machine state and, as such, has its own highly optimized structure, which changes relatively quickly from one version to another. Reverse engineering documentation is still scarce. Additionally, when an application is compiled, it statically loads all the necessary libraries (there is no dynamic linking), so the space to study within this snapshot is vast. Moreover, remember that this is not code to be executed directly; instead, the Dart virtual machine will handle execution (recall, it is cross-platform). Therefore, even when disassembled, the code lacks immediate meaning. Lastly, reverse engineering relies on applying existing knowledge about the object to analyze (breaking down the whole into parts). This process inevitably depends on a range of well-tested and optimized tools developed by the community or specialized companies. But, what happens if these tools do not exist or are still in a very early stage? This is precisely the current state of reverse Dart AOT snapshots: optimized and ever-changing formats, poor documentation, and a lack of tools—a perfect storm for an analysis lab and an opportunity window for malware, as we have seen. What can we do to combat malware using Flutter (Dart)? There are some options, of course. In this field, curiosity is never in short supply, and fortunately, some share their findings. For example, the "reFlutter" project is a patched version of the original framework to help trace the application, which we can use to work with it in Frida or examine its traffic with an HTTP intercepting proxy. Additionally, there are more publications addressing Dart AOT snapshot handling for reverse engineering. An excellent example is the recently published article "Reversing Dart AOT Snapshot" in the renowned ezine Phrack. Conclusion Clearly, this is yet another new challenge for malware analysts. It's an area previously unknown, now exploited by malware creators. However, the community is quick to respond, and the continuous flow of published findings keeps a steady stream of new tools and knowledge to shrink the space available to attackers. Cyber Security Name the malware you have, and I'll tell you which botnet you belong to September 14, 2022 Image: Freepik.

One constant thing in cyber security is that everything is changing... constantly. We refer, of course, to the evolution of techniques used by APT groups to evade defenses, avoid detection and make attribution more difficult. Let's stay on this last point. Attribution means assigning techniques, tactics and procedures (ATT&CK) to a particular group of attackers who will almost always fall into a geopolitical zone with interests that may vary from sabotage to espionage or purely economic. In some cases, it is even desired to pervert this attribution to divert it and blame another APT group (and by association, in many cases, another nation) for the operation. This is what has been called, since time immemorial, a false flag operation. For example, making believe that a sabotage has been the work of another APT group. (Nothing sophisticated, these similar situations have occurred from the family environment to the school). ℹ️ Advanced Persistent Threats (APT) are meticulously planned cyberattacks that remain covert within a system for extended periods, aiming to spy or steal information without being detected. This false flag can range from something as simple as using coding in the enemy's language or using proxies in the enemy's home country, to using expressions typical of their language and details to the clinical eye that leave false clues. Some of them even do it in such a way that they express themselves in other languages but using typical mistakes of the speakers of the country that is the object of the false attribution. And of course, with generative AI these techniques have been improved (anyone remember phishings with misspellings?). However, in many cases the attribution is not desired, nor is it intended to place the blame on a third party in a self-serving way. In fact, on the contrary, the aim is to mask the attack for many different reasons: anonymity and concealment of the infrastructure that is so difficult to set up and which, once discovered, is often the subject of demolition and analysis for later cases. Now, for the purpose of this article, let's look at one type of network infrastructure that can take many forms and layers to serve as a quagmire against the best efforts of the incident response infantry. Their mission is to succeed in operating by making infrastructure attribution and discovery difficult. The ninja concept but APT style. The different approaches to “fronting” on the web A few years ago (well, quite a few years now) malware writers started to employ techniques to make their assets more difficult to discover on the network. If they wanted to deploy, for example, a malware download point or a phishing attack, they would use the botnet itself (a network of infected computers under the control of cybercrime). A DNS record with a very low refresh time (TTL) was added to advertise the IP addresses where this point was located. In this way, the IP address of a domain would rotate relatively quickly, throwing analysts off the scent and making it difficult to target their efforts at a single point. The technique is called Fast-Flux. Another technique consists of creating a network of proxies chained together (proxy-chaining) to add a network of pivoting nodes to add layers of complexity to the attribution. This form has been captured by cinema and surely evokes everyone's personal imagination: the image of the world map and traffic packets traveling from city to city across the globe while someone is pounding the keyboard as if playing Prokofiev's Toccata Op.11. Then, with the advent of Bullet-Proof servers, and the brazenness institutionally supported, APT groups were (and are) operating in the open through servers hosted in their own or allied territory, so it was easy and straightforward to attribute the operation. There is not much of a layer to peel back in these cases. Despite administrative channels, requests to shut down such infrastructures fell on deaf ears or were delayed without explanation. Let us add to our small and involuntary anthology the Tor-type networks, which, although they are useful in places where freedom of expression is questioned, are also used for morally and ethically questionable purposes. However, these types of networks are, in many cases, banned in EDRs and other devices, so their use is minor, and we could even say naïve. Operational Relay Box And we reach the consequence. If a group wants to make it difficult to attribute and share information about its activities, it must make its exposure a big problem for the analyst. A botnet? No good to us, too much noise and they can throw it away and even analyze it to detect who's behind it. Tor? Come on, man, let's be serious. Do we use bulletproof servers? Not if we want to go undercover. Proxy chaining? Arianna's thread is never stiff enough or probably long enough. What if a third party rents us an amalgamation of everything discussed above? What if we copy the Tor-like networking infrastructure, but without Tor? Well, that's more or less a type of network infrastructure that has been dubbed ORB and that conjugates elements of VPS servers acting as a proxy and an assortment of compromised devices, viz: IoT, routers, etc. Having all that gear in constant motion makes it difficult to keep track of a thread because the thread is continually changing shape, size and color. And sometimes even that thread frays into several strands like a hydra on the rampage. An ordeal for the researcher. In fact, there are voices proclaiming the death of IOC with this type of infrastructure. Like Michael Raggi, from the Mandiant team (now part of Google Cloud) who has researched, written, and presented on this issue. We recommend reading his article. Why 'the death of the IOC'? The IOC aims to be a fixed point. If we have a malware that uses a set of domains and their corresponding IP addresses, we will have a picture on which to rely to deploy prophylactic measures or diagnose an attack. That is, if your defenses detect outbound traffic trying to contact a malicious domain, then we would have an infection pointing to a certain malware (and pulling on the thread, who knows, a specific APT). So, what happens when IP addresses dance in a matter of hours or a few days? What fixed point do we grab to feed our threat feed? What Snort rule do we add to our already crowded jar of healing essences? It's simple to understand and it's going to be a challenge to counter. The cost of detection is going to be higher without fixed points of reference, the cost of attribution is going to be higher, and so the landscape ahead of us (well, we've had this here for a while now) is challenging in many ways. For those of us who enjoy looking for solutions to new problems it is going to be a tempting challenge, for those who are suffering right now and have nowhere to hold on it is a real pain. But hey, it's the same old story, in this race the mouse will always have the advantage. Cyber Security Malware distributed during fraudulent job interviews May 8, 2024 Photo: Taylor Vick / Unsplash.

We owe Tony Hoare or Sir C.A.R. Hoare (1934) a great deal for his discoveries in the field of computing. His name may not ring a bell, but if you have studied computer science you will almost certainly have come across one or two (or more) of his discoveries, for example, the QuickSort algorithm. With that business card, perhaps I should add that he also invented a formal language for concurrent processes to live in harmony: CSP or Concurrent Sequential Processes, which today is used in the Go (or Golang) programming language as the basis for its multithreaded implementation. Suffice it to say that it is one of the "Turings". The prize was awarded to him in 1980 for his contributions to the design of programming languages. And it is precisely here, at this stop, that we get off and continue on foot. ALGOL's cursed inheritance In 1965, Hoare was working on the successor to ALGOL 60, a programming language that at the time was a modest commercial success and was mainly used in research. This language, called ALGOL W in a display of originality, was a joint design with another of the greats, Niklaus Wirth, and was in turn based on a version of ALGOL named (surprise) ALGOL X. ALGOL W failed to win the hearts of the ALGOL 60 programmers as its successor. In the end, they opted for the other pretender to the throne: ALGOL 68 (yes, naming languages with catchy names was not mainstream until decades later). But what ALGOL W did achieve was to lay the foundations for something bigger. Much bigger: Niklaus Wirth adopted it to create the basis for one of the most widely used programming languages of all time and the one for which Wirth is perhaps best known: Pascal. However, Pascal and its lineage are the subject of another article. Let's keep the focus on ALGOL W. He possessed something evil inside him. A diabulus in music that inadvertently but consciously crept into its design and of which, to this day, many programming languages carry like a cursed inheritance... An example of a null reference Let's give an example to help us get the hang of it. Moreover, we will do it in Java, since it will surely sound familiar to those of you who have programmed a little in this language: The example is almost self-explanatory: In line 3 we declare a variable with the String type but we initialise it to 'null'. That is, the variable 'string' does not have an object of the String class, in fact, it does not have anything. null' is indicative that the variable is waiting to reference a String object. The small catastrophe is about to happen on line 6, when we are going to use that variable. At that moment, when we indicate that we are going to use the "length" method that objects of the String class possess, the Java runtime invokes the reference that should possess 'string', but in doing so there is nothing, zero, null. What does this cause? Well, first of all, the program stops its execution if we do not control the exception. The problem is not that it stops in this trivial but illustrative example: can you imagine if it stops in a program that is responsible for calculating the approach to the runway in the middle of a commercial flight? ⚠️ This error, when it manifests itself (and it is not difficult for it to do so) has led to tremendous repair costs. In fact, Tony Hoare calls it "the trillion dollar bug" and his conference with the same title is famous, where he exposes real cases in which in one way or another his "invention" has been involved. If we think of a program as a big state machine, we will realise that, at a given instant of time, a variable does not always contain an object, but the variable is there. In addition, that variable can be used as a parameter to a function or method, be inserted into a collection: a vector, list, etc. If we delete the object pointed to by that variable or its memory is collected, the variable will contain a null reference ready for disaster when it is invoked. We have already seen in previous posts how in languages like C or C++, where we handle memory manually, we can run into variables that no longer contain the object they point to in memory. How do we avoid null references? Let's continue on the basis of Java, for illustrative purposes. We assume that the rest of the programming languages where there is the possibility of an object having a null reference have or are developing similar defensive mechanisms. First of all, the curious thing is that if we have a String type, why does the compiler allow us to initialise it with 'null'? Does 'null' possess a generic type? Not really, it doesn't. In the particular case of Java, it is a generic type. In the particular case of Java, it is a language keyword without any type. In fact, strictly speaking, since it has no type, the mere fact of assigning 'null' should be an exception since it does not match the type to which the variable is designated. That is, the very fact of doing "String string = null;" should cause a compilation error and at runtime, if a variable has a 'null' value, it should at least cause a warning even before the variable is used. And this is precisely a strategy to define a class that "root" does not admit to be null: Optional. Such a class does not free us from the evils of null, but it does give us a very powerful mechanism to manage its risks. That is to say, instead of hiding the use of null, what is done is just the opposite: it is openly manifested. An example to see it better: In this case we have a Person class, if we want to get the age, we have a getter method for it. So far so normal, but the funny thing is that as we don't know if that class will have the field 'age' with a value or not or if it is null, what it returns is not an integer but an object of the Optional class. This object is not the value itself but a wrapper on which to work safely. In this case, let's take an example with the Person class in place: As we can see, we do not have the value directly (notice line 6), but we are somehow forced to treat it as what it is, something optional. In this case, we simply ask (line 16) if it has some well-defined value and use it, or else we handle its absence appropriately (no exceptions in between). Are there languages without null types? Almost all languages in which there is the possibility that a reference (or pointer, etc.) is null, it is a headache to eliminate them through good programming practices, defensive programming or simple error correction. The possibility of an object being invalid (null) is present in practically all languages, but what does change is the way the language sees it and, above all, the tools it provides to programmers. Rust, Haskell, Kotlin and Swift, among many others, have native or integrated capabilities from day one in their development and execution environment. The basis is: if a type can be null, you must handle it accordingly. Almost all of them work in a similar way to Optional (present in Java since version 8). An example in Rust (the code repetition is for teaching purposes): The treatment of dividing by zero is similar to not having a proper reference or the type being null. As we know we cannot divide by zero and so the function is designed to be prepared in case the divisor is zero (lines 1 to 7). The function does not return the result of the division but an Option on which we must do a pattern capture (lines 12 to 15, for example) to determine if the operation has been correct or not. You may ask yourself... But is it possible to do the operation WRONG and not return an Option, but directly an f64? Of course, you can: But it is precisely how not to program a function. If you design and implement it to be fail-safe, you don't deliver back a value f64 that might be null or invalid, but you force the user of the function to worry that the value is present and to deal appropriately with the case that it is not. Conclusions A software bug can cost us a lot of money. Both in repairing it and in mitigating the damage it has caused; not to mention cases that have made headlines in the general media for the disaster they have caused. One of the most common bugs and present since the infancy of programming languages are null references. Invented precisely by one of the most laureate scientists in the field. Modern languages have mechanisms that make it easier for us to deal with these errors and more difficult for us to try to commit them. Finally, a simile from real life: Imagine a letter that has no sender. Where does the letter return to if the address is wrong? If there is a sender, it has a mechanism in case the value or destination address is not valid: return it to the sender. As we see, we often deal with old problems and we don't see that the solution was already there centuries ago... Cyber Security The Hacktivist, an online documentary about Cyber Security pioneer Andrew Huang January 2, 2024 Image: Freepik.

One of the principles of cybercriminals (and on the side of the good guys, the pentesters or read team) is that both the exploits and tools used are detected by defense systems, both networked and local. Although they are not infallible systems, they can ruin the job if they catch suspicious activity and raise an alarm. As a pentester, it hurts your pride to get an email or a call saying you've been caught in the act; but it's part of the game. At the end of the life cycle of malware, exploits and other malicious elements there is always the "police" token in the form of a signature, rule or pattern that serves to detect it and raise alarms. Thus, malware in general has an expiration date, sooner or later it will be hunted down, and each use is a ticket to end up at the castle gates or in the crocodile pit. Malware in general has an expiration date: each use is a ticket to end up in the crocodile pit. Given this situation, someone thought that why bother writing a complex malware with rootkit functions and end up being discovered a few days later. After all, what is such malware going to do, search for information on the computer and exfiltrate it? steal domain administrator account tokens? And ended up answering: why not use the tools that are already in the system? On the one hand, they are all whitelisted and on the other hand you save a good handful of hours developing an original piece that is going to end up x-rayed on a yara ruler. What are LOLBins These types of techniques were given a name: "Living off the land" (LOLBins). It is the military equivalent of being released alone in the middle of a mountain to make a living and get to a specific point dozens of kilometers away. A frightening but enriching test, which teaches you to achieve a goal with few or no resources and no tools other than your own hands. LOLBins techniques are a goldmine for cybercriminals, but it is also a fertile area for pentesters and researchers. Something so obvious and simple a priori, has become a lode for cybercriminals, but also a fertile area for pentesters and researchers who use this type of techniques and tools ethically but with the opposite intention. In light of this reality, lists of this type of tools have been appearing for some time now to classify them in terms of use, location in systems, techniques used, etc. Authentic catalogs to consult and use them depending on the context in which we find ourselves. Let's take a look at a handful of them, many of them spin-offs of the original project and that choose to specialize in a particular operating system or a specific area. LOLBAS (Living off the land binaries) URL: https://lolbas-project.github.io It is arguably the original project. It accumulates about 200 entries between binaries, scripts, and libraries. It also has a unique virtue and that is that it maps each entry with its corresponding entry in the ATT&CK framework; widely used in advanced threat modeling. One of the popular entries is Rundll32.exe, used to execute DLL functions (libraries), which makes them practically executable; one of the most used and omnipresent (and all-powerful) in Microsoft Windows operating systems along with the king of LOLBins: Powershell. For each entry we have additional information about its use and in which APT operations it has been used; together with ATT&CK we have a very instructive overview. All of this started with Microsoft Windows systems, but when it was seen to be productive, the same scheme was transferred to other systems. For example, the LOOBins (Living off the orchard) project, devoted to the Apple system: MacOS. It is basically the same idea as the original but transferred to the operating system of the Cupertino company. URL: https://www.loobins.io/binaries/ We even found a classification by the tactic to be employed (once again, ATT&CK). GTFOBins Same scheme but focused on UNIX systems with Linux in particular. In a way, as MacOS is a UNIX derivative, it would also be useful for that system, in addition to the one we have seen above. URL: https://gtfobins.github.io It is surprising that with almost 400 detailed tools (UNIX systems are rich in this aspect), it is possible to take advantage of such unlikely tools for this aspect as "bc", the classic UNIX command line calculator. These are the main and most important ones, but it doesn't end there. The idea of listing items that pass under the radar of EDR (Endpoint Detection and Response) systems has flourished to such an extent that we no longer only have sites dedicated to binaries, but to domains that are usually whitelisted and known not to be blocked. They also often allow the uploading of files or text that is then used as a download point or, on the contrary, to exfiltrate information. An example is https://lots-project.com (Living off Trusted Sites): A clear example is docs.google.com or github, services widely used for phishing schemes, malware, etc. Another with file extensions (!), https://filesec.io o drivers en https://www.loldrivers.io We left out a few, such as APIs: https://malapi.io or cheat sheets of code snippets that can be used in Windows pentesting and Active Directory in particular: https://wadcoms.github.io. Surprisingly, there is an arsenal available for an unauthorized user wandering around in the systems themselves. As we can see, it doesn't take complex, latest tools to achieve results. We have a vast repertoire of techniques at our disposal with the ingenuity and knowledge that these types of compendiums possess. It is the maxim of doing a lot with practically nothing: Something similar to what MacGyver would do with his legendary little Swiss Army Knife, but in a digital version. Cyber Security Dangerous friendships (or how a disguised collaboration on Github can ruin your day) October 12, 2023

◾In the previous posts we talked about virtual environments, rights and wrongs, and about virtual worlds in Python. Now let's talk about the "third dragon": dependencies. The third dragon When we talked about installing dependencies and projects stepping on each other's toes, we solved it with virtual environments. That erases at a stroke any altercation between neighbors. However, what happens if within the dependencies, two packages have overlapping and incompatible dependencies between them? Well, that's our third problem. As many of you know, Python has a central repository from which the projects are fed to install the dependencies. This is typically handled with the command line tool 'pip'. The problem with pip is that it does not watch for one package to step on the version-level dependencies of another. Let's illustrate this. If I have a library named tech.py that requires version 2.0 of the Pydantic package or library and I have a dependency that uses Pydantic but in its version 1.2, I still have the same problem that I had before with projects that use the same interpreter. The difference is that I have the problem between packages of the same project and before I had it between projects. Additionally, pip will not handle that dependency resolution problem without help from the project administrator. In fact, if we persist in our installation and force the inclusion of packages that are not compatible, it will fail at some point in its execution, since the functionality expected by some of the packages will not be found. Welcome to the world of package and dependency managers for Python Dependencies are not the only thing we will be concerned about when we are developing, testing, or deploying the application. Throughout the life of the application, we will want to keep the packages free of vulnerabilities and that means applying a strict security policy that leads us to check that they are free of bugs, when new updates are released and update our dependencies if they are released. Be careful, because solving the dependency conflict network is a rather big problem that could throw us into the abyss of computational complexity since it can become a problem within the NP-Hard category. We have already given examples of dependency conflicts, but within a project this problem can be exacerbated because some packages will depend on a particular version while others exclude the use of that particular version. The conflict will be assured and often the solution is to adjust the project to tip the balance on one side or the other, with no room for Solomonic decisions. Well, having described the problem and discarded pip for the dependency management of a moderately serious project (for small projects, pip can be enough) we need a tool that monitors conflicts, updates packages without compatibility problems (that is, for example, using semver and respecting the version specifications) and even builds and publishes our own Python packages. The problem is not that such tools do not exist, but precisely the opposite. There is such an abundance of dependency managers that it is hard to commit to just one. Similar to when you have free time to watch a series and you spend it browsing through the whole catalog without deciding to "hit play". Just kidding, let's not exaggerate. There are several managers, and it is difficult to decide on one because they are all competent in their work and the features practically overlap. In the end, the decision comes down more to use and comfort as we try them out than to a rigorous examination and technical test. A practical graphic example Here are three of them, with a lot in common but with different tastes and ways of doing things: Poetry, PDM y Rye. Since they have very common elements, we will see examples in Poetry that are easily adaptable to the other managers. These types of tools are more than just a package and dependency manager. Poetry, for example, allows us to build the project, package it and publish it in PyPI (the Python repository par excellence). For the resolution of dependencies it relies heavily on the metadata of the packages published and accessible in PyPI following the versioning scheme standardized in pip-440 and not the semver that we have already discussed above. One interesting thing is that when we start a new project it creates a directory and file structure typical in a Python project, saving us some input work. Here is an example: The "new" command, at a stroke, has given us the directory for the sources (which optionally can also be src) and one to encourage us to write the always necessary but procrastination victims, tests. Let's install a dependency, for example the requests package: As we can see, it installs the package, its dependencies and the best thing is that instead of using the usual requirements.txt (we can export to that format) it declares it in a pyproject.toml file that is already defined in the Python standards. Let's look at the content of this file: If we observe well, the dependencies are not detailed. This is on purpose. What we have is a "clear" view of what the project needs, but we abstract from the dependencies of each package. Those third-party dependencies are defined in a "lock" file that we have already seen but we replicate to see how it looks like after the installation of a required library: What we see is the end of the file, as it is dense and difficult to process in a cursory reading. It is more of a record for the package manager to get a snapshot of "working" versions. Then, when we want to replicate the execution environment in a deployment, there won't be (or shouldn't be) unnecessary conflicts because it is a practically cloned environment. PDM and Rye are similar in terms of operation, although it goes without saying that Rye opts for a different path and even has the functionality of pyenv, that is to say, it has its own "Pythons" manager and will install the desired version (if available) and attach it to the project. The case of Rye is curious because it is designed and implemented by the well-known Armin Ronacher "Mitsuhiko". This developer is the leader of projects such as Flask, Click and other widely used libraries. Armin has merged the needs of Python package management but from the perspective of Rust, since for some years now, Ronacher has been a fan of this language which we have already talked about on more than one occasion from here. Bonus track When a dependency manager "locks" the packages required by a project, their hashes are kept in a file (usually with the extension .lock) where we can see the list of packages, their dependencies and all the hashes according to the installed version. This allows us to compare the hash of the package being installed and if it does not match the hash noted in the file the manager will notify this disagreement. The protection is obvious: The package may have been altered, either accidentally or as part of an attack on the supply chain. Mind you, these are not cryptographic signatures. Still, something is in place to ensure that the package that arrives does not come with a loose loop. Conclusion Having a portable execution environment (in the sense of moving it from machine to machine) in Python should not be a penance to Cvstodia. Of course, it is necessary to stumble over all the stones on the road to gain experience. It is ironic for a language with the motto "There should be one-- and preferably only one--obvious way to do it." that there are so many options to get to the same point, so accidentally. There is no orthodox way, and we are not trying to be prescriptive here, but it is practical and, as they say, "battle tested". We have used it in several of the projects we lead from the Innovation and Lab area and so far no one has broken the build ;) —CONTINUING THIS SERIES Telefónica Tech The ironic state of dependency management in Python (I) November 27, 2023 Telefónica Tech The ironic state of dependency management in Python (II) December 11, 2023

◾In the previous post of this series we talked about environments and interpreters and how to adjust them to every need. Now let's talk about virtual worlds. Python virtual worlds Let's assume that we already have the right interpreter running. Now it's time to install the project dependencies. One of them is the tech.py library version 2.0. Since it is a system that hosts several projects, it so happens that Python already has that package installed because another project makes use of that library. But there is a problem. The library that is installed is version 1.2 of tech.py and our project uses many functions and classes defined in version 2.0. Even if we force the installation, when we run the project, it will give error. What do we do? Because we only have one Python interpreter with that version. The Python project itself tried to provide a solution to this problem in pep-582. It aimed to share interpreter but at the same time, make the libraries of each project independent of each other by putting them in a folder: __pypackages__ local to the project. In this way, there would be no collision and the same interpreter would use some libraries or others when starting from a different project. A good idea... that ended up being rejected by the committee itself. How can we have multiple projects and separate their dependencies so that they don't get in each other's way? The solution is to have a copy of the interpreter, its associated libraries and binaries and change the paths pointing to the python binary. In other words, duplicate the environment and isolate it from the outside by making the shell "believe" that Python is installed in a different and alternate path to the system installation (and even to that of pyenv). It is what is called, with no further mystery: virtual environment. And that is exactly what a virtual environment management tool or library does: it copies or binds the same executable and provides an environment of associated libraries, as long as the appropriate environment variables are manipulated to activate it. So now, when we install Python packages or project dependencies, they will be isolated to that virtual environment, without affecting any other environment at all. This solves the problem of having one interpreter serving multiple projects. How do we use it? We have several options, the most direct one is using the venv module included in all Python distributions from version 3.3 onwards (by the way, its respective pep is here). As easy as this: That creates an environment called 'mientornovirtual' in the same directory of the project. However, we must activate it before working with it in a similar way to how we activate the Python versions with pyenv: From there, everything we do will be on the environment that is in that folder. When we finish working, we only have to deactivate it to return to the original interpreter: What are the alternatives to the management of virtual environments? We have the project that originated the creation of virtual environments in Python, virtualenv. In fact, it is the tool to be used instead of the module (which we would use only if we do not have virtualenv). The usage is exactly the same, but we would invoke virtualenv instead of the module. The bonus if you are using pyenv In case you are using pyenv to install and activate interpreters, there is a plugin or pyenv that allows us to manage virtual environments. Everything is integrated in a single tool with this plugin: installation and selection of the interpreter and also the creation and management of virtual environments. Another advantage is that it does not create a folder in the project, but in a directory as a cache of virtual environments so that in projects that we consider common (that do not have problems of dependencies) we can reuse the environments already created only with its name. In addition, if we indicate the name of the virtual environment in the .python-version file, when we enter the project directory, it will be activated automagically. Top. Are we sure that we already have everything that technology gives us to have a reproducible Python environment? Well, no. There is still a "small" detail that we will see in the next post. —CONTINUING THIS SERIES Telefónica Tech The ironic state of dependency management in Python (I) November 27, 2023 Telefónica Tech The ironic state of dependency management in Python (III) December 26, 2023

There is no doubt that if there is one programming language that has gained respect and admiration in the last decade, (among others), it would be Python Artificial intelligence? Python. Web applications? Python. Learning to program? Python. Learning Python? Python. Few tasks escape this dynamic language that began displacing the king of scripting in the 1990s, Perl. Created by Dutchman Guido Van Rossum in the late 80s, it saw its first version in 91 and since then has been gaining positions in the various annual rankings in which programming languages are positioned in various criteria. It is worth asking if there is something that this language does not do well and the answer would be: Of course. One of them, quite repeated in the community, is the management system of packages, dependencies, and versions of the interpreter when you get into work with a project. Even the great creator of the XKCD comic, Randall Munroe, dedicated one of his vignettes to this problem: "It worked on my computer" Engineers like reproducible systems with no hindrances. Which means you design, develop, test, and when everything is ready, theory says that you should deploy and run it without problems. Practice, however, teaches us that the result is quite the opposite. Therefore, one of the purposes of containers or virtualization is to try to blur the line between the two universes in constant discrepancy: the development environment and the production environment. ◾If we remember, when Java was announced back in '96 one of the slogans was: Write Once, Run Everywhere. And it was all thanks to the Java Virtual Machine, the JVM, which relieved programmers of two burdens: 1. manual memory management thanks to a garbage collector and 2. the abstraction layer over the operating system that hosted the environment. While not perfect, the JVM did a very good job in that regard. Regarding the Python ecosystem, it is cross-platform in the sense that the Python interpreter is available for multiple systems, and you also abstract (with identical limitations to Java) from the underlying platform. So far so good, except that on an operating system the version of Python installed may not be the same as the one on which the application was developed. What's more, a few years ago the Python world split in two, like a schism, with the advent of version 3 of the language. On one side were those who were reluctant to change from the then popular Python 2.7. On the other side were the group of adventurers who were entering the then wasteland that was Python 3 (also known as Python 3000); we say wasteland because the ecosystem of libraries was still extremely incipient in terms of versions for Python 3. As a result, we are faced with the problem that the project we are developing is tied to a specific version and any other target system will not support it. How do we solve it? We obviously have to install a new interpreter with the required version and make it coexist with the original one. There are projects that allow us to do exactly that, install as many Python interpreters as we want with a single and discreet drawback (that you can even automate): You must take care of selecting the right version before launching the project or any feature you need for testing or continuous integration. Pyenv, an open-source and available for UNIX systems (there is a fork for Windows) is a project that installs a command line utility to manage Python's. It is simple to use once you understand it. It is simple to use once you understand the concepts with which it works. First of all, it installs the interpreter of your choice. The list is endless. You can install from deprecated versions (mostly used for regression testing on still supported libraries), alternative interpreters (Pypy, Jython...) and popular distributions, such as Anaconda. Once we have the interpreter (or the ones we want) installed, it will not be active yet. Here we have a variety of options to make it flexible and adapt it to our needs. First, we can choose to activate an interpreter globally, affecting the whole system. Any new shell we create will have that interpreter as its main interpreter and any process that calls Python will also have that interpreter as its target. Say, for example, that I want the operating system in general to recognize Python 3.9.17 as the interpreter to use. The command is simple: pyenv global 3.9.17 With that command all "python" will be 3.9.17. But, of course, affecting the whole system in general is something that can produce undesirable side effects. We want to reduce it to the level of the folder we are working on. That is to say, that all the shell sessions that we open in that folder have that interpreter as reference. It is simple, instead of global, we will use local. This will create a file called .python-version which only contains the name with which pyenv will alter the paths of the environment so that when we type python and we are in that folder, the interpreter will be the appropriate one. Magic. In the screenshot we can see it illustrated. We tell pyenv that version 3.9.17 will be used in that directory and any shell we open there will be tied to that interpreter. Even more flexible. Let's suppose that inside the directory we want to do a small test with another version of Python. What can we do? pyenv shell 3.11.5 That alters only that shell, which will now point to Python version 3.11.5. No other session will be altered, only what we run there. Let's look at the screenshot: This is Hollywood. Once we finish our tests, we do a pyenv shell --unset and go back to where we were. With this, we practically solve the problem of the interpreters. We can have as many as we want. —CONTINUING THIS SERIES Telefónica Tech The ironic state of dependency management in Python (II) December 11, 2023 Telefónica Tech The ironic state of dependency management in Python (III) December 26, 2023

It is always fascinating to observe how evil makes use of creativity to cultivate new crops. This phenomenon occurs especially when the techniques that were once productive for them end up burning out; either because the trick has been seen too many times or because the technology concerned is retired. Of course, there are real revivals, for example, the resurgence of malware that infects through Office macros. Something that was thought to be over and that, by a quirk of fate (actually, no, there are technical reasons that help the trick to work again) is back in the limelight. In fact, in reality, it is not so much the technique that matters, but whether or not it is profitable. We have already said several times that finding a new vulnerability is a costly undertaking, especially if we expect its impact to be severe and its vector to require the least user interaction. The easy way out is for the user himself to contribute to getting infected, of course, at the expense of setting up a good hoax. Why would you rack your brains looking for a serious bug when human behavior already comes with a few of them as standard? This is what has happened to a handful of repositories on Github, which have received altruistic collaborations disguised as DependaBot (we'll define this later) with malicious code that was intended to steal credentials in a naughty way. Source: https://thehackernews.com/2023/09/github-repositories-hit-by-password.html If some clueless developer accepted the code collaboration (in the form of a pull request) what it did was to inject malicious code into the repository to steal the data marked as "secret" (not shared with the public). In addition, it modified the Javascript files of the project to intercept requests coming from forms and steal them. Attack sequence The sequence goes like this: you are a maintainer of a popular project on Github. A Python library that is used in a multitude of web application projects. That is, you are a dependency for those projects, a third party that provides them. In turn, you will have dependencies of other third parties in your own project. All normal and correct. In addition, Github has a wonderful bot that manages those dependencies and tells you when you should update them, especially for security bug fixes (great tool). That bot is DependaBot and it solves the problem we mentioned: being aware of the updates of your dependencies. The thing is that you get so used to DependaBot that you attribute a lot of reliability to it and many developers, when they receive a request from DependaBot, press the "merge" button (accept the modifications to your project) without thinking twice. There is trust, the value of collaboration is high and if you do not collaborate it means that you have a vulnerable dependency to solve. Well, this bond of trust is the one that has been exploited. Under the guise of DependaBot, from third party accounts, they have taken its icon and name to make changes in certain projects. As the developers trust DependaBot, many of them accepted the changes under the premise that we have already mentioned: trust, without knowing that they were actually making more changes than they should have and leaving the library infected. Trawling for cedentials and user data The next step was to wait for other projects that depend on that library to update to the vulnerable version and, once the project was deployed to production, trawling took place, capturing user credentials and data through the forms of the affected website. Basically, it is a poor variant of more sophisticated supply chain attacks that we have witnessed. Where to attack a third party, they first approached the company that owned control of a dependency and slyly injected their fishing gear into a bookstore to open the attackers' kitchen door. In fact, it is already a chapter of your own to manage not only the security of your company or your intellectual property, but also to keep a close eye on the dependencies you let into your projects and, from now on, the public collaborations, both yours and of those you invite to your home. See the corresponding section in ATT&CK. Something to take away Here, once again, trust is exploited. That human component as necessary as it is weak to build a society. Paying attention to the scenario set up to give credibility to the hoax, it is not much different from any well-done phishing scam, using the image and the common verb to pull the wool over the eyes of the public. In fact, this would fall more into the category of phishing using an uncommon vehicle than a supply chain attack, although it has traits of both constituencies. The former is not scary, but confusing, and the latter you don't see it coming and by the time it hits you it's too late for surprise. In the end, the same of these occasions: In cyber security, if something flies and has wings it may not be a bird but a cannonball on its way to the Santa Barbara. Cyber Security The Hacktivist, an online documentary about Cyber Security pioneer Andrew Huang January 2, 2024 Image from Yancy Min on Unsplash.

Holidays are over and we're back at work. We switch on the computer, open the email manager and... Oh, the password! It is expired. You send a request to IT to have it reset and you have to wait. A few floors down or a building down the street or perhaps on the other side of the Atlantic, someone from the IT department opens the ticket manager, looks at the screen, crunches his fingers and takes a sip of the coffee that will inevitably cool down before he can drink it again. He sighs, shifts his chair, and settles a back ready for a couple of contractures to land 500 password reset tickets later. Is it a good security measure for a password to expire like a carton of milk? That's what Bret Arsenault (Microsoft's CISO) asked himself when he abolished the company's internal policy requiring passwords to be changed every 71 days. Arsenault is critical of passwords. What's more, we fall short. It is not really a question of whether or not password refreshing is a productive security measure, but rather suggests that the very use of passwords is inadequate, unreliable, and an outdated model. https://www.linkedin.com/posts/bret-arsenault_in-the-future-there-will-be-no-passwordsbecause-activity-6993973391414210560-CfQx Of course, withdrawing such a measure seems counterproductive. A leap of faith. An inch gained by flexibility at the cost of sacrificing security in this eternal tug of war between the two irreconcilable extremes. Initially, it's scary to think that an attacker could get hold of a password and gain access to the organisation indefinitely. It's even more complicated. You have to be optimistic to think that an attacker is going to keep a credential indefinitely and only move from that point, like clocking in every day with that same card. In fact, the equivalent is getting an employee card and pretending that we can go through the turnstile for months without anyone noticing an intrusion. Wrong. That is indeed the point at issue: does having something as weak as a password give us a free hand to do whatever we want? Do we have to trust that identity? Not at all. That's why we have security guards, video surveillance cameras, etc. Or are we going to wait until that card is renewed or the employee reports the theft before we realise that something is wrong? The "zero trust" model is based on the assumption that things go wrong until the opposite is verified. It is the same principle that has to be transferred to the network and already exists: the "zero trust" model. This in turn is based on the assumption that things go wrong until the opposite is verified. Just the reverse of "nothing has to go wrong unless we detect a divergence in the system". Three types of factors and why the password is fuzzy in this regard You may have read this more than once: Authentication factors are based on three classes in an abstract way: what you know, what you own and what you are. The first is a secret, the second is an object, and the third is you (biometrics, of course). Where would you put the password: in your memory (you know it) or in your password manager (you own it)? If it's in your memory, I bet it's weak. Moreover, I dare you, I dare you twice that I am able to figure it out sooner or later. Memory is brittle and unless you are a genius at memorising long and complex strings of characters, your password is weak for sure. As we can see, the password is a fuzzy factor that can go from something you know (you memorise it) to something you own (the password manager). Like the old-fashioned "security questions", it is a counterproductive measure that is fortunately becoming less and less common. There is no worse security measure than one that we trust but ignore that it is useless. It must be conceded that, given that a well-operated password manager pivots passwords towards an "object you have" and allows us, at least, to create secure passwords and, as an essential measure, not to share them between different sites or domains. Why are passwords so bad Their bad reputation is no coincidence but has been built on facts. What are the famous, headline-grabbing leaks mainly about: biometric data or OTP device seeds? They are mostly about personal data and passwords (well, hashes, for another article). Passwords are also stored on disk or in memory. Within the reach of any malicious process (https://github.com/AlessandroZ/LaZagne, a wonder in the world of computers), not to mention how relatively easy it is for someone to "give you" the password with a few drops of social engineering. "Hi, I'm the support technician...". On top of that, if we refresh passwords there are users whose pattern is based on changing a couple of digits sequentially at most: passwordA2021, passwordA2022... That's not security, it's a false security measure. It adds nothing. As we have already mentioned, what if that password is also shared on several sites? It is possible that it is exfiltrated on a totally unrelated site and because it is shared for convenience, wham, the password has already fallen into the hands of someone who should not have it. Some employees even log on to sites using their business email address, which is totally inappropriate, both from a security and reputational point of view, and ethically reprehensible, especially in the case of a public leak. So? Neither unique passwords nor passwords that are difficult to remember. We have already touched on this in this article: mandatory second authentication factor and an architecture that provides the minimum trust and confidence in the identity. Changing passwords only leads to increased user frustration and wastes valuable hours for the IT team. The second factor (and why not even the third factor) gives us an additional safety net; though of course, it is not a panacea either. Rather than letting someone through the security check and then ignoring their actions, mistrust is the most important tool here. If someone must enter your organisation to perform a particular task at a particular time, what you have to make sure is that they are the right person, that they are on the right path and do what they have to do in the right window of time, and that they do it with the necessary permissions and resources and nothing else. Instead, it is a security event to which we must pay attention. As it is said, "You trust, but then you check”. Cyber Security How to use Passkey, Google's password substitute May 17, 2023 Imagen: Freepik.

We saw in the previous article the problems of manual memory management, but also the pitfalls of automatic memory management in languages like Java. But what if there was a middle ground, what if we could find a way to get rid of rubbish collector pauses, but at the same time not have to manage memory manually? How do we do it? Automatic management relies on the garbage collector, which runs at runtime, along with the program. Manual management falls to the programmer, who must do it at development time. If we discard the runtime collector and take away the programmer's task during development, what are we left with? Easy: the compiler. The third way: the compiler The third way is to introduce (or rather, to make the compiler, which is another of the elements in play, responsible for the memory management options. That is to say, to make it responsible for identifying who requests memory, how it is used and when it stops being used in order to reclaim it. The compiler, as an element of the development chain, has a broad view of all the elements that make up a program. It knows when memory is being requested and keeps track of the life of an object because it knows which symbol is being referenced, how and where. And of course, most importantly, when it stops being referenced. This is what programming languages like Rust do, whose model is based on ownership with the following rules: Each value in Rust must have an "owner". Only one "owner" can exist at a time. No more than one "owner" can exist. When the "owner" goes out of scope or visibility, the object is discarded and any memory it may contain is properly freed. Rules are simple, but in practice it takes some getting used to and a high tolerance for frustration, as the compiler will interpose itself in the face of any slip-up that inadvertently leads to a violation of the above rules. The system used by the compiler is called the borrow checker. It is basically the part of the compilation dedicated to check that the concept of owner is respected with respect to an object and that if an owner "borrows" the object, this borrowing is resolved when the scope changes and the concept of single owner is still maintained. Either because the recipient of the object takes responsibility for it or because he/she returns the ownership of the object. An example: If we look at the compiler complaints and the code, we see that the variable "s" has the string property "hello". In line 3, we declare a variable called "t" that borrows (and does not return) the string "hello". Then, in line 4, we make "s" add a new string and complete the classic "hello world" sentence, but the compiler won't let it: it's an error and lets us know. What has happened here? The one-owner rule comes into play. The compiler has detected that "s" no longer owns the string property, which now resides in "t", so it is illegal for it to make use of or attempt to modify the object it once owned, since it now doesn't belong to it. This is just the basics, but it is intended to give us an idea of how this "policing" of the rules by the compiler works. Image by Freepik. By the way, where is the toll here? Of course, in very long compilation times compared to C language or even Java, e.g. Conclusions Rust goes a third way in terms of memory management and only sacrifices compile time, programmer frustration while developing, and a certain steep curve in getting used to memory management rules (we have illustrated the basics, but Rust has other notions that complement borrowing, such as lifetimes, etc.). Despite the price paid (even so, the compiler improves timings with each new version), however, we will be almost absolutely certain that our program is free of memory errors (and also most of the race conditions) that could cause vulnerabilities. Will Rust save the world? Time will tell, but the reception and adoption of the language is remarkable and will improve the outlook for memory error-based vulnerabilities, which is no small thing. Cyber Security Will Rust save the world? (I) May 4, 2023 Featured image: Creativeart on Freepik.

The issue Programming bugs related to memory usage take a big slice of the CVE (Common Vulnerabilities and Exposures) pie every year. It has been many years since Elias Levi's (then known as Aleph One) classic and historic article, Smashing the Stack for Fun and Profit put in black and white how stack overflows could be exploited in an era where available information was scarce and not exactly easy to access. It was soon realised that memory management required attention and expertise that not all programmers possessed. When you create an object using dynamic memory (malloc), you have to keep track of its "lifetime" and free it when it is no longer useful so that the memory resources it owns can be reused. Memory management requires attention and expertise that not all programmers possess. This ecology is simple to set up, but devilishly complex to put into practice when a program exceeds the adjective of simple. It is easy to understand: when an object is created, it does not live in isolation from the rest of the program, it tends to create numerous references (pointers) and establish relationships (sometimes multi-layered) that finally lead us to ask ourselves the question: What if we release the object, will there be a reference that still requires the use of that object? What happens if we use an object that has already been released? What happens if I mistakenly release the same object twice? Rubbish collectors: a solution with drawbacks In the mid-1990s a programming language tried, successfully, to solve the problem with software created by languages with manual memory management: mainly C and C++. Unsurprisingly, it was Java. The language of the now defunct Sun Microsystems company (curious note: remember that there is an IDE for Java called... Eclipse), combined the incipient rise of object orientation and added concepts that were not new, but were well executed: a virtual machine and a memory collector, among others. Photo: Kelly Sikkema / Unsplash Java was a leap in quality and quantity. Programmers could (almost) abstract from memory management and were more productive. In addition, because Java was easier to learn (or rather, to master) than C and C++, it reduced the entry level for new programmers, which made it more affordable for companies, which in turn were able to complete more complex projects in a shorter period of time. Go language, also known as Golang, born precisely out of frustration with the use of C++ and… Java. Even today, other programming languages have emulated this initiative. The clearest example is the language Go, also known as Golang. Born precisely out of frustration with the use of C++ and… Java. The use of memory collectors in both of them erases the problem created by manual management, something that promotes the simplicity of the language and, once again, lowers the barrier to entry for new programmers. There is only one drawback, and it is not one that can be overlooked. Automatic memory management has a performance cost. The biggest drawback Although new memory management algorithms are increasingly advanced and the development capabilities of techniques allow nanoseconds of performance to be shaved off, the use of memory rubbish collectors has a toll to take during the collection of "rubbish" (unreferenced or unused memory) the world grinds to a halt. Indeed, when it is time to collect memory to be recycled, the program stops what it is doing, calls a garbage collector procedure, and routinely frees memory. Whatever the program is doing at that moment, its world stops for a fraction of a second, and it goes about cleaning up memory. Once it is done, it goes back to what it was doing as if nothing had happened. This Stop the world for many companies, on a large enough scale, translates into an added cost to the server bill. There are dozens of articles, almost all of which list the same reasons or complaints: the cost of the rubbish collector. Source: https://discord.com/blog/why-discord-is-switching-from-go-to-rust It may seem miniscule, but the response spikes accumulated in large, high-demand applications translate into an economic bill at the end of the chain. No matter how well you program, no matter how many algorithms and data structures you use, the world stops to collect rubbish and don't come back until the memory is clean as a whistle. The dilemma So, On the one hand, we have the languages that have great performance and exceptional execution, but which require suitably qualified personnel (and even so...) and tools that mitigate management errors. On the other hand, there are languages that increase productivity, drastically reduce memory management errors (although let's not forget about NullPointerException), but on the other hand, they have a little hindrance that makes them a bigger swallower of computational resources. It is a real dilemma in some cases, although it also becomes evident depending on the nature of the project (you are not going to write a Linux kernel module in Perl, nor are you going to implement a web application in C, as was done in the 90s). Barring the extremes, the options are to think about and calibrate what we sacrifice and what advantages are more appropriate. The third way However, what if there was a middle ground? What if we could find a way to get rid of rubbish collector pauses, but still not have to manage memory manually? How do we do that? Let's see it in next article. Cyber Security Will Rust save the world? (II) May 24, 2023 Featured photo: Mark Fletcher Brown / Unsplash.

Legend has it that it wasn't long after someone invented doors that locks were created. In ancient Egypt, an ingenious primitive mechanism was already in use that allowed a hook to be inserted through a hole as a key to move the wedges that acted as cylinders on a pin. These ingenious devices were perfected until they became the modern locks that we have in front of us today to curb the excesses of curiosity or empathy for other people's belongings. Moving away from the physical plane, in the digital world, locks are forms with two little boxes and a button, and keys are strings of characters. In the early days, when modern internet users could fit into a couple of first division football stadiums, those "keys" were as simple as counting to five, i.e., "12345". Of course, it was just the seed for a well-known disaster. We don't know how to create strong passwords And we can't memorise them either. Let's add to this mix the unhealthy ability we have to reuse passwords on dozens of sites where we open accounts. But isn't it easy to carry a simple master key to open all our doors? Of course, it is, and that's what the (now digitalised) thieves love to do every time they get your pet's name or the date of your birthday. One password, thousands of websites. A bargain. Second authentication factor, secure key generation algorithm, CAPTCHA systems to detect bots, password managers in the cloud... patches and patches and patches and patches on top of these to mend an exhausted system, concluded, obsolete, called to extinction, vilified and outdated... but, mind you, universal, ubiquitous, unanimous and anointed for being unique and unrivalled. What other alternatives do we have? As on the physical plane, locks do not have many competitors, although they change shape (and there are even those with facial recognition), in the end it is the same thing: a shutter that moves, leaving the way open to the supposed bearers of the right to access and enjoy the contents of that which they guard. However, doors and locks are not as driven by research and initiatives as the digital world in which we move on a daily basis (and at ungodly hours too), so the progressive abandonment of the headache of passwords was a problem that needed to be addressed and whose solution was to put an agreement on the table to standardise "something" that would allow, once and for all, to discard a mechanism more typical of the previous century than of this interesting and busy century. Transparent passwords At last, half of GAFAM and the FIDO Alliance in conjunction have signed a principle to push for a common mechanism to free our memory (the grey one, not the silicon one) from the tediousness of remembering or managing passwords. Could we be living in the historical moment where the last human being enters a password to access his or her binary piggy banks? Probably not yet. It's a long way off, but as we said at the beginning, it's a start. A first step already taken to walk a long road that is not going to be exactly flat. OK, but how exactly are we going to stop using passwords? What would be used instead? Well, strangely enough, we are not going to stop using them in a certain way or, rather, form. You simply won't use passwords, it will be a transparent action for you. It takes advantage of a number of items that are already widespread among the public: diversity of devices federated into a personal or distinguishable identity and the ability of these devices to offer biometric sensor features. Case studies All with a front camera and two of them with fingerprint reading capability. They don't have to be from the same manufacturer, but surely, in most cases, the same person has the same identity under one, two or even three providers. Do you access the mobile terminal? You can do it with your fingerprint or with facial recognition. Even laptops allow access with fingerprint (Apple's Touch ID) or facial recognition (Microsoft's Windows Hello). Now picture that all these manufacturers (who are also, let's not forget, identity providers) agree and allow a single identity to be used for all devices. You open your account once and that same identity is used on all your other systems. Sounds good, doesn't it? What if you now make it possible for a third party, on a website, to use that same system? Easy, you enter the identity saved on your device and on that or another device (from laptop to mobile and vice versa) it asks you to put your fingerprint or face. No password travelling, no password or hash is stored on the website, there is nothing an attacker can use to guess your password because it simply doesn't exist. There's no point in snooping through your social networks to guess your birthday or a pet's name Some of the challenges Of course, in all these advantages, there are some points that need to be elaborated in order to be fair. The first and most important of all is privacy. That quality that no one misses until it is too late to repent and remedy. Once you hand over control of your bunch of digital keys to a third party they will know at all times where, when and with which device you are visiting which site. The second, less likely but not impossible, is that an integrity failure in the identity provider can be catastrophic. Very catastrophic. Security is better (albeit more complex) when the weights (the risk) are distributed and it has never been a principle of this science to place the weight of a whole virtual life on a single point. Let's also talk about the third point: availability. What if you need to make an urgent transfer and your provider's servers have a meltdown at that moment and do not recover until many moments later? Bonus, fourth point: What if you wake up one morning and suddenly a machine learning-based automated system randomly decides that your account is fraudulent and deletes it? Where is the customer service desk? Hello, is anyone there? Hello? Can anyone hear me? My name is K, and I can't access my money? We can only celebrate the beginning of the farewell to passwords, even if there is still a long way to go, but we must also be critical of those who claim to be guarding what is ours, and we must be wary of betting with caution. We will see what the arrival of these new capabilities has in store for us and, above all, what alternatives are available for those who do not want or find it an inconvenient convenience.

It all starts with a tweet from a researcher (Allan Liska from RecordedFuture) announcing that he is compiling a list of vulnerabilities currently being exploited by organised groups in ransomware operations. It was, and still is, a good idea, so the good side of the Internet began to work and collaborations began to arrive, extending the set of vulnerabilities. In the end, a more or less fixed picture was reached (we all know that in technology, years pass in days): These are, to a large extent, to blame for many headaches and losses of millions nowadays. This list will change, some CVEs will fall due to exhaustion while new ones will enter and replace the old ones in a perverse cycle that seems to have no end. If we take a good look at the image, we can see that they correspond to vulnerabilities in products that can reside in the network perimeter of our organisation as well as in the desktop systems or in the cloud. There is heterogeneity in the classification and it corresponds directly to this other New Zealand CERT publication which illustrates perfectly how a ransomware operation works broadly.: The above table would enter the first, initial phase, where the first contact takes place. Thus, for example, vulnerabilities affecting Microsoft Office are triggered through the connection "Email -> Malicious Document -> Malware", while those affecting a product located at the perimeter of the network exposed to the Internet would be in "Exploit software weaknesses". The connections do not end here. Vulnerabilities that are specific to operating systems often involve elevation of privileges that guarantee two main things: access and persistence; in the network: "Take control -> ...". Once they have entered the perimeter, the focus shifts to the discovery of internal systems, exploitation, take control and elevation of privileges. From this point onwards, the company's value, its data, is sought. And not just live data, but also attempts to wipe out backups, the only viable solution against ransomware once all preventative controls have failed. Basically, the following three points can be summarised as the basic pillars against which the criminal groups strike: Vulnerabilities that allow taking control of the device exposed to the Internet. The human factor as a point of failure exploitable by social engineering. Poor configuration and implementation. The technical pillar (exploiting critical vulnerabilities) In the first case, the control is prevention and warning. As has been said many times, equipment must always be up to date. There is no excuse. If we have a dependency on an endangered technology, it is a countdown until it is replaced. So it is better to advance its replacement than to postpone it indefinitely. Moreover, it is not just a matter of waiting for the manufacturer's patch; as soon as we hear of the appearance of a vulnerability, we must put some kind of countermeasure in place to take it for granted that they are going to exploit it while we make our move. There are infrastructures that are designed to be staked at a particular point on the perimeter, and when that point falls, the consequences are devastating. We cannot place all the responsibility on a single control. The planning of a defence must take for granted that the commitment of that point in the network can occur at any given moment. For a team to be part of an internal network should not engender trust. Imagine a stranger who hangs a name tag on his shirt and walks around the departments of an office as he pleases. In fact, a handful of vulnerabilities are discovered when he is already wreaking havoc. In other words, a zero-day, which is discovered precisely because of its activity, is not detected by any antiviral solution or the like. There is no signature, it has not been seen before, it is not suspicious and yet it knocks down computers and systems. You have to be mentally and technically prepared to deal with such a blow. You have your computers properly updated and yet they are compromised. The human pillar (phishing and social engineering in general) In this case, we are talking about malware that needs the help of a human to act. This is no longer a vulnerability that can directly take control of a computer or at least run as a process. What we have is an unwitting helping hand with a finger that makes the terrible decision to send two keystrokes through the mouse and trigger a cascade of actions that end badly. That decision is made because false information has been provided that creates a situation perceived as safe by a person. A theatre. The king of this is email, but even now we have operations that are set up and run by posing as managers or department heads. Social engineering works. Always. Does awareness-raising work as a countermeasure? It is paradoxical. Imagine in the Middle Ages a castle that wants to defend itself against a possible surprise takeover. The sergeant of the guard lectures the watchmen every night to be vigilant. A state of alertness is induced which the soldiers internalise, but which they end up normalising as they see that night after night nothing happens. Until the moment comes when the walls are stormed and they are caught... with their guard down despite the poor sergeant's constant warnings and harangues. Perhaps the problem is that we call awareness to what we should call (and do) training. Training is what generates a tailored response to a particular problem. Make your employees understand the problem they face. Give them the opportunity to learn through simulated exercises. If instead of continuous alert the sergeant had trained his men with night raids, they might have interpreted the early signs of an invasion and would not now be under enemy fire. Social engineering works not because you are not on continuous alert but because you do not know how to identify the right signals to detect that you are walking into a trap. Let's remember that even in 2021, classic scams like the “pigeon drop” still work. The Pillar of Carelessness (All [wrong] by default) This type of breach is somewhere between technical and human failure. The former for bringing to market a product or system with a configuration that is not very demanding in terms of security, and the latter for deploying and integrating without giving importance to changing the parameters or performing the bastioning. The clearest case is the system with an account with default credentials. There have been (and continue to be) hundreds of well-documented cases. When doing a pentest, there is always the phase of knocking on doors hoping that one of the usual keys will open the door. A very bleeding case is CVE-2000-1209 or the classic Microsoft SQL Server with the account 'sa' and no password (rather, password to 'null') that filled audit and pentest reports for many years. In fact, in the early 2000s, several worms exploiting this oversight emerged. Mirai also had a field day with this kind of oversight. The IoT botnet reached a large number of nodes thanks to a simple list of default accounts on all kinds of network systems, set up and left to their own devices on the Internet. In the cinema, there is a very famous cliché in which one of the protagonists struggles with a door until he exhausts himself. Then another one of them, with a certain degree of mockery, approaches the door and opens it by simply turning the knob. The image should stick in our minds. We are giving cybercriminals the freedom to turn the knob and open the door. It's an example that shows that sometimes it doesn't take a lot of effort to find a zero-day vulnerability that allows us to execute arbitrary code. They are the worst because it is an evil that could have been avoided in an extremely simple way: by changing the default password to a suitable and robust one. Usually, this is done for several reasons, for example: rushing to finish things due to poor planning, staff not trained in cyber security, thinking that the manufacturer has a secure default configuration, lack of a security policy (no guidelines, no controls), etc. Conclusion As we have seen, ransomware comes in through the door at the slightest opening. Once inside, it makes itself at home, where we let our guard down and finally, when it is perhaps too late, it ruins us in dimensions ranging from a bad afternoon to a complete business shutdown. Identifying possible avenues of entry and their techniques are fundamental skills to learn in order to plan our defence. Either that or give in to luck and miss out on the lottery, the one that has as its ticket number: "All your files have been encrypted...". Download our new guide created in partnership with Palo Alto to help you prepare, plan, and respond to Ransomware attacks

OWASP, the foundation focused on web application security, has recently updated its ranking of the most prominent risks. Let's take a look at the new reorganisation of the top, commenting briefly on what aspects of security each of them touches on. What is the "OWASP top 10"? The OWASP top 10 is a fairly popular ranking in the world of web security and auditing, albeit nuanced and not without some controversy in the past. The list was first compiled in 2003 and updates have been published more or less regularly every three years since then. Of course, the list is not representative of all risks in web applications. In fact, it is likely that some of the items on this list do not apply to certain types of applications, while other risks that are not on the list could critically affect them. It should be taken as a "where the frontiers of the global risk map are moving", rather than a detailed and particular blueprint of where the security emphasis should be placed. The rankings are based on a study of a significant number of web applications and a survey of the industry, where practitioners indicate their greatest concerns about the various risks to their applications. In first place, we have a notable rise from fifth: A01:2021-Broken Access Control. This category covers all risks that allow an attacker to circumvent an imposed control in order to gain access to a resource. These types of vulnerabilities can emanate from the development itself to errors in the configuration during deployment. A simple example would be being able to access a privileged API because it does not correctly examine the access permissions of the source of the request. 2. The second place (climbing from third) goes to A02:2021-Cryptographic Failures. Cryptography is difficult, and if it is already complex to understand, it is even more difficult to apply its concepts in a practical and error-free manner. Add to this the fact that failures in cryptographic libraries are often cross-cutting. That is, a bug affects a myriad of libraries and programs that depend on them. Problems in the cryptographic chapter fill (rather flood) the audit reports: use of an obsolete hash function, insufficient encryption, insecure block cipher mode… and of course: absence of any kind of encryption… 3. The third place is a surprise, because after years of reigning at the top, injection vulnerabilities fall several notches. A03:2021-Injection groups together all those vulnerabilities in which the vector is the input of external code and is executed both on the client (browser) and on the server. In other words, both cross-site scripting and SQL injection. Undoubtedly, the progress made in terms of security, both in programming languages and in secure programming libraries and techniques, has something to do with this. It's nothing to let our guard down, but it may be good news. 4. Fourth place goes to a new category, unheard of in the ranking: A04:2021-Insecure Design. However, it is an amalgam of possible flaws in the design and implementation of the logical controls that should ensure the correct functioning of the application. For example, allowing products in the shopping cart with a negative number of items, storing the credentials in clear text, etc. As we can see, it is a catch-all for all kinds of errors that put the security of the application at risk. 5. A05:2021-Security Misconfiguration moves up one place, from sixth to fifth. Although this category could be confused with the previous one, it is easy to see that it refers to aspects of security that need to be reviewed in the context of configuration rather than secure design and implementation. The clearest example would be allowing a web server to list entire directories or allowing traces of request execution errors to be published in response to a request. Typically, third-party products provide an open default configuration to create as few conflicts as possible. Adapting the configuration in a secure way should be part of the natural development and deployment cycle. 6. Sixth place belongs to A06:2021-Vulnerable and Outdated Components, which moves up from ninth. It is almost self-explanatory: that the application has vulnerable or outdated components and the most direct example would be a WordPress plugin that is outdated and vulnerable or abandoned by the developers. It is obvious that a system is not built and deployed and then forgotten about. Software has to be tended like a garden or the bugs will end up eating the fruits of the garden. 7. A07:2021-Identification and Authentication Failures was formerly known as Broken Authentication. It also features another major drop: from second to seventh place. Easily confused with access controls, identification and authentication is the first step in obtaining permissions and privileges at the application level. This category not only identifies risks at the point of authentication, but we are also capturing here the lifecycle of the user session. That is, aspects such as the robustness of the authentication token, the exposure of the token and its validity over time. 8. The eighth place is taken by a new category that covers certain risks that have been in many headlines: A08:2021-Software and Data Integrity Failures. This section takes care of everything related to the integrity and verification of sources when we install, update or have supporting infrastructure (continuous integration, …) Remember the "supply chain" attacks? Well, this new category is the result of recognising this type of attack as a critical and important threat that we cannot ignore. We must not only keep an eye on our domains but also on what we sit at our table. 9. A09:2021-Security Logging and Monitoring Failures moves up one place and updates its name. From Insufficient Logging & Monitoring to the current one, no wonder. An application generates an astronomical number of events. How many of these events are security alerts? And better yet, which of these are real and critical alerts? This chapter covers everything from the absence of event logging to the proper storage and management of events. There is no point in having done your homework in all other aspects and not knowing what is happening and not being able to act in time. 10. The ranking is closed by A10:2021-Server-Side Request Forgery as the industry's number one concern (from the OWASP industry survey). It is practically the only category with a specific type of vulnerability. Not surprisingly, the vulnerability can pose a very high risk in a range of repercussions ranging from the discovery of exposed services on the internal network, access to non-public resources (files, databases, …) and even arbitrary code execution. A new scenario for cybersecurity This is the new scenario that OWASP paints in 2021. The changes are obvious, and although it is shown in an ordered list, we cannot weight the risk in relation to its position on the list. In fact, you only have to go through the security test guide to see how high the risks are. The top is a tasting of what is the hottest or most important points, but we must not lose sight of the rest. Doing so can put us in an awkward position if we fail to give importance to chapters that do not fall into the ten categories listed. Read more: https://owasp.org/Top10/ https://owasp.org/www-project-web-security-testing-guide/stable/

If we must choose a particularly damaging vulnerability, it would most likely be arbitrary code execution, and even more so if it can be exploited remotely. In the first blog entry we introduced the issues that can be caused by a poorly managed memory. The second one was about double free, and the third one was focused on dangling pointers and memory leaks. Let's close this set of blog posts with the use of uninitialized memory and the conclusions. Use of Uninitialized Memory For efficiency purposes, when we call 'malloc' or use the 'new' operator in C++, the memory area allocated is not initialized. What does this mean? That it doesn't contain a default value, but data that will seem random to us and doesn't make sense in the context of our process. Let's see it: We get a block of 10,000 integers, fill it with random integers and free it up. In theory, according to the standard of the C library, the memory coming from 'malloc' should not be initialized, but in certain systems (particularly modern ones) it is likely to be initialized to zero. That is, the whole reserved area is full of zeros. In the program, we make use of a reserved memory area and then free it up. But when we use this type of memory again, the system returns that same block with the content it already had. This content probably does not make sense in the current execution context. Let's see the output: What would happen if we used that data accidentally? Let's see it, again, within code. We modify the program so that the second reserve is made for a structure that we have defined: As we can see, we make use of 'p' by filling that area with random data. We free that block and now call one for a structure that should contain a pointer to a string and two integer values. Let's check a series of executions: As we see, the structure is initialized with "garbage", and making use of this "garbage" is problematic, if not worrying, and completely unsecure. Imagine that these values are used for a critical application. In addition to those already mentioned, the issues related to manual memory management do not end here. What we have seen is just a small sample and the list would be endless: pointer arithmetic, out-of-bounds write, and so on. New Management Mechanisms C++ has greatly improved manual management so that if already in the first steps of the language the need to use functions through operators ("new" and "delete") was removed, the new standard extends and improves memory management through "smart" pointers. That is, memory containers that call their own destructor when they detect that they are no longer useful. For example, when an object goes out of a scope and is no longer referenced by any other variable or object. Still, even with smart pointers, there is room for surprise and even for cases where we must use the traditional method, either for efficiency or for limitations in the libraries used by a given application. Another method of memory management that does not require a collector is the system used by languages such as Swift or Rust. The first one uses an "immediate" type of memory collector that does not require pauses, ARC or Automatic Reference Counting. This is a method that relies on the compiler to insert into the code the appropriate instructions to free up memory when this one is no longer going to be used. Rust, a relatively modern language, uses a method based on the concepts of "borrowing" and "ownership" related to the objects created with dynamic memory. An intermediate commitment between not having to carry the burden of a memory collector and the inconvenience of the programmer having to worry minimally about the logic of "borrowing" an object to other methods. Conclusions It is clear that manual memory management causes a tremendous vortex of issues that can (and usually do) lead to serious security problems. On the other hand, it requires a good capacity, attention and experience from programmers who use languages like C or C++. This doesn't mean that these languages should be abandoned because they are complex to use in certain aspects. As we said at the beginning, you can't avoid using them in certain types of applications. Don't miss the previous entries of this post: CYBER SECURITY Bestiary of a Poorly Managed Memory (I) April 30, 2020 CYBER SECURITY Bestiary of a Poorly Managed Memory (II) May 7, 2020 CYBER SECURITY Bestiary of a Poorly Managed Memory (III) May 14, 2020

If we must choose a particularly damaging vulnerability, it would most likely be arbitrary code execution, and even more so if it can be exploited remotely. In the first blog entry we introduced the issues that can be caused by a poorly managed memory. The second one was on double free. Now we are going to see more examples. Dangling Pointers Manual memory management is complex so attention must be paid to the order of operations, where resources are obtained from and where we stop using them in order to free them under good conditions. It also requires tracking copies of pointers or references that, if freed too early, may cause pointers to become "dangling". That is, making use of a resource that has already been freed. Let’s see an example: Let’s run: This leaves us with a pointer pointing to a memory area (heap) that is not valid (note that it does not print anything after "(p2) apunta a...". Moreover, there is no way to know if a resource whose address has been copied is still valid, just as it is not possible to recover a memory leak if its reference is lost (we will see this later). To tag that a pointer is not valid, we assign the NULL macro to that pointer (in "modern" C++ we would assign nullptr) to somehow warn that it is not pointing at anything. But if that NULL isn't verified, this is useless. Therefore, for every pointer using a resource, its "non NULLity" must be verified. The good practice is, therefore: once we free up memory, we assign NULL or nullptr (in C++) to tag that the pointer is no longer pointing at anything valid. Also, before making use of it, both to copy it and to de-reference it, we must verify if it's valid. Memory Leaks The opposite of using a memory area that is no longer valid is to have no pointer pointing at a valid memory area. Once the reference is lost, we can no longer free that reserved memory and it will occupy that space indefinitely until the program ends. This is a big issue if the program does not finish − such as a server that normally runs until the machine is shut down or some other unavoidable interruption occurs. An example (if you want to replicate it, do it in a virtualised system for testing): The code on the right gets parts of memory until all the heap memory is used up. This causes the system to run out of RAM, start swapping and finally the OOM-killer will kill the process for overconsuming memory. What is the OOM-killer? It is a special kernel procedure (on Linux systems) to end processes in memory so that the system is not destabilised. In the screenshot we can see the output of the command 'dmesg', where the kill of our process is showed due to the cost of resources it represents to the system. If we analyse the code, we see that we get into an endless loop where memory is reserved and the same pointer is reallocated to new blocks of that memory. Previous references are not freed and are lost, which triggers a relentless memory leak (exactly like a burst pipe) that ends drastically. This is obviously a dramatization of what would happen in a real program, but actually it occurs that way. The issue is that the reserved memory is not controlled at a point, so lost references are accumulated, and it ends up becoming a problem. It is possible that in applications with memory leaks that we only use for a few hours, we only notice a slowdown (this was more evident in times when the RAM was more limited) or a memory buildup. However, regarding servers the issue commonly leads to service drop. In the next post we will see the use of uninitialized memory. Don't forget to read the previous entries of this post: CYBER SECURITY Bestiary of a Poorly Managed Memory (I) April 30, 2020 CYBER SECURITY Bestiary of a Poorly Managed Memory (II) May 7, 2020

If we must choose a particularly damaging vulnerability, it would most likely be arbitrary code execution, and even more so if it can be exploited remotely. In the previous blog entry we introduced the issues that can be caused by a poorly managed memory. Now we are going to see concrete examples. Double Free: A Basic Example This issue occurs when we free the same block of reserved memory twice. Let's take a look at a program that does it "properly": We reserve two blocks, copy a string into them and free them as we no longer need them (note the calls for 'free'). Let's see the execution: All right, now we are going to deliberately make a slip. We are going to free the block that we had already freed. This does not prove the vulnerability itself (which is more complex to exploit) but allow us to check out what happens when we get it wrong in the dynamic memory structures of the heap. Let's run and see what happens: As we can see, the second string was not printed on the terminal, as it was in the previous program. What happened? By freeing a block of heap, this space has been left free. We have required another space for the variable p2 and we have been assigned another block. But remember that even though p1 has been freed, it still points to its original block, now belonging to p2. Since we are freeing p a second time, what we are actually doing is freeing memory that is using another pointer not related to the p object. The use of p2 becomes unstable since we are using a region of memory already freed. Let's see how the memory addresses were made: As we can see, p2 takes the address of the block that already had p, but this one has been freed. Let's see the same situation, but freeing the two blocks at the end of the function: If the situation is well managed, both point to different blocks, resources are not freed at the wrong time and everything goes as planned. Again: Manual memory management is complex, very complex. Attention must be paid to the order of operations, where resources are obtained and where we stop using them in order to free them in good condition. In the next entry we will talk about dangling pointers. Don't miss the first part of this post here: CYBER SECURITY Bestiary of a Poorly Managed Memory (I) April 30, 2020

If we must choose a particularly damaging vulnerability, it would most likely be arbitrary code execution, and even more so if it can be exploited remotely. The consequences may be fatal, as we have seen many times (Conficker for malware analysts, MS08-067 and EternalBlue for pentesters, WannaCry for everyone, etc.). Arbitrary code execution has been and remains one of the most loss-and-repair programming errors in the history of silicon. By the way, it is called arbitrary because actually the CPU is already executing code. The point of "arbitrary" is that it is left to the attacker to decide what code is executed, since it is the one taking control of the process. That's what an exploitation of this type is all about: diverting the normal and determined execution of a process to a foreign agent introduced in an arbitrary way by an attacker through an exploit. How Exactly Does This Happen? There are many ways to execute code (from here we will understand arbitrary). By the way, the definition is not limited to native executables. Cross-site scripting is an injection of foreign code that, again, diverts the execution of a script to the injected code snippet. One of the factors in the execution of code at the native level is the one derived from memory management issues. We will review the most common types of errors, focusing on how they occur and how operating systems and programming languages are evolving to mitigate the effect of these failures when they are maliciously exploited. Going back in time, not all languages had a manual management of the use of the memory. In fact, John McCarthy, one of the fathers of Artificial Intelligence and creator of LISP, coined the concept of automatic garbage collection (memory freed during the execution of a process) in the sixties. However, even though the garbage collectors made life easier for programmers (detaching themselves from manual management), it was an overload on resource consumption that some systems could not afford. To get an idea, it would be as if the real-time flight tracking of an airport control tower stopped for a few seconds to eliminate the freed memory. That's why languages like C or C++ keep a huge weight when programming system applications. They are languages without garbage collector (although it is possible to make use of them through libraries), so the programmer is fully responsible for the management of the memory. And of course, we all know what happens when you leave the work of a machine in the hands of a human. On the contrary, freeing the resources consumed by a collector means an enormous increase in the performance and response of the program − and this is translated into a lower cost in terms of hardware. Is It So Difficult to Manage the Memory Manually? Of course, it is a very open question and the answer will depend on our level of familiarity with this type of programming and on the facilities given by the language − added to the use of external tools and technology implemented in the compiler. Let's see an example: imagine that we want to associate a text string to a variable. A trivial operation in languages with automatic memory management, for example in Python (the following is an example of code, we are not going to bother with its correction): def asociar_cadena(cadena): mi_cadena = input()  # … # procesamos mi_cadena # … return mi_cadena Well, this in C language has some interesting points. First of all, we don't know the length of the string. That amount does not come "by default" with the string, it must be found or added as a parameter to the function. Secondly, since we do not know its length, we do not know either what memory we are going to need to save it. Thirdly: Who is in charge of warning us when we do not need that memory anymore? Let's look at a code snippet (there are multiple ways to implement this, safer and better, but this one will allow us to illustrate what we mean, for example, using strdup, "%ms", etc.): As we can see, we haven't even started to manipulate the string when we already have to write code to detect the end of a string, reserve memory, watch the limits of the array in the stack, etc. However, the important thing is to look at line 28, that "free" function used to tell the system to free the piece of memory we had reserved in the "read" function. Here the situation is clear: we no longer use that memory, so we return it. In an example of code, it is easy to make use of the memory but what if we continue to use that reserved memory 200 lines of code later? What if we have to pass that pointer through several functions? How is it clear who is in charge of the memory, the function called or who is calling that function? In the subsequent blog entries, we will review some scenarios that turn into vulnerabilities because of this type of oversight: double free, use of uninitialized memory, memory leaks and dangling pointers.

Over the last entry we focused on analyzing the content of these files from a critical point of view, this is: on clarifying that when a massive leak freeing millions of passwords is announced, the reality is not entirely what it seems to be. After all, what it has been filtered is the collection of leaks, gathered over time by a certain group of people or by someone. The leak we have examined has 640 Gb of content. We must clarify that it is not just the leak called "Collection #1" or the subsequent "Collection #2" and so on (the best-known ones). These types of collections are on the Internet, on several forums or uploaded on servers where anyone, with some patience, can access. Even considering that the content of these files is not always the latest one, or that many data can be completely irrelevant, it is not only this aspect what we are worried about. These types of leaks make us feel vulnerable and show us sharply how privacy is marketed. However, there are other aspects to be analyzed. For instance, thanks to these leaks we can apprehend what are the interests of these traders, how these collections are built, what are the different origins of the files and (above all) what they are later used for. From a constructive point of view, we are going to examine how the collection is structured as well as the potential origin of these files. We say "potential" because in most cases we cannot state categorically their origin with certainty. On some files, the organization consists in TLD domains attributed to groups and countries. This would allow to target some kind of attacks (phishing and scam, in general) towards a certain type of organization or group with the same idiosyncrasy. On this organization we can observe lists of leaks (very likely) coming from sites that could have been compromised, for extracting their databases as well as for injecting JavaScript code and consequently stealing the data from the form fields filled by the website visitors (who become then the second victims together with the website). Sometimes, lists of thematic websites are gathered. This is interesting for attackers, since it allows them to successfully perform very targeted campaigns. Let’s imagine that the users of these sites receive an e-mail inviting them to enter their credit card data to gain a free month subscription or a discount. The attackers could even show the user’s password to be trusted. Of course, in case of pornographic or adults’ relations sites, they may also use the consumption of this kind of services as a mean of blackmailing users. In the same way, they also have lists related to video game selling sites: As well as related to Bitcoin -or cryptocurrency in general- sites: There are more thematic divisions based on different types of services: purchases, streaming sites, etc. The files usually include e-mails and passwords on a classic format: [email]:[password]. In other cases, information is rawly organized. This is, for instance, a direct database dump: As a curiosity, we have created statistics based on the frequency of e-mail address domains in order to examine those that are more repeated within the various leaks. On the one hand, we must consider that some e-mails can be repeated in various files (we previously said that a high number of them were repetitions of the same e-mail within different leaks). On the other hand, certain e-mail services are more popular than other ones. Moreover, we must consider as well in which countries this leak can be more or less useful (in case of campaigns targeted by location). Six of the domains are focused on Russia, two on France and two further domains on the United Kingdom. QQ is a service mainly used in China. You may also be interested in: » The hugest collection of usernames and passwords has been filtered…or not (I)