Expanding Neto capabilities: how to develop new analysis plugins

In previous posts we have introduced Neto as a browser extension analyzer. The first version we released, 0.5.x included a CLI, a JSON-RPC interface and could be used directly from your scripts. In the 0.6.x series we have gained stability and added some interesting features like the interactive console which makes the analyzer a tool to interact with. However, we have not yet discussed how we can extend Neto's functionality to suit our needs.

A system of plugins to gain flexibility

Despite the research needs that we may have from ElevenPaths, it may happen that other security analysts also want to carry out other tasks that we have not thought about. In order to make its use as flexible as possible, we have thought of a system of plugins that allows you to design your own modules. Remember at this point that we can always install the latest version from PyPI with:

But first, we will give you a brief description of how Neto works. Each extension is represented in Python in an object that loads the official analysis methods that we have included in neto/plugins/analysis. Neto will automatically execute the function defined as runAnalysis in which we will receive two different parameters that we can use according to our needs:

In this way, depending on what we want to do, we can choose one of these options. For example, if we want to work only with the.png files present in the extension, it is easier to do it using unzippedFiles but if we want to analyze the file itself we can use extensionFile. It depends on our needs. What we have to take into account is that you should always return a list in which the key is the name we give to our procedure and the value of the results. Thus, this new attribute will be added to the rest of the elements already obtained.

To define our own analysis modules in these first versions of Neto it will be enough to generate a few small scripts in Python, that it will store in its local folder ~/.config/ElevenPaths/Neto/plugins/. The characteristics of these user modules are identical to those of the official modules only that will be loaded upon request.

Creating our first plugin for Neto

In order to make the process easier for us, we have included a template of a plugin with each installation in ~/.config/ElevenPaths/Neto/plugins/template.py.sampleIt is easy to start developing from this screen and in order to see it we will make a simple plugin, which will count the number of files which the extension contains.

def runAnalysis(**kwargs):
    """
    Method that runs an analysis

    This method is dinamically loaded by neto.lib.extensions.Extension objects
    to conduct an analysis. The analyst can choose to perform the analysis on
    kwargs["extensionFile"] or on kwargs["unzippedFiles"]. It SHOULD return a
    dictionary with the results of the analysis that will be updated to the
    features property of the Extension.

    Args:
    -----
        kwargs: It currently contains:
            - extensionFile: A string to the local path of the extension.
            - unzippedFiles: A dictionary where the key is the relative path to
                the file and the the value the absolute path to the extension.
                {
                    "manifest.json": "/tmp/extension/manifest.json"
                    …
                }
    Returns:
    --------
        A dictionary where the key is the name given to the analysis and the
            value is the result of the analysis. This result can be of any
            format.
    """
    results = {}

    # Iterate through all the files in the folder
    for f, realPath in kwargs["unzippedFiles"].items():
        if os.path.isfile(realPath):
            # TODO: Your code here for each file
            pass

    return {__name__: results}

Based on the original code, we will utilize the stored information in kwargs["unzippedFiles"]and we will reutilize the loop which we already have to count those elements which are files increasing the variable myCounter, which we initiated at the start of the method.

    myCounter = 0

    # Iterate through all the files in the folder
    for f, realPath in kwargs["unzippedFiles"].items():
        if os.path.isfile(realPath):
            # TODO: Your code here for each file
            myCounter += 1

    return {"num_files": myCounter}

Now we will keep the file in the folder in question as ~/.config/ElevenPaths/Neto/plugins/hello_world.py for example. All that's left to do is start Neto with a new extension (for example, with the CLI) and to check the exit:

$ neto analyser -e ./my_demo.xpi
$ cat /home/USER/.config/ElevenPaths/Neto/data/analysis/854…78f.json | grep num_files
    "num_files": 151,

We now have our first plugin for Neto!

Now how can I share my plugins with the rest?

Once you have defined your plugin and you have tried it in a local instance, we will ask you to share it with us in order to merge it with the main project. Logged in with your username, make a fork of the project in your platform and clone your bifurcated repository in your system. We do it this way in order to prevent undesired circumstances, due to pushear the content of the main Github repository will be rejected because it is not authorized.

$ git clone https://github.com/USER/neto
$ cd neto

Once it is downloaded, copy the file which has already been tested locally to the repository. For example, in a GNU/Linux system you can retrieve the plugin from the file ~/.config/ElevenPaths/Neto/plugins/hello_world.py and copy it into the file of neto/plugins/analysis.

$ cp ~/.config/ElevenPaths/Neto/plugins/hello_world.py neto/plugins/analysis

Once the file is added, simply add it, make the changes and put it in your repository.

$ git add neto/plugins/analyser
$ git commit -m "Add hello_world plugin following the tutorial"
$ git push origin master

Once it is authenticated with your user, the only thing left is to make the pull request so that we can revise and merge it with the main project. Sometime in this revision process we will ask you to clarify some things, so that it is convenient to maintain a certain homogeneity we will utilize the guidelines marked in the style by PEP-8 wherever possible.

Anyway, the only general condition is that the generated response is a list in which the key is an element which identifies your analysis in a unique way and does not cause conflict with the rest of the implemented methods. Take into account that in the case that your plugin depends on another packet that is not found by default in Python 3, it will be necessary to update the setup.pyso that they satisfy the corresponding dependencies. Even so, you will not be in the process alone. Do you fancy trying it out?

Félix Brezo

Innovation and Laboratory Team ElevenPaths

@febrezo

felix.brezo@11paths.com