The subject of working with XML and DOM comes up quite often in network engineering when we start to work with data structure and automation. Usually, this is in the context of understanding and making use of pieces of information we received in the XML format and we need to transfer the information in a programming language, such as Python, to make the data useful. This is confusing because we are introducing two new elements at the same time: XML and the particular Python library we are using. The situation can be even more confusing if we are introducing YANG model at the same time, this is yet another new element (no pun intended) if the reader has no prior experience with YANG. I am guilty of doing this in some of my courses and my book.
In this article, I intend to do a quick introduction to separate these elements. In doing so, hopefully, we can compartmentalize each in our minds. The next time we see the term we can start to focus on the basic characteristics.
I am almost afraid to bring up the YANG model for the fear of confusing the subject at hand. But it probably makes sense to at least do a cursory mention. YANG is a data modeling language that defines both configuration data as well as state data of network elements (source: https://en.wikipedia.org/wiki/YANG). It has its own set of terms, definitions, RFCs, and implementations. For the sake of our discussion, YANG is doing the job of representing our network element and state. We can then take that state and encode, or put it into a format we can give to others.
If we look at the example given on https://en.wikipedia.org/wiki/YANG, we can see the example model was written in the following format:
module example-sports {
..<skip>
}
This can then be put into either XML format:
<data xmlnx="">
..<sports>
....<skip>
..</sports>
</data>
or JSON:
{
.."example-sports:sports": {
....<skip>
..}
}
Note that both formats represent the same set of data given from the YANG model, you say "tomato", I say "tomahto", same thing.
From the YANG model, we can convert that data into different formats optimized for our application. XML is one of the data formats commonly used to represent YANG models, amongst others.
XML stands for Extensible Markup Language, much like HTML, it uses tags <> to describe the data at hand. The tags usually come in pairs where the opening tag <first_tag> is met with a closing tag </first_tag> of the same name preceded with a '/' in the front. For hierarchy data, the tags can be enclosed within other tags. The data, if any, are placed in-between the opening and closing tags. Within each tag, there can be attributes that describe the tag themselves. These features make XML ideal for a self-contained, self-describing, tree-like data format. Note that XML can also have separate documents used to further describe the document itself.
Here is an example of an XML representation:
<filter>
..<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
....<interface></interface>
..</interfaces>
</filter>
If we need to transport data over the Internet and it needs to be read by both machines and humans, XML would be a great choice. Amongst its many features, it has support for Unicode (tags can be in different human languages), can be compacted (since we do not need white space to represent structure), and self-documented (another buzz word).
In order to read the XML documents we typically need to convert them into a format that can be understood by the computer. There are two ways we can read, or parse, the XML document. Which way we pick typically has to do with the document's size and whether the data it contains are dynamic or static in nature.
For our use cases, I can't recall if I have ever come across needing to use the SAX API for parsing. The documents are typically static in nature, and with modern's computers' memory capacity, the size of the document usually does not impose any limitation, even with hundreds of lines (it might make the engineer's brain hurt, however). I'd imagine some enclosed system where the same vendor provides both the equipment and management station would find the SAX parsing useful, but as end-users, I don't see it being implemented. If we need the event-driven nature SAX provides, we typically implement that in a higher layer than try to accomplish it in a single XML document.
Now that we have some understanding between XML and its parsing techniques, let's look at the different Python library we can use to work with XML documents.
Python provides XML parsing modules within the standard library, appropriately named xml. Here are the two parsers for DOM and SAX:
Should we go jump straight to xml.dom and start processing XML data? Not so fast, Python also provides the xml.etree.ElementTree module that is a hybrid between the DOM and SAX approach. It needs less memory for XML tree than than DOM by using iterparse like SAX.
ElementTree represents the XML tree as a structure of lists and attributes in dictionaries. They are both native structures of Python. For this reason, we almost always pick ElementTree to begin with, unless we have specific needs to use DOM or SAX.
I know we did not include any actual Python code examples in this post. This is by design. By keeping the discussion at an abstract level, we can better categorize the approaches without getting into the muddy details. If you'd like to see some actual code example, you can take a look at the code on my book's repository:
Next time somebody asks you, "Is Python ElementTree a DOM parser or SAX parser?" You can say, "Both", with a confident smile.
Let me know what you think of this post by leaving me a comment below!
Happy Coding,
Eric